2023-05-16T06:43:18.413 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-16T06:43:18.414 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061 --name yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/orig.config.yaml 2023-05-16T06:43:18.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-05-16T06:43:18.543 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061 branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7275061' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 20531b61f434caac5e29cea07411fac6911e8e91 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 20531b61f434caac5e29cea07411fac6911e8e91 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 sha1: 20531b61f434caac5e29cea07411fac6911e8e91 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6166 sha1: 20531b61f434caac5e29cea07411fac6911e8e91 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 20531b61f434caac5e29cea07411fac6911e8e91 targets: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCogJToiq7Bj+4LXDaTzkRPivwioX4EePZ/pr/wuw7kazuX66qqj+UsTcjkQsxH16yUR7aqgG8BuzZZZot4jagbc2yiK6FpFc1Scs1mug+neYOc7vaRg+4rlSuEDenCXHChSM8Usrys/mmqt+to/W6JC0+F6i8DZT3qlCf2lyYQXNQcUuEJZh7gqtjdGUC6w08V9O9ET18IFk0e67tGdbiY8YhVxW6GiU4chMGdJWqkRHBO5evT5C/fCFScvzT6g+t1XcBYCm5WxcQ7PJg9wDlDIZSTUX0Dudj1h8Wzr5feNqW9/iFhp9RYjAtwWfxy/LLEwhkLY7/aCQwWwChEQCtJs9VzyPPaJbSXkieNPnpqYDbxYwYG9gvPHhKo5/JcuqKtf4ZYv2F+iw9zmMy9c6RWgB3/2tiyAMOPqzHvvdZk+PVySYuV2DfgzwCYCQTIMJp7225sGih4Ns/AQo6NF3qMzEmBIYSmxrfU/CA5DiZ2Rt7PIBQ7wfkzlZdNpG0o548= smithi158.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdf9TST+bZAMveHtQb+bAEliZIsauT4reJIVKcMcsRrC907QFoHkmkAQguH0kVa1zxA4RLPzlufnJ06+syo7rI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 69caab8ec6f44c61c991ae693e8e7b8cd859a896 timestamp: 2023-05-15_21:56:33 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-05-16T06:43:18.543 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa; will attempt to use it 2023-05-16T06:43:18.544 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa/tasks 2023-05-16T06:43:18.545 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-05-16T06:43:18.547 INFO:teuthology.task.internal:Checking packages... 2023-05-16T06:43:18.584 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '20531b61f434caac5e29cea07411fac6911e8e91' 2023-05-16T06:43:18.584 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-05-16T06:43:18.585 INFO:teuthology.packaging:ref: None 2023-05-16T06:43:18.585 INFO:teuthology.packaging:tag: None 2023-05-16T06:43:18.586 INFO:teuthology.packaging:branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 2023-05-16T06:43:18.586 INFO:teuthology.packaging:sha1: 20531b61f434caac5e29cea07411fac6911e8e91 2023-05-16T06:43:18.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri2-testing-2023-05-15-0810-pacific_2 2023-05-16T06:43:18.731 INFO:teuthology.task.internal:Found packages for ceph version 16.2.13-145.g20531b61 2023-05-16T06:43:18.731 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-05-16T06:43:18.738 INFO:teuthology.task.internal:no buildpackages task found 2023-05-16T06:43:18.738 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-05-16T06:43:18.751 INFO:teuthology.task.internal:Saving configuration 2023-05-16T06:43:18.774 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-05-16T06:43:18.781 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-16T06:43:18.810 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-16 06:37:28.807553', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCu2Pr3146oPyIU6NZAciIh5QiwqFnrc6Q+zHmjB/1K+hv3fvjDMLRpfYz7l4ynApDGZiVI4/ikVcn3sJ9G5Xo='} 2023-05-16T06:43:18.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-16 06:37:28.805570', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdf9TST+bZAMveHtQb+bAEliZIsauT4reJIVKcMcsRrC907QFoHkmkAQguH0kVa1zxA4RLPzlufnJ06+syo7rI='} 2023-05-16T06:43:18.840 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-05-16T06:43:18.847 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-05-16T06:43:18.847 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-05-16T06:43:18.847 INFO:teuthology.run_tasks:Running task console_log... 2023-05-16T06:43:18.935 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f56de1d4ee0>, signals=[15]) 2023-05-16T06:43:18.937 INFO:teuthology.run_tasks:Running task internal.connect... 2023-05-16T06:43:18.944 INFO:teuthology.task.internal:Opening connections... 2023-05-16T06:43:18.944 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T06:43:18.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-16T06:43:19.016 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T06:43:19.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-16T06:43:19.083 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-05-16T06:43:19.089 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-05-16T06:43:19.108 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-05-16T06:43:19.109 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-05-16T06:43:19.164 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Red Hat Enterprise Linux" 2023-05-16T06:43:19.165 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8.4 (Ootpa)" 2023-05-16T06:43:19.165 INFO:teuthology.orchestra.run.smithi092.stdout:ID="rhel" 2023-05-16T06:43:19.165 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="fedora" 2023-05-16T06:43:19.165 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8.4" 2023-05-16T06:43:19.166 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-05-16T06:43:19.166 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-05-16T06:43:19.166 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-05-16T06:43:19.166 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-05-16T06:43:19.167 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.redhat.com/" 2023-05-16T06:43:19.167 INFO:teuthology.orchestra.run.smithi092.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-05-16T06:43:19.167 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-05-16T06:43:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T06:43:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-05-16T06:43:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-05-16T06:43:19.168 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-05-16T06:43:19.169 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-05-16T06:43:19.170 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-05-16T06:43:19.197 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2023-05-16T06:43:19.214 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2023-05-16T06:43:19.214 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2023-05-16T06:43:19.270 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux" 2023-05-16T06:43:19.270 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.4 (Ootpa)" 2023-05-16T06:43:19.270 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel" 2023-05-16T06:43:19.270 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora" 2023-05-16T06:43:19.270 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.4" 2023-05-16T06:43:19.270 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-05-16T06:43:19.271 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:43:19.272 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-05-16T06:43:19.272 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-05-16T06:43:19.272 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-05-16T06:43:19.272 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-05-16T06:43:19.272 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2023-05-16T06:43:19.296 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-05-16T06:43:19.306 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-05-16T06:43:19.326 INFO:teuthology.task.internal:Checking for old test directory... 2023-05-16T06:43:19.326 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-05-16T06:43:19.330 DEBUG:teuthology.orchestra.run.smithi158:> test '!' -e /home/ubuntu/cephtest 2023-05-16T06:43:19.345 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-05-16T06:43:19.351 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-05-16T06:43:19.352 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-05-16T06:43:19.385 DEBUG:teuthology.orchestra.run.smithi158:> test -z $(ls -A /var/lib/ceph) 2023-05-16T06:43:19.399 INFO:teuthology.orchestra.run.smithi092.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-05-16T06:43:19.401 INFO:teuthology.orchestra.run.smithi158.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-05-16T06:43:19.402 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-05-16T06:43:19.465 INFO:teuthology.run_tasks:Running task kernel... 2023-05-16T06:43:19.485 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-05-16T06:43:19.486 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-05-16T06:43:19.486 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-05-16T06:43:19.486 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-05-16T06:43:19.487 DEBUG:teuthology.orchestra.run.smithi158:> test -f /run/.containerenv -o -f /.dockerenv 2023-05-16T06:43:19.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T06:43:19.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T06:43:19.504 DEBUG:teuthology.orchestra.run.smithi158:> uname -r 2023-05-16T06:43:19.504 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-05-16T06:43:19.560 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-05-16T06:43:19.561 INFO:teuthology.task.kernel:Running kernel on smithi092: 4.18.0-305.25.1.el8_4.x86_64 2023-05-16T06:43:19.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2023-05-16T06:43:19.562 INFO:teuthology.orchestra.run.smithi158.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-05-16T06:43:19.562 INFO:teuthology.task.kernel:Running kernel on smithi158: 4.18.0-305.25.1.el8_4.x86_64 2023-05-16T06:43:19.562 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum install -y kernel 2023-05-16T06:43:21.009 INFO:teuthology.orchestra.run.smithi158.stdout:Updating Subscription Management repositories. 2023-05-16T06:43:21.011 INFO:teuthology.orchestra.run.smithi158.stdout:Unable to read consumer identity 2023-05-16T06:43:21.052 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-05-16T06:43:21.052 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-05-16T06:43:24.646 INFO:teuthology.orchestra.run.smithi158.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-05-16T06:43:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-05-16T06:43:25.211 INFO:teuthology.orchestra.run.smithi158.stdout:Extra Packages for Enterprise Linux 26 MB/s | 14 MB 00:00 2023-05-16T06:43:25.646 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 23 MB/s | 14 MB 00:00 2023-05-16T06:43:28.081 INFO:teuthology.orchestra.run.smithi158.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-05-16T06:43:28.549 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-05-16T06:43:28.784 INFO:teuthology.orchestra.run.smithi158.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-05-16T06:43:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-05-16T06:43:28.785 INFO:teuthology.orchestra.run.smithi158.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-05-16T06:43:28.926 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-05-16T06:43:28.927 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-05-16T06:43:28.927 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-05-16T06:43:29.041 DEBUG:teuthology.orchestra.run.smithi158:> echo no | sudo yum reinstall kernel || true 2023-05-16T06:43:29.216 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-05-16T06:43:29.216 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-05-16T06:43:29.217 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-05-16T06:43:29.348 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-05-16T06:43:29.349 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2023-05-16T06:43:29.350 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-05-16T06:43:29.386 INFO:teuthology.orchestra.run.smithi158.stdout:Updating Subscription Management repositories. 2023-05-16T06:43:29.386 INFO:teuthology.orchestra.run.smithi158.stdout:Unable to read consumer identity 2023-05-16T06:43:29.461 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2023-05-16T06:43:29.672 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:00:01 ago on Tue 16 May 2023 06:43:28 AM UTC. 2023-05-16T06:43:29.805 INFO:teuthology.orchestra.run.smithi158.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-05-16T06:43:29.805 INFO:teuthology.orchestra.run.smithi158.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:29.805 INFO:teuthology.orchestra.run.smithi158.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:29.806 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-05-16T06:43:29.807 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-05-16T06:43:29.828 INFO:teuthology.orchestra.run.smithi158.stderr:Error: No packages marked for reinstall. 2023-05-16T06:43:29.877 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum reinstall -y kernel || true 2023-05-16T06:43:30.088 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Tue 16 May 2023 06:43:28 AM UTC. 2023-05-16T06:43:30.219 INFO:teuthology.orchestra.run.smithi158.stdout:Updating Subscription Management repositories. 2023-05-16T06:43:30.220 INFO:teuthology.orchestra.run.smithi158.stdout:Unable to read consumer identity 2023-05-16T06:43:30.260 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-05-16T06:43:30.260 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:30.260 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:30.287 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-05-16T06:43:30.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2023-05-16T06:43:30.498 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:00:02 ago on Tue 16 May 2023 06:43:28 AM UTC. 2023-05-16T06:43:30.628 INFO:teuthology.orchestra.run.smithi158.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-05-16T06:43:30.628 INFO:teuthology.orchestra.run.smithi158.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:30.628 INFO:teuthology.orchestra.run.smithi158.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:30.636 INFO:teuthology.orchestra.run.smithi158.stderr:Error: No packages marked for reinstall. 2023-05-16T06:43:30.680 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-05-16T06:43:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-05-16T06:43:30.686 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q kernel --last | head -n 1 2023-05-16T06:43:30.810 INFO:teuthology.orchestra.run.smithi158.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-05-16T06:43:30.811 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-05-16T06:43:30.812 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-05-16T06:43:30.813 DEBUG:teuthology.parallel:result is None 2023-05-16T06:43:30.958 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Tue 16 May 2023 06:43:28 AM UTC. 2023-05-16T06:43:31.086 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-05-16T06:43:31.086 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:31.086 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-05-16T06:43:31.099 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-05-16T06:43:31.148 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel --last | head -n 1 2023-05-16T06:43:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-05-16T06:43:31.274 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-05-16T06:43:31.275 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-05-16T06:43:31.275 DEBUG:teuthology.parallel:result is None 2023-05-16T06:43:31.275 INFO:teuthology.run_tasks:Running task internal.base... 2023-05-16T06:43:31.282 INFO:teuthology.task.internal:Creating test directory... 2023-05-16T06:43:31.283 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-05-16T06:43:31.285 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-05-16T06:43:31.300 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-05-16T06:43:31.413 INFO:teuthology.run_tasks:Running task internal.archive... 2023-05-16T06:43:31.420 INFO:teuthology.task.internal:Creating archive directory... 2023-05-16T06:43:31.421 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-05-16T06:43:31.423 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-05-16T06:43:31.450 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-05-16T06:43:31.459 INFO:teuthology.task.internal:Enabling coredump saving... 2023-05-16T06:43:31.460 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-05-16T06:43:31.487 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-05-16T06:43:31.514 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-16T06:43:31.516 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-16T06:43:31.528 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-16T06:43:31.535 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-16T06:43:31.537 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-05-16T06:43:31.552 INFO:teuthology.task.internal:Configuring sudo... 2023-05-16T06:43:31.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-05-16T06:43:31.580 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-05-16T06:43:31.605 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-05-16T06:43:31.626 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-05-16T06:43:31.626 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-05-16T06:43:31.643 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-05-16T06:43:31.660 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-16T06:43:31.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-16T06:43:31.776 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-16T06:43:31.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-16T06:43:31.894 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T06:43:31.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-05-16T06:43:31.956 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-16T06:43:32.000 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-16T06:43:32.046 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-16T06:43:32.134 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-16T06:43:32.168 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T06:43:32.168 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-05-16T06:43:32.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-05-16T06:43:32.233 DEBUG:teuthology.orchestra.run.smithi158:> sudo service rsyslog restart 2023-05-16T06:43:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-05-16T06:43:32.318 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-05-16T06:43:32.714 INFO:teuthology.run_tasks:Running task internal.timer... 2023-05-16T06:43:32.725 INFO:teuthology.task.internal:Starting timer... 2023-05-16T06:43:32.725 INFO:teuthology.run_tasks:Running task pcp... 2023-05-16T06:43:32.744 INFO:teuthology.run_tasks:Running task selinux... 2023-05-16T06:43:32.755 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-05-16T06:43:32.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-05-16T06:43:32.809 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: [ OK ] 2023-05-16T06:43:32.811 DEBUG:teuthology.orchestra.run.smithi158:> sudo service auditd rotate 2023-05-16T06:43:32.865 INFO:teuthology.orchestra.run.smithi158.stdout:Rotating logs: [ OK ] 2023-05-16T06:43:32.867 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-05-16T06:43:32.867 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-05-16T06:43:32.899 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-05-16T06:43:32.899 DEBUG:teuthology.orchestra.run.smithi158:> /usr/sbin/getenforce 2023-05-16T06:43:32.921 INFO:teuthology.orchestra.run.smithi158.stdout:Permissive 2023-05-16T06:43:32.922 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi158.front.sepia.ceph.com': 'permissive'} 2023-05-16T06:43:32.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-05-16T06:43:32.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T06:43:32.964 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-05-16T06:43:32.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T06:43:32.990 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-05-16T06:43:32.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-05-16T06:43:33.029 DEBUG:teuthology.orchestra.run.smithi158:> sudo /usr/sbin/setenforce permissive 2023-05-16T06:43:33.060 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-05-16T06:43:33.075 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-05-16T06:43:33.076 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-05-16T06:43:33.097 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-05-16T06:43:33.098 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi158.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-05-16T06:55:11.301 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi158.front.sepia.ceph.com')] 2023-05-16T06:55:11.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-05-16T06:55:11.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-16T06:55:11.374 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-05-16T06:55:11.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-05-16T06:55:11.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-16T06:55:11.505 DEBUG:teuthology.orchestra.run.smithi158:> true 2023-05-16T06:55:11.571 INFO:teuthology.run_tasks:Running task clock... 2023-05-16T06:55:11.584 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-05-16T06:55:11.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-05-16T06:55:11.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-05-16T06:55:11.587 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-05-16T06:55:11.588 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-05-16T06:55:11.614 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-05-16T06:55:11.628 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-05-16T06:55:11.640 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-05-16T06:55:11.653 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-05-16T06:55:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-05-16T06:55:11.671 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-05-16T06:55:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-05-16T06:55:11.685 INFO:teuthology.orchestra.run.smithi158.stderr:sudo: ntpd: command not found 2023-05-16T06:55:11.695 INFO:teuthology.orchestra.run.smithi158.stdout:506 Cannot talk to daemon 2023-05-16T06:55:11.697 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-05-16T06:55:11.709 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-05-16T06:55:11.722 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-05-16T06:55:11.762 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-05-16T06:55:11.765 INFO:teuthology.orchestra.run.smithi092.stdout:210 Number of sources = 4 2023-05-16T06:55:11.765 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-05-16T06:55:11.765 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-05-16T06:55:11.765 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.766 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.766 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.766 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.793 INFO:teuthology.orchestra.run.smithi158.stderr:bash: ntpq: command not found 2023-05-16T06:55:11.795 INFO:teuthology.orchestra.run.smithi158.stdout:210 Number of sources = 4 2023-05-16T06:55:11.795 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-05-16T06:55:11.796 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2023-05-16T06:55:11.796 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.796 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.796 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.796 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T06:55:11.797 INFO:teuthology.run_tasks:Running task install... 2023-05-16T06:55:11.805 DEBUG:teuthology.task.install:project ceph 2023-05-16T06:55:11.806 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91'}} 2023-05-16T06:55:11.806 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91'} 2023-05-16T06:55:11.806 INFO:teuthology.task.install:Using flavor: default 2023-05-16T06:55:11.813 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-05-16T06:55:11.814 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-05-16T06:55:11.815 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91', 'tag': None, 'wait_for_package': False} 2023-05-16T06:55:11.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=20531b61f434caac5e29cea07411fac6911e8e91 2023-05-16T06:55:11.819 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91', 'tag': None, 'wait_for_package': False} 2023-05-16T06:55:11.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=20531b61f434caac5e29cea07411fac6911e8e91 2023-05-16T06:55:11.965 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/ 2023-05-16T06:55:11.965 INFO:teuthology.task.install.rpm:Package version is 16.2.13-145.g20531b61 2023-05-16T06:55:11.969 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/ 2023-05-16T06:55:11.970 INFO:teuthology.task.install.rpm:Package version is 16.2.13-145.g20531b61 2023-05-16T06:55:12.076 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-05-16T06:55:12.076 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T06:55:12.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-05-16T06:55:12.086 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-05-15-0810-pacific_2/20531b61f434caac5e29cea07411fac6911e8e91/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-05-16T06:55:12.086 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T06:55:12.087 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-05-16T06:55:12.101 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-05-16T06:55:12.102 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/20531b61f434caac5e29cea07411fac6911e8e91/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-05-16T06:55:12.111 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-05-16T06:55:12.112 DEBUG:teuthology.orchestra.run.smithi158:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/20531b61f434caac5e29cea07411fac6911e8e91/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-05-16T06:55:12.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-05-16T06:55:12.175 DEBUG:teuthology.orchestra.run.smithi158:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-05-16T06:55:12.238 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-05-16T06:55:12.249 DEBUG:teuthology.orchestra.run.smithi158:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-05-16T06:55:12.300 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-05-16T06:55:12.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-05-16T06:55:12.312 INFO:teuthology.orchestra.run.smithi158.stdout:check_obsoletes = 1 2023-05-16T06:55:12.313 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum clean all 2023-05-16T06:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-05-16T06:55:12.697 INFO:teuthology.orchestra.run.smithi158.stdout:Updating Subscription Management repositories. 2023-05-16T06:55:12.979 INFO:teuthology.orchestra.run.smithi092.stdout:44 files removed 2023-05-16T06:55:12.987 INFO:teuthology.orchestra.run.smithi158.stdout:44 files removed 2023-05-16T06:55:13.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-05-16T06:55:13.037 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-05-16T06:55:13.371 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-05-16T06:55:13.379 INFO:teuthology.orchestra.run.smithi158.stdout:Updating Subscription Management repositories. 2023-05-16T06:55:14.015 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 349 kB/s | 76 kB 00:00 2023-05-16T06:55:14.028 INFO:teuthology.orchestra.run.smithi158.stdout:ceph packages for x86_64 339 kB/s | 76 kB 00:00 2023-05-16T06:55:14.239 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 84 kB/s | 15 kB 00:00 2023-05-16T06:55:14.252 INFO:teuthology.orchestra.run.smithi158.stdout:ceph noarch packages 84 kB/s | 15 kB 00:00 2023-05-16T06:55:14.454 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 7.8 kB/s | 1.4 kB 00:00 2023-05-16T06:55:14.466 INFO:teuthology.orchestra.run.smithi158.stdout:ceph source packages 7.8 kB/s | 1.4 kB 00:00 2023-05-16T06:55:17.060 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-05-16T06:55:17.276 INFO:teuthology.orchestra.run.smithi158.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-05-16T06:55:17.532 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 31 MB/s | 14 MB 00:00 2023-05-16T06:55:18.341 INFO:teuthology.orchestra.run.smithi158.stdout:Extra Packages for Enterprise Linux 13 MB/s | 14 MB 00:01 2023-05-16T06:55:20.445 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 1.7 MB/s | 24 kB 00:00 2023-05-16T06:55:20.849 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 117 MB/s | 38 MB 00:00 2023-05-16T06:55:21.243 INFO:teuthology.orchestra.run.smithi158.stdout:lab-extras 4.9 MB/s | 24 kB 00:00 2023-05-16T06:55:21.621 INFO:teuthology.orchestra.run.smithi158.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 127 MB/s | 38 MB 00:00 2023-05-16T06:55:26.352 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 125 MB/s | 33 MB 00:00 2023-05-16T06:55:27.170 INFO:teuthology.orchestra.run.smithi158.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 122 MB/s | 33 MB 00:00 2023-05-16T06:55:34.330 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-16T06:55:34.331 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-16T06:55:34.341 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-05-16T06:55:34.342 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-05-16T06:55:34.343 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-05-16T06:55:34.344 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-05-16T06:55:34.345 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-05-16T06:55:34.348 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-05-16T06:55:34.349 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-05-16T06:55:34.841 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-05-16T06:55:34.852 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-05-16T06:55:34.853 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-05-16T06:55:34.853 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-05-16T06:55:34.853 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-05-16T06:55:34.853 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-05-16T06:55:34.853 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:16.2.13-145.g20531b61.el8 ceph 6.4 k 2023-05-16T06:55:34.853 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:16.2.13-145.g20531b61.el8 ceph 6.1 M 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:16.2.13-145.g20531b61.el8 ceph 908 k 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:16.2.13-145.g20531b61.el8 ceph 161 k 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:16.2.13-145.g20531b61.el8 ceph 1.6 M 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 116 k 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 3.1 M 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 8.6 M 2023-05-16T06:55:34.854 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 27 k 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:16.2.13-145.g20531b61.el8 ceph 12 M 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:16.2.13-145.g20531b61.el8 ceph 51 M 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 84 k 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:16.2.13-145.g20531b61.el8 ceph 256 k 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 18 k 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-05-16T06:55:34.855 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:16.2.13-145.g20531b61.el8 ceph 25 k 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:16.2.13-145.g20531b61.el8 ceph 822 k 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:16.2.13-145.g20531b61.el8 ceph 131 k 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-05-16T06:55:34.856 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:16.2.13-145.g20531b61.el8 ceph 215 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:16.2.13-145.g20531b61.el8 ceph 389 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:16.2.13-145.g20531b61.el8 ceph 368 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:16.2.13-145.g20531b61.el8 ceph 114 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:16.2.13-145.g20531b61.el8 ceph 88 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:16.2.13-145.g20531b61.el8 ceph 3.7 M 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:16.2.13-145.g20531b61.el8 ceph 173 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-05-16T06:55:34.857 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:16.2.13-145.g20531b61.el8 ceph 3.7 M 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:16.2.13-145.g20531b61.el8 ceph 4.2 M 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:16.2.13-145.g20531b61.el8 ceph 24 M 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 24 k 2023-05-16T06:55:34.858 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:16.2.13-145.g20531b61.el8 ceph 2.4 M 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 244 k 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:16.2.13-145.g20531b61.el8 ceph 5.0 M 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:16.2.13-145.g20531b61.el8 ceph 18 M 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 15 k 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:16.2.13-145.g20531b61.el8 ceph 26 k 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-05-16T06:55:34.859 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:16.2.13-145.g20531b61.el8 ceph 174 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-05-16T06:55:34.860 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:16.2.13-145.g20531b61.el8 ceph 493 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:16.2.13-145.g20531b61.el8 ceph 3.8 M 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-05-16T06:55:34.861 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-05-16T06:55:34.862 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:16.2.13-145.g20531b61.el8 ceph 45 k 2023-05-16T06:55:34.863 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:16.2.13-145.g20531b61.el8 ceph 97 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-05-16T06:55:34.864 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-05-16T06:55:34.865 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-05-16T06:55:34.866 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-05-16T06:55:34.867 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-05-16T06:55:34.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-05-16T06:55:34.869 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-05-16T06:55:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T06:55:34.870 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-05-16T06:55:34.870 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-05-16T06:55:34.870 INFO:teuthology.orchestra.run.smithi092.stdout:Install 118 Packages 2023-05-16T06:55:34.870 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2023-05-16T06:55:34.870 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T06:55:34.871 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 200 M 2023-05-16T06:55:34.871 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-05-16T06:55:35.116 INFO:teuthology.orchestra.run.smithi092.stdout:(1/120): ceph-16.2.13-145.g20531b61.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2023-05-16T06:55:35.329 INFO:teuthology.orchestra.run.smithi158.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-16T06:55:35.330 INFO:teuthology.orchestra.run.smithi158.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-16T06:55:35.340 INFO:teuthology.orchestra.run.smithi158.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-05-16T06:55:35.341 INFO:teuthology.orchestra.run.smithi158.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-05-16T06:55:35.341 INFO:teuthology.orchestra.run.smithi158.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-05-16T06:55:35.342 INFO:teuthology.orchestra.run.smithi158.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-05-16T06:55:35.344 INFO:teuthology.orchestra.run.smithi158.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-05-16T06:55:35.346 INFO:teuthology.orchestra.run.smithi158.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-05-16T06:55:35.347 INFO:teuthology.orchestra.run.smithi158.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-05-16T06:55:35.784 INFO:teuthology.orchestra.run.smithi092.stdout:(2/120): ceph-fuse-16.2.13-145.g20531b61.el8.x8 1.3 MB/s | 908 kB 00:00 2023-05-16T06:55:35.817 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-05-16T06:55:35.829 INFO:teuthology.orchestra.run.smithi158.stdout:====================================================================================================================== 2023-05-16T06:55:35.829 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-05-16T06:55:35.829 INFO:teuthology.orchestra.run.smithi158.stdout:====================================================================================================================== 2023-05-16T06:55:35.829 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-05-16T06:55:35.829 INFO:teuthology.orchestra.run.smithi158.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph x86_64 2:16.2.13-145.g20531b61.el8 ceph 6.4 k 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base x86_64 2:16.2.13-145.g20531b61.el8 ceph 6.1 M 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse x86_64 2:16.2.13-145.g20531b61.el8 ceph 908 k 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache x86_64 2:16.2.13-145.g20531b61.el8 ceph 161 k 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr x86_64 2:16.2.13-145.g20531b61.el8 ceph 1.6 M 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 116 k 2023-05-16T06:55:35.830 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 3.1 M 2023-05-16T06:55:35.831 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 8.6 M 2023-05-16T06:55:35.831 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 27 k 2023-05-16T06:55:35.831 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw x86_64 2:16.2.13-145.g20531b61.el8 ceph 12 M 2023-05-16T06:55:35.831 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test x86_64 2:16.2.13-145.g20531b61.el8 ceph 51 M 2023-05-16T06:55:35.831 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 84 k 2023-05-16T06:55:35.831 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-mirror x86_64 2:16.2.13-145.g20531b61.el8 ceph 256 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-top noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 18 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel x86_64 2:16.2.13-145.g20531b61.el8 ceph 25 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2 x86_64 2:16.2.13-145.g20531b61.el8 ceph 822 k 2023-05-16T06:55:35.832 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel x86_64 2:16.2.13-145.g20531b61.el8 ceph 131 k 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs x86_64 2:16.2.13-145.g20531b61.el8 ceph 215 k 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados x86_64 2:16.2.13-145.g20531b61.el8 ceph 389 k 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd x86_64 2:16.2.13-145.g20531b61.el8 ceph 368 k 2023-05-16T06:55:35.833 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw x86_64 2:16.2.13-145.g20531b61.el8 ceph 114 k 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse x86_64 2:16.2.13-145.g20531b61.el8 ceph 88 k 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror x86_64 2:16.2.13-145.g20531b61.el8 ceph 3.7 M 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd x86_64 2:16.2.13-145.g20531b61.el8 ceph 173 k 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrading: 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout: librados2 x86_64 2:16.2.13-145.g20531b61.el8 ceph 3.7 M 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1 x86_64 2:16.2.13-145.g20531b61.el8 ceph 4.2 M 2023-05-16T06:55:35.834 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common x86_64 2:16.2.13-145.g20531b61.el8 ceph 24 M 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 24 k 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds x86_64 2:16.2.13-145.g20531b61.el8 ceph 2.4 M 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 244 k 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon x86_64 2:16.2.13-145.g20531b61.el8 ceph 5.0 M 2023-05-16T06:55:35.835 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd x86_64 2:16.2.13-145.g20531b61.el8 ceph 18 M 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts noarch 2:16.2.13-145.g20531b61.el8 ceph-noarch 15 k 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux x86_64 2:16.2.13-145.g20531b61.el8 ceph 26 k 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-05-16T06:55:35.836 INFO:teuthology.orchestra.run.smithi158.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite x86_64 2:16.2.13-145.g20531b61.el8 ceph 174 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-05-16T06:55:35.837 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1 x86_64 2:16.2.13-145.g20531b61.el8 ceph 493 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2 x86_64 2:16.2.13-145.g20531b61.el8 ceph 3.8 M 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-05-16T06:55:35.838 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-05-16T06:55:35.839 INFO:teuthology.orchestra.run.smithi158.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse x86_64 2:16.2.13-145.g20531b61.el8 ceph 45 k 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common x86_64 2:16.2.13-145.g20531b61.el8 ceph 97 k 2023-05-16T06:55:35.840 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-05-16T06:55:35.841 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-05-16T06:55:35.842 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-05-16T06:55:35.843 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-05-16T06:55:35.844 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-05-16T06:55:35.845 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-05-16T06:55:35.846 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-05-16T06:55:35.846 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-05-16T06:55:35.846 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-05-16T06:55:35.846 INFO:teuthology.orchestra.run.smithi158.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-05-16T06:55:35.846 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-05-16T06:55:35.846 INFO:teuthology.orchestra.run.smithi158.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-05-16T06:55:35.847 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:55:35.847 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-05-16T06:55:35.847 INFO:teuthology.orchestra.run.smithi158.stdout:====================================================================================================================== 2023-05-16T06:55:35.847 INFO:teuthology.orchestra.run.smithi158.stdout:Install 118 Packages 2023-05-16T06:55:35.847 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrade 2 Packages 2023-05-16T06:55:35.847 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:55:35.848 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 200 M 2023-05-16T06:55:35.848 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-05-16T06:55:35.905 INFO:teuthology.orchestra.run.smithi092.stdout:(3/120): ceph-immutable-object-cache-16.2.13-14 1.3 MB/s | 161 kB 00:00 2023-05-16T06:55:36.193 INFO:teuthology.orchestra.run.smithi158.stdout:(1/120): ceph-16.2.13-145.g20531b61.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-05-16T06:55:37.048 INFO:teuthology.orchestra.run.smithi092.stdout:(4/120): ceph-base-16.2.13-145.g20531b61.el8.x8 2.8 MB/s | 6.1 MB 00:02 2023-05-16T06:55:37.218 INFO:teuthology.orchestra.run.smithi092.stdout:(5/120): ceph-mds-16.2.13-145.g20531b61.el8.x86 1.9 MB/s | 2.4 MB 00:01 2023-05-16T06:55:37.443 INFO:teuthology.orchestra.run.smithi092.stdout:(6/120): ceph-mgr-16.2.13-145.g20531b61.el8.x86 4.1 MB/s | 1.6 MB 00:00 2023-05-16T06:55:37.739 INFO:teuthology.orchestra.run.smithi158.stdout:(2/120): ceph-fuse-16.2.13-145.g20531b61.el8.x8 587 kB/s | 908 kB 00:01 2023-05-16T06:55:38.129 INFO:teuthology.orchestra.run.smithi158.stdout:(3/120): ceph-immutable-object-cache-16.2.13-14 413 kB/s | 161 kB 00:00 2023-05-16T06:55:39.645 INFO:teuthology.orchestra.run.smithi158.stdout:(4/120): ceph-base-16.2.13-145.g20531b61.el8.x8 1.6 MB/s | 6.1 MB 00:03 2023-05-16T06:55:39.918 INFO:teuthology.orchestra.run.smithi092.stdout:(7/120): ceph-mon-16.2.13-145.g20531b61.el8.x86 1.9 MB/s | 5.0 MB 00:02 2023-05-16T06:55:40.318 INFO:teuthology.orchestra.run.smithi158.stdout:(5/120): ceph-mgr-16.2.13-145.g20531b61.el8.x86 2.4 MB/s | 1.6 MB 00:00 2023-05-16T06:55:41.308 INFO:teuthology.orchestra.run.smithi158.stdout:(6/120): ceph-mds-16.2.13-145.g20531b61.el8.x86 786 kB/s | 2.4 MB 00:03 2023-05-16T06:55:41.503 INFO:teuthology.orchestra.run.smithi092.stdout:(8/120): ceph-osd-16.2.13-145.g20531b61.el8.x86 4.5 MB/s | 18 MB 00:04 2023-05-16T06:55:41.567 INFO:teuthology.orchestra.run.smithi092.stdout:(9/120): ceph-selinux-16.2.13-145.g20531b61.el8 399 kB/s | 26 kB 00:00 2023-05-16T06:55:42.667 INFO:teuthology.orchestra.run.smithi158.stdout:(7/120): ceph-mon-16.2.13-145.g20531b61.el8.x86 2.1 MB/s | 5.0 MB 00:02 2023-05-16T06:55:42.921 INFO:teuthology.orchestra.run.smithi092.stdout:(10/120): ceph-common-16.2.13-145.g20531b61.el8 3.0 MB/s | 24 MB 00:08 2023-05-16T06:55:43.058 INFO:teuthology.orchestra.run.smithi092.stdout:(11/120): cephfs-mirror-16.2.13-145.g20531b61.e 1.8 MB/s | 256 kB 00:00 2023-05-16T06:55:43.123 INFO:teuthology.orchestra.run.smithi092.stdout:(12/120): libcephfs-devel-16.2.13-145.g20531b61 393 kB/s | 25 kB 00:00 2023-05-16T06:55:43.406 INFO:teuthology.orchestra.run.smithi092.stdout:(13/120): libcephfs2-16.2.13-145.g20531b61.el8. 2.8 MB/s | 822 kB 00:00 2023-05-16T06:55:43.476 INFO:teuthology.orchestra.run.smithi092.stdout:(14/120): libcephsqlite-16.2.13-145.g20531b61.e 2.4 MB/s | 174 kB 00:00 2023-05-16T06:55:43.546 INFO:teuthology.orchestra.run.smithi092.stdout:(15/120): librados-devel-16.2.13-145.g20531b61. 1.9 MB/s | 131 kB 00:00 2023-05-16T06:55:43.748 INFO:teuthology.orchestra.run.smithi092.stdout:(16/120): libradosstriper1-16.2.13-145.g20531b6 2.4 MB/s | 493 kB 00:00 2023-05-16T06:55:44.714 INFO:teuthology.orchestra.run.smithi092.stdout:(17/120): librgw2-16.2.13-145.g20531b61.el8.x86 3.9 MB/s | 3.8 MB 00:00 2023-05-16T06:55:44.773 INFO:teuthology.orchestra.run.smithi092.stdout:(18/120): python3-ceph-argparse-16.2.13-145.g20 765 kB/s | 45 kB 00:00 2023-05-16T06:55:44.835 INFO:teuthology.orchestra.run.smithi092.stdout:(19/120): python3-ceph-common-16.2.13-145.g2053 1.5 MB/s | 97 kB 00:00 2023-05-16T06:55:44.902 INFO:teuthology.orchestra.run.smithi092.stdout:(20/120): python3-cephfs-16.2.13-145.g20531b61. 3.2 MB/s | 215 kB 00:00 2023-05-16T06:55:45.033 INFO:teuthology.orchestra.run.smithi092.stdout:(21/120): python3-rados-16.2.13-145.g20531b61.e 2.9 MB/s | 389 kB 00:00 2023-05-16T06:55:45.163 INFO:teuthology.orchestra.run.smithi092.stdout:(22/120): python3-rbd-16.2.13-145.g20531b61.el8 2.8 MB/s | 368 kB 00:00 2023-05-16T06:55:45.227 INFO:teuthology.orchestra.run.smithi092.stdout:(23/120): python3-rgw-16.2.13-145.g20531b61.el8 1.8 MB/s | 114 kB 00:00 2023-05-16T06:55:45.289 INFO:teuthology.orchestra.run.smithi092.stdout:(24/120): rbd-fuse-16.2.13-145.g20531b61.el8.x8 1.4 MB/s | 88 kB 00:00 2023-05-16T06:55:45.346 INFO:teuthology.orchestra.run.smithi092.stdout:(25/120): ceph-radosgw-16.2.13-145.g20531b61.el 2.1 MB/s | 12 MB 00:05 2023-05-16T06:55:45.416 INFO:teuthology.orchestra.run.smithi092.stdout:(26/120): rbd-nbd-16.2.13-145.g20531b61.el8.x86 2.4 MB/s | 173 kB 00:00 2023-05-16T06:55:45.480 INFO:teuthology.orchestra.run.smithi092.stdout:(27/120): ceph-grafana-dashboards-16.2.13-145.g 381 kB/s | 24 kB 00:00 2023-05-16T06:55:45.544 INFO:teuthology.orchestra.run.smithi092.stdout:(28/120): ceph-mgr-cephadm-16.2.13-145.g20531b6 1.8 MB/s | 116 kB 00:00 2023-05-16T06:55:46.305 INFO:teuthology.orchestra.run.smithi092.stdout:(29/120): rbd-mirror-16.2.13-145.g20531b61.el8. 3.6 MB/s | 3.7 MB 00:01 2023-05-16T06:55:46.809 INFO:teuthology.orchestra.run.smithi092.stdout:(30/120): ceph-mgr-dashboard-16.2.13-145.g20531 2.5 MB/s | 3.1 MB 00:01 2023-05-16T06:55:46.877 INFO:teuthology.orchestra.run.smithi158.stdout:(8/120): ceph-radosgw-16.2.13-145.g20531b61.el8 2.7 MB/s | 12 MB 00:04 2023-05-16T06:55:46.947 INFO:teuthology.orchestra.run.smithi158.stdout:(9/120): ceph-selinux-16.2.13-145.g20531b61.el8 371 kB/s | 26 kB 00:00 2023-05-16T06:55:46.947 INFO:teuthology.orchestra.run.smithi092.stdout:(31/120): ceph-mgr-modules-core-16.2.13-145.g20 1.7 MB/s | 244 kB 00:00 2023-05-16T06:55:47.018 INFO:teuthology.orchestra.run.smithi092.stdout:(32/120): ceph-mgr-rook-16.2.13-145.g20531b61.e 390 kB/s | 27 kB 00:00 2023-05-16T06:55:47.088 INFO:teuthology.orchestra.run.smithi092.stdout:(33/120): ceph-prometheus-alerts-16.2.13-145.g2 211 kB/s | 15 kB 00:00 2023-05-16T06:55:47.164 INFO:teuthology.orchestra.run.smithi092.stdout:(34/120): cephadm-16.2.13-145.g20531b61.el8.noa 1.1 MB/s | 84 kB 00:00 2023-05-16T06:55:47.236 INFO:teuthology.orchestra.run.smithi092.stdout:(35/120): cephfs-top-16.2.13-145.g20531b61.el8. 248 kB/s | 18 kB 00:00 2023-05-16T06:55:48.119 INFO:teuthology.orchestra.run.smithi158.stdout:(10/120): ceph-common-16.2.13-145.g20531b61.el8 2.0 MB/s | 24 MB 00:12 2023-05-16T06:55:48.200 INFO:teuthology.orchestra.run.smithi158.stdout:(11/120): cephfs-mirror-16.2.13-145.g20531b61.e 3.1 MB/s | 256 kB 00:00 2023-05-16T06:55:48.272 INFO:teuthology.orchestra.run.smithi158.stdout:(12/120): libcephfs-devel-16.2.13-145.g20531b61 356 kB/s | 25 kB 00:00 2023-05-16T06:55:48.573 INFO:teuthology.orchestra.run.smithi158.stdout:(13/120): libcephfs2-16.2.13-145.g20531b61.el8. 2.7 MB/s | 822 kB 00:00 2023-05-16T06:55:48.640 INFO:teuthology.orchestra.run.smithi158.stdout:(14/120): libcephsqlite-16.2.13-145.g20531b61.e 2.6 MB/s | 174 kB 00:00 2023-05-16T06:55:48.648 INFO:teuthology.orchestra.run.smithi092.stdout:(36/120): ceph-mgr-diskprediction-local-16.2.13 3.7 MB/s | 8.6 MB 00:02 2023-05-16T06:55:48.697 INFO:teuthology.orchestra.run.smithi158.stdout:(15/120): librados-devel-16.2.13-145.g20531b61. 2.2 MB/s | 131 kB 00:00 2023-05-16T06:55:48.887 INFO:teuthology.orchestra.run.smithi158.stdout:(16/120): libradosstriper1-16.2.13-145.g20531b6 2.5 MB/s | 493 kB 00:00 2023-05-16T06:55:49.978 INFO:teuthology.orchestra.run.smithi158.stdout:(17/120): ceph-osd-16.2.13-145.g20531b61.el8.x8 2.1 MB/s | 18 MB 00:08 2023-05-16T06:55:49.990 INFO:teuthology.orchestra.run.smithi158.stdout:(18/120): librgw2-16.2.13-145.g20531b61.el8.x86 3.4 MB/s | 3.8 MB 00:01 2023-05-16T06:55:50.025 INFO:teuthology.orchestra.run.smithi158.stdout:(19/120): python3-ceph-argparse-16.2.13-145.g20 954 kB/s | 45 kB 00:00 2023-05-16T06:55:50.053 INFO:teuthology.orchestra.run.smithi158.stdout:(20/120): python3-ceph-common-16.2.13-145.g2053 1.5 MB/s | 97 kB 00:00 2023-05-16T06:55:50.087 INFO:teuthology.orchestra.run.smithi158.stdout:(21/120): python3-cephfs-16.2.13-145.g20531b61. 3.4 MB/s | 215 kB 00:00 2023-05-16T06:55:50.200 INFO:teuthology.orchestra.run.smithi158.stdout:(22/120): python3-rados-16.2.13-145.g20531b61.e 2.6 MB/s | 389 kB 00:00 2023-05-16T06:55:50.229 INFO:teuthology.orchestra.run.smithi158.stdout:(23/120): python3-rbd-16.2.13-145.g20531b61.el8 2.5 MB/s | 368 kB 00:00 2023-05-16T06:55:50.260 INFO:teuthology.orchestra.run.smithi158.stdout:(24/120): python3-rgw-16.2.13-145.g20531b61.el8 1.9 MB/s | 114 kB 00:00 2023-05-16T06:55:50.296 INFO:teuthology.orchestra.run.smithi158.stdout:(25/120): rbd-fuse-16.2.13-145.g20531b61.el8.x8 1.3 MB/s | 88 kB 00:00 2023-05-16T06:55:50.368 INFO:teuthology.orchestra.run.smithi158.stdout:(26/120): rbd-nbd-16.2.13-145.g20531b61.el8.x86 2.4 MB/s | 173 kB 00:00 2023-05-16T06:55:50.424 INFO:teuthology.orchestra.run.smithi158.stdout:(27/120): ceph-grafana-dashboards-16.2.13-145.g 434 kB/s | 24 kB 00:00 2023-05-16T06:55:50.488 INFO:teuthology.orchestra.run.smithi158.stdout:(28/120): ceph-mgr-cephadm-16.2.13-145.g20531b6 1.8 MB/s | 116 kB 00:00 2023-05-16T06:55:51.231 INFO:teuthology.orchestra.run.smithi158.stdout:(29/120): rbd-mirror-16.2.13-145.g20531b61.el8. 3.8 MB/s | 3.7 MB 00:00 2023-05-16T06:55:51.320 INFO:teuthology.orchestra.run.smithi158.stdout:(30/120): ceph-mgr-dashboard-16.2.13-145.g20531 3.8 MB/s | 3.1 MB 00:00 2023-05-16T06:55:51.383 INFO:teuthology.orchestra.run.smithi158.stdout:(31/120): ceph-mgr-modules-core-16.2.13-145.g20 3.8 MB/s | 244 kB 00:00 2023-05-16T06:55:51.443 INFO:teuthology.orchestra.run.smithi158.stdout:(32/120): ceph-mgr-rook-16.2.13-145.g20531b61.e 461 kB/s | 27 kB 00:00 2023-05-16T06:55:51.489 INFO:teuthology.orchestra.run.smithi158.stdout:(33/120): ceph-prometheus-alerts-16.2.13-145.g2 322 kB/s | 15 kB 00:00 2023-05-16T06:55:51.554 INFO:teuthology.orchestra.run.smithi158.stdout:(34/120): cephadm-16.2.13-145.g20531b61.el8.noa 1.3 MB/s | 84 kB 00:00 2023-05-16T06:55:51.618 INFO:teuthology.orchestra.run.smithi158.stdout:(35/120): cephfs-top-16.2.13-145.g20531b61.el8. 282 kB/s | 18 kB 00:00 2023-05-16T06:55:51.868 INFO:teuthology.orchestra.run.smithi158.stdout:(36/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-05-16T06:55:51.909 INFO:teuthology.orchestra.run.smithi158.stdout:(37/120): leveldb-1.22-1.el8.x86_64.rpm 4.3 MB/s | 181 kB 00:00 2023-05-16T06:55:51.949 INFO:teuthology.orchestra.run.smithi158.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.5 MB/s | 59 kB 00:00 2023-05-16T06:55:51.989 INFO:teuthology.orchestra.run.smithi158.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.9 MB/s | 75 kB 00:00 2023-05-16T06:55:52.028 INFO:teuthology.orchestra.run.smithi158.stdout:(40/120): python3-apipkg-1.5-6.el8.noarch.rpm 543 kB/s | 21 kB 00:00 2023-05-16T06:55:52.068 INFO:teuthology.orchestra.run.smithi158.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.1 MB/s | 44 kB 00:00 2023-05-16T06:55:52.109 INFO:teuthology.orchestra.run.smithi158.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 4.4 MB/s | 185 kB 00:00 2023-05-16T06:55:52.149 INFO:teuthology.orchestra.run.smithi158.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 840 kB/s | 33 kB 00:00 2023-05-16T06:55:52.188 INFO:teuthology.orchestra.run.smithi158.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 417 kB/s | 16 kB 00:00 2023-05-16T06:55:52.230 INFO:teuthology.orchestra.run.smithi158.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-05-16T06:55:52.310 INFO:teuthology.orchestra.run.smithi158.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 4.7 MB/s | 384 kB 00:00 2023-05-16T06:55:52.391 INFO:teuthology.orchestra.run.smithi158.stdout:(47/120): python3-execnet-1.7.1-1.el8.noarch.rp 4.3 MB/s | 355 kB 00:00 2023-05-16T06:55:52.431 INFO:teuthology.orchestra.run.smithi158.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 2.0 MB/s | 81 kB 00:00 2023-05-16T06:55:52.472 INFO:teuthology.orchestra.run.smithi158.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 270 kB/s | 11 kB 00:00 2023-05-16T06:55:52.512 INFO:teuthology.orchestra.run.smithi158.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 447 kB/s | 18 kB 00:00 2023-05-16T06:55:52.710 INFO:teuthology.orchestra.run.smithi158.stdout:(51/120): python3-kubernetes-11.0.0-6.el8.noarc 5.9 MB/s | 1.2 MB 00:00 2023-05-16T06:55:52.750 INFO:teuthology.orchestra.run.smithi158.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 1.2 MB/s | 49 kB 00:00 2023-05-16T06:55:52.789 INFO:teuthology.orchestra.run.smithi158.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.5 MB/s | 59 kB 00:00 2023-05-16T06:55:52.832 INFO:teuthology.orchestra.run.smithi158.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2023-05-16T06:55:52.871 INFO:teuthology.orchestra.run.smithi158.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 398 kB/s | 16 kB 00:00 2023-05-16T06:55:52.895 INFO:teuthology.orchestra.run.smithi158.stdout:(56/120): ceph-mgr-diskprediction-local-16.2.13 5.2 MB/s | 8.6 MB 00:01 2023-05-16T06:55:52.911 INFO:teuthology.orchestra.run.smithi158.stdout:(57/120): python3-remoto-1.2.1-1.el8.noarch.rpm 1.2 MB/s | 49 kB 00:00 2023-05-16T06:55:52.952 INFO:teuthology.orchestra.run.smithi158.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 4.7 MB/s | 196 kB 00:00 2023-05-16T06:55:52.992 INFO:teuthology.orchestra.run.smithi158.stdout:(59/120): python3-rsa-4.8-1.el8.noarch.rpm 1.5 MB/s | 61 kB 00:00 2023-05-16T06:55:53.002 INFO:teuthology.orchestra.run.smithi158.stdout:(60/120): python3-repoze-lru-0.7-6.el8.noarch.r 312 kB/s | 34 kB 00:00 2023-05-16T06:55:53.031 INFO:teuthology.orchestra.run.smithi158.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 477 kB/s | 19 kB 00:00 2023-05-16T06:55:53.039 INFO:teuthology.orchestra.run.smithi158.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 661 kB/s | 24 kB 00:00 2023-05-16T06:55:53.071 INFO:teuthology.orchestra.run.smithi158.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 744 kB/s | 29 kB 00:00 2023-05-16T06:55:53.076 INFO:teuthology.orchestra.run.smithi158.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 753 kB/s | 27 kB 00:00 2023-05-16T06:55:53.113 INFO:teuthology.orchestra.run.smithi158.stdout:(65/120): python3-waitress-1.2.1-2.el8.1.noarch 5.6 MB/s | 241 kB 00:00 2023-05-16T06:55:53.156 INFO:teuthology.orchestra.run.smithi158.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-05-16T06:55:53.187 INFO:teuthology.orchestra.run.smithi158.stdout:(67/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.2 MB/s | 251 kB 00:00 2023-05-16T06:55:53.205 INFO:teuthology.orchestra.run.smithi158.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-05-16T06:55:53.209 INFO:teuthology.orchestra.run.smithi158.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-05-16T06:55:53.227 INFO:teuthology.orchestra.run.smithi158.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 592 kB/s | 23 kB 00:00 2023-05-16T06:55:53.250 INFO:teuthology.orchestra.run.smithi158.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 459 kB/s | 19 kB 00:00 2023-05-16T06:55:53.277 INFO:teuthology.orchestra.run.smithi158.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 837 kB/s | 43 kB 00:00 2023-05-16T06:55:53.283 INFO:teuthology.orchestra.run.smithi158.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 4.7 MB/s | 155 kB 00:00 2023-05-16T06:55:53.308 INFO:teuthology.orchestra.run.smithi158.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 7.1 MB/s | 220 kB 00:00 2023-05-16T06:55:53.320 INFO:teuthology.orchestra.run.smithi158.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 1.3 MB/s | 48 kB 00:00 2023-05-16T06:55:53.341 INFO:teuthology.orchestra.run.smithi158.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 6.8 MB/s | 229 kB 00:00 2023-05-16T06:55:53.349 INFO:teuthology.orchestra.run.smithi158.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 1.5 MB/s | 43 kB 00:00 2023-05-16T06:55:53.377 INFO:teuthology.orchestra.run.smithi158.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 657 kB/s | 18 kB 00:00 2023-05-16T06:55:53.398 INFO:teuthology.orchestra.run.smithi158.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 41 MB/s | 2.3 MB 00:00 2023-05-16T06:55:53.417 INFO:teuthology.orchestra.run.smithi158.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 15 MB/s | 605 kB 00:00 2023-05-16T06:55:53.428 INFO:teuthology.orchestra.run.smithi158.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 2.7 MB/s | 83 kB 00:00 2023-05-16T06:55:53.447 INFO:teuthology.orchestra.run.smithi158.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 1.9 MB/s | 59 kB 00:00 2023-05-16T06:55:53.460 INFO:teuthology.orchestra.run.smithi158.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 5.2 MB/s | 169 kB 00:00 2023-05-16T06:55:53.477 INFO:teuthology.orchestra.run.smithi158.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.2 MB/s | 38 kB 00:00 2023-05-16T06:55:53.491 INFO:teuthology.orchestra.run.smithi158.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 2.8 MB/s | 86 kB 00:00 2023-05-16T06:55:53.505 INFO:teuthology.orchestra.run.smithi158.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.0 MB/s | 58 kB 00:00 2023-05-16T06:55:53.517 INFO:teuthology.orchestra.run.smithi158.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.7 MB/s | 47 kB 00:00 2023-05-16T06:55:53.538 INFO:teuthology.orchestra.run.smithi158.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 17 MB/s | 559 kB 00:00 2023-05-16T06:55:53.556 INFO:teuthology.orchestra.run.smithi158.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 16 MB/s | 639 kB 00:00 2023-05-16T06:55:53.572 INFO:teuthology.orchestra.run.smithi158.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-05-16T06:55:53.583 INFO:teuthology.orchestra.run.smithi158.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 7.2 MB/s | 200 kB 00:00 2023-05-16T06:55:53.604 INFO:teuthology.orchestra.run.smithi158.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 3.0 MB/s | 97 kB 00:00 2023-05-16T06:55:53.630 INFO:teuthology.orchestra.run.smithi158.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-05-16T06:55:53.653 INFO:teuthology.orchestra.run.smithi158.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.2 MB/s | 27 kB 00:00 2023-05-16T06:55:53.677 INFO:teuthology.orchestra.run.smithi158.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 6.5 MB/s | 161 kB 00:00 2023-05-16T06:55:53.705 INFO:teuthology.orchestra.run.smithi158.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 12 MB/s | 320 kB 00:00 2023-05-16T06:55:53.735 INFO:teuthology.orchestra.run.smithi158.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 15 MB/s | 457 kB 00:00 2023-05-16T06:55:53.842 INFO:teuthology.orchestra.run.smithi158.stdout:(98/120): python3-scipy-1.0.0-20.module+el8.1.0 56 MB/s | 15 MB 00:00 2023-05-16T06:55:53.869 INFO:teuthology.orchestra.run.smithi158.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 1.4 MB/s | 39 kB 00:00 2023-05-16T06:55:53.900 INFO:teuthology.orchestra.run.smithi158.stdout:(100/120): python3-babel-2.5.1-5.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-05-16T06:55:53.908 INFO:teuthology.orchestra.run.smithi158.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 18 MB/s | 688 kB 00:00 2023-05-16T06:55:53.929 INFO:teuthology.orchestra.run.smithi158.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 8.8 MB/s | 259 kB 00:00 2023-05-16T06:55:53.941 INFO:teuthology.orchestra.run.smithi158.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 6.7 MB/s | 225 kB 00:00 2023-05-16T06:55:53.960 INFO:teuthology.orchestra.run.smithi158.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.9 MB/s | 126 kB 00:00 2023-05-16T06:55:53.990 INFO:teuthology.orchestra.run.smithi158.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 6.3 MB/s | 187 kB 00:00 2023-05-16T06:55:54.008 INFO:teuthology.orchestra.run.smithi158.stdout:(106/120): python3-numpy-1.14.3-9.el8.x86_64.rp 55 MB/s | 3.7 MB 00:00 2023-05-16T06:55:54.023 INFO:teuthology.orchestra.run.smithi158.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 16 MB/s | 538 kB 00:00 2023-05-16T06:55:54.038 INFO:teuthology.orchestra.run.smithi158.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 9.9 MB/s | 302 kB 00:00 2023-05-16T06:55:54.054 INFO:teuthology.orchestra.run.smithi158.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 5.1 MB/s | 157 kB 00:00 2023-05-16T06:55:54.066 INFO:teuthology.orchestra.run.smithi158.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 3.9 MB/s | 110 kB 00:00 2023-05-16T06:55:54.086 INFO:teuthology.orchestra.run.smithi158.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 11 MB/s | 353 kB 00:00 2023-05-16T06:55:54.093 INFO:teuthology.orchestra.run.smithi158.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 754 kB/s | 20 kB 00:00 2023-05-16T06:55:54.125 INFO:teuthology.orchestra.run.smithi158.stdout:(113/120): libtool-2.4.6-25.el8.x86_64.rpm 18 MB/s | 709 kB 00:00 2023-05-16T06:55:54.126 INFO:teuthology.orchestra.run.smithi158.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-05-16T06:55:54.218 INFO:teuthology.orchestra.run.smithi158.stdout:(115/120): openblas-0.3.12-1.el8.x86_64.rpm 50 MB/s | 4.6 MB 00:00 2023-05-16T06:55:54.235 INFO:teuthology.orchestra.run.smithi158.stdout:(116/120): openblas-threads-0.3.12-1.el8.x86_64 43 MB/s | 4.7 MB 00:00 2023-05-16T06:55:54.249 INFO:teuthology.orchestra.run.smithi158.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 22 MB/s | 713 kB 00:00 2023-05-16T06:55:54.472 INFO:teuthology.orchestra.run.smithi092.stdout:(37/120): ceph-test-16.2.13-145.g20531b61.el8.x 4.0 MB/s | 51 MB 00:12 2023-05-16T06:55:54.876 INFO:teuthology.orchestra.run.smithi158.stdout:(118/120): ceph-test-16.2.13-145.g20531b61.el8. 6.5 MB/s | 51 MB 00:07 2023-05-16T06:55:54.931 INFO:teuthology.orchestra.run.smithi158.stdout:(119/120): librados2-16.2.13-145.g20531b61.el8. 5.4 MB/s | 3.7 MB 00:00 2023-05-16T06:55:54.986 INFO:teuthology.orchestra.run.smithi158.stdout:(120/120): librbd1-16.2.13-145.g20531b61.el8.x8 5.7 MB/s | 4.2 MB 00:00 2023-05-16T06:55:54.990 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-05-16T06:55:54.990 INFO:teuthology.orchestra.run.smithi158.stdout:Total 10 MB/s | 200 MB 00:19 2023-05-16T06:55:55.042 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-05-16T06:55:55.157 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-05-16T06:55:55.157 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-05-16T06:55:56.809 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-05-16T06:55:56.813 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-05-16T06:56:01.271 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-05-16T06:56:01.478 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-05-16T06:56:01.664 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/122 2023-05-16T06:56:01.835 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-05-16T06:56:01.959 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-05-16T06:56:02.152 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-05-16T06:56:02.525 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-05-16T06:56:02.751 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librados2-2:16.2.13-145.g20531b61.el8.x86_64 5/122 2023-05-16T06:56:03.167 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:16.2.13-145.g20531b61.el8.x86_64 5/122 2023-05-16T06:56:03.335 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librbd1-2:16.2.13-145.g20531b61.el8.x86_64 6/122 2023-05-16T06:56:03.463 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:16.2.13-145.g20531b61.el8.x86_64 6/122 2023-05-16T06:56:03.685 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 7/122 2023-05-16T06:56:03.785 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 7/122 2023-05-16T06:56:03.975 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rados-2:16.2.13-145.g20531b61.el8.x86_64 8/122 2023-05-16T06:56:04.438 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/122 2023-05-16T06:56:04.585 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librgw2-2:16.2.13-145.g20531b61.el8.x86_64 10/122 2023-05-16T06:56:04.692 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librgw2-2:16.2.13-145.g20531b61.el8.x86_64 10/122 2023-05-16T06:56:04.958 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-05-16T06:56:05.292 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-05-16T06:56:05.526 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 13/122 2023-05-16T06:56:05.632 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 13/122 2023-05-16T06:56:05.795 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libradosstriper1-2:16.2.13-145.g20531b61.el8.x86 14/122 2023-05-16T06:56:05.859 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libradosstriper1-2:16.2.13-145.g20531b61.el8.x86 14/122 2023-05-16T06:56:06.034 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-05-16T06:56:06.325 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-05-16T06:56:06.525 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-05-16T06:56:06.653 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-05-16T06:56:06.818 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-05-16T06:56:06.961 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-05-16T06:56:07.109 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-05-16T06:56:07.654 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-05-16T06:56:07.834 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-05-16T06:56:08.370 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-05-16T06:56:08.551 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-05-16T06:56:09.266 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-05-16T06:56:09.537 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-05-16T06:56:09.759 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-05-16T06:56:10.023 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-05-16T06:56:10.173 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-common-2:16.2.13-145.g20531b61.el8. 26/122 2023-05-16T06:56:10.289 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-argparse-2:16.2.13-145.g20531b61.el 27/122 2023-05-16T06:56:10.550 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cephfs-2:16.2.13-145.g20531b61.el8.x86_6 28/122 2023-05-16T06:56:11.042 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-05-16T06:56:11.335 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-05-16T06:56:11.525 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-05-16T06:56:13.588 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-05-16T06:56:13.963 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-05-16T06:56:14.962 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-05-16T06:56:15.303 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-05-16T06:56:15.553 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-05-16T06:56:15.788 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-05-16T06:56:15.979 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-05-16T06:56:16.176 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rgw-2:16.2.13-145.g20531b61.el8.x86_64 38/122 2023-05-16T06:56:16.347 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rbd-2:16.2.13-145.g20531b61.el8.x86_64 39/122 2023-05-16T06:56:16.641 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librados-devel-2:16.2.13-145.g20531b61.el8.x86_6 40/122 2023-05-16T06:56:16.856 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-05-16T06:56:17.065 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-05-16T06:56:17.263 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-05-16T06:56:17.444 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-05-16T06:56:17.599 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-05-16T06:56:17.710 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-05-16T06:56:17.883 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-05-16T06:56:18.061 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-05-16T06:56:18.210 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-05-16T06:56:18.380 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-05-16T06:56:18.528 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-05-16T06:56:18.669 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-05-16T06:56:18.804 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-05-16T06:56:19.025 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-05-16T06:56:19.170 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-05-16T06:56:19.483 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-05-16T06:56:19.753 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-05-16T06:56:19.983 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-05-16T06:56:20.251 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-05-16T06:56:20.399 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-05-16T06:56:20.563 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-05-16T06:56:20.828 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-05-16T06:56:21.041 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 62/122 2023-05-16T06:56:21.217 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-05-16T06:56:21.346 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-05-16T06:56:21.520 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-05-16T06:56:21.705 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-05-16T06:56:21.881 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-05-16T06:56:22.061 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 68/122 2023-05-16T06:56:22.241 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 69/122 2023-05-16T06:56:22.470 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 70/122 2023-05-16T06:56:22.670 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 71/122 2023-05-16T06:56:22.914 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 72/122 2023-05-16T06:56:23.212 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-portend-2.6-1.el8.noarch 73/122 2023-05-16T06:56:23.465 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 74/122 2023-05-16T06:56:23.650 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 75/122 2023-05-16T06:56:23.886 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 76/122 2023-05-16T06:56:25.047 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 77/122 2023-05-16T06:56:25.337 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 78/122 2023-05-16T06:56:25.558 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 79/122 2023-05-16T06:56:25.851 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 80/122 2023-05-16T06:56:26.175 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 81/122 2023-05-16T06:56:26.346 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/122 2023-05-16T06:56:26.612 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 83/122 2023-05-16T06:56:26.853 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 84/122 2023-05-16T06:56:27.040 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 85/122 2023-05-16T06:56:27.223 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 86/122 2023-05-16T06:56:27.456 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 87/122 2023-05-16T06:56:32.012 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 88/122 2023-05-16T06:56:32.222 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 88/122 2023-05-16T06:56:32.938 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 88/122 2023-05-16T06:56:33.248 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 89/122 2023-05-16T06:56:33.324 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 89/122 2023-05-16T06:56:33.438 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 90/122 2023-05-16T06:56:51.105 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 90/122 2023-05-16T06:56:51.106 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /sys 2023-05-16T06:56:51.106 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /proc 2023-05-16T06:56:51.106 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /dev 2023-05-16T06:56:51.106 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /run 2023-05-16T06:56:51.106 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /mnt 2023-05-16T06:56:51.107 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /var/tmp 2023-05-16T06:56:51.107 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /home 2023-05-16T06:56:51.107 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /tmp 2023-05-16T06:56:51.107 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /dev 2023-05-16T06:56:51.107 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:56:51.502 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 91/122 2023-05-16T06:56:51.931 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 91/122 2023-05-16T06:56:52.136 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 92/122 2023-05-16T06:56:53.635 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 92/122 2023-05-16T06:56:53.810 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 93/122 2023-05-16T06:56:53.831 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 93/122 2023-05-16T06:56:56.358 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephadm-2:16.2.13-145.g20531b61.el8.noarch 94/122 2023-05-16T06:56:56.480 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephadm-2:16.2.13-145.g20531b61.el8.noarch 94/122 2023-05-16T06:56:56.721 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-prometheus-alerts-2:16.2.13-145.g20531b61.e 95/122 2023-05-16T06:56:56.913 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-grafana-dashboards-2:16.2.13-145.g20531b61. 96/122 2023-05-16T06:56:57.068 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noa 97/122 2023-05-16T06:56:58.011 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noa 97/122 2023-05-16T06:56:58.235 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.n 98/122 2023-05-16T06:56:59.272 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.n 98/122 2023-05-16T06:56:59.434 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.13-145.g205 99/122 2023-05-16T06:56:59.528 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.13-145.g205 99/122 2023-05-16T06:56:59.941 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-modules-core-2:16.2.13-145.g20531b61.el 100/122 2023-05-16T06:57:00.110 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 101/122 2023-05-16T06:57:00.135 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 101/122 2023-05-16T06:57:00.267 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 102/122 2023-05-16T06:57:00.278 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 102/122 2023-05-16T06:57:01.344 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-2:16.2.13-145.g20531b61.el8.x86_64 103/122 2023-05-16T06:57:01.543 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 104/122 2023-05-16T06:57:01.632 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 104/122 2023-05-16T06:57:01.801 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-immutable-object-cache-2:16.2.13-145.g20531 105/122 2023-05-16T06:57:01.841 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.13-145.g20531 105/122 2023-05-16T06:57:01.976 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 106/122 2023-05-16T06:57:02.318 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 106/122 2023-05-16T06:57:02.467 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 107/122 2023-05-16T06:57:06.951 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 107/122 2023-05-16T06:57:07.863 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-test-2:16.2.13-145.g20531b61.el8.x86_64 108/122 2023-05-16T06:57:08.199 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-05-16T06:57:08.402 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-fuse-2:16.2.13-145.g20531b61.el8.x86_64 110/122 2023-05-16T06:57:08.633 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-05-16T06:57:08.815 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-05-16T06:57:08.826 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-05-16T06:57:08.962 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs-devel-2:16.2.13-145.g20531b61.el8.x86_ 113/122 2023-05-16T06:57:09.087 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-05-16T06:57:09.231 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-top-2:16.2.13-145.g20531b61.el8.noarch 115/122 2023-05-16T06:57:09.422 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-fuse-2:16.2.13-145.g20531b61.el8.x86_64 116/122 2023-05-16T06:57:09.675 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-nbd-2:16.2.13-145.g20531b61.el8.x86_64 117/122 2023-05-16T06:57:09.840 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-05-16T06:57:09.896 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-05-16T06:57:10.115 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-05-16T06:57:10.123 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-05-16T06:57:10.234 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-05-16T06:57:10.234 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-05-16T06:57:10.435 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-05-16T06:57:10.435 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-05-16T06:57:10.938 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-05-16T06:57:10.939 INFO:teuthology.orchestra.run.smithi158.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-05-16T06:57:10.939 INFO:teuthology.orchestra.run.smithi158.stdout:All rules containing unresolvable specifiers will be skipped. 2023-05-16T06:57:10.939 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:57:11.258 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-2:16.2.13-145.g20531b61.el8.x86_64 1/122 2023-05-16T06:57:11.258 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 2/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 3/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-fuse-2:16.2.13-145.g20531b61.el8.x86_64 4/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-immutable-object-cache-2:16.2.13-145.g20531 5/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 6/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 7/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 8/122 2023-05-16T06:57:11.259 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 9/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 10/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 11/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-test-2:16.2.13-145.g20531b61.el8.x86_64 12/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 13/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs-devel-2:16.2.13-145.g20531b61.el8.x86_ 14/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 15/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 16/122 2023-05-16T06:57:11.260 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados-devel-2:16.2.13-145.g20531b61.el8.x86_6 17/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libradosstriper1-2:16.2.13-145.g20531b61.el8.x86 18/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librgw2-2:16.2.13-145.g20531b61.el8.x86_64 19/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-argparse-2:16.2.13-145.g20531b61.el 20/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-common-2:16.2.13-145.g20531b61.el8. 21/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cephfs-2:16.2.13-145.g20531b61.el8.x86_6 22/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rados-2:16.2.13-145.g20531b61.el8.x86_64 23/122 2023-05-16T06:57:11.261 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rbd-2:16.2.13-145.g20531b61.el8.x86_64 24/122 2023-05-16T06:57:11.262 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rgw-2:16.2.13-145.g20531b61.el8.x86_64 25/122 2023-05-16T06:57:11.268 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-fuse-2:16.2.13-145.g20531b61.el8.x86_64 26/122 2023-05-16T06:57:11.268 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 27/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-nbd-2:16.2.13-145.g20531b61.el8.x86_64 28/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-grafana-dashboards-2:16.2.13-145.g20531b61. 29/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noa 30/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.n 31/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.13-145.g205 32/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-modules-core-2:16.2.13-145.g20531b61.el 33/122 2023-05-16T06:57:11.269 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 34/122 2023-05-16T06:57:11.270 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-prometheus-alerts-2:16.2.13-145.g20531b61.e 35/122 2023-05-16T06:57:11.270 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephadm-2:16.2.13-145.g20531b61.el8.noarch 36/122 2023-05-16T06:57:11.270 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-top-2:16.2.13-145.g20531b61.el8.noarch 37/122 2023-05-16T06:57:11.270 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 38/122 2023-05-16T06:57:11.271 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : leveldb-1.22-1.el8.x86_64 39/122 2023-05-16T06:57:11.271 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-05-16T06:57:11.271 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-05-16T06:57:11.271 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 42/122 2023-05-16T06:57:11.271 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-05-16T06:57:11.272 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-05-16T06:57:11.272 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-05-16T06:57:11.272 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-05-16T06:57:11.272 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-05-16T06:57:11.272 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-05-16T06:57:11.272 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 49/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-05-16T06:57:11.273 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 58/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 61/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/122 2023-05-16T06:57:11.274 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 66/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-05-16T06:57:11.275 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-05-16T06:57:11.276 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-05-16T06:57:11.277 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-05-16T06:57:11.277 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-05-16T06:57:11.277 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-05-16T06:57:11.277 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-05-16T06:57:11.277 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-05-16T06:57:11.278 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-05-16T06:57:11.279 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-05-16T06:57:11.280 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-05-16T06:57:11.280 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-05-16T06:57:11.280 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-05-16T06:57:11.280 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-05-16T06:57:11.280 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-05-16T06:57:11.280 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-05-16T06:57:11.281 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:16.2.13-145.g20531b61.el8.x86_64 119/122 2023-05-16T06:57:11.282 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-05-16T06:57:11.283 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:16.2.13-145.g20531b61.el8.x86_64 121/122 2023-05-16T06:57:13.437 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-05-16T06:57:13.437 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:01:46 ago on Tue 16 May 2023 06:55:27 AM UTC. 2023-05-16T06:57:15.496 INFO:teuthology.orchestra.run.smithi158.stdout:Installed products updated. 2023-05-16T06:57:15.657 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:57:15.657 INFO:teuthology.orchestra.run.smithi158.stdout:Upgraded: 2023-05-16T06:57:15.657 INFO:teuthology.orchestra.run.smithi158.stdout: librados2-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.657 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout: automake-1.16.1-7.el8.noarch 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout: bison-3.0.4-10.el8.x86_64 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.658 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.659 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.660 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-top-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: flex-2.6.1-9.el8.x86_64 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: fuse-2.9.7-12.el8.x86_64 2023-05-16T06:57:15.661 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: jq-1.5-12.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: leveldb-1.22-1.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.662 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: liboath-2.6.2-3.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.663 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: libtool-2.4.6-25.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind-1.3.1-3.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-05-16T06:57:15.664 INFO:teuthology.orchestra.run.smithi158.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-0.3.12-1.el8.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-05-16T06:57:15.665 INFO:teuthology.orchestra.run.smithi158.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-apipkg-1.5-6.el8.noarch 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel-2.5.1-5.el8.noarch 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.666 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-6.2-6.el8.noarch 2023-05-16T06:57:15.667 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-05-16T06:57:15.668 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako-1.0.6-13.el8.noarch 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend-2.6-1.el8.noarch 2023-05-16T06:57:15.669 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz-2017.2-9.el8.noarch 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-05-16T06:57:15.670 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes-2.4.1-12.el8.noarch 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa-4.8-1.el8.noarch 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-05-16T06:57:15.671 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-05-16T06:57:15.672 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-05-16T06:57:15.673 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.673 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.673 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T06:57:15.673 INFO:teuthology.orchestra.run.smithi158.stdout: socat-1.7.3.3-2.el8.x86_64 2023-05-16T06:57:15.673 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-05-16T06:57:15.673 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-05-16T06:57:15.674 INFO:teuthology.orchestra.run.smithi158.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-05-16T06:57:15.674 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T06:57:15.674 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-05-16T06:57:15.960 DEBUG:teuthology.parallel:result is None 2023-05-16T06:57:56.287 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-05-16T06:57:56.511 INFO:teuthology.orchestra.run.smithi092.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.4 kB/s | 306 kB 02:09 2023-05-16T06:58:00.382 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] leveldb-1.22-1.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/leveldb-1.22-1.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-05-16T06:58:00.528 INFO:teuthology.orchestra.run.smithi092.stdout:(39/120): leveldb-1.22-1.el8.x86_64.rpm 1.4 kB/s | 181 kB 02:11 2023-05-16T06:58:04.478 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-05-16T06:58:04.597 INFO:teuthology.orchestra.run.smithi092.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 462 B/s | 59 kB 02:10 2023-05-16T06:58:04.648 INFO:teuthology.orchestra.run.smithi092.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 848 kB/s | 44 kB 00:00 2023-05-16T06:58:04.722 INFO:teuthology.orchestra.run.smithi092.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.5 MB/s | 185 kB 00:00 2023-05-16T06:58:04.759 INFO:teuthology.orchestra.run.smithi092.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 905 kB/s | 33 kB 00:00 2023-05-16T06:58:04.795 INFO:teuthology.orchestra.run.smithi092.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 449 kB/s | 16 kB 00:00 2023-05-16T06:58:04.834 INFO:teuthology.orchestra.run.smithi092.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.4 MB/s | 173 kB 00:00 2023-05-16T06:58:04.908 INFO:teuthology.orchestra.run.smithi092.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 5.1 MB/s | 384 kB 00:00 2023-05-16T06:58:04.952 INFO:teuthology.orchestra.run.smithi092.stdout:(47/120): python3-execnet-1.7.1-1.el8.noarch.rp 7.8 MB/s | 355 kB 00:00 2023-05-16T06:58:04.991 INFO:teuthology.orchestra.run.smithi092.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 2.1 MB/s | 81 kB 00:00 2023-05-16T06:58:05.029 INFO:teuthology.orchestra.run.smithi092.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 294 kB/s | 11 kB 00:00 2023-05-16T06:58:05.065 INFO:teuthology.orchestra.run.smithi092.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 483 kB/s | 18 kB 00:00 2023-05-16T06:58:05.186 INFO:teuthology.orchestra.run.smithi092.stdout:(51/120): python3-kubernetes-11.0.0-6.el8.noarc 9.7 MB/s | 1.2 MB 00:00 2023-05-16T06:58:05.225 INFO:teuthology.orchestra.run.smithi092.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 1.2 MB/s | 49 kB 00:00 2023-05-16T06:58:05.262 INFO:teuthology.orchestra.run.smithi092.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.6 MB/s | 59 kB 00:00 2023-05-16T06:58:05.302 INFO:teuthology.orchestra.run.smithi092.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 7.0 MB/s | 283 kB 00:00 2023-05-16T06:58:05.338 INFO:teuthology.orchestra.run.smithi092.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 430 kB/s | 16 kB 00:00 2023-05-16T06:58:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:(56/120): python3-remoto-1.2.1-1.el8.noarch.rpm 1.3 MB/s | 49 kB 00:00 2023-05-16T06:58:05.412 INFO:teuthology.orchestra.run.smithi092.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 923 kB/s | 34 kB 00:00 2023-05-16T06:58:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 4.9 MB/s | 196 kB 00:00 2023-05-16T06:58:05.488 INFO:teuthology.orchestra.run.smithi092.stdout:(59/120): python3-rsa-4.8-1.el8.noarch.rpm 1.6 MB/s | 61 kB 00:00 2023-05-16T06:58:05.525 INFO:teuthology.orchestra.run.smithi092.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 509 kB/s | 19 kB 00:00 2023-05-16T06:58:05.562 INFO:teuthology.orchestra.run.smithi092.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 658 kB/s | 24 kB 00:00 2023-05-16T06:58:05.600 INFO:teuthology.orchestra.run.smithi092.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 776 kB/s | 29 kB 00:00 2023-05-16T06:58:05.644 INFO:teuthology.orchestra.run.smithi092.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 628 kB/s | 27 kB 00:00 2023-05-16T06:58:05.684 INFO:teuthology.orchestra.run.smithi092.stdout:(64/120): python3-waitress-1.2.1-2.el8.1.noarch 5.9 MB/s | 241 kB 00:00 2023-05-16T06:58:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 5.9 MB/s | 251 kB 00:00 2023-05-16T06:58:05.762 INFO:teuthology.orchestra.run.smithi092.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.6 MB/s | 61 kB 00:00 2023-05-16T06:58:05.799 INFO:teuthology.orchestra.run.smithi092.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 2.3 MB/s | 85 kB 00:00 2023-05-16T06:58:05.839 INFO:teuthology.orchestra.run.smithi092.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 581 kB/s | 23 kB 00:00 2023-05-16T06:58:05.844 INFO:teuthology.orchestra.run.smithi092.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-05-16T06:58:05.877 INFO:teuthology.orchestra.run.smithi092.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 583 kB/s | 19 kB 00:00 2023-05-16T06:58:05.892 INFO:teuthology.orchestra.run.smithi092.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.7 MB/s | 43 kB 00:00 2023-05-16T06:58:05.908 INFO:teuthology.orchestra.run.smithi092.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-05-16T06:58:05.924 INFO:teuthology.orchestra.run.smithi092.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-05-16T06:58:05.938 INFO:teuthology.orchestra.run.smithi092.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.5 MB/s | 48 kB 00:00 2023-05-16T06:58:05.953 INFO:teuthology.orchestra.run.smithi092.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-05-16T06:58:05.966 INFO:teuthology.orchestra.run.smithi092.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.4 MB/s | 43 kB 00:00 2023-05-16T06:58:06.003 INFO:teuthology.orchestra.run.smithi092.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 63 MB/s | 2.3 MB 00:00 2023-05-16T06:58:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.4 MB/s | 18 kB 00:00 2023-05-16T06:58:06.034 INFO:teuthology.orchestra.run.smithi092.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 33 MB/s | 605 kB 00:00 2023-05-16T06:58:06.049 INFO:teuthology.orchestra.run.smithi092.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 5.2 MB/s | 83 kB 00:00 2023-05-16T06:58:06.063 INFO:teuthology.orchestra.run.smithi092.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.2 MB/s | 59 kB 00:00 2023-05-16T06:58:06.079 INFO:teuthology.orchestra.run.smithi092.stdout:(82/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-05-16T06:58:06.094 INFO:teuthology.orchestra.run.smithi092.stdout:(83/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.6 MB/s | 38 kB 00:00 2023-05-16T06:58:06.109 INFO:teuthology.orchestra.run.smithi092.stdout:(84/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.6 MB/s | 86 kB 00:00 2023-05-16T06:58:06.126 INFO:teuthology.orchestra.run.smithi092.stdout:(85/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.5 MB/s | 58 kB 00:00 2023-05-16T06:58:06.144 INFO:teuthology.orchestra.run.smithi092.stdout:(86/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.6 MB/s | 47 kB 00:00 2023-05-16T06:58:06.163 INFO:teuthology.orchestra.run.smithi092.stdout:(87/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 28 MB/s | 559 kB 00:00 2023-05-16T06:58:06.184 INFO:teuthology.orchestra.run.smithi092.stdout:(88/120): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-05-16T06:58:06.198 INFO:teuthology.orchestra.run.smithi092.stdout:(89/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.1 MB/s | 70 kB 00:00 2023-05-16T06:58:06.215 INFO:teuthology.orchestra.run.smithi092.stdout:(90/120): libselinux-devel-2.9-5.el8.x86_64.rpm 12 MB/s | 200 kB 00:00 2023-05-16T06:58:06.228 INFO:teuthology.orchestra.run.smithi092.stdout:(91/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 7.2 MB/s | 97 kB 00:00 2023-05-16T06:58:06.454 INFO:teuthology.orchestra.run.smithi092.stdout:(92/120): python3-scipy-1.0.0-20.module+el8.1.0 65 MB/s | 15 MB 00:00 2023-05-16T06:58:06.466 INFO:teuthology.orchestra.run.smithi092.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.6 MB/s | 44 kB 00:00 2023-05-16T06:58:06.478 INFO:teuthology.orchestra.run.smithi092.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.3 MB/s | 27 kB 00:00 2023-05-16T06:58:06.490 INFO:teuthology.orchestra.run.smithi092.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 12 MB/s | 161 kB 00:00 2023-05-16T06:58:06.507 INFO:teuthology.orchestra.run.smithi092.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 19 MB/s | 320 kB 00:00 2023-05-16T06:58:06.524 INFO:teuthology.orchestra.run.smithi092.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 27 MB/s | 457 kB 00:00 2023-05-16T06:58:06.581 INFO:teuthology.orchestra.run.smithi092.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 84 MB/s | 4.8 MB 00:00 2023-05-16T06:58:06.595 INFO:teuthology.orchestra.run.smithi092.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.8 MB/s | 39 kB 00:00 2023-05-16T06:58:06.614 INFO:teuthology.orchestra.run.smithi092.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 35 MB/s | 688 kB 00:00 2023-05-16T06:58:06.630 INFO:teuthology.orchestra.run.smithi092.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 17 MB/s | 259 kB 00:00 2023-05-16T06:58:06.645 INFO:teuthology.orchestra.run.smithi092.stdout:(102/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 15 MB/s | 225 kB 00:00 2023-05-16T06:58:06.660 INFO:teuthology.orchestra.run.smithi092.stdout:(103/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.3 MB/s | 126 kB 00:00 2023-05-16T06:58:06.706 INFO:teuthology.orchestra.run.smithi092.stdout:(104/120): python3-numpy-1.14.3-9.el8.x86_64.rp 82 MB/s | 3.7 MB 00:00 2023-05-16T06:58:06.720 INFO:teuthology.orchestra.run.smithi092.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 14 MB/s | 187 kB 00:00 2023-05-16T06:58:06.735 INFO:teuthology.orchestra.run.smithi092.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 34 MB/s | 538 kB 00:00 2023-05-16T06:58:06.750 INFO:teuthology.orchestra.run.smithi092.stdout:(107/120): socat-1.7.3.3-2.el8.x86_64.rpm 20 MB/s | 302 kB 00:00 2023-05-16T06:58:06.763 INFO:teuthology.orchestra.run.smithi092.stdout:(108/120): python3-mako-1.0.6-13.el8.noarch.rpm 12 MB/s | 157 kB 00:00 2023-05-16T06:58:06.775 INFO:teuthology.orchestra.run.smithi092.stdout:(109/120): python3-pyasn1-modules-0.3.7-6.el8.n 9.0 MB/s | 110 kB 00:00 2023-05-16T06:58:06.789 INFO:teuthology.orchestra.run.smithi092.stdout:(110/120): librdkafka-0.11.4-1.el8.x86_64.rpm 24 MB/s | 353 kB 00:00 2023-05-16T06:58:06.802 INFO:teuthology.orchestra.run.smithi092.stdout:(111/120): python3-jsonpointer-1.10-11.el8.noar 1.7 MB/s | 20 kB 00:00 2023-05-16T06:58:06.818 INFO:teuthology.orchestra.run.smithi092.stdout:(112/120): libtool-2.4.6-25.el8.x86_64.rpm 40 MB/s | 709 kB 00:00 2023-05-16T06:58:06.831 INFO:teuthology.orchestra.run.smithi092.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.4 MB/s | 54 kB 00:00 2023-05-16T06:58:06.883 INFO:teuthology.orchestra.run.smithi092.stdout:(114/120): openblas-threads-0.3.12-1.el8.x86_64 90 MB/s | 4.7 MB 00:00 2023-05-16T06:58:06.936 INFO:teuthology.orchestra.run.smithi092.stdout:(115/120): openblas-0.3.12-1.el8.x86_64.rpm 88 MB/s | 4.6 MB 00:00 2023-05-16T06:58:06.954 INFO:teuthology.orchestra.run.smithi092.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 40 MB/s | 713 kB 00:00 2023-05-16T06:58:07.579 INFO:teuthology.orchestra.run.smithi092.stdout:(117/120): librados2-16.2.13-145.g20531b61.el8. 6.0 MB/s | 3.7 MB 00:00 2023-05-16T06:58:07.762 INFO:teuthology.orchestra.run.smithi092.stdout:(118/120): librbd1-16.2.13-145.g20531b61.el8.x8 23 MB/s | 4.2 MB 00:00 2023-05-16T07:00:07.358 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-05-16T07:00:07.522 INFO:teuthology.orchestra.run.smithi092.stdout:(119/120): libunwind-1.3.1-3.el8.x86_64.rpm 586 B/s | 75 kB 02:11 2023-05-16T07:00:11.454 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] python3-apipkg-1.5-6.el8.noarch.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/p/python3-apipkg-1.5-6.el8.noarch.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-05-16T07:00:11.531 INFO:teuthology.orchestra.run.smithi092.stdout:(120/120): python3-apipkg-1.5-6.el8.noarch.rpm 166 B/s | 21 kB 02:11 2023-05-16T07:00:11.532 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-05-16T07:00:11.532 INFO:teuthology.orchestra.run.smithi092.stdout:Total 742 kB/s | 200 MB 04:36 2023-05-16T07:00:11.584 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-05-16T07:00:11.699 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-05-16T07:00:11.699 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-05-16T07:00:13.406 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-05-16T07:00:13.410 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-05-16T07:00:17.892 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-05-16T07:00:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-05-16T07:00:18.299 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/122 2023-05-16T07:00:18.514 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-05-16T07:00:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-05-16T07:00:18.830 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-05-16T07:00:19.231 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-05-16T07:00:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:16.2.13-145.g20531b61.el8.x86_64 5/122 2023-05-16T07:00:19.847 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.13-145.g20531b61.el8.x86_64 5/122 2023-05-16T07:00:20.030 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:16.2.13-145.g20531b61.el8.x86_64 6/122 2023-05-16T07:00:20.161 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.13-145.g20531b61.el8.x86_64 6/122 2023-05-16T07:00:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 7/122 2023-05-16T07:00:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 7/122 2023-05-16T07:00:20.676 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:16.2.13-145.g20531b61.el8.x86_64 8/122 2023-05-16T07:00:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/122 2023-05-16T07:00:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:16.2.13-145.g20531b61.el8.x86_64 10/122 2023-05-16T07:00:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:16.2.13-145.g20531b61.el8.x86_64 10/122 2023-05-16T07:00:21.708 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-05-16T07:00:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-05-16T07:00:22.121 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 13/122 2023-05-16T07:00:22.227 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 13/122 2023-05-16T07:00:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:16.2.13-145.g20531b61.el8.x86 14/122 2023-05-16T07:00:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:16.2.13-145.g20531b61.el8.x86 14/122 2023-05-16T07:00:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-05-16T07:00:22.902 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-05-16T07:00:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-05-16T07:00:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-05-16T07:00:23.421 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-05-16T07:00:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-05-16T07:00:23.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-05-16T07:00:24.266 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-05-16T07:00:24.437 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-05-16T07:00:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-05-16T07:00:25.129 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-05-16T07:00:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-05-16T07:00:26.116 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-05-16T07:00:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-05-16T07:00:26.592 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-05-16T07:00:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:16.2.13-145.g20531b61.el8. 26/122 2023-05-16T07:00:26.900 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:16.2.13-145.g20531b61.el 27/122 2023-05-16T07:00:27.144 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:16.2.13-145.g20531b61.el8.x86_6 28/122 2023-05-16T07:00:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-05-16T07:00:27.659 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-05-16T07:00:27.895 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-05-16T07:00:29.925 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-05-16T07:00:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-05-16T07:00:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-05-16T07:00:31.763 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-05-16T07:00:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-05-16T07:00:32.282 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-05-16T07:00:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-05-16T07:00:32.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:16.2.13-145.g20531b61.el8.x86_64 38/122 2023-05-16T07:00:32.933 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:16.2.13-145.g20531b61.el8.x86_64 39/122 2023-05-16T07:00:33.261 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:16.2.13-145.g20531b61.el8.x86_6 40/122 2023-05-16T07:00:33.481 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-05-16T07:00:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-05-16T07:00:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-05-16T07:00:34.064 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-05-16T07:00:34.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-05-16T07:00:34.339 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-05-16T07:00:34.510 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-05-16T07:00:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-05-16T07:00:34.886 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-05-16T07:00:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-05-16T07:00:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-05-16T07:00:35.429 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-05-16T07:00:35.564 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-05-16T07:00:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-05-16T07:00:35.930 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-05-16T07:00:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-05-16T07:00:36.546 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-05-16T07:00:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-05-16T07:00:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-05-16T07:00:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-05-16T07:00:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-05-16T07:00:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-05-16T07:00:37.951 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 62/122 2023-05-16T07:00:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-05-16T07:00:38.331 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-05-16T07:00:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-05-16T07:00:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-05-16T07:00:38.912 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-05-16T07:00:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 68/122 2023-05-16T07:00:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 69/122 2023-05-16T07:00:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 70/122 2023-05-16T07:00:39.646 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 71/122 2023-05-16T07:00:39.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 72/122 2023-05-16T07:00:40.155 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-2.6-1.el8.noarch 73/122 2023-05-16T07:00:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 74/122 2023-05-16T07:00:40.557 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 75/122 2023-05-16T07:00:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 76/122 2023-05-16T07:00:41.937 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 77/122 2023-05-16T07:00:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 78/122 2023-05-16T07:00:42.485 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 79/122 2023-05-16T07:00:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 80/122 2023-05-16T07:00:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 81/122 2023-05-16T07:00:43.290 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/122 2023-05-16T07:00:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 83/122 2023-05-16T07:00:43.821 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 84/122 2023-05-16T07:00:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 85/122 2023-05-16T07:00:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 86/122 2023-05-16T07:00:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 87/122 2023-05-16T07:00:48.987 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 88/122 2023-05-16T07:00:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 88/122 2023-05-16T07:00:49.940 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 88/122 2023-05-16T07:00:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 89/122 2023-05-16T07:00:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 89/122 2023-05-16T07:00:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 90/122 2023-05-16T07:01:08.297 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 90/122 2023-05-16T07:01:08.298 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-05-16T07:01:08.298 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-05-16T07:01:08.298 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-05-16T07:01:08.298 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /run 2023-05-16T07:01:08.298 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-05-16T07:01:08.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-05-16T07:01:08.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-05-16T07:01:08.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-05-16T07:01:08.299 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-05-16T07:01:08.299 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:08.661 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 91/122 2023-05-16T07:01:09.094 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 91/122 2023-05-16T07:01:09.261 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 92/122 2023-05-16T07:01:10.753 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 92/122 2023-05-16T07:01:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 93/122 2023-05-16T07:01:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 93/122 2023-05-16T07:01:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:16.2.13-145.g20531b61.el8.noarch 94/122 2023-05-16T07:01:13.620 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:16.2.13-145.g20531b61.el8.noarch 94/122 2023-05-16T07:01:13.753 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:16.2.13-145.g20531b61.e 95/122 2023-05-16T07:01:13.937 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:16.2.13-145.g20531b61. 96/122 2023-05-16T07:01:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noa 97/122 2023-05-16T07:01:15.073 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noa 97/122 2023-05-16T07:01:15.328 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.n 98/122 2023-05-16T07:01:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.n 98/122 2023-05-16T07:01:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.13-145.g205 99/122 2023-05-16T07:01:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.13-145.g205 99/122 2023-05-16T07:01:17.002 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:16.2.13-145.g20531b61.el 100/122 2023-05-16T07:01:17.185 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 101/122 2023-05-16T07:01:17.211 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 101/122 2023-05-16T07:01:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 102/122 2023-05-16T07:01:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 102/122 2023-05-16T07:01:18.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:16.2.13-145.g20531b61.el8.x86_64 103/122 2023-05-16T07:01:18.668 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 104/122 2023-05-16T07:01:18.766 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 104/122 2023-05-16T07:01:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:16.2.13-145.g20531 105/122 2023-05-16T07:01:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.13-145.g20531 105/122 2023-05-16T07:01:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 106/122 2023-05-16T07:01:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 106/122 2023-05-16T07:01:19.600 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 107/122 2023-05-16T07:01:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 107/122 2023-05-16T07:01:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:16.2.13-145.g20531b61.el8.x86_64 108/122 2023-05-16T07:01:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-05-16T07:01:25.573 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:16.2.13-145.g20531b61.el8.x86_64 110/122 2023-05-16T07:01:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-05-16T07:01:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-05-16T07:01:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-05-16T07:01:26.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:16.2.13-145.g20531b61.el8.x86_ 113/122 2023-05-16T07:01:26.378 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-05-16T07:01:26.553 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:16.2.13-145.g20531b61.el8.noarch 115/122 2023-05-16T07:01:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:16.2.13-145.g20531b61.el8.x86_64 116/122 2023-05-16T07:01:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:16.2.13-145.g20531b61.el8.x86_64 117/122 2023-05-16T07:01:27.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-05-16T07:01:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-05-16T07:01:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-05-16T07:01:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-05-16T07:01:27.584 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-05-16T07:01:27.584 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-05-16T07:01:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-05-16T07:01:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-05-16T07:01:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-05-16T07:01:28.360 INFO:teuthology.orchestra.run.smithi092.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-05-16T07:01:28.360 INFO:teuthology.orchestra.run.smithi092.stdout:All rules containing unresolvable specifiers will be skipped. 2023-05-16T07:01:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:28.748 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:16.2.13-145.g20531b61.el8.x86_64 1/122 2023-05-16T07:01:28.749 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 2/122 2023-05-16T07:01:28.749 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 3/122 2023-05-16T07:01:28.749 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:16.2.13-145.g20531b61.el8.x86_64 4/122 2023-05-16T07:01:28.749 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:16.2.13-145.g20531 5/122 2023-05-16T07:01:28.749 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 6/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 7/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 8/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 9/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 10/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 11/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:16.2.13-145.g20531b61.el8.x86_64 12/122 2023-05-16T07:01:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 13/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:16.2.13-145.g20531b61.el8.x86_ 14/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 15/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 16/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:16.2.13-145.g20531b61.el8.x86_6 17/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:16.2.13-145.g20531b61.el8.x86 18/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:16.2.13-145.g20531b61.el8.x86_64 19/122 2023-05-16T07:01:28.751 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:16.2.13-145.g20531b61.el 20/122 2023-05-16T07:01:28.752 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:16.2.13-145.g20531b61.el8. 21/122 2023-05-16T07:01:28.752 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:16.2.13-145.g20531b61.el8.x86_6 22/122 2023-05-16T07:01:28.752 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:16.2.13-145.g20531b61.el8.x86_64 23/122 2023-05-16T07:01:28.752 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:16.2.13-145.g20531b61.el8.x86_64 24/122 2023-05-16T07:01:28.752 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:16.2.13-145.g20531b61.el8.x86_64 25/122 2023-05-16T07:01:28.760 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:16.2.13-145.g20531b61.el8.x86_64 26/122 2023-05-16T07:01:28.760 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 27/122 2023-05-16T07:01:28.760 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:16.2.13-145.g20531b61.el8.x86_64 28/122 2023-05-16T07:01:28.760 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:16.2.13-145.g20531b61. 29/122 2023-05-16T07:01:28.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noa 30/122 2023-05-16T07:01:28.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.n 31/122 2023-05-16T07:01:28.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.13-145.g205 32/122 2023-05-16T07:01:28.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:16.2.13-145.g20531b61.el 33/122 2023-05-16T07:01:28.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 34/122 2023-05-16T07:01:28.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:16.2.13-145.g20531b61.e 35/122 2023-05-16T07:01:28.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:16.2.13-145.g20531b61.el8.noarch 36/122 2023-05-16T07:01:28.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:16.2.13-145.g20531b61.el8.noarch 37/122 2023-05-16T07:01:28.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 38/122 2023-05-16T07:01:28.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : leveldb-1.22-1.el8.x86_64 39/122 2023-05-16T07:01:28.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-05-16T07:01:28.764 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-05-16T07:01:28.764 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 42/122 2023-05-16T07:01:28.764 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-05-16T07:01:28.764 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-05-16T07:01:28.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-05-16T07:01:28.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-05-16T07:01:28.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-05-16T07:01:28.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-05-16T07:01:28.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 49/122 2023-05-16T07:01:28.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/122 2023-05-16T07:01:28.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/122 2023-05-16T07:01:28.767 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/122 2023-05-16T07:01:28.767 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/122 2023-05-16T07:01:28.767 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/122 2023-05-16T07:01:28.767 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/122 2023-05-16T07:01:28.768 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-05-16T07:01:28.768 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-05-16T07:01:28.768 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 58/122 2023-05-16T07:01:28.769 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/122 2023-05-16T07:01:28.769 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/122 2023-05-16T07:01:28.769 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 61/122 2023-05-16T07:01:28.769 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/122 2023-05-16T07:01:28.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/122 2023-05-16T07:01:28.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/122 2023-05-16T07:01:28.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/122 2023-05-16T07:01:28.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 66/122 2023-05-16T07:01:28.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/122 2023-05-16T07:01:28.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-05-16T07:01:28.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-05-16T07:01:28.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-05-16T07:01:28.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-05-16T07:01:28.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-05-16T07:01:28.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-05-16T07:01:28.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.13-145.g20531b61.el8.x86_64 119/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-05-16T07:01:28.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.13-145.g20531b61.el8.x86_64 121/122 2023-05-16T07:01:30.990 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-05-16T07:01:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:06:04 ago on Tue 16 May 2023 06:55:26 AM UTC. 2023-05-16T07:01:33.128 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2023-05-16T07:01:33.278 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:33.278 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-05-16T07:01:33.279 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.279 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.279 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-05-16T07:01:33.279 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.1-7.el8.noarch 2023-05-16T07:01:33.279 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.0.4-10.el8.x86_64 2023-05-16T07:01:33.279 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.281 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:16.2.13-145.g20531b61.el8.noarch 2023-05-16T07:01:33.282 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.1-9.el8.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.7-12.el8.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.5-12.el8.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-05-16T07:01:33.283 INFO:teuthology.orchestra.run.smithi092.stdout: leveldb-1.22-1.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-05-16T07:01:33.284 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.2-3.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.285 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-25.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.3.1-3.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-05-16T07:01:33.286 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.12-1.el8.x86_64 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: python3-apipkg-1.5-6.el8.noarch 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.5.1-5.el8.noarch 2023-05-16T07:01:33.287 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-05-16T07:01:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-05-16T07:01:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-05-16T07:01:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.288 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-6.2-6.el8.noarch 2023-05-16T07:01:33.289 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.0.6-13.el8.noarch 2023-05-16T07:01:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-2.6-1.el8.noarch 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-05-16T07:01:33.291 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2017.2-9.el8.noarch 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-05-16T07:01:33.292 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-05-16T07:01:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.4.1-12.el8.noarch 2023-05-16T07:01:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.8-1.el8.noarch 2023-05-16T07:01:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-05-16T07:01:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-05-16T07:01:33.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-05-16T07:01:33.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:16.2.13-145.g20531b61.el8.x86_64 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.3.3-2.el8.x86_64 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-05-16T07:01:33.295 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-05-16T07:01:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-05-16T07:01:33.296 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:33.296 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-05-16T07:01:33.495 DEBUG:teuthology.parallel:result is None 2023-05-16T07:01:33.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=20531b61f434caac5e29cea07411fac6911e8e91 2023-05-16T07:01:33.631 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-05-16T07:01:33.651 INFO:teuthology.orchestra.run.smithi092.stdout:16.2.13-145.g20531b61.el8 2023-05-16T07:01:33.652 INFO:teuthology.packaging:The installed version of ceph is 16.2.13-145.g20531b61.el8 2023-05-16T07:01:33.652 INFO:teuthology.task.install:The correct ceph version 16.2.13-145.g20531b61 is installed. 2023-05-16T07:01:33.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=20531b61f434caac5e29cea07411fac6911e8e91 2023-05-16T07:01:33.790 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-05-16T07:01:33.817 INFO:teuthology.orchestra.run.smithi158.stdout:16.2.13-145.g20531b61.el8 2023-05-16T07:01:33.818 INFO:teuthology.packaging:The installed version of ceph is 16.2.13-145.g20531b61.el8 2023-05-16T07:01:33.818 INFO:teuthology.task.install:The correct ceph version 16.2.13-145.g20531b61 is installed. 2023-05-16T07:01:33.818 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-05-16T07:01:33.819 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:33.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-05-16T07:01:33.856 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:33.856 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-05-16T07:01:33.898 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-05-16T07:01:33.898 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:33.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-05-16T07:01:33.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-05-16T07:01:33.984 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:33.985 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/daemon-helper 2023-05-16T07:01:34.010 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-05-16T07:01:34.075 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-05-16T07:01:34.076 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:34.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-05-16T07:01:34.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-05-16T07:01:34.161 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:34.162 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/adjust-ulimits 2023-05-16T07:01:34.185 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-05-16T07:01:34.249 INFO:teuthology.run_tasks:Running task ceph... 2023-05-16T07:01:34.330 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-05-16T07:01:34.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-05-16T07:01:34.333 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 777 /var/log/ceph 2023-05-16T07:01:34.357 INFO:tasks.ceph:Disabling ceph logrotate... 2023-05-16T07:01:34.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-05-16T07:01:34.396 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /etc/logrotate.d/ceph 2023-05-16T07:01:34.420 INFO:tasks.ceph:Creating extra log directories... 2023-05-16T07:01:34.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-05-16T07:01:34.459 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-05-16T07:01:34.491 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-05-16T07:01:34.491 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-05-16T07:01:34.491 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-05-16T07:01:34.492 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:34.493 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-05-16T07:01:34.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-05-16T07:01:34.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-05-16T07:01:34.674 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:34.674 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-05-16T07:01:34.689 DEBUG:teuthology.orchestra.run.smithi158:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-05-16T07:01:34.805 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-05-16T07:01:34.842 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-05-16T07:01:34.843 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-05-16T07:01:34.843 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061', 'branch': 'wip-yuri2-testing-2023-05-15-0810-pacific_2', 'description': 'fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7275061', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2023-05-15-0810-pacific_2'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2023-05-15-0810-pacific_2', 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6166, 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri2-testing-2023-05-15-0810-pacific_2', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '20531b61f434caac5e29cea07411fac6911e8e91', 'targets': {'smithi092.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCogJToiq7Bj+4LXDaTzkRPivwioX4EePZ/pr/wuw7kazuX66qqj+UsTcjkQsxH16yUR7aqgG8BuzZZZot4jagbc2yiK6FpFc1Scs1mug+neYOc7vaRg+4rlSuEDenCXHChSM8Usrys/mmqt+to/W6JC0+F6i8DZT3qlCf2lyYQXNQcUuEJZh7gqtjdGUC6w08V9O9ET18IFk0e67tGdbiY8YhVxW6GiU4chMGdJWqkRHBO5evT5C/fCFScvzT6g+t1XcBYCm5WxcQ7PJg9wDlDIZSTUX0Dudj1h8Wzr5feNqW9/iFhp9RYjAtwWfxy/LLEwhkLY7/aCQwWwChEQCtJs9VzyPPaJbSXkieNPnpqYDbxYwYG9gvPHhKo5/JcuqKtf4ZYv2F+iw9zmMy9c6RWgB3/2tiyAMOPqzHvvdZk+PVySYuV2DfgzwCYCQTIMJp7225sGih4Ns/AQo6NF3qMzEmBIYSmxrfU/CA5DiZ2Rt7PIBQ7wfkzlZdNpG0o548=', 'smithi158.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdf9TST+bZAMveHtQb+bAEliZIsauT4reJIVKcMcsRrC907QFoHkmkAQguH0kVa1zxA4RLPzlufnJ06+syo7rI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '20531b61f434caac5e29cea07411fac6911e8e91'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '69caab8ec6f44c61c991ae693e8e7b8cd859a896', 'timestamp': '2023-05-15_21:56:33', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-05-16T07:01:34.844 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-05-16T07:01:34.846 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-05-16T07:01:34.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-05-16T07:01:34.899 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -d -m0777 -- /var/run/ceph 2023-05-16T07:01:34.954 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:34.954 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-05-16T07:01:34.974 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-05-16T07:01:34.974 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-05-16T07:01:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-05-16T07:01:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:35.029 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 206660 Links: 1 2023-05-16T07:01:35.029 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:35.030 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:35.030 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-05-16 07:01:28.275343470 +0000 2023-05-16T07:01:35.030 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-05-16 06:52:33.651314186 +0000 2023-05-16T07:01:35.030 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-05-16 06:52:33.651314186 +0000 2023-05-16T07:01:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-05-16T07:01:35.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-05-16T07:01:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-05-16T07:01:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-05-16T07:01:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00025318 s, 2.0 MB/s 2023-05-16T07:01:35.093 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-05-16T07:01:35.148 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-05-16T07:01:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-05-16T07:01:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:35.203 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 211388 Links: 1 2023-05-16T07:01:35.204 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:35.204 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:35.204 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-05-16 07:01:28.275343470 +0000 2023-05-16T07:01:35.204 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-05-16 06:52:34.160299716 +0000 2023-05-16T07:01:35.204 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-05-16 06:52:34.160299716 +0000 2023-05-16T07:01:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-05-16T07:01:35.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-05-16T07:01:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-05-16T07:01:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-05-16T07:01:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00021402 s, 2.4 MB/s 2023-05-16T07:01:35.266 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-05-16T07:01:35.321 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-05-16T07:01:35.374 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-05-16T07:01:35.375 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:35.375 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 212338 Links: 1 2023-05-16T07:01:35.375 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:35.375 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:35.375 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-05-16 07:01:28.275343470 +0000 2023-05-16T07:01:35.376 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-05-16 06:52:34.655285644 +0000 2023-05-16T07:01:35.376 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-05-16 06:52:34.655285644 +0000 2023-05-16T07:01:35.376 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-05-16T07:01:35.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-05-16T07:01:35.437 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-05-16T07:01:35.437 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-05-16T07:01:35.437 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000224462 s, 2.3 MB/s 2023-05-16T07:01:35.438 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-05-16T07:01:35.492 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-05-16T07:01:35.545 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-05-16T07:01:35.546 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:35.546 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 211479 Links: 1 2023-05-16T07:01:35.546 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:35.546 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:35.546 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-05-16 07:01:28.275343470 +0000 2023-05-16T07:01:35.546 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-05-16 06:52:35.159271316 +0000 2023-05-16T07:01:35.547 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-05-16 06:52:35.159271316 +0000 2023-05-16T07:01:35.547 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-05-16T07:01:35.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-05-16T07:01:35.608 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-05-16T07:01:35.608 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-05-16T07:01:35.608 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000184923 s, 2.8 MB/s 2023-05-16T07:01:35.609 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-05-16T07:01:35.669 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:35.670 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:35.670 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/scratch_devs of=/dev/stdout 2023-05-16T07:01:35.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-05-16T07:01:35.686 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_1 2023-05-16T07:01:35.742 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-05-16T07:01:35.742 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:35.742 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 6h/6d Inode: 204703 Links: 1 2023-05-16T07:01:35.742 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:35.742 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:35.743 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-05-16 06:57:10.857837054 +0000 2023-05-16T07:01:35.743 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-05-16 06:54:42.936589598 +0000 2023-05-16T07:01:35.743 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-05-16 06:54:42.936589598 +0000 2023-05-16T07:01:35.743 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: - 2023-05-16T07:01:35.743 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-05-16T07:01:35.805 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-05-16T07:01:35.806 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-05-16T07:01:35.806 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000139092 s, 3.7 MB/s 2023-05-16T07:01:35.808 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-05-16T07:01:35.864 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_2 2023-05-16T07:01:35.919 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-05-16T07:01:35.919 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:35.920 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 6h/6d Inode: 209399 Links: 1 2023-05-16T07:01:35.920 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:35.920 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:35.920 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-05-16 06:57:10.857837054 +0000 2023-05-16T07:01:35.921 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-05-16 06:54:43.442583700 +0000 2023-05-16T07:01:35.921 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-05-16 06:54:43.442583700 +0000 2023-05-16T07:01:35.921 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: - 2023-05-16T07:01:35.924 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-05-16T07:01:35.981 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-05-16T07:01:35.982 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-05-16T07:01:35.982 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000139362 s, 3.7 MB/s 2023-05-16T07:01:35.983 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-05-16T07:01:36.038 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_3 2023-05-16T07:01:36.093 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-05-16T07:01:36.093 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:36.093 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 6h/6d Inode: 212116 Links: 1 2023-05-16T07:01:36.093 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:36.093 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:36.094 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-05-16 06:57:10.857837054 +0000 2023-05-16T07:01:36.094 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-05-16 06:54:43.940577896 +0000 2023-05-16T07:01:36.094 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-05-16 06:54:43.940577896 +0000 2023-05-16T07:01:36.094 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: - 2023-05-16T07:01:36.094 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-05-16T07:01:36.155 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-05-16T07:01:36.155 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-05-16T07:01:36.156 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 9.4559e-05 s, 5.4 MB/s 2023-05-16T07:01:36.156 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-05-16T07:01:36.212 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_4 2023-05-16T07:01:36.266 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-05-16T07:01:36.266 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-16T07:01:36.266 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 6h/6d Inode: 213087 Links: 1 2023-05-16T07:01:36.267 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-16T07:01:36.267 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-05-16T07:01:36.267 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-05-16 06:57:10.857837054 +0000 2023-05-16T07:01:36.267 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-05-16 06:54:44.430572184 +0000 2023-05-16T07:01:36.267 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-05-16 06:54:44.430572184 +0000 2023-05-16T07:01:36.267 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: - 2023-05-16T07:01:36.268 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-05-16T07:01:36.328 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-05-16T07:01:36.328 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-05-16T07:01:36.329 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 9.4989e-05 s, 5.4 MB/s 2023-05-16T07:01:36.329 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-05-16T07:01:36.384 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:36.385 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi158.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-05-16T07:01:36.386 INFO:tasks.ceph:Generating config... 2023-05-16T07:01:36.387 INFO:tasks.ceph:[client] client mount timeout = 600 2023-05-16T07:01:36.387 INFO:tasks.ceph:[client] debug client = 20 2023-05-16T07:01:36.387 INFO:tasks.ceph:[client] debug ms = 1 2023-05-16T07:01:36.387 INFO:tasks.ceph:[client] pid file = 2023-05-16T07:01:36.388 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-05-16T07:01:36.388 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-05-16T07:01:36.388 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-05-16T07:01:36.388 INFO:tasks.ceph:[mds] debug mds = 20 2023-05-16T07:01:36.389 INFO:tasks.ceph:[mds] debug ms = 1 2023-05-16T07:01:36.389 INFO:tasks.ceph:[mds] mds debug frag = True 2023-05-16T07:01:36.389 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-05-16T07:01:36.389 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-05-16T07:01:36.389 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-05-16T07:01:36.389 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-05-16T07:01:36.390 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-05-16T07:01:36.390 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-05-16T07:01:36.390 INFO:tasks.ceph:[mgr] debug client = 20 2023-05-16T07:01:36.390 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-05-16T07:01:36.390 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-05-16T07:01:36.391 INFO:tasks.ceph:[mgr] debug ms = 1 2023-05-16T07:01:36.391 INFO:tasks.ceph:[mon] debug mon = 20 2023-05-16T07:01:36.391 INFO:tasks.ceph:[mon] debug ms = 1 2023-05-16T07:01:36.391 INFO:tasks.ceph:[mon] debug paxos = 20 2023-05-16T07:01:36.391 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-05-16T07:01:36.392 INFO:tasks.ceph:[osd] bdev async discard = True 2023-05-16T07:01:36.392 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-05-16T07:01:36.392 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-05-16T07:01:36.392 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-05-16T07:01:36.393 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-05-16T07:01:36.393 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-05-16T07:01:36.393 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-05-16T07:01:36.394 INFO:tasks.ceph:[osd] debug ms = 1 2023-05-16T07:01:36.394 INFO:tasks.ceph:[osd] debug osd = 20 2023-05-16T07:01:36.394 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-05-16T07:01:36.394 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-05-16T07:01:36.395 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-05-16T07:01:36.395 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-05-16T07:01:36.395 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-05-16T07:01:36.395 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-05-16T07:01:36.395 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-05-16T07:01:36.396 INFO:tasks.ceph:Setting up mon.a... 2023-05-16T07:01:36.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-05-16T07:01:36.444 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2023-05-16T07:01:36.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-05-16T07:01:36.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-05-16T07:01:36.550 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.158'), ('mon.c', '[v2:172.21.15.158:3301,v1:172.21.15.158:6790]')] 2023-05-16T07:01:36.551 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.158,[v2:172.21.15.158:3301,v1:172.21.15.158:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-05-16T07:01:36.552 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:36.552 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-05-16T07:01:36.604 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.158 --addv c '[v2:172.21.15.158:3301,v1:172.21.15.158:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:36.684 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:36.685 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 7dc962fc-4181-4aa6-8ae9-07b8e654738a 2023-05-16T07:01:36.685 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2023-05-16T07:01:36.685 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 7dc962fc-4181-4aa6-8ae9-07b8e654738a 2023-05-16T07:01:36.685 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2023-05-16T07:01:36.681125+0000 2023-05-16T07:01:36.686 INFO:teuthology.orchestra.run.smithi092.stdout:created 2023-05-16T07:01:36.681125+0000 2023-05-16T07:01:36.686 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 0 (unknown) 2023-05-16T07:01:36.686 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2023-05-16T07:01:36.687 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2023-05-16T07:01:36.687 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] mon.b 2023-05-16T07:01:36.687 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] mon.c 2023-05-16T07:01:36.687 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-05-16T07:01:36.689 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-05-16T07:01:36.743 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7dc962fc-4181-4aa6-8ae9-07b8e654738a... 2023-05-16T07:01:36.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-16T07:01:36.784 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-16T07:01:36.825 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-05-16T07:01:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-05-16T07:01:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-16T07:01:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-05-16T07:01:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-05-16T07:01:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-05-16T07:01:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-05-16T07:01:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-05-16T07:01:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-05-16T07:01:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-05-16T07:01:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-16T07:01:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-05-16T07:01:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-05-16T07:01:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-16T07:01:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-05-16T07:01:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-05-16T07:01:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-05-16T07:01:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-05-16T07:01:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-05-16T07:01:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-05-16T07:01:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-05-16T07:01:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-05-16T07:01:36.834 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.158,[v2:172.21.15.158:3301,v1:172.21.15.158:6790]" 2023-05-16T07:01:36.835 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on skipped message = False 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 7dc962fc-4181-4aa6-8ae9-07b8e654738a 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-05-16T07:01:36.836 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-05-16T07:01:36.837 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.838 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2023-05-16T07:01:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-05-16T07:01:36.840 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-05-16T07:01:36.841 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-05-16T07:01:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout: debug finisher = 20 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-05-16T07:01:36.843 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-05-16T07:01:36.844 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-16T07:01:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-16T07:01:36.846 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-05-16T07:01:36.847 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-05-16T07:01:36.848 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-05-16T07:01:36.849 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-05-16T07:01:36.850 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-05-16T07:01:36.851 INFO:teuthology.orchestra.run.smithi158.stdout:[global] 2023-05-16T07:01:36.851 INFO:teuthology.orchestra.run.smithi158.stdout: chdir = "" 2023-05-16T07:01:36.851 INFO:teuthology.orchestra.run.smithi158.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-16T07:01:36.851 INFO:teuthology.orchestra.run.smithi158.stdout: auth supported = cephx 2023-05-16T07:01:36.852 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.852 INFO:teuthology.orchestra.run.smithi158.stdout: filestore xattr use omap = true 2023-05-16T07:01:36.852 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.852 INFO:teuthology.orchestra.run.smithi158.stdout: mon clock drift allowed = 1.000 2023-05-16T07:01:36.852 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.852 INFO:teuthology.orchestra.run.smithi158.stdout: osd crush chooseleaf type = 0 2023-05-16T07:01:36.853 INFO:teuthology.orchestra.run.smithi158.stdout: auth debug = true 2023-05-16T07:01:36.853 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.853 INFO:teuthology.orchestra.run.smithi158.stdout: ms die on old message = true 2023-05-16T07:01:36.853 INFO:teuthology.orchestra.run.smithi158.stdout: ms die on bug = true 2023-05-16T07:01:36.853 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.853 INFO:teuthology.orchestra.run.smithi158.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: mon pg warn max object skew = 0 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: # disable pg_autoscaler by default for new pools 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: osd pool default size = 2 2023-05-16T07:01:36.854 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd allow primary affinity = true 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd allow pg remap = true 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on legacy crush tunables = false 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on crush straw calc version zero = false 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on no sortbitwise = false 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on osd down out interval zero = false 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on too few osds = false 2023-05-16T07:01:36.855 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: mon_allow_pool_size_one = true 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: osd default data pool replay window = 5 2023-05-16T07:01:36.856 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: mon allow pool delete = true 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: mon cluster log file level = debug 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: debug asserts on shutdown = true 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: mon health detail to clog = false 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: mon host = "172.21.15.92,172.21.15.158,[v2:172.21.15.158:3301,v1:172.21.15.158:6790]" 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: ms die on skipped message = False 2023-05-16T07:01:36.857 INFO:teuthology.orchestra.run.smithi158.stdout: fsid = 7dc962fc-4181-4aa6-8ae9-07b8e654738a 2023-05-16T07:01:36.858 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.858 INFO:teuthology.orchestra.run.smithi158.stdout:[osd] 2023-05-16T07:01:36.858 INFO:teuthology.orchestra.run.smithi158.stdout: osd journal size = 100 2023-05-16T07:01:36.858 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.858 INFO:teuthology.orchestra.run.smithi158.stdout: osd scrub load threshold = 5.0 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: osd scrub max interval = 600 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: osd recover clone overlap = true 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: osd recovery max chunk = 1048576 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug shutdown = true 2023-05-16T07:01:36.859 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug op order = true 2023-05-16T07:01:36.860 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug verify stray on activate = true 2023-05-16T07:01:36.860 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.860 INFO:teuthology.orchestra.run.smithi158.stdout: osd open classes on start = true 2023-05-16T07:01:36.860 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug pg log writeout = true 2023-05-16T07:01:36.860 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.861 INFO:teuthology.orchestra.run.smithi158.stdout: osd deep scrub update digest min age = 30 2023-05-16T07:01:36.861 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.861 INFO:teuthology.orchestra.run.smithi158.stdout: osd map max advance = 10 2023-05-16T07:01:36.861 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.862 INFO:teuthology.orchestra.run.smithi158.stdout: journal zero on create = true 2023-05-16T07:01:36.862 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.862 INFO:teuthology.orchestra.run.smithi158.stdout: filestore ondisk finisher threads = 3 2023-05-16T07:01:36.862 INFO:teuthology.orchestra.run.smithi158.stdout: filestore apply finisher threads = 3 2023-05-16T07:01:36.863 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.863 INFO:teuthology.orchestra.run.smithi158.stdout: bdev debug aio = true 2023-05-16T07:01:36.863 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug misdirected ops = true 2023-05-16T07:01:36.864 INFO:teuthology.orchestra.run.smithi158.stdout: bdev async discard = True 2023-05-16T07:01:36.864 INFO:teuthology.orchestra.run.smithi158.stdout: bdev enable discard = True 2023-05-16T07:01:36.864 INFO:teuthology.orchestra.run.smithi158.stdout: bluestore allocator = bitmap 2023-05-16T07:01:36.864 INFO:teuthology.orchestra.run.smithi158.stdout: bluestore block size = 96636764160 2023-05-16T07:01:36.865 INFO:teuthology.orchestra.run.smithi158.stdout: bluestore fsck on mount = True 2023-05-16T07:01:36.865 INFO:teuthology.orchestra.run.smithi158.stdout: debug bluefs = 1/20 2023-05-16T07:01:36.866 INFO:teuthology.orchestra.run.smithi158.stdout: debug bluestore = 1/20 2023-05-16T07:01:36.866 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-05-16T07:01:36.866 INFO:teuthology.orchestra.run.smithi158.stdout: debug osd = 20 2023-05-16T07:01:36.866 INFO:teuthology.orchestra.run.smithi158.stdout: debug rocksdb = 4/10 2023-05-16T07:01:36.866 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd backfillfull_ratio = 0.85 2023-05-16T07:01:36.866 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd full ratio = 0.9 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd nearfull ratio = 0.8 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: osd failsafe full ratio = 0.95 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: osd objectstore = bluestore 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: osd op complaint time = 180 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout:[mgr] 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-05-16T07:01:36.867 INFO:teuthology.orchestra.run.smithi158.stdout: debug mgr = 20 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: debug mon = 20 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: debug auth = 20 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min pgs per osd = 4 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min bytes per osd = 10 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: mgr/telemetry/nag = false 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: debug client = 20 2023-05-16T07:01:36.868 INFO:teuthology.orchestra.run.smithi158.stdout: debug finisher = 20 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout:[mon] 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: debug mon = 20 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: debug paxos = 20 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: debug auth = 20 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: mon data avail warn = 5 2023-05-16T07:01:36.869 INFO:teuthology.orchestra.run.smithi158.stdout: mon mgr mkfs grace = 240 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min pgs per osd = 4 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd reporter subtree level = osd 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd prime pg temp = true 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min bytes per osd = 10 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: auth mon ticket ttl = 660 # 11m 2023-05-16T07:01:36.870 INFO:teuthology.orchestra.run.smithi158.stdout: auth service ticket ttl = 240 # 4m 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout: # don't complain about insecure global_id in the test suite 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout: mon op complaint time = 120 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-05-16T07:01:36.871 INFO:teuthology.orchestra.run.smithi158.stdout:[client] 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: rgw cache enabled = true 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: rgw enable ops log = true 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: rgw enable usage log = true 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: client mount timeout = 600 2023-05-16T07:01:36.872 INFO:teuthology.orchestra.run.smithi158.stdout: debug client = 20 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout: pid file = "" 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout: rados mon op timeout = 900 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout: rados osd op timeout = 900 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout:[mon.a] 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout:[mon.b] 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout:[mon.c] 2023-05-16T07:01:36.873 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.a] 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.c] 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.b] 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.d] 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout:[mds] 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout: debug mds = 20 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout: mds debug frag = True 2023-05-16T07:01:36.874 INFO:teuthology.orchestra.run.smithi158.stdout: mds debug scatterstat = True 2023-05-16T07:01:36.875 INFO:teuthology.orchestra.run.smithi158.stdout: mds op complaint time = 180 2023-05-16T07:01:36.875 INFO:teuthology.orchestra.run.smithi158.stdout: mds verify scatter = True 2023-05-16T07:01:36.875 INFO:teuthology.orchestra.run.smithi158.stdout: osd op complaint time = 180 2023-05-16T07:01:36.875 INFO:teuthology.orchestra.run.smithi158.stdout: rados mon op timeout = 900 2023-05-16T07:01:36.875 INFO:teuthology.orchestra.run.smithi158.stdout: rados osd op timeout = 900 2023-05-16T07:01:36.875 INFO:tasks.ceph:Creating admin key on mon.a... 2023-05-16T07:01:36.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-05-16T07:01:36.940 INFO:tasks.ceph:Copying monmap to all nodes... 2023-05-16T07:01:36.940 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:36.940 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-05-16T07:01:36.957 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:36.957 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-05-16T07:01:37.011 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:01:37.011 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:37.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-05-16T07:01:37.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-05-16T07:01:37.083 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:01:37.083 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:37.137 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:01:37.137 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:37.137 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/ceph/ceph.keyring 2023-05-16T07:01:37.138 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-05-16T07:01:37.173 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:01:37.173 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:37.228 INFO:tasks.ceph:Setting up mon nodes... 2023-05-16T07:01:37.228 INFO:tasks.ceph:Setting up mgr nodes... 2023-05-16T07:01:37.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-05-16T07:01:37.273 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-05-16T07:01:37.275 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-05-16T07:01:37.328 INFO:teuthology.orchestra.run.smithi158.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-05-16T07:01:37.330 INFO:tasks.ceph:Setting up mds nodes... 2023-05-16T07:01:37.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-05-16T07:01:37.376 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-05-16T07:01:37.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-05-16T07:01:37.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-05-16T07:01:37.488 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-05-16T07:01:37.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-05-16T07:01:37.513 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-05-16T07:01:37.559 INFO:teuthology.orchestra.run.smithi158.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-05-16T07:01:37.561 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-05-16T07:01:37.584 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-05-16T07:01:37.669 INFO:teuthology.orchestra.run.smithi158.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-05-16T07:01:37.671 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-05-16T07:01:37.695 INFO:tasks.ceph_client:Setting up client nodes... 2023-05-16T07:01:37.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-05-16T07:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-05-16T07:01:37.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-05-16T07:01:37.820 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-05-16T07:01:37.833 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-05-16T07:01:37.869 INFO:teuthology.orchestra.run.smithi158.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-05-16T07:01:37.881 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-05-16T07:01:37.957 INFO:teuthology.orchestra.run.smithi158.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-05-16T07:01:37.969 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-05-16T07:01:37.970 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi158.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-05-16T07:01:37.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-05-16T07:01:37.993 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:37.994 INFO:tasks.ceph:role: osd.0 2023-05-16T07:01:37.994 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:01:37.994 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-05-16T07:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-05-16T07:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:38.421 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-05-16T07:01:38.422 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-05-16T07:01:38.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-05-16T07:01:38.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-05-16T07:01:38.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-05-16T07:01:38.646 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:38.646 INFO:tasks.ceph:role: osd.1 2023-05-16T07:01:38.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:01:38.647 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-05-16T07:01:38.716 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:38.716 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:38.716 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:38.716 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-05-16T07:01:38.716 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:38.717 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:38.717 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:38.717 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:38.717 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:38.717 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:39.075 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-05-16T07:01:39.077 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-05-16T07:01:39.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-05-16T07:01:39.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-05-16T07:01:39.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-05-16T07:01:39.252 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:39.252 INFO:tasks.ceph:role: osd.2 2023-05-16T07:01:39.253 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:01:39.253 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-05-16T07:01:39.323 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:39.323 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:39.323 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:39.324 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:39.685 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-05-16T07:01:39.687 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-05-16T07:01:39.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-05-16T07:01:39.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-05-16T07:01:39.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-05-16T07:01:39.880 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:39.881 INFO:tasks.ceph:role: osd.3 2023-05-16T07:01:39.881 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:01:39.881 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-05-16T07:01:39.951 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:39.951 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:39.951 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:39.952 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:40.320 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-05-16T07:01:40.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-05-16T07:01:40.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-05-16T07:01:40.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-05-16T07:01:40.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:40.529+0000 7f7cdf68a200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-05-16T07:01:40.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:40.532+0000 7f7cdf68a200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-05-16T07:01:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:40.532+0000 7f7cdf68a200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-05-16T07:01:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:40.532+0000 7f7cdf68a200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-05-16T07:01:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:40.532+0000 7f7cdf68a200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-05-16T07:01:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:40.536+0000 7f7cdf68a200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-05-16T07:01:43.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-05-16T07:01:43.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:43.617+0000 7f2ff6d9b200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-05-16T07:01:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:43.617+0000 7f2ff6d9b200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-05-16T07:01:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:43.617+0000 7f2ff6d9b200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-05-16T07:01:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:43.617+0000 7f2ff6d9b200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-05-16T07:01:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:43.617+0000 7f2ff6d9b200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-05-16T07:01:43.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:43.617+0000 7f2ff6d9b200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-05-16T07:01:46.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-05-16T07:01:46.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:46.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:46.648+0000 7f8e2bda4200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-05-16T07:01:46.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:46.648+0000 7f8e2bda4200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-05-16T07:01:46.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:46.648+0000 7f8e2bda4200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-05-16T07:01:46.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:46.648+0000 7f8e2bda4200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-05-16T07:01:46.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:46.648+0000 7f8e2bda4200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-05-16T07:01:46.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:46.649+0000 7f8e2bda4200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-05-16T07:01:49.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-05-16T07:01:49.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:49.685+0000 7f26d5fa3200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-05-16T07:01:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:49.685+0000 7f26d5fa3200 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-05-16T07:01:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:49.685+0000 7f26d5fa3200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-05-16T07:01:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:49.685+0000 7f26d5fa3200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-05-16T07:01:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:49.685+0000 7f26d5fa3200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-05-16T07:01:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:01:49.685+0000 7f26d5fa3200 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-05-16T07:01:52.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-05-16T07:01:52.628 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-05-16T07:01:52.658 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:52.658 INFO:tasks.ceph:role: osd.4 2023-05-16T07:01:52.659 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:01:52.659 DEBUG:teuthology.orchestra.run.smithi158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-05-16T07:01:52.725 INFO:teuthology.orchestra.run.smithi158.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:52.725 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout: = reflink=1 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:52.726 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:52.727 INFO:teuthology.orchestra.run.smithi158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:52.747 INFO:teuthology.orchestra.run.smithi158.stdout:Discarding blocks...Done. 2023-05-16T07:01:52.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi158.front.sepia.ceph.com -o noatime 2023-05-16T07:01:52.749 DEBUG:teuthology.orchestra.run.smithi158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-05-16T07:01:52.827 DEBUG:teuthology.orchestra.run.smithi158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-05-16T07:01:52.896 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-05-16T07:01:52.960 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:52.960 INFO:tasks.ceph:role: osd.5 2023-05-16T07:01:52.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:01:52.961 DEBUG:teuthology.orchestra.run.smithi158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-05-16T07:01:53.023 INFO:teuthology.orchestra.run.smithi158.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:53.023 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:53.023 INFO:teuthology.orchestra.run.smithi158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout: = reflink=1 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:53.024 INFO:teuthology.orchestra.run.smithi158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:53.045 INFO:teuthology.orchestra.run.smithi158.stdout:Discarding blocks...Done. 2023-05-16T07:01:53.046 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi158.front.sepia.ceph.com -o noatime 2023-05-16T07:01:53.046 DEBUG:teuthology.orchestra.run.smithi158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-05-16T07:01:53.122 DEBUG:teuthology.orchestra.run.smithi158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-05-16T07:01:53.189 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-05-16T07:01:53.251 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:53.252 INFO:tasks.ceph:role: osd.6 2023-05-16T07:01:53.252 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:01:53.252 DEBUG:teuthology.orchestra.run.smithi158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-05-16T07:01:53.314 INFO:teuthology.orchestra.run.smithi158.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:53.314 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:53.314 INFO:teuthology.orchestra.run.smithi158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:53.314 INFO:teuthology.orchestra.run.smithi158.stdout: = reflink=1 2023-05-16T07:01:53.314 INFO:teuthology.orchestra.run.smithi158.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:53.315 INFO:teuthology.orchestra.run.smithi158.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:53.315 INFO:teuthology.orchestra.run.smithi158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:53.315 INFO:teuthology.orchestra.run.smithi158.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:53.315 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:53.315 INFO:teuthology.orchestra.run.smithi158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:53.336 INFO:teuthology.orchestra.run.smithi158.stdout:Discarding blocks...Done. 2023-05-16T07:01:53.337 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi158.front.sepia.ceph.com -o noatime 2023-05-16T07:01:53.338 DEBUG:teuthology.orchestra.run.smithi158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-05-16T07:01:53.413 DEBUG:teuthology.orchestra.run.smithi158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-05-16T07:01:53.480 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-05-16T07:01:53.544 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-16T07:01:53.545 INFO:tasks.ceph:role: osd.7 2023-05-16T07:01:53.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:01:53.545 DEBUG:teuthology.orchestra.run.smithi158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-05-16T07:01:53.607 INFO:teuthology.orchestra.run.smithi158.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-05-16T07:01:53.607 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-16T07:01:53.607 INFO:teuthology.orchestra.run.smithi158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-16T07:01:53.607 INFO:teuthology.orchestra.run.smithi158.stdout: = reflink=1 2023-05-16T07:01:53.608 INFO:teuthology.orchestra.run.smithi158.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-16T07:01:53.608 INFO:teuthology.orchestra.run.smithi158.stdout: = sunit=0 swidth=0 blks 2023-05-16T07:01:53.608 INFO:teuthology.orchestra.run.smithi158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-16T07:01:53.608 INFO:teuthology.orchestra.run.smithi158.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-16T07:01:53.608 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-16T07:01:53.608 INFO:teuthology.orchestra.run.smithi158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-16T07:01:53.630 INFO:teuthology.orchestra.run.smithi158.stdout:Discarding blocks...Done. 2023-05-16T07:01:53.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi158.front.sepia.ceph.com -o noatime 2023-05-16T07:01:53.632 DEBUG:teuthology.orchestra.run.smithi158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-05-16T07:01:53.707 DEBUG:teuthology.orchestra.run.smithi158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-05-16T07:01:53.774 DEBUG:teuthology.orchestra.run.smithi158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:53.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:53.850+0000 7fb691d42200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-05-16T07:01:53.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:53.850+0000 7fb691d42200 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-05-16T07:01:53.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:53.850+0000 7fb691d42200 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-05-16T07:01:53.855 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:53.850+0000 7fb691d42200 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-05-16T07:01:53.855 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:53.850+0000 7fb691d42200 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-05-16T07:01:53.855 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:53.850+0000 7fb691d42200 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-05-16T07:01:56.703 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-05-16T07:01:56.772 DEBUG:teuthology.orchestra.run.smithi158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:56.857 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:56.853+0000 7fd782a36200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-05-16T07:01:56.857 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:56.853+0000 7fd782a36200 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-05-16T07:01:56.857 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:56.853+0000 7fd782a36200 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-05-16T07:01:56.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:56.853+0000 7fd782a36200 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-05-16T07:01:56.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:56.853+0000 7fd782a36200 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-05-16T07:01:56.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:56.854+0000 7fd782a36200 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-05-16T07:01:59.712 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-05-16T07:01:59.739 DEBUG:teuthology.orchestra.run.smithi158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:01:59.823 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:59.819+0000 7f68bbcb5200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-05-16T07:01:59.823 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:59.819+0000 7f68bbcb5200 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-05-16T07:01:59.824 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:59.819+0000 7f68bbcb5200 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-05-16T07:01:59.824 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:59.819+0000 7f68bbcb5200 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-05-16T07:01:59.824 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:59.819+0000 7f68bbcb5200 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-05-16T07:01:59.824 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:01:59.820+0000 7f68bbcb5200 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-05-16T07:02:02.666 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-05-16T07:02:02.733 DEBUG:teuthology.orchestra.run.smithi158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:02:02.814 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:02.810+0000 7fe8937ad200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-05-16T07:02:02.814 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:02.810+0000 7fe8937ad200 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-05-16T07:02:02.814 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:02.810+0000 7fe8937ad200 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-05-16T07:02:02.815 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:02.810+0000 7fe8937ad200 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-05-16T07:02:02.815 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:02.810+0000 7fe8937ad200 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-05-16T07:02:02.815 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:02.811+0000 7fe8937ad200 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-05-16T07:02:04.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:02:04.848 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:02:05.663 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-05-16T07:02:05.689 INFO:tasks.ceph:Reading keys from all nodes... 2023-05-16T07:02:05.690 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:05.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-05-16T07:02:05.718 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:05.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-05-16T07:02:05.784 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:05.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-05-16T07:02:05.850 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:05.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-05-16T07:02:05.917 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:05.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-05-16T07:02:05.982 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:05.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-05-16T07:02:06.046 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:06.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-05-16T07:02:06.110 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.110 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-05-16T07:02:06.135 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.136 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-05-16T07:02:06.200 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.200 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-05-16T07:02:06.264 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.264 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-05-16T07:02:06.326 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.327 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-05-16T07:02:06.389 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.389 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-05-16T07:02:06.452 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.452 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-05-16T07:02:06.515 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:06.515 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-05-16T07:02:06.530 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:06.530 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-05-16T07:02:06.584 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.586 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-05-16T07:02:06.600 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:06.601 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-05-16T07:02:06.654 INFO:tasks.ceph:Adding keys to all mons... 2023-05-16T07:02:06.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2023-05-16T07:02:06.658 DEBUG:teuthology.orchestra.run.smithi158:> sudo tee -a /etc/ceph/ceph.keyring 2023-05-16T07:02:06.698 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2023-05-16T07:02:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w== 2023-05-16T07:02:06.699 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-05-16T07:02:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ== 2023-05-16T07:02:06.699 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-05-16T07:02:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg== 2023-05-16T07:02:06.700 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2023-05-16T07:02:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg== 2023-05-16T07:02:06.700 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2023-05-16T07:02:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ== 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw== 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw== 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg== 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-05-16T07:02:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ== 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g== 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw== 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg== 2023-05-16T07:02:06.702 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg== 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg== 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ== 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw== 2023-05-16T07:02:06.703 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2023-05-16T07:02:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ== 2023-05-16T07:02:06.704 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2023-05-16T07:02:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g== 2023-05-16T07:02:06.716 INFO:teuthology.orchestra.run.smithi158.stdout:[mgr.y] 2023-05-16T07:02:06.716 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w== 2023-05-16T07:02:06.716 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.a] 2023-05-16T07:02:06.716 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ== 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.c] 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg== 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.0] 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg== 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.1] 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ== 2023-05-16T07:02:06.717 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.2] 2023-05-16T07:02:06.718 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw== 2023-05-16T07:02:06.718 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.3] 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw== 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout:[mgr.x] 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg== 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.b] 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ== 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout:[mds.d] 2023-05-16T07:02:06.719 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g== 2023-05-16T07:02:06.720 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.4] 2023-05-16T07:02:06.720 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw== 2023-05-16T07:02:06.720 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.5] 2023-05-16T07:02:06.720 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg== 2023-05-16T07:02:06.721 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.6] 2023-05-16T07:02:06.721 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg== 2023-05-16T07:02:06.721 INFO:teuthology.orchestra.run.smithi158.stdout:[osd.7] 2023-05-16T07:02:06.721 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg== 2023-05-16T07:02:06.722 INFO:teuthology.orchestra.run.smithi158.stdout:[client.0] 2023-05-16T07:02:06.722 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ== 2023-05-16T07:02:06.722 INFO:teuthology.orchestra.run.smithi158.stdout:[client.2] 2023-05-16T07:02:06.722 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw== 2023-05-16T07:02:06.722 INFO:teuthology.orchestra.run.smithi158.stdout:[client.1] 2023-05-16T07:02:06.722 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ== 2023-05-16T07:02:06.723 INFO:teuthology.orchestra.run.smithi158.stdout:[client.3] 2023-05-16T07:02:06.723 INFO:teuthology.orchestra.run.smithi158.stdout: key = AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g== 2023-05-16T07:02:06.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-16T07:02:06.748 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-16T07:02:06.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:06.833 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:06.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:06.883 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:06.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:06.932 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:06.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.017 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.065 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.149 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-16T07:02:07.235 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-16T07:02:07.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:07.320 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:07.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:07.370 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-16T07:02:07.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.419 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.467 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.551 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.598 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-16T07:02:07.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.684 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.734 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.784 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.834 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-16T07:02:07.880 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-05-16T07:02:07.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-05-16T07:02:07.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-05-16T07:02:07.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-05-16T07:02:08.018 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-05-16T07:02:08.042 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-05-16T07:02:08.131 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-05-16T07:02:08.153 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-05-16T07:02:08.217 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-05-16T07:02:08.306 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-05-16T07:02:08.330 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:02:08.332 DEBUG:teuthology.orchestra.run.smithi158:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-05-16T07:02:08.384 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-05-16T07:02:08.385 INFO:tasks.ceph.mon.a:Restarting daemon 2023-05-16T07:02:08.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-05-16T07:02:08.387 INFO:tasks.ceph.mon.a:Started 2023-05-16T07:02:08.387 INFO:tasks.ceph.mon.b:Restarting daemon 2023-05-16T07:02:08.387 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-05-16T07:02:08.426 INFO:tasks.ceph.mon.b:Started 2023-05-16T07:02:08.427 INFO:tasks.ceph.mon.c:Restarting daemon 2023-05-16T07:02:08.427 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-05-16T07:02:08.429 INFO:tasks.ceph.mon.c:Started 2023-05-16T07:02:08.429 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-05-16T07:02:08.429 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-05-16T07:02:08.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-05-16T07:02:08.431 INFO:tasks.ceph.mgr.y:Started 2023-05-16T07:02:08.432 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-05-16T07:02:08.432 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-05-16T07:02:08.433 INFO:tasks.ceph.mgr.x:Started 2023-05-16T07:02:08.434 INFO:tasks.ceph:Setting crush tunables to default 2023-05-16T07:02:08.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2023-05-16T07:02:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.543+0000 7f3bb6174700 1 Processor -- start 2023-05-16T07:02:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.543+0000 7f3bb6174700 1 -- start start 2023-05-16T07:02:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3bb6174700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 0x7f3bb00a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3bb6174700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 0x7f3bb00a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3bb6174700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb0064e90 con 0x7f3bb0150bb0 2023-05-16T07:02:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3bb6174700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb0065a10 con 0x7f3bb0150f90 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3bb6174700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb00a7160 con 0x7f3bb0068670 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 0x7f3bb00a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 0x7f3bb00a6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39898/0 (socket says 172.21.15.92:39898) 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 learned_addr learned my addr 172.21.15.92:0/2013030233 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 msgr2=0x7f3bb00a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 msgr2=0x7f3bb00a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-05-16T07:02:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 0x7f3bb00a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-05-16T07:02:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baffff700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 0x7f3bb00a6c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:02:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 0x7f3bb00a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baeffd700 1 -- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 msgr2=0x7f3bb00a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-05-16T07:02:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baeffd700 1 -- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 msgr2=0x7f3bb00a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-05-16T07:02:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 0x7f3bb00a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-05-16T07:02:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.544+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 0x7f3bb00a6780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:02:08.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237370603 0 0) 0x7f3bb0064e90 con 0x7f3bb0150bb0 2023-05-16T07:02:08.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b980033f0 con 0x7f3bb0150bb0 2023-05-16T07:02:08.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 <== mon.1 v1:172.21.15.158:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058054901 0 0) 0x7f3b980033f0 con 0x7f3bb0150bb0 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 msgr2=0x7f3bb00a6780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.158:3300/0 conn(0x7f3bb0068670 0x7f3bb00a6780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 msgr2=0x7f3bb00a6c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 --2- 172.21.15.92:0/2013030233 >> v2:172.21.15.92:3300/0 conn(0x7f3bb0150f90 0x7f3bb00a6c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 --> v1:172.21.15.158:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00635b0 con 0x7f3bb0150bb0 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 <== mon.1 v1:172.21.15.158:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f3ba00017f0 con 0x7f3bb0150bb0 2023-05-16T07:02:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 >> v1:172.21.15.158:6789/0 conn(0x7f3bb0150bb0 legacy=0x7f3bb0067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3b980039b0 0x7f3b98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98004690 0x7f3b98004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b98005050 0x7f3b9800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb0065a10 con 0x7f3b98005050 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00a7160 con 0x7f3b980039b0 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9800bfb0 con 0x7f3b98004690 2023-05-16T07:02:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3baffff700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b98005050 0x7f3b9800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3baf7fe700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3b980039b0 0x7f3b98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.700+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98004690 0x7f3b98004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98004690 msgr2=0x7f3b98004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98004690 0x7f3b98004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3b980039b0 msgr2=0x7f3b98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3b980039b0 0x7f3b98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980033f0 con 0x7f3b98005050 2023-05-16T07:02:08.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 -- 172.21.15.92:0/2013030233 reap_dead start 2023-05-16T07:02:08.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.701+0000 7f3baffff700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b98005050 0x7f3b9800b8e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3ba8006e70 tx=0x7f3ba8008040).ready entity=mon.0 client_cookie=dcc7d6ac14a1c652 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.702+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800b3f0 con 0x7f3b98005050 2023-05-16T07:02:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.702+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba800be90 con 0x7f3b98005050 2023-05-16T07:02:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.702+0000 7f3bae7fc700 1 -- 172.21.15.92:0/2013030233 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800d7e0 con 0x7f3b98005050 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.702+0000 7f3bb6174700 1 -- 172.21.15.92:0/2013030233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b98005050 msgr2=0x7f3b9800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.702+0000 7f3bb6174700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b98005050 0x7f3b9800b8e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3ba8006e70 tx=0x7f3ba8008040).stop 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.703+0000 7f3bb6174700 1 -- 172.21.15.92:0/2013030233 shutdown_connections 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.703+0000 7f3bb6174700 1 --2- 172.21.15.92:0/2013030233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b98005050 0x7f3b9800b8e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.703+0000 7f3bb6174700 1 -- 172.21.15.92:0/2013030233 >> 172.21.15.92:0/2013030233 conn(0x7f3bb00c8420 msgr2=0x7f3bb00c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.703+0000 7f3bb6174700 1 -- 172.21.15.92:0/2013030233 shutdown_connections 2023-05-16T07:02:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.703+0000 7f3bb6174700 1 -- 172.21.15.92:0/2013030233 wait complete. 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.704+0000 7f3bb6174700 1 Processor -- start 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.704+0000 7f3bb6174700 1 -- start start 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98005050 0x7f3bb014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 0x7f3bb0149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bb014a1e0 0x7f3bb014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00a76a0 con 0x7f3bb0150320 2023-05-16T07:02:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb014a640 con 0x7f3bb014a1e0 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00a78f0 con 0x7f3b98005050 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 0x7f3bb0149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 0x7f3bb0149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39918/0 (socket says 172.21.15.92:39918) 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 -- 172.21.15.92:0/3280263767 learned_addr learned my addr 172.21.15.92:0/3280263767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baffff700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bb014a1e0 0x7f3bb014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baf7fe700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98005050 0x7f3bb014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:08.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 -- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98005050 msgr2=0x7f3bb014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98005050 0x7f3bb014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 -- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bb014a1e0 msgr2=0x7f3bb014b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bb014a1e0 0x7f3bb014b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 -- 172.21.15.92:0/3280263767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00a7bd0 con 0x7f3bb0150320 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3baeffd700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 0x7f3bb0149cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3ba400f6e0 tx=0x7f3ba4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4012570 con 0x7f3bb0150320 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba40126d0 con 0x7f3bb0150320 2023-05-16T07:02:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb00a7eb0 con 0x7f3bb0150320 2023-05-16T07:02:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4012a70 con 0x7f3bb0150320 2023-05-16T07:02:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb00a8470 con 0x7f3bb0150320 2023-05-16T07:02:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.705+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f3ba4005ae0 con 0x7f3bb0150320 2023-05-16T07:02:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.706+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f3ba4006650 con 0x7f3bb0150320 2023-05-16T07:02:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.706+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb00a8690 con 0x7f3bb0150320 2023-05-16T07:02:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.708+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f3ba4026490 con 0x7f3bb0150320 2023-05-16T07:02:08.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.778+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3bb006c350 con 0x7f3bb0150320 2023-05-16T07:02:08.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3b977fe700 1 -- 172.21.15.92:0/3280263767 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3ba4026630 con 0x7f3bb0150320 2023-05-16T07:02:08.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 msgr2=0x7f3bb0149cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 0x7f3bb0149cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3ba400f6e0 tx=0x7f3ba4008040).stop 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 shutdown_connections 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3b98005050 0x7f3bb014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bb014a1e0 0x7f3bb014b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 --2- 172.21.15.92:0/3280263767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb0150320 0x7f3bb0149cf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.781+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 >> 172.21.15.92:0/3280263767 conn(0x7f3bb00c8420 msgr2=0x7f3bb00c28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:08.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.782+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 shutdown_connections 2023-05-16T07:02:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:08.782+0000 7f3bb6174700 1 -- 172.21.15.92:0/3280263767 wait complete. 2023-05-16T07:02:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2023-05-16T07:02:08.796 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-05-16T07:02:08.797 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:08.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-05-16T07:02:08.820 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:08.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-05-16T07:02:08.831 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:08.828+0000 7fc83a07b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:02:08.882 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:08.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-05-16T07:02:08.899 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:08.895+0000 7fc83a07b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:02:08.946 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-05-16T07:02:08.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-05-16T07:02:09.008 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:09.009 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-05-16T07:02:09.032 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:09.032 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-05-16T07:02:09.096 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:09.096 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-05-16T07:02:09.160 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-05-16T07:02:09.160 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-05-16T07:02:09.223 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new 7a2cf1f9-d953-439c-a6b2-9aaa74344bdd 0 2023-05-16T07:02:09.228 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:09.224+0000 7f14d02be680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:02:09.296 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:09.292+0000 7f14d02be680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:02:09.375 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.371+0000 7f1b30d95700 1 Processor -- start 2023-05-16T07:02:09.375 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.371+0000 7f1b30d95700 1 -- start start 2023-05-16T07:02:09.378 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b30d95700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b2c153f90 0x7f1b2c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.378 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b30d95700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.378 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b30d95700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2c0c6670 con 0x7f1b2c0c0150 2023-05-16T07:02:09.378 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b30d95700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2c069f60 con 0x7f1b2c153f90 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b30d95700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c06a9e0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2bfff700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f1b2c0c0150 0x7f1b2c0c05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:44868/0 (socket says 172.21.15.158:44868) 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2bfff700 1 -- 172.21.15.158:0/4159671816 learned_addr learned my addr 172.21.15.158:0/4159671816 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2b7fe700 1 --2- 172.21.15.158:0/4159671816 >> v2:172.21.15.92:3300/0 conn(0x7f1b2c153f90 0x7f1b2c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2a7fc700 1 -- 172.21.15.158:0/4159671816 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604770551 0 0) 0x7f1b2c0c6670 con 0x7f1b2c0c0150 2023-05-16T07:02:09.379 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2a7fc700 1 -- 172.21.15.158:0/4159671816 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b140033f0 con 0x7f1b2c0c0150 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2affd700 1 -- 172.21.15.158:0/4159671816 >> v2:172.21.15.92:3300/0 conn(0x7f1b2c153f90 msgr2=0x7f1b2c0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2affd700 1 --2- 172.21.15.158:0/4159671816 >> v2:172.21.15.92:3300/0 conn(0x7f1b2c153f90 0x7f1b2c0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2affd700 1 -- 172.21.15.158:0/4159671816 >> v1:172.21.15.158:6789/0 conn(0x7f1b2c0c0150 legacy=0x7f1b2c0c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2affd700 1 -- 172.21.15.158:0/4159671816 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c06a140 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2b7fe700 1 --2- 172.21.15.158:0/4159671816 >> v2:172.21.15.92:3300/0 conn(0x7f1b2c153f90 0x7f1b2c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2affd700 1 --2- 172.21.15.158:0/4159671816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c0c6180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1b20009760 tx=0x7f1b20009060).ready entity=mon.2 client_cookie=fabbfc392c6c2873 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:09.380 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2a7fc700 1 -- 172.21.15.158:0/4159671816 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000f510 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.381 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2a7fc700 1 -- 172.21.15.158:0/4159671816 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2000f670 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.381 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.372+0000 7f1b2a7fc700 1 -- 172.21.15.158:0/4159671816 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000f9f0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.381 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 -- 172.21.15.158:0/4159671816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 msgr2=0x7f1b2c0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.381 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 --2- 172.21.15.158:0/4159671816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c0c6180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1b20009760 tx=0x7f1b20009060).stop 2023-05-16T07:02:09.381 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 -- 172.21.15.158:0/4159671816 shutdown_connections 2023-05-16T07:02:09.381 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 --2- 172.21.15.158:0/4159671816 >> v2:172.21.15.92:3300/0 conn(0x7f1b2c153f90 0x7f1b2c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.382 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 --2- 172.21.15.158:0/4159671816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c0c6180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.382 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 -- 172.21.15.158:0/4159671816 >> 172.21.15.158:0/4159671816 conn(0x7f1b2c0cb600 msgr2=0x7f1b2c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:09.382 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 -- 172.21.15.158:0/4159671816 shutdown_connections 2023-05-16T07:02:09.382 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.373+0000 7f1b30d95700 1 -- 172.21.15.158:0/4159671816 wait complete. 2023-05-16T07:02:09.382 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.374+0000 7f1b30d95700 1 Processor -- start 2023-05-16T07:02:09.382 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.374+0000 7f1b30d95700 1 -- start start 2023-05-16T07:02:09.385 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b30d95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b2c0c0150 0x7f1b2c14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.385 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b30d95700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.385 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b30d95700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 0x7f1b2c167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.385 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b30d95700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c167f70 con 0x7f1b2c0c0150 2023-05-16T07:02:09.385 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b30d95700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0c2610 con 0x7f1b2c153f90 2023-05-16T07:02:09.385 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b30d95700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0c1ce0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.386 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2bfff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 0x7f1b2c167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.386 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.386 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c167590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41090/0 (socket says 172.21.15.158:41090) 2023-05-16T07:02:09.386 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2bfff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 0x7f1b2c167a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53824/0 (socket says 172.21.15.158:53824) 2023-05-16T07:02:09.386 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 -- 172.21.15.158:0/425686554 learned_addr learned my addr 172.21.15.158:0/425686554 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:09.386 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2b7fe700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b2c0c0150 0x7f1b2c14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 -- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 msgr2=0x7f1b2c167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 0x7f1b2c167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 -- 172.21.15.158:0/425686554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b2c0c0150 msgr2=0x7f1b2c14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b2c0c0150 0x7f1b2c14da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2affd700 1 -- 172.21.15.158:0/425686554 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b200092a0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.375+0000 7f1b2bfff700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 0x7f1b2c167a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b2affd700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c167590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f1b2c0aa770 tx=0x7f1b20017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:09.387 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000f7e0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2000f940 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c0c1fc0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2001b660 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c0afa50 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f1b2002c070 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.376+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f1b20023770 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.388 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.377+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c083120 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.389 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.379+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f1b2001be50 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.392 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:09.389+0000 7fc83a07b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:02:09.462 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.459+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7a2cf1f9-d953-439c-a6b2-9aaa74344bdd", "id": 0} v 0) v1 -- 0x7f1b2c047150 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.465 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.462+0000 7f1b137fe700 1 -- 172.21.15.158:0/425686554 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7a2cf1f9-d953-439c-a6b2-9aaa74344bdd", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1b200350b0 con 0x7f1b2c0c5d00 2023-05-16T07:02:09.466 INFO:teuthology.orchestra.run.smithi158.stdout:0 2023-05-16T07:02:09.466 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 msgr2=0x7f1b2c167590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c167590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f1b2c0aa770 tx=0x7f1b20017000).stop 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 shutdown_connections 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b2c153f90 0x7f1b2c167a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b2c0c0150 0x7f1b2c14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 --2- 172.21.15.158:0/425686554 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b2c0c5d00 0x7f1b2c167590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 >> 172.21.15.158:0/425686554 conn(0x7f1b2c0cb600 msgr2=0x7f1b2c0c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:09.467 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 shutdown_connections 2023-05-16T07:02:09.468 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.463+0000 7f1b30d95700 1 -- 172.21.15.158:0/425686554 wait complete. 2023-05-16T07:02:09.479 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new 9997c8eb-1fdd-4bfe-ae2d-b08544f22d90 1 2023-05-16T07:02:09.590 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.586+0000 7fdc16f18700 1 Processor -- start 2023-05-16T07:02:09.590 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.586+0000 7fdc16f18700 1 -- start start 2023-05-16T07:02:09.592 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc16f18700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 0x7fdc1014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.592 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc16f18700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdc10157be0 0x7fdc10158060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.592 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc16f18700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc10158550 con 0x7fdc10068ba0 2023-05-16T07:02:09.592 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc16f18700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1006a470 con 0x7fdc10157be0 2023-05-16T07:02:09.592 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc16f18700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc10137670 con 0x7fdc1014e210 2023-05-16T07:02:09.592 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc0ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdc10157be0 0x7fdc10158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc14cb4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 0x7fdc1014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc14cb4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 0x7fdc1014e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41094/0 (socket says 172.21.15.158:41094) 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc0ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdc10157be0 0x7fdc10158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53828/0 (socket says 172.21.15.158:53828) 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc14cb4700 1 -- 172.21.15.158:0/4103843926 learned_addr learned my addr 172.21.15.158:0/4103843926 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.587+0000 7fdc154b5700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fdc10068ba0 0x7fdc10068ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:44880/0 (socket says 172.21.15.158:44880) 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0ffff700 1 -- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 msgr2=0x7fdc1014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0ffff700 1 --2- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 0x7fdc1014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.593 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0ffff700 1 -- 172.21.15.158:0/4103843926 >> v1:172.21.15.158:6789/0 conn(0x7fdc10068ba0 legacy=0x7fdc10068ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0f7fe700 1 -- 172.21.15.158:0/4103843926 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014792403 0 0) 0x7fdc10158550 con 0x7fdc10068ba0 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0ffff700 1 -- 172.21.15.158:0/4103843926 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100697f0 con 0x7fdc10157be0 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc14cb4700 1 --2- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 0x7fdc1014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0ffff700 1 --2- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10157be0 0x7fdc10158060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdbfc009760 tx=0x7fdbfc009060).ready entity=mon.1 client_cookie=c77d55e4cd321eaa server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0f7fe700 1 -- 172.21.15.158:0/4103843926 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00f510 con 0x7fdc10157be0 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0f7fe700 1 -- 172.21.15.158:0/4103843926 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc00f670 con 0x7fdc10157be0 2023-05-16T07:02:09.594 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.588+0000 7fdc0f7fe700 1 -- 172.21.15.158:0/4103843926 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00f9f0 con 0x7fdc10157be0 2023-05-16T07:02:09.597 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 -- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10157be0 msgr2=0x7fdc10158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.597 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10157be0 0x7fdc10158060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdbfc009760 tx=0x7fdbfc009060).stop 2023-05-16T07:02:09.597 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 -- 172.21.15.158:0/4103843926 shutdown_connections 2023-05-16T07:02:09.597 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc1014e210 0x7fdc1014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.597 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4103843926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10157be0 0x7fdc10158060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.597 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 -- 172.21.15.158:0/4103843926 >> 172.21.15.158:0/4103843926 conn(0x7fdc100c4600 msgr2=0x7fdc10066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 -- 172.21.15.158:0/4103843926 shutdown_connections 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.589+0000 7fdc16f18700 1 -- 172.21.15.158:0/4103843926 wait complete. 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.590+0000 7fdc16f18700 1 Processor -- start 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.590+0000 7fdc16f18700 1 -- start start 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 0x7fdc1014d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1014e210 0x7fdc100ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc10157be0 0x7fdc10065790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.598 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1014ed50 con 0x7fdc1014e210 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1014f030 con 0x7fdc10068ba0 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1014f310 con 0x7fdc10157be0 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 0x7fdc1014d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc154b5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc10157be0 0x7fdc10065790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 0x7fdc1014d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53832/0 (socket says 172.21.15.158:53832) 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 -- 172.21.15.158:0/4216783256 learned_addr learned my addr 172.21.15.158:0/4216783256 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:09.599 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc0ffff700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1014e210 0x7fdc100ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 -- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc10157be0 msgr2=0x7fdc10065790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc10157be0 0x7fdc10065790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 -- 172.21.15.158:0/4216783256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1014e210 msgr2=0x7fdc100ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1014e210 0x7fdc100ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 -- 172.21.15.158:0/4216783256 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04009370 con 0x7fdc10068ba0 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc14cb4700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 0x7fdc1014d890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdc0400ea20 tx=0x7fdc040090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04009650 con 0x7fdc10068ba0 2023-05-16T07:02:09.600 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0400c850 con 0x7fdc10068ba0 2023-05-16T07:02:09.601 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc0092a0 con 0x7fdc10068ba0 2023-05-16T07:02:09.601 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400cbd0 con 0x7fdc10068ba0 2023-05-16T07:02:09.601 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.591+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1014f8d0 con 0x7fdc10068ba0 2023-05-16T07:02:09.601 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.592+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fdc04022070 con 0x7fdc10068ba0 2023-05-16T07:02:09.601 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.592+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fdc040123f0 con 0x7fdc10068ba0 2023-05-16T07:02:09.601 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.592+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1014fc40 con 0x7fdc10068ba0 2023-05-16T07:02:09.602 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.595+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fdc0403ae50 con 0x7fdc10068ba0 2023-05-16T07:02:09.712 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.708+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9997c8eb-1fdd-4bfe-ae2d-b08544f22d90", "id": 1} v 0) v1 -- 0x7fdc101366a0 con 0x7fdc10068ba0 2023-05-16T07:02:09.719 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.716+0000 7fdc0cff9700 1 -- 172.21.15.158:0/4216783256 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9997c8eb-1fdd-4bfe-ae2d-b08544f22d90", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc04029030 con 0x7fdc10068ba0 2023-05-16T07:02:09.720 INFO:teuthology.orchestra.run.smithi158.stdout:1 2023-05-16T07:02:09.721 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 msgr2=0x7fdc1014d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.721 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 0x7fdc1014d890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdc0400ea20 tx=0x7fdc040090f0).stop 2023-05-16T07:02:09.721 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 shutdown_connections 2023-05-16T07:02:09.721 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc10157be0 0x7fdc10065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.721 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc10068ba0 0x7fdc1014d890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.721 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 --2- 172.21.15.158:0/4216783256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1014e210 0x7fdc100ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.722 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 >> 172.21.15.158:0/4216783256 conn(0x7fdc100c4600 msgr2=0x7fdc10154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:09.722 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 shutdown_connections 2023-05-16T07:02:09.722 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.717+0000 7fdc16f18700 1 -- 172.21.15.158:0/4216783256 wait complete. 2023-05-16T07:02:09.732 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new 637f304d-b1f2-4217-8b73-1f40aa9115f5 2 2023-05-16T07:02:09.758 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:09.755+0000 7f14d02be680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:02:09.845 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.841+0000 7fbf28849700 1 Processor -- start 2023-05-16T07:02:09.845 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf28849700 1 -- start start 2023-05-16T07:02:09.848 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf28849700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf240a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.848 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf28849700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbf24155380 0x7fbf240c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.848 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf28849700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240c6670 con 0x7fbf240c5d00 2023-05-16T07:02:09.848 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf28849700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf24069f60 con 0x7fbf24155380 2023-05-16T07:02:09.849 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf28849700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2406a9e0 con 0x7fbf24153f90 2023-05-16T07:02:09.849 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.842+0000 7fbf22d9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf240a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.849 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf2259c700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fbf240c5d00 0x7fbf240c0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:44890/0 (socket says 172.21.15.158:44890) 2023-05-16T07:02:09.849 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf2259c700 1 -- 172.21.15.158:0/1436107956 learned_addr learned my addr 172.21.15.158:0/1436107956 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-05-16T07:02:09.849 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf2359e700 1 --2- 172.21.15.158:0/1436107956 >> v2:172.21.15.92:3300/0 conn(0x7fbf24155380 0x7fbf240c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.850 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf22d9d700 1 -- 172.21.15.158:0/1436107956 >> v2:172.21.15.92:3300/0 conn(0x7fbf24155380 msgr2=0x7fbf240c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:09.850 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf22d9d700 1 --2- 172.21.15.158:0/1436107956 >> v2:172.21.15.92:3300/0 conn(0x7fbf24155380 0x7fbf240c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.850 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf21d9b700 1 -- 172.21.15.158:0/1436107956 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337053989 0 0) 0x7fbf240c6670 con 0x7fbf240c5d00 2023-05-16T07:02:09.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf22d9d700 1 -- 172.21.15.158:0/1436107956 >> v1:172.21.15.158:6789/0 conn(0x7fbf240c5d00 legacy=0x7fbf240c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf22d9d700 1 -- 172.21.15.158:0/1436107956 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2406a140 con 0x7fbf24153f90 2023-05-16T07:02:09.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf22d9d700 1 --2- 172.21.15.158:0/1436107956 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf240a9d20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbf14009760 tx=0x7fbf14009060).ready entity=mon.2 client_cookie=d5e83455a7ea4d1e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:09.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf21d9b700 1 -- 172.21.15.158:0/1436107956 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400b580 con 0x7fbf24153f90 2023-05-16T07:02:09.852 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf21d9b700 1 -- 172.21.15.158:0/1436107956 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400b6e0 con 0x7fbf24153f90 2023-05-16T07:02:09.852 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.843+0000 7fbf21d9b700 1 -- 172.21.15.158:0/1436107956 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400ba60 con 0x7fbf24153f90 2023-05-16T07:02:09.853 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 -- 172.21.15.158:0/1436107956 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 msgr2=0x7fbf240a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.853 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 --2- 172.21.15.158:0/1436107956 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf240a9d20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbf14009760 tx=0x7fbf14009060).stop 2023-05-16T07:02:09.853 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 -- 172.21.15.158:0/1436107956 shutdown_connections 2023-05-16T07:02:09.853 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 --2- 172.21.15.158:0/1436107956 >> v2:172.21.15.92:3300/0 conn(0x7fbf24155380 0x7fbf240c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 --2- 172.21.15.158:0/1436107956 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf240a9d20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 -- 172.21.15.158:0/1436107956 >> 172.21.15.158:0/1436107956 conn(0x7fbf240cb600 msgr2=0x7fbf240c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:09.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 -- 172.21.15.158:0/1436107956 shutdown_connections 2023-05-16T07:02:09.854 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.844+0000 7fbf28849700 1 -- 172.21.15.158:0/1436107956 wait complete. 2023-05-16T07:02:09.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.845+0000 7fbf28849700 1 Processor -- start 2023-05-16T07:02:09.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.845+0000 7fbf28849700 1 -- start start 2023-05-16T07:02:09.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf240c5d00 0x7fbf240ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.859 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf24163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.859 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf24155380 0x7fbf240b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:09.859 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ac170 con 0x7fbf24155380 2023-05-16T07:02:09.859 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ab560 con 0x7fbf240c5d00 2023-05-16T07:02:09.860 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ab840 con 0x7fbf24153f90 2023-05-16T07:02:09.860 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf24163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.860 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf24163bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41114/0 (socket says 172.21.15.158:41114) 2023-05-16T07:02:09.860 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 -- 172.21.15.158:0/2719999934 learned_addr learned my addr 172.21.15.158:0/2719999934 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:09.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf22d9d700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf240c5d00 0x7fbf240ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2359e700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf24155380 0x7fbf240b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:09.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 -- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf240c5d00 msgr2=0x7fbf240ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf240c5d00 0x7fbf240ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 -- 172.21.15.158:0/2719999934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf24155380 msgr2=0x7fbf240b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf24155380 0x7fbf240b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 -- 172.21.15.158:0/2719999934 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140092a0 con 0x7fbf24153f90 2023-05-16T07:02:09.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf2259c700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf24163bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fbf1800c490 tx=0x7fbf18007e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:09.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180070d0 con 0x7fbf24153f90 2023-05-16T07:02:09.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf18007230 con 0x7fbf24153f90 2023-05-16T07:02:09.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240abb20 con 0x7fbf24153f90 2023-05-16T07:02:09.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180075d0 con 0x7fbf24153f90 2023-05-16T07:02:09.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.846+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf241640a0 con 0x7fbf24153f90 2023-05-16T07:02:09.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.847+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fbf1801a060 con 0x7fbf24153f90 2023-05-16T07:02:09.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.847+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fbf18013c10 con 0x7fbf24153f90 2023-05-16T07:02:09.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.847+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24083120 con 0x7fbf24153f90 2023-05-16T07:02:09.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.849+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fbf18018070 con 0x7fbf24153f90 2023-05-16T07:02:09.932 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.929+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "osd new", "uuid": "637f304d-b1f2-4217-8b73-1f40aa9115f5", "id": 2} v 0) v1 -- 0x7fbf24047150 con 0x7fbf24153f90 2023-05-16T07:02:09.940 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.937+0000 7fbf0affd700 1 -- 172.21.15.158:0/2719999934 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "637f304d-b1f2-4217-8b73-1f40aa9115f5", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf180137c0 con 0x7fbf24153f90 2023-05-16T07:02:09.940 INFO:teuthology.orchestra.run.smithi158.stdout:2 2023-05-16T07:02:09.941 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 msgr2=0x7fbf24163bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:09.942 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf24163bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fbf1800c490 tx=0x7fbf18007e60).stop 2023-05-16T07:02:09.942 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 shutdown_connections 2023-05-16T07:02:09.942 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf24155380 0x7fbf240b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.942 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf240c5d00 0x7fbf240ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.942 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 --2- 172.21.15.158:0/2719999934 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf24153f90 0x7fbf24163bb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:09.942 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 >> 172.21.15.158:0/2719999934 conn(0x7fbf240cb600 msgr2=0x7fbf240c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:09.943 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 shutdown_connections 2023-05-16T07:02:09.943 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:09.938+0000 7fbf28849700 1 -- 172.21.15.158:0/2719999934 wait complete. 2023-05-16T07:02:09.954 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new f061ab26-f80b-471b-93b4-a2c5ed339458 3 2023-05-16T07:02:10.066 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.062+0000 7f102859a700 1 Processor -- start 2023-05-16T07:02:10.066 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.062+0000 7f102859a700 1 -- start start 2023-05-16T07:02:10.068 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.063+0000 7f102859a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10200aa210 0x7f10200c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.069 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.063+0000 7f102859a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f1020153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.069 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.063+0000 7f102859a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10200c8830 con 0x7f10200c5d30 2023-05-16T07:02:10.069 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.063+0000 7f102859a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10200ca600 con 0x7f10200aa210 2023-05-16T07:02:10.069 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.063+0000 7f102859a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c8b10 con 0x7f1020153aa0 2023-05-16T07:02:10.069 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f1020153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.069 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f1020153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41122/0 (socket says 172.21.15.158:41122) 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 -- 172.21.15.158:0/164859004 learned_addr learned my addr 172.21.15.158:0/164859004 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1025b35700 1 --2- 172.21.15.158:0/164859004 >> v2:172.21.15.92:3300/0 conn(0x7f10200aa210 0x7f10200c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 -- 172.21.15.158:0/164859004 >> v2:172.21.15.92:3300/0 conn(0x7f10200aa210 msgr2=0x7f10200c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 --2- 172.21.15.158:0/164859004 >> v2:172.21.15.92:3300/0 conn(0x7f10200aa210 0x7f10200c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 -- 172.21.15.158:0/164859004 >> v1:172.21.15.92:6789/0 conn(0x7f10200c5d30 legacy=0x7f1020069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 -- 172.21.15.158:0/164859004 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200c8df0 con 0x7f1020153aa0 2023-05-16T07:02:10.070 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1026b37700 1 --2- 172.21.15.158:0/164859004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f1020153ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f1010006fd0 tx=0x7f10100090f0).ready entity=mon.2 client_cookie=7acac2814028516 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1025334700 1 -- 172.21.15.158:0/164859004 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000b580 con 0x7f1020153aa0 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1025334700 1 -- 172.21.15.158:0/164859004 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101000b6e0 con 0x7f1020153aa0 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.064+0000 7f1025334700 1 -- 172.21.15.158:0/164859004 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000ba60 con 0x7f1020153aa0 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 -- 172.21.15.158:0/164859004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 msgr2=0x7f1020153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 --2- 172.21.15.158:0/164859004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f1020153ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f1010006fd0 tx=0x7f10100090f0).stop 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 -- 172.21.15.158:0/164859004 shutdown_connections 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 --2- 172.21.15.158:0/164859004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f1020153ef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.071 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 --2- 172.21.15.158:0/164859004 >> v2:172.21.15.92:3300/0 conn(0x7f10200aa210 0x7f10200c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.072 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 -- 172.21.15.158:0/164859004 >> 172.21.15.158:0/164859004 conn(0x7f10200cb700 msgr2=0x7f10200c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.072 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 -- 172.21.15.158:0/164859004 shutdown_connections 2023-05-16T07:02:10.072 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.065+0000 7f102859a700 1 -- 172.21.15.158:0/164859004 wait complete. 2023-05-16T07:02:10.074 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.066+0000 7f102859a700 1 Processor -- start 2023-05-16T07:02:10.074 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.066+0000 7f102859a700 1 -- start start 2023-05-16T07:02:10.074 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10200aa210 0x7f10200c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10200c5d30 0x7f102014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f10201672f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c2af0 con 0x7f10200aa210 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c1e70 con 0x7f10200c5d30 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c2150 con 0x7f1020153aa0 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1025b35700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10200c5d30 0x7f102014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f10201672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.075 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1025b35700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10200c5d30 0x7f102014e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53858/0 (socket says 172.21.15.158:53858) 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f10201672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41128/0 (socket says 172.21.15.158:41128) 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1025b35700 1 -- 172.21.15.158:0/2744739768 learned_addr learned my addr 172.21.15.158:0/2744739768 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026336700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10200aa210 0x7f10200c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 -- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10200c5d30 msgr2=0x7f102014e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10200c5d30 0x7f102014e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 -- 172.21.15.158:0/2744739768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10200aa210 msgr2=0x7f10200c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.076 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10200aa210 0x7f10200c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 -- 172.21.15.158:0/2744739768 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009370 con 0x7f1020153aa0 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f1026b37700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f10201672f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f1010014000 tx=0x7f1010017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000b850 con 0x7f1020153aa0 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101000b9b0 con 0x7f1020153aa0 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10200c2430 con 0x7f1020153aa0 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001c670 con 0x7f1020153aa0 2023-05-16T07:02:10.077 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.067+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200c2f10 con 0x7f1020153aa0 2023-05-16T07:02:10.078 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.068+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f1010022020 con 0x7f1020153aa0 2023-05-16T07:02:10.078 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.068+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f1010025960 con 0x7f1020153aa0 2023-05-16T07:02:10.078 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.068+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10200c2c30 con 0x7f1020153aa0 2023-05-16T07:02:10.078 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.070+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f10200c2c30 con 0x7f1020153aa0 2023-05-16T07:02:10.182 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:10.179+0000 7fc83a07b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:02:10.187 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.184+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f061ab26-f80b-471b-93b4-a2c5ed339458", "id": 3} v 0) v1 -- 0x7f102006c350 con 0x7f1020153aa0 2023-05-16T07:02:10.195 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.191+0000 7f101e7fc700 1 -- 172.21.15.158:0/2744739768 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f061ab26-f80b-471b-93b4-a2c5ed339458", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f10100254a0 con 0x7f1020153aa0 2023-05-16T07:02:10.195 INFO:teuthology.orchestra.run.smithi158.stdout:3 2023-05-16T07:02:10.196 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 msgr2=0x7f10201672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.196 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f10201672f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f1010014000 tx=0x7f1010017000).stop 2023-05-16T07:02:10.196 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 shutdown_connections 2023-05-16T07:02:10.197 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1020153aa0 0x7f10201672f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.197 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10200aa210 0x7f10200c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.197 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 --2- 172.21.15.158:0/2744739768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10200c5d30 0x7f102014e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.197 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 >> 172.21.15.158:0/2744739768 conn(0x7f10200cb700 msgr2=0x7f1020151320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.198 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 shutdown_connections 2023-05-16T07:02:10.198 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.192+0000 7f102859a700 1 -- 172.21.15.158:0/2744739768 wait complete. 2023-05-16T07:02:10.207 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new c2795304-ad42-4a2d-bd4a-bf8ad7aec574 4 2023-05-16T07:02:10.253 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:10.250+0000 7fc83a07b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:02:10.322 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.318+0000 7fef76afd700 1 Processor -- start 2023-05-16T07:02:10.322 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.318+0000 7fef76afd700 1 -- start start 2023-05-16T07:02:10.323 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef76afd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fef700c6110 0x7fef701541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef76afd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700ca600 con 0x7fef700c5d30 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef76afd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700c8830 con 0x7fef700aa210 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef76afd700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef700c6560 con 0x7fef700c6110 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef7509a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fef700c6110 0x7fef701541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef7509a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fef700c6110 0x7fef701541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53866/0 (socket says 172.21.15.158:53866) 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.319+0000 7fef7509a700 1 -- 172.21.15.158:0/1875928303 learned_addr learned my addr 172.21.15.158:0/1875928303 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.324 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef7509a700 1 -- 172.21.15.158:0/1875928303 >> v1:172.21.15.158:6789/0 conn(0x7fef700aa210 legacy=0x7fef70153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.325 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef7509a700 1 -- 172.21.15.158:0/1875928303 >> v1:172.21.15.92:6789/0 conn(0x7fef700c5d30 legacy=0x7fef70069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.325 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef7509a700 1 -- 172.21.15.158:0/1875928303 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700c8b10 con 0x7fef700c6110 2023-05-16T07:02:10.325 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef7509a700 1 --2- 172.21.15.158:0/1875928303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef701541b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fef68009f20 tx=0x7fef68009840).ready entity=mon.1 client_cookie=321762e8f2608908 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.325 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef6f7fe700 1 -- 172.21.15.158:0/1875928303 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800e070 con 0x7fef700c6110 2023-05-16T07:02:10.325 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef6f7fe700 1 -- 172.21.15.158:0/1875928303 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6800ba80 con 0x7fef700c6110 2023-05-16T07:02:10.325 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef6f7fe700 1 -- 172.21.15.158:0/1875928303 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800be00 con 0x7fef700c6110 2023-05-16T07:02:10.328 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef76afd700 1 -- 172.21.15.158:0/1875928303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 msgr2=0x7fef701541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.328 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef76afd700 1 --2- 172.21.15.158:0/1875928303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef701541b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fef68009f20 tx=0x7fef68009840).stop 2023-05-16T07:02:10.328 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.320+0000 7fef76afd700 1 -- 172.21.15.158:0/1875928303 shutdown_connections 2023-05-16T07:02:10.328 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.321+0000 7fef76afd700 1 --2- 172.21.15.158:0/1875928303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef701541b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.328 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.321+0000 7fef76afd700 1 -- 172.21.15.158:0/1875928303 >> 172.21.15.158:0/1875928303 conn(0x7fef700cb700 msgr2=0x7fef700c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.321+0000 7fef76afd700 1 -- 172.21.15.158:0/1875928303 shutdown_connections 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.321+0000 7fef76afd700 1 -- 172.21.15.158:0/1875928303 wait complete. 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.322+0000 7fef76afd700 1 Processor -- start 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.322+0000 7fef76afd700 1 -- start start 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.322+0000 7fef76afd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef700aa210 0x7fef70155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.322+0000 7fef76afd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef700c5d30 0x7fef70156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.329 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef76afd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef70150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef76afd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70155840 con 0x7fef700aa210 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef76afd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70156d10 con 0x7fef700c6110 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef76afd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700c1e50 con 0x7fef700c5d30 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef70150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef70150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53872/0 (socket says 172.21.15.158:53872) 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef700c5d30 0x7fef70156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.330 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 -- 172.21.15.158:0/1007206521 learned_addr learned my addr 172.21.15.158:0/1007206521 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef74899700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef700aa210 0x7fef70155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 -- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef700c5d30 msgr2=0x7fef70156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef700c5d30 0x7fef70156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 -- 172.21.15.158:0/1007206521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef700aa210 msgr2=0x7fef70155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef700aa210 0x7fef70155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 -- 172.21.15.158:0/1007206521 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009ae0 con 0x7fef700c6110 2023-05-16T07:02:10.331 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6ffff700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef700c5d30 0x7fef70156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef7509a700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef70150990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fef70083c50 tx=0x7fef68009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680092e0 con 0x7fef700c6110 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6801d410 con 0x7fef700c6110 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef70156f20 con 0x7fef700c6110 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6801d7d0 con 0x7fef700c6110 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef701574e0 con 0x7fef700c6110 2023-05-16T07:02:10.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fef6801b040 con 0x7fef700c6110 2023-05-16T07:02:10.333 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.323+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7fef68027c10 con 0x7fef700c6110 2023-05-16T07:02:10.333 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.324+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef70066730 con 0x7fef700c6110 2023-05-16T07:02:10.333 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.326+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fef68018040 con 0x7fef700c6110 2023-05-16T07:02:10.409 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.406+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c2795304-ad42-4a2d-bd4a-bf8ad7aec574", "id": 4} v 0) v1 -- 0x7fef700c2010 con 0x7fef700c6110 2023-05-16T07:02:10.418 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.414+0000 7fef6cff9700 1 -- 172.21.15.158:0/1007206521 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2795304-ad42-4a2d-bd4a-bf8ad7aec574", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef680276e0 con 0x7fef700c6110 2023-05-16T07:02:10.419 INFO:teuthology.orchestra.run.smithi158.stdout:4 2023-05-16T07:02:10.420 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 msgr2=0x7fef70150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.420 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef70150990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fef70083c50 tx=0x7fef68009840).stop 2023-05-16T07:02:10.420 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 shutdown_connections 2023-05-16T07:02:10.421 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef700c6110 0x7fef70150990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.421 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef700aa210 0x7fef70155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.421 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 --2- 172.21.15.158:0/1007206521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef700c5d30 0x7fef70156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.421 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 >> 172.21.15.158:0/1007206521 conn(0x7fef700cb700 msgr2=0x7fef700c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.422 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 shutdown_connections 2023-05-16T07:02:10.422 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.415+0000 7fef76afd700 1 -- 172.21.15.158:0/1007206521 wait complete. 2023-05-16T07:02:10.431 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new 0cd9fd10-239c-442c-a3fc-f6ec09b33c26 5 2023-05-16T07:02:10.449 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:10.446+0000 7fc83a07b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:02:10.478 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:10.474+0000 7f14d02be680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:02:10.543 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.540+0000 7f1a4bae3700 1 Processor -- start 2023-05-16T07:02:10.544 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.540+0000 7f1a4bae3700 1 -- start start 2023-05-16T07:02:10.544 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:10.540+0000 7f14d02be680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:02:10.546 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.541+0000 7f1a4bae3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a440aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.546 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.541+0000 7f1a4bae3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 0x7f1a44068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.546 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.541+0000 7f1a4bae3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440c94d0 con 0x7f1a44154470 2023-05-16T07:02:10.546 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.541+0000 7f1a4bae3700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440cb7e0 con 0x7f1a44068160 2023-05-16T07:02:10.546 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4bae3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44068a70 con 0x7f1a44154090 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a440aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4a080700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 0x7f1a44068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4a080700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 0x7f1a44068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53876/0 (socket says 172.21.15.158:53876) 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a440aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41142/0 (socket says 172.21.15.158:41142) 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4a080700 1 -- 172.21.15.158:0/1562561322 learned_addr learned my addr 172.21.15.158:0/1562561322 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 -- 172.21.15.158:0/1562561322 >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 msgr2=0x7f1a44068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:10.547 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 --2- 172.21.15.158:0/1562561322 >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 0x7f1a44068580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.548 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 -- 172.21.15.158:0/1562561322 >> v1:172.21.15.92:6789/0 conn(0x7f1a44154470 legacy=0x7f1a440a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.548 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 -- 172.21.15.158:0/1562561322 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440c88c0 con 0x7f1a44154090 2023-05-16T07:02:10.548 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4a080700 1 --2- 172.21.15.158:0/1562561322 >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 0x7f1a44068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:10.548 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4987f700 1 --2- 172.21.15.158:0/1562561322 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a440aaca0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f1a40009760 tx=0x7f1a40009060).ready entity=mon.2 client_cookie=79555bcfc0a9bed4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.548 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4887d700 1 -- 172.21.15.158:0/1562561322 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000f510 con 0x7f1a44154090 2023-05-16T07:02:10.548 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4887d700 1 -- 172.21.15.158:0/1562561322 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000f670 con 0x7f1a44154090 2023-05-16T07:02:10.549 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.542+0000 7f1a4887d700 1 -- 172.21.15.158:0/1562561322 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000f9f0 con 0x7f1a44154090 2023-05-16T07:02:10.551 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 -- 172.21.15.158:0/1562561322 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 msgr2=0x7f1a440aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.551 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/1562561322 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a440aaca0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f1a40009760 tx=0x7f1a40009060).stop 2023-05-16T07:02:10.551 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 -- 172.21.15.158:0/1562561322 shutdown_connections 2023-05-16T07:02:10.551 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/1562561322 >> v2:172.21.15.158:3300/0 conn(0x7f1a44068160 0x7f1a44068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.552 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/1562561322 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a440aaca0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.552 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 -- 172.21.15.158:0/1562561322 >> 172.21.15.158:0/1562561322 conn(0x7f1a44067920 msgr2=0x7f1a440cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.552 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 -- 172.21.15.158:0/1562561322 shutdown_connections 2023-05-16T07:02:10.552 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.543+0000 7f1a4bae3700 1 -- 172.21.15.158:0/1562561322 wait complete. 2023-05-16T07:02:10.552 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.545+0000 7f1a4bae3700 1 Processor -- start 2023-05-16T07:02:10.552 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.545+0000 7f1a4bae3700 1 -- start start 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4bae3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 0x7f1a4414f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4bae3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a44152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4bae3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154470 0x7f1a44152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4bae3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44151f60 con 0x7f1a44154470 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4bae3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4414f9b0 con 0x7f1a44068160 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4bae3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44153460 con 0x7f1a44154090 2023-05-16T07:02:10.553 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4987f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 0x7f1a4414f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4987f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 0x7f1a4414f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53878/0 (socket says 172.21.15.158:53878) 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4987f700 1 -- 172.21.15.158:0/4064093685 learned_addr learned my addr 172.21.15.158:0/4064093685 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.546+0000 7f1a4907e700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a44152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4a080700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154470 0x7f1a44152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4987f700 1 -- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 msgr2=0x7f1a44152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4987f700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a44152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.554 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4987f700 1 -- 172.21.15.158:0/4064093685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154470 msgr2=0x7f1a44152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4987f700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154470 0x7f1a44152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4987f700 1 -- 172.21.15.158:0/4064093685 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c009ae0 con 0x7f1a44068160 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4907e700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a44152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4987f700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 0x7f1a4414f4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1a40009e00 tx=0x7f1a40017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000f7e0 con 0x7f1a44068160 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000f940 con 0x7f1a44068160 2023-05-16T07:02:10.555 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4001b660 con 0x7f1a44068160 2023-05-16T07:02:10.556 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a400092a0 con 0x7f1a44068160 2023-05-16T07:02:10.556 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440af3a0 con 0x7f1a44068160 2023-05-16T07:02:10.556 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f1a4002b070 con 0x7f1a44068160 2023-05-16T07:02:10.556 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.547+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f1a40024a00 con 0x7f1a44068160 2023-05-16T07:02:10.557 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.548+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a440af680 con 0x7f1a44068160 2023-05-16T07:02:10.557 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.552+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f1a40029060 con 0x7f1a44068160 2023-05-16T07:02:10.634 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.631+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0cd9fd10-239c-442c-a3fc-f6ec09b33c26", "id": 5} v 0) v1 -- 0x7f1a4406c350 con 0x7f1a44068160 2023-05-16T07:02:10.644 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.638+0000 7f1a39ffb700 1 -- 172.21.15.158:0/4064093685 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0cd9fd10-239c-442c-a3fc-f6ec09b33c26", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a4001f070 con 0x7f1a44068160 2023-05-16T07:02:10.645 INFO:teuthology.orchestra.run.smithi158.stdout:5 2023-05-16T07:02:10.645 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 msgr2=0x7f1a4414f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 0x7f1a4414f4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1a40009e00 tx=0x7f1a40017000).stop 2023-05-16T07:02:10.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 shutdown_connections 2023-05-16T07:02:10.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154470 0x7f1a44152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a44068160 0x7f1a4414f4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 --2- 172.21.15.158:0/4064093685 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154090 0x7f1a44152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 >> 172.21.15.158:0/4064093685 conn(0x7f1a44067920 msgr2=0x7f1a440ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 shutdown_connections 2023-05-16T07:02:10.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.639+0000 7f1a4bae3700 1 -- 172.21.15.158:0/4064093685 wait complete. 2023-05-16T07:02:10.655 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new f3047d3d-d21e-4666-a4b1-72e17b0cd879 6 2023-05-16T07:02:10.730 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:10.726+0000 7f14d02be680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:02:10.808 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.804+0000 7f79221f5700 1 Processor -- start 2023-05-16T07:02:10.825 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.804+0000 7f79221f5700 1 -- start start 2023-05-16T07:02:10.829 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f79221f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f791c154090 0x7f791c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.829 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f79221f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0c94d0 con 0x7f791c068160 2023-05-16T07:02:10.829 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f79221f5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0cb7e0 con 0x7f791c154470 2023-05-16T07:02:10.829 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f79221f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791c0c88c0 con 0x7f791c154090 2023-05-16T07:02:10.830 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f7912ffd700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f791c154470 0x7f791c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:44936/0 (socket says 172.21.15.158:44936) 2023-05-16T07:02:10.830 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f7912ffd700 1 -- 172.21.15.158:0/2745735403 learned_addr learned my addr 172.21.15.158:0/2745735403 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-05-16T07:02:10.830 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.805+0000 7f791b7fe700 1 --2- 172.21.15.158:0/2745735403 >> v2:172.21.15.92:3300/0 conn(0x7f791c154090 0x7f791c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.830 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791affd700 1 -- 172.21.15.158:0/2745735403 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906572451 0 0) 0x7f791c0cb7e0 con 0x7f791c154470 2023-05-16T07:02:10.830 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791affd700 1 -- 172.21.15.158:0/2745735403 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79080033f0 con 0x7f791c154470 2023-05-16T07:02:10.830 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791b7fe700 1 -- 172.21.15.158:0/2745735403 >> v1:172.21.15.158:6789/0 conn(0x7f791c154470 legacy=0x7f791c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.831 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791b7fe700 1 -- 172.21.15.158:0/2745735403 >> v1:172.21.15.92:6789/0 conn(0x7f791c068160 legacy=0x7f791c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.831 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791affd700 1 -- 172.21.15.158:0/2745735403 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092141782 0 0) 0x7f791c0c94d0 con 0x7f791c068160 2023-05-16T07:02:10.831 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791b7fe700 1 -- 172.21.15.158:0/2745735403 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0c8ba0 con 0x7f791c154090 2023-05-16T07:02:10.831 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791b7fe700 1 --2- 172.21.15.158:0/2745735403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154090 0x7f791c0aaca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f790c009760 tx=0x7f790c009060).ready entity=mon.0 client_cookie=1237b5b34615b4ba server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.831 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791affd700 1 -- 172.21.15.158:0/2745735403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00f510 con 0x7f791c154090 2023-05-16T07:02:10.831 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791affd700 1 -- 172.21.15.158:0/2745735403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c00f670 con 0x7f791c154090 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.806+0000 7f791affd700 1 -- 172.21.15.158:0/2745735403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00f9f0 con 0x7f791c154090 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 -- 172.21.15.158:0/2745735403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154090 msgr2=0x7f791c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 --2- 172.21.15.158:0/2745735403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154090 0x7f791c0aaca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f790c009760 tx=0x7f790c009060).stop 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 -- 172.21.15.158:0/2745735403 shutdown_connections 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 --2- 172.21.15.158:0/2745735403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154090 0x7f791c0aaca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 -- 172.21.15.158:0/2745735403 >> 172.21.15.158:0/2745735403 conn(0x7f791c067920 msgr2=0x7f791c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 -- 172.21.15.158:0/2745735403 shutdown_connections 2023-05-16T07:02:10.832 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.807+0000 7f79221f5700 1 -- 172.21.15.158:0/2745735403 wait complete. 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.808+0000 7f79221f5700 1 Processor -- start 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.808+0000 7f79221f5700 1 -- start start 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 0x7f791c164910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f791c154090 0x7f791c14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154470 0x7f791c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c151c50 con 0x7f791c154470 2023-05-16T07:02:10.833 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c39c0 con 0x7f791c068160 2023-05-16T07:02:10.834 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c164e00 con 0x7f791c154090 2023-05-16T07:02:10.834 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 0x7f791c164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.834 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 0x7f791c164910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53890/0 (socket says 172.21.15.158:53890) 2023-05-16T07:02:10.834 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 -- 172.21.15.158:0/1244055400 learned_addr learned my addr 172.21.15.158:0/1244055400 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:10.834 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f7912ffd700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f791c154090 0x7f791c14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791bfff700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154470 0x7f791c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 -- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f791c154090 msgr2=0x7f791c14eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f791c154090 0x7f791c14eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 -- 172.21.15.158:0/1244055400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154470 msgr2=0x7f791c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154470 0x7f791c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 -- 172.21.15.158:0/1244055400 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c0092a0 con 0x7f791c068160 2023-05-16T07:02:10.835 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f7912ffd700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f791c154090 0x7f791c14eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:10.836 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f791b7fe700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 0x7f791c164910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f790c009e00 tx=0x7f790c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:10.836 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00f7e0 con 0x7f791c068160 2023-05-16T07:02:10.836 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c00f940 con 0x7f791c068160 2023-05-16T07:02:10.836 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c153550 con 0x7f791c068160 2023-05-16T07:02:10.836 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c01b660 con 0x7f791c068160 2023-05-16T07:02:10.837 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.809+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c0c3370 con 0x7f791c068160 2023-05-16T07:02:10.837 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.810+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f790c02b070 con 0x7f791c068160 2023-05-16T07:02:10.837 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.810+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f790c024a70 con 0x7f791c068160 2023-05-16T07:02:10.837 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.810+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c066730 con 0x7f791c068160 2023-05-16T07:02:10.837 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.812+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f790c029060 con 0x7f791c068160 2023-05-16T07:02:10.896 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.892+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f3047d3d-d21e-4666-a4b1-72e17b0cd879", "id": 6} v 0) v1 -- 0x7f791c1519c0 con 0x7f791c068160 2023-05-16T07:02:10.903 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.900+0000 7f7913fff700 1 -- 172.21.15.158:0/1244055400 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f3047d3d-d21e-4666-a4b1-72e17b0cd879", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f790c01f070 con 0x7f791c068160 2023-05-16T07:02:10.904 INFO:teuthology.orchestra.run.smithi158.stdout:6 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 msgr2=0x7f791c164910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 0x7f791c164910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f790c009e00 tx=0x7f790c017000).stop 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 shutdown_connections 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f791c154470 0x7f791c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f791c068160 0x7f791c164910 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 --2- 172.21.15.158:0/1244055400 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f791c154090 0x7f791c14eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:10.905 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 >> 172.21.15.158:0/1244055400 conn(0x7f791c067920 msgr2=0x7f791c0ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:10.906 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 shutdown_connections 2023-05-16T07:02:10.906 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:10.901+0000 7f79221f5700 1 -- 172.21.15.158:0/1244055400 wait complete. 2023-05-16T07:02:10.909 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:10.905+0000 7fc83a07b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:02:10.917 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new 09acff50-ea9d-4909-b83f-269422145185 7 2023-05-16T07:02:11.016 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:11.012+0000 7fc83a07b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:02:11.029 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.025+0000 7f96ed4f7700 1 Processor -- start 2023-05-16T07:02:11.029 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.025+0000 7f96ed4f7700 1 -- start start 2023-05-16T07:02:11.031 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96ed4f7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.032 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96ed4f7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80c94d0 con 0x7f96e8154090 2023-05-16T07:02:11.032 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96ed4f7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80cb7e0 con 0x7f96e8068160 2023-05-16T07:02:11.032 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96ed4f7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80c88c0 con 0x7f96e8154470 2023-05-16T07:02:11.032 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e67fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.032 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e77fe700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f96e8068160 0x7f96e80c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:44950/0 (socket says 172.21.15.158:44950) 2023-05-16T07:02:11.032 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e67fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41166/0 (socket says 172.21.15.158:41166) 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e77fe700 1 -- 172.21.15.158:0/38977967 learned_addr learned my addr 172.21.15.158:0/38977967 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e5ffb700 1 -- 172.21.15.158:0/38977967 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459413095 0 0) 0x7f96e80cb7e0 con 0x7f96e8068160 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e5ffb700 1 -- 172.21.15.158:0/38977967 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d40033f0 con 0x7f96e8068160 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e67fc700 1 -- 172.21.15.158:0/38977967 >> v1:172.21.15.158:6789/0 conn(0x7f96e8068160 legacy=0x7f96e80c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e67fc700 1 -- 172.21.15.158:0/38977967 >> v1:172.21.15.92:6789/0 conn(0x7f96e8154090 legacy=0x7f96e80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e67fc700 1 -- 172.21.15.158:0/38977967 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80c8ba0 con 0x7f96e8154470 2023-05-16T07:02:11.033 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e67fc700 1 --2- 172.21.15.158:0/38977967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80aaca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f96d0009790 tx=0x7f96d0009060).ready entity=mon.2 client_cookie=82f492f301e7251d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:11.034 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.027+0000 7f96e5ffb700 1 -- 172.21.15.158:0/38977967 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000f510 con 0x7f96e8154470 2023-05-16T07:02:11.034 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96e5ffb700 1 -- 172.21.15.158:0/38977967 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d000f670 con 0x7f96e8154470 2023-05-16T07:02:11.034 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96e5ffb700 1 -- 172.21.15.158:0/38977967 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000f9f0 con 0x7f96e8154470 2023-05-16T07:02:11.036 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 -- 172.21.15.158:0/38977967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 msgr2=0x7f96e80aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.036 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 --2- 172.21.15.158:0/38977967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80aaca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f96d0009790 tx=0x7f96d0009060).stop 2023-05-16T07:02:11.037 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 -- 172.21.15.158:0/38977967 shutdown_connections 2023-05-16T07:02:11.037 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 --2- 172.21.15.158:0/38977967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80aaca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.037 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 -- 172.21.15.158:0/38977967 >> 172.21.15.158:0/38977967 conn(0x7f96e8067920 msgr2=0x7f96e80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:11.037 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 -- 172.21.15.158:0/38977967 shutdown_connections 2023-05-16T07:02:11.037 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.028+0000 7f96ed4f7700 1 -- 172.21.15.158:0/38977967 wait complete. 2023-05-16T07:02:11.037 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.029+0000 7f96ed4f7700 1 Processor -- start 2023-05-16T07:02:11.038 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.029+0000 7f96ed4f7700 1 -- start start 2023-05-16T07:02:11.038 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96ed4f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8068160 0x7f96e80b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.038 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96ed4f7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 0x7f96e814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.038 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96ed4f7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.038 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96ed4f7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8164fa0 con 0x7f96e8068160 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96ed4f7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8154f40 con 0x7f96e8154090 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96ed4f7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80b0b50 con 0x7f96e8154470 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e67fc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 0x7f96e814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e77fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e67fc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 0x7f96e814ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:53904/0 (socket says 172.21.15.158:53904) 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e67fc700 1 -- 172.21.15.158:0/289258563 learned_addr learned my addr 172.21.15.158:0/289258563 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e6ffd700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8068160 0x7f96e80b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.039 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e77fe700 1 -- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 msgr2=0x7f96e814ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e77fe700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 0x7f96e814ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.030+0000 7f96e77fe700 1 -- 172.21.15.158:0/289258563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8068160 msgr2=0x7f96e80b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96e77fe700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8068160 0x7f96e80b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96e77fe700 1 -- 172.21.15.158:0/289258563 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00092a0 con 0x7f96e8154470 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96e67fc700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 0x7f96e814ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96e77fe700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80adbe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f96e000c790 tx=0x7f96e0007f10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:11.040 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0005d30 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0005e90 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e81651b0 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00072e0 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e81548f0 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f96e0010070 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.031+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f96e001c3f0 con 0x7f96e8154470 2023-05-16T07:02:11.041 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.032+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e8066730 con 0x7f96e8154470 2023-05-16T07:02:11.042 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.034+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f96e00137c0 con 0x7f96e8154470 2023-05-16T07:02:11.117 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.114+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "osd new", "uuid": "09acff50-ea9d-4909-b83f-269422145185", "id": 7} v 0) v1 -- 0x7f96e8155080 con 0x7f96e8154470 2023-05-16T07:02:11.121 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.118+0000 7f96deffd700 1 -- 172.21.15.158:0/289258563 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09acff50-ea9d-4909-b83f-269422145185", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f96e0013960 con 0x7f96e8154470 2023-05-16T07:02:11.122 INFO:teuthology.orchestra.run.smithi158.stdout:7 2023-05-16T07:02:11.123 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 msgr2=0x7f96e80adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.123 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80adbe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f96e000c790 tx=0x7f96e0007f10).stop 2023-05-16T07:02:11.123 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 shutdown_connections 2023-05-16T07:02:11.123 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f96e8154470 0x7f96e80adbe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.123 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96e8068160 0x7f96e80b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.123 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 --2- 172.21.15.158:0/289258563 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96e8154090 0x7f96e814ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.124 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 >> 172.21.15.158:0/289258563 conn(0x7f96e8067920 msgr2=0x7f96e80c1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:11.124 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 shutdown_connections 2023-05-16T07:02:11.124 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:11.119+0000 7f96ed4f7700 1 -- 172.21.15.158:0/289258563 wait complete. 2023-05-16T07:02:11.135 INFO:tasks.ceph.osd.0:Restarting daemon 2023-05-16T07:02:11.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-05-16T07:02:11.138 INFO:tasks.ceph.osd.0:Started 2023-05-16T07:02:11.138 INFO:tasks.ceph.osd.1:Restarting daemon 2023-05-16T07:02:11.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-05-16T07:02:11.140 INFO:tasks.ceph.osd.1:Started 2023-05-16T07:02:11.140 INFO:tasks.ceph.osd.2:Restarting daemon 2023-05-16T07:02:11.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-05-16T07:02:11.143 INFO:tasks.ceph.osd.2:Started 2023-05-16T07:02:11.143 INFO:tasks.ceph.osd.3:Restarting daemon 2023-05-16T07:02:11.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-05-16T07:02:11.145 INFO:tasks.ceph.osd.3:Started 2023-05-16T07:02:11.146 INFO:tasks.ceph.osd.4:Restarting daemon 2023-05-16T07:02:11.146 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-05-16T07:02:11.148 INFO:tasks.ceph.osd.4:Started 2023-05-16T07:02:11.148 INFO:tasks.ceph.osd.5:Restarting daemon 2023-05-16T07:02:11.149 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-05-16T07:02:11.150 INFO:tasks.ceph.osd.5:Started 2023-05-16T07:02:11.151 INFO:tasks.ceph.osd.6:Restarting daemon 2023-05-16T07:02:11.151 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-05-16T07:02:11.153 INFO:tasks.ceph.osd.6:Started 2023-05-16T07:02:11.153 INFO:tasks.ceph.osd.7:Restarting daemon 2023-05-16T07:02:11.154 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-05-16T07:02:11.156 INFO:tasks.ceph.osd.7:Started 2023-05-16T07:02:11.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:11.173 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:11.169+0000 7f14d02be680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:02:11.182 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:11.179+0000 7fc83a07b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:02:11.268 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:11.264+0000 7fc83a07b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:02:11.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.287+0000 7fb01fcc0700 1 Processor -- start 2023-05-16T07:02:11.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.288+0000 7fb01fcc0700 1 -- start start 2023-05-16T07:02:11.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01fcc0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb018154550 0x7fb0180696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01fcc0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb018069b90 0x7fb0180c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01fcc0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0180c0660 con 0x7fb018154170 2023-05-16T07:02:11.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01fcc0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb018069fc0 con 0x7fb018154550 2023-05-16T07:02:11.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01fcc0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0180bfd30 con 0x7fb018069b90 2023-05-16T07:02:11.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb018154550 0x7fb0180696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb018154550 0x7fb0180696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39958/0 (socket says 172.21.15.92:39958) 2023-05-16T07:02:11.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 -- 172.21.15.92:0/2858873049 learned_addr learned my addr 172.21.15.92:0/2858873049 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01da5c700 1 --1- 172.21.15.92:0/2858873049 >> v1:172.21.15.92:6789/0 conn(0x7fb018154170 0x7fb0180c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50704/0 (socket says 172.21.15.92:50704) 2023-05-16T07:02:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01e25d700 1 --2- 172.21.15.92:0/2858873049 >> v2:172.21.15.158:3300/0 conn(0x7fb018069b90 0x7fb0180c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 -- 172.21.15.92:0/2858873049 >> v2:172.21.15.158:3300/0 conn(0x7fb018069b90 msgr2=0x7fb0180c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 --2- 172.21.15.92:0/2858873049 >> v2:172.21.15.158:3300/0 conn(0x7fb018069b90 0x7fb0180c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 -- 172.21.15.92:0/2858873049 >> v1:172.21.15.92:6789/0 conn(0x7fb018154170 legacy=0x7fb0180c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01ca5a700 1 -- 172.21.15.92:0/2858873049 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716750664 0 0) 0x7fb0180c0660 con 0x7fb018154170 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 -- 172.21.15.92:0/2858873049 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180c0010 con 0x7fb018154550 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01d25b700 1 --2- 172.21.15.92:0/2858873049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018154550 0x7fb0180696a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb0180c5210 tx=0x7fb00c0090f0).ready entity=mon.0 client_cookie=efed2f6739d53103 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01ca5a700 1 -- 172.21.15.92:0/2858873049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00b520 con 0x7fb018154550 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01ca5a700 1 -- 172.21.15.92:0/2858873049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00b680 con 0x7fb018154550 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.289+0000 7fb01ca5a700 1 -- 172.21.15.92:0/2858873049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00ba00 con 0x7fb018154550 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 -- 172.21.15.92:0/2858873049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018154550 msgr2=0x7fb0180696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/2858873049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018154550 0x7fb0180696a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb0180c5210 tx=0x7fb00c0090f0).stop 2023-05-16T07:02:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 -- 172.21.15.92:0/2858873049 shutdown_connections 2023-05-16T07:02:11.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/2858873049 >> v2:172.21.15.158:3300/0 conn(0x7fb018069b90 0x7fb0180c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/2858873049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018154550 0x7fb0180696a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 -- 172.21.15.92:0/2858873049 >> 172.21.15.92:0/2858873049 conn(0x7fb0180cb750 msgr2=0x7fb0180c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:11.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 -- 172.21.15.92:0/2858873049 shutdown_connections 2023-05-16T07:02:11.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.290+0000 7fb01fcc0700 1 -- 172.21.15.92:0/2858873049 wait complete. 2023-05-16T07:02:11.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.291+0000 7fb01fcc0700 1 Processor -- start 2023-05-16T07:02:11.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.291+0000 7fb01fcc0700 1 -- start start 2023-05-16T07:02:11.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 0x7fb018130090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb018154170 0x7fb018156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb018154550 0x7fb018157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018157510 con 0x7fb018069b90 2023-05-16T07:02:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018143eb0 con 0x7fb018154550 2023-05-16T07:02:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018144460 con 0x7fb018154170 2023-05-16T07:02:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 0x7fb018130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 0x7fb018130090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39962/0 (socket says 172.21.15.92:39962) 2023-05-16T07:02:11.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 -- 172.21.15.92:0/3933346939 learned_addr learned my addr 172.21.15.92:0/3933346939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:11.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01e25d700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb018154550 0x7fb018157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01d25b700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb018154170 0x7fb018156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:11.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 -- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb018154170 msgr2=0x7fb018156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb018154170 0x7fb018156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 -- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb018154550 msgr2=0x7fb018157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb018154550 0x7fb018157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 -- 172.21.15.92:0/3933346939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c009370 con 0x7fb018069b90 2023-05-16T07:02:11.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01d25b700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb018154170 0x7fb018156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:11.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01da5c700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 0x7fb018130090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb01400c560 tx=0x7fb014007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:11.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140072e0 con 0x7fb018069b90 2023-05-16T07:02:11.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014007440 con 0x7fb018069b90 2023-05-16T07:02:11.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb018159880 con 0x7fb018069b90 2023-05-16T07:02:11.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.292+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140058f0 con 0x7fb018069b90 2023-05-16T07:02:11.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.293+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fb01401a060 con 0x7fb018069b90 2023-05-16T07:02:11.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.293+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018130580 con 0x7fb018069b90 2023-05-16T07:02:11.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.293+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7fb01401d440 con 0x7fb018069b90 2023-05-16T07:02:11.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.293+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0180666c0 con 0x7fb018069b90 2023-05-16T07:02:11.309 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:11.294+0000 7f14d02be680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:02:11.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.296+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fb014018070 con 0x7fb018069b90 2023-05-16T07:02:11.338 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:11.334+0000 7fc83a07b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:02:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.362+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb018159c70 con 0x7fb018069b90 2023-05-16T07:02:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.363+0000 7fb009ffb700 1 -- 172.21.15.92:0/3933346939 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7fb0140137c0 con 0x7fb018069b90 2023-05-16T07:02:11.367 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:11.368 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:11.115703+0000","last_up_change":"0.000000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 msgr2=0x7fb018130090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 0x7fb018130090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb01400c560 tx=0x7fb014007c90).stop 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 shutdown_connections 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb018154550 0x7fb018157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb018069b90 0x7fb018130090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 --2- 172.21.15.92:0/3933346939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb018154170 0x7fb018156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 >> 172.21.15.92:0/3933346939 conn(0x7fb0180cb750 msgr2=0x7fb0180ac6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 shutdown_connections 2023-05-16T07:02:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:11.364+0000 7fb01fcc0700 1 -- 172.21.15.92:0/3933346939 wait complete. 2023-05-16T07:02:11.380 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-05-16T07:02:11.381 INFO:tasks.ceph:Waiting for OSDs to come up 2023-05-16T07:02:11.453 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:11.449+0000 7f14d02be680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:02:11.530 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:11.526+0000 7f14d02be680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:02:11.598 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:11.594+0000 7f14d02be680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:02:11.600 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:11.596+0000 7fc83a07b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:02:11.751 INFO:tasks.ceph.osd.4.smithi158.stderr:2023-05-16T07:02:11.747+0000 7fb43fd0f200 -1 Falling back to public interface 2023-05-16T07:02:11.756 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-05-16T07:02:11.752+0000 7f6ae4b8d200 -1 Falling back to public interface 2023-05-16T07:02:11.762 INFO:tasks.ceph.osd.7.smithi158.stderr:2023-05-16T07:02:11.758+0000 7fbf8b09d200 -1 Falling back to public interface 2023-05-16T07:02:11.764 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-05-16T07:02:11.760+0000 7f8bff7e3200 -1 Falling back to public interface 2023-05-16T07:02:11.765 INFO:tasks.ceph.osd.5.smithi158.stderr:2023-05-16T07:02:11.762+0000 7fa4ba634200 -1 Falling back to public interface 2023-05-16T07:02:11.769 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-05-16T07:02:11.765+0000 7faf5e180200 -1 Falling back to public interface 2023-05-16T07:02:11.770 INFO:tasks.ceph.osd.6.smithi158.stderr:2023-05-16T07:02:11.766+0000 7fa40dc90200 -1 Falling back to public interface 2023-05-16T07:02:11.770 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-05-16T07:02:11.766+0000 7f1e0b6b0200 -1 Falling back to public interface 2023-05-16T07:02:11.802 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:11.798+0000 7fc83a07b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:02:11.860 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:11.856+0000 7f14d02be680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:02:12.052 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:12.048+0000 7f14d02be680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:02:12.187 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:12.291 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.288+0000 7ff155ea9700 1 Processor -- start 2023-05-16T07:02:12.292 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.288+0000 7ff155ea9700 1 -- start start 2023-05-16T07:02:12.294 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.288+0000 7ff155ea9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff150154170 0x7ff1500c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:12.294 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff155ea9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1500c0660 con 0x7ff150069db0 2023-05-16T07:02:12.295 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff155ea9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1500bfd30 con 0x7ff150154550 2023-05-16T07:02:12.295 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff155ea9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1500c0010 con 0x7ff150154170 2023-05-16T07:02:12.295 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff150154170 0x7ff1500c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:12.295 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff150069db0 0x7ff1500c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50720/0 (socket says 172.21.15.92:50720) 2023-05-16T07:02:12.295 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14ffff700 1 -- 172.21.15.92:0/2340502832 learned_addr learned my addr 172.21.15.92:0/2340502832 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:12.295 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2340502832 >> v1:172.21.15.158:6789/0 conn(0x7ff150154550 legacy=0x7ff1500696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:12.296 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2340502832 >> v1:172.21.15.92:6789/0 conn(0x7ff150069db0 legacy=0x7ff1500c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:12.296 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14e7fc700 1 -- 172.21.15.92:0/2340502832 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898864663 0 0) 0x7ff1500c0660 con 0x7ff150069db0 2023-05-16T07:02:12.296 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2340502832 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500c02f0 con 0x7ff150154170 2023-05-16T07:02:12.296 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14f7fe700 1 --2- 172.21.15.92:0/2340502832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150154170 0x7ff1500c5c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff140009760 tx=0x7ff140009060).ready entity=mon.0 client_cookie=f3d4aee550fdcb16 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:12.297 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14e7fc700 1 -- 172.21.15.92:0/2340502832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000b580 con 0x7ff150154170 2023-05-16T07:02:12.297 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14e7fc700 1 -- 172.21.15.92:0/2340502832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14000b6e0 con 0x7ff150154170 2023-05-16T07:02:12.297 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.289+0000 7ff14e7fc700 1 -- 172.21.15.92:0/2340502832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000ba60 con 0x7ff150154170 2023-05-16T07:02:12.297 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 -- 172.21.15.92:0/2340502832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150154170 msgr2=0x7ff1500c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:12.297 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 --2- 172.21.15.92:0/2340502832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150154170 0x7ff1500c5c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff140009760 tx=0x7ff140009060).stop 2023-05-16T07:02:12.298 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 -- 172.21.15.92:0/2340502832 shutdown_connections 2023-05-16T07:02:12.298 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 --2- 172.21.15.92:0/2340502832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150154170 0x7ff1500c5c70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:12.298 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 -- 172.21.15.92:0/2340502832 >> 172.21.15.92:0/2340502832 conn(0x7ff1500cb750 msgr2=0x7ff1500c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:12.298 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 -- 172.21.15.92:0/2340502832 shutdown_connections 2023-05-16T07:02:12.298 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.290+0000 7ff155ea9700 1 -- 172.21.15.92:0/2340502832 wait complete. 2023-05-16T07:02:12.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.291+0000 7ff155ea9700 1 Processor -- start 2023-05-16T07:02:12.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.291+0000 7ff155ea9700 1 -- start start 2023-05-16T07:02:12.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 0x7ff1500ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:12.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff150154170 0x7ff150151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:12.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff150154550 0x7ff150158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:12.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500c3370 con 0x7ff150069db0 2023-05-16T07:02:12.302 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500c3060 con 0x7ff150154170 2023-05-16T07:02:12.302 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500aeeb0 con 0x7ff150154550 2023-05-16T07:02:12.302 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 0x7ff1500ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:12.302 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 0x7ff1500ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39974/0 (socket says 172.21.15.92:39974) 2023-05-16T07:02:12.302 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2306119207 learned_addr learned my addr 172.21.15.92:0/2306119207 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:12.302 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14effd700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff150154170 0x7ff150151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:12.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14ffff700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff150154550 0x7ff150158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:12.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff150154550 msgr2=0x7ff150158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:12.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff150154550 0x7ff150158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:12.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff150154170 msgr2=0x7ff150151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff150154170 0x7ff150151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 -- 172.21.15.92:0/2306119207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400092a0 con 0x7ff150069db0 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14effd700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff150154170 0x7ff150151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff14f7fe700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 0x7ff1500ae620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff14000b580 tx=0x7ff140017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000ba60 con 0x7ff150069db0 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140000c90 con 0x7ff150069db0 2023-05-16T07:02:12.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1500af190 con 0x7ff150069db0 2023-05-16T07:02:12.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.292+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14001b640 con 0x7ff150069db0 2023-05-16T07:02:12.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.293+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1500aeb10 con 0x7ff150069db0 2023-05-16T07:02:12.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.294+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1500af6b0 con 0x7ff150069db0 2023-05-16T07:02:12.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.294+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7ff14000f070 con 0x7ff150069db0 2023-05-16T07:02:12.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.294+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7ff140023b00 con 0x7ff150069db0 2023-05-16T07:02:12.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.297+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ff1500af6b0 con 0x7ff150069db0 2023-05-16T07:02:12.367 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.364+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1500666c0 con 0x7ff150069db0 2023-05-16T07:02:12.368 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff1377fe700 1 -- 172.21.15.92:0/2306119207 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7ff140033090 con 0x7ff150069db0 2023-05-16T07:02:12.369 INFO:teuthology.misc.health.smithi092.stdout: 2023-05-16T07:02:12.369 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:11.115703+0000","last_up_change":"0.000000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:12.370 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 msgr2=0x7ff1500ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:12.370 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 0x7ff1500ae620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff14000b580 tx=0x7ff140017000).stop 2023-05-16T07:02:12.370 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 shutdown_connections 2023-05-16T07:02:12.371 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff150154550 0x7ff150158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:12.371 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff150069db0 0x7ff1500ae620 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:12.371 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 --2- 172.21.15.92:0/2306119207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff150154170 0x7ff150151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:12.372 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.365+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 >> 172.21.15.92:0/2306119207 conn(0x7ff1500cb750 msgr2=0x7ff1500a9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:12.372 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.366+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 shutdown_connections 2023-05-16T07:02:12.372 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:12.366+0000 7ff155ea9700 1 -- 172.21.15.92:0/2306119207 wait complete. 2023-05-16T07:02:12.380 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-05-16T07:02:12.461 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:12.457+0000 7fc83a07b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:02:12.528 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:12.524+0000 7fc83a07b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:02:12.605 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:12.602+0000 7fc83a07b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:02:12.724 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:12.720+0000 7f14d02be680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:02:12.753 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:12.749+0000 7fc83a07b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:02:12.791 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:12.787+0000 7f14d02be680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:02:12.823 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:12.819+0000 7fc83a07b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:02:12.869 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:12.865+0000 7f14d02be680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:02:12.939 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:12.936+0000 7fc83a07b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:02:13.015 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:13.011+0000 7f14d02be680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:02:13.043 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:13.040+0000 7fc83a07b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:02:13.084 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:13.081+0000 7f14d02be680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:02:13.202 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:13.198+0000 7f14d02be680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:02:13.307 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:13.303+0000 7f14d02be680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:02:13.362 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:13.358+0000 7fc83a07b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:02:13.433 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:13.429+0000 7fc83a07b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:02:13.626 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:13.622+0000 7f14d02be680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:02:13.698 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:02:13.694+0000 7f14d02be680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:02:14.458 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:14.454+0000 7fc7f02c7700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-05-16T07:02:14.458 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:02:14.454+0000 7fc7f02c7700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-05-16T07:02:15.252 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-05-16T07:02:15.249+0000 7f6ae4b8d200 -1 osd.3 0 log_to_monitors {default=true} 2023-05-16T07:02:15.255 INFO:tasks.ceph.osd.7.smithi158.stderr:2023-05-16T07:02:15.251+0000 7fbf8b09d200 -1 osd.7 0 log_to_monitors {default=true} 2023-05-16T07:02:15.289 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-05-16T07:02:15.285+0000 7f8bff7e3200 -1 osd.0 0 log_to_monitors {default=true} 2023-05-16T07:02:15.290 INFO:tasks.ceph.osd.4.smithi158.stderr:2023-05-16T07:02:15.286+0000 7fb43fd0f200 -1 osd.4 0 log_to_monitors {default=true} 2023-05-16T07:02:15.294 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-05-16T07:02:15.290+0000 7faf5e180200 -1 osd.2 0 log_to_monitors {default=true} 2023-05-16T07:02:15.299 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-05-16T07:02:15.296+0000 7f1e0b6b0200 -1 osd.1 0 log_to_monitors {default=true} 2023-05-16T07:02:15.300 INFO:tasks.ceph.osd.6.smithi158.stderr:2023-05-16T07:02:15.297+0000 7fa40dc90200 -1 osd.6 0 log_to_monitors {default=true} 2023-05-16T07:02:15.302 INFO:tasks.ceph.osd.5.smithi158.stderr:2023-05-16T07:02:15.298+0000 7fa4ba634200 -1 osd.5 0 log_to_monitors {default=true} 2023-05-16T07:02:16.724 INFO:tasks.ceph.osd.6.smithi158.stderr:2023-05-16T07:02:16.721+0000 7fa40464e700 -1 osd.6 0 waiting for initial osdmap 2023-05-16T07:02:16.725 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-05-16T07:02:16.720+0000 7faf54b3e700 -1 osd.2 0 waiting for initial osdmap 2023-05-16T07:02:16.725 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-05-16T07:02:16.720+0000 7f8bf61a1700 -1 osd.0 0 waiting for initial osdmap 2023-05-16T07:02:16.726 INFO:tasks.ceph.osd.4.smithi158.stderr:2023-05-16T07:02:16.722+0000 7fb4366cd700 -1 osd.4 0 waiting for initial osdmap 2023-05-16T07:02:16.726 INFO:tasks.ceph.osd.7.smithi158.stderr:2023-05-16T07:02:16.722+0000 7fbf81a5b700 -1 osd.7 0 waiting for initial osdmap 2023-05-16T07:02:16.726 INFO:tasks.ceph.osd.5.smithi158.stderr:2023-05-16T07:02:16.723+0000 7fa4b0ff2700 -1 osd.5 0 waiting for initial osdmap 2023-05-16T07:02:16.727 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-05-16T07:02:16.722+0000 7f1e0206e700 -1 osd.1 0 waiting for initial osdmap 2023-05-16T07:02:16.727 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-05-16T07:02:16.722+0000 7f6adb54b700 -1 osd.3 0 waiting for initial osdmap 2023-05-16T07:02:16.731 INFO:tasks.ceph.osd.6.smithi158.stderr:2023-05-16T07:02:16.727+0000 7fa400dcb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.731 INFO:tasks.ceph.osd.7.smithi158.stderr:2023-05-16T07:02:16.728+0000 7fbf7e1d8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.732 INFO:tasks.ceph.osd.4.smithi158.stderr:2023-05-16T07:02:16.728+0000 7fb432e4a700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.732 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-05-16T07:02:16.727+0000 7faf512bb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.734 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-05-16T07:02:16.730+0000 7f8bf291e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.735 INFO:tasks.ceph.osd.5.smithi158.stderr:2023-05-16T07:02:16.731+0000 7fa4ad76f700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.735 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-05-16T07:02:16.730+0000 7f1dfe7eb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:16.735 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-05-16T07:02:16.730+0000 7f6ad7cc8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-16T07:02:19.185 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:19.300 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.295+0000 7f624e03a700 1 Processor -- start 2023-05-16T07:02:19.301 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.295+0000 7f624e03a700 1 -- start start 2023-05-16T07:02:19.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f624e03a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6248154170 0x7f62480c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f624e03a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154550 0x7f62480c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f624e03a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62480696a0 0x7f6248069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f624e03a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62480c0660 con 0x7f62480696a0 2023-05-16T07:02:19.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f624e03a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624806a020 con 0x7f6248154170 2023-05-16T07:02:19.303 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f624e03a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480bfd30 con 0x7f6248154550 2023-05-16T07:02:19.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f6247fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62480696a0 0x7f6248069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f6247fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62480696a0 0x7f6248069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40138/0 (socket says 172.21.15.92:40138) 2023-05-16T07:02:19.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f6247fff700 1 -- 172.21.15.92:0/3921928534 learned_addr learned my addr 172.21.15.92:0/3921928534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:19.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f6246ffd700 1 --2- 172.21.15.92:0/3921928534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154550 0x7f62480c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.296+0000 7f62477fe700 1 --2- 172.21.15.92:0/3921928534 >> v2:172.21.15.158:3300/0 conn(0x7f6248154170 0x7f62480c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.304 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f6247fff700 1 -- 172.21.15.92:0/3921928534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154550 msgr2=0x7f62480c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f6247fff700 1 --2- 172.21.15.92:0/3921928534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154550 0x7f62480c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f6247fff700 1 -- 172.21.15.92:0/3921928534 >> v2:172.21.15.158:3300/0 conn(0x7f6248154170 msgr2=0x7f62480c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f6247fff700 1 --2- 172.21.15.92:0/3921928534 >> v2:172.21.15.158:3300/0 conn(0x7f6248154170 0x7f62480c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f6247fff700 1 -- 172.21.15.92:0/3921928534 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62480c0010 con 0x7f62480696a0 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f6247fff700 1 --2- 172.21.15.92:0/3921928534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62480696a0 0x7f6248069b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6248083030 tx=0x7f6240009840).ready entity=mon.0 client_cookie=b5a1f3a0607e5565 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f62467fc700 1 -- 172.21.15.92:0/3921928534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000e070 con 0x7f62480696a0 2023-05-16T07:02:19.305 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f62467fc700 1 -- 172.21.15.92:0/3921928534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624000ba50 con 0x7f62480696a0 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f62467fc700 1 -- 172.21.15.92:0/3921928534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000bdd0 con 0x7f62480696a0 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 -- 172.21.15.92:0/3921928534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62480696a0 msgr2=0x7f6248069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 --2- 172.21.15.92:0/3921928534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62480696a0 0x7f6248069b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6248083030 tx=0x7f6240009840).stop 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 -- 172.21.15.92:0/3921928534 shutdown_connections 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 --2- 172.21.15.92:0/3921928534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62480696a0 0x7f6248069b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 --2- 172.21.15.92:0/3921928534 >> v2:172.21.15.158:3300/0 conn(0x7f6248154170 0x7f62480c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 --2- 172.21.15.92:0/3921928534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154550 0x7f62480c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.306 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 -- 172.21.15.92:0/3921928534 >> 172.21.15.92:0/3921928534 conn(0x7f62480cb700 msgr2=0x7f62480c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:19.307 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.297+0000 7f624e03a700 1 -- 172.21.15.92:0/3921928534 shutdown_connections 2023-05-16T07:02:19.307 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.298+0000 7f624e03a700 1 -- 172.21.15.92:0/3921928534 wait complete. 2023-05-16T07:02:19.309 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.299+0000 7f624e03a700 1 Processor -- start 2023-05-16T07:02:19.309 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.299+0000 7f624e03a700 1 -- start start 2023-05-16T07:02:19.309 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62480696a0 0x7f624814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.309 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154170 0x7f62480aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.309 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 0x7f62481678e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.310 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c2150 con 0x7f6248154550 2023-05-16T07:02:19.310 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624814b380 con 0x7f62480696a0 2023-05-16T07:02:19.310 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624814b660 con 0x7f6248154170 2023-05-16T07:02:19.310 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 0x7f62481678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.310 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 0x7f62481678e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40146/0 (socket says 172.21.15.92:40146) 2023-05-16T07:02:19.310 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 -- 172.21.15.92:0/579234480 learned_addr learned my addr 172.21.15.92:0/579234480 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6246ffd700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154170 0x7f62480aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f62477fe700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62480696a0 0x7f624814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 -- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154170 msgr2=0x7f62480aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154170 0x7f62480aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 -- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62480696a0 msgr2=0x7f624814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62480696a0 0x7f624814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.311 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 -- 172.21.15.92:0/579234480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240009ae0 con 0x7f6248154550 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f6247fff700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 0x7f62481678e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6240009fd0 tx=0x7f6240009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62400092e0 con 0x7f6248154550 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624000bdd0 con 0x7f6248154550 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624814b940 con 0x7f6248154550 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.300+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624001d690 con 0x7f6248154550 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.301+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624814af70 con 0x7f6248154550 2023-05-16T07:02:19.312 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.301+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62480c1fa0 con 0x7f6248154550 2023-05-16T07:02:19.313 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.302+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6240024030 con 0x7f6248154550 2023-05-16T07:02:19.313 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.302+0000 7f622f7fe700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f623405ed90 0x7f6234061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.313 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.302+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f624007aad0 con 0x7f6248154550 2023-05-16T07:02:19.313 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.302+0000 7f62477fe700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f623405ed90 0x7f6234061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.313 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.303+0000 7f62477fe700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f623405ed90 0x7f6234061250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f623800bbc0 tx=0x7f6238008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:19.314 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.306+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f624004c5f0 con 0x7f6248154550 2023-05-16T07:02:19.477 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.473+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62480666c0 con 0x7f6248154550 2023-05-16T07:02:19.478 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.473+0000 7f622f7fe700 1 -- 172.21.15.92:0/579234480 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+12228 (secure 0 0 0) 0x7f6240022030 con 0x7f6248154550 2023-05-16T07:02:19.479 INFO:teuthology.misc.health.smithi092.stdout: 2023-05-16T07:02:19.479 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":14,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:18.719326+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:19.480 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f623405ed90 msgr2=0x7f6234061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.480 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f623405ed90 0x7f6234061250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f623800bbc0 tx=0x7f6238008040).stop 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 msgr2=0x7f62481678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 0x7f62481678e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6240009fd0 tx=0x7f6240009840).stop 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 shutdown_connections 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f623405ed90 0x7f6234061250 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248154550 0x7f62481678e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62480696a0 0x7f624814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.481 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 --2- 172.21.15.92:0/579234480 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6248154170 0x7f62480aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.482 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 >> 172.21.15.92:0/579234480 conn(0x7f62480cb700 msgr2=0x7f62480aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:19.482 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 shutdown_connections 2023-05-16T07:02:19.482 INFO:teuthology.misc.health.smithi092.stderr:2023-05-16T07:02:19.475+0000 7f624e03a700 1 -- 172.21.15.92:0/579234480 wait complete. 2023-05-16T07:02:19.492 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-05-16T07:02:19.492 INFO:tasks.ceph:Creating RBD pool 2023-05-16T07:02:19.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2023-05-16T07:02:19.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.598+0000 7fceec31e700 1 Processor -- start 2023-05-16T07:02:19.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.598+0000 7fceec31e700 1 -- start start 2023-05-16T07:02:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceec31e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcee414e6f0 0x7fcee4155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceec31e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcee40622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceec31e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee414ead0 con 0x7fcee40627a0 2023-05-16T07:02:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceec31e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee4154cb0 con 0x7fcee414e6f0 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceec31e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4062c10 con 0x7fcee40c4d40 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceea8bb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcee40627a0 0x7fcee4063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50902/0 (socket says 172.21.15.92:50902) 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceea8bb700 1 -- 172.21.15.92:0/3159662651 learned_addr learned my addr 172.21.15.92:0/3159662651 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fcee98b9700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcee40622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fceea0ba700 1 --2- 172.21.15.92:0/3159662651 >> v2:172.21.15.158:3300/0 conn(0x7fcee414e6f0 0x7fcee4155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915941091 0 0) 0x7fcee414ead0 con 0x7fcee40627a0 2023-05-16T07:02:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.599+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcecc0033f0 con 0x7fcee40627a0 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161036816 0 0) 0x7fcecc0033f0 con 0x7fcee40627a0 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 msgr2=0x7fcee40622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcee40622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 >> v2:172.21.15.158:3300/0 conn(0x7fcee414e6f0 msgr2=0x7fcee4155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 --2- 172.21.15.92:0/3159662651 >> v2:172.21.15.158:3300/0 conn(0x7fcee414e6f0 0x7fcee4155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40c6020 con 0x7fcee40627a0 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fcedc001470 con 0x7fcee40627a0 2023-05-16T07:02:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 --2- 172.21.15.92:0/3159662651 >> v2:172.21.15.158:3300/0 conn(0x7fcee414e6f0 0x7fcee4155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee98b9700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcee40622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 >> v1:172.21.15.92:6789/0 conn(0x7fcee40627a0 legacy=0x7fcee4063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcecc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcecc004690 0x7fcecc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcecc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4154cb0 con 0x7fcecc0038e0 2023-05-16T07:02:19.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4062c10 con 0x7fcecc004690 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc00b8e0 con 0x7fcee40c4d40 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcecc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee98b9700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcecc004690 0x7fcecc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea8bb700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcecc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 -- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 msgr2=0x7fcecc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcecc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 -- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcecc004690 msgr2=0x7fcecc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcecc004690 0x7fcecc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 -- 172.21.15.92:0/3159662651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80092a0 con 0x7fcecc0038e0 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea8bb700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40c4d40 0x7fcecc005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fceea0ba700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcecc003d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fced400e3c0 tx=0x7fced400b040).ready entity=mon.0 client_cookie=5e8ffc4f03c05a9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4009c50 con 0x7fcecc0038e0 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced400f040 con 0x7fcecc0038e0 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.600+0000 7fcee90b8700 1 -- 172.21.15.92:0/3159662651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced400ee80 con 0x7fcecc0038e0 2023-05-16T07:02:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceec31e700 1 -- 172.21.15.92:0/3159662651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 msgr2=0x7fcecc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceec31e700 1 --2- 172.21.15.92:0/3159662651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcecc003d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fced400e3c0 tx=0x7fced400b040).stop 2023-05-16T07:02:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceea8bb700 1 -- 172.21.15.92:0/3159662651 reap_dead start 2023-05-16T07:02:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceec31e700 1 -- 172.21.15.92:0/3159662651 shutdown_connections 2023-05-16T07:02:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceec31e700 1 -- 172.21.15.92:0/3159662651 >> 172.21.15.92:0/3159662651 conn(0x7fcee405e820 msgr2=0x7fcee40c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceec31e700 1 -- 172.21.15.92:0/3159662651 shutdown_connections 2023-05-16T07:02:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.601+0000 7fceec31e700 1 -- 172.21.15.92:0/3159662651 wait complete. 2023-05-16T07:02:19.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.602+0000 7fceec31e700 1 Processor -- start 2023-05-16T07:02:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.602+0000 7fceec31e700 1 -- start start 2023-05-16T07:02:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceec31e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcee414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceec31e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40ad910 0x7fcee40add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceec31e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcee40ac410 0x7fcee40ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceec31e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40c0180 con 0x7fcecc0038e0 2023-05-16T07:02:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceec31e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ae280 con 0x7fcee40ac410 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceec31e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ad0f0 con 0x7fcee40ad910 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcee414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcee414c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40162/0 (socket says 172.21.15.92:40162) 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 -- 172.21.15.92:0/1938825326 learned_addr learned my addr 172.21.15.92:0/1938825326 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fcee98b9700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40ad910 0x7fcee40add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea8bb700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcee40ac410 0x7fcee40ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 -- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40ad910 msgr2=0x7fcee40add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40ad910 0x7fcee40add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 -- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcee40ac410 msgr2=0x7fcee40ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcee40ac410 0x7fcee40ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 -- 172.21.15.92:0/1938825326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40ad3d0 con 0x7fcecc0038e0 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea8bb700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcee40ac410 0x7fcee40ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.603+0000 7fceea0ba700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcee414c750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fced4001190 tx=0x7fced401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.604+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4013dd0 con 0x7fcecc0038e0 2023-05-16T07:02:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.604+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4023070 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.604+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee40bfb30 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.604+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced401c450 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.604+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee414cc40 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.604+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee406c350 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.605+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fced402f070 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.605+0000 7fcee27fc700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fced005ed60 0x7fced0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.605+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7fced407b890 con 0x7fcecc0038e0 2023-05-16T07:02:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.605+0000 7fcee98b9700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fced005ed60 0x7fced0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.606+0000 7fcee98b9700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fced005ed60 0x7fced0061220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fcee4068700 tx=0x7fced8009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.609+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fced404d300 con 0x7fcecc0038e0 2023-05-16T07:02:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:19.776+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fcee4064830 con 0x7fcecc0038e0 2023-05-16T07:02:20.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.769+0000 7fcee27fc700 1 -- 172.21.15.92:0/1938825326 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fced402d070 con 0x7fcecc0038e0 2023-05-16T07:02:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.771+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fced005ed60 msgr2=0x7fced0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fced005ed60 0x7fced0061220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fcee4068700 tx=0x7fced8009060).stop 2023-05-16T07:02:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 msgr2=0x7fcee414c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcee414c750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fced4001190 tx=0x7fced401a000).stop 2023-05-16T07:02:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 shutdown_connections 2023-05-16T07:02:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fced005ed60 0x7fced0061220 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcee40ac410 0x7fcee40ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcecc0038e0 0x7fcee414c750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 --2- 172.21.15.92:0/1938825326 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcee40ad910 0x7fcee40add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 >> 172.21.15.92:0/1938825326 conn(0x7fcee405e820 msgr2=0x7fcee4067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 shutdown_connections 2023-05-16T07:02:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.772+0000 7fceec31e700 1 -- 172.21.15.92:0/1938825326 wait complete. 2023-05-16T07:02:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2023-05-16T07:02:20.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-05-16T07:02:20.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 Processor -- start 2023-05-16T07:02:20.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 -- start start 2023-05-16T07:02:20.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f833413b420 0x7f83340be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f8334066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8334067340 0x7f83340692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83340622b0 con 0x7f8334067340 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8334063110 con 0x7f833413b420 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f833ab97700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340677d0 con 0x7f83340be590 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.905+0000 7f8339134700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8334067340 0x7f83340692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8334067340 0x7f83340692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40178/0 (socket says 172.21.15.92:40178) 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 -- 172.21.15.92:0/3057633500 learned_addr learned my addr 172.21.15.92:0/3057633500 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8338933700 1 --2- 172.21.15.92:0/3057633500 >> v2:172.21.15.158:3300/0 conn(0x7f833413b420 0x7f83340be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8333fff700 1 --2- 172.21.15.92:0/3057633500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f8334066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 -- 172.21.15.92:0/3057633500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 msgr2=0x7f8334066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 --2- 172.21.15.92:0/3057633500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f8334066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 -- 172.21.15.92:0/3057633500 >> v2:172.21.15.158:3300/0 conn(0x7f833413b420 msgr2=0x7f83340be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:20.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 --2- 172.21.15.92:0/3057633500 >> v2:172.21.15.158:3300/0 conn(0x7f833413b420 0x7f83340be0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 -- 172.21.15.92:0/3057633500 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334063600 con 0x7f8334067340 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f8339134700 1 --2- 172.21.15.92:0/3057633500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340692b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f83340c7870 tx=0x7f832c009840).ready entity=mon.0 client_cookie=f41701265442e26 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f83337fe700 1 -- 172.21.15.92:0/3057633500 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00e070 con 0x7f8334067340 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f83337fe700 1 -- 172.21.15.92:0/3057633500 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c00b970 con 0x7f8334067340 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f83337fe700 1 -- 172.21.15.92:0/3057633500 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00bcf0 con 0x7f8334067340 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f833ab97700 1 -- 172.21.15.92:0/3057633500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 msgr2=0x7f83340692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.906+0000 7f833ab97700 1 --2- 172.21.15.92:0/3057633500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340692b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f83340c7870 tx=0x7f832c009840).stop 2023-05-16T07:02:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 -- 172.21.15.92:0/3057633500 shutdown_connections 2023-05-16T07:02:20.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 --2- 172.21.15.92:0/3057633500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340692b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 --2- 172.21.15.92:0/3057633500 >> v2:172.21.15.158:3300/0 conn(0x7f833413b420 0x7f83340be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 --2- 172.21.15.92:0/3057633500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f8334066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 -- 172.21.15.92:0/3057633500 >> 172.21.15.92:0/3057633500 conn(0x7f83340c8470 msgr2=0x7f83340c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:20.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 -- 172.21.15.92:0/3057633500 shutdown_connections 2023-05-16T07:02:20.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.907+0000 7f833ab97700 1 -- 172.21.15.92:0/3057633500 wait complete. 2023-05-16T07:02:20.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.908+0000 7f833ab97700 1 Processor -- start 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.908+0000 7f833ab97700 1 -- start start 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f83340ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f833413b420 0x7f8334139500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340bf7c0 con 0x7f8334067340 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340bf490 con 0x7f833413b420 2023-05-16T07:02:20.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340abb20 con 0x7f83340be590 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40180/0 (socket says 172.21.15.92:40180) 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 -- 172.21.15.92:0/4078942988 learned_addr learned my addr 172.21.15.92:0/4078942988 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8333fff700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f83340ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 -- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 msgr2=0x7f83340ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8339134700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f833413b420 0x7f8334139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f83340ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 -- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f833413b420 msgr2=0x7f8334139500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f833413b420 0x7f8334139500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 -- 172.21.15.92:0/4078942988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009ae0 con 0x7f8334067340 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8339134700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f833413b420 0x7f8334139500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8338933700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340ad130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f832800e930 tx=0x7f8328009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800eed0 con 0x7f8334067340 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832800c850 con 0x7f8334067340 2023-05-16T07:02:20.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83340abe00 con 0x7f8334067340 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800cbd0 con 0x7f8334067340 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.909+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83340ab810 con 0x7f8334067340 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.910+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83340ac320 con 0x7f8334067340 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.910+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8328022070 con 0x7f8334067340 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.911+0000 7f8330ff9700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f831805ed90 0x7f8318061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.911+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5258+0+0 (secure 0 0 0) 0x7f8328079bd0 con 0x7f8334067340 2023-05-16T07:02:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.911+0000 7f8333fff700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f831805ed90 0x7f8318061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:20.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.911+0000 7f8333fff700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f831805ed90 0x7f8318061250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8334151f90 tx=0x7f8320008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:20.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:20.915+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f83340ac320 con 0x7f8334067340 2023-05-16T07:02:21.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.079+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f83340ac320 con 0x7f8334067340 2023-05-16T07:02:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.770+0000 7f8330ff9700 1 -- 172.21.15.92:0/4078942988 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f83340ac320 con 0x7f8334067340 2023-05-16T07:02:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f831805ed90 msgr2=0x7f8318061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f831805ed90 0x7f8318061250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8334151f90 tx=0x7f8320008040).stop 2023-05-16T07:02:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 msgr2=0x7f83340ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340ad130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f832800e930 tx=0x7f8328009060).stop 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 shutdown_connections 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f831805ed90 0x7f8318061250 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f833413b420 0x7f8334139500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8334067340 0x7f83340ad130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 --2- 172.21.15.92:0/4078942988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83340be590 0x7f83340ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 >> 172.21.15.92:0/4078942988 conn(0x7f83340c8470 msgr2=0x7f83340c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 shutdown_connections 2023-05-16T07:02:21.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.773+0000 7f833ab97700 1 -- 172.21.15.92:0/4078942988 wait complete. 2023-05-16T07:02:21.779 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2023-05-16T07:02:21.791 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-05-16T07:02:21.792 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:02:21.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:02:21.794 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:02:21.794 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:02:21.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:02:21.796 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:02:21.797 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:02:21.797 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:02:21.799 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:02:21.800 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:02:21.800 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:02:21.802 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:02:21.802 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-05-16T07:02:21.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:21.867 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:02:21.868 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:02:21.880 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:02:21.882 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:02:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.920+0000 7f524a57c700 1 Processor -- start 2023-05-16T07:02:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.920+0000 7f524a57c700 1 -- start start 2023-05-16T07:02:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f524a57c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5244151bd0 0x7f5244156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f524a57c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52440a76f0 0x7f52440a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f524a57c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52440c8750 con 0x7f52441517f0 2023-05-16T07:02:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f524a57c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5244155d70 con 0x7f5244151bd0 2023-05-16T07:02:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f524a57c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52440a8030 con 0x7f52440a76f0 2023-05-16T07:02:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5244151bd0 0x7f5244156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5244151bd0 0x7f5244156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40262/0 (socket says 172.21.15.92:40262) 2023-05-16T07:02:21.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 -- 172.21.15.92:0/2197397541 learned_addr learned my addr 172.21.15.92:0/2197397541 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:21.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5249d7b700 1 --2- 172.21.15.92:0/2197397541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52440a76f0 0x7f52440a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:21.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 -- 172.21.15.92:0/2197397541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52440a76f0 msgr2=0x7f52440a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 --2- 172.21.15.92:0/2197397541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52440a76f0 0x7f52440a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 -- 172.21.15.92:0/2197397541 >> v1:172.21.15.158:6789/0 conn(0x7f52441517f0 legacy=0x7f5244152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 -- 172.21.15.92:0/2197397541 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52440c8470 con 0x7f5244151bd0 2023-05-16T07:02:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.921+0000 7f5248d79700 1 --2- 172.21.15.92:0/2197397541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244151bd0 0x7f5244156200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5240006fd0 tx=0x7f52400090f0).ready entity=mon.0 client_cookie=abf3a371611310e1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f523bfff700 1 -- 172.21.15.92:0/2197397541 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524000b600 con 0x7f5244151bd0 2023-05-16T07:02:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f523bfff700 1 -- 172.21.15.92:0/2197397541 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524000b760 con 0x7f5244151bd0 2023-05-16T07:02:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f523bfff700 1 -- 172.21.15.92:0/2197397541 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524000bb00 con 0x7f5244151bd0 2023-05-16T07:02:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 -- 172.21.15.92:0/2197397541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244151bd0 msgr2=0x7f5244156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 --2- 172.21.15.92:0/2197397541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244151bd0 0x7f5244156200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5240006fd0 tx=0x7f52400090f0).stop 2023-05-16T07:02:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 -- 172.21.15.92:0/2197397541 shutdown_connections 2023-05-16T07:02:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 --2- 172.21.15.92:0/2197397541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52440a76f0 0x7f52440a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 --2- 172.21.15.92:0/2197397541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244151bd0 0x7f5244156200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 -- 172.21.15.92:0/2197397541 >> 172.21.15.92:0/2197397541 conn(0x7f524405e820 msgr2=0x7f52440622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 -- 172.21.15.92:0/2197397541 shutdown_connections 2023-05-16T07:02:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.922+0000 7f524a57c700 1 -- 172.21.15.92:0/2197397541 wait complete. 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.923+0000 7f524a57c700 1 Processor -- start 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.923+0000 7f524a57c700 1 -- start start 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524a57c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 0x7f524414af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524a57c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52441517f0 0x7f52440c2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524a57c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5244151bd0 0x7f52440c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524a57c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5244161b70 con 0x7f52440a76f0 2023-05-16T07:02:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524a57c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52441629b0 con 0x7f52441517f0 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524a57c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5244161da0 con 0x7f5244151bd0 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 0x7f524414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 0x7f524414af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40264/0 (socket says 172.21.15.92:40264) 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 -- 172.21.15.92:0/1435896652 learned_addr learned my addr 172.21.15.92:0/1435896652 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f5248d79700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52441517f0 0x7f52440c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f5249d7b700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5244151bd0 0x7f52440c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 -- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5244151bd0 msgr2=0x7f52440c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5244151bd0 0x7f52440c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 -- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52441517f0 msgr2=0x7f52440c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52441517f0 0x7f52440c2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.924+0000 7f524957a700 1 -- 172.21.15.92:0/1435896652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240009370 con 0x7f52440a76f0 2023-05-16T07:02:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.925+0000 7f524957a700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 0x7f524414af60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f523400c4c0 tx=0x7f5234007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.925+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400ce70 con 0x7f52440a76f0 2023-05-16T07:02:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.925+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234007020 con 0x7f52440a76f0 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.925+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340073e0 con 0x7f52440a76f0 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.925+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5244162080 con 0x7f52440a76f0 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.925+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5244162640 con 0x7f52440a76f0 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.926+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5244047150 con 0x7f52440a76f0 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.926+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5234010070 con 0x7f52440a76f0 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.926+0000 7f52397fa700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f522805ed90 0x7f5228061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.926+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5234078f60 con 0x7f52440a76f0 2023-05-16T07:02:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.926+0000 7f5248d79700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f522805ed90 0x7f5228061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.927+0000 7f5248d79700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f522805ed90 0x7f5228061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f524000b3c0 tx=0x7f5240017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:21.930+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f523404a7f0 con 0x7f52440a76f0 2023-05-16T07:02:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.091+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5244155720 con 0x7f52440a76f0 2023-05-16T07:02:22.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.092+0000 7f52397fa700 1 -- 172.21.15.92:0/1435896652 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13770 (secure 0 0 0) 0x7f523404de10 con 0x7f52440a76f0 2023-05-16T07:02:22.096 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:22.097 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:21.766139+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f522805ed90 msgr2=0x7f5228061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f522805ed90 0x7f5228061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f524000b3c0 tx=0x7f5240017000).stop 2023-05-16T07:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 msgr2=0x7f524414af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 0x7f524414af60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f523400c4c0 tx=0x7f5234007df0).stop 2023-05-16T07:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 shutdown_connections 2023-05-16T07:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f522805ed90 0x7f5228061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5244151bd0 0x7f52440c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52440a76f0 0x7f524414af60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 --2- 172.21.15.92:0/1435896652 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52441517f0 0x7f52440c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 >> 172.21.15.92:0/1435896652 conn(0x7f524405e820 msgr2=0x7f52441544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 shutdown_connections 2023-05-16T07:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.094+0000 7f524a57c700 1 -- 172.21.15.92:0/1435896652 wait complete. 2023-05-16T07:02:22.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:02:22.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.219+0000 7f05a14a1700 1 Processor -- start 2023-05-16T07:02:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.219+0000 7f05a14a1700 1 -- start start 2023-05-16T07:02:22.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f05a14a1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f05a14a1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c062c30 con 0x7f059c0627a0 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f05a14a1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c0bf620 con 0x7f059c1388f0 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f05a14a1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c138cd0 con 0x7f059c0c4ef0 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f059b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f059c0627a0 0x7f059c0a8690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51024/0 (socket says 172.21.15.92:51024) 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f059b7fe700 1 -- 172.21.15.92:0/3220729104 learned_addr learned my addr 172.21.15.92:0/3220729104 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.220+0000 7f059a7fc700 1 --2- 172.21.15.92:0/3220729104 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619789561 0 0) 0x7f059c062c30 con 0x7f059c0627a0 2023-05-16T07:02:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05880033f0 con 0x7f059c0627a0 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302428747 0 0) 0x7f059c0bf620 con 0x7f059c1388f0 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f059c062c30 con 0x7f059c1388f0 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983982975 0 0) 0x7f05880033f0 con 0x7f059c0627a0 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f059a7fc700 1 -- 172.21.15.92:0/3220729104 >> v1:172.21.15.158:6789/0 conn(0x7f059c1388f0 legacy=0x7f059c0bda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f059a7fc700 1 -- 172.21.15.92:0/3220729104 >> v1:172.21.15.92:6789/0 conn(0x7f059c0627a0 legacy=0x7f059c0a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f059a7fc700 1 -- 172.21.15.92:0/3220729104 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0c3f30 con 0x7f059c0c4ef0 2023-05-16T07:02:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f059a7fc700 1 --2- 172.21.15.92:0/3220729104 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0622b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f0584009790 tx=0x7f0584009060).ready entity=mon.2 client_cookie=90bbd9e6f6d091c7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594001470 con 0x7f059c0c4ef0 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0584011400 con 0x7f059c0c4ef0 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f0599ffb700 1 -- 172.21.15.92:0/3220729104 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584011780 con 0x7f059c0c4ef0 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f05a14a1700 1 -- 172.21.15.92:0/3220729104 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 msgr2=0x7f059c0622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.221+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3220729104 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0622b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f0584009790 tx=0x7f0584009060).stop 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.222+0000 7f05a14a1700 1 -- 172.21.15.92:0/3220729104 shutdown_connections 2023-05-16T07:02:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.222+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3220729104 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0622b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.222+0000 7f05a14a1700 1 -- 172.21.15.92:0/3220729104 >> 172.21.15.92:0/3220729104 conn(0x7f059c0c8ae0 msgr2=0x7f059c0c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.222+0000 7f05a14a1700 1 -- 172.21.15.92:0/3220729104 shutdown_connections 2023-05-16T07:02:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.222+0000 7f05a14a1700 1 -- 172.21.15.92:0/3220729104 wait complete. 2023-05-16T07:02:22.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.223+0000 7f05a14a1700 1 Processor -- start 2023-05-16T07:02:22.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.223+0000 7f05a14a1700 1 -- start start 2023-05-16T07:02:22.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.223+0000 7f05a14a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 0x7f059c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.223+0000 7f05a14a1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f05a14a1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f059c1388f0 0x7f059c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f05a14a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0aeb30 con 0x7f059c0627a0 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f05a14a1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c05c0 con 0x7f059c1388f0 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f05a14a1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c0290 con 0x7f059c0c4ef0 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 0x7f059c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 0x7f059c155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40278/0 (socket says 172.21.15.92:40278) 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 -- 172.21.15.92:0/3138934739 learned_addr learned my addr 172.21.15.92:0/3138934739 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059b7fe700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f059c1388f0 0x7f059c14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059a7fc700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 -- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 msgr2=0x7f059c0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 -- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f059c1388f0 msgr2=0x7f059c14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f059c1388f0 0x7f059c14dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 -- 172.21.15.92:0/3138934739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840092a0 con 0x7f059c0627a0 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059b7fe700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f059c1388f0 0x7f059c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f059affd700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 0x7f059c155010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f058c00c810 tx=0x7f058c0026f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00a5c0 con 0x7f059c0627a0 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00a720 con 0x7f059c0627a0 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0ae570 con 0x7f059c0627a0 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00aac0 con 0x7f059c0627a0 2023-05-16T07:02:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.224+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0ae1a0 con 0x7f059c0627a0 2023-05-16T07:02:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.225+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c0c00f0 con 0x7f059c0627a0 2023-05-16T07:02:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.225+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f058c010070 con 0x7f059c0627a0 2023-05-16T07:02:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.226+0000 7f0592ffd700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f057c05ed90 0x7f057c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.226+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f058c079670 con 0x7f059c0627a0 2023-05-16T07:02:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.226+0000 7f059a7fc700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f057c05ed90 0x7f057c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.226+0000 7f059a7fc700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f057c05ed90 0x7f057c061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f05840096e0 tx=0x7f0584016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.230+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f059c0c00f0 con 0x7f059c0627a0 2023-05-16T07:02:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.393+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f059c0c00f0 con 0x7f059c0627a0 2023-05-16T07:02:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.393+0000 7f0592ffd700 1 -- 172.21.15.92:0/3138934739 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f059c0c00f0 con 0x7f059c0627a0 2023-05-16T07:02:22.398 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f057c05ed90 msgr2=0x7f057c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f057c05ed90 0x7f057c061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f05840096e0 tx=0x7f0584016000).stop 2023-05-16T07:02:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 msgr2=0x7f059c155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 0x7f059c155010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f058c00c810 tx=0x7f058c0026f0).stop 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 shutdown_connections 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f057c05ed90 0x7f057c061250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f059c1388f0 0x7f059c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f059c0627a0 0x7f059c155010 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 --2- 172.21.15.92:0/3138934739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f059c0c4ef0 0x7f059c0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 >> 172.21.15.92:0/3138934739 conn(0x7f059c0c8ae0 msgr2=0x7f059c0bda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 shutdown_connections 2023-05-16T07:02:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.395+0000 7f05a14a1700 1 -- 172.21.15.92:0/3138934739 wait complete. 2023-05-16T07:02:22.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.520+0000 7feee712d700 1 Processor -- start 2023-05-16T07:02:22.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.520+0000 7feee712d700 1 -- start start 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee712d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee0155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee712d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feee00629c0 0x7feee0063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee712d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee014ead0 con 0x7feee00c4d40 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee712d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee0154cb0 con 0x7feee00629c0 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee712d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee00c6020 con 0x7feee014e6f0 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feee00629c0 0x7feee0063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feee00629c0 0x7feee0063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40288/0 (socket says 172.21.15.92:40288) 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 -- 172.21.15.92:0/4059244708 learned_addr learned my addr 172.21.15.92:0/4059244708 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feed7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7feee00c4d40 0x7feee00622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51038/0 (socket says 172.21.15.92:51038) 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee4ec9700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 -- 172.21.15.92:0/4059244708 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 msgr2=0x7feee0155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee0155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 -- 172.21.15.92:0/4059244708 >> v1:172.21.15.92:6789/0 conn(0x7feee00c4d40 legacy=0x7feee00622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feed77fe700 1 -- 172.21.15.92:0/4059244708 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388848299 0 0) 0x7feee014ead0 con 0x7feee00c4d40 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 -- 172.21.15.92:0/4059244708 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee0154660 con 0x7feee00629c0 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee4ec9700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee0155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feee56ca700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee0063600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7feed8006fd0 tx=0x7feed80090f0).ready entity=mon.0 client_cookie=c328bb8641c1aff server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feed77fe700 1 -- 172.21.15.92:0/4059244708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed800b600 con 0x7feee00629c0 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feed77fe700 1 -- 172.21.15.92:0/4059244708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed800b760 con 0x7feee00629c0 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.521+0000 7feed77fe700 1 -- 172.21.15.92:0/4059244708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed800bae0 con 0x7feee00629c0 2023-05-16T07:02:22.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 -- 172.21.15.92:0/4059244708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 msgr2=0x7feee0063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee0063600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7feed8006fd0 tx=0x7feed80090f0).stop 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 -- 172.21.15.92:0/4059244708 shutdown_connections 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee0155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 --2- 172.21.15.92:0/4059244708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee0063600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 -- 172.21.15.92:0/4059244708 >> 172.21.15.92:0/4059244708 conn(0x7feee005e820 msgr2=0x7feee00c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 -- 172.21.15.92:0/4059244708 shutdown_connections 2023-05-16T07:02:22.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.522+0000 7feee712d700 1 -- 172.21.15.92:0/4059244708 wait complete. 2023-05-16T07:02:22.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.523+0000 7feee712d700 1 Processor -- start 2023-05-16T07:02:22.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.523+0000 7feee712d700 1 -- start start 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feee00c4d40 0x7feee00ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee00c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee014ce20 con 0x7feee00629c0 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee0165270 con 0x7feee00c4d40 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee0153200 con 0x7feee014e6f0 2023-05-16T07:02:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee014c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40292/0 (socket says 172.21.15.92:40292) 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 -- 172.21.15.92:0/311739532 learned_addr learned my addr 172.21.15.92:0/311739532 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feed7fff700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feee00c4d40 0x7feee00ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee56ca700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee00c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 -- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 msgr2=0x7feee00c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee00c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 -- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feee00c4d40 msgr2=0x7feee00ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feee00c4d40 0x7feee00ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 -- 172.21.15.92:0/311739532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8009370 con 0x7feee00629c0 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee56ca700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee00c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee4ec9700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee014c930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7feedc007f90 tx=0x7feedc007870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc016400 con 0x7feee00629c0 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc016560 con 0x7feee00629c0 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0168e0 con 0x7feee00629c0 2023-05-16T07:02:22.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.524+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee01525f0 con 0x7feee00629c0 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.525+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee0152bb0 con 0x7feee00629c0 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.525+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee01534e0 con 0x7feee00629c0 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.526+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7feedc0203f0 con 0x7feee00629c0 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.526+0000 7feed4ff9700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feecc05ed60 0x7feecc061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.526+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7feedc07a850 con 0x7feee00629c0 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.526+0000 7feed7fff700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feecc05ed60 0x7feecc061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.527+0000 7feed7fff700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feecc05ed60 0x7feecc061220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7feee00c2c10 tx=0x7feed0007180).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.530+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7feedc04bfd0 con 0x7feee00629c0 2023-05-16T07:02:22.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.691+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feee006c350 con 0x7feee00629c0 2023-05-16T07:02:22.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.691+0000 7feed4ff9700 1 -- 172.21.15.92:0/311739532 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7feedc04f5f0 con 0x7feee00629c0 2023-05-16T07:02:22.696 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:22.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feecc05ed60 msgr2=0x7feecc061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feecc05ed60 0x7feecc061220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7feee00c2c10 tx=0x7feed0007180).stop 2023-05-16T07:02:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 msgr2=0x7feee014c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee014c930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7feedc007f90 tx=0x7feedc007870).stop 2023-05-16T07:02:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 shutdown_connections 2023-05-16T07:02:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feecc05ed60 0x7feecc061220 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feee014e6f0 0x7feee00c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee00629c0 0x7feee014c930 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 --2- 172.21.15.92:0/311739532 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feee00c4d40 0x7feee00ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 >> 172.21.15.92:0/311739532 conn(0x7feee005e820 msgr2=0x7feee0067d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:22.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 shutdown_connections 2023-05-16T07:02:22.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.693+0000 7feee712d700 1 -- 172.21.15.92:0/311739532 wait complete. 2023-05-16T07:02:22.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:22.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.818+0000 7fd2cd61e700 1 Processor -- start 2023-05-16T07:02:22.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.818+0000 7fd2cd61e700 1 -- start start 2023-05-16T07:02:22.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2cd61e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd2c8154550 0x7fd2c80aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2cd61e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd2c80c5cb0 0x7fd2c80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2cd61e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c80c6650 con 0x7fd2c8154170 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2cd61e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80ab200 con 0x7fd2c8154550 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2cd61e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80c08b0 con 0x7fd2c80c5cb0 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2c67fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd2c8154550 0x7fd2c80aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2c67fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd2c8154550 0x7fd2c80aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40302/0 (socket says 172.21.15.92:40302) 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.819+0000 7fd2c67fc700 1 -- 172.21.15.92:0/3022385382 learned_addr learned my addr 172.21.15.92:0/3022385382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c77fe700 1 --2- 172.21.15.92:0/3022385382 >> v2:172.21.15.158:3300/0 conn(0x7fd2c80c5cb0 0x7fd2c80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c67fc700 1 -- 172.21.15.92:0/3022385382 >> v2:172.21.15.158:3300/0 conn(0x7fd2c80c5cb0 msgr2=0x7fd2c80c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c67fc700 1 --2- 172.21.15.92:0/3022385382 >> v2:172.21.15.158:3300/0 conn(0x7fd2c80c5cb0 0x7fd2c80c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c67fc700 1 -- 172.21.15.92:0/3022385382 >> v1:172.21.15.158:6789/0 conn(0x7fd2c8154170 legacy=0x7fd2c80a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c67fc700 1 -- 172.21.15.92:0/3022385382 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80bfca0 con 0x7fd2c8154550 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c77fe700 1 --2- 172.21.15.92:0/3022385382 >> v2:172.21.15.158:3300/0 conn(0x7fd2c80c5cb0 0x7fd2c80c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c67fc700 1 --2- 172.21.15.92:0/3022385382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80aad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd2c80c0820 tx=0x7fd2b00090f0).ready entity=mon.0 client_cookie=73e6657366852d04 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c5ffb700 1 -- 172.21.15.92:0/3022385382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000b5a0 con 0x7fd2c8154550 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c5ffb700 1 -- 172.21.15.92:0/3022385382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000b700 con 0x7fd2c8154550 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2c5ffb700 1 -- 172.21.15.92:0/3022385382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000ba80 con 0x7fd2c8154550 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3022385382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 msgr2=0x7fd2c80aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3022385382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80aad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd2c80c0820 tx=0x7fd2b00090f0).stop 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3022385382 shutdown_connections 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3022385382 >> v2:172.21.15.158:3300/0 conn(0x7fd2c80c5cb0 0x7fd2c80c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3022385382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80aad10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3022385382 >> 172.21.15.92:0/3022385382 conn(0x7fd2c80cb6d0 msgr2=0x7fd2c80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3022385382 shutdown_connections 2023-05-16T07:02:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.820+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3022385382 wait complete. 2023-05-16T07:02:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 Processor -- start 2023-05-16T07:02:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 -- start start 2023-05-16T07:02:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c80c5cb0 0x7fd2c81388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2c8154170 0x7fd2c80c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8159ee0 con 0x7fd2c8154550 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8138de0 con 0x7fd2c80c5cb0 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2cd61e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80ae780 con 0x7fd2c8154170 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.822+0000 7fd2c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40310/0 (socket says 172.21.15.92:40310) 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 -- 172.21.15.92:0/3881824423 learned_addr learned my addr 172.21.15.92:0/3881824423 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:22.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c67fc700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2c8154170 0x7fd2c80c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c6ffd700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c80c5cb0 0x7fd2c81388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 -- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2c8154170 msgr2=0x7fd2c80c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2c8154170 0x7fd2c80c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 -- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c80c5cb0 msgr2=0x7fd2c81388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c80c5cb0 0x7fd2c81388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 -- 172.21.15.92:0/3881824423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0009370 con 0x7fd2c8154550 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2c77fe700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80c3200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fd2c000c880 tx=0x7fd2c0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0010580 con 0x7fd2c8154550 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c00106e0 con 0x7fd2c8154550 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c80aea60 con 0x7fd2c8154550 2023-05-16T07:02:22.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0010a60 con 0x7fd2c8154550 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.823+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c8157190 con 0x7fd2c8154550 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.824+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c80af260 con 0x7fd2c8154550 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.824+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd2c001b460 con 0x7fd2c8154550 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.825+0000 7fd2beffd700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2a805ed60 0x7fd2a8061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.825+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd2c00797a0 con 0x7fd2c8154550 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.825+0000 7fd2c6ffd700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2a805ed60 0x7fd2a8061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.825+0000 7fd2c6ffd700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2a805ed60 0x7fd2a8061220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd2b8006fd0 tx=0x7fd2b8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:22.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:22.829+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd2c80af260 con 0x7fd2c8154550 2023-05-16T07:02:23.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.035+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c80af260 con 0x7fd2c8154550 2023-05-16T07:02:23.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.036+0000 7fd2beffd700 1 -- 172.21.15.92:0/3881824423 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7fd2c80af260 con 0x7fd2c8154550 2023-05-16T07:02:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:23.041 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-05-16T07:02:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2a805ed60 msgr2=0x7fd2a8061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2a805ed60 0x7fd2a8061220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd2b8006fd0 tx=0x7fd2b8008040).stop 2023-05-16T07:02:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 msgr2=0x7fd2c80c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80c3200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fd2c000c880 tx=0x7fd2c0008040).stop 2023-05-16T07:02:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 shutdown_connections 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2a805ed60 0x7fd2a8061220 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8154550 0x7fd2c80c3200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c80c5cb0 0x7fd2c81388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 --2- 172.21.15.92:0/3881824423 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2c8154170 0x7fd2c80c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 >> 172.21.15.92:0/3881824423 conn(0x7fd2c80cb6d0 msgr2=0x7fd2c80ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 shutdown_connections 2023-05-16T07:02:23.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.038+0000 7fd2cd61e700 1 -- 172.21.15.92:0/3881824423 wait complete. 2023-05-16T07:02:23.044 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2023-05-16T07:02:23.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.159+0000 7fabb7e74700 1 Processor -- start 2023-05-16T07:02:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.160+0000 7fabb7e74700 1 -- start start 2023-05-16T07:02:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.160+0000 7fabb7e74700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 0x7fabb00c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.160+0000 7fabb7e74700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fabb0068670 0x7fabb01549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.160+0000 7fabb7e74700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb00c3340 con 0x7fabb013a9a0 2023-05-16T07:02:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.160+0000 7fabb7e74700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb00c2ab0 con 0x7fabb0068670 2023-05-16T07:02:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.160+0000 7fabb7e74700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c2d10 con 0x7fabb013a5c0 2023-05-16T07:02:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 0x7fabb00c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb6411700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fabb0068670 0x7fabb01549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 0x7fabb00c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:32890/0 (socket says 172.21.15.92:32890) 2023-05-16T07:02:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 -- 172.21.15.92:0/1903787734 learned_addr learned my addr 172.21.15.92:0/1903787734 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb540f700 1 --1- 172.21.15.92:0/1903787734 >> v1:172.21.15.158:6789/0 conn(0x7fabb013a9a0 0x7fabb0067f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:54668/0 (socket says 172.21.15.92:54668) 2023-05-16T07:02:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 -- 172.21.15.92:0/1903787734 >> v2:172.21.15.158:3300/0 conn(0x7fabb0068670 msgr2=0x7fabb01549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:23.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 --2- 172.21.15.92:0/1903787734 >> v2:172.21.15.158:3300/0 conn(0x7fabb0068670 0x7fabb01549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 -- 172.21.15.92:0/1903787734 >> v1:172.21.15.158:6789/0 conn(0x7fabb013a9a0 legacy=0x7fabb0067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 -- 172.21.15.92:0/1903787734 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0155670 con 0x7fabb013a5c0 2023-05-16T07:02:23.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb6411700 1 --2- 172.21.15.92:0/1903787734 >> v2:172.21.15.158:3300/0 conn(0x7fabb0068670 0x7fabb01549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:23.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb5c10700 1 --2- 172.21.15.92:0/1903787734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 0x7fabb00c37d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fabac006fd0 tx=0x7fabac0090f0).ready entity=mon.2 client_cookie=58b085f8f796868c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb4c0e700 1 -- 172.21.15.92:0/1903787734 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00b580 con 0x7fabb013a5c0 2023-05-16T07:02:23.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb4c0e700 1 -- 172.21.15.92:0/1903787734 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac00b6e0 con 0x7fabb013a5c0 2023-05-16T07:02:23.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.161+0000 7fabb4c0e700 1 -- 172.21.15.92:0/1903787734 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00ba60 con 0x7fabb013a5c0 2023-05-16T07:02:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 -- 172.21.15.92:0/1903787734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 msgr2=0x7fabb00c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 --2- 172.21.15.92:0/1903787734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 0x7fabb00c37d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fabac006fd0 tx=0x7fabac0090f0).stop 2023-05-16T07:02:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 -- 172.21.15.92:0/1903787734 shutdown_connections 2023-05-16T07:02:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 --2- 172.21.15.92:0/1903787734 >> v2:172.21.15.158:3300/0 conn(0x7fabb0068670 0x7fabb01549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 --2- 172.21.15.92:0/1903787734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb013a5c0 0x7fabb00c37d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 -- 172.21.15.92:0/1903787734 >> 172.21.15.92:0/1903787734 conn(0x7fabb00c4430 msgr2=0x7fabb00c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 -- 172.21.15.92:0/1903787734 shutdown_connections 2023-05-16T07:02:23.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.162+0000 7fabb7e74700 1 -- 172.21.15.92:0/1903787734 wait complete. 2023-05-16T07:02:23.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.163+0000 7fabb7e74700 1 Processor -- start 2023-05-16T07:02:23.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.163+0000 7fabb7e74700 1 -- start start 2023-05-16T07:02:23.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabb0068670 0x7fabb0136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 0x7fabb0138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb0138c10 0x7fabb00beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0162520 con 0x7fabb013a5c0 2023-05-16T07:02:23.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0162800 con 0x7fabb0068670 2023-05-16T07:02:23.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0137390 con 0x7fabb0138c10 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 0x7fabb0138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 0x7fabb0138720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40324/0 (socket says 172.21.15.92:40324) 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 -- 172.21.15.92:0/3878231741 learned_addr learned my addr 172.21.15.92:0/3878231741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb5c10700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabb0068670 0x7fabb0136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb6411700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb0138c10 0x7fabb00beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 -- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb0138c10 msgr2=0x7fabb00beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb0138c10 0x7fabb00beb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 -- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabb0068670 msgr2=0x7fabb0136ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabb0068670 0x7fabb0136ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 -- 172.21.15.92:0/3878231741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009a20 con 0x7fabb013a5c0 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb540f700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 0x7fabb0138720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7faba000c4a0 tx=0x7faba0007d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba000ce80 con 0x7fabb013a5c0 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba00072e0 con 0x7fabb013a5c0 2023-05-16T07:02:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac009370 con 0x7fabb013a5c0 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.164+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba00056e0 con 0x7fabb013a5c0 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.165+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb00bf850 con 0x7fabb013a5c0 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.166+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb006c350 con 0x7fabb013a5c0 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.166+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7faba0010070 con 0x7fabb013a5c0 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.166+0000 7faba5ffb700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fab9405ed90 0x7fab94061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.166+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7faba0078d90 con 0x7fabb013a5c0 2023-05-16T07:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.166+0000 7fabb5c10700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fab9405ed90 0x7fab94061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.167+0000 7fabb5c10700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fab9405ed90 0x7fab94061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fabac009130 tx=0x7fabac017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.170+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7faba004a680 con 0x7fabb013a5c0 2023-05-16T07:02:23.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.335+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabb0162940 con 0x7fabb013a5c0 2023-05-16T07:02:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.336+0000 7faba5ffb700 1 -- 172.21.15.92:0/3878231741 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13757 (secure 0 0 0) 0x7faba004dca0 con 0x7fabb013a5c0 2023-05-16T07:02:23.340 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:23.340 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:22.770131+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:23.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fab9405ed90 msgr2=0x7fab94061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fab9405ed90 0x7fab94061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fabac009130 tx=0x7fabac017000).stop 2023-05-16T07:02:23.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 msgr2=0x7fabb0138720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 0x7fabb0138720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7faba000c4a0 tx=0x7faba0007d50).stop 2023-05-16T07:02:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 shutdown_connections 2023-05-16T07:02:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fab9405ed90 0x7fab94061250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fabb0138c10 0x7fabb00beb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabb0068670 0x7fabb0136ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 --2- 172.21.15.92:0/3878231741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabb013a5c0 0x7fabb0138720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 >> 172.21.15.92:0/3878231741 conn(0x7fabb00c4430 msgr2=0x7fabb00ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 shutdown_connections 2023-05-16T07:02:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.338+0000 7fabb7e74700 1 -- 172.21.15.92:0/3878231741 wait complete. 2023-05-16T07:02:23.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:02:23.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:23.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.465+0000 7f0427f5d700 1 Processor -- start 2023-05-16T07:02:23.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.465+0000 7f0427f5d700 1 -- start start 2023-05-16T07:02:23.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0427f5d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f04200bc290 0x7f0420150380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0427f5d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 0x7f04200bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0427f5d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04200bc770 con 0x7f042014ffa0 2023-05-16T07:02:23.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0427f5d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042006a510 con 0x7f04200bc290 2023-05-16T07:02:23.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0427f5d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04200bca50 con 0x7f042005dd50 2023-05-16T07:02:23.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0425cf9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f042014ffa0 0x7f0420069e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51078/0 (socket says 172.21.15.92:51078) 2023-05-16T07:02:23.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0425cf9700 1 -- 172.21.15.92:0/2998059775 learned_addr learned my addr 172.21.15.92:0/2998059775 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f04254f8700 1 --2- 172.21.15.92:0/2998059775 >> v2:172.21.15.158:3300/0 conn(0x7f04200bc290 0x7f0420150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f04264fa700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 0x7f04200bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941018504 0 0) 0x7f04200bc770 con 0x7f042014ffa0 2023-05-16T07:02:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f040c0033f0 con 0x7f042014ffa0 2023-05-16T07:02:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2000480031 0 0) 0x7f040c0033f0 con 0x7f042014ffa0 2023-05-16T07:02:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 msgr2=0x7f04200bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 0x7f04200bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 >> v2:172.21.15.158:3300/0 conn(0x7f04200bc290 msgr2=0x7f0420150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 --2- 172.21.15.92:0/2998059775 >> v2:172.21.15.158:3300/0 conn(0x7f04200bc290 0x7f0420150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200bcd30 con 0x7f042014ffa0 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f041c001470 con 0x7f042014ffa0 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f04254f8700 1 --2- 172.21.15.92:0/2998059775 >> v2:172.21.15.158:3300/0 conn(0x7f04200bc290 0x7f0420150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 >> v1:172.21.15.92:6789/0 conn(0x7f042014ffa0 legacy=0x7f0420069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f040c0038e0 0x7f040c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.466+0000 7f0424cf7700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f040c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 0x7f040c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042006a510 con 0x7f040c004690 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200bca50 con 0x7f040c0038e0 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c00b8e0 con 0x7f042005dd50 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f040c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0425cf9700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f040c0038e0 0x7f040c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04264fa700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 0x7f040c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 -- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 msgr2=0x7f040c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f042005dd50 0x7f040c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 -- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f040c0038e0 msgr2=0x7f040c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f040c0038e0 0x7f040c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 -- 172.21.15.92:0/2998059775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0033f0 con 0x7f040c004690 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f04254f8700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f040c004b10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0410009c70 tx=0x7f0410009060).ready entity=mon.0 client_cookie=af3f3c0d0404c3b5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100095c0 con 0x7f040c004690 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041000e6f0 con 0x7f040c004690 2023-05-16T07:02:23.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.467+0000 7f0424cf7700 1 -- 172.21.15.92:0/2998059775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000ea70 con 0x7f040c004690 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f0427f5d700 1 -- 172.21.15.92:0/2998059775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 msgr2=0x7f040c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f0427f5d700 1 --2- 172.21.15.92:0/2998059775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f040c004b10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0410009c70 tx=0x7f0410009060).stop 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f04264fa700 1 -- 172.21.15.92:0/2998059775 reap_dead start 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f0427f5d700 1 -- 172.21.15.92:0/2998059775 shutdown_connections 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f0427f5d700 1 -- 172.21.15.92:0/2998059775 >> 172.21.15.92:0/2998059775 conn(0x7f04200c4620 msgr2=0x7f0420066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f0427f5d700 1 -- 172.21.15.92:0/2998059775 shutdown_connections 2023-05-16T07:02:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.468+0000 7f0427f5d700 1 -- 172.21.15.92:0/2998059775 wait complete. 2023-05-16T07:02:23.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 Processor -- start 2023-05-16T07:02:23.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 -- start start 2023-05-16T07:02:23.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f0420147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f04201498a0 0x7f0420149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0420137660 0x7f0420137ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200c0060 con 0x7f040c004690 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042014a9a0 con 0x7f0420137660 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.469+0000 7f0427f5d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042014ae30 con 0x7f04201498a0 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f0420147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f0420147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40342/0 (socket says 172.21.15.92:40342) 2023-05-16T07:02:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 -- 172.21.15.92:0/1439607147 learned_addr learned my addr 172.21.15.92:0/1439607147 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f04264fa700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0420137660 0x7f0420137ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f04254f8700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f04201498a0 0x7f0420149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 -- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f04201498a0 msgr2=0x7f0420149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f04201498a0 0x7f0420149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 -- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0420137660 msgr2=0x7f0420137ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0420137660 0x7f0420137ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 -- 172.21.15.92:0/1439607147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200bfd80 con 0x7f040c004690 2023-05-16T07:02:23.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f04254f8700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f04201498a0 0x7f0420149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0425cf9700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f0420147d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f041c007fd0 tx=0x7f041c001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c011840 con 0x7f040c004690 2023-05-16T07:02:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c0119a0 con 0x7f040c004690 2023-05-16T07:02:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042014a350 con 0x7f040c004690 2023-05-16T07:02:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c011d20 con 0x7f040c004690 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.470+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0420137fb0 con 0x7f040c004690 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.471+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f041c013040 con 0x7f040c004690 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.471+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042006c350 con 0x7f040c004690 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.472+0000 7f0415ffb700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f040405ed90 0x7f0404061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.472+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f041c07bd80 con 0x7f040c004690 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.472+0000 7f04254f8700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f040405ed90 0x7f0404061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.472+0000 7f04254f8700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f040405ed90 0x7f0404061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f04200c0f30 tx=0x7f041001a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.476+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f041c04d670 con 0x7f040c004690 2023-05-16T07:02:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.638+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f04200bd010 con 0x7f040c004690 2023-05-16T07:02:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.639+0000 7f0415ffb700 1 -- 172.21.15.92:0/1439607147 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f041c07d070 con 0x7f040c004690 2023-05-16T07:02:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f040405ed90 msgr2=0x7f0404061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f040405ed90 0x7f0404061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f04200c0f30 tx=0x7f041001a000).stop 2023-05-16T07:02:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 msgr2=0x7f0420147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f0420147d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f041c007fd0 tx=0x7f041c001e90).stop 2023-05-16T07:02:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 shutdown_connections 2023-05-16T07:02:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f040405ed90 0x7f0404061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0420137660 0x7f0420137ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f040c004690 0x7f0420147d20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 --2- 172.21.15.92:0/1439607147 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f04201498a0 0x7f0420149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 >> 172.21.15.92:0/1439607147 conn(0x7f04200c4620 msgr2=0x7f04200bd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 shutdown_connections 2023-05-16T07:02:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.641+0000 7f0427f5d700 1 -- 172.21.15.92:0/1439607147 wait complete. 2023-05-16T07:02:23.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.766+0000 7f8d40ae6700 1 Processor -- start 2023-05-16T07:02:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.766+0000 7f8d40ae6700 1 -- start start 2023-05-16T07:02:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d40ae6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d3c157a20 0x7f8d3c066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d40ae6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c067340 0x7f8d3c065840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d40ae6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0653b0 con 0x7f8d3c14e440 2023-05-16T07:02:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d40ae6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c0677d0 con 0x7f8d3c157a20 2023-05-16T07:02:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d40ae6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c158000 con 0x7f8d3c067340 2023-05-16T07:02:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d39d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d3c157a20 0x7f8d3c066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d39d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d3c157a20 0x7f8d3c066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40354/0 (socket says 172.21.15.92:40354) 2023-05-16T07:02:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d39d9b700 1 -- 172.21.15.92:0/332119955 learned_addr learned my addr 172.21.15.92:0/332119955 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d3ad9d700 1 --2- 172.21.15.92:0/332119955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c067340 0x7f8d3c065840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d39d9b700 1 -- 172.21.15.92:0/332119955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c067340 msgr2=0x7f8d3c065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d39d9b700 1 --2- 172.21.15.92:0/332119955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c067340 0x7f8d3c065840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.767+0000 7f8d39d9b700 1 -- 172.21.15.92:0/332119955 >> v1:172.21.15.158:6789/0 conn(0x7f8d3c14e440 legacy=0x7f8d3c154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d39d9b700 1 -- 172.21.15.92:0/332119955 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c1582e0 con 0x7f8d3c157a20 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d39d9b700 1 --2- 172.21.15.92:0/332119955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c157a20 0x7f8d3c066e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8d24006fd0 tx=0x7f8d240090f0).ready entity=mon.0 client_cookie=3416d897bf711cab server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d3959a700 1 -- 172.21.15.92:0/332119955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400b600 con 0x7f8d3c157a20 2023-05-16T07:02:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d3959a700 1 -- 172.21.15.92:0/332119955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400b760 con 0x7f8d3c157a20 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d3959a700 1 -- 172.21.15.92:0/332119955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400bae0 con 0x7f8d3c157a20 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 -- 172.21.15.92:0/332119955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c157a20 msgr2=0x7f8d3c066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/332119955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c157a20 0x7f8d3c066e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8d24006fd0 tx=0x7f8d240090f0).stop 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 -- 172.21.15.92:0/332119955 shutdown_connections 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/332119955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c067340 0x7f8d3c065840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/332119955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c157a20 0x7f8d3c066e50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 -- 172.21.15.92:0/332119955 >> 172.21.15.92:0/332119955 conn(0x7f8d3c0c8470 msgr2=0x7f8d3c0c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 -- 172.21.15.92:0/332119955 shutdown_connections 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.768+0000 7f8d40ae6700 1 -- 172.21.15.92:0/332119955 wait complete. 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.769+0000 7f8d40ae6700 1 Processor -- start 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.769+0000 7f8d40ae6700 1 -- start start 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d40ae6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d3c067340 0x7f8d3c0ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d40ae6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 0x7f8d3c0a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d40ae6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c157a20 0x7f8d3c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d40ae6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0a86b0 con 0x7f8d3c14e440 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d40ae6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0a83d0 con 0x7f8d3c067340 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d40ae6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0aaa80 con 0x7f8d3c157a20 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d39d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 0x7f8d3c0a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d39d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 0x7f8d3c0a7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40358/0 (socket says 172.21.15.92:40358) 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d39d9b700 1 -- 172.21.15.92:0/3923771162 learned_addr learned my addr 172.21.15.92:0/3923771162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d3a59c700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d3c067340 0x7f8d3c0ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.770+0000 7f8d3ad9d700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c157a20 0x7f8d3c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d39d9b700 1 -- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c157a20 msgr2=0x7f8d3c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d39d9b700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c157a20 0x7f8d3c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d39d9b700 1 -- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d3c067340 msgr2=0x7f8d3c0ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d39d9b700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d3c067340 0x7f8d3c0ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d39d9b700 1 -- 172.21.15.92:0/3923771162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009370 con 0x7f8d3c14e440 2023-05-16T07:02:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d39d9b700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 0x7f8d3c0a7a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8d3c064ad0 tx=0x7f8d24017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400bea0 con 0x7f8d3c14e440 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400b8d0 con 0x7f8d3c14e440 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c0aad60 con 0x7f8d3c14e440 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2401c670 con 0x7f8d3c14e440 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.771+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c0a7f50 con 0x7f8d3c14e440 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.772+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c0a8170 con 0x7f8d3c14e440 2023-05-16T07:02:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.772+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8d2402c070 con 0x7f8d3c14e440 2023-05-16T07:02:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.772+0000 7f8d327fc700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d1c05ed90 0x7f8d1c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.772+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8d2407b270 con 0x7f8d3c14e440 2023-05-16T07:02:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.772+0000 7f8d3a59c700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d1c05ed90 0x7f8d1c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.773+0000 7f8d3a59c700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d1c05ed90 0x7f8d1c061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8d2c004290 tx=0x7f8d2c001560).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.776+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8d2404cb60 con 0x7f8d3c14e440 2023-05-16T07:02:23.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.940+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8d3c06c350 con 0x7f8d3c14e440 2023-05-16T07:02:23.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.941+0000 7f8d327fc700 1 -- 172.21.15.92:0/3923771162 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8d2402a020 con 0x7f8d3c14e440 2023-05-16T07:02:23.945 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.942+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d1c05ed90 msgr2=0x7f8d1c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.942+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d1c05ed90 0x7f8d1c061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8d2c004290 tx=0x7f8d2c001560).stop 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.942+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 msgr2=0x7f8d3c0a7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.942+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 0x7f8d3c0a7a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8d3c064ad0 tx=0x7f8d24017000).stop 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 shutdown_connections 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d1c05ed90 0x7f8d1c061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d3c157a20 0x7f8d3c0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d3c067340 0x7f8d3c0ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 --2- 172.21.15.92:0/3923771162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d3c14e440 0x7f8d3c0a7a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 >> 172.21.15.92:0/3923771162 conn(0x7f8d3c0c8470 msgr2=0x7f8d3c0c56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 shutdown_connections 2023-05-16T07:02:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:23.943+0000 7f8d40ae6700 1 -- 172.21.15.92:0/3923771162 wait complete. 2023-05-16T07:02:23.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:02:24.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.064+0000 7fbdad7e7700 1 Processor -- start 2023-05-16T07:02:24.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.065+0000 7fbdad7e7700 1 -- start start 2023-05-16T07:02:24.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.065+0000 7fbdad7e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 0x7fbda80bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.065+0000 7fbdad7e7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 0x7fbda80c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.065+0000 7fbdad7e7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda80bdbb0 con 0x7fbda813a5a0 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.065+0000 7fbdad7e7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda8154b80 con 0x7fbda80bd780 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.065+0000 7fbdad7e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8154e60 con 0x7fbda8068060 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 0x7fbda80bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda77fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 0x7fbda80c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 0x7fbda80bd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:32942/0 (socket says 172.21.15.92:32942) 2023-05-16T07:02:24.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 -- 172.21.15.92:0/560278411 learned_addr learned my addr 172.21.15.92:0/560278411 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:24.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda77fe700 1 --2- 172.21.15.92:0/560278411 >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 0x7fbda80c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44004/0 (socket says 172.21.15.92:44004) 2023-05-16T07:02:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 -- 172.21.15.92:0/560278411 >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 msgr2=0x7fbda80c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:24.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 --2- 172.21.15.92:0/560278411 >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 0x7fbda80c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 -- 172.21.15.92:0/560278411 >> v1:172.21.15.158:6789/0 conn(0x7fbda813a5a0 legacy=0x7fbda8068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 -- 172.21.15.92:0/560278411 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8155140 con 0x7fbda8068060 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda77fe700 1 --2- 172.21.15.92:0/560278411 >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 0x7fbda80c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda67fc700 1 --2- 172.21.15.92:0/560278411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 0x7fbda80bd290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbd90006fd0 tx=0x7fbd9000b3b0).ready entity=mon.2 client_cookie=a39243f3a10ff3dc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda5ffb700 1 -- 172.21.15.92:0/560278411 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000da70 con 0x7fbda8068060 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda5ffb700 1 -- 172.21.15.92:0/560278411 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000dbd0 con 0x7fbda8068060 2023-05-16T07:02:24.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.066+0000 7fbda5ffb700 1 -- 172.21.15.92:0/560278411 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90010470 con 0x7fbda8068060 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 -- 172.21.15.92:0/560278411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 msgr2=0x7fbda80bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/560278411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 0x7fbda80bd290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbd90006fd0 tx=0x7fbd9000b3b0).stop 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 -- 172.21.15.92:0/560278411 shutdown_connections 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/560278411 >> v2:172.21.15.158:3300/0 conn(0x7fbda80bd780 0x7fbda80c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/560278411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda8068060 0x7fbda80bd290 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 -- 172.21.15.92:0/560278411 >> 172.21.15.92:0/560278411 conn(0x7fbda80c4430 msgr2=0x7fbda80c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:24.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 -- 172.21.15.92:0/560278411 shutdown_connections 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.067+0000 7fbdad7e7700 1 -- 172.21.15.92:0/560278411 wait complete. 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.068+0000 7fbdad7e7700 1 Processor -- start 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.068+0000 7fbdad7e7700 1 -- start start 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbdad7e7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda8068060 0x7fbda8138150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbdad7e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda80bd780 0x7fbda814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbdad7e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 0x7fbda814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbdad7e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80ab060 con 0x7fbda813a5a0 2023-05-16T07:02:24.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbdad7e7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80aa9c0 con 0x7fbda8068060 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbdad7e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda814e780 con 0x7fbda80bd780 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 0x7fbda814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 0x7fbda814e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40376/0 (socket says 172.21.15.92:40376) 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 -- 172.21.15.92:0/3761029007 learned_addr learned my addr 172.21.15.92:0/3761029007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda6ffd700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda8068060 0x7fbda8138150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda67fc700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda80bd780 0x7fbda814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 -- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda80bd780 msgr2=0x7fbda814dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda80bd780 0x7fbda814dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 -- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda8068060 msgr2=0x7fbda8138150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda8068060 0x7fbda8138150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 -- 172.21.15.92:0/3761029007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda00092a0 con 0x7fbda813a5a0 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbda77fe700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 0x7fbda814e190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fbda000e980 tx=0x7fbda0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda000c850 con 0x7fbda813a5a0 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda000c9b0 con 0x7fbda813a5a0 2023-05-16T07:02:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.069+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda000cd50 con 0x7fbda813a5a0 2023-05-16T07:02:24.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.070+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9000b610 con 0x7fbda813a5a0 2023-05-16T07:02:24.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.070+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80aac80 con 0x7fbda813a5a0 2023-05-16T07:02:24.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.070+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda80ab1a0 con 0x7fbda813a5a0 2023-05-16T07:02:24.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.071+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbda0022070 con 0x7fbda813a5a0 2023-05-16T07:02:24.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.071+0000 7fbd9effd700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ed90 0x7fbd88061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.071+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fbda0079cb0 con 0x7fbda813a5a0 2023-05-16T07:02:24.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.071+0000 7fbda6ffd700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ed90 0x7fbd88061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.072+0000 7fbda6ffd700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ed90 0x7fbd88061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fbd98003e70 tx=0x7fbd98009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:24.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.075+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbda80ab1a0 con 0x7fbda813a5a0 2023-05-16T07:02:24.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.239+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbda806c350 con 0x7fbda813a5a0 2023-05-16T07:02:24.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.240+0000 7fbd9effd700 1 -- 172.21.15.92:0/3761029007 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7fbda0016020 con 0x7fbda813a5a0 2023-05-16T07:02:24.244 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:24.244 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:02:24.245 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:02:24.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:02:24.246 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:02:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ed90 msgr2=0x7fbd88061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ed90 0x7fbd88061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fbd98003e70 tx=0x7fbd98009040).stop 2023-05-16T07:02:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 msgr2=0x7fbda814e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 0x7fbda814e190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fbda000e980 tx=0x7fbda0009060).stop 2023-05-16T07:02:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 shutdown_connections 2023-05-16T07:02:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ed90 0x7fbd88061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda813a5a0 0x7fbda814e190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda8068060 0x7fbda8138150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.241+0000 7fbdad7e7700 1 --2- 172.21.15.92:0/3761029007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda80bd780 0x7fbda814dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.242+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 >> 172.21.15.92:0/3761029007 conn(0x7fbda80c4430 msgr2=0x7fbda81393d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:24.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.242+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 shutdown_connections 2023-05-16T07:02:24.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.242+0000 7fbdad7e7700 1 -- 172.21.15.92:0/3761029007 wait complete. 2023-05-16T07:02:24.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:02:24.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:02:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.369+0000 7fcf29bc4700 1 Processor -- start 2023-05-16T07:02:24.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.369+0000 7fcf29bc4700 1 -- start start 2023-05-16T07:02:24.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf29bc4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 0x7fcf240aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf29bc4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcf24062f20 0x7fcf240a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf29bc4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf24069440 con 0x7fcf240698d0 2023-05-16T07:02:24.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf29bc4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf24069110 con 0x7fcf24062f20 2023-05-16T07:02:24.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf29bc4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240a7aa0 con 0x7fcf24062b40 2023-05-16T07:02:24.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf293c3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcf240698d0 0x7fcf240aaa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51128/0 (socket says 172.21.15.92:51128) 2023-05-16T07:02:24.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf293c3700 1 -- 172.21.15.92:0/2501792155 learned_addr learned my addr 172.21.15.92:0/2501792155 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:24.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf28bc2700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 0x7fcf240aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.370+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454182922 0 0) 0x7fcf24069440 con 0x7fcf240698d0 2023-05-16T07:02:24.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf23fff700 1 --2- 172.21.15.92:0/2501792155 >> v2:172.21.15.158:3300/0 conn(0x7fcf24062f20 0x7fcf240a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf0c0033f0 con 0x7fcf240698d0 2023-05-16T07:02:24.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2801893648 0 0) 0x7fcf0c0033f0 con 0x7fcf240698d0 2023-05-16T07:02:24.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 msgr2=0x7fcf240aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 0x7fcf240aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 >> v2:172.21.15.158:3300/0 conn(0x7fcf24062f20 msgr2=0x7fcf240a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 --2- 172.21.15.92:0/2501792155 >> v2:172.21.15.158:3300/0 conn(0x7fcf24062f20 0x7fcf240a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24065bb0 con 0x7fcf240698d0 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fcf1c0017f0 con 0x7fcf240698d0 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 >> v1:172.21.15.92:6789/0 conn(0x7fcf240698d0 legacy=0x7fcf240aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf0c0038e0 0x7fcf0c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 0x7fcf0c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf0c004b80 0x7fcf0c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf24069110 con 0x7fcf0c004b80 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240a7aa0 con 0x7fcf0c0038e0 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c00b8e0 con 0x7fcf24062b40 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf0c004b80 0x7fcf0c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf28bc2700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf0c0038e0 0x7fcf0c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf23fff700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 0x7fcf0c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 -- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 msgr2=0x7fcf0c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf24062b40 0x7fcf0c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 -- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf0c0038e0 msgr2=0x7fcf0c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf0c0038e0 0x7fcf0c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 -- 172.21.15.92:0/2501792155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0033f0 con 0x7fcf0c004b80 2023-05-16T07:02:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf293c3700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf0c004b80 0x7fcf0c005000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fcf1c00c530 tx=0x7fcf1c001d90).ready entity=mon.0 client_cookie=68042e60b4f27c55 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00ce60 con 0x7fcf0c004b80 2023-05-16T07:02:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c005b20 con 0x7fcf0c004b80 2023-05-16T07:02:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.371+0000 7fcf237fe700 1 -- 172.21.15.92:0/2501792155 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0070d0 con 0x7fcf0c004b80 2023-05-16T07:02:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf29bc4700 1 -- 172.21.15.92:0/2501792155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf0c004b80 msgr2=0x7fcf0c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/2501792155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf0c004b80 0x7fcf0c005000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fcf1c00c530 tx=0x7fcf1c001d90).stop 2023-05-16T07:02:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf293c3700 1 -- 172.21.15.92:0/2501792155 reap_dead start 2023-05-16T07:02:24.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf29bc4700 1 -- 172.21.15.92:0/2501792155 shutdown_connections 2023-05-16T07:02:24.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf29bc4700 1 -- 172.21.15.92:0/2501792155 >> 172.21.15.92:0/2501792155 conn(0x7fcf24068a30 msgr2=0x7fcf240c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:24.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf29bc4700 1 -- 172.21.15.92:0/2501792155 shutdown_connections 2023-05-16T07:02:24.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.372+0000 7fcf29bc4700 1 -- 172.21.15.92:0/2501792155 wait complete. 2023-05-16T07:02:24.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.373+0000 7fcf29bc4700 1 Processor -- start 2023-05-16T07:02:24.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.373+0000 7fcf29bc4700 1 -- start start 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf29bc4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 0x7fcf240a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf29bc4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf241493d0 0x7fcf2414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf29bc4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf2414ec00 0x7fcf240aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf29bc4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf24163e30 con 0x7fcf24148ff0 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf29bc4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241677e0 con 0x7fcf241493d0 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf29bc4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf24167ac0 con 0x7fcf2414ec00 2023-05-16T07:02:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf28bc2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 0x7fcf240a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf28bc2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 0x7fcf240a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40390/0 (socket says 172.21.15.92:40390) 2023-05-16T07:02:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf28bc2700 1 -- 172.21.15.92:0/641966466 learned_addr learned my addr 172.21.15.92:0/641966466 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf293c3700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf2414ec00 0x7fcf240aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf23fff700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf241493d0 0x7fcf2414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf28bc2700 1 -- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf2414ec00 msgr2=0x7fcf240aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.374+0000 7fcf28bc2700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf2414ec00 0x7fcf240aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf28bc2700 1 -- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf241493d0 msgr2=0x7fcf2414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf28bc2700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf241493d0 0x7fcf2414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf28bc2700 1 -- 172.21.15.92:0/641966466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24167da0 con 0x7fcf24148ff0 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf28bc2700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 0x7fcf240a4bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fcf140119c0 tx=0x7fcf14009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400f700 con 0x7fcf24148ff0 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1400f860 con 0x7fcf24148ff0 2023-05-16T07:02:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf24168080 con 0x7fcf24148ff0 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400fbe0 con 0x7fcf24148ff0 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.375+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf24167260 con 0x7fcf24148ff0 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.376+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf24163c30 con 0x7fcf24148ff0 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.376+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcf14014440 con 0x7fcf24148ff0 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.376+0000 7fcf20ff9700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf1005edc0 0x7fcf10061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.377+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fcf14079db0 con 0x7fcf24148ff0 2023-05-16T07:02:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.377+0000 7fcf23fff700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf1005edc0 0x7fcf10061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:24.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.377+0000 7fcf23fff700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf1005edc0 0x7fcf10061280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fcf241539a0 tx=0x7fcf18009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:24.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.381+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcf1404b640 con 0x7fcf24148ff0 2023-05-16T07:02:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.546+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fcf24047150 con 0x7fcf24148ff0 2023-05-16T07:02:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.884+0000 7fcf20ff9700 1 -- 172.21.15.92:0/641966466 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 152+0+0 (secure 0 0 0) 0x7fcf1404ec60 con 0x7fcf24148ff0 2023-05-16T07:02:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.887+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf1005edc0 msgr2=0x7fcf10061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.887+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf1005edc0 0x7fcf10061280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fcf241539a0 tx=0x7fcf18009060).stop 2023-05-16T07:02:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.887+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 msgr2=0x7fcf240a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.887+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 0x7fcf240a4bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fcf140119c0 tx=0x7fcf14009060).stop 2023-05-16T07:02:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 shutdown_connections 2023-05-16T07:02:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf1005edc0 0x7fcf10061280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf2414ec00 0x7fcf240aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf24148ff0 0x7fcf240a4bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 --2- 172.21.15.92:0/641966466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf241493d0 0x7fcf2414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 >> 172.21.15.92:0/641966466 conn(0x7fcf24068a30 msgr2=0x7fcf2414bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 shutdown_connections 2023-05-16T07:02:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:24.891+0000 7fcf29bc4700 1 -- 172.21.15.92:0/641966466 wait complete. 2023-05-16T07:02:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:02:24.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:02:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.021+0000 7fddaf2e6700 1 Processor -- start 2023-05-16T07:02:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.021+0000 7fddaf2e6700 1 -- start start 2023-05-16T07:02:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddaf2e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddaf2e6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdda80622b0 0x7fdda8062760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddaf2e6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda8062c50 con 0x7fdda8152e40 2023-05-16T07:02:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddaf2e6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda8063690 con 0x7fdda80622b0 2023-05-16T07:02:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddaf2e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda814fd70 con 0x7fdda814f990 2023-05-16T07:02:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddac881700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdda8152e40 0x7fdda80c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51146/0 (socket says 172.21.15.92:51146) 2023-05-16T07:02:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddac881700 1 -- 172.21.15.92:0/1266284017 learned_addr learned my addr 172.21.15.92:0/1266284017 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad082700 1 --2- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad883700 1 --2- 172.21.15.92:0/1266284017 >> v2:172.21.15.158:3300/0 conn(0x7fdda80622b0 0x7fdda8062760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fdd9ffff700 1 -- 172.21.15.92:0/1266284017 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226402292 0 0) 0x7fdd98001110 con 0x7fdda8152e40 2023-05-16T07:02:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fdd9ffff700 1 -- 172.21.15.92:0/1266284017 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd940033f0 con 0x7fdda8152e40 2023-05-16T07:02:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad883700 1 -- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 msgr2=0x7fdda80c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad883700 1 --2- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad883700 1 -- 172.21.15.92:0/1266284017 >> v1:172.21.15.92:6789/0 conn(0x7fdda8152e40 legacy=0x7fdda80c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:25.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad883700 1 -- 172.21.15.92:0/1266284017 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda80c52f0 con 0x7fdda80622b0 2023-05-16T07:02:25.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fddad883700 1 --2- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda8062760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdda0009e60 tx=0x7fdda0009780).ready entity=mon.1 client_cookie=7657ddababd6d95b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:25.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.022+0000 7fdd9ffff700 1 -- 172.21.15.92:0/1266284017 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda000e070 con 0x7fdda80622b0 2023-05-16T07:02:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fdd9ffff700 1 -- 172.21.15.92:0/1266284017 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda000bab0 con 0x7fdda80622b0 2023-05-16T07:02:25.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fdd9ffff700 1 -- 172.21.15.92:0/1266284017 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda000be30 con 0x7fdda80622b0 2023-05-16T07:02:25.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 -- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 msgr2=0x7fdda8062760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:25.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda8062760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdda0009e60 tx=0x7fdda0009780).stop 2023-05-16T07:02:25.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 -- 172.21.15.92:0/1266284017 shutdown_connections 2023-05-16T07:02:25.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda8062760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/1266284017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 -- 172.21.15.92:0/1266284017 >> 172.21.15.92:0/1266284017 conn(0x7fdda805e820 msgr2=0x7fdda80c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:25.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 -- 172.21.15.92:0/1266284017 shutdown_connections 2023-05-16T07:02:25.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.023+0000 7fddaf2e6700 1 -- 172.21.15.92:0/1266284017 wait complete. 2023-05-16T07:02:25.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.024+0000 7fddaf2e6700 1 Processor -- start 2023-05-16T07:02:25.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.024+0000 7fddaf2e6700 1 -- start start 2023-05-16T07:02:25.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddaf2e6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda80c07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:25.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddaf2e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddaf2e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 0x7fdda80c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddaf2e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda80bfb30 con 0x7fdda8152e40 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddaf2e6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda8163170 con 0x7fdda80622b0 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddaf2e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda80bfd10 con 0x7fdda814f990 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 0x7fdda80c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 0x7fdda80c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40406/0 (socket says 172.21.15.92:40406) 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 -- 172.21.15.92:0/2409635571 learned_addr learned my addr 172.21.15.92:0/2409635571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddac881700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad082700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda80c07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 -- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 msgr2=0x7fdda80acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 -- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 msgr2=0x7fdda80c07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda80c07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddad883700 1 -- 172.21.15.92:0/2409635571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0009a20 con 0x7fdda8152e40 2023-05-16T07:02:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.025+0000 7fddac881700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.026+0000 7fddad883700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 0x7fdda80c15e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fdda8066c40 tx=0x7fdda0009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.026+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda001d630 con 0x7fdda8152e40 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.026+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda001d790 con 0x7fdda8152e40 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.026+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda80bfff0 con 0x7fdda8152e40 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.026+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda001db50 con 0x7fdda8152e40 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.026+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda80ad240 con 0x7fdda8152e40 2023-05-16T07:02:25.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.027+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda80ad3d0 con 0x7fdda8152e40 2023-05-16T07:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.027+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdda0026410 con 0x7fdda8152e40 2023-05-16T07:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.027+0000 7fdd9d7fa700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdd8c05ed60 0x7fdd8c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.028+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fdda007b980 con 0x7fdda8152e40 2023-05-16T07:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.028+0000 7fddad082700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdd8c05ed60 0x7fdd8c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.028+0000 7fddad082700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdd8c05ed60 0x7fdd8c061220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdda4009c30 tx=0x7fdda4009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.032+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdda004d0a0 con 0x7fdda8152e40 2023-05-16T07:02:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.194+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdda806c350 con 0x7fdda8152e40 2023-05-16T07:02:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.883+0000 7fdd9d7fa700 1 -- 172.21.15.92:0/2409635571 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdda00506c0 con 0x7fdda8152e40 2023-05-16T07:02:25.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.886+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdd8c05ed60 msgr2=0x7fdd8c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:25.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.886+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdd8c05ed60 0x7fdd8c061220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdda4009c30 tx=0x7fdda4009060).stop 2023-05-16T07:02:25.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.886+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 msgr2=0x7fdda80c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:25.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.886+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 0x7fdda80c15e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fdda8066c40 tx=0x7fdda0009780).stop 2023-05-16T07:02:25.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.890+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 shutdown_connections 2023-05-16T07:02:25.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.890+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdd8c05ed60 0x7fdd8c061220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.890+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdda8152e40 0x7fdda80c15e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.890+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdda80622b0 0x7fdda80c07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.890+0000 7fddaf2e6700 1 --2- 172.21.15.92:0/2409635571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdda814f990 0x7fdda80acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:25.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.890+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 >> 172.21.15.92:0/2409635571 conn(0x7fdda805e820 msgr2=0x7fdda80c3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.893+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 shutdown_connections 2023-05-16T07:02:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:25.894+0000 7fddaf2e6700 1 -- 172.21.15.92:0/2409635571 wait complete. 2023-05-16T07:02:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:02:25.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:02:26.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.029+0000 7f1233889700 1 Processor -- start 2023-05-16T07:02:26.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.029+0000 7f1233889700 1 -- start start 2023-05-16T07:02:26.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1233889700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f122c154090 0x7f122c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1233889700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c068160 0x7f122c0685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1233889700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c0c94d0 con 0x7f122c154470 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1233889700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c0cb7e0 con 0x7f122c154090 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1233889700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c068aa0 con 0x7f122c068160 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1231625700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f122c154090 0x7f122c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1231e26700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c068160 0x7f122c0685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1231625700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f122c154090 0x7f122c0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44048/0 (socket says 172.21.15.92:44048) 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1231625700 1 -- 172.21.15.92:0/3866005801 learned_addr learned my addr 172.21.15.92:0/3866005801 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.030+0000 7f1230e24700 1 --1- 172.21.15.92:0/3866005801 >> v1:172.21.15.158:6789/0 conn(0x7f122c154470 0x7f122c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:54758/0 (socket says 172.21.15.92:54758) 2023-05-16T07:02:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1231625700 1 -- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c068160 msgr2=0x7f122c0685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1231625700 1 --2- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c068160 0x7f122c0685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1231625700 1 -- 172.21.15.92:0/3866005801 >> v1:172.21.15.158:6789/0 conn(0x7f122c154470 legacy=0x7f122c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f122bfff700 1 -- 172.21.15.92:0/3866005801 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242998563 0 0) 0x7f122c0c94d0 con 0x7f122c154470 2023-05-16T07:02:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1231625700 1 -- 172.21.15.92:0/3866005801 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0c88c0 con 0x7f122c154090 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1231e26700 1 --2- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c068160 0x7f122c0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1231625700 1 --2- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154090 0x7f122c0aaca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f121c009f20 tx=0x7f121c009840).ready entity=mon.1 client_cookie=783a849e32c934e3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f122bfff700 1 -- 172.21.15.92:0/3866005801 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00e070 con 0x7f122c154090 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f122bfff700 1 -- 172.21.15.92:0/3866005801 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c00b9d0 con 0x7f122c154090 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f122bfff700 1 -- 172.21.15.92:0/3866005801 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00bd50 con 0x7f122c154090 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.031+0000 7f1233889700 1 -- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154090 msgr2=0x7f122c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 --2- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154090 0x7f122c0aaca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f121c009f20 tx=0x7f121c009840).stop 2023-05-16T07:02:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 -- 172.21.15.92:0/3866005801 shutdown_connections 2023-05-16T07:02:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 --2- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154090 0x7f122c0aaca0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 --2- 172.21.15.92:0/3866005801 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c068160 0x7f122c0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 -- 172.21.15.92:0/3866005801 >> 172.21.15.92:0/3866005801 conn(0x7f122c067920 msgr2=0x7f122c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 -- 172.21.15.92:0/3866005801 shutdown_connections 2023-05-16T07:02:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.032+0000 7f1233889700 1 -- 172.21.15.92:0/3866005801 wait complete. 2023-05-16T07:02:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.033+0000 7f1233889700 1 Processor -- start 2023-05-16T07:02:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.033+0000 7f1233889700 1 -- start start 2023-05-16T07:02:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 0x7f122c1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c154090 0x7f122c1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154470 0x7f122c0c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0c3d70 con 0x7f122c068160 2023-05-16T07:02:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1539c0 con 0x7f122c154470 2023-05-16T07:02:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0c3160 con 0x7f122c154090 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 0x7f122c1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 0x7f122c1502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40416/0 (socket says 172.21.15.92:40416) 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 -- 172.21.15.92:0/3876656057 learned_addr learned my addr 172.21.15.92:0/3876656057 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231e26700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154470 0x7f122c0c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1230e24700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c154090 0x7f122c1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 -- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c154090 msgr2=0x7f122c1534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c154090 0x7f122c1534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 -- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154470 msgr2=0x7f122c0c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154470 0x7f122c0c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 -- 172.21.15.92:0/3876656057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240092a0 con 0x7f122c068160 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1230e24700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c154090 0x7f122c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1231625700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 0x7f122c1502a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f121c00b610 tx=0x7f121c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00b640 con 0x7f122c068160 2023-05-16T07:02:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c00bb40 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c009ae0 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c01e6a0 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.034+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c0c3720 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.035+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c0c4050 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.036+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f121c025030 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.036+0000 7f12297fa700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121805ed90 0x7f1218061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.036+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f121c07b110 con 0x7f122c068160 2023-05-16T07:02:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.036+0000 7f1230e24700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121805ed90 0x7f1218061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.037+0000 7f1230e24700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121805ed90 0x7f1218061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f122c1462d0 tx=0x7f1220007c40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.040+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f121c04c710 con 0x7f122c068160 2023-05-16T07:02:26.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.243+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f122c0c71c0 con 0x7f122c068160 2023-05-16T07:02:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.250+0000 7f12297fa700 1 -- 172.21.15.92:0/3876656057 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f121c023070 con 0x7f122c068160 2023-05-16T07:02:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121805ed90 msgr2=0x7f1218061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121805ed90 0x7f1218061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f122c1462d0 tx=0x7f1220007c40).stop 2023-05-16T07:02:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 msgr2=0x7f122c1502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 0x7f122c1502a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f121c00b610 tx=0x7f121c009840).stop 2023-05-16T07:02:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 shutdown_connections 2023-05-16T07:02:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121805ed90 0x7f1218061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f122c154470 0x7f122c0c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f122c068160 0x7f122c1502a0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0x7f121c00b610 tx=0x7f121c009840).stop 2023-05-16T07:02:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 --2- 172.21.15.92:0/3876656057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f122c154090 0x7f122c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 >> 172.21.15.92:0/3876656057 conn(0x7f122c067920 msgr2=0x7f122c0c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:26.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 shutdown_connections 2023-05-16T07:02:26.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.264+0000 7f1233889700 1 -- 172.21.15.92:0/3876656057 wait complete. 2023-05-16T07:02:26.271 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 3 and data pool 4 2023-05-16T07:02:26.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:26.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.391+0000 7fb38759e700 1 Processor -- start 2023-05-16T07:02:26.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.391+0000 7fb38759e700 1 -- start start 2023-05-16T07:02:26.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb38759e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb3880aada0 0x7fb388155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb38759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3880c8470 0x7fb3880c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb38759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880c6910 con 0x7fb3880c7290 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb38759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb388155ef0 con 0x7fb3880c8470 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb38759e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3881561d0 con 0x7fb3880aada0 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb385d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3880c8470 0x7fb3880c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb386d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3880c7290 0x7fb3880a76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51184/0 (socket says 172.21.15.92:51184) 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb385d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3880c8470 0x7fb3880c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40436/0 (socket says 172.21.15.92:40436) 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb386d9d700 1 -- 172.21.15.92:0/1951056261 learned_addr learned my addr 172.21.15.92:0/1951056261 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.392+0000 7fb38659c700 1 --2- 172.21.15.92:0/1951056261 >> v2:172.21.15.158:3300/0 conn(0x7fb3880aada0 0x7fb388155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb385d9b700 1 -- 172.21.15.92:0/1951056261 >> v2:172.21.15.158:3300/0 conn(0x7fb3880aada0 msgr2=0x7fb388155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb385d9b700 1 --2- 172.21.15.92:0/1951056261 >> v2:172.21.15.158:3300/0 conn(0x7fb3880aada0 0x7fb388155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb385d9b700 1 -- 172.21.15.92:0/1951056261 >> v1:172.21.15.92:6789/0 conn(0x7fb3880c7290 legacy=0x7fb3880a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb38559a700 1 -- 172.21.15.92:0/1951056261 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729064954 0 0) 0x7fb3880c6910 con 0x7fb3880c7290 2023-05-16T07:02:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb385d9b700 1 -- 172.21.15.92:0/1951056261 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880c65a0 con 0x7fb3880c8470 2023-05-16T07:02:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb385d9b700 1 --2- 172.21.15.92:0/1951056261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb3880c6da0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb388060680 tx=0x7fb37c009060).ready entity=mon.0 client_cookie=540f5d52a9338e2b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb38559a700 1 -- 172.21.15.92:0/1951056261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00f510 con 0x7fb3880c8470 2023-05-16T07:02:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb38559a700 1 -- 172.21.15.92:0/1951056261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00f670 con 0x7fb3880c8470 2023-05-16T07:02:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb38559a700 1 -- 172.21.15.92:0/1951056261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00f9f0 con 0x7fb3880c8470 2023-05-16T07:02:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb38759e700 1 -- 172.21.15.92:0/1951056261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 msgr2=0x7fb3880c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.393+0000 7fb38759e700 1 --2- 172.21.15.92:0/1951056261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb3880c6da0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb388060680 tx=0x7fb37c009060).stop 2023-05-16T07:02:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.394+0000 7fb38759e700 1 -- 172.21.15.92:0/1951056261 shutdown_connections 2023-05-16T07:02:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.394+0000 7fb38759e700 1 --2- 172.21.15.92:0/1951056261 >> v2:172.21.15.158:3300/0 conn(0x7fb3880aada0 0x7fb388155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.394+0000 7fb38759e700 1 --2- 172.21.15.92:0/1951056261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb3880c6da0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.394+0000 7fb38759e700 1 -- 172.21.15.92:0/1951056261 >> 172.21.15.92:0/1951056261 conn(0x7fb38805e820 msgr2=0x7fb3880622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.394+0000 7fb38759e700 1 -- 172.21.15.92:0/1951056261 shutdown_connections 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.394+0000 7fb38759e700 1 -- 172.21.15.92:0/1951056261 wait complete. 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 Processor -- start 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 -- start start 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb3880aada0 0x7fb3881538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3880c7290 0x7fb3880ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb388162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880ad830 con 0x7fb3880c8470 2023-05-16T07:02:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881630e0 con 0x7fb3880c7290 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.395+0000 7fb38759e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880addd0 con 0x7fb3880aada0 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb388162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb388162100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40442/0 (socket says 172.21.15.92:40442) 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 -- 172.21.15.92:0/4108337969 learned_addr learned my addr 172.21.15.92:0/4108337969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb385d9b700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3880c7290 0x7fb3880ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb38659c700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb3880aada0 0x7fb3881538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 -- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb3880aada0 msgr2=0x7fb3881538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb3880aada0 0x7fb3881538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 -- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3880c7290 msgr2=0x7fb3880ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3880c7290 0x7fb3880ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 -- 172.21.15.92:0/4108337969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0092a0 con 0x7fb3880c8470 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb386d9d700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb388162100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb38000c7e0 tx=0x7fb380007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380005d30 con 0x7fb3880c8470 2023-05-16T07:02:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380005e90 con 0x7fb3880c8470 2023-05-16T07:02:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb388162b20 con 0x7fb3880c8470 2023-05-16T07:02:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380007320 con 0x7fb3880c8470 2023-05-16T07:02:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.396+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3880adae0 con 0x7fb3880c8470 2023-05-16T07:02:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.397+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb388153e70 con 0x7fb3880c8470 2023-05-16T07:02:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.397+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb380010030 con 0x7fb3880c8470 2023-05-16T07:02:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.398+0000 7fb3767fc700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb36805ed60 0x7fb368061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.398+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb380079ad0 con 0x7fb3880c8470 2023-05-16T07:02:26.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.398+0000 7fb38659c700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb36805ed60 0x7fb368061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.399+0000 7fb38659c700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb36805ed60 0x7fb368061220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb3880ad620 tx=0x7fb378008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.402+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb388153e70 con 0x7fb3880c8470 2023-05-16T07:02:26.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.566+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb388153e70 con 0x7fb3880c8470 2023-05-16T07:02:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.567+0000 7fb3767fc700 1 -- 172.21.15.92:0/4108337969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16989 (secure 0 0 0) 0x7fb388153e70 con 0x7fb3880c8470 2023-05-16T07:02:26.571 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:26.571 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:26.244399+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb36805ed60 msgr2=0x7fb368061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb36805ed60 0x7fb368061220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb3880ad620 tx=0x7fb378008040).stop 2023-05-16T07:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 msgr2=0x7fb388162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb388162100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb38000c7e0 tx=0x7fb380007f90).stop 2023-05-16T07:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 shutdown_connections 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb36805ed60 0x7fb368061220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb3880aada0 0x7fb3881538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3880c8470 0x7fb388162100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 --2- 172.21.15.92:0/4108337969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3880c7290 0x7fb3880ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 >> 172.21.15.92:0/4108337969 conn(0x7fb38805e820 msgr2=0x7fb3880be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 shutdown_connections 2023-05-16T07:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.569+0000 7fb38759e700 1 -- 172.21.15.92:0/4108337969 wait complete. 2023-05-16T07:02:26.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.690+0000 7fb770af5700 1 Processor -- start 2023-05-16T07:02:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.690+0000 7fb770af5700 1 -- start start 2023-05-16T07:02:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb770af5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c13b550 0x7fb76c0c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb770af5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb76c13b930 0x7fb76c0abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb770af5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb76c062c80 0x7fb76c063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb770af5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76c0c5400 con 0x7fb76c062c80 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb770af5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76c0bdc30 con 0x7fb76c13b930 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb770af5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c063130 con 0x7fb76c13b550 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb76ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb76c062c80 0x7fb76c063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb76ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb76c062c80 0x7fb76c063720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40448/0 (socket says 172.21.15.92:40448) 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb76ad9d700 1 -- 172.21.15.92:0/1707495892 learned_addr learned my addr 172.21.15.92:0/1707495892 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb769d9b700 1 --2- 172.21.15.92:0/1707495892 >> v2:172.21.15.158:3300/0 conn(0x7fb76c13b930 0x7fb76c0abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.691+0000 7fb76a59c700 1 --2- 172.21.15.92:0/1707495892 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c13b550 0x7fb76c0c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76ad9d700 1 -- 172.21.15.92:0/1707495892 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c13b550 msgr2=0x7fb76c0c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76ad9d700 1 --2- 172.21.15.92:0/1707495892 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c13b550 0x7fb76c0c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76ad9d700 1 -- 172.21.15.92:0/1707495892 >> v2:172.21.15.158:3300/0 conn(0x7fb76c13b930 msgr2=0x7fb76c0abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76ad9d700 1 --2- 172.21.15.92:0/1707495892 >> v2:172.21.15.158:3300/0 conn(0x7fb76c13b930 0x7fb76c0abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76ad9d700 1 -- 172.21.15.92:0/1707495892 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0c4e40 con 0x7fb76c062c80 2023-05-16T07:02:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76ad9d700 1 --2- 172.21.15.92:0/1707495892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c062c80 0x7fb76c063720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb76c065860 tx=0x7fb764009840).ready entity=mon.0 client_cookie=1eb3597826a0e77a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76959a700 1 -- 172.21.15.92:0/1707495892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400e070 con 0x7fb76c062c80 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76959a700 1 -- 172.21.15.92:0/1707495892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76400b970 con 0x7fb76c062c80 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb76959a700 1 -- 172.21.15.92:0/1707495892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400bcf0 con 0x7fb76c062c80 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 -- 172.21.15.92:0/1707495892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c062c80 msgr2=0x7fb76c063720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 --2- 172.21.15.92:0/1707495892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c062c80 0x7fb76c063720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb76c065860 tx=0x7fb764009840).stop 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 -- 172.21.15.92:0/1707495892 shutdown_connections 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 --2- 172.21.15.92:0/1707495892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c062c80 0x7fb76c063720 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 --2- 172.21.15.92:0/1707495892 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c13b550 0x7fb76c0c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 --2- 172.21.15.92:0/1707495892 >> v2:172.21.15.158:3300/0 conn(0x7fb76c13b930 0x7fb76c0abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 -- 172.21.15.92:0/1707495892 >> 172.21.15.92:0/1707495892 conn(0x7fb76c0c8aa0 msgr2=0x7fb76c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:26.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 -- 172.21.15.92:0/1707495892 shutdown_connections 2023-05-16T07:02:26.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.692+0000 7fb770af5700 1 -- 172.21.15.92:0/1707495892 wait complete. 2023-05-16T07:02:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.693+0000 7fb770af5700 1 Processor -- start 2023-05-16T07:02:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 -- start start 2023-05-16T07:02:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c062c80 0x7fb76c067f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb76c13b550 0x7fb76c157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 0x7fb76c0aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c069900 con 0x7fb76c13b930 2023-05-16T07:02:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c068860 con 0x7fb76c13b550 2023-05-16T07:02:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb770af5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c068b40 con 0x7fb76c062c80 2023-05-16T07:02:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb76ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 0x7fb76c0aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb76ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 0x7fb76c0aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40454/0 (socket says 172.21.15.92:40454) 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb76ad9d700 1 -- 172.21.15.92:0/2335868927 learned_addr learned my addr 172.21.15.92:0/2335868927 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb76a59c700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c062c80 0x7fb76c067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.694+0000 7fb769d9b700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb76c13b550 0x7fb76c157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb76ad9d700 1 -- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c062c80 msgr2=0x7fb76c067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb76ad9d700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c062c80 0x7fb76c067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb76ad9d700 1 -- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb76c13b550 msgr2=0x7fb76c157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb76ad9d700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb76c13b550 0x7fb76c157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb76ad9d700 1 -- 172.21.15.92:0/2335868927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764009ae0 con 0x7fb76c13b930 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb76ad9d700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 0x7fb76c0aab90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fb76400b610 tx=0x7fb764009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400bee0 con 0x7fb76c13b930 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76400bcf0 con 0x7fb76c13b930 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c068e20 con 0x7fb76c13b930 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76401d6b0 con 0x7fb76c13b930 2023-05-16T07:02:26.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.695+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c0693e0 con 0x7fb76c13b930 2023-05-16T07:02:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.696+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76c06c350 con 0x7fb76c13b930 2023-05-16T07:02:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.696+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb764024030 con 0x7fb76c13b930 2023-05-16T07:02:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.697+0000 7fb75a7fc700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb74c05ed60 0x7fb74c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.697+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb76407ad20 con 0x7fb76c13b930 2023-05-16T07:02:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.697+0000 7fb76a59c700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb74c05ed60 0x7fb74c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.697+0000 7fb76a59c700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb74c05ed60 0x7fb74c061220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb76c135f70 tx=0x7fb75c009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.701+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb76404c370 con 0x7fb76c13b930 2023-05-16T07:02:26.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.862+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb76c0c5120 con 0x7fb76c13b930 2023-05-16T07:02:26.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.863+0000 7fb75a7fc700 1 -- 172.21.15.92:0/2335868927 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16989 (secure 0 0 0) 0x7fb764022030 con 0x7fb76c13b930 2023-05-16T07:02:26.867 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:26.867 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:26.244399+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:26.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb74c05ed60 msgr2=0x7fb74c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb74c05ed60 0x7fb74c061220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb76c135f70 tx=0x7fb75c009060).stop 2023-05-16T07:02:26.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 msgr2=0x7fb76c0aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:26.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 0x7fb76c0aab90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fb76400b610 tx=0x7fb764009840).stop 2023-05-16T07:02:26.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 shutdown_connections 2023-05-16T07:02:26.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb74c05ed60 0x7fb74c061220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb76c062c80 0x7fb76c067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb76c13b930 0x7fb76c0aab90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 --2- 172.21.15.92:0/2335868927 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb76c13b550 0x7fb76c157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 >> 172.21.15.92:0/2335868927 conn(0x7fb76c0c8aa0 msgr2=0x7fb76c0adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:26.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 shutdown_connections 2023-05-16T07:02:26.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.865+0000 7fb770af5700 1 -- 172.21.15.92:0/2335868927 wait complete. 2023-05-16T07:02:26.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:02:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.991+0000 7faea09d1700 1 Processor -- start 2023-05-16T07:02:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.991+0000 7faea09d1700 1 -- start start 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.992+0000 7faea09d1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c153f90 0x7fae9c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.992+0000 7faea09d1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fae9c0c5d00 0x7fae9c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.992+0000 7faea09d1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c0c6670 con 0x7fae9c0c0150 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.992+0000 7faea09d1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c069f60 con 0x7fae9c0c5d00 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.992+0000 7faea09d1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c06a9e0 con 0x7fae9c153f90 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9b59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fae9c0c0150 0x7fae9c0c05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51210/0 (socket says 172.21.15.92:51210) 2023-05-16T07:02:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9b59e700 1 -- 172.21.15.92:0/3709889450 learned_addr learned my addr 172.21.15.92:0/3709889450 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9a59c700 1 --2- 172.21.15.92:0/3709889450 >> v2:172.21.15.158:3300/0 conn(0x7fae9c0c5d00 0x7fae9c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9ad9d700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c153f90 0x7fae9c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae99d9b700 1 -- 172.21.15.92:0/3709889450 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874892988 0 0) 0x7fae9c0c6670 con 0x7fae9c0c0150 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae99d9b700 1 -- 172.21.15.92:0/3709889450 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae840033f0 con 0x7fae9c0c0150 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9a59c700 1 -- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c153f90 msgr2=0x7fae9c0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9a59c700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c153f90 0x7fae9c0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9a59c700 1 -- 172.21.15.92:0/3709889450 >> v1:172.21.15.92:6789/0 conn(0x7fae9c0c0150 legacy=0x7fae9c0c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9a59c700 1 -- 172.21.15.92:0/3709889450 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c06a140 con 0x7fae9c0c5d00 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9ad9d700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c153f90 0x7fae9c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae9a59c700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c0c5d00 0x7fae9c0c6180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fae90009760 tx=0x7fae90009060).ready entity=mon.1 client_cookie=5b11cd946f5a937 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae99d9b700 1 -- 172.21.15.92:0/3709889450 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000d4a0 con 0x7fae9c0c5d00 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae99d9b700 1 -- 172.21.15.92:0/3709889450 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9000d600 con 0x7fae9c0c5d00 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.993+0000 7fae99d9b700 1 -- 172.21.15.92:0/3709889450 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000d980 con 0x7fae9c0c5d00 2023-05-16T07:02:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 -- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c0c5d00 msgr2=0x7fae9c0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c0c5d00 0x7fae9c0c6180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fae90009760 tx=0x7fae90009060).stop 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 -- 172.21.15.92:0/3709889450 shutdown_connections 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c153f90 0x7fae9c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 --2- 172.21.15.92:0/3709889450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c0c5d00 0x7fae9c0c6180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 -- 172.21.15.92:0/3709889450 >> 172.21.15.92:0/3709889450 conn(0x7fae9c0cb600 msgr2=0x7fae9c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 -- 172.21.15.92:0/3709889450 shutdown_connections 2023-05-16T07:02:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.994+0000 7faea09d1700 1 -- 172.21.15.92:0/3709889450 wait complete. 2023-05-16T07:02:27.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.995+0000 7faea09d1700 1 Processor -- start 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.995+0000 7faea09d1700 1 -- start start 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 0x7fae9c0c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c0c5d00 0x7fae9c1646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c153f90 0x7fae9c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0c3090 con 0x7fae9c0c0150 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c164ba0 con 0x7fae9c153f90 2023-05-16T07:02:27.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0c3e90 con 0x7fae9c0c5d00 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 0x7fae9c0c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 0x7fae9c0c2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40464/0 (socket says 172.21.15.92:40464) 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 -- 172.21.15.92:0/4098233424 learned_addr learned my addr 172.21.15.92:0/4098233424 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9a59c700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c0c5d00 0x7fae9c1646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9b59e700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c153f90 0x7fae9c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 -- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c0c5d00 msgr2=0x7fae9c1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c0c5d00 0x7fae9c1646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 -- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c153f90 msgr2=0x7fae9c14dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c153f90 0x7fae9c14dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 -- 172.21.15.92:0/4098233424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0092a0 con 0x7fae9c0c0150 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae9ad9d700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 0x7fae9c0c2ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fae8c00ea20 tx=0x7fae8c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00c850 con 0x7fae9c0c0150 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c00c9b0 con 0x7fae9c0c0150 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae900092a0 con 0x7fae9c0c0150 2023-05-16T07:02:27.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00cd30 con 0x7fae9c0c0150 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.996+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c0c3560 con 0x7fae9c0c0150 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.997+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c083120 con 0x7fae9c0c0150 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.998+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fae8c022070 con 0x7fae9c0c0150 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.998+0000 7fae82ffd700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae8805ed90 0x7fae88061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.998+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fae8c07a010 con 0x7fae9c0c0150 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.998+0000 7fae9a59c700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae8805ed90 0x7fae88061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:26.998+0000 7fae9a59c700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae8805ed90 0x7fae88061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fae90009760 tx=0x7fae90016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:27.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.002+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fae8c04b5b0 con 0x7fae9c0c0150 2023-05-16T07:02:27.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.211+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae9c0adc10 con 0x7fae9c0c0150 2023-05-16T07:02:27.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.248+0000 7fae82ffd700 1 -- 172.21.15.92:0/4098233424 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae8c04ebd0 con 0x7fae9c0c0150 2023-05-16T07:02:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae8805ed90 msgr2=0x7fae88061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae8805ed90 0x7fae88061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fae90009760 tx=0x7fae90016000).stop 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 msgr2=0x7fae9c0c2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 0x7fae9c0c2ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fae8c00ea20 tx=0x7fae8c009060).stop 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 shutdown_connections 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae8805ed90 0x7fae88061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae9c153f90 0x7fae9c14dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae9c0c0150 0x7fae9c0c2ba0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 --2- 172.21.15.92:0/4098233424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae9c0c5d00 0x7fae9c1646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 >> 172.21.15.92:0/4098233424 conn(0x7fae9c0cb600 msgr2=0x7fae9c068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 shutdown_connections 2023-05-16T07:02:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.250+0000 7faea09d1700 1 -- 172.21.15.92:0/4098233424 wait complete. 2023-05-16T07:02:27.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:02:27.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.376+0000 7f72cbfc6700 1 Processor -- start 2023-05-16T07:02:27.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.376+0000 7f72cbfc6700 1 -- start start 2023-05-16T07:02:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72cbfc6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72c40be280 0x7f72c41542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72cbfc6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c4155870 con 0x7f72c4062cc0 2023-05-16T07:02:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72cbfc6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c41548b0 con 0x7f72c4153ee0 2023-05-16T07:02:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72cbfc6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c4063150 con 0x7f72c40be280 2023-05-16T07:02:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72c9561700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72c40be280 0x7f72c41542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72ca563700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f72c4062cc0 0x7f72c4064e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51222/0 (socket says 172.21.15.92:51222) 2023-05-16T07:02:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72c9561700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72c40be280 0x7f72c41542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40474/0 (socket says 172.21.15.92:40474) 2023-05-16T07:02:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.377+0000 7f72ca563700 1 -- 172.21.15.92:0/3709842272 learned_addr learned my addr 172.21.15.92:0/3709842272 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c9561700 1 -- 172.21.15.92:0/3709842272 >> v1:172.21.15.158:6789/0 conn(0x7f72c4153ee0 legacy=0x7f72c40bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c9561700 1 -- 172.21.15.92:0/3709842272 >> v1:172.21.15.92:6789/0 conn(0x7f72c4062cc0 legacy=0x7f72c4064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c9561700 1 -- 172.21.15.92:0/3709842272 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4154b90 con 0x7f72c40be280 2023-05-16T07:02:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c8d60700 1 -- 172.21.15.92:0/3709842272 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760833586 0 0) 0x7f72c4155870 con 0x7f72c4062cc0 2023-05-16T07:02:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c9561700 1 --2- 172.21.15.92:0/3709842272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40be280 0x7f72c41542c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f72b8009760 tx=0x7f72b8009060).ready entity=mon.0 client_cookie=f12e4793e605364f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c8d60700 1 -- 172.21.15.92:0/3709842272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800b580 con 0x7f72c40be280 2023-05-16T07:02:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c8d60700 1 -- 172.21.15.92:0/3709842272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800b6e0 con 0x7f72c40be280 2023-05-16T07:02:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72c8d60700 1 -- 172.21.15.92:0/3709842272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800ba60 con 0x7f72c40be280 2023-05-16T07:02:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72cbfc6700 1 -- 172.21.15.92:0/3709842272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40be280 msgr2=0x7f72c41542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/3709842272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40be280 0x7f72c41542c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f72b8009760 tx=0x7f72b8009060).stop 2023-05-16T07:02:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72cbfc6700 1 -- 172.21.15.92:0/3709842272 shutdown_connections 2023-05-16T07:02:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.378+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/3709842272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40be280 0x7f72c41542c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.379+0000 7f72cbfc6700 1 -- 172.21.15.92:0/3709842272 >> 172.21.15.92:0/3709842272 conn(0x7f72c40c8420 msgr2=0x7f72c40c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.379+0000 7f72cbfc6700 1 -- 172.21.15.92:0/3709842272 shutdown_connections 2023-05-16T07:02:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.379+0000 7f72cbfc6700 1 -- 172.21.15.92:0/3709842272 wait complete. 2023-05-16T07:02:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 Processor -- start 2023-05-16T07:02:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 -- start start 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f72c4062cc0 0x7f72c4168930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f72c40be280 0x7f72c40c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 0x7f72c4164080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c414dfb0 con 0x7f72c4153ee0 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40be9f0 con 0x7f72c40be280 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72cbfc6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40bec40 con 0x7f72c4062cc0 2023-05-16T07:02:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.380+0000 7f72ca563700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 0x7f72c4164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 0x7f72c4164080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40482/0 (socket says 172.21.15.92:40482) 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 -- 172.21.15.92:0/2570211522 learned_addr learned my addr 172.21.15.92:0/2570211522 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72c9561700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f72c40be280 0x7f72c40c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72c9d62700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f72c4062cc0 0x7f72c4168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 -- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f72c4062cc0 msgr2=0x7f72c4168930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f72c4062cc0 0x7f72c4168930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 -- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f72c40be280 msgr2=0x7f72c40c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:27.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f72c40be280 0x7f72c40c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 -- 172.21.15.92:0/2570211522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80092a0 con 0x7f72c4153ee0 2023-05-16T07:02:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72c9561700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f72c40be280 0x7f72c40c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72ca563700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 0x7f72c4164080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f72bc00c440 tx=0x7f72bc007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0072e0 con 0x7f72c4153ee0 2023-05-16T07:02:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc007440 con 0x7f72c4153ee0 2023-05-16T07:02:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c40bef20 con 0x7f72c4153ee0 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0057a0 con 0x7f72c4153ee0 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.381+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c40bf4e0 con 0x7f72c4153ee0 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.382+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c40bf6d0 con 0x7f72c4153ee0 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.382+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f72bc01a060 con 0x7f72c4153ee0 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.383+0000 7f72c1ffb700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f72b005ed60 0x7f72b0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.383+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f72bc079250 con 0x7f72c4153ee0 2023-05-16T07:02:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.383+0000 7f72c9d62700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f72b005ed60 0x7f72b0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.383+0000 7f72c9d62700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f72b005ed60 0x7f72b0061220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f72b4003dd0 tx=0x7f72b4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.387+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f72bc04a7f0 con 0x7f72c4153ee0 2023-05-16T07:02:27.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:27.593+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72c406c350 con 0x7f72c4153ee0 2023-05-16T07:02:28.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.254+0000 7f72c1ffb700 1 -- 172.21.15.92:0/2570211522 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72bc018020 con 0x7f72c4153ee0 2023-05-16T07:02:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.256+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f72b005ed60 msgr2=0x7f72b0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.256+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f72b005ed60 0x7f72b0061220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f72b4003dd0 tx=0x7f72b4009040).stop 2023-05-16T07:02:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.256+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 msgr2=0x7f72c4164080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.256+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 0x7f72c4164080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f72bc00c440 tx=0x7f72bc007bb0).stop 2023-05-16T07:02:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 shutdown_connections 2023-05-16T07:02:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f72b005ed60 0x7f72b0061220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f72c4062cc0 0x7f72c4168930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c4153ee0 0x7f72c4164080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 --2- 172.21.15.92:0/2570211522 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f72c40be280 0x7f72c40c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 >> 172.21.15.92:0/2570211522 conn(0x7f72c40c8420 msgr2=0x7f72c40c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 shutdown_connections 2023-05-16T07:02:28.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.257+0000 7f72cbfc6700 1 -- 172.21.15.92:0/2570211522 wait complete. 2023-05-16T07:02:28.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:28.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.381+0000 7f7f059f2700 1 Processor -- start 2023-05-16T07:02:28.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.381+0000 7f7f059f2700 1 -- start start 2023-05-16T07:02:28.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7f059f2700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7f00154180 0x7f7f00063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7f059f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f00062fa0 0x7f7f00063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7f059f2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f00062b10 con 0x7f7f00153da0 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7f059f2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f00064100 con 0x7f7f00062fa0 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7f059f2700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f00068620 con 0x7f7f00154180 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7efeffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f00153da0 0x7f7f00069a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51234/0 (socket says 172.21.15.92:51234) 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7efeffd700 1 -- 172.21.15.92:0/2689976670 learned_addr learned my addr 172.21.15.92:0/2689976670 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:28.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7eff7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f00062fa0 0x7f7f00063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7eff7fe700 1 -- 172.21.15.92:0/2689976670 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154180 msgr2=0x7f7f00063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:28.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7efe7fc700 1 --2- 172.21.15.92:0/2689976670 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154180 0x7f7f00063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7efdffb700 1 -- 172.21.15.92:0/2689976670 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313504572 0 0) 0x7f7f00062b10 con 0x7f7f00153da0 2023-05-16T07:02:28.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7eff7fe700 1 --2- 172.21.15.92:0/2689976670 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154180 0x7f7f00063720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7eff7fe700 1 -- 172.21.15.92:0/2689976670 >> v1:172.21.15.92:6789/0 conn(0x7f7f00153da0 legacy=0x7f7f00069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7eff7fe700 1 -- 172.21.15.92:0/2689976670 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00068340 con 0x7f7f00062fa0 2023-05-16T07:02:28.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.382+0000 7f7eff7fe700 1 --2- 172.21.15.92:0/2689976670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00062fa0 0x7f7f00063c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f7f00065250 tx=0x7f7ef8009060).ready entity=mon.0 client_cookie=2c6a5373bf70eabe server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:28.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7efdffb700 1 -- 172.21.15.92:0/2689976670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800b580 con 0x7f7f00062fa0 2023-05-16T07:02:28.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7efdffb700 1 -- 172.21.15.92:0/2689976670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef800b6e0 con 0x7f7f00062fa0 2023-05-16T07:02:28.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7efdffb700 1 -- 172.21.15.92:0/2689976670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800ba60 con 0x7f7f00062fa0 2023-05-16T07:02:28.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 -- 172.21.15.92:0/2689976670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00062fa0 msgr2=0x7f7f00063c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 --2- 172.21.15.92:0/2689976670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00062fa0 0x7f7f00063c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f7f00065250 tx=0x7f7ef8009060).stop 2023-05-16T07:02:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 -- 172.21.15.92:0/2689976670 shutdown_connections 2023-05-16T07:02:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 --2- 172.21.15.92:0/2689976670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00062fa0 0x7f7f00063c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 --2- 172.21.15.92:0/2689976670 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154180 0x7f7f00063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 -- 172.21.15.92:0/2689976670 >> 172.21.15.92:0/2689976670 conn(0x7f7f000c8aa0 msgr2=0x7f7f000c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:28.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 -- 172.21.15.92:0/2689976670 shutdown_connections 2023-05-16T07:02:28.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.383+0000 7f7f059f2700 1 -- 172.21.15.92:0/2689976670 wait complete. 2023-05-16T07:02:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.384+0000 7f7f059f2700 1 Processor -- start 2023-05-16T07:02:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.384+0000 7f7f059f2700 1 -- start start 2023-05-16T07:02:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7f059f2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00062fa0 0x7f7f0014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7f059f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 0x7f7f00150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7f059f2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f00154180 0x7f7f00163e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7f059f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000c58e0 con 0x7f7f00153da0 2023-05-16T07:02:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7f059f2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0014f240 con 0x7f7f00062fa0 2023-05-16T07:02:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7f059f2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00151840 con 0x7f7f00154180 2023-05-16T07:02:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 0x7f7f00150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 0x7f7f00150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40494/0 (socket says 172.21.15.92:40494) 2023-05-16T07:02:28.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 -- 172.21.15.92:0/355744858 learned_addr learned my addr 172.21.15.92:0/355744858 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:28.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efeffd700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00062fa0 0x7f7f0014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7eff7fe700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f00154180 0x7f7f00163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 -- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f00154180 msgr2=0x7f7f00163e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f00154180 0x7f7f00163e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 -- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00062fa0 msgr2=0x7f7f0014ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.385+0000 7f7efe7fc700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00062fa0 0x7f7f0014ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7efe7fc700 1 -- 172.21.15.92:0/355744858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80092a0 con 0x7f7f00153da0 2023-05-16T07:02:28.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7eff7fe700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f00154180 0x7f7f00163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7efeffd700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00062fa0 0x7f7f0014ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7efe7fc700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 0x7f7f00150ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f7ee800c650 tx=0x7f7ee8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8010580 con 0x7f7f00153da0 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee80106e0 con 0x7f7f00153da0 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f00151b20 con 0x7f7f00153da0 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8010a60 con 0x7f7f00153da0 2023-05-16T07:02:28.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.386+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f000c5570 con 0x7f7f00153da0 2023-05-16T07:02:28.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.387+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0006c350 con 0x7f7f00153da0 2023-05-16T07:02:28.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.387+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7ee801b460 con 0x7f7f00153da0 2023-05-16T07:02:28.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.387+0000 7f7ef6ffd700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7ee005ed90 0x7f7ee0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.387+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ee8079c00 con 0x7f7f00153da0 2023-05-16T07:02:28.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.387+0000 7f7efeffd700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7ee005ed90 0x7f7ee0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.388+0000 7f7efeffd700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7ee005ed90 0x7f7ee0061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7ef0001e40 tx=0x7f7ef0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:28.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.391+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7ee807f020 con 0x7f7f00153da0 2023-05-16T07:02:28.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.595+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f00067160 con 0x7f7f00153da0 2023-05-16T07:02:28.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.595+0000 7f7ef6ffd700 1 -- 172.21.15.92:0/355744858 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f7ee804b140 con 0x7f7f00153da0 2023-05-16T07:02:28.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:28.600 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:28.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7ee005ed90 msgr2=0x7f7ee0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7ee005ed90 0x7f7ee0061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7ef0001e40 tx=0x7f7ef0009040).stop 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 msgr2=0x7f7f00150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 0x7f7f00150ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f7ee800c650 tx=0x7f7ee8008040).stop 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 shutdown_connections 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7ee005ed90 0x7f7ee0061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f00154180 0x7f7f00163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00062fa0 0x7f7f0014ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 --2- 172.21.15.92:0/355744858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00153da0 0x7f7f00150ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.597+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 >> 172.21.15.92:0/355744858 conn(0x7f7f000c8aa0 msgr2=0x7f7f000ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.598+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 shutdown_connections 2023-05-16T07:02:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.598+0000 7f7f059f2700 1 -- 172.21.15.92:0/355744858 wait complete. 2023-05-16T07:02:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:28.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.719+0000 7fdafa8ec700 1 Processor -- start 2023-05-16T07:02:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.719+0000 7fdafa8ec700 1 -- start start 2023-05-16T07:02:28.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.720+0000 7fdafa8ec700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdaf414e5e0 0x7fdaf4157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.720+0000 7fdafa8ec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40683d0 con 0x7fdaf414e200 2023-05-16T07:02:28.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.720+0000 7fdafa8ec700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40be060 con 0x7fdaf4157f40 2023-05-16T07:02:28.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.720+0000 7fdafa8ec700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf4137470 con 0x7fdaf414e5e0 2023-05-16T07:02:28.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.720+0000 7fdaf3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdaf414e200 0x7fdaf4066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51248/0 (socket says 172.21.15.92:51248) 2023-05-16T07:02:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf3fff700 1 -- 172.21.15.92:0/437533365 learned_addr learned my addr 172.21.15.92:0/437533365 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/437533365 >> v2:172.21.15.158:3300/0 conn(0x7fdaf414e5e0 0x7fdaf4157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/437533365 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259998249 0 0) 0x7fdaf40683d0 con 0x7fdaf414e200 2023-05-16T07:02:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/437533365 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdadc0033f0 con 0x7fdaf414e200 2023-05-16T07:02:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/437533365 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874922669 0 0) 0x7fdadc0033f0 con 0x7fdaf414e200 2023-05-16T07:02:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf37fe700 1 -- 172.21.15.92:0/437533365 >> v1:172.21.15.158:6789/0 conn(0x7fdaf4157f40 legacy=0x7fdaf415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf37fe700 1 -- 172.21.15.92:0/437533365 >> v1:172.21.15.92:6789/0 conn(0x7fdaf414e200 legacy=0x7fdaf4066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf37fe700 1 -- 172.21.15.92:0/437533365 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4067560 con 0x7fdaf414e5e0 2023-05-16T07:02:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/437533365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf414e5e0 0x7fdaf4157a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdae8009760 tx=0x7fdae8009060).ready entity=mon.1 client_cookie=24db033039f124fa server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/437533365 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4001470 con 0x7fdaf414e5e0 2023-05-16T07:02:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/437533365 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8009130 con 0x7fdaf414e5e0 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.721+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/437533365 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8012620 con 0x7fdaf414e5e0 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 -- 172.21.15.92:0/437533365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf414e5e0 msgr2=0x7fdaf4157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/437533365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf414e5e0 0x7fdaf4157a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdae8009760 tx=0x7fdae8009060).stop 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 -- 172.21.15.92:0/437533365 shutdown_connections 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/437533365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf414e5e0 0x7fdaf4157a50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 -- 172.21.15.92:0/437533365 >> 172.21.15.92:0/437533365 conn(0x7fdaf40c7bc0 msgr2=0x7fdaf40c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:28.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 -- 172.21.15.92:0/437533365 shutdown_connections 2023-05-16T07:02:28.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.722+0000 7fdafa8ec700 1 -- 172.21.15.92:0/437533365 wait complete. 2023-05-16T07:02:28.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.723+0000 7fdafa8ec700 1 Processor -- start 2023-05-16T07:02:28.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.723+0000 7fdafa8ec700 1 -- start start 2023-05-16T07:02:28.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 0x7fdaf4168610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdaf414e5e0 0x7fdaf40ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf4157f40 0x7fdaf41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40bed70 con 0x7fdaf414e200 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40bf050 con 0x7fdaf4157f40 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40bf330 con 0x7fdaf414e5e0 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 0x7fdaf4168610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 0x7fdaf4168610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40506/0 (socket says 172.21.15.92:40506) 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 -- 172.21.15.92:0/3598858016 learned_addr learned my addr 172.21.15.92:0/3598858016 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:28.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdaf414e5e0 0x7fdaf40ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 -- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdaf414e5e0 msgr2=0x7fdaf40ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf8e89700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf4157f40 0x7fdaf41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdaf414e5e0 0x7fdaf40ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 -- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf4157f40 msgr2=0x7fdaf41568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf4157f40 0x7fdaf41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 -- 172.21.15.92:0/3598858016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80092a0 con 0x7fdaf414e200 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdaf3fff700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 0x7fdaf4168610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fdae400c730 tx=0x7fdae4001aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:28.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4001b10 con 0x7fdaf414e200 2023-05-16T07:02:28.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae4007170 con 0x7fdaf414e200 2023-05-16T07:02:28.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf40bf610 con 0x7fdaf414e200 2023-05-16T07:02:28.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4007530 con 0x7fdaf414e200 2023-05-16T07:02:28.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.724+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf40bea20 con 0x7fdaf414e200 2023-05-16T07:02:28.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.725+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf40bf850 con 0x7fdaf414e200 2023-05-16T07:02:28.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.726+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdae401b060 con 0x7fdaf414e200 2023-05-16T07:02:28.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.726+0000 7fdadbfff700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdae005edc0 0x7fdae0061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:28.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.726+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdae4079530 con 0x7fdaf414e200 2023-05-16T07:02:28.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.726+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdae005edc0 0x7fdae0061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:28.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.727+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdae005edc0 0x7fdae0061280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdaf4167970 tx=0x7fdae8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:28.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.730+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdae404aad0 con 0x7fdaf414e200 2023-05-16T07:02:28.882 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-05-16T07:02:28.878+0000 7f1859941700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:02:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.893+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf40beeb0 con 0x7fdaf414e200 2023-05-16T07:02:28.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.894+0000 7fdadbfff700 1 -- 172.21.15.92:0/3598858016 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fdaf40beeb0 con 0x7fdaf414e200 2023-05-16T07:02:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:28.899 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:28.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdae005edc0 msgr2=0x7fdae0061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdae005edc0 0x7fdae0061280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdaf4167970 tx=0x7fdae8015000).stop 2023-05-16T07:02:28.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 msgr2=0x7fdaf4168610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:28.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 0x7fdaf4168610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fdae400c730 tx=0x7fdae4001aa0).stop 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 shutdown_connections 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdae005edc0 0x7fdae0061280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaf4157f40 0x7fdaf41568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf414e200 0x7fdaf4168610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 --2- 172.21.15.92:0/3598858016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdaf414e5e0 0x7fdaf40ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 >> 172.21.15.92:0/3598858016 conn(0x7fdaf40c7bc0 msgr2=0x7fdaf40a4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:28.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 shutdown_connections 2023-05-16T07:02:28.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:28.896+0000 7fdafa8ec700 1 -- 172.21.15.92:0/3598858016 wait complete. 2023-05-16T07:02:28.912 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:02:28.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:02:29.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.019+0000 7f29b62e1700 1 Processor -- start 2023-05-16T07:02:29.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.019+0000 7f29b62e1700 1 -- start start 2023-05-16T07:02:29.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b62e1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f29b01517f0 0x7f29b0156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b62e1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f29b0151bd0 0x7f29b0152a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b62e1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b00c8750 con 0x7f29b00a76f0 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b62e1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b0155d70 con 0x7f29b0151bd0 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b62e1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b00c8470 con 0x7f29b01517f0 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b4ade700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f29b0151bd0 0x7f29b0152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b5ae0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f29b00a76f0 0x7f29b00a7af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51262/0 (socket says 172.21.15.92:51262) 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b5ae0700 1 -- 172.21.15.92:0/2210343182 learned_addr learned my addr 172.21.15.92:0/2210343182 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:29.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b52df700 1 --2- 172.21.15.92:0/2210343182 >> v2:172.21.15.158:3300/0 conn(0x7f29b01517f0 0x7f29b0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b4ade700 1 -- 172.21.15.92:0/2210343182 >> v2:172.21.15.158:3300/0 conn(0x7f29b01517f0 msgr2=0x7f29b0156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b4ade700 1 --2- 172.21.15.92:0/2210343182 >> v2:172.21.15.158:3300/0 conn(0x7f29b01517f0 0x7f29b0156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b4ade700 1 -- 172.21.15.92:0/2210343182 >> v1:172.21.15.92:6789/0 conn(0x7f29b00a76f0 legacy=0x7f29b00a7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29b4ade700 1 -- 172.21.15.92:0/2210343182 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0155720 con 0x7f29b0151bd0 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.020+0000 7f29affff700 1 -- 172.21.15.92:0/2210343182 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614981112 0 0) 0x7f29b00c8750 con 0x7f29b00a76f0 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b4ade700 1 --2- 172.21.15.92:0/2210343182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b0151bd0 0x7f29b0152a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f29b0060670 tx=0x7f29a4009060).ready entity=mon.0 client_cookie=75a3b8afc8d733ca server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29affff700 1 -- 172.21.15.92:0/2210343182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a40090d0 con 0x7f29b0151bd0 2023-05-16T07:02:29.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29affff700 1 -- 172.21.15.92:0/2210343182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a400b580 con 0x7f29b0151bd0 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29affff700 1 -- 172.21.15.92:0/2210343182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a400b900 con 0x7f29b0151bd0 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 -- 172.21.15.92:0/2210343182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b0151bd0 msgr2=0x7f29b0152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 --2- 172.21.15.92:0/2210343182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b0151bd0 0x7f29b0152a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f29b0060670 tx=0x7f29a4009060).stop 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 -- 172.21.15.92:0/2210343182 shutdown_connections 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 --2- 172.21.15.92:0/2210343182 >> v2:172.21.15.158:3300/0 conn(0x7f29b01517f0 0x7f29b0156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 --2- 172.21.15.92:0/2210343182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b0151bd0 0x7f29b0152a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 -- 172.21.15.92:0/2210343182 >> 172.21.15.92:0/2210343182 conn(0x7f29b005e820 msgr2=0x7f29b00622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:29.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 -- 172.21.15.92:0/2210343182 shutdown_connections 2023-05-16T07:02:29.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.021+0000 7f29b62e1700 1 -- 172.21.15.92:0/2210343182 wait complete. 2023-05-16T07:02:29.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.022+0000 7f29b62e1700 1 Processor -- start 2023-05-16T07:02:29.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.022+0000 7f29b62e1700 1 -- start start 2023-05-16T07:02:29.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b62e1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29b00a76f0 0x7f29b00a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b62e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 0x7f29b0165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b62e1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29b0151bd0 0x7f29b0165610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b62e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b014b9a0 con 0x7f29b01517f0 2023-05-16T07:02:29.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b62e1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0165b00 con 0x7f29b00a76f0 2023-05-16T07:02:29.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b62e1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00bff70 con 0x7f29b0151bd0 2023-05-16T07:02:29.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b4ade700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 0x7f29b0165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b4ade700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 0x7f29b0165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40520/0 (socket says 172.21.15.92:40520) 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b4ade700 1 -- 172.21.15.92:0/1751176290 learned_addr learned my addr 172.21.15.92:0/1751176290 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b52df700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29b00a76f0 0x7f29b00a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.023+0000 7f29b5ae0700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29b0151bd0 0x7f29b0165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b4ade700 1 -- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29b0151bd0 msgr2=0x7f29b0165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b4ade700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29b0151bd0 0x7f29b0165610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b4ade700 1 -- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29b00a76f0 msgr2=0x7f29b00a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b4ade700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29b00a76f0 0x7f29b00a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b4ade700 1 -- 172.21.15.92:0/1751176290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40092a0 con 0x7f29b01517f0 2023-05-16T07:02:29.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b52df700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29b00a76f0 0x7f29b00a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:29.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b4ade700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 0x7f29b0165120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f29a4009fd0 tx=0x7f29a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a400b900 con 0x7f29b01517f0 2023-05-16T07:02:29.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a400bcc0 con 0x7f29b01517f0 2023-05-16T07:02:29.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a401b440 con 0x7f29b01517f0 2023-05-16T07:02:29.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b00aa250 con 0x7f29b01517f0 2023-05-16T07:02:29.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.024+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b00aa810 con 0x7f29b01517f0 2023-05-16T07:02:29.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.025+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b0047150 con 0x7f29b01517f0 2023-05-16T07:02:29.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.025+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f29a400f070 con 0x7f29b01517f0 2023-05-16T07:02:29.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.026+0000 7f29ad7fa700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299c05ed90 0x7f299c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.026+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f29a407b230 con 0x7f29b01517f0 2023-05-16T07:02:29.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.026+0000 7f29b52df700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299c05ed90 0x7f299c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.026+0000 7f29b52df700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299c05ed90 0x7f299c061250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f29a0009fd0 tx=0x7f29a000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.030+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f29a404c770 con 0x7f29b01517f0 2023-05-16T07:02:29.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.189+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29b0155a00 con 0x7f299c05ed90 2023-05-16T07:02:29.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.199+0000 7f29ad7fa700 1 -- 172.21.15.92:0/1751176290 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+325082 (secure 0 0 0) 0x7f29b0155a00 con 0x7f299c05ed90 2023-05-16T07:02:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:29.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299c05ed90 msgr2=0x7f299c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299c05ed90 0x7f299c061250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f29a0009fd0 tx=0x7f29a000b040).stop 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 msgr2=0x7f29b0165120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 0x7f29b0165120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f29a4009fd0 tx=0x7f29a4017000).stop 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 shutdown_connections 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299c05ed90 0x7f299c061250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29b0151bd0 0x7f29b0165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29b00a76f0 0x7f29b00a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 --2- 172.21.15.92:0/1751176290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29b01517f0 0x7f29b0165120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 >> 172.21.15.92:0/1751176290 conn(0x7f29b005e820 msgr2=0x7f29b0154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 shutdown_connections 2023-05-16T07:02:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.202+0000 7f29b62e1700 1 -- 172.21.15.92:0/1751176290 wait complete. 2023-05-16T07:02:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:02:29.217 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:02:27.451620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":39556,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":38144,"kb_avail":754935164,"statfs":{"total":773094113280,"available":773053607936,"internally_reserved":0,"allocated":1380352,"data_stored":439437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39059456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:22.259510+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:02:22.259510+0000","last_peered":"2023-05-16T07:02:22.259510+0000","last_clean":"2023-05-16T07:02:22.259510+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:02:22.259510+0000","last_undegraded":"2023-05-16T07:02:22.259510+0000","last_fullsized":"2023-05-16T07:02:22.259510+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.778771+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:02:21.778771+0000","last_peered":"2023-05-16T07:02:21.778771+0000","last_clean":"2023-05-16T07:02:21.778771+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:02:21.778771+0000","last_undegraded":"2023-05-16T07:02:21.778771+0000","last_fullsized":"2023-05-16T07:02:21.778771+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.777934+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:02:21.777934+0000","last_peered":"2023-05-16T07:02:21.777934+0000","last_clean":"2023-05-16T07:02:21.777934+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:02:21.777934+0000","last_undegraded":"2023-05-16T07:02:21.777934+0000","last_fullsized":"2023-05-16T07:02:21.777934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.777020+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:02:21.777020+0000","last_peered":"2023-05-16T07:02:21.777020+0000","last_clean":"2023-05-16T07:02:21.777020+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:02:21.777020+0000","last_undegraded":"2023-05-16T07:02:21.777020+0000","last_fullsized":"2023-05-16T07:02:21.777020+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.156291+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:02:21.156291+0000","last_peered":"2023-05-16T07:02:21.156291+0000","last_clean":"2023-05-16T07:02:21.156291+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:02:21.156291+0000","last_undegraded":"2023-05-16T07:02:21.156291+0000","last_fullsized":"2023-05-16T07:02:21.156291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.775370+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:02:21.775370+0000","last_peered":"2023-05-16T07:02:21.775370+0000","last_clean":"2023-05-16T07:02:21.775370+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:02:21.775370+0000","last_undegraded":"2023-05-16T07:02:21.775370+0000","last_fullsized":"2023-05-16T07:02:21.775370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:22.259995+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:02:22.259995+0000","last_peered":"2023-05-16T07:02:22.259995+0000","last_clean":"2023-05-16T07:02:22.259995+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:02:22.259995+0000","last_undegraded":"2023-05-16T07:02:22.259995+0000","last_fullsized":"2023-05-16T07:02:22.259995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.774949+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:02:21.774949+0000","last_peered":"2023-05-16T07:02:21.774949+0000","last_clean":"2023-05-16T07:02:21.774949+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:02:21.774949+0000","last_undegraded":"2023-05-16T07:02:21.774949+0000","last_fullsized":"2023-05-16T07:02:21.774949+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-05-16T07:02:21.776096+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:02:21.776096+0000","last_peered":"2023-05-16T07:02:21.776096+0000","last_clean":"2023-05-16T07:02:21.776096+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:02:21.776096+0000","last_undegraded":"2023-05-16T07:02:21.776096+0000","last_fullsized":"2023-05-16T07:02:21.776096+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:25.880105+0000","last_change":"2023-05-16T07:02:25.880105+0000","last_active":"2023-05-16T07:02:25.880105+0000","last_peered":"2023-05-16T07:02:25.880105+0000","last_clean":"2023-05-16T07:02:25.880105+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:25.880105+0000","last_undegraded":"2023-05-16T07:02:25.880105+0000","last_fullsized":"2023-05-16T07:02:25.880105+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:02:24.878069+0000","last_change":"2023-05-16T07:02:24.878069+0000","last_active":"2023-05-16T07:02:24.878069+0000","last_peered":"2023-05-16T07:02:24.878069+0000","last_clean":"2023-05-16T07:02:24.878069+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:02:24.878069+0000","last_undegraded":"2023-05-16T07:02:24.878069+0000","last_fullsized":"2023-05-16T07:02:24.878069+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4972,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366868,"statfs":{"total":96636764160,"available":96631672832,"internally_reserved":0,"allocated":167936,"data_stored":52755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":58554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4972,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366868,"statfs":{"total":96636764160,"available":96631672832,"internally_reserved":0,"allocated":167936,"data_stored":52755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4908,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366932,"statfs":{"total":96636764160,"available":96631738368,"internally_reserved":0,"allocated":167936,"data_stored":52755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4920,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366920,"statfs":{"total":96636764160,"available":96631726080,"internally_reserved":0,"allocated":180224,"data_stored":58554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4908,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366932,"statfs":{"total":96636764160,"available":96631738368,"internally_reserved":0,"allocated":167936,"data_stored":52755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":58554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4908,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366932,"statfs":{"total":96636764160,"available":96631738368,"internally_reserved":0,"allocated":167936,"data_stored":52755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:02:29.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:02:29.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.363+0000 7f1a4a218700 1 Processor -- start 2023-05-16T07:02:29.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.363+0000 7f1a4a218700 1 -- start start 2023-05-16T07:02:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a4a218700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a44154170 0x7f1a440c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a4a218700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a440c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a4a218700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1a440696a0 0x7f1a44069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a4a218700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440c0660 con 0x7f1a44154170 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a4a218700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4406a020 con 0x7f1a440696a0 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a4a218700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440bfd30 con 0x7f1a44154550 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a437fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a44154170 0x7f1a440c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a437fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a44154170 0x7f1a440c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40528/0 (socket says 172.21.15.92:40528) 2023-05-16T07:02:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a437fe700 1 -- 172.21.15.92:0/1449211778 learned_addr learned my addr 172.21.15.92:0/1449211778 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a42ffd700 1 --2- 172.21.15.92:0/1449211778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a440c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.364+0000 7f1a43fff700 1 --2- 172.21.15.92:0/1449211778 >> v2:172.21.15.158:3300/0 conn(0x7f1a440696a0 0x7f1a44069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a437fe700 1 -- 172.21.15.92:0/1449211778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 msgr2=0x7f1a440c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a437fe700 1 --2- 172.21.15.92:0/1449211778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a440c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a437fe700 1 -- 172.21.15.92:0/1449211778 >> v2:172.21.15.158:3300/0 conn(0x7f1a440696a0 msgr2=0x7f1a44069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a437fe700 1 --2- 172.21.15.92:0/1449211778 >> v2:172.21.15.158:3300/0 conn(0x7f1a440696a0 0x7f1a44069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a437fe700 1 -- 172.21.15.92:0/1449211778 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440c0010 con 0x7f1a44154170 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a43fff700 1 --2- 172.21.15.92:0/1449211778 >> v2:172.21.15.158:3300/0 conn(0x7f1a440696a0 0x7f1a44069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a437fe700 1 --2- 172.21.15.92:0/1449211778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a440c5c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f1a44158f90 tx=0x7f1a340090f0).ready entity=mon.0 client_cookie=46b0417ba0766f4e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a427fc700 1 -- 172.21.15.92:0/1449211778 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400b600 con 0x7f1a44154170 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a427fc700 1 -- 172.21.15.92:0/1449211778 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3400b760 con 0x7f1a44154170 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a427fc700 1 -- 172.21.15.92:0/1449211778 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400bb00 con 0x7f1a44154170 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 -- 172.21.15.92:0/1449211778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 msgr2=0x7f1a440c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 --2- 172.21.15.92:0/1449211778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a440c5c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f1a44158f90 tx=0x7f1a340090f0).stop 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 -- 172.21.15.92:0/1449211778 shutdown_connections 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 --2- 172.21.15.92:0/1449211778 >> v2:172.21.15.158:3300/0 conn(0x7f1a440696a0 0x7f1a44069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 --2- 172.21.15.92:0/1449211778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a440c5c70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 --2- 172.21.15.92:0/1449211778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a440c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 -- 172.21.15.92:0/1449211778 >> 172.21.15.92:0/1449211778 conn(0x7f1a440cb700 msgr2=0x7f1a440c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 -- 172.21.15.92:0/1449211778 shutdown_connections 2023-05-16T07:02:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.365+0000 7f1a4a218700 1 -- 172.21.15.92:0/1449211778 wait complete. 2023-05-16T07:02:29.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 Processor -- start 2023-05-16T07:02:29.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 -- start start 2023-05-16T07:02:29.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a440696a0 0x7f1a44151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a44158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a44164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c2af0 con 0x7f1a44154170 2023-05-16T07:02:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c1ee0 con 0x7f1a440696a0 2023-05-16T07:02:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a4a218700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c21c0 con 0x7f1a44154550 2023-05-16T07:02:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a42ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a44158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a42ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a44158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40536/0 (socket says 172.21.15.92:40536) 2023-05-16T07:02:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a42ffd700 1 -- 172.21.15.92:0/3530632925 learned_addr learned my addr 172.21.15.92:0/3530632925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a437fe700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a440696a0 0x7f1a44151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.367+0000 7f1a43fff700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a44164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a42ffd700 1 -- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 msgr2=0x7f1a44164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a42ffd700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a44164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a42ffd700 1 -- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a440696a0 msgr2=0x7f1a44151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a42ffd700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a440696a0 0x7f1a44151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a42ffd700 1 -- 172.21.15.92:0/3530632925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34009370 con 0x7f1a44154170 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a42ffd700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a44158040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f1a3800d860 tx=0x7f1a38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38011590 con 0x7f1a44154170 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a380116f0 con 0x7f1a44154170 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38011a70 con 0x7f1a44154170 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a440c24a0 con 0x7f1a44154170 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.368+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a441524c0 con 0x7f1a44154170 2023-05-16T07:02:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.369+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a440666c0 con 0x7f1a44154170 2023-05-16T07:02:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.369+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1a38012420 con 0x7f1a44154170 2023-05-16T07:02:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.370+0000 7f1a2b7fe700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1a3005ed90 0x7f1a30061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.370+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1a380795b0 con 0x7f1a44154170 2023-05-16T07:02:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.370+0000 7f1a437fe700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1a3005ed90 0x7f1a30061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.370+0000 7f1a437fe700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1a3005ed90 0x7f1a30061250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1a3400b3c0 tx=0x7f1a34017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.373+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1a3804aaf0 con 0x7f1a44154170 2023-05-16T07:02:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.532+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a441656e0 con 0x7f1a3005ed90 2023-05-16T07:02:29.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.542+0000 7f1a2b7fe700 1 -- 172.21.15.92:0/3530632925 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336341 (secure 0 0 0) 0x7f1a441656e0 con 0x7f1a3005ed90 2023-05-16T07:02:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:29.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1a3005ed90 msgr2=0x7f1a30061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1a3005ed90 0x7f1a30061250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1a3400b3c0 tx=0x7f1a34017000).stop 2023-05-16T07:02:29.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 msgr2=0x7f1a44158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a44158040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f1a3800d860 tx=0x7f1a38008040).stop 2023-05-16T07:02:29.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 shutdown_connections 2023-05-16T07:02:29.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1a3005ed90 0x7f1a30061250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a44154550 0x7f1a44164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a440696a0 0x7f1a44151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 --2- 172.21.15.92:0/3530632925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a44154170 0x7f1a44158040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 >> 172.21.15.92:0/3530632925 conn(0x7f1a440cb700 msgr2=0x7f1a440a9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 shutdown_connections 2023-05-16T07:02:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.545+0000 7f1a4a218700 1 -- 172.21.15.92:0/3530632925 wait complete. 2023-05-16T07:02:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:02:29.561 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-05-16T07:02:29.452181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":256,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41512,"kb_used_data":1832,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":754933208,"statfs":{"total":773094113280,"available":773051604992,"internally_reserved":0,"allocated":1875968,"data_stored":673900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266087+0000","last_change":"2023-05-16T07:02:26.266087+0000","last_active":"2023-05-16T07:02:26.266087+0000","last_peered":"2023-05-16T07:02:26.266087+0000","last_clean":"2023-05-16T07:02:26.266087+0000","last_became_active":"2023-05-16T07:02:26.265870+0000","last_became_peered":"2023-05-16T07:02:26.265870+0000","last_unstale":"2023-05-16T07:02:26.266087+0000","last_undegraded":"2023-05-16T07:02:26.266087+0000","last_fullsized":"2023-05-16T07:02:26.266087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260827+0000","last_change":"2023-05-16T07:02:25.902715+0000","last_active":"2023-05-16T07:02:26.260827+0000","last_peered":"2023-05-16T07:02:26.260827+0000","last_clean":"2023-05-16T07:02:26.260827+0000","last_became_active":"2023-05-16T07:02:25.902401+0000","last_became_peered":"2023-05-16T07:02:25.902401+0000","last_unstale":"2023-05-16T07:02:26.260827+0000","last_undegraded":"2023-05-16T07:02:26.260827+0000","last_fullsized":"2023-05-16T07:02:26.260827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270248+0000","last_change":"2023-05-16T07:02:26.269154+0000","last_active":"2023-05-16T07:02:26.270248+0000","last_peered":"2023-05-16T07:02:26.270248+0000","last_clean":"2023-05-16T07:02:26.270248+0000","last_became_active":"2023-05-16T07:02:26.268835+0000","last_became_peered":"2023-05-16T07:02:26.268835+0000","last_unstale":"2023-05-16T07:02:26.270248+0000","last_undegraded":"2023-05-16T07:02:26.270248+0000","last_fullsized":"2023-05-16T07:02:26.270248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.903344+0000","last_change":"2023-05-16T07:02:25.899378+0000","last_active":"2023-05-16T07:02:25.903344+0000","last_peered":"2023-05-16T07:02:25.903344+0000","last_clean":"2023-05-16T07:02:25.903344+0000","last_became_active":"2023-05-16T07:02:25.899114+0000","last_became_peered":"2023-05-16T07:02:25.899114+0000","last_unstale":"2023-05-16T07:02:25.903344+0000","last_undegraded":"2023-05-16T07:02:25.903344+0000","last_fullsized":"2023-05-16T07:02:25.903344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.258180+0000","last_change":"2023-05-16T07:02:26.255359+0000","last_active":"2023-05-16T07:02:26.258180+0000","last_peered":"2023-05-16T07:02:26.258180+0000","last_clean":"2023-05-16T07:02:26.258180+0000","last_became_active":"2023-05-16T07:02:26.254336+0000","last_became_peered":"2023-05-16T07:02:26.254336+0000","last_unstale":"2023-05-16T07:02:26.258180+0000","last_undegraded":"2023-05-16T07:02:26.258180+0000","last_fullsized":"2023-05-16T07:02:26.258180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.906898+0000","last_change":"2023-05-16T07:02:25.903697+0000","last_active":"2023-05-16T07:02:25.906898+0000","last_peered":"2023-05-16T07:02:25.906898+0000","last_clean":"2023-05-16T07:02:25.906898+0000","last_became_active":"2023-05-16T07:02:25.903431+0000","last_became_peered":"2023-05-16T07:02:25.903431+0000","last_unstale":"2023-05-16T07:02:25.906898+0000","last_undegraded":"2023-05-16T07:02:25.906898+0000","last_fullsized":"2023-05-16T07:02:25.906898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272040+0000","last_change":"2023-05-16T07:02:26.272040+0000","last_active":"2023-05-16T07:02:26.272040+0000","last_peered":"2023-05-16T07:02:26.272040+0000","last_clean":"2023-05-16T07:02:26.272040+0000","last_became_active":"2023-05-16T07:02:26.271774+0000","last_became_peered":"2023-05-16T07:02:26.271774+0000","last_unstale":"2023-05-16T07:02:26.272040+0000","last_undegraded":"2023-05-16T07:02:26.272040+0000","last_fullsized":"2023-05-16T07:02:26.272040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261304+0000","last_change":"2023-05-16T07:02:25.899639+0000","last_active":"2023-05-16T07:02:26.261304+0000","last_peered":"2023-05-16T07:02:26.261304+0000","last_clean":"2023-05-16T07:02:26.261304+0000","last_became_active":"2023-05-16T07:02:25.899189+0000","last_became_peered":"2023-05-16T07:02:25.899189+0000","last_unstale":"2023-05-16T07:02:26.261304+0000","last_undegraded":"2023-05-16T07:02:26.261304+0000","last_fullsized":"2023-05-16T07:02:26.261304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263614+0000","last_change":"2023-05-16T07:02:26.263614+0000","last_active":"2023-05-16T07:02:26.263614+0000","last_peered":"2023-05-16T07:02:26.263614+0000","last_clean":"2023-05-16T07:02:26.263614+0000","last_became_active":"2023-05-16T07:02:26.263333+0000","last_became_peered":"2023-05-16T07:02:26.263333+0000","last_unstale":"2023-05-16T07:02:26.263614+0000","last_undegraded":"2023-05-16T07:02:26.263614+0000","last_fullsized":"2023-05-16T07:02:26.263614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252139+0000","last_change":"2023-05-16T07:02:25.902127+0000","last_active":"2023-05-16T07:02:26.252139+0000","last_peered":"2023-05-16T07:02:26.252139+0000","last_clean":"2023-05-16T07:02:26.252139+0000","last_became_active":"2023-05-16T07:02:25.901841+0000","last_became_peered":"2023-05-16T07:02:25.901841+0000","last_unstale":"2023-05-16T07:02:26.252139+0000","last_undegraded":"2023-05-16T07:02:26.252139+0000","last_fullsized":"2023-05-16T07:02:26.252139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266427+0000","last_change":"2023-05-16T07:02:26.266427+0000","last_active":"2023-05-16T07:02:26.266427+0000","last_peered":"2023-05-16T07:02:26.266427+0000","last_clean":"2023-05-16T07:02:26.266427+0000","last_became_active":"2023-05-16T07:02:26.265962+0000","last_became_peered":"2023-05-16T07:02:26.265962+0000","last_unstale":"2023-05-16T07:02:26.266427+0000","last_undegraded":"2023-05-16T07:02:26.266427+0000","last_fullsized":"2023-05-16T07:02:26.266427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270470+0000","last_change":"2023-05-16T07:02:25.906258+0000","last_active":"2023-05-16T07:02:26.270470+0000","last_peered":"2023-05-16T07:02:26.270470+0000","last_clean":"2023-05-16T07:02:26.270470+0000","last_became_active":"2023-05-16T07:02:25.905996+0000","last_became_peered":"2023-05-16T07:02:25.905996+0000","last_unstale":"2023-05-16T07:02:26.270470+0000","last_undegraded":"2023-05-16T07:02:26.270470+0000","last_fullsized":"2023-05-16T07:02:26.270470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273736+0000","last_change":"2023-05-16T07:02:26.269375+0000","last_active":"2023-05-16T07:02:26.273736+0000","last_peered":"2023-05-16T07:02:26.273736+0000","last_clean":"2023-05-16T07:02:26.273736+0000","last_became_active":"2023-05-16T07:02:26.269086+0000","last_became_peered":"2023-05-16T07:02:26.269086+0000","last_unstale":"2023-05-16T07:02:26.273736+0000","last_undegraded":"2023-05-16T07:02:26.273736+0000","last_fullsized":"2023-05-16T07:02:26.273736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275465+0000","last_change":"2023-05-16T07:02:25.903795+0000","last_active":"2023-05-16T07:02:26.275465+0000","last_peered":"2023-05-16T07:02:26.275465+0000","last_clean":"2023-05-16T07:02:26.275465+0000","last_became_active":"2023-05-16T07:02:25.903251+0000","last_became_peered":"2023-05-16T07:02:25.903251+0000","last_unstale":"2023-05-16T07:02:26.275465+0000","last_undegraded":"2023-05-16T07:02:26.275465+0000","last_fullsized":"2023-05-16T07:02:26.275465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266963+0000","last_change":"2023-05-16T07:02:26.266963+0000","last_active":"2023-05-16T07:02:26.266963+0000","last_peered":"2023-05-16T07:02:26.266963+0000","last_clean":"2023-05-16T07:02:26.266963+0000","last_became_active":"2023-05-16T07:02:26.266311+0000","last_became_peered":"2023-05-16T07:02:26.266311+0000","last_unstale":"2023-05-16T07:02:26.266963+0000","last_undegraded":"2023-05-16T07:02:26.266963+0000","last_fullsized":"2023-05-16T07:02:26.266963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272749+0000","last_change":"2023-05-16T07:02:25.909821+0000","last_active":"2023-05-16T07:02:26.272749+0000","last_peered":"2023-05-16T07:02:26.272749+0000","last_clean":"2023-05-16T07:02:26.272749+0000","last_became_active":"2023-05-16T07:02:25.909518+0000","last_became_peered":"2023-05-16T07:02:25.909518+0000","last_unstale":"2023-05-16T07:02:26.272749+0000","last_undegraded":"2023-05-16T07:02:26.272749+0000","last_fullsized":"2023-05-16T07:02:26.272749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266094+0000","last_change":"2023-05-16T07:02:26.263808+0000","last_active":"2023-05-16T07:02:26.266094+0000","last_peered":"2023-05-16T07:02:26.266094+0000","last_clean":"2023-05-16T07:02:26.266094+0000","last_became_active":"2023-05-16T07:02:26.263527+0000","last_became_peered":"2023-05-16T07:02:26.263527+0000","last_unstale":"2023-05-16T07:02:26.266094+0000","last_undegraded":"2023-05-16T07:02:26.266094+0000","last_fullsized":"2023-05-16T07:02:26.266094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254202+0000","last_change":"2023-05-16T07:02:25.901918+0000","last_active":"2023-05-16T07:02:26.254202+0000","last_peered":"2023-05-16T07:02:26.254202+0000","last_clean":"2023-05-16T07:02:26.254202+0000","last_became_active":"2023-05-16T07:02:25.901641+0000","last_became_peered":"2023-05-16T07:02:25.901641+0000","last_unstale":"2023-05-16T07:02:26.254202+0000","last_undegraded":"2023-05-16T07:02:26.254202+0000","last_fullsized":"2023-05-16T07:02:26.254202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269556+0000","last_change":"2023-05-16T07:02:26.267982+0000","last_active":"2023-05-16T07:02:26.269556+0000","last_peered":"2023-05-16T07:02:26.269556+0000","last_clean":"2023-05-16T07:02:26.269556+0000","last_became_active":"2023-05-16T07:02:26.267703+0000","last_became_peered":"2023-05-16T07:02:26.267703+0000","last_unstale":"2023-05-16T07:02:26.269556+0000","last_undegraded":"2023-05-16T07:02:26.269556+0000","last_fullsized":"2023-05-16T07:02:26.269556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254313+0000","last_change":"2023-05-16T07:02:25.909768+0000","last_active":"2023-05-16T07:02:26.254313+0000","last_peered":"2023-05-16T07:02:26.254313+0000","last_clean":"2023-05-16T07:02:26.254313+0000","last_became_active":"2023-05-16T07:02:25.909522+0000","last_became_peered":"2023-05-16T07:02:25.909522+0000","last_unstale":"2023-05-16T07:02:26.254313+0000","last_undegraded":"2023-05-16T07:02:26.254313+0000","last_fullsized":"2023-05-16T07:02:26.254313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.268669+0000","last_change":"2023-05-16T07:02:26.268669+0000","last_active":"2023-05-16T07:02:26.268669+0000","last_peered":"2023-05-16T07:02:26.268669+0000","last_clean":"2023-05-16T07:02:26.268669+0000","last_became_active":"2023-05-16T07:02:26.268378+0000","last_became_peered":"2023-05-16T07:02:26.268378+0000","last_unstale":"2023-05-16T07:02:26.268669+0000","last_undegraded":"2023-05-16T07:02:26.268669+0000","last_fullsized":"2023-05-16T07:02:26.268669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269854+0000","last_change":"2023-05-16T07:02:25.901955+0000","last_active":"2023-05-16T07:02:26.269854+0000","last_peered":"2023-05-16T07:02:26.269854+0000","last_clean":"2023-05-16T07:02:26.269854+0000","last_became_active":"2023-05-16T07:02:25.901688+0000","last_became_peered":"2023-05-16T07:02:25.901688+0000","last_unstale":"2023-05-16T07:02:26.269854+0000","last_undegraded":"2023-05-16T07:02:26.269854+0000","last_fullsized":"2023-05-16T07:02:26.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273371+0000","last_change":"2023-05-16T07:02:26.270760+0000","last_active":"2023-05-16T07:02:26.273371+0000","last_peered":"2023-05-16T07:02:26.273371+0000","last_clean":"2023-05-16T07:02:26.273371+0000","last_became_active":"2023-05-16T07:02:26.270396+0000","last_became_peered":"2023-05-16T07:02:26.270396+0000","last_unstale":"2023-05-16T07:02:26.273371+0000","last_undegraded":"2023-05-16T07:02:26.273371+0000","last_fullsized":"2023-05-16T07:02:26.273371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274528+0000","last_change":"2023-05-16T07:02:25.909903+0000","last_active":"2023-05-16T07:02:26.274528+0000","last_peered":"2023-05-16T07:02:26.274528+0000","last_clean":"2023-05-16T07:02:26.274528+0000","last_became_active":"2023-05-16T07:02:25.909569+0000","last_became_peered":"2023-05-16T07:02:25.909569+0000","last_unstale":"2023-05-16T07:02:26.274528+0000","last_undegraded":"2023-05-16T07:02:26.274528+0000","last_fullsized":"2023-05-16T07:02:26.274528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273702+0000","last_change":"2023-05-16T07:02:26.268910+0000","last_active":"2023-05-16T07:02:26.273702+0000","last_peered":"2023-05-16T07:02:26.273702+0000","last_clean":"2023-05-16T07:02:26.273702+0000","last_became_active":"2023-05-16T07:02:26.268594+0000","last_became_peered":"2023-05-16T07:02:26.268594+0000","last_unstale":"2023-05-16T07:02:26.273702+0000","last_undegraded":"2023-05-16T07:02:26.273702+0000","last_fullsized":"2023-05-16T07:02:26.273702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273618+0000","last_change":"2023-05-16T07:02:25.911781+0000","last_active":"2023-05-16T07:02:26.273618+0000","last_peered":"2023-05-16T07:02:26.273618+0000","last_clean":"2023-05-16T07:02:26.273618+0000","last_became_active":"2023-05-16T07:02:25.911476+0000","last_became_peered":"2023-05-16T07:02:25.911476+0000","last_unstale":"2023-05-16T07:02:26.273618+0000","last_undegraded":"2023-05-16T07:02:26.273618+0000","last_fullsized":"2023-05-16T07:02:26.273618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273691+0000","last_change":"2023-05-16T07:02:26.271378+0000","last_active":"2023-05-16T07:02:26.273691+0000","last_peered":"2023-05-16T07:02:26.273691+0000","last_clean":"2023-05-16T07:02:26.273691+0000","last_became_active":"2023-05-16T07:02:26.271109+0000","last_became_peered":"2023-05-16T07:02:26.271109+0000","last_unstale":"2023-05-16T07:02:26.273691+0000","last_undegraded":"2023-05-16T07:02:26.273691+0000","last_fullsized":"2023-05-16T07:02:26.273691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255189+0000","last_change":"2023-05-16T07:02:25.908805+0000","last_active":"2023-05-16T07:02:26.255189+0000","last_peered":"2023-05-16T07:02:26.255189+0000","last_clean":"2023-05-16T07:02:26.255189+0000","last_became_active":"2023-05-16T07:02:25.906682+0000","last_became_peered":"2023-05-16T07:02:25.906682+0000","last_unstale":"2023-05-16T07:02:26.255189+0000","last_undegraded":"2023-05-16T07:02:26.255189+0000","last_fullsized":"2023-05-16T07:02:26.255189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269698+0000","last_change":"2023-05-16T07:02:26.267973+0000","last_active":"2023-05-16T07:02:26.269698+0000","last_peered":"2023-05-16T07:02:26.269698+0000","last_clean":"2023-05-16T07:02:26.269698+0000","last_became_active":"2023-05-16T07:02:26.267715+0000","last_became_peered":"2023-05-16T07:02:26.267715+0000","last_unstale":"2023-05-16T07:02:26.269698+0000","last_undegraded":"2023-05-16T07:02:26.269698+0000","last_fullsized":"2023-05-16T07:02:26.269698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252005+0000","last_change":"2023-05-16T07:02:25.902602+0000","last_active":"2023-05-16T07:02:26.252005+0000","last_peered":"2023-05-16T07:02:26.252005+0000","last_clean":"2023-05-16T07:02:26.252005+0000","last_became_active":"2023-05-16T07:02:25.901915+0000","last_became_peered":"2023-05-16T07:02:25.901915+0000","last_unstale":"2023-05-16T07:02:26.252005+0000","last_undegraded":"2023-05-16T07:02:26.252005+0000","last_fullsized":"2023-05-16T07:02:26.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274675+0000","last_change":"2023-05-16T07:02:26.272831+0000","last_active":"2023-05-16T07:02:26.274675+0000","last_peered":"2023-05-16T07:02:26.274675+0000","last_clean":"2023-05-16T07:02:26.274675+0000","last_became_active":"2023-05-16T07:02:26.270437+0000","last_became_peered":"2023-05-16T07:02:26.270437+0000","last_unstale":"2023-05-16T07:02:26.274675+0000","last_undegraded":"2023-05-16T07:02:26.274675+0000","last_fullsized":"2023-05-16T07:02:26.274675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251751+0000","last_change":"2023-05-16T07:02:25.905337+0000","last_active":"2023-05-16T07:02:26.251751+0000","last_peered":"2023-05-16T07:02:26.251751+0000","last_clean":"2023-05-16T07:02:26.251751+0000","last_became_active":"2023-05-16T07:02:25.904994+0000","last_became_peered":"2023-05-16T07:02:25.904994+0000","last_unstale":"2023-05-16T07:02:26.251751+0000","last_undegraded":"2023-05-16T07:02:26.251751+0000","last_fullsized":"2023-05-16T07:02:26.251751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273663+0000","last_change":"2023-05-16T07:02:26.269107+0000","last_active":"2023-05-16T07:02:26.273663+0000","last_peered":"2023-05-16T07:02:26.273663+0000","last_clean":"2023-05-16T07:02:26.273663+0000","last_became_active":"2023-05-16T07:02:26.268131+0000","last_became_peered":"2023-05-16T07:02:26.268131+0000","last_unstale":"2023-05-16T07:02:26.273663+0000","last_undegraded":"2023-05-16T07:02:26.273663+0000","last_fullsized":"2023-05-16T07:02:26.273663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250472+0000","last_change":"2023-05-16T07:02:25.897709+0000","last_active":"2023-05-16T07:02:26.250472+0000","last_peered":"2023-05-16T07:02:26.250472+0000","last_clean":"2023-05-16T07:02:26.250472+0000","last_became_active":"2023-05-16T07:02:25.897436+0000","last_became_peered":"2023-05-16T07:02:25.897436+0000","last_unstale":"2023-05-16T07:02:26.250472+0000","last_undegraded":"2023-05-16T07:02:26.250472+0000","last_fullsized":"2023-05-16T07:02:26.250472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273849+0000","last_change":"2023-05-16T07:02:26.269529+0000","last_active":"2023-05-16T07:02:26.273849+0000","last_peered":"2023-05-16T07:02:26.273849+0000","last_clean":"2023-05-16T07:02:26.273849+0000","last_became_active":"2023-05-16T07:02:26.269233+0000","last_became_peered":"2023-05-16T07:02:26.269233+0000","last_unstale":"2023-05-16T07:02:26.273849+0000","last_undegraded":"2023-05-16T07:02:26.273849+0000","last_fullsized":"2023-05-16T07:02:26.273849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.257675+0000","last_change":"2023-05-16T07:02:25.901600+0000","last_active":"2023-05-16T07:02:26.257675+0000","last_peered":"2023-05-16T07:02:26.257675+0000","last_clean":"2023-05-16T07:02:26.257675+0000","last_became_active":"2023-05-16T07:02:25.901168+0000","last_became_peered":"2023-05-16T07:02:25.901168+0000","last_unstale":"2023-05-16T07:02:26.257675+0000","last_undegraded":"2023-05-16T07:02:26.257675+0000","last_fullsized":"2023-05-16T07:02:26.257675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263820+0000","last_change":"2023-05-16T07:02:26.263820+0000","last_active":"2023-05-16T07:02:26.263820+0000","last_peered":"2023-05-16T07:02:26.263820+0000","last_clean":"2023-05-16T07:02:26.263820+0000","last_became_active":"2023-05-16T07:02:26.263188+0000","last_became_peered":"2023-05-16T07:02:26.263188+0000","last_unstale":"2023-05-16T07:02:26.263820+0000","last_undegraded":"2023-05-16T07:02:26.263820+0000","last_fullsized":"2023-05-16T07:02:26.263820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.906879+0000","last_change":"2023-05-16T07:02:25.903683+0000","last_active":"2023-05-16T07:02:25.906879+0000","last_peered":"2023-05-16T07:02:25.906879+0000","last_clean":"2023-05-16T07:02:25.906879+0000","last_became_active":"2023-05-16T07:02:25.903358+0000","last_became_peered":"2023-05-16T07:02:25.903358+0000","last_unstale":"2023-05-16T07:02:25.906879+0000","last_undegraded":"2023-05-16T07:02:25.906879+0000","last_fullsized":"2023-05-16T07:02:25.906879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.884095+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:02:25.884095+0000","last_peered":"2023-05-16T07:02:25.884095+0000","last_clean":"2023-05-16T07:02:25.884095+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:02:25.884095+0000","last_undegraded":"2023-05-16T07:02:25.884095+0000","last_fullsized":"2023-05-16T07:02:25.884095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252187+0000","last_change":"2023-05-16T07:02:25.903947+0000","last_active":"2023-05-16T07:02:26.252187+0000","last_peered":"2023-05-16T07:02:26.252187+0000","last_clean":"2023-05-16T07:02:26.252187+0000","last_became_active":"2023-05-16T07:02:25.903647+0000","last_became_peered":"2023-05-16T07:02:25.903647+0000","last_unstale":"2023-05-16T07:02:26.252187+0000","last_undegraded":"2023-05-16T07:02:26.252187+0000","last_fullsized":"2023-05-16T07:02:26.252187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260004+0000","last_change":"2023-05-16T07:02:26.260004+0000","last_active":"2023-05-16T07:02:26.260004+0000","last_peered":"2023-05-16T07:02:26.260004+0000","last_clean":"2023-05-16T07:02:26.260004+0000","last_became_active":"2023-05-16T07:02:26.256118+0000","last_became_peered":"2023-05-16T07:02:26.256118+0000","last_unstale":"2023-05-16T07:02:26.260004+0000","last_undegraded":"2023-05-16T07:02:26.260004+0000","last_fullsized":"2023-05-16T07:02:26.260004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252201+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:02:26.252201+0000","last_peered":"2023-05-16T07:02:26.252201+0000","last_clean":"2023-05-16T07:02:26.252201+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:02:26.252201+0000","last_undegraded":"2023-05-16T07:02:26.252201+0000","last_fullsized":"2023-05-16T07:02:26.252201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260286+0000","last_change":"2023-05-16T07:02:25.902850+0000","last_active":"2023-05-16T07:02:26.260286+0000","last_peered":"2023-05-16T07:02:26.260286+0000","last_clean":"2023-05-16T07:02:26.260286+0000","last_became_active":"2023-05-16T07:02:25.902557+0000","last_became_peered":"2023-05-16T07:02:25.902557+0000","last_unstale":"2023-05-16T07:02:26.260286+0000","last_undegraded":"2023-05-16T07:02:26.260286+0000","last_fullsized":"2023-05-16T07:02:26.260286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265589+0000","last_change":"2023-05-16T07:02:26.265589+0000","last_active":"2023-05-16T07:02:26.265589+0000","last_peered":"2023-05-16T07:02:26.265589+0000","last_clean":"2023-05-16T07:02:26.265589+0000","last_became_active":"2023-05-16T07:02:26.264804+0000","last_became_peered":"2023-05-16T07:02:26.264804+0000","last_unstale":"2023-05-16T07:02:26.265589+0000","last_undegraded":"2023-05-16T07:02:26.265589+0000","last_fullsized":"2023-05-16T07:02:26.265589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263668+0000","last_change":"2023-05-16T07:02:26.263668+0000","last_active":"2023-05-16T07:02:26.263668+0000","last_peered":"2023-05-16T07:02:26.263668+0000","last_clean":"2023-05-16T07:02:26.263668+0000","last_became_active":"2023-05-16T07:02:26.262937+0000","last_became_peered":"2023-05-16T07:02:26.262937+0000","last_unstale":"2023-05-16T07:02:26.263668+0000","last_undegraded":"2023-05-16T07:02:26.263668+0000","last_fullsized":"2023-05-16T07:02:26.263668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252944+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:02:26.252944+0000","last_peered":"2023-05-16T07:02:26.252944+0000","last_clean":"2023-05-16T07:02:26.252944+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:02:26.252944+0000","last_undegraded":"2023-05-16T07:02:26.252944+0000","last_fullsized":"2023-05-16T07:02:26.252944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.905931+0000","last_change":"2023-05-16T07:02:25.900932+0000","last_active":"2023-05-16T07:02:25.905931+0000","last_peered":"2023-05-16T07:02:25.905931+0000","last_clean":"2023-05-16T07:02:25.905931+0000","last_became_active":"2023-05-16T07:02:25.899982+0000","last_became_peered":"2023-05-16T07:02:25.899982+0000","last_unstale":"2023-05-16T07:02:25.905931+0000","last_undegraded":"2023-05-16T07:02:25.905931+0000","last_fullsized":"2023-05-16T07:02:25.905931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274431+0000","last_change":"2023-05-16T07:02:26.271648+0000","last_active":"2023-05-16T07:02:26.274431+0000","last_peered":"2023-05-16T07:02:26.274431+0000","last_clean":"2023-05-16T07:02:26.274431+0000","last_became_active":"2023-05-16T07:02:26.271391+0000","last_became_peered":"2023-05-16T07:02:26.271391+0000","last_unstale":"2023-05-16T07:02:26.274431+0000","last_undegraded":"2023-05-16T07:02:26.274431+0000","last_fullsized":"2023-05-16T07:02:26.274431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251974+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:02:26.251974+0000","last_peered":"2023-05-16T07:02:26.251974+0000","last_clean":"2023-05-16T07:02:26.251974+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:02:26.251974+0000","last_undegraded":"2023-05-16T07:02:26.251974+0000","last_fullsized":"2023-05-16T07:02:26.251974+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255213+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:02:26.255213+0000","last_peered":"2023-05-16T07:02:26.255213+0000","last_clean":"2023-05-16T07:02:26.255213+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:02:26.255213+0000","last_undegraded":"2023-05-16T07:02:26.255213+0000","last_fullsized":"2023-05-16T07:02:26.255213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.907383+0000","last_change":"2023-05-16T07:02:25.906617+0000","last_active":"2023-05-16T07:02:25.907383+0000","last_peered":"2023-05-16T07:02:25.907383+0000","last_clean":"2023-05-16T07:02:25.907383+0000","last_became_active":"2023-05-16T07:02:25.906269+0000","last_became_peered":"2023-05-16T07:02:25.906269+0000","last_unstale":"2023-05-16T07:02:25.907383+0000","last_undegraded":"2023-05-16T07:02:25.907383+0000","last_fullsized":"2023-05-16T07:02:25.907383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274440+0000","last_change":"2023-05-16T07:02:26.273104+0000","last_active":"2023-05-16T07:02:26.274440+0000","last_peered":"2023-05-16T07:02:26.274440+0000","last_clean":"2023-05-16T07:02:26.274440+0000","last_became_active":"2023-05-16T07:02:26.272816+0000","last_became_peered":"2023-05-16T07:02:26.272816+0000","last_unstale":"2023-05-16T07:02:26.274440+0000","last_undegraded":"2023-05-16T07:02:26.274440+0000","last_fullsized":"2023-05-16T07:02:26.274440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.257669+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:02:26.257669+0000","last_peered":"2023-05-16T07:02:26.257669+0000","last_clean":"2023-05-16T07:02:26.257669+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:02:26.257669+0000","last_undegraded":"2023-05-16T07:02:26.257669+0000","last_fullsized":"2023-05-16T07:02:26.257669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251964+0000","last_change":"2023-05-16T07:02:25.902199+0000","last_active":"2023-05-16T07:02:26.251964+0000","last_peered":"2023-05-16T07:02:26.251964+0000","last_clean":"2023-05-16T07:02:26.251964+0000","last_became_active":"2023-05-16T07:02:25.901913+0000","last_became_peered":"2023-05-16T07:02:25.901913+0000","last_unstale":"2023-05-16T07:02:26.251964+0000","last_undegraded":"2023-05-16T07:02:26.251964+0000","last_fullsized":"2023-05-16T07:02:26.251964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255055+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:02:26.255055+0000","last_peered":"2023-05-16T07:02:26.255055+0000","last_clean":"2023-05-16T07:02:26.255055+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:02:26.255055+0000","last_undegraded":"2023-05-16T07:02:26.255055+0000","last_fullsized":"2023-05-16T07:02:26.255055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.905559+0000","last_change":"2023-05-16T07:02:25.900748+0000","last_active":"2023-05-16T07:02:25.905559+0000","last_peered":"2023-05-16T07:02:25.905559+0000","last_clean":"2023-05-16T07:02:25.905559+0000","last_became_active":"2023-05-16T07:02:25.900481+0000","last_became_peered":"2023-05-16T07:02:25.900481+0000","last_unstale":"2023-05-16T07:02:25.905559+0000","last_undegraded":"2023-05-16T07:02:25.905559+0000","last_fullsized":"2023-05-16T07:02:25.905559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265039+0000","last_change":"2023-05-16T07:02:26.260513+0000","last_active":"2023-05-16T07:02:26.265039+0000","last_peered":"2023-05-16T07:02:26.265039+0000","last_clean":"2023-05-16T07:02:26.265039+0000","last_became_active":"2023-05-16T07:02:26.260265+0000","last_became_peered":"2023-05-16T07:02:26.260265+0000","last_unstale":"2023-05-16T07:02:26.265039+0000","last_undegraded":"2023-05-16T07:02:26.265039+0000","last_fullsized":"2023-05-16T07:02:26.265039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.268834+0000","last_change":"2023-05-16T07:02:25.900313+0000","last_active":"2023-05-16T07:02:26.268834+0000","last_peered":"2023-05-16T07:02:26.268834+0000","last_clean":"2023-05-16T07:02:26.268834+0000","last_became_active":"2023-05-16T07:02:25.900056+0000","last_became_peered":"2023-05-16T07:02:25.900056+0000","last_unstale":"2023-05-16T07:02:26.268834+0000","last_undegraded":"2023-05-16T07:02:26.268834+0000","last_fullsized":"2023-05-16T07:02:26.268834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.884019+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:02:25.884019+0000","last_peered":"2023-05-16T07:02:25.884019+0000","last_clean":"2023-05-16T07:02:25.884019+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:02:25.884019+0000","last_undegraded":"2023-05-16T07:02:25.884019+0000","last_fullsized":"2023-05-16T07:02:25.884019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270645+0000","last_change":"2023-05-16T07:02:26.270645+0000","last_active":"2023-05-16T07:02:26.270645+0000","last_peered":"2023-05-16T07:02:26.270645+0000","last_clean":"2023-05-16T07:02:26.270645+0000","last_became_active":"2023-05-16T07:02:26.270312+0000","last_became_peered":"2023-05-16T07:02:26.270312+0000","last_unstale":"2023-05-16T07:02:26.270645+0000","last_undegraded":"2023-05-16T07:02:26.270645+0000","last_fullsized":"2023-05-16T07:02:26.270645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250955+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:02:26.250955+0000","last_peered":"2023-05-16T07:02:26.250955+0000","last_clean":"2023-05-16T07:02:26.250955+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:02:26.250955+0000","last_undegraded":"2023-05-16T07:02:26.250955+0000","last_fullsized":"2023-05-16T07:02:26.250955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.258392+0000","last_change":"2023-05-16T07:02:25.902709+0000","last_active":"2023-05-16T07:02:26.258392+0000","last_peered":"2023-05-16T07:02:26.258392+0000","last_clean":"2023-05-16T07:02:26.258392+0000","last_became_active":"2023-05-16T07:02:25.902421+0000","last_became_peered":"2023-05-16T07:02:25.902421+0000","last_unstale":"2023-05-16T07:02:26.258392+0000","last_undegraded":"2023-05-16T07:02:26.258392+0000","last_fullsized":"2023-05-16T07:02:26.258392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272520+0000","last_change":"2023-05-16T07:02:26.263603+0000","last_active":"2023-05-16T07:02:26.272520+0000","last_peered":"2023-05-16T07:02:26.272520+0000","last_clean":"2023-05-16T07:02:26.272520+0000","last_became_active":"2023-05-16T07:02:26.260959+0000","last_became_peered":"2023-05-16T07:02:26.260959+0000","last_unstale":"2023-05-16T07:02:26.272520+0000","last_undegraded":"2023-05-16T07:02:26.272520+0000","last_fullsized":"2023-05-16T07:02:26.272520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273598+0000","last_change":"2023-05-16T07:02:26.272233+0000","last_active":"2023-05-16T07:02:26.273598+0000","last_peered":"2023-05-16T07:02:26.273598+0000","last_clean":"2023-05-16T07:02:26.273598+0000","last_became_active":"2023-05-16T07:02:26.271964+0000","last_became_peered":"2023-05-16T07:02:26.271964+0000","last_unstale":"2023-05-16T07:02:26.273598+0000","last_undegraded":"2023-05-16T07:02:26.273598+0000","last_fullsized":"2023-05-16T07:02:26.273598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253633+0000","last_change":"2023-05-16T07:02:25.902264+0000","last_active":"2023-05-16T07:02:26.253633+0000","last_peered":"2023-05-16T07:02:26.253633+0000","last_clean":"2023-05-16T07:02:26.253633+0000","last_became_active":"2023-05-16T07:02:25.901192+0000","last_became_peered":"2023-05-16T07:02:25.901192+0000","last_unstale":"2023-05-16T07:02:26.253633+0000","last_undegraded":"2023-05-16T07:02:26.253633+0000","last_fullsized":"2023-05-16T07:02:26.253633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270652+0000","last_change":"2023-05-16T07:02:26.264892+0000","last_active":"2023-05-16T07:02:26.270652+0000","last_peered":"2023-05-16T07:02:26.270652+0000","last_clean":"2023-05-16T07:02:26.270652+0000","last_became_active":"2023-05-16T07:02:26.264659+0000","last_became_peered":"2023-05-16T07:02:26.264659+0000","last_unstale":"2023-05-16T07:02:26.270652+0000","last_undegraded":"2023-05-16T07:02:26.270652+0000","last_fullsized":"2023-05-16T07:02:26.270652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252103+0000","last_change":"2023-05-16T07:02:25.899001+0000","last_active":"2023-05-16T07:02:26.252103+0000","last_peered":"2023-05-16T07:02:26.252103+0000","last_clean":"2023-05-16T07:02:26.252103+0000","last_became_active":"2023-05-16T07:02:25.898724+0000","last_became_peered":"2023-05-16T07:02:25.898724+0000","last_unstale":"2023-05-16T07:02:26.252103+0000","last_undegraded":"2023-05-16T07:02:26.252103+0000","last_fullsized":"2023-05-16T07:02:26.252103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274782+0000","last_change":"2023-05-16T07:02:26.270764+0000","last_active":"2023-05-16T07:02:26.274782+0000","last_peered":"2023-05-16T07:02:26.274782+0000","last_clean":"2023-05-16T07:02:26.274782+0000","last_became_active":"2023-05-16T07:02:26.270171+0000","last_became_peered":"2023-05-16T07:02:26.270171+0000","last_unstale":"2023-05-16T07:02:26.274782+0000","last_undegraded":"2023-05-16T07:02:26.274782+0000","last_fullsized":"2023-05-16T07:02:26.274782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270540+0000","last_change":"2023-05-16T07:02:25.907500+0000","last_active":"2023-05-16T07:02:26.270540+0000","last_peered":"2023-05-16T07:02:26.270540+0000","last_clean":"2023-05-16T07:02:26.270540+0000","last_became_active":"2023-05-16T07:02:25.907202+0000","last_became_peered":"2023-05-16T07:02:25.907202+0000","last_unstale":"2023-05-16T07:02:26.270540+0000","last_undegraded":"2023-05-16T07:02:26.270540+0000","last_fullsized":"2023-05-16T07:02:26.270540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.262010+0000","last_change":"2023-05-16T07:02:26.262010+0000","last_active":"2023-05-16T07:02:26.262010+0000","last_peered":"2023-05-16T07:02:26.262010+0000","last_clean":"2023-05-16T07:02:26.262010+0000","last_became_active":"2023-05-16T07:02:26.261641+0000","last_became_peered":"2023-05-16T07:02:26.261641+0000","last_unstale":"2023-05-16T07:02:26.262010+0000","last_undegraded":"2023-05-16T07:02:26.262010+0000","last_fullsized":"2023-05-16T07:02:26.262010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253325+0000","last_change":"2023-05-16T07:02:25.900614+0000","last_active":"2023-05-16T07:02:26.253325+0000","last_peered":"2023-05-16T07:02:26.253325+0000","last_clean":"2023-05-16T07:02:26.253325+0000","last_became_active":"2023-05-16T07:02:25.898929+0000","last_became_peered":"2023-05-16T07:02:25.898929+0000","last_unstale":"2023-05-16T07:02:26.253325+0000","last_undegraded":"2023-05-16T07:02:26.253325+0000","last_fullsized":"2023-05-16T07:02:26.253325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273729+0000","last_change":"2023-05-16T07:02:26.269314+0000","last_active":"2023-05-16T07:02:26.273729+0000","last_peered":"2023-05-16T07:02:26.273729+0000","last_clean":"2023-05-16T07:02:26.273729+0000","last_became_active":"2023-05-16T07:02:26.269002+0000","last_became_peered":"2023-05-16T07:02:26.269002+0000","last_unstale":"2023-05-16T07:02:26.273729+0000","last_undegraded":"2023-05-16T07:02:26.273729+0000","last_fullsized":"2023-05-16T07:02:26.273729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.267696+0000","last_change":"2023-05-16T07:02:25.899453+0000","last_active":"2023-05-16T07:02:26.267696+0000","last_peered":"2023-05-16T07:02:26.267696+0000","last_clean":"2023-05-16T07:02:26.267696+0000","last_became_active":"2023-05-16T07:02:25.898915+0000","last_became_peered":"2023-05-16T07:02:25.898915+0000","last_unstale":"2023-05-16T07:02:26.267696+0000","last_undegraded":"2023-05-16T07:02:26.267696+0000","last_fullsized":"2023-05-16T07:02:26.267696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263984+0000","last_change":"2023-05-16T07:02:26.263984+0000","last_active":"2023-05-16T07:02:26.263984+0000","last_peered":"2023-05-16T07:02:26.263984+0000","last_clean":"2023-05-16T07:02:26.263984+0000","last_became_active":"2023-05-16T07:02:26.263440+0000","last_became_peered":"2023-05-16T07:02:26.263440+0000","last_unstale":"2023-05-16T07:02:26.263984+0000","last_undegraded":"2023-05-16T07:02:26.263984+0000","last_fullsized":"2023-05-16T07:02:26.263984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251262+0000","last_change":"2023-05-16T07:02:25.903155+0000","last_active":"2023-05-16T07:02:26.251262+0000","last_peered":"2023-05-16T07:02:26.251262+0000","last_clean":"2023-05-16T07:02:26.251262+0000","last_became_active":"2023-05-16T07:02:25.901362+0000","last_became_peered":"2023-05-16T07:02:25.901362+0000","last_unstale":"2023-05-16T07:02:26.251262+0000","last_undegraded":"2023-05-16T07:02:26.251262+0000","last_fullsized":"2023-05-16T07:02:26.251262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.262016+0000","last_change":"2023-05-16T07:02:26.262016+0000","last_active":"2023-05-16T07:02:26.262016+0000","last_peered":"2023-05-16T07:02:26.262016+0000","last_clean":"2023-05-16T07:02:26.262016+0000","last_became_active":"2023-05-16T07:02:26.261803+0000","last_became_peered":"2023-05-16T07:02:26.261803+0000","last_unstale":"2023-05-16T07:02:26.262016+0000","last_undegraded":"2023-05-16T07:02:26.262016+0000","last_fullsized":"2023-05-16T07:02:26.262016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253045+0000","last_change":"2023-05-16T07:02:25.904392+0000","last_active":"2023-05-16T07:02:26.253045+0000","last_peered":"2023-05-16T07:02:26.253045+0000","last_clean":"2023-05-16T07:02:26.253045+0000","last_became_active":"2023-05-16T07:02:25.904025+0000","last_became_peered":"2023-05-16T07:02:25.904025+0000","last_unstale":"2023-05-16T07:02:26.253045+0000","last_undegraded":"2023-05-16T07:02:26.253045+0000","last_fullsized":"2023-05-16T07:02:26.253045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266065+0000","last_change":"2023-05-16T07:02:26.263661+0000","last_active":"2023-05-16T07:02:26.266065+0000","last_peered":"2023-05-16T07:02:26.266065+0000","last_clean":"2023-05-16T07:02:26.266065+0000","last_became_active":"2023-05-16T07:02:26.263328+0000","last_became_peered":"2023-05-16T07:02:26.263328+0000","last_unstale":"2023-05-16T07:02:26.266065+0000","last_undegraded":"2023-05-16T07:02:26.266065+0000","last_fullsized":"2023-05-16T07:02:26.266065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252146+0000","last_change":"2023-05-16T07:02:25.905914+0000","last_active":"2023-05-16T07:02:26.252146+0000","last_peered":"2023-05-16T07:02:26.252146+0000","last_clean":"2023-05-16T07:02:26.252146+0000","last_became_active":"2023-05-16T07:02:25.905585+0000","last_became_peered":"2023-05-16T07:02:25.905585+0000","last_unstale":"2023-05-16T07:02:26.252146+0000","last_undegraded":"2023-05-16T07:02:26.252146+0000","last_fullsized":"2023-05-16T07:02:26.252146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271450+0000","last_change":"2023-05-16T07:02:26.271450+0000","last_active":"2023-05-16T07:02:26.271450+0000","last_peered":"2023-05-16T07:02:26.271450+0000","last_clean":"2023-05-16T07:02:26.271450+0000","last_became_active":"2023-05-16T07:02:26.269701+0000","last_became_peered":"2023-05-16T07:02:26.269701+0000","last_unstale":"2023-05-16T07:02:26.271450+0000","last_undegraded":"2023-05-16T07:02:26.271450+0000","last_fullsized":"2023-05-16T07:02:26.271450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253701+0000","last_change":"2023-05-16T07:02:25.908650+0000","last_active":"2023-05-16T07:02:26.253701+0000","last_peered":"2023-05-16T07:02:26.253701+0000","last_clean":"2023-05-16T07:02:26.253701+0000","last_became_active":"2023-05-16T07:02:25.908385+0000","last_became_peered":"2023-05-16T07:02:25.908385+0000","last_unstale":"2023-05-16T07:02:26.253701+0000","last_undegraded":"2023-05-16T07:02:26.253701+0000","last_fullsized":"2023-05-16T07:02:26.253701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272118+0000","last_change":"2023-05-16T07:02:26.272118+0000","last_active":"2023-05-16T07:02:26.272118+0000","last_peered":"2023-05-16T07:02:26.272118+0000","last_clean":"2023-05-16T07:02:26.272118+0000","last_became_active":"2023-05-16T07:02:26.271865+0000","last_became_peered":"2023-05-16T07:02:26.271865+0000","last_unstale":"2023-05-16T07:02:26.272118+0000","last_undegraded":"2023-05-16T07:02:26.272118+0000","last_fullsized":"2023-05-16T07:02:26.272118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.907097+0000","last_change":"2023-05-16T07:02:25.905216+0000","last_active":"2023-05-16T07:02:25.907097+0000","last_peered":"2023-05-16T07:02:25.907097+0000","last_clean":"2023-05-16T07:02:25.907097+0000","last_became_active":"2023-05-16T07:02:25.904954+0000","last_became_peered":"2023-05-16T07:02:25.904954+0000","last_unstale":"2023-05-16T07:02:25.907097+0000","last_undegraded":"2023-05-16T07:02:25.907097+0000","last_fullsized":"2023-05-16T07:02:25.907097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274803+0000","last_change":"2023-05-16T07:02:26.270884+0000","last_active":"2023-05-16T07:02:26.274803+0000","last_peered":"2023-05-16T07:02:26.274803+0000","last_clean":"2023-05-16T07:02:26.274803+0000","last_became_active":"2023-05-16T07:02:26.270473+0000","last_became_peered":"2023-05-16T07:02:26.270473+0000","last_unstale":"2023-05-16T07:02:26.274803+0000","last_undegraded":"2023-05-16T07:02:26.274803+0000","last_fullsized":"2023-05-16T07:02:26.274803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.903395+0000","last_change":"2023-05-16T07:02:25.900138+0000","last_active":"2023-05-16T07:02:25.903395+0000","last_peered":"2023-05-16T07:02:25.903395+0000","last_clean":"2023-05-16T07:02:25.903395+0000","last_became_active":"2023-05-16T07:02:25.899831+0000","last_became_peered":"2023-05-16T07:02:25.899831+0000","last_unstale":"2023-05-16T07:02:25.903395+0000","last_undegraded":"2023-05-16T07:02:25.903395+0000","last_fullsized":"2023-05-16T07:02:25.903395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273190+0000","last_change":"2023-05-16T07:02:26.273190+0000","last_active":"2023-05-16T07:02:26.273190+0000","last_peered":"2023-05-16T07:02:26.273190+0000","last_clean":"2023-05-16T07:02:26.273190+0000","last_became_active":"2023-05-16T07:02:26.272912+0000","last_became_peered":"2023-05-16T07:02:26.272912+0000","last_unstale":"2023-05-16T07:02:26.273190+0000","last_undegraded":"2023-05-16T07:02:26.273190+0000","last_fullsized":"2023-05-16T07:02:26.273190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250712+0000","last_change":"2023-05-16T07:02:25.902881+0000","last_active":"2023-05-16T07:02:26.250712+0000","last_peered":"2023-05-16T07:02:26.250712+0000","last_clean":"2023-05-16T07:02:26.250712+0000","last_became_active":"2023-05-16T07:02:25.902555+0000","last_became_peered":"2023-05-16T07:02:25.902555+0000","last_unstale":"2023-05-16T07:02:26.250712+0000","last_undegraded":"2023-05-16T07:02:26.250712+0000","last_fullsized":"2023-05-16T07:02:26.250712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266078+0000","last_change":"2023-05-16T07:02:26.262732+0000","last_active":"2023-05-16T07:02:26.266078+0000","last_peered":"2023-05-16T07:02:26.266078+0000","last_clean":"2023-05-16T07:02:26.266078+0000","last_became_active":"2023-05-16T07:02:26.258080+0000","last_became_peered":"2023-05-16T07:02:26.258080+0000","last_unstale":"2023-05-16T07:02:26.266078+0000","last_undegraded":"2023-05-16T07:02:26.266078+0000","last_fullsized":"2023-05-16T07:02:26.266078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.904405+0000","last_change":"2023-05-16T07:02:25.899623+0000","last_active":"2023-05-16T07:02:25.904405+0000","last_peered":"2023-05-16T07:02:25.904405+0000","last_clean":"2023-05-16T07:02:25.904405+0000","last_became_active":"2023-05-16T07:02:25.899225+0000","last_became_peered":"2023-05-16T07:02:25.899225+0000","last_unstale":"2023-05-16T07:02:25.904405+0000","last_undegraded":"2023-05-16T07:02:25.904405+0000","last_fullsized":"2023-05-16T07:02:25.904405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269443+0000","last_change":"2023-05-16T07:02:26.267767+0000","last_active":"2023-05-16T07:02:26.269443+0000","last_peered":"2023-05-16T07:02:26.269443+0000","last_clean":"2023-05-16T07:02:26.269443+0000","last_became_active":"2023-05-16T07:02:26.267496+0000","last_became_peered":"2023-05-16T07:02:26.267496+0000","last_unstale":"2023-05-16T07:02:26.269443+0000","last_undegraded":"2023-05-16T07:02:26.269443+0000","last_fullsized":"2023-05-16T07:02:26.269443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251732+0000","last_change":"2023-05-16T07:02:25.905706+0000","last_active":"2023-05-16T07:02:26.251732+0000","last_peered":"2023-05-16T07:02:26.251732+0000","last_clean":"2023-05-16T07:02:26.251732+0000","last_became_active":"2023-05-16T07:02:25.905387+0000","last_became_peered":"2023-05-16T07:02:25.905387+0000","last_unstale":"2023-05-16T07:02:26.251732+0000","last_undegraded":"2023-05-16T07:02:26.251732+0000","last_fullsized":"2023-05-16T07:02:26.251732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273026+0000","last_change":"2023-05-16T07:02:26.264777+0000","last_active":"2023-05-16T07:02:26.273026+0000","last_peered":"2023-05-16T07:02:26.273026+0000","last_clean":"2023-05-16T07:02:26.273026+0000","last_became_active":"2023-05-16T07:02:26.264481+0000","last_became_peered":"2023-05-16T07:02:26.264481+0000","last_unstale":"2023-05-16T07:02:26.273026+0000","last_undegraded":"2023-05-16T07:02:26.273026+0000","last_fullsized":"2023-05-16T07:02:26.273026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.909203+0000","last_change":"2023-05-16T07:02:25.907961+0000","last_active":"2023-05-16T07:02:25.909203+0000","last_peered":"2023-05-16T07:02:25.909203+0000","last_clean":"2023-05-16T07:02:25.909203+0000","last_became_active":"2023-05-16T07:02:25.907665+0000","last_became_peered":"2023-05-16T07:02:25.907665+0000","last_unstale":"2023-05-16T07:02:25.909203+0000","last_undegraded":"2023-05-16T07:02:25.909203+0000","last_fullsized":"2023-05-16T07:02:25.909203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261333+0000","last_change":"2023-05-16T07:02:26.257095+0000","last_active":"2023-05-16T07:02:26.261333+0000","last_peered":"2023-05-16T07:02:26.261333+0000","last_clean":"2023-05-16T07:02:26.261333+0000","last_became_active":"2023-05-16T07:02:26.256845+0000","last_became_peered":"2023-05-16T07:02:26.256845+0000","last_unstale":"2023-05-16T07:02:26.261333+0000","last_undegraded":"2023-05-16T07:02:26.261333+0000","last_fullsized":"2023-05-16T07:02:26.261333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252260+0000","last_change":"2023-05-16T07:02:25.907092+0000","last_active":"2023-05-16T07:02:26.252260+0000","last_peered":"2023-05-16T07:02:26.252260+0000","last_clean":"2023-05-16T07:02:26.252260+0000","last_became_active":"2023-05-16T07:02:25.906488+0000","last_became_peered":"2023-05-16T07:02:25.906488+0000","last_unstale":"2023-05-16T07:02:26.252260+0000","last_undegraded":"2023-05-16T07:02:26.252260+0000","last_fullsized":"2023-05-16T07:02:26.252260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274666+0000","last_change":"2023-05-16T07:02:26.272727+0000","last_active":"2023-05-16T07:02:26.274666+0000","last_peered":"2023-05-16T07:02:26.274666+0000","last_clean":"2023-05-16T07:02:26.274666+0000","last_became_active":"2023-05-16T07:02:26.270729+0000","last_became_peered":"2023-05-16T07:02:26.270729+0000","last_unstale":"2023-05-16T07:02:26.274666+0000","last_undegraded":"2023-05-16T07:02:26.274666+0000","last_fullsized":"2023-05-16T07:02:26.274666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270991+0000","last_change":"2023-05-16T07:02:25.902010+0000","last_active":"2023-05-16T07:02:26.270991+0000","last_peered":"2023-05-16T07:02:26.270991+0000","last_clean":"2023-05-16T07:02:26.270991+0000","last_became_active":"2023-05-16T07:02:25.900891+0000","last_became_peered":"2023-05-16T07:02:25.900891+0000","last_unstale":"2023-05-16T07:02:26.270991+0000","last_undegraded":"2023-05-16T07:02:26.270991+0000","last_fullsized":"2023-05-16T07:02:26.270991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271884+0000","last_change":"2023-05-16T07:02:26.268192+0000","last_active":"2023-05-16T07:02:26.271884+0000","last_peered":"2023-05-16T07:02:26.271884+0000","last_clean":"2023-05-16T07:02:26.271884+0000","last_became_active":"2023-05-16T07:02:26.263201+0000","last_became_peered":"2023-05-16T07:02:26.263201+0000","last_unstale":"2023-05-16T07:02:26.271884+0000","last_undegraded":"2023-05-16T07:02:26.271884+0000","last_fullsized":"2023-05-16T07:02:26.271884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250995+0000","last_change":"2023-05-16T07:02:25.899500+0000","last_active":"2023-05-16T07:02:26.250995+0000","last_peered":"2023-05-16T07:02:26.250995+0000","last_clean":"2023-05-16T07:02:26.250995+0000","last_became_active":"2023-05-16T07:02:25.898726+0000","last_became_peered":"2023-05-16T07:02:25.898726+0000","last_unstale":"2023-05-16T07:02:26.250995+0000","last_undegraded":"2023-05-16T07:02:26.250995+0000","last_fullsized":"2023-05-16T07:02:26.250995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274483+0000","last_change":"2023-05-16T07:02:26.273388+0000","last_active":"2023-05-16T07:02:26.274483+0000","last_peered":"2023-05-16T07:02:26.274483+0000","last_clean":"2023-05-16T07:02:26.274483+0000","last_became_active":"2023-05-16T07:02:26.273112+0000","last_became_peered":"2023-05-16T07:02:26.273112+0000","last_unstale":"2023-05-16T07:02:26.274483+0000","last_undegraded":"2023-05-16T07:02:26.274483+0000","last_fullsized":"2023-05-16T07:02:26.274483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270551+0000","last_change":"2023-05-16T07:02:25.905564+0000","last_active":"2023-05-16T07:02:26.270551+0000","last_peered":"2023-05-16T07:02:26.270551+0000","last_clean":"2023-05-16T07:02:26.270551+0000","last_became_active":"2023-05-16T07:02:25.905277+0000","last_became_peered":"2023-05-16T07:02:25.905277+0000","last_unstale":"2023-05-16T07:02:26.270551+0000","last_undegraded":"2023-05-16T07:02:26.270551+0000","last_fullsized":"2023-05-16T07:02:26.270551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265761+0000","last_change":"2023-05-16T07:02:26.265761+0000","last_active":"2023-05-16T07:02:26.265761+0000","last_peered":"2023-05-16T07:02:26.265761+0000","last_clean":"2023-05-16T07:02:26.265761+0000","last_became_active":"2023-05-16T07:02:26.265515+0000","last_became_peered":"2023-05-16T07:02:26.265515+0000","last_unstale":"2023-05-16T07:02:26.265761+0000","last_undegraded":"2023-05-16T07:02:26.265761+0000","last_fullsized":"2023-05-16T07:02:26.265761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254410+0000","last_change":"2023-05-16T07:02:25.900333+0000","last_active":"2023-05-16T07:02:26.254410+0000","last_peered":"2023-05-16T07:02:26.254410+0000","last_clean":"2023-05-16T07:02:26.254410+0000","last_became_active":"2023-05-16T07:02:25.900071+0000","last_became_peered":"2023-05-16T07:02:25.900071+0000","last_unstale":"2023-05-16T07:02:26.254410+0000","last_undegraded":"2023-05-16T07:02:26.254410+0000","last_fullsized":"2023-05-16T07:02:26.254410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271814+0000","last_change":"2023-05-16T07:02:26.269160+0000","last_active":"2023-05-16T07:02:26.271814+0000","last_peered":"2023-05-16T07:02:26.271814+0000","last_clean":"2023-05-16T07:02:26.271814+0000","last_became_active":"2023-05-16T07:02:26.268896+0000","last_became_peered":"2023-05-16T07:02:26.268896+0000","last_unstale":"2023-05-16T07:02:26.271814+0000","last_undegraded":"2023-05-16T07:02:26.271814+0000","last_fullsized":"2023-05-16T07:02:26.271814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253776+0000","last_change":"2023-05-16T07:02:25.902998+0000","last_active":"2023-05-16T07:02:26.253776+0000","last_peered":"2023-05-16T07:02:26.253776+0000","last_clean":"2023-05-16T07:02:26.253776+0000","last_became_active":"2023-05-16T07:02:25.900472+0000","last_became_peered":"2023-05-16T07:02:25.900472+0000","last_unstale":"2023-05-16T07:02:26.253776+0000","last_undegraded":"2023-05-16T07:02:26.253776+0000","last_fullsized":"2023-05-16T07:02:26.253776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269460+0000","last_change":"2023-05-16T07:02:26.268175+0000","last_active":"2023-05-16T07:02:26.269460+0000","last_peered":"2023-05-16T07:02:26.269460+0000","last_clean":"2023-05-16T07:02:26.269460+0000","last_became_active":"2023-05-16T07:02:26.267874+0000","last_became_peered":"2023-05-16T07:02:26.267874+0000","last_unstale":"2023-05-16T07:02:26.269460+0000","last_undegraded":"2023-05-16T07:02:26.269460+0000","last_fullsized":"2023-05-16T07:02:26.269460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265758+0000","last_change":"2023-05-16T07:02:25.903192+0000","last_active":"2023-05-16T07:02:26.265758+0000","last_peered":"2023-05-16T07:02:26.265758+0000","last_clean":"2023-05-16T07:02:26.265758+0000","last_became_active":"2023-05-16T07:02:25.902859+0000","last_became_peered":"2023-05-16T07:02:25.902859+0000","last_unstale":"2023-05-16T07:02:26.265758+0000","last_undegraded":"2023-05-16T07:02:26.265758+0000","last_fullsized":"2023-05-16T07:02:26.265758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269471+0000","last_change":"2023-05-16T07:02:26.267628+0000","last_active":"2023-05-16T07:02:26.269471+0000","last_peered":"2023-05-16T07:02:26.269471+0000","last_clean":"2023-05-16T07:02:26.269471+0000","last_became_active":"2023-05-16T07:02:26.267196+0000","last_became_peered":"2023-05-16T07:02:26.267196+0000","last_unstale":"2023-05-16T07:02:26.269471+0000","last_undegraded":"2023-05-16T07:02:26.269471+0000","last_fullsized":"2023-05-16T07:02:26.269471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.894868+0000","last_change":"2023-05-16T07:02:25.890842+0000","last_active":"2023-05-16T07:02:25.894868+0000","last_peered":"2023-05-16T07:02:25.894868+0000","last_clean":"2023-05-16T07:02:25.894868+0000","last_became_active":"2023-05-16T07:02:25.890548+0000","last_became_peered":"2023-05-16T07:02:25.890548+0000","last_unstale":"2023-05-16T07:02:25.894868+0000","last_undegraded":"2023-05-16T07:02:25.894868+0000","last_fullsized":"2023-05-16T07:02:25.894868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273817+0000","last_change":"2023-05-16T07:02:26.269745+0000","last_active":"2023-05-16T07:02:26.273817+0000","last_peered":"2023-05-16T07:02:26.273817+0000","last_clean":"2023-05-16T07:02:26.273817+0000","last_became_active":"2023-05-16T07:02:26.269454+0000","last_became_peered":"2023-05-16T07:02:26.269454+0000","last_unstale":"2023-05-16T07:02:26.273817+0000","last_undegraded":"2023-05-16T07:02:26.273817+0000","last_fullsized":"2023-05-16T07:02:26.273817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254294+0000","last_change":"2023-05-16T07:02:25.892810+0000","last_active":"2023-05-16T07:02:26.254294+0000","last_peered":"2023-05-16T07:02:26.254294+0000","last_clean":"2023-05-16T07:02:26.254294+0000","last_became_active":"2023-05-16T07:02:25.892538+0000","last_became_peered":"2023-05-16T07:02:25.892538+0000","last_unstale":"2023-05-16T07:02:26.254294+0000","last_undegraded":"2023-05-16T07:02:26.254294+0000","last_fullsized":"2023-05-16T07:02:26.254294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272472+0000","last_change":"2023-05-16T07:02:26.271699+0000","last_active":"2023-05-16T07:02:26.272472+0000","last_peered":"2023-05-16T07:02:26.272472+0000","last_clean":"2023-05-16T07:02:26.272472+0000","last_became_active":"2023-05-16T07:02:26.271437+0000","last_became_peered":"2023-05-16T07:02:26.271437+0000","last_unstale":"2023-05-16T07:02:26.272472+0000","last_undegraded":"2023-05-16T07:02:26.272472+0000","last_fullsized":"2023-05-16T07:02:26.272472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253404+0000","last_change":"2023-05-16T07:02:25.899544+0000","last_active":"2023-05-16T07:02:26.253404+0000","last_peered":"2023-05-16T07:02:26.253404+0000","last_clean":"2023-05-16T07:02:26.253404+0000","last_became_active":"2023-05-16T07:02:25.898789+0000","last_became_peered":"2023-05-16T07:02:25.898789+0000","last_unstale":"2023-05-16T07:02:26.253404+0000","last_undegraded":"2023-05-16T07:02:26.253404+0000","last_fullsized":"2023-05-16T07:02:26.253404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.264508+0000","last_change":"2023-05-16T07:02:26.262702+0000","last_active":"2023-05-16T07:02:26.264508+0000","last_peered":"2023-05-16T07:02:26.264508+0000","last_clean":"2023-05-16T07:02:26.264508+0000","last_became_active":"2023-05-16T07:02:26.262411+0000","last_became_peered":"2023-05-16T07:02:26.262411+0000","last_unstale":"2023-05-16T07:02:26.264508+0000","last_undegraded":"2023-05-16T07:02:26.264508+0000","last_fullsized":"2023-05-16T07:02:26.264508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.276391+0000","last_change":"2023-05-16T07:02:25.898185+0000","last_active":"2023-05-16T07:02:26.276391+0000","last_peered":"2023-05-16T07:02:26.276391+0000","last_clean":"2023-05-16T07:02:26.276391+0000","last_became_active":"2023-05-16T07:02:25.897879+0000","last_became_peered":"2023-05-16T07:02:25.897879+0000","last_unstale":"2023-05-16T07:02:26.276391+0000","last_undegraded":"2023-05-16T07:02:26.276391+0000","last_fullsized":"2023-05-16T07:02:26.276391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261293+0000","last_change":"2023-05-16T07:02:26.259483+0000","last_active":"2023-05-16T07:02:26.261293+0000","last_peered":"2023-05-16T07:02:26.261293+0000","last_clean":"2023-05-16T07:02:26.261293+0000","last_became_active":"2023-05-16T07:02:26.259204+0000","last_became_peered":"2023-05-16T07:02:26.259204+0000","last_unstale":"2023-05-16T07:02:26.261293+0000","last_undegraded":"2023-05-16T07:02:26.261293+0000","last_fullsized":"2023-05-16T07:02:26.261293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251063+0000","last_change":"2023-05-16T07:02:25.904881+0000","last_active":"2023-05-16T07:02:26.251063+0000","last_peered":"2023-05-16T07:02:26.251063+0000","last_clean":"2023-05-16T07:02:26.251063+0000","last_became_active":"2023-05-16T07:02:25.904608+0000","last_became_peered":"2023-05-16T07:02:25.904608+0000","last_unstale":"2023-05-16T07:02:26.251063+0000","last_undegraded":"2023-05-16T07:02:26.251063+0000","last_fullsized":"2023-05-16T07:02:26.251063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271560+0000","last_change":"2023-05-16T07:02:26.267010+0000","last_active":"2023-05-16T07:02:26.271560+0000","last_peered":"2023-05-16T07:02:26.271560+0000","last_clean":"2023-05-16T07:02:26.271560+0000","last_became_active":"2023-05-16T07:02:26.266536+0000","last_became_peered":"2023-05-16T07:02:26.266536+0000","last_unstale":"2023-05-16T07:02:26.271560+0000","last_undegraded":"2023-05-16T07:02:26.271560+0000","last_fullsized":"2023-05-16T07:02:26.271560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250515+0000","last_change":"2023-05-16T07:02:25.904051+0000","last_active":"2023-05-16T07:02:26.250515+0000","last_peered":"2023-05-16T07:02:26.250515+0000","last_clean":"2023-05-16T07:02:26.250515+0000","last_became_active":"2023-05-16T07:02:25.903808+0000","last_became_peered":"2023-05-16T07:02:25.903808+0000","last_unstale":"2023-05-16T07:02:26.250515+0000","last_undegraded":"2023-05-16T07:02:26.250515+0000","last_fullsized":"2023-05-16T07:02:26.250515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270685+0000","last_change":"2023-05-16T07:02:26.264443+0000","last_active":"2023-05-16T07:02:26.270685+0000","last_peered":"2023-05-16T07:02:26.270685+0000","last_clean":"2023-05-16T07:02:26.270685+0000","last_became_active":"2023-05-16T07:02:26.263678+0000","last_became_peered":"2023-05-16T07:02:26.263678+0000","last_unstale":"2023-05-16T07:02:26.270685+0000","last_undegraded":"2023-05-16T07:02:26.270685+0000","last_fullsized":"2023-05-16T07:02:26.270685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254540+0000","last_change":"2023-05-16T07:02:25.908234+0000","last_active":"2023-05-16T07:02:26.254540+0000","last_peered":"2023-05-16T07:02:26.254540+0000","last_clean":"2023-05-16T07:02:26.254540+0000","last_became_active":"2023-05-16T07:02:25.907873+0000","last_became_peered":"2023-05-16T07:02:25.907873+0000","last_unstale":"2023-05-16T07:02:26.254540+0000","last_undegraded":"2023-05-16T07:02:26.254540+0000","last_fullsized":"2023-05-16T07:02:26.254540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266820+0000","last_change":"2023-05-16T07:02:26.264091+0000","last_active":"2023-05-16T07:02:26.266820+0000","last_peered":"2023-05-16T07:02:26.266820+0000","last_clean":"2023-05-16T07:02:26.266820+0000","last_became_active":"2023-05-16T07:02:26.263428+0000","last_became_peered":"2023-05-16T07:02:26.263428+0000","last_unstale":"2023-05-16T07:02:26.266820+0000","last_undegraded":"2023-05-16T07:02:26.266820+0000","last_fullsized":"2023-05-16T07:02:26.266820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.907226+0000","last_change":"2023-05-16T07:02:25.906021+0000","last_active":"2023-05-16T07:02:25.907226+0000","last_peered":"2023-05-16T07:02:25.907226+0000","last_clean":"2023-05-16T07:02:25.907226+0000","last_became_active":"2023-05-16T07:02:25.905737+0000","last_became_peered":"2023-05-16T07:02:25.905737+0000","last_unstale":"2023-05-16T07:02:25.907226+0000","last_undegraded":"2023-05-16T07:02:25.907226+0000","last_fullsized":"2023-05-16T07:02:25.907226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269451+0000","last_change":"2023-05-16T07:02:26.267451+0000","last_active":"2023-05-16T07:02:26.269451+0000","last_peered":"2023-05-16T07:02:26.269451+0000","last_clean":"2023-05-16T07:02:26.269451+0000","last_became_active":"2023-05-16T07:02:26.266832+0000","last_became_peered":"2023-05-16T07:02:26.266832+0000","last_unstale":"2023-05-16T07:02:26.269451+0000","last_undegraded":"2023-05-16T07:02:26.269451+0000","last_fullsized":"2023-05-16T07:02:26.269451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254418+0000","last_change":"2023-05-16T07:02:25.908040+0000","last_active":"2023-05-16T07:02:26.254418+0000","last_peered":"2023-05-16T07:02:26.254418+0000","last_clean":"2023-05-16T07:02:26.254418+0000","last_became_active":"2023-05-16T07:02:25.907139+0000","last_became_peered":"2023-05-16T07:02:25.907139+0000","last_unstale":"2023-05-16T07:02:26.254418+0000","last_undegraded":"2023-05-16T07:02:26.254418+0000","last_fullsized":"2023-05-16T07:02:26.254418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273639+0000","last_change":"2023-05-16T07:02:26.268915+0000","last_active":"2023-05-16T07:02:26.273639+0000","last_peered":"2023-05-16T07:02:26.273639+0000","last_clean":"2023-05-16T07:02:26.273639+0000","last_became_active":"2023-05-16T07:02:26.263749+0000","last_became_peered":"2023-05-16T07:02:26.263749+0000","last_unstale":"2023-05-16T07:02:26.273639+0000","last_undegraded":"2023-05-16T07:02:26.273639+0000","last_fullsized":"2023-05-16T07:02:26.273639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261994+0000","last_change":"2023-05-16T07:02:25.904081+0000","last_active":"2023-05-16T07:02:26.261994+0000","last_peered":"2023-05-16T07:02:26.261994+0000","last_clean":"2023-05-16T07:02:26.261994+0000","last_became_active":"2023-05-16T07:02:25.903797+0000","last_became_peered":"2023-05-16T07:02:25.903797+0000","last_unstale":"2023-05-16T07:02:26.261994+0000","last_undegraded":"2023-05-16T07:02:26.261994+0000","last_fullsized":"2023-05-16T07:02:26.261994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261837+0000","last_change":"2023-05-16T07:02:26.258411+0000","last_active":"2023-05-16T07:02:26.261837+0000","last_peered":"2023-05-16T07:02:26.261837+0000","last_clean":"2023-05-16T07:02:26.261837+0000","last_became_active":"2023-05-16T07:02:26.258108+0000","last_became_peered":"2023-05-16T07:02:26.258108+0000","last_unstale":"2023-05-16T07:02:26.261837+0000","last_undegraded":"2023-05-16T07:02:26.261837+0000","last_fullsized":"2023-05-16T07:02:26.261837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.268905+0000","last_change":"2023-05-16T07:02:25.900588+0000","last_active":"2023-05-16T07:02:26.268905+0000","last_peered":"2023-05-16T07:02:26.268905+0000","last_clean":"2023-05-16T07:02:26.268905+0000","last_became_active":"2023-05-16T07:02:25.900336+0000","last_became_peered":"2023-05-16T07:02:25.900336+0000","last_unstale":"2023-05-16T07:02:26.268905+0000","last_undegraded":"2023-05-16T07:02:26.268905+0000","last_fullsized":"2023-05-16T07:02:26.268905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269680+0000","last_change":"2023-05-16T07:02:26.267380+0000","last_active":"2023-05-16T07:02:26.269680+0000","last_peered":"2023-05-16T07:02:26.269680+0000","last_clean":"2023-05-16T07:02:26.269680+0000","last_became_active":"2023-05-16T07:02:26.267111+0000","last_became_peered":"2023-05-16T07:02:26.267111+0000","last_unstale":"2023-05-16T07:02:26.269680+0000","last_undegraded":"2023-05-16T07:02:26.269680+0000","last_fullsized":"2023-05-16T07:02:26.269680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263166+0000","last_change":"2023-05-16T07:02:25.902771+0000","last_active":"2023-05-16T07:02:26.263166+0000","last_peered":"2023-05-16T07:02:26.263166+0000","last_clean":"2023-05-16T07:02:26.263166+0000","last_became_active":"2023-05-16T07:02:25.902327+0000","last_became_peered":"2023-05-16T07:02:25.902327+0000","last_unstale":"2023-05-16T07:02:26.263166+0000","last_undegraded":"2023-05-16T07:02:26.263166+0000","last_fullsized":"2023-05-16T07:02:26.263166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275129+0000","last_change":"2023-05-16T07:02:26.270942+0000","last_active":"2023-05-16T07:02:26.275129+0000","last_peered":"2023-05-16T07:02:26.275129+0000","last_clean":"2023-05-16T07:02:26.275129+0000","last_became_active":"2023-05-16T07:02:26.270608+0000","last_became_peered":"2023-05-16T07:02:26.270608+0000","last_unstale":"2023-05-16T07:02:26.275129+0000","last_undegraded":"2023-05-16T07:02:26.275129+0000","last_fullsized":"2023-05-16T07:02:26.275129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251411+0000","last_change":"2023-05-16T07:02:25.901785+0000","last_active":"2023-05-16T07:02:26.251411+0000","last_peered":"2023-05-16T07:02:26.251411+0000","last_clean":"2023-05-16T07:02:26.251411+0000","last_became_active":"2023-05-16T07:02:25.901529+0000","last_became_peered":"2023-05-16T07:02:25.901529+0000","last_unstale":"2023-05-16T07:02:26.251411+0000","last_undegraded":"2023-05-16T07:02:26.251411+0000","last_fullsized":"2023-05-16T07:02:26.251411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271585+0000","last_change":"2023-05-16T07:02:26.267468+0000","last_active":"2023-05-16T07:02:26.271585+0000","last_peered":"2023-05-16T07:02:26.271585+0000","last_clean":"2023-05-16T07:02:26.271585+0000","last_became_active":"2023-05-16T07:02:26.266500+0000","last_became_peered":"2023-05-16T07:02:26.266500+0000","last_unstale":"2023-05-16T07:02:26.271585+0000","last_undegraded":"2023-05-16T07:02:26.271585+0000","last_fullsized":"2023-05-16T07:02:26.271585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-05-16T07:02:25.906312+0000","last_change":"2023-05-16T07:02:25.904335+0000","last_active":"2023-05-16T07:02:25.906312+0000","last_peered":"2023-05-16T07:02:25.906312+0000","last_clean":"2023-05-16T07:02:25.906312+0000","last_became_active":"2023-05-16T07:02:25.903977+0000","last_became_peered":"2023-05-16T07:02:25.903977+0000","last_unstale":"2023-05-16T07:02:25.906312+0000","last_undegraded":"2023-05-16T07:02:25.906312+0000","last_fullsized":"2023-05-16T07:02:25.906312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266029+0000","last_change":"2023-05-16T07:02:26.264992+0000","last_active":"2023-05-16T07:02:26.266029+0000","last_peered":"2023-05-16T07:02:26.266029+0000","last_clean":"2023-05-16T07:02:26.266029+0000","last_became_active":"2023-05-16T07:02:26.264703+0000","last_became_peered":"2023-05-16T07:02:26.264703+0000","last_unstale":"2023-05-16T07:02:26.266029+0000","last_undegraded":"2023-05-16T07:02:26.266029+0000","last_fullsized":"2023-05-16T07:02:26.266029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271717+0000","last_change":"2023-05-16T07:02:25.899178+0000","last_active":"2023-05-16T07:02:26.271717+0000","last_peered":"2023-05-16T07:02:26.271717+0000","last_clean":"2023-05-16T07:02:26.271717+0000","last_became_active":"2023-05-16T07:02:25.898835+0000","last_became_peered":"2023-05-16T07:02:25.898835+0000","last_unstale":"2023-05-16T07:02:26.271717+0000","last_undegraded":"2023-05-16T07:02:26.271717+0000","last_fullsized":"2023-05-16T07:02:26.271717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5252,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366588,"statfs":{"total":96636764160,"available":96631386112,"internally_reserved":0,"allocated":258048,"data_stored":86751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5168,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366672,"statfs":{"total":96636764160,"available":96631472128,"internally_reserved":0,"allocated":237568,"data_stored":86783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5148,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366692,"statfs":{"total":96636764160,"available":96631492608,"internally_reserved":0,"allocated":217088,"data_stored":78633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":233472,"data_stored":85407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5216,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366624,"statfs":{"total":96636764160,"available":96631422976,"internally_reserved":0,"allocated":221184,"data_stored":78723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5244,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366596,"statfs":{"total":96636764160,"available":96631394304,"internally_reserved":0,"allocated":249856,"data_stored":86929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5160,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366680,"statfs":{"total":96636764160,"available":96631480320,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:02:29.570 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:02:29.570 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-05-16T07:02:29.571 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-05-16T07:02:29.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-05-16T07:02:29.572 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-05-16T07:02:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.708+0000 7f94edaf0700 1 Processor -- start 2023-05-16T07:02:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.708+0000 7f94edaf0700 1 -- start start 2023-05-16T07:02:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94edaf0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e80696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94edaf0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94e8069b90 0x7f94e80c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94edaf0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e80c0660 con 0x7f94e8154170 2023-05-16T07:02:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94edaf0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e806a020 con 0x7f94e8069b90 2023-05-16T07:02:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94edaf0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80bfd30 con 0x7f94e8154550 2023-05-16T07:02:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94e8069b90 0x7f94e80c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f94e8154170 0x7f94e80c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51290/0 (socket says 172.21.15.92:51290) 2023-05-16T07:02:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e77fe700 1 -- 172.21.15.92:0/3131748151 learned_addr learned my addr 172.21.15.92:0/3131748151 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e6ffd700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e80696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e7fff700 1 -- 172.21.15.92:0/3131748151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 msgr2=0x7f94e80696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e7fff700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e80696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e7fff700 1 -- 172.21.15.92:0/3131748151 >> v1:172.21.15.92:6789/0 conn(0x7f94e8154170 legacy=0x7f94e80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e7fff700 1 -- 172.21.15.92:0/3131748151 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80c0010 con 0x7f94e8069b90 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e67fc700 1 -- 172.21.15.92:0/3131748151 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051605039 0 0) 0x7f94e80c0660 con 0x7f94e8154170 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e6ffd700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e80696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e7fff700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8069b90 0x7f94e80c0af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f94e0009e60 tx=0x7f94e0009780).ready entity=mon.0 client_cookie=3e33b100ee857064 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.709+0000 7f94e67fc700 1 -- 172.21.15.92:0/3131748151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000e070 con 0x7f94e8069b90 2023-05-16T07:02:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94e67fc700 1 -- 172.21.15.92:0/3131748151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e000bc30 con 0x7f94e8069b90 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94e67fc700 1 -- 172.21.15.92:0/3131748151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0012410 con 0x7f94e8069b90 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 -- 172.21.15.92:0/3131748151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8069b90 msgr2=0x7f94e80c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8069b90 0x7f94e80c0af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f94e0009e60 tx=0x7f94e0009780).stop 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 -- 172.21.15.92:0/3131748151 shutdown_connections 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8069b90 0x7f94e80c0af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 --2- 172.21.15.92:0/3131748151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e80696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 -- 172.21.15.92:0/3131748151 >> 172.21.15.92:0/3131748151 conn(0x7f94e80cb700 msgr2=0x7f94e80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 -- 172.21.15.92:0/3131748151 shutdown_connections 2023-05-16T07:02:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.710+0000 7f94edaf0700 1 -- 172.21.15.92:0/3131748151 wait complete. 2023-05-16T07:02:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.711+0000 7f94edaf0700 1 Processor -- start 2023-05-16T07:02:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.711+0000 7f94edaf0700 1 -- start start 2023-05-16T07:02:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94edaf0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e8069b90 0x7f94e8158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94edaf0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 0x7f94e8158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94edaf0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e8163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94edaf0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80adc30 con 0x7f94e8154170 2023-05-16T07:02:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94edaf0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e8151550 con 0x7f94e8069b90 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94edaf0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80ade80 con 0x7f94e8154550 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 0x7f94e8158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 0x7f94e8158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40550/0 (socket says 172.21.15.92:40550) 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 -- 172.21.15.92:0/2620354531 learned_addr learned my addr 172.21.15.92:0/2620354531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 -- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 msgr2=0x7f94e8163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e8163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 -- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e8069b90 msgr2=0x7f94e8158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e8069b90 0x7f94e8158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 -- 172.21.15.92:0/2620354531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0009a20 con 0x7f94e8154170 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.712+0000 7f94e6ffd700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 0x7f94e8158950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f94dc00c670 tx=0x7f94dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.713+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc015070 con 0x7f94e8154170 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.713+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc010580 con 0x7f94e8154170 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.713+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc010900 con 0x7f94e8154170 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.713+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e80ae160 con 0x7f94e8154170 2023-05-16T07:02:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.713+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e80ae720 con 0x7f94e8154170 2023-05-16T07:02:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.713+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e80666c0 con 0x7f94e8154170 2023-05-16T07:02:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.714+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f94dc010d30 con 0x7f94e8154170 2023-05-16T07:02:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.714+0000 7f94cf7fe700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f94d405ed90 0x7f94d4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.714+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f94dc0797b0 con 0x7f94e8154170 2023-05-16T07:02:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.714+0000 7f94e77fe700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f94d405ed90 0x7f94d4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.715+0000 7f94e77fe700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f94d405ed90 0x7f94d4061250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f94d80011e0 tx=0x7f94d8007850).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.718+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f94dc04ad50 con 0x7f94e8154170 2023-05-16T07:02:29.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.925+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f94e806c350 con 0x7f94e8154170 2023-05-16T07:02:29.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.928+0000 7f94cf7fe700 1 -- 172.21.15.92:0/2620354531 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+148523 (secure 0 0 0) 0x7f94dc04e370 con 0x7f94e8154170 2023-05-16T07:02:29.933 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:29.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f94d405ed90 msgr2=0x7f94d4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f94d405ed90 0x7f94d4061250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f94d80011e0 tx=0x7f94d8007850).stop 2023-05-16T07:02:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 msgr2=0x7f94e8158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 0x7f94e8158950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f94dc00c670 tx=0x7f94dc008040).stop 2023-05-16T07:02:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 shutdown_connections 2023-05-16T07:02:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f94d405ed90 0x7f94d4061250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94e8154550 0x7f94e8163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:29.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e8069b90 0x7f94e8158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:29.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 --2- 172.21.15.92:0/2620354531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94e8154170 0x7f94e8158950 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:29.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 >> 172.21.15.92:0/2620354531 conn(0x7f94e80cb700 msgr2=0x7f94e8154960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:29.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.930+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 shutdown_connections 2023-05-16T07:02:29.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:29.931+0000 7f94edaf0700 1 -- 172.21.15.92:0/2620354531 wait complete. 2023-05-16T07:02:29.947 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":4,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":86301},{"type":"v1","addr":"172.21.15.158:6833","nonce":86301}]},"active_addr":"172.21.15.158:6833/86301","active_change":"2023-05-16T07:02:13.444004+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":2276917568}]},{"addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":1428741227}]}]} 2023-05-16T07:02:29.952 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-05-16T07:02:29.952 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-05-16T07:02:29.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.060+0000 7f1ebefb4700 1 Processor -- start 2023-05-16T07:02:30.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.060+0000 7f1ebefb4700 1 -- start start 2023-05-16T07:02:30.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebefb4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1eb81517f0 0x7f1eb8156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebefb4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1eb80a76f0 0x7f1eb80a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebefb4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80c8750 con 0x7f1eb8151bd0 2023-05-16T07:02:30.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebefb4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb8155d70 con 0x7f1eb80a76f0 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebefb4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb80c8470 con 0x7f1eb81517f0 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1eb80a76f0 0x7f1eb80a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1eb80a76f0 0x7f1eb80a7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40560/0 (socket says 172.21.15.92:40560) 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 -- 172.21.15.92:0/75629157 learned_addr learned my addr 172.21.15.92:0/75629157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebdfb2700 1 --2- 172.21.15.92:0/75629157 >> v2:172.21.15.158:3300/0 conn(0x7f1eb81517f0 0x7f1eb8156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 -- 172.21.15.92:0/75629157 >> v2:172.21.15.158:3300/0 conn(0x7f1eb81517f0 msgr2=0x7f1eb8156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 --2- 172.21.15.92:0/75629157 >> v2:172.21.15.158:3300/0 conn(0x7f1eb81517f0 0x7f1eb8156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 -- 172.21.15.92:0/75629157 >> v1:172.21.15.158:6789/0 conn(0x7f1eb8151bd0 legacy=0x7f1eb8152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 -- 172.21.15.92:0/75629157 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8155720 con 0x7f1eb80a76f0 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebe7b3700 1 --2- 172.21.15.92:0/75629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb80a76f0 0x7f1eb80a7af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1eb8060670 tx=0x7f1eb40090f0).ready entity=mon.0 client_cookie=1d674647d8d188a0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebcfb0700 1 -- 172.21.15.92:0/75629157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400b600 con 0x7f1eb80a76f0 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.061+0000 7f1ebcfb0700 1 -- 172.21.15.92:0/75629157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb400b760 con 0x7f1eb80a76f0 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebcfb0700 1 -- 172.21.15.92:0/75629157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400bb00 con 0x7f1eb80a76f0 2023-05-16T07:02:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 -- 172.21.15.92:0/75629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb80a76f0 msgr2=0x7f1eb80a7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/75629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb80a76f0 0x7f1eb80a7af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1eb8060670 tx=0x7f1eb40090f0).stop 2023-05-16T07:02:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 -- 172.21.15.92:0/75629157 shutdown_connections 2023-05-16T07:02:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/75629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb80a76f0 0x7f1eb80a7af0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/75629157 >> v2:172.21.15.158:3300/0 conn(0x7f1eb81517f0 0x7f1eb8156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 -- 172.21.15.92:0/75629157 >> 172.21.15.92:0/75629157 conn(0x7f1eb805e820 msgr2=0x7f1eb80622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 -- 172.21.15.92:0/75629157 shutdown_connections 2023-05-16T07:02:30.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.062+0000 7f1ebefb4700 1 -- 172.21.15.92:0/75629157 wait complete. 2023-05-16T07:02:30.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.063+0000 7f1ebefb4700 1 Processor -- start 2023-05-16T07:02:30.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.063+0000 7f1ebefb4700 1 -- start start 2023-05-16T07:02:30.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebefb4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1eb80a76f0 0x7f1eb8150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebefb4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 0x7f1eb80ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebefb4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1eb8151bd0 0x7f1eb80ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebefb4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81628e0 con 0x7f1eb81517f0 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebefb4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8161cd0 con 0x7f1eb8151bd0 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebefb4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8161fb0 con 0x7f1eb80a76f0 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 0x7f1eb80ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 0x7f1eb80ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40564/0 (socket says 172.21.15.92:40564) 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 -- 172.21.15.92:0/1545016913 learned_addr learned my addr 172.21.15.92:0/1545016913 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebe7b3700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1eb8151bd0 0x7f1eb80ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebdfb2700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1eb80a76f0 0x7f1eb8150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 -- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1eb80a76f0 msgr2=0x7f1eb8150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1eb80a76f0 0x7f1eb8150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 -- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1eb8151bd0 msgr2=0x7f1eb80ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1eb8151bd0 0x7f1eb80ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 -- 172.21.15.92:0/1545016913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009370 con 0x7f1eb81517f0 2023-05-16T07:02:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebdfb2700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1eb80a76f0 0x7f1eb8150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1ebd7b1700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 0x7f1eb80ae170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f1ea800c440 tx=0x7f1ea8007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.064+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8012400 con 0x7f1eb81517f0 2023-05-16T07:02:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.065+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8012560 con 0x7f1eb81517f0 2023-05-16T07:02:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.065+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb8162290 con 0x7f1eb81517f0 2023-05-16T07:02:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.065+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8012900 con 0x7f1eb81517f0 2023-05-16T07:02:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.065+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb8150ef0 con 0x7f1eb81517f0 2023-05-16T07:02:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.065+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb8047150 con 0x7f1eb81517f0 2023-05-16T07:02:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.066+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1ea8019030 con 0x7f1eb81517f0 2023-05-16T07:02:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.066+0000 7f1eadffb700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1e9c05edc0 0x7f1e9c061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.067+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1ea8079170 con 0x7f1eb81517f0 2023-05-16T07:02:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.067+0000 7f1ebdfb2700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1e9c05edc0 0x7f1e9c061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.067+0000 7f1ebdfb2700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1e9c05edc0 0x7f1e9c061280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f1eb0009fd0 tx=0x7f1eb000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.070+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1ea804a6b0 con 0x7f1eb81517f0 2023-05-16T07:02:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.233+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eb8155a00 con 0x7f1eb81517f0 2023-05-16T07:02:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.234+0000 7f1eadffb700 1 -- 172.21.15.92:0/1545016913 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f1ea804dcd0 con 0x7f1eb81517f0 2023-05-16T07:02:30.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:30.239 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:30.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1e9c05edc0 msgr2=0x7f1e9c061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1e9c05edc0 0x7f1e9c061280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f1eb0009fd0 tx=0x7f1eb000b040).stop 2023-05-16T07:02:30.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 msgr2=0x7f1eb80ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 0x7f1eb80ae170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f1ea800c440 tx=0x7f1ea8007db0).stop 2023-05-16T07:02:30.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 shutdown_connections 2023-05-16T07:02:30.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1e9c05edc0 0x7f1e9c061280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1eb80a76f0 0x7f1eb8150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1eb8151bd0 0x7f1eb80ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 --2- 172.21.15.92:0/1545016913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1eb81517f0 0x7f1eb80ae170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 >> 172.21.15.92:0/1545016913 conn(0x7f1eb805e820 msgr2=0x7f1eb80c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 shutdown_connections 2023-05-16T07:02:30.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.236+0000 7f1ebefb4700 1 -- 172.21.15.92:0/1545016913 wait complete. 2023-05-16T07:02:30.252 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-05-16T07:02:30.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.359+0000 7f655da57700 1 Processor -- start 2023-05-16T07:02:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.359+0000 7f655da57700 1 -- start start 2023-05-16T07:02:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f655da57700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6558154170 0x7f65580c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f655da57700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6558154550 0x7f65580c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f655da57700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65580c0660 con 0x7f65580696a0 2023-05-16T07:02:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f655da57700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65580c1b00 con 0x7f6558154550 2023-05-16T07:02:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f655da57700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65580bfd30 con 0x7f6558154170 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6558154550 0x7f65580c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6558154550 0x7f65580c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40574/0 (socket says 172.21.15.92:40574) 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 -- 172.21.15.92:0/3730963324 learned_addr learned my addr 172.21.15.92:0/3730963324 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f6556ffd700 1 --2- 172.21.15.92:0/3730963324 >> v2:172.21.15.158:3300/0 conn(0x7f6558154170 0x7f65580c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 -- 172.21.15.92:0/3730963324 >> v2:172.21.15.158:3300/0 conn(0x7f6558154170 msgr2=0x7f65580c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 --2- 172.21.15.92:0/3730963324 >> v2:172.21.15.158:3300/0 conn(0x7f6558154170 0x7f65580c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 -- 172.21.15.92:0/3730963324 >> v1:172.21.15.158:6789/0 conn(0x7f65580696a0 legacy=0x7f6558157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 -- 172.21.15.92:0/3730963324 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580c0010 con 0x7f6558154550 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f65567fc700 1 --2- 172.21.15.92:0/3730963324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154550 0x7f65580c6160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f6558083030 tx=0x7f654c0090f0).ready entity=mon.0 client_cookie=9275e2a2a90f0c3f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f6555ffb700 1 -- 172.21.15.92:0/3730963324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00b600 con 0x7f6558154550 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f6555ffb700 1 -- 172.21.15.92:0/3730963324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c00b760 con 0x7f6558154550 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.360+0000 7f6555ffb700 1 -- 172.21.15.92:0/3730963324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00bb00 con 0x7f6558154550 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 -- 172.21.15.92:0/3730963324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154550 msgr2=0x7f65580c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 --2- 172.21.15.92:0/3730963324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154550 0x7f65580c6160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f6558083030 tx=0x7f654c0090f0).stop 2023-05-16T07:02:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 -- 172.21.15.92:0/3730963324 shutdown_connections 2023-05-16T07:02:30.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 --2- 172.21.15.92:0/3730963324 >> v2:172.21.15.158:3300/0 conn(0x7f6558154170 0x7f65580c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 --2- 172.21.15.92:0/3730963324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154550 0x7f65580c6160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 -- 172.21.15.92:0/3730963324 >> 172.21.15.92:0/3730963324 conn(0x7f65580cb700 msgr2=0x7f65580c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 -- 172.21.15.92:0/3730963324 shutdown_connections 2023-05-16T07:02:30.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.361+0000 7f655da57700 1 -- 172.21.15.92:0/3730963324 wait complete. 2023-05-16T07:02:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.362+0000 7f655da57700 1 Processor -- start 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.362+0000 7f655da57700 1 -- start start 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f655da57700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65580696a0 0x7f65581642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f655da57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 0x7f655814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f655da57700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6558154550 0x7f65580c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f655da57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580c3b70 con 0x7f6558154170 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f655da57700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580c2d10 con 0x7f65580696a0 2023-05-16T07:02:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f655da57700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580c2f60 con 0x7f6558154550 2023-05-16T07:02:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 0x7f655814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 0x7f655814d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40578/0 (socket says 172.21.15.92:40578) 2023-05-16T07:02:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 -- 172.21.15.92:0/943146881 learned_addr learned my addr 172.21.15.92:0/943146881 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 -- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6558154550 msgr2=0x7f65580c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6558154550 0x7f65580c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f6556ffd700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65580696a0 0x7f65581642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 -- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65580696a0 msgr2=0x7f65581642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65580696a0 0x7f65581642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 -- 172.21.15.92:0/943146881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009370 con 0x7f6558154170 2023-05-16T07:02:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.363+0000 7f65567fc700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 0x7f655814d8e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f654c011000 tx=0x7f654c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.364+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00bec0 con 0x7f6558154170 2023-05-16T07:02:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.364+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c00b8d0 con 0x7f6558154170 2023-05-16T07:02:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.364+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65580c3240 con 0x7f6558154170 2023-05-16T07:02:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.364+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c024470 con 0x7f6558154170 2023-05-16T07:02:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.364+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580c3800 con 0x7f6558154170 2023-05-16T07:02:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.364+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65580666c0 con 0x7f6558154170 2023-05-16T07:02:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.365+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f654c00f070 con 0x7f6558154170 2023-05-16T07:02:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.365+0000 7f653effd700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f654405ed60 0x7f6544061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.365+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f654c07b700 con 0x7f6558154170 2023-05-16T07:02:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.365+0000 7f6556ffd700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f654405ed60 0x7f6544061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.366+0000 7f6556ffd700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f654405ed60 0x7f6544061220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f6550007460 tx=0x7f655000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.369+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f654c04cca0 con 0x7f6558154170 2023-05-16T07:02:30.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.533+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65580c4d00 con 0x7f6558154170 2023-05-16T07:02:30.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.534+0000 7f653effd700 1 -- 172.21.15.92:0/943146881 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f654c0502c0 con 0x7f6558154170 2023-05-16T07:02:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:30.539 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:30.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f654405ed60 msgr2=0x7f6544061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f654405ed60 0x7f6544061220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f6550007460 tx=0x7f655000b040).stop 2023-05-16T07:02:30.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 msgr2=0x7f655814d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 0x7f655814d8e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f654c011000 tx=0x7f654c017000).stop 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 shutdown_connections 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f654405ed60 0x7f6544061220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6558154550 0x7f65580c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65580696a0 0x7f65581642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 --2- 172.21.15.92:0/943146881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6558154170 0x7f655814d8e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 >> 172.21.15.92:0/943146881 conn(0x7f65580cb700 msgr2=0x7f65580ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 shutdown_connections 2023-05-16T07:02:30.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.536+0000 7f655da57700 1 -- 172.21.15.92:0/943146881 wait complete. 2023-05-16T07:02:30.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-05-16T07:02:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.660+0000 7fb9d7ddb700 1 Processor -- start 2023-05-16T07:02:30.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.660+0000 7fb9d7ddb700 1 -- start start 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d7ddb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d7ddb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9d0066280 0x7fb9d0066730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d7ddb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d00c2240 con 0x7fb9d0150ac0 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d7ddb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d00c31a0 con 0x7fb9d0066280 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d7ddb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00c4680 con 0x7fb9d0154ec0 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d6378700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9d0066280 0x7fb9d0066730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d5b77700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb9d0150ac0 0x7fb9d01547b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51332/0 (socket says 172.21.15.92:51332) 2023-05-16T07:02:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d6378700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9d0066280 0x7fb9d0066730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40588/0 (socket says 172.21.15.92:40588) 2023-05-16T07:02:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.661+0000 7fb9d5b77700 1 -- 172.21.15.92:0/1483467263 learned_addr learned my addr 172.21.15.92:0/1483467263 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d5376700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d6378700 1 -- 172.21.15.92:0/1483467263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 msgr2=0x7fb9d00c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d6378700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d6378700 1 -- 172.21.15.92:0/1483467263 >> v1:172.21.15.92:6789/0 conn(0x7fb9d0150ac0 legacy=0x7fb9d01547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d4b75700 1 -- 172.21.15.92:0/1483467263 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314224563 0 0) 0x7fb9d00c2240 con 0x7fb9d0150ac0 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d6378700 1 -- 172.21.15.92:0/1483467263 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00c24b0 con 0x7fb9d0066280 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d5376700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d6378700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d0066730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fb9c8009760 tx=0x7fb9c8009060).ready entity=mon.0 client_cookie=d49e6e5208434449 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d4b75700 1 -- 172.21.15.92:0/1483467263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800b580 con 0x7fb9d0066280 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d4b75700 1 -- 172.21.15.92:0/1483467263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c800b6e0 con 0x7fb9d0066280 2023-05-16T07:02:30.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d4b75700 1 -- 172.21.15.92:0/1483467263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800ba60 con 0x7fb9d0066280 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/1483467263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 msgr2=0x7fb9d0066730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.662+0000 7fb9d7ddb700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d0066730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fb9c8009760 tx=0x7fb9c8009060).stop 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.663+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/1483467263 shutdown_connections 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.663+0000 7fb9d7ddb700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d0066730 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.663+0000 7fb9d7ddb700 1 --2- 172.21.15.92:0/1483467263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.663+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/1483467263 >> 172.21.15.92:0/1483467263 conn(0x7fb9d00c7bc0 msgr2=0x7fb9d00c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.663+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/1483467263 shutdown_connections 2023-05-16T07:02:30.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.663+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/1483467263 wait complete. 2023-05-16T07:02:30.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 Processor -- start 2023-05-16T07:02:30.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 -- start start 2023-05-16T07:02:30.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9d0150ac0 0x7fb9d014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d015ae80 con 0x7fb9d0066280 2023-05-16T07:02:30.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d015a270 con 0x7fb9d0150ac0 2023-05-16T07:02:30.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.664+0000 7fb9d7ddb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d015a550 con 0x7fb9d0154ec0 2023-05-16T07:02:30.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d015b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40592/0 (socket says 172.21.15.92:40592) 2023-05-16T07:02:30.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 -- 172.21.15.92:0/3107134998 learned_addr learned my addr 172.21.15.92:0/3107134998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5376700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9d0150ac0 0x7fb9d014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d6378700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 -- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 msgr2=0x7fb9d00acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9d0154ec0 0x7fb9d00acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 -- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9d0150ac0 msgr2=0x7fb9d014ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9d0150ac0 0x7fb9d014ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 -- 172.21.15.92:0/3107134998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80092a0 con 0x7fb9d0066280 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d5b77700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d015b310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb9c000c960 tx=0x7fb9c0001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0005d30 con 0x7fb9d0066280 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0005e90 con 0x7fb9d0066280 2023-05-16T07:02:30.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d015a830 con 0x7fb9d0066280 2023-05-16T07:02:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0007320 con 0x7fb9d0066280 2023-05-16T07:02:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.665+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d014b2d0 con 0x7fb9d0066280 2023-05-16T07:02:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.666+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9d006c350 con 0x7fb9d0066280 2023-05-16T07:02:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.666+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb9c0010030 con 0x7fb9d0066280 2023-05-16T07:02:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9cdffb700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb9bc05ed90 0x7fb9bc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb9c0079bc0 con 0x7fb9d0066280 2023-05-16T07:02:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9cdffb700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7fb9bc068130 0x7fb9bc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 --> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9bc06ac10 con 0x7fb9bc068130 2023-05-16T07:02:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c007f220 con 0x7fb9d0066280 2023-05-16T07:02:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9d5376700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb9bc05ed90 0x7fb9bc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9d6378700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7fb9bc068130 0x7fb9bc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9d6378700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7fb9bc068130 0x7fb9bc06a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.667+0000 7fb9d5376700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb9bc05ed90 0x7fb9bc061250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb9d00a9d60 tx=0x7fb9c4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.668+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== osd.0 v2:172.21.15.92:6808/116543 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fb9bc06ac10 con 0x7fb9bc068130 2023-05-16T07:02:30.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.682+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 --> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9d015afc0 con 0x7fb9bc068130 2023-05-16T07:02:30.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.682+0000 7fb9cdffb700 1 -- 172.21.15.92:0/3107134998 <== osd.0 v2:172.21.15.92:6808/116543 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9c800b850 con 0x7fb9bc068130 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.682+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7fb9bc068130 msgr2=0x7fb9bc06a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.682+0000 7fb9d7ddb700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7fb9bc068130 0x7fb9bc06a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb9bc05ed90 msgr2=0x7fb9bc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb9bc05ed90 0x7fb9bc061250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb9d00a9d60 tx=0x7fb9c4008040).stop 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 msgr2=0x7fb9d015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 --2- 172.21.15.92:0/3107134998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d0066280 0x7fb9d015b310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb9c000c960 tx=0x7fb9c0001630).stop 2023-05-16T07:02:30.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d6378700 1 -- 172.21.15.92:0/3107134998 reap_dead start 2023-05-16T07:02:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 shutdown_connections 2023-05-16T07:02:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 >> 172.21.15.92:0/3107134998 conn(0x7fb9d00c7bc0 msgr2=0x7fb9d00c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 shutdown_connections 2023-05-16T07:02:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.683+0000 7fb9d7ddb700 1 -- 172.21.15.92:0/3107134998 wait complete. 2023-05-16T07:02:30.697 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-05-16T07:02:30.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-05-16T07:02:30.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 Processor -- start 2023-05-16T07:02:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 -- start start 2023-05-16T07:02:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d10068060 0x7f3d100bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d100c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d100bdbd0 con 0x7f3d1013a5a0 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d10154b80 con 0x7f3d10068060 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d15b83700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10154e60 con 0x7f3d100bd780 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.806+0000 7f3d0effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d10068060 0x7f3d100bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d10068060 0x7f3d100bd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40604/0 (socket says 172.21.15.92:40604) 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 -- 172.21.15.92:0/97312551 learned_addr learned my addr 172.21.15.92:0/97312551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0ffff700 1 --2- 172.21.15.92:0/97312551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d100c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 -- 172.21.15.92:0/97312551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 msgr2=0x7f3d100c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 --2- 172.21.15.92:0/97312551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d100c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 -- 172.21.15.92:0/97312551 >> v1:172.21.15.158:6789/0 conn(0x7f3d1013a5a0 legacy=0x7f3d10068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 -- 172.21.15.92:0/97312551 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10155140 con 0x7f3d10068060 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0effd700 1 --2- 172.21.15.92:0/97312551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d10068060 0x7f3d100bd290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3d04006fd0 tx=0x7f3d040090f0).ready entity=mon.0 client_cookie=8b258f22f11ca364 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/97312551 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400b600 con 0x7f3d10068060 2023-05-16T07:02:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/97312551 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0400b760 con 0x7f3d10068060 2023-05-16T07:02:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d0e7fc700 1 -- 172.21.15.92:0/97312551 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400bae0 con 0x7f3d10068060 2023-05-16T07:02:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d15b83700 1 -- 172.21.15.92:0/97312551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d10068060 msgr2=0x7f3d100bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.807+0000 7f3d15b83700 1 --2- 172.21.15.92:0/97312551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d10068060 0x7f3d100bd290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3d04006fd0 tx=0x7f3d040090f0).stop 2023-05-16T07:02:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.808+0000 7f3d15b83700 1 -- 172.21.15.92:0/97312551 shutdown_connections 2023-05-16T07:02:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.808+0000 7f3d15b83700 1 --2- 172.21.15.92:0/97312551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d100c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.808+0000 7f3d15b83700 1 --2- 172.21.15.92:0/97312551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d10068060 0x7f3d100bd290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.808+0000 7f3d15b83700 1 -- 172.21.15.92:0/97312551 >> 172.21.15.92:0/97312551 conn(0x7f3d100c4430 msgr2=0x7f3d100c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.808+0000 7f3d15b83700 1 -- 172.21.15.92:0/97312551 shutdown_connections 2023-05-16T07:02:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.808+0000 7f3d15b83700 1 -- 172.21.15.92:0/97312551 wait complete. 2023-05-16T07:02:30.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.809+0000 7f3d15b83700 1 Processor -- start 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.809+0000 7f3d15b83700 1 -- start start 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.809+0000 7f3d15b83700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d10068060 0x7f3d100c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.809+0000 7f3d15b83700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d10137590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.809+0000 7f3d15b83700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1013a5a0 0x7f3d10151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.809+0000 7f3d15b83700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100acc50 con 0x7f3d1013a5a0 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d15b83700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100ad0e0 con 0x7f3d10068060 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d15b83700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100ac040 con 0x7f3d100bd780 2023-05-16T07:02:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1013a5a0 0x7f3d10151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1013a5a0 0x7f3d10151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40610/0 (socket says 172.21.15.92:40610) 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 -- 172.21.15.92:0/4167761015 learned_addr learned my addr 172.21.15.92:0/4167761015 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d10068060 0x7f3d100c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0effd700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d10137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 -- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 msgr2=0x7f3d10137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d100bd780 0x7f3d10137590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 -- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d10068060 msgr2=0x7f3d100c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d10068060 0x7f3d100c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 -- 172.21.15.92:0/4167761015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04009370 con 0x7f3d1013a5a0 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d0ffff700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1013a5a0 0x7f3d10151650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f3d0800d890 tx=0x7f3d08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d080125f0 con 0x7f3d1013a5a0 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08012750 con 0x7f3d1013a5a0 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d100ac320 con 0x7f3d1013a5a0 2023-05-16T07:02:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08012ad0 con 0x7f3d1013a5a0 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.810+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d100ac8e0 con 0x7f3d1013a5a0 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.811+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d1006c350 con 0x7f3d1013a5a0 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.811+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f3d08013480 con 0x7f3d1013a5a0 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3cf77fe700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cfc05ed90 0x7f3cfc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3d0807ab40 con 0x7f3d1013a5a0 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3cf77fe700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f3cfc068130 0x7f3cfc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 --> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cfc06ac10 con 0x7f3cfc068130 2023-05-16T07:02:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d0807af40 con 0x7f3d1013a5a0 2023-05-16T07:02:30.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cfc05ed90 0x7f3cfc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3d0effd700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f3cfc068130 0x7f3cfc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3d0effd700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f3cfc068130 0x7f3cfc06a590 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.812+0000 7f3d0f7fe700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cfc05ed90 0x7f3cfc061250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3d00003e70 tx=0x7f3d00009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.813+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== osd.1 v2:172.21.15.92:6820/116552 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f3cfc06ac10 con 0x7f3cfc068130 2023-05-16T07:02:30.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 --> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d100c5370 con 0x7f3cfc068130 2023-05-16T07:02:30.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3cf77fe700 1 -- 172.21.15.92:0/4167761015 <== osd.1 v2:172.21.15.92:6820/116552 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d100c5370 con 0x7f3cfc068130 2023-05-16T07:02:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f3cfc068130 msgr2=0x7f3cfc06a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f3cfc068130 0x7f3cfc06a590 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cfc05ed90 msgr2=0x7f3cfc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cfc05ed90 0x7f3cfc061250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3d00003e70 tx=0x7f3d00009040).stop 2023-05-16T07:02:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1013a5a0 msgr2=0x7f3d10151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 --2- 172.21.15.92:0/4167761015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1013a5a0 0x7f3d10151650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f3d0800d890 tx=0x7f3d08008040).stop 2023-05-16T07:02:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d0ffff700 1 -- 172.21.15.92:0/4167761015 reap_dead start 2023-05-16T07:02:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 shutdown_connections 2023-05-16T07:02:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 >> 172.21.15.92:0/4167761015 conn(0x7f3d100c4430 msgr2=0x7f3d100c2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 shutdown_connections 2023-05-16T07:02:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.827+0000 7f3d15b83700 1 -- 172.21.15.92:0/4167761015 wait complete. 2023-05-16T07:02:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-05-16T07:02:30.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-05-16T07:02:30.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.949+0000 7f2f2897d700 1 Processor -- start 2023-05-16T07:02:30.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.949+0000 7f2f2897d700 1 -- start start 2023-05-16T07:02:30.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f2897d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f24069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f2897d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f24062b10 con 0x7f2f24154180 2023-05-16T07:02:30.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f2897d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f24068340 con 0x7f2f24063e30 2023-05-16T07:02:30.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f2897d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24068620 con 0x7f2f24153da0 2023-05-16T07:02:30.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f21d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2f24154180 0x7f2f24063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51364/0 (socket says 172.21.15.92:51364) 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f21d9b700 1 -- 172.21.15.92:0/2532907800 learned_addr learned my addr 172.21.15.92:0/2532907800 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.950+0000 7f2f2259c700 1 --2- 172.21.15.92:0/2532907800 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f24069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277806581 0 0) 0x7f2f24062b10 con 0x7f2f24154180 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f100033f0 con 0x7f2f24154180 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995437071 0 0) 0x7f2f24068340 con 0x7f2f24063e30 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f24062b10 con 0x7f2f24063e30 2023-05-16T07:02:30.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 907847040 0 0) 0x7f2f100033f0 con 0x7f2f24154180 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2259c700 1 -- 172.21.15.92:0/2532907800 >> v1:172.21.15.158:6789/0 conn(0x7f2f24063e30 legacy=0x7f2f240bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2259c700 1 -- 172.21.15.92:0/2532907800 >> v1:172.21.15.92:6789/0 conn(0x7f2f24154180 legacy=0x7f2f24063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2259c700 1 -- 172.21.15.92:0/2532907800 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f240624c0 con 0x7f2f24153da0 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2259c700 1 --2- 172.21.15.92:0/2532907800 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f24069a10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f2f14009e90 tx=0x7f2f14009780).ready entity=mon.2 client_cookie=7413d41a33469e79 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400e070 con 0x7f2f24153da0 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1400b7a0 con 0x7f2f24153da0 2023-05-16T07:02:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.951+0000 7f2f2159a700 1 -- 172.21.15.92:0/2532907800 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c001470 con 0x7f2f24153da0 2023-05-16T07:02:30.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 -- 172.21.15.92:0/2532907800 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 msgr2=0x7f2f24069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 --2- 172.21.15.92:0/2532907800 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f24069a10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f2f14009e90 tx=0x7f2f14009780).stop 2023-05-16T07:02:30.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 -- 172.21.15.92:0/2532907800 shutdown_connections 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 --2- 172.21.15.92:0/2532907800 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f24069a10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 -- 172.21.15.92:0/2532907800 >> 172.21.15.92:0/2532907800 conn(0x7f2f240c8aa0 msgr2=0x7f2f240c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 -- 172.21.15.92:0/2532907800 shutdown_connections 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.952+0000 7f2f2897d700 1 -- 172.21.15.92:0/2532907800 wait complete. 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.953+0000 7f2f2897d700 1 Processor -- start 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.953+0000 7f2f2897d700 1 -- start start 2023-05-16T07:02:30.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.953+0000 7f2f2897d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f24063e30 0x7f2f24145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.953+0000 7f2f2897d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f241677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2897d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f24154180 0x7f2f24167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2897d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241681a0 con 0x7f2f24154180 2023-05-16T07:02:30.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2897d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240c6b50 con 0x7f2f24063e30 2023-05-16T07:02:30.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2897d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240a5090 con 0x7f2f24153da0 2023-05-16T07:02:30.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f24154180 0x7f2f24167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f24154180 0x7f2f24167cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40626/0 (socket says 172.21.15.92:40626) 2023-05-16T07:02:30.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 -- 172.21.15.92:0/2094296822 learned_addr learned my addr 172.21.15.92:0/2094296822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:30.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f21d9b700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f241677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2259c700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f24063e30 0x7f2f24145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 -- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 msgr2=0x7f2f241677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f24153da0 0x7f2f241677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 -- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f24063e30 msgr2=0x7f2f24145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f24063e30 0x7f2f24145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 -- 172.21.15.92:0/2094296822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009a20 con 0x7f2f24154180 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2259c700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f24063e30 0x7f2f24145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:30.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f22d9d700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f24154180 0x7f2f24167cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2f1c00c420 tx=0x7f2f1c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00f470 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00f5d0 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00f950 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f240c6540 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.954+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241683a0 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.955+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f2406c350 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.955+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2f1c023070 con 0x7f2f24154180 2023-05-16T07:02:30.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f1a7fc700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f0405ed90 0x7f2f04061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2f1c079a00 con 0x7f2f24154180 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f1a7fc700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7f2f04068130 0x7f2f0406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 --> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f0406ac10 con 0x7f2f04068130 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f1c079e00 con 0x7f2f24154180 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f2259c700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f0405ed90 0x7f2f04061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f21d9b700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7f2f04068130 0x7f2f0406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:30.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f21d9b700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7f2f04068130 0x7f2f0406a590 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.956+0000 7f2f2259c700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f0405ed90 0x7f2f04061250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2f14009de0 tx=0x7f2f14009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:30.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.957+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== osd.2 v2:172.21.15.92:6816/116553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f2f0406ac10 con 0x7f2f04068130 2023-05-16T07:02:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.971+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 --> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f240c6ce0 con 0x7f2f04068130 2023-05-16T07:02:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.971+0000 7f2f1a7fc700 1 -- 172.21.15.92:0/2094296822 <== osd.2 v2:172.21.15.92:6816/116553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f240c6ce0 con 0x7f2f04068130 2023-05-16T07:02:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.971+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7f2f04068130 msgr2=0x7f2f0406a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.971+0000 7f2f2897d700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7f2f04068130 0x7f2f0406a590 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f0405ed90 msgr2=0x7f2f04061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f0405ed90 0x7f2f04061250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2f14009de0 tx=0x7f2f14009780).stop 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f24154180 msgr2=0x7f2f24167cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 --2- 172.21.15.92:0/2094296822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f24154180 0x7f2f24167cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2f1c00c420 tx=0x7f2f1c007bd0).stop 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f22d9d700 1 -- 172.21.15.92:0/2094296822 reap_dead start 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 shutdown_connections 2023-05-16T07:02:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 >> 172.21.15.92:0/2094296822 conn(0x7f2f240c8aa0 msgr2=0x7f2f240ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 shutdown_connections 2023-05-16T07:02:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:30.972+0000 7f2f2897d700 1 -- 172.21.15.92:0/2094296822 wait complete. 2023-05-16T07:02:30.986 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-05-16T07:02:30.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-05-16T07:02:31.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.094+0000 7f9390d4a700 1 Processor -- start 2023-05-16T07:02:31.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.094+0000 7f9390d4a700 1 -- start start 2023-05-16T07:02:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9390d4a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c13a560 0x7f938c065c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9390d4a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f938c13a940 0x7f938c062cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9390d4a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f938c065000 0x7f938c0654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9390d4a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c0685c0 con 0x7f938c13a940 2023-05-16T07:02:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9390d4a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c0676f0 con 0x7f938c065000 2023-05-16T07:02:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9390d4a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0c2030 con 0x7f938c13a560 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9389d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f938c13a940 0x7f938c062cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9389d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f938c13a940 0x7f938c062cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40634/0 (socket says 172.21.15.92:40634) 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.095+0000 7f9389d9b700 1 -- 172.21.15.92:0/980995870 learned_addr learned my addr 172.21.15.92:0/980995870 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f938a59c700 1 --2- 172.21.15.92:0/980995870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c13a560 0x7f938c065c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9389d9b700 1 -- 172.21.15.92:0/980995870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c13a560 msgr2=0x7f938c065c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f938ad9d700 1 --2- 172.21.15.92:0/980995870 >> v2:172.21.15.158:3300/0 conn(0x7f938c065000 0x7f938c0654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9389d9b700 1 --2- 172.21.15.92:0/980995870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c13a560 0x7f938c065c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9389d9b700 1 -- 172.21.15.92:0/980995870 >> v2:172.21.15.158:3300/0 conn(0x7f938c065000 msgr2=0x7f938c0654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9389d9b700 1 --2- 172.21.15.92:0/980995870 >> v2:172.21.15.158:3300/0 conn(0x7f938c065000 0x7f938c0654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9389d9b700 1 -- 172.21.15.92:0/980995870 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0659a0 con 0x7f938c13a940 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9389d9b700 1 --2- 172.21.15.92:0/980995870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c062cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f938c0a73f0 tx=0x7f93740090f0).ready entity=mon.0 client_cookie=205cd05bd973b171 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f938959a700 1 -- 172.21.15.92:0/980995870 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400b600 con 0x7f938c13a940 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f938959a700 1 -- 172.21.15.92:0/980995870 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937400b760 con 0x7f938c13a940 2023-05-16T07:02:31.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f938959a700 1 -- 172.21.15.92:0/980995870 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400bae0 con 0x7f938c13a940 2023-05-16T07:02:31.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 -- 172.21.15.92:0/980995870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 msgr2=0x7f938c062cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 --2- 172.21.15.92:0/980995870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c062cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f938c0a73f0 tx=0x7f93740090f0).stop 2023-05-16T07:02:31.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 -- 172.21.15.92:0/980995870 shutdown_connections 2023-05-16T07:02:31.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 --2- 172.21.15.92:0/980995870 >> v2:172.21.15.158:3300/0 conn(0x7f938c065000 0x7f938c0654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 --2- 172.21.15.92:0/980995870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c13a560 0x7f938c065c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 --2- 172.21.15.92:0/980995870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c062cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.096+0000 7f9390d4a700 1 -- 172.21.15.92:0/980995870 >> 172.21.15.92:0/980995870 conn(0x7f938c0c4430 msgr2=0x7f938c0c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.097+0000 7f9390d4a700 1 -- 172.21.15.92:0/980995870 shutdown_connections 2023-05-16T07:02:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.097+0000 7f9390d4a700 1 -- 172.21.15.92:0/980995870 wait complete. 2023-05-16T07:02:31.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 Processor -- start 2023-05-16T07:02:31.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 -- start start 2023-05-16T07:02:31.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c065000 0x7f938c164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f938c13a560 0x7f938c13d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c0a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c13b300 con 0x7f938c13a940 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1684b0 con 0x7f938c13a560 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f9390d4a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c164770 con 0x7f938c065000 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f938ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c0a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f938ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c0a6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40642/0 (socket says 172.21.15.92:40642) 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.098+0000 7f938ad9d700 1 -- 172.21.15.92:0/1385814843 learned_addr learned my addr 172.21.15.92:0/1385814843 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938a59c700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c065000 0x7f938c164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f9389d9b700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f938c13a560 0x7f938c13d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938ad9d700 1 -- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c065000 msgr2=0x7f938c164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938ad9d700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c065000 0x7f938c164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938ad9d700 1 -- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f938c13a560 msgr2=0x7f938c13d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938ad9d700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f938c13a560 0x7f938c13d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938ad9d700 1 -- 172.21.15.92:0/1385814843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9374009370 con 0x7f938c13a940 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f9389d9b700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f938c13a560 0x7f938c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938a59c700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f938c065000 0x7f938c164c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f938ad9d700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c0a6340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f938400c550 tx=0x7f9384008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384010580 con 0x7f938c13a940 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93840106e0 con 0x7f938c13a940 2023-05-16T07:02:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c13d9b0 con 0x7f938c13a940 2023-05-16T07:02:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384010a60 con 0x7f938c13a940 2023-05-16T07:02:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.099+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c164120 con 0x7f938c13a940 2023-05-16T07:02:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.100+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f938c163f50 con 0x7f938c13a940 2023-05-16T07:02:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.100+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f938401b460 con 0x7f938c13a940 2023-05-16T07:02:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f93827fc700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f936c05ed90 0x7f936c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9384079c90 con 0x7f938c13a940 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f93827fc700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f936c068130 0x7f936c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 --> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f936c06ac10 con 0x7f936c068130 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938407c1f0 con 0x7f938c13a940 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f938a59c700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f936c05ed90 0x7f936c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f9389d9b700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f936c068130 0x7f936c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f9389d9b700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f936c068130 0x7f936c06a590 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.101+0000 7f938a59c700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f936c05ed90 0x7f936c061250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f937c00d460 tx=0x7f937c0075e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.102+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== osd.3 v2:172.21.15.92:6800/116542 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f936c06ac10 con 0x7f936c068130 2023-05-16T07:02:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.115+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 --> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f938c06c350 con 0x7f936c068130 2023-05-16T07:02:31.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f93827fc700 1 -- 172.21.15.92:0/1385814843 <== osd.3 v2:172.21.15.92:6800/116542 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f938c06c350 con 0x7f936c068130 2023-05-16T07:02:31.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f936c068130 msgr2=0x7f936c06a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f936c068130 0x7f936c06a590 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f936c05ed90 msgr2=0x7f936c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f936c05ed90 0x7f936c061250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f937c00d460 tx=0x7f937c0075e0).stop 2023-05-16T07:02:31.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 msgr2=0x7f938c0a6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 --2- 172.21.15.92:0/1385814843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938c13a940 0x7f938c0a6340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f938400c550 tx=0x7f9384008040).stop 2023-05-16T07:02:31.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f938ad9d700 1 -- 172.21.15.92:0/1385814843 reap_dead start 2023-05-16T07:02:31.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 shutdown_connections 2023-05-16T07:02:31.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 >> 172.21.15.92:0/1385814843 conn(0x7f938c0c4430 msgr2=0x7f938c0c25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 shutdown_connections 2023-05-16T07:02:31.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.116+0000 7f9390d4a700 1 -- 172.21.15.92:0/1385814843 wait complete. 2023-05-16T07:02:31.131 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:31.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-05-16T07:02:31.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.239+0000 7f09c83ac700 1 Processor -- start 2023-05-16T07:02:31.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.239+0000 7f09c83ac700 1 -- start start 2023-05-16T07:02:31.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c83ac700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f09c0154170 0x7f09c00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c83ac700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 0x7f09c00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c83ac700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00c0660 con 0x7f09c00696a0 2023-05-16T07:02:31.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c83ac700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c00c1b00 con 0x7f09c0154170 2023-05-16T07:02:31.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c83ac700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00bfd30 con 0x7f09c0154550 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c5947700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 0x7f09c00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c6148700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f09c0154170 0x7f09c00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c5947700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 0x7f09c00c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:33224/0 (socket says 172.21.15.92:33224) 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c6148700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f09c0154170 0x7f09c00c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44284/0 (socket says 172.21.15.92:44284) 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.240+0000 7f09c5947700 1 -- 172.21.15.92:0/2101081364 learned_addr learned my addr 172.21.15.92:0/2101081364 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5146700 1 -- 172.21.15.92:0/2101081364 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359770547 0 0) 0x7f09c00c0660 con 0x7f09c00696a0 2023-05-16T07:02:31.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5947700 1 -- 172.21.15.92:0/2101081364 >> v2:172.21.15.158:3300/0 conn(0x7f09c0154170 msgr2=0x7f09c00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5947700 1 --2- 172.21.15.92:0/2101081364 >> v2:172.21.15.158:3300/0 conn(0x7f09c0154170 0x7f09c00c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5947700 1 -- 172.21.15.92:0/2101081364 >> v1:172.21.15.158:6789/0 conn(0x7f09c00696a0 legacy=0x7f09c0157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5947700 1 -- 172.21.15.92:0/2101081364 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00c0010 con 0x7f09c0154550 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5947700 1 --2- 172.21.15.92:0/2101081364 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 0x7f09c00c6160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f09b4006fd0 tx=0x7f09b40090f0).ready entity=mon.2 client_cookie=f5e599f9922cfcfc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5146700 1 -- 172.21.15.92:0/2101081364 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b400b580 con 0x7f09c0154550 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5146700 1 -- 172.21.15.92:0/2101081364 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b400b6e0 con 0x7f09c0154550 2023-05-16T07:02:31.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c5146700 1 -- 172.21.15.92:0/2101081364 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b400ba80 con 0x7f09c0154550 2023-05-16T07:02:31.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c83ac700 1 -- 172.21.15.92:0/2101081364 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 msgr2=0x7f09c00c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.241+0000 7f09c83ac700 1 --2- 172.21.15.92:0/2101081364 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 0x7f09c00c6160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f09b4006fd0 tx=0x7f09b40090f0).stop 2023-05-16T07:02:31.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.242+0000 7f09c83ac700 1 -- 172.21.15.92:0/2101081364 shutdown_connections 2023-05-16T07:02:31.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.242+0000 7f09c83ac700 1 --2- 172.21.15.92:0/2101081364 >> v2:172.21.15.158:3300/0 conn(0x7f09c0154170 0x7f09c00c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.242+0000 7f09c83ac700 1 --2- 172.21.15.92:0/2101081364 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c0154550 0x7f09c00c6160 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.242+0000 7f09c83ac700 1 -- 172.21.15.92:0/2101081364 >> 172.21.15.92:0/2101081364 conn(0x7f09c00cb700 msgr2=0x7f09c00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.242+0000 7f09c83ac700 1 -- 172.21.15.92:0/2101081364 shutdown_connections 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.242+0000 7f09c83ac700 1 -- 172.21.15.92:0/2101081364 wait complete. 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 Processor -- start 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 -- start start 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c00696a0 0x7f09c00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c0154170 0x7f09c0168360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c0154550 0x7f09c014bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0168850 con 0x7f09c0154550 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c014c4e0 con 0x7f09c0154170 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.243+0000 7f09c83ac700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00afc40 con 0x7f09c00696a0 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c0154550 0x7f09c014bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c0154550 0x7f09c014bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40658/0 (socket says 172.21.15.92:40658) 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c5947700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c0154170 0x7f09c0168360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 -- 172.21.15.92:0/3632557379 learned_addr learned my addr 172.21.15.92:0/3632557379 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6148700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c00696a0 0x7f09c00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 -- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c00696a0 msgr2=0x7f09c00b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09c00696a0 0x7f09c00b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 -- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c0154170 msgr2=0x7f09c0168360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09c0154170 0x7f09c0168360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 -- 172.21.15.92:0/3632557379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4009370 con 0x7f09c0154550 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c6949700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c0154550 0x7f09c014bff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f09b800c490 tx=0x7f09b8007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8012400 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8012560 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c0158440 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8012900 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.244+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c00b01c0 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.245+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09c0158bb0 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f09b8019030 con 0x7f09c0154550 2023-05-16T07:02:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09be7fc700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f09ac05ed90 0x7f09ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f09b8079930 con 0x7f09c0154550 2023-05-16T07:02:31.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09be7fc700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f09ac068130 0x7f09ac06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 --> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09ac06ac10 con 0x7f09ac068130 2023-05-16T07:02:31.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b8079d30 con 0x7f09c0154550 2023-05-16T07:02:31.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09c6148700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f09ac05ed90 0x7f09ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09c5947700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f09ac068130 0x7f09ac06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.246+0000 7f09c6148700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f09ac05ed90 0x7f09ac061250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f09b000f570 tx=0x7f09b000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.247+0000 7f09c5947700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f09ac068130 0x7f09ac06a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.247+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== osd.4 v2:172.21.15.158:6800/86891 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f09ac06ac10 con 0x7f09ac068130 2023-05-16T07:02:31.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 --> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09c00666c0 con 0x7f09ac068130 2023-05-16T07:02:31.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09be7fc700 1 -- 172.21.15.92:0/3632557379 <== osd.4 v2:172.21.15.158:6800/86891 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09c00666c0 con 0x7f09ac068130 2023-05-16T07:02:31.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f09ac068130 msgr2=0x7f09ac06a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f09ac068130 0x7f09ac06a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f09ac05ed90 msgr2=0x7f09ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f09ac05ed90 0x7f09ac061250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f09b000f570 tx=0x7f09b000b040).stop 2023-05-16T07:02:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c0154550 msgr2=0x7f09c014bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 --2- 172.21.15.92:0/3632557379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c0154550 0x7f09c014bff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f09b800c490 tx=0x7f09b8007bf0).stop 2023-05-16T07:02:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c6949700 1 -- 172.21.15.92:0/3632557379 reap_dead start 2023-05-16T07:02:31.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 shutdown_connections 2023-05-16T07:02:31.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.262+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 >> 172.21.15.92:0/3632557379 conn(0x7f09c00cb700 msgr2=0x7f09c014d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.263+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 shutdown_connections 2023-05-16T07:02:31.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.263+0000 7f09c83ac700 1 -- 172.21.15.92:0/3632557379 wait complete. 2023-05-16T07:02:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:31.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-05-16T07:02:31.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.385+0000 7ff18371c700 1 Processor -- start 2023-05-16T07:02:31.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.385+0000 7ff18371c700 1 -- start start 2023-05-16T07:02:31.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff18371c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff17c154550 0x7ff17c0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff18371c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0c0660 con 0x7ff17c069b90 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff18371c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c069fc0 con 0x7ff17c154170 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff18371c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0bfd30 con 0x7ff17c154550 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff180cb7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff17c154550 0x7ff17c0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff181cb9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff17c069b90 0x7ff17c0c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51416/0 (socket says 172.21.15.92:51416) 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff181cb9700 1 -- 172.21.15.92:0/4175462557 learned_addr learned my addr 172.21.15.92:0/4175462557 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff180cb7700 1 -- 172.21.15.92:0/4175462557 >> v1:172.21.15.158:6789/0 conn(0x7ff17c154170 legacy=0x7ff17c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff180cb7700 1 -- 172.21.15.92:0/4175462557 >> v1:172.21.15.92:6789/0 conn(0x7ff17c069b90 legacy=0x7ff17c0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff17bfff700 1 -- 172.21.15.92:0/4175462557 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066709690 0 0) 0x7ff17c0c0660 con 0x7ff17c069b90 2023-05-16T07:02:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.386+0000 7ff180cb7700 1 -- 172.21.15.92:0/4175462557 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0c0010 con 0x7ff17c154550 2023-05-16T07:02:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff180cb7700 1 --2- 172.21.15.92:0/4175462557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c0696a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff170009760 tx=0x7ff170009060).ready entity=mon.0 client_cookie=90e828352fb11e80 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff17bfff700 1 -- 172.21.15.92:0/4175462557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1700090d0 con 0x7ff17c154550 2023-05-16T07:02:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff17bfff700 1 -- 172.21.15.92:0/4175462557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17000b520 con 0x7ff17c154550 2023-05-16T07:02:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff17bfff700 1 -- 172.21.15.92:0/4175462557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000b8a0 con 0x7ff17c154550 2023-05-16T07:02:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 -- 172.21.15.92:0/4175462557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 msgr2=0x7ff17c0696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 --2- 172.21.15.92:0/4175462557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c0696a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff170009760 tx=0x7ff170009060).stop 2023-05-16T07:02:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 -- 172.21.15.92:0/4175462557 shutdown_connections 2023-05-16T07:02:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 --2- 172.21.15.92:0/4175462557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c0696a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 -- 172.21.15.92:0/4175462557 >> 172.21.15.92:0/4175462557 conn(0x7ff17c0cb700 msgr2=0x7ff17c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 -- 172.21.15.92:0/4175462557 shutdown_connections 2023-05-16T07:02:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.387+0000 7ff18371c700 1 -- 172.21.15.92:0/4175462557 wait complete. 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.388+0000 7ff18371c700 1 Processor -- start 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.388+0000 7ff18371c700 1 -- start start 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff18371c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff17c069b90 0x7ff17c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff18371c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff17c154170 0x7ff17c14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff18371c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff18371c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c157600 con 0x7ff17c154550 2023-05-16T07:02:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff18371c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1569f0 con 0x7ff17c154170 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff18371c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c156cd0 con 0x7ff17c069b90 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff181cb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff181cb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c068090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40672/0 (socket says 172.21.15.92:40672) 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.389+0000 7ff181cb9700 1 -- 172.21.15.92:0/3067608080 learned_addr learned my addr 172.21.15.92:0/3067608080 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff180cb7700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff17c154170 0x7ff17c14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff1814b8700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff17c069b90 0x7ff17c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff181cb9700 1 -- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff17c069b90 msgr2=0x7ff17c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff181cb9700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff17c069b90 0x7ff17c155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff181cb9700 1 -- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff17c154170 msgr2=0x7ff17c14ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff181cb9700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff17c154170 0x7ff17c14ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff181cb9700 1 -- 172.21.15.92:0/3067608080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1700092a0 con 0x7ff17c154550 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff180cb7700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff17c154170 0x7ff17c14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff1814b8700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff17c069b90 0x7ff17c155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff181cb9700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c068090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7ff17400c490 tx=0x7ff174007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1740072e0 con 0x7ff17c154550 2023-05-16T07:02:31.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174007440 con 0x7ff17c154550 2023-05-16T07:02:31.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c156fb0 con 0x7ff17c154550 2023-05-16T07:02:31.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1740057a0 con 0x7ff17c154550 2023-05-16T07:02:31.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.390+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c155d20 con 0x7ff17c154550 2023-05-16T07:02:31.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.391+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff17c0666c0 con 0x7ff17c154550 2023-05-16T07:02:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.391+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff174010070 con 0x7ff17c154550 2023-05-16T07:02:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1797fa700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff16805ed90 0x7ff168061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff1740791e0 con 0x7ff17c154550 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1797fa700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7ff168068130 0x7ff16806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 --> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff16806ac10 con 0x7ff168068130 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff17407b850 con 0x7ff17c154550 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1814b8700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff16805ed90 0x7ff168061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff180cb7700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7ff168068130 0x7ff16806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff1814b8700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff16805ed90 0x7ff168061250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff16c007fd0 tx=0x7ff16c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.392+0000 7ff180cb7700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7ff168068130 0x7ff16806a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.393+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== osd.5 v2:172.21.15.158:6816/86917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7ff16806ac10 con 0x7ff168068130 2023-05-16T07:02:31.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.407+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 --> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff17c06c350 con 0x7ff168068130 2023-05-16T07:02:31.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff1797fa700 1 -- 172.21.15.92:0/3067608080 <== osd.5 v2:172.21.15.158:6816/86917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff17c06c350 con 0x7ff168068130 2023-05-16T07:02:31.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7ff168068130 msgr2=0x7ff16806a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7ff168068130 0x7ff16806a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff16805ed90 msgr2=0x7ff168061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff16805ed90 0x7ff168061250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff16c007fd0 tx=0x7ff16c009040).stop 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 msgr2=0x7ff17c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 --2- 172.21.15.92:0/3067608080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff17c154550 0x7ff17c068090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7ff17400c490 tx=0x7ff174007c10).stop 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff181cb9700 1 -- 172.21.15.92:0/3067608080 reap_dead start 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 shutdown_connections 2023-05-16T07:02:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 >> 172.21.15.92:0/3067608080 conn(0x7ff17c0cb700 msgr2=0x7ff17c14db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 shutdown_connections 2023-05-16T07:02:31.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.408+0000 7ff18371c700 1 -- 172.21.15.92:0/3067608080 wait complete. 2023-05-16T07:02:31.423 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:31.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-05-16T07:02:31.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4b1ef2700 1 Processor -- start 2023-05-16T07:02:31.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4b1ef2700 1 -- start start 2023-05-16T07:02:31.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4b1ef2700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd4ac158160 0x7fd4ac15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4b1ef2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0683d0 con 0x7fd4ac14e5e0 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4b1ef2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0be060 con 0x7fd4ac14e200 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4b1ef2700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac137470 con 0x7fd4ac158160 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.532+0000 7fd4aaffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd4ac14e5e0 0x7fd4ac157a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51430/0 (socket says 172.21.15.92:51430) 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 -- 172.21.15.92:0/825357241 learned_addr learned my addr 172.21.15.92:0/825357241 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4abfff700 1 --2- 172.21.15.92:0/825357241 >> v2:172.21.15.158:3300/0 conn(0x7fd4ac158160 0x7fd4ac15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417752184 0 0) 0x7fd4ac0683d0 con 0x7fd4ac14e5e0 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4940033f0 con 0x7fd4ac14e5e0 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3902638858 0 0) 0x7fd4940033f0 con 0x7fd4ac14e5e0 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 >> v2:172.21.15.158:3300/0 conn(0x7fd4ac158160 msgr2=0x7fd4ac15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 --2- 172.21.15.92:0/825357241 >> v2:172.21.15.158:3300/0 conn(0x7fd4ac158160 0x7fd4ac15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 >> v1:172.21.15.158:6789/0 conn(0x7fd4ac14e200 legacy=0x7fd4ac066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac067560 con 0x7fd4ac14e5e0 2023-05-16T07:02:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd4a0001470 con 0x7fd4ac14e5e0 2023-05-16T07:02:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 >> v1:172.21.15.92:6789/0 conn(0x7fd4ac14e5e0 legacy=0x7fd4ac157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4940038e0 0x7fd494003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd494004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd494005000 0x7fd49400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac137470 con 0x7fd494004690 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49400bf60 con 0x7fd494005000 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49400c0f0 con 0x7fd4940038e0 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd494004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4ab7fe700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4940038e0 0x7fd494003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4abfff700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd494005000 0x7fd49400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 -- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4940038e0 msgr2=0x7fd494003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4940038e0 0x7fd494003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 -- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd494005000 msgr2=0x7fd49400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd494005000 0x7fd49400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4aaffd700 1 -- 172.21.15.92:0/825357241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940033f0 con 0x7fd494004690 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4ab7fe700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4940038e0 0x7fd494003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4abfff700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd494005000 0x7fd49400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.533+0000 7fd4abfff700 1 -- 172.21.15.92:0/825357241 reap_dead start 2023-05-16T07:02:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd494004b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd4a000c160 tx=0x7fd4ac158a30).ready entity=mon.0 client_cookie=9bf4b60fcde93c24 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000ca90 con 0x7fd494004690 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a000cbf0 con 0x7fd494004690 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4aa7fc700 1 -- 172.21.15.92:0/825357241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00070d0 con 0x7fd494004690 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/825357241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 msgr2=0x7fd494004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd494004b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd4a000c160 tx=0x7fd4ac158a30).stop 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/825357241 shutdown_connections 2023-05-16T07:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 --2- 172.21.15.92:0/825357241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd494004b10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/825357241 >> 172.21.15.92:0/825357241 conn(0x7fd4ac0c7bc0 msgr2=0x7fd4ac0c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/825357241 shutdown_connections 2023-05-16T07:02:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.534+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/825357241 wait complete. 2023-05-16T07:02:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.535+0000 7fd4b1ef2700 1 Processor -- start 2023-05-16T07:02:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.535+0000 7fd4b1ef2700 1 -- start start 2023-05-16T07:02:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4b1ef2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd4ac138c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4b1ef2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4ac1375b0 0x7fd4ac137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4b1ef2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4ac164c90 0x7fd4ac165110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4b1ef2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac165a30 con 0x7fd494004690 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4b1ef2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac137f20 con 0x7fd4ac164c90 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4b1ef2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac165600 con 0x7fd4ac1375b0 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd4ac138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd4ac138c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40692/0 (socket says 172.21.15.92:40692) 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 -- 172.21.15.92:0/3078061714 learned_addr learned my addr 172.21.15.92:0/3078061714 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4ac1375b0 0x7fd4ac137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4abfff700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4ac164c90 0x7fd4ac165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 -- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4ac1375b0 msgr2=0x7fd4ac137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4ac1375b0 0x7fd4ac137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 -- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4ac164c90 msgr2=0x7fd4ac165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4ac164c90 0x7fd4ac165110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.536+0000 7fd4ab7fe700 1 -- 172.21.15.92:0/3078061714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac1394c0 con 0x7fd494004690 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4ab7fe700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd4ac138c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fd49c00e850 tx=0x7fd49c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00c600 con 0x7fd494004690 2023-05-16T07:02:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c00c760 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac1397a0 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00cae0 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac139160 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.537+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4ac139cc0 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd49c0133e0 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4937fe700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd49805ed90 0x7fd498061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd49c0797d0 con 0x7fd494004690 2023-05-16T07:02:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4937fe700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fd498068130 0x7fd49806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 --> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd49806ac10 con 0x7fd498068130 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd49805ed90 0x7fd498061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.538+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49c07e220 con 0x7fd494004690 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.539+0000 7fd4abfff700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fd498068130 0x7fd49806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.539+0000 7fd4aaffd700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd49805ed90 0x7fd498061250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd4ac155320 tx=0x7fd4a0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.539+0000 7fd4abfff700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fd498068130 0x7fd49806a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.540+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== osd.6 v2:172.21.15.158:6824/86922 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fd49806ac10 con 0x7fd498068130 2023-05-16T07:02:31.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.554+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 --> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd4ac165b70 con 0x7fd498068130 2023-05-16T07:02:31.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.554+0000 7fd4937fe700 1 -- 172.21.15.92:0/3078061714 <== osd.6 v2:172.21.15.158:6824/86922 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd4a4009750 con 0x7fd498068130 2023-05-16T07:02:31.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fd498068130 msgr2=0x7fd49806a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fd498068130 0x7fd49806a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd49805ed90 msgr2=0x7fd498061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd49805ed90 0x7fd498061250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd4ac155320 tx=0x7fd4a0015000).stop 2023-05-16T07:02:31.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 msgr2=0x7fd4ac138c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 --2- 172.21.15.92:0/3078061714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd494004690 0x7fd4ac138c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fd49c00e850 tx=0x7fd49c009040).stop 2023-05-16T07:02:31.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4abfff700 1 -- 172.21.15.92:0/3078061714 reap_dead start 2023-05-16T07:02:31.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 shutdown_connections 2023-05-16T07:02:31.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 >> 172.21.15.92:0/3078061714 conn(0x7fd4ac0c7bc0 msgr2=0x7fd4ac0a4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 shutdown_connections 2023-05-16T07:02:31.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.555+0000 7fd4b1ef2700 1 -- 172.21.15.92:0/3078061714 wait complete. 2023-05-16T07:02:31.570 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:31.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-05-16T07:02:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.679+0000 7fca4eb28700 1 Processor -- start 2023-05-16T07:02:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.679+0000 7fca4eb28700 1 -- start start 2023-05-16T07:02:31.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4eb28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca48154180 0x7fca48063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4eb28700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca48063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4eb28700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca48062b10 con 0x7fca48153da0 2023-05-16T07:02:31.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4eb28700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca48064100 con 0x7fca48154180 2023-05-16T07:02:31.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4eb28700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca48068620 con 0x7fca48062fa0 2023-05-16T07:02:31.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4c8c4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fca48153da0 0x7fca48069a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51450/0 (socket says 172.21.15.92:51450) 2023-05-16T07:02:31.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4c8c4700 1 -- 172.21.15.92:0/1210074898 learned_addr learned my addr 172.21.15.92:0/1210074898 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:31.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca48154180 0x7fca48063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca4d0c5700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca48063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca477fe700 1 -- 172.21.15.92:0/1210074898 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642245166 0 0) 0x7fca48062b10 con 0x7fca48153da0 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.680+0000 7fca477fe700 1 -- 172.21.15.92:0/1210074898 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca300033f0 con 0x7fca48153da0 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca47fff700 1 -- 172.21.15.92:0/1210074898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 msgr2=0x7fca48063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca47fff700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca48063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca47fff700 1 -- 172.21.15.92:0/1210074898 >> v1:172.21.15.92:6789/0 conn(0x7fca48153da0 legacy=0x7fca48069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca47fff700 1 -- 172.21.15.92:0/1210074898 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48068340 con 0x7fca48154180 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4d0c5700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca48063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca47fff700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48154180 0x7fca48063720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fca3c009760 tx=0x7fca3c009060).ready entity=mon.0 client_cookie=cbee21dc4e2cbf43 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca477fe700 1 -- 172.21.15.92:0/1210074898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00f510 con 0x7fca48154180 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca477fe700 1 -- 172.21.15.92:0/1210074898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00f670 con 0x7fca48154180 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca477fe700 1 -- 172.21.15.92:0/1210074898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00f9f0 con 0x7fca48154180 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 -- 172.21.15.92:0/1210074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48154180 msgr2=0x7fca48063720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48154180 0x7fca48063720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fca3c009760 tx=0x7fca3c009060).stop 2023-05-16T07:02:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 -- 172.21.15.92:0/1210074898 shutdown_connections 2023-05-16T07:02:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca48063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 --2- 172.21.15.92:0/1210074898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48154180 0x7fca48063720 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 -- 172.21.15.92:0/1210074898 >> 172.21.15.92:0/1210074898 conn(0x7fca480c8aa0 msgr2=0x7fca480c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 -- 172.21.15.92:0/1210074898 shutdown_connections 2023-05-16T07:02:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.681+0000 7fca4eb28700 1 -- 172.21.15.92:0/1210074898 wait complete. 2023-05-16T07:02:31.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.682+0000 7fca4eb28700 1 Processor -- start 2023-05-16T07:02:31.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 -- start start 2023-05-16T07:02:31.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca480c0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48153da0 0x7fca4814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca48154180 0x7fca480aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c0c00 con 0x7fca48153da0 2023-05-16T07:02:31.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c2c80 con 0x7fca48154180 2023-05-16T07:02:31.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca4eb28700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c20b0 con 0x7fca48062fa0 2023-05-16T07:02:31.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48153da0 0x7fca4814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48153da0 0x7fca4814f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40712/0 (socket says 172.21.15.92:40712) 2023-05-16T07:02:31.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.683+0000 7fca47fff700 1 -- 172.21.15.92:0/4115858583 learned_addr learned my addr 172.21.15.92:0/4115858583 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca4c8c4700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca480c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca4d0c5700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca48154180 0x7fca480aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca47fff700 1 -- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 msgr2=0x7fca480c0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca47fff700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca480c0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca47fff700 1 -- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca48154180 msgr2=0x7fca480aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca47fff700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca48154180 0x7fca480aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca47fff700 1 -- 172.21.15.92:0/4115858583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0092a0 con 0x7fca48153da0 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca4d0c5700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca48154180 0x7fca480aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:31.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca4c8c4700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca48062fa0 0x7fca480c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca47fff700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48153da0 0x7fca4814f670 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fca3c009e00 tx=0x7fca3c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00f9f0 con 0x7fca48153da0 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c000c90 con 0x7fca48153da0 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca480c2350 con 0x7fca48153da0 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c01b690 con 0x7fca48153da0 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.684+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca480c2910 con 0x7fca48153da0 2023-05-16T07:02:31.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.685+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca4806c350 con 0x7fca48153da0 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.685+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fca3c00f070 con 0x7fca48153da0 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.685+0000 7fca44ff9700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca3405ed90 0x7fca34061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.685+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fca3c07b250 con 0x7fca48153da0 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca44ff9700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7fca34068130 0x7fca3406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 --> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca3406ac10 con 0x7fca34068130 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c07d8c0 con 0x7fca48153da0 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca4c8c4700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca3405ed90 0x7fca34061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca4d0c5700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7fca34068130 0x7fca3406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca4c8c4700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca3405ed90 0x7fca34061250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fca38007fd0 tx=0x7fca38009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.686+0000 7fca4d0c5700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7fca34068130 0x7fca3406a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.687+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== osd.7 v2:172.21.15.158:6808/86892 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fca3406ac10 con 0x7fca34068130 2023-05-16T07:02:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.701+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 --> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca480c2e10 con 0x7fca34068130 2023-05-16T07:02:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.701+0000 7fca44ff9700 1 -- 172.21.15.92:0/4115858583 <== osd.7 v2:172.21.15.158:6808/86892 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fca480c2e10 con 0x7fca34068130 2023-05-16T07:02:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7fca34068130 msgr2=0x7fca3406a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7fca34068130 0x7fca3406a590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca3405ed90 msgr2=0x7fca34061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca3405ed90 0x7fca34061250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fca38007fd0 tx=0x7fca38009040).stop 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48153da0 msgr2=0x7fca4814f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 --2- 172.21.15.92:0/4115858583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca48153da0 0x7fca4814f670 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fca3c009e00 tx=0x7fca3c017000).stop 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4d0c5700 1 -- 172.21.15.92:0/4115858583 reap_dead start 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 shutdown_connections 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 >> 172.21.15.92:0/4115858583 conn(0x7fca480c8aa0 msgr2=0x7fca480ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 shutdown_connections 2023-05-16T07:02:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.702+0000 7fca4eb28700 1 -- 172.21.15.92:0/4115858583 wait complete. 2023-05-16T07:02:31.717 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:31.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-05-16T07:02:31.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.827+0000 7f14978b8700 1 Processor -- start 2023-05-16T07:02:31.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.827+0000 7f14978b8700 1 -- start start 2023-05-16T07:02:31.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f14978b8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1490154170 0x7f14900c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f14978b8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490069db0 0x7f14900c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f14978b8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900c0660 con 0x7f1490154550 2023-05-16T07:02:31.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f14978b8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900c1b00 con 0x7f1490154170 2023-05-16T07:02:31.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f14978b8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900bfd30 con 0x7f1490069db0 2023-05-16T07:02:31.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1494e53700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1490154550 0x7f14900696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51466/0 (socket says 172.21.15.92:51466) 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1494e53700 1 -- 172.21.15.92:0/132673384 learned_addr learned my addr 172.21.15.92:0/132673384 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495654700 1 --2- 172.21.15.92:0/132673384 >> v2:172.21.15.158:3300/0 conn(0x7f1490154170 0x7f14900c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495e55700 1 --2- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490069db0 0x7f14900c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1487fff700 1 -- 172.21.15.92:0/132673384 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844776253 0 0) 0x7f14900c0660 con 0x7f1490154550 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1487fff700 1 -- 172.21.15.92:0/132673384 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f147c0033f0 con 0x7f1490154550 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495654700 1 -- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490069db0 msgr2=0x7f14900c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495654700 1 --2- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490069db0 0x7f14900c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495654700 1 -- 172.21.15.92:0/132673384 >> v1:172.21.15.92:6789/0 conn(0x7f1490154550 legacy=0x7f14900696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1487fff700 1 -- 172.21.15.92:0/132673384 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2288190022 0 0) 0x7f147c0033f0 con 0x7f1490154550 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495654700 1 -- 172.21.15.92:0/132673384 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900c0010 con 0x7f1490154170 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1495654700 1 --2- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900c5c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f148c009760 tx=0x7f148c009060).ready entity=mon.1 client_cookie=717c1a864f743b08 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.828+0000 7f1487fff700 1 -- 172.21.15.92:0/132673384 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480001470 con 0x7f1490154170 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f1487fff700 1 -- 172.21.15.92:0/132673384 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c0090d0 con 0x7f1490154170 2023-05-16T07:02:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f1487fff700 1 -- 172.21.15.92:0/132673384 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c011610 con 0x7f1490154170 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 -- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 msgr2=0x7f14900c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 --2- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900c5c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f148c009760 tx=0x7f148c009060).stop 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 -- 172.21.15.92:0/132673384 shutdown_connections 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 --2- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490069db0 0x7f14900c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 --2- 172.21.15.92:0/132673384 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900c5c70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 -- 172.21.15.92:0/132673384 >> 172.21.15.92:0/132673384 conn(0x7f14900cb700 msgr2=0x7f14900c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 -- 172.21.15.92:0/132673384 shutdown_connections 2023-05-16T07:02:31.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.829+0000 7f14978b8700 1 -- 172.21.15.92:0/132673384 wait complete. 2023-05-16T07:02:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.830+0000 7f14978b8700 1 Processor -- start 2023-05-16T07:02:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.830+0000 7f14978b8700 1 -- start start 2023-05-16T07:02:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f14978b8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 0x7f149013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f14978b8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f14978b8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490154550 0x7f14900abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f14978b8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900ac0e0 con 0x7f1490069db0 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f14978b8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490140190 con 0x7f1490154170 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f14978b8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1490140470 con 0x7f1490154550 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 0x7f149013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 0x7f149013e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40722/0 (socket says 172.21.15.92:40722) 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 -- 172.21.15.92:0/3286494875 learned_addr learned my addr 172.21.15.92:0/3286494875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495e55700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490154550 0x7f14900abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1494e53700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 -- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490154550 msgr2=0x7f14900abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490154550 0x7f14900abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 -- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 msgr2=0x7f14900ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 -- 172.21.15.92:0/3286494875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0092a0 con 0x7f1490069db0 2023-05-16T07:02:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1494e53700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.831+0000 7f1495654700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 0x7f149013e810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f148c009d50 tx=0x7f148c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.832+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0107c0 con 0x7f1490069db0 2023-05-16T07:02:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.832+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c010920 con 0x7f1490069db0 2023-05-16T07:02:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.832+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1490140750 con 0x7f1490069db0 2023-05-16T07:02:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.832+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c010cc0 con 0x7f1490069db0 2023-05-16T07:02:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.832+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1490140d10 con 0x7f1490069db0 2023-05-16T07:02:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.833+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14900666c0 con 0x7f1490069db0 2023-05-16T07:02:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.833+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f148c00f070 con 0x7f1490069db0 2023-05-16T07:02:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.833+0000 7f14857fa700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f147405ed60 0x7f1474061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.833+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f148c07af10 con 0x7f1490069db0 2023-05-16T07:02:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.833+0000 7f1494e53700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f147405ed60 0x7f1474061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.834+0000 7f1494e53700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f147405ed60 0x7f1474061220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f1490154930 tx=0x7f1480009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:31.837+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f148c04c4b0 con 0x7f1490069db0 2023-05-16T07:02:32.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.002+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f149006c350 con 0x7f1490069db0 2023-05-16T07:02:32.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.002+0000 7f14857fa700 1 -- 172.21.15.92:0/3286494875 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f148c04fad0 con 0x7f1490069db0 2023-05-16T07:02:32.006 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:32.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f147405ed60 msgr2=0x7f1474061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:32.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f147405ed60 0x7f1474061220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f1490154930 tx=0x7f1480009040).stop 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 msgr2=0x7f149013e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 0x7f149013e810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f148c009d50 tx=0x7f148c016000).stop 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 shutdown_connections 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f147405ed60 0x7f1474061220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1490154550 0x7f14900abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1490069db0 0x7f149013e810 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:32.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 --2- 172.21.15.92:0/3286494875 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1490154170 0x7f14900ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:32.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 >> 172.21.15.92:0/3286494875 conn(0x7f14900cb700 msgr2=0x7f14900b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:32.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 shutdown_connections 2023-05-16T07:02:32.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:32.004+0000 7f14978b8700 1 -- 172.21.15.92:0/3286494875 wait complete. 2023-05-16T07:02:32.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2023-05-16T07:02:33.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-05-16T07:02:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.138+0000 7f39aeab6700 1 Processor -- start 2023-05-16T07:02:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.138+0000 7f39aeab6700 1 -- start start 2023-05-16T07:02:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39aeab6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f39a8069750 0x7f39a8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39aeab6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39a80aab40 0x7f39a80c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39aeab6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39aeab6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a80c65f0 con 0x7f39a80aab40 2023-05-16T07:02:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39aeab6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a806a920 con 0x7f39a8069750 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39aeab6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c63c0 con 0x7f39a80c5f70 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39a80aab40 0x7f39a80c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39a80aab40 0x7f39a80c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40732/0 (socket says 172.21.15.92:40732) 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 -- 172.21.15.92:0/4004161732 learned_addr learned my addr 172.21.15.92:0/4004161732 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ae2b5700 1 --2- 172.21.15.92:0/4004161732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39adab4700 1 --2- 172.21.15.92:0/4004161732 >> v2:172.21.15.158:3300/0 conn(0x7f39a8069750 0x7f39a8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 -- 172.21.15.92:0/4004161732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 msgr2=0x7f39a8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 --2- 172.21.15.92:0/4004161732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 -- 172.21.15.92:0/4004161732 >> v2:172.21.15.158:3300/0 conn(0x7f39a8069750 msgr2=0x7f39a8069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 --2- 172.21.15.92:0/4004161732 >> v2:172.21.15.158:3300/0 conn(0x7f39a8069750 0x7f39a8069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 -- 172.21.15.92:0/4004161732 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80bfa30 con 0x7f39a80aab40 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.139+0000 7f39ad2b3700 1 --2- 172.21.15.92:0/4004161732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80c5a80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f39a80c74d0 tx=0x7f39980090f0).ready entity=mon.0 client_cookie=a41e0590cd8942cd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39acab2700 1 -- 172.21.15.92:0/4004161732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800f510 con 0x7f39a80aab40 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39acab2700 1 -- 172.21.15.92:0/4004161732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800f670 con 0x7f39a80aab40 2023-05-16T07:02:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39acab2700 1 -- 172.21.15.92:0/4004161732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800f9f0 con 0x7f39a80aab40 2023-05-16T07:02:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 -- 172.21.15.92:0/4004161732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 msgr2=0x7f39a80c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 --2- 172.21.15.92:0/4004161732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80c5a80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f39a80c74d0 tx=0x7f39980090f0).stop 2023-05-16T07:02:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 -- 172.21.15.92:0/4004161732 shutdown_connections 2023-05-16T07:02:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 --2- 172.21.15.92:0/4004161732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 --2- 172.21.15.92:0/4004161732 >> v2:172.21.15.158:3300/0 conn(0x7f39a8069750 0x7f39a8069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 --2- 172.21.15.92:0/4004161732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80c5a80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 -- 172.21.15.92:0/4004161732 >> 172.21.15.92:0/4004161732 conn(0x7f39a80cb550 msgr2=0x7f39a80c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 -- 172.21.15.92:0/4004161732 shutdown_connections 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.140+0000 7f39aeab6700 1 -- 172.21.15.92:0/4004161732 wait complete. 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.141+0000 7f39aeab6700 1 Processor -- start 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.141+0000 7f39aeab6700 1 -- start start 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39aeab6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39a8069750 0x7f39a80ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39aeab6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39aeab6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39aeab6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80ae460 con 0x7f39a80aab40 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39aeab6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a81545d0 con 0x7f39a8069750 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39aeab6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8153a00 con 0x7f39a80c5f70 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40736/0 (socket says 172.21.15.92:40736) 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 -- 172.21.15.92:0/777531512 learned_addr learned my addr 172.21.15.92:0/777531512 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:33.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ae2b5700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39adab4700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39a8069750 0x7f39a80ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 -- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 msgr2=0x7f39a8154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 -- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39a8069750 msgr2=0x7f39a80ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39a8069750 0x7f39a80ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 -- 172.21.15.92:0/777531512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009370 con 0x7f39a80aab40 2023-05-16T07:02:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.142+0000 7f39ad2b3700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80adf70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f3998011000 tx=0x7f3998017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.143+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800f7e0 con 0x7f39a80aab40 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.143+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800f940 con 0x7f39a80aab40 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.143+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a8153ca0 con 0x7f39a80aab40 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.143+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399801c670 con 0x7f39a80aab40 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.143+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a8154260 con 0x7f39a80aab40 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.144+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a8083040 con 0x7f39a80aab40 2023-05-16T07:02:33.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.144+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f399802c070 con 0x7f39a80aab40 2023-05-16T07:02:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.144+0000 7f399dffb700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f398c05ed90 0x7f398c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.144+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f399807b870 con 0x7f39a80aab40 2023-05-16T07:02:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.144+0000 7f39adab4700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f398c05ed90 0x7f398c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.145+0000 7f39adab4700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f398c05ed90 0x7f398c061250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f39a00076a0 tx=0x7f39a000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.148+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f399804cdb0 con 0x7f39a80aab40 2023-05-16T07:02:33.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.311+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39a80bfc80 con 0x7f39a80aab40 2023-05-16T07:02:33.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.311+0000 7f399dffb700 1 -- 172.21.15.92:0/777531512 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3998020070 con 0x7f39a80aab40 2023-05-16T07:02:33.315 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f398c05ed90 msgr2=0x7f398c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f398c05ed90 0x7f398c061250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f39a00076a0 tx=0x7f39a000b040).stop 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 msgr2=0x7f39a80adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80adf70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f3998011000 tx=0x7f3998017000).stop 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 shutdown_connections 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f398c05ed90 0x7f398c061250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39a80c5f70 0x7f39a8154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39a8069750 0x7f39a80ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 --2- 172.21.15.92:0/777531512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39a80aab40 0x7f39a80adf70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 >> 172.21.15.92:0/777531512 conn(0x7f39a80cb550 msgr2=0x7f39a80c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 shutdown_connections 2023-05-16T07:02:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.313+0000 7f39aeab6700 1 -- 172.21.15.92:0/777531512 wait complete. 2023-05-16T07:02:33.328 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2023-05-16T07:02:33.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-05-16T07:02:33.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.437+0000 7f297007e700 1 Processor -- start 2023-05-16T07:02:33.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.437+0000 7f297007e700 1 -- start start 2023-05-16T07:02:33.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f297007e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f296814e200 0x7f296814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f297007e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29680683d0 con 0x7f2968066e50 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f297007e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29680be060 con 0x7f2968158160 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f297007e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2968137470 con 0x7f296814e200 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296de1a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f296814e200 0x7f296814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296d619700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2968066e50 0x7f2968157a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51494/0 (socket says 172.21.15.92:51494) 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296d619700 1 -- 172.21.15.92:0/3108902335 learned_addr learned my addr 172.21.15.92:0/3108902335 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296de1a700 1 -- 172.21.15.92:0/3108902335 >> v1:172.21.15.158:6789/0 conn(0x7f2968158160 legacy=0x7f296815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:33.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296de1a700 1 -- 172.21.15.92:0/3108902335 >> v1:172.21.15.92:6789/0 conn(0x7f2968066e50 legacy=0x7f2968157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296ce18700 1 -- 172.21.15.92:0/3108902335 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699713922 0 0) 0x7f29680683d0 con 0x7f2968066e50 2023-05-16T07:02:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.438+0000 7f296de1a700 1 -- 172.21.15.92:0/3108902335 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680677c0 con 0x7f296814e200 2023-05-16T07:02:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f296de1a700 1 --2- 172.21.15.92:0/3108902335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f296814e200 0x7f296814e5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f2958009e60 tx=0x7f2958009780).ready entity=mon.0 client_cookie=a51d6082c2e363c7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f296ce18700 1 -- 172.21.15.92:0/3108902335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295800e070 con 0x7f296814e200 2023-05-16T07:02:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f296ce18700 1 -- 172.21.15.92:0/3108902335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295800bbf0 con 0x7f296814e200 2023-05-16T07:02:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f296ce18700 1 -- 172.21.15.92:0/3108902335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958012410 con 0x7f296814e200 2023-05-16T07:02:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 -- 172.21.15.92:0/3108902335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f296814e200 msgr2=0x7f296814e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 --2- 172.21.15.92:0/3108902335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f296814e200 0x7f296814e5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f2958009e60 tx=0x7f2958009780).stop 2023-05-16T07:02:33.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 -- 172.21.15.92:0/3108902335 shutdown_connections 2023-05-16T07:02:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 --2- 172.21.15.92:0/3108902335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f296814e200 0x7f296814e5e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 -- 172.21.15.92:0/3108902335 >> 172.21.15.92:0/3108902335 conn(0x7f29680c7bc0 msgr2=0x7f29680c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 -- 172.21.15.92:0/3108902335 shutdown_connections 2023-05-16T07:02:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.439+0000 7f297007e700 1 -- 172.21.15.92:0/3108902335 wait complete. 2023-05-16T07:02:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.440+0000 7f297007e700 1 Processor -- start 2023-05-16T07:02:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.440+0000 7f297007e700 1 -- start start 2023-05-16T07:02:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f297007e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2968066e50 0x7f29680a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f297007e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f296814e200 0x7f2968136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f297007e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 0x7f29680a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f297007e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29680a7930 con 0x7f2968158160 2023-05-16T07:02:33.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f297007e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29680a6d20 con 0x7f2968066e50 2023-05-16T07:02:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f297007e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29680a7000 con 0x7f296814e200 2023-05-16T07:02:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 0x7f29680a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 0x7f29680a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40752/0 (socket says 172.21.15.92:40752) 2023-05-16T07:02:33.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 -- 172.21.15.92:0/160030216 learned_addr learned my addr 172.21.15.92:0/160030216 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 -- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f296814e200 msgr2=0x7f2968136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296de1a700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2968066e50 0x7f29680a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296d619700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f296814e200 0x7f2968136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f296814e200 0x7f2968136a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 -- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2968066e50 msgr2=0x7f29680a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2968066e50 0x7f29680a7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.441+0000 7f296e61b700 1 -- 172.21.15.92:0/160030216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958009a20 con 0x7f2968158160 2023-05-16T07:02:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.442+0000 7f296e61b700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 0x7f29680a4ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f2960009470 tx=0x7f2960006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.442+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960013460 con 0x7f2968158160 2023-05-16T07:02:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.442+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960013600 con 0x7f2968158160 2023-05-16T07:02:33.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.442+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600139c0 con 0x7f2968158160 2023-05-16T07:02:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.442+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29680a72e0 con 0x7f2968158160 2023-05-16T07:02:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.442+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2968136f70 con 0x7f2968158160 2023-05-16T07:02:33.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.443+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29680a7aa0 con 0x7f2968158160 2023-05-16T07:02:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.443+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f296000f070 con 0x7f2968158160 2023-05-16T07:02:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.443+0000 7f2965ffb700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f295405ed90 0x7f2954061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.443+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2960079200 con 0x7f2968158160 2023-05-16T07:02:33.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.443+0000 7f296de1a700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f295405ed90 0x7f2954061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.444+0000 7f296de1a700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f295405ed90 0x7f2954061250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2958009e60 tx=0x7f2958009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.447+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f296004a7a0 con 0x7f2968158160 2023-05-16T07:02:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.611+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29680a82b0 con 0x7f2968158160 2023-05-16T07:02:33.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.611+0000 7f2965ffb700 1 -- 172.21.15.92:0/160030216 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f296004ddc0 con 0x7f2968158160 2023-05-16T07:02:33.615 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-05-16T07:02:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f295405ed90 msgr2=0x7f2954061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f295405ed90 0x7f2954061250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2958009e60 tx=0x7f2958009780).stop 2023-05-16T07:02:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 msgr2=0x7f29680a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 0x7f29680a4ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f2960009470 tx=0x7f2960006ea0).stop 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 shutdown_connections 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f295405ed90 0x7f2954061250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2968158160 0x7f29680a4ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2968066e50 0x7f29680a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 --2- 172.21.15.92:0/160030216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f296814e200 0x7f2968136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 >> 172.21.15.92:0/160030216 conn(0x7f29680c7bc0 msgr2=0x7f29681572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 shutdown_connections 2023-05-16T07:02:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.613+0000 7f297007e700 1 -- 172.21.15.92:0/160030216 wait complete. 2023-05-16T07:02:33.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2023-05-16T07:02:33.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-05-16T07:02:33.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.737+0000 7fe0c54eb700 1 Processor -- start 2023-05-16T07:02:33.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.737+0000 7fe0c54eb700 1 -- start start 2023-05-16T07:02:33.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0c54eb700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe0c0154550 0x7fe0c00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0c54eb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0c0069b90 0x7fe0c00c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0c54eb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c00c0660 con 0x7fe0c0154170 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0c54eb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c0069fc0 con 0x7fe0c0069b90 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0c54eb700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00bfd30 con 0x7fe0c0154550 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0c0069b90 0x7fe0c00c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0c0069b90 0x7fe0c00c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40758/0 (socket says 172.21.15.92:40758) 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0beffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe0c0154170 0x7fe0c00c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51504/0 (socket says 172.21.15.92:51504) 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2817175899 learned_addr learned my addr 172.21.15.92:0/2817175899 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:33.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2817175899 >> v2:172.21.15.158:3300/0 conn(0x7fe0c0154550 msgr2=0x7fe0c00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 --2- 172.21.15.92:0/2817175899 >> v2:172.21.15.158:3300/0 conn(0x7fe0c0154550 0x7fe0c00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2817175899 >> v1:172.21.15.92:6789/0 conn(0x7fe0c0154170 legacy=0x7fe0c00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2817175899 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00c0010 con 0x7fe0c0069b90 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.738+0000 7fe0bdffb700 1 -- 172.21.15.92:0/2817175899 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331326965 0 0) 0x7fe0c00c0660 con 0x7fe0c0154170 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0bf7fe700 1 --2- 172.21.15.92:0/2817175899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c0af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe0c00844e0 tx=0x7fe0b80090f0).ready entity=mon.0 client_cookie=e17cdc2892c416a3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0bdffb700 1 -- 172.21.15.92:0/2817175899 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800b600 con 0x7fe0c0069b90 2023-05-16T07:02:33.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0bdffb700 1 -- 172.21.15.92:0/2817175899 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800b760 con 0x7fe0c0069b90 2023-05-16T07:02:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0bdffb700 1 -- 172.21.15.92:0/2817175899 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800bb00 con 0x7fe0c0069b90 2023-05-16T07:02:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 -- 172.21.15.92:0/2817175899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 msgr2=0x7fe0c00c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/2817175899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c0af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe0c00844e0 tx=0x7fe0b80090f0).stop 2023-05-16T07:02:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 -- 172.21.15.92:0/2817175899 shutdown_connections 2023-05-16T07:02:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/2817175899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c0af0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/2817175899 >> v2:172.21.15.158:3300/0 conn(0x7fe0c0154550 0x7fe0c00696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 -- 172.21.15.92:0/2817175899 >> 172.21.15.92:0/2817175899 conn(0x7fe0c00cb700 msgr2=0x7fe0c00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 -- 172.21.15.92:0/2817175899 shutdown_connections 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.739+0000 7fe0c54eb700 1 -- 172.21.15.92:0/2817175899 wait complete. 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.740+0000 7fe0c54eb700 1 Processor -- start 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 -- start start 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c0154170 0x7fe0c00c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0154550 0x7fe0c00c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c2720 con 0x7fe0c0069b90 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c014d450 con 0x7fe0c0154170 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0c54eb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ae450 con 0x7fe0c0154550 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40762/0 (socket says 172.21.15.92:40762) 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.741+0000 7fe0beffd700 1 -- 172.21.15.92:0/618312434 learned_addr learned my addr 172.21.15.92:0/618312434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:33.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0beffd700 1 -- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0154550 msgr2=0x7fe0c00c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0be7fc700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c0154170 0x7fe0c00c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0bf7fe700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0154550 0x7fe0c00c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0beffd700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0154550 0x7fe0c00c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0beffd700 1 -- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c0154170 msgr2=0x7fe0c00c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0beffd700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c0154170 0x7fe0c00c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0beffd700 1 -- 172.21.15.92:0/618312434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8009370 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0beffd700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c3e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe0b0001560 tx=0x7fe0b00016c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0005c70 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0005dd0 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0007320 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c00ae6f0 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.742+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00aecb0 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.743+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00666c0 con 0x7fe0c0069b90 2023-05-16T07:02:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.743+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe0b0010030 con 0x7fe0c0069b90 2023-05-16T07:02:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.743+0000 7fe0b6ffd700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.743+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe0b0079ab0 con 0x7fe0c0069b90 2023-05-16T07:02:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.744+0000 7fe0be7fc700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.744+0000 7fe0be7fc700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe0c0154930 tx=0x7fe0a8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:33.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.748+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe0b004b050 con 0x7fe0c0069b90 2023-05-16T07:02:33.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.912+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe0c006c350 con 0x7fe0c0069b90 2023-05-16T07:02:33.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.913+0000 7fe0b6ffd700 1 -- 172.21.15.92:0/618312434 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0b004e670 con 0x7fe0c0069b90 2023-05-16T07:02:33.917 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-05-16T07:02:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 msgr2=0x7fe0a0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe0c0154930 tx=0x7fe0a8008040).stop 2023-05-16T07:02:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 msgr2=0x7fe0c00c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c3e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe0b0001560 tx=0x7fe0b00016c0).stop 2023-05-16T07:02:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 shutdown_connections 2023-05-16T07:02:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0154550 0x7fe0c00c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c0069b90 0x7fe0c00c3e00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 --2- 172.21.15.92:0/618312434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c0154170 0x7fe0c00c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 >> 172.21.15.92:0/618312434 conn(0x7fe0c00cb700 msgr2=0x7fe0c00aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 shutdown_connections 2023-05-16T07:02:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:33.915+0000 7fe0c54eb700 1 -- 172.21.15.92:0/618312434 wait complete. 2023-05-16T07:02:33.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-05-16T07:02:33.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-05-16T07:02:34.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 Processor -- start 2023-05-16T07:02:34.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 -- start start 2023-05-16T07:02:34.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64c0063e30 0x7f64c00c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b930 0x7f64c00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c00bdc30 con 0x7f64c013b550 2023-05-16T07:02:34.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c00c5400 con 0x7f64c0063e30 2023-05-16T07:02:34.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.038+0000 7f64c58e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c4e40 con 0x7f64c013b930 2023-05-16T07:02:34.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64c0063e30 0x7f64c00c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64c0063e30 0x7f64c00c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40772/0 (socket says 172.21.15.92:40772) 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 -- 172.21.15.92:0/1918116582 learned_addr learned my addr 172.21.15.92:0/1918116582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64bf7fe700 1 --2- 172.21.15.92:0/1918116582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b930 0x7f64c00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 -- 172.21.15.92:0/1918116582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b930 msgr2=0x7f64c00abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 --2- 172.21.15.92:0/1918116582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b930 0x7f64c00abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 -- 172.21.15.92:0/1918116582 >> v1:172.21.15.158:6789/0 conn(0x7f64c013b550 legacy=0x7f64c0063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 -- 172.21.15.92:0/1918116582 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00c5120 con 0x7f64c0063e30 2023-05-16T07:02:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64be7fc700 1 --2- 172.21.15.92:0/1918116582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c31f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f64b4006fd0 tx=0x7f64b40090f0).ready entity=mon.0 client_cookie=9e0f1937ea475fdc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64bdffb700 1 -- 172.21.15.92:0/1918116582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b400b600 con 0x7f64c0063e30 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64bdffb700 1 -- 172.21.15.92:0/1918116582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b400b760 con 0x7f64c0063e30 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.039+0000 7f64bdffb700 1 -- 172.21.15.92:0/1918116582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b400bae0 con 0x7f64c0063e30 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 -- 172.21.15.92:0/1918116582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 msgr2=0x7f64c00c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1918116582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c31f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f64b4006fd0 tx=0x7f64b40090f0).stop 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 -- 172.21.15.92:0/1918116582 shutdown_connections 2023-05-16T07:02:34.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1918116582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b930 0x7f64c00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1918116582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c31f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 -- 172.21.15.92:0/1918116582 >> 172.21.15.92:0/1918116582 conn(0x7f64c00c8aa0 msgr2=0x7f64c00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 -- 172.21.15.92:0/1918116582 shutdown_connections 2023-05-16T07:02:34.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.040+0000 7f64c58e6700 1 -- 172.21.15.92:0/1918116582 wait complete. 2023-05-16T07:02:34.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.041+0000 7f64c58e6700 1 Processor -- start 2023-05-16T07:02:34.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.041+0000 7f64c58e6700 1 -- start start 2023-05-16T07:02:34.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b550 0x7f64c00c0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64c013b930 0x7f64c0137780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c55d0 con 0x7f64c0063e30 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c0d30 con 0x7f64c013b930 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00c1010 con 0x7f64c013b550 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40776/0 (socket says 172.21.15.92:40776) 2023-05-16T07:02:34.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 -- 172.21.15.92:0/1322132752 learned_addr learned my addr 172.21.15.92:0/1322132752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64be7fc700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b550 0x7f64c00c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64bf7fe700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64c013b930 0x7f64c0137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 -- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b550 msgr2=0x7f64c00c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b550 0x7f64c00c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 -- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64c013b930 msgr2=0x7f64c0137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64c013b930 0x7f64c0137780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 -- 172.21.15.92:0/1322132752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4009370 con 0x7f64c0063e30 2023-05-16T07:02:34.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64be7fc700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b550 0x7f64c00c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:34.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64bf7fe700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64c013b930 0x7f64c0137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:34.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64beffd700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c24e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f64b000c840 tx=0x7f64b0001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0006a20 con 0x7f64c0063e30 2023-05-16T07:02:34.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b0006b80 con 0x7f64c0063e30 2023-05-16T07:02:34.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c00c12f0 con 0x7f64c0063e30 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0007020 con 0x7f64c0063e30 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.042+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c00c0a40 con 0x7f64c0063e30 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.043+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c0063930 con 0x7f64c0063e30 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.044+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f64b0018020 con 0x7f64c0063e30 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.044+0000 7f64a6ffd700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64ac05ed90 0x7f64ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.044+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f64b0079170 con 0x7f64c0063e30 2023-05-16T07:02:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.044+0000 7f64be7fc700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64ac05ed90 0x7f64ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.044+0000 7f64be7fc700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64ac05ed90 0x7f64ac061250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f64c0064230 tx=0x7f64b4017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.048+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f64b00488f0 con 0x7f64c0063e30 2023-05-16T07:02:34.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.209+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64c006c350 con 0x7f64c0063e30 2023-05-16T07:02:34.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.210+0000 7f64a6ffd700 1 -- 172.21.15.92:0/1322132752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64b004a710 con 0x7f64c0063e30 2023-05-16T07:02:34.216 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:34.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.211+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64ac05ed90 msgr2=0x7f64ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.211+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64ac05ed90 0x7f64ac061250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f64c0064230 tx=0x7f64b4017000).stop 2023-05-16T07:02:34.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 msgr2=0x7f64c00c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c24e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f64b000c840 tx=0x7f64b0001660).stop 2023-05-16T07:02:34.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 shutdown_connections 2023-05-16T07:02:34.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64ac05ed90 0x7f64ac061250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64c013b930 0x7f64c0137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64c0063e30 0x7f64c00c24e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 --2- 172.21.15.92:0/1322132752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64c013b550 0x7f64c00c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 >> 172.21.15.92:0/1322132752 conn(0x7f64c00c8aa0 msgr2=0x7f64c0154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 shutdown_connections 2023-05-16T07:02:34.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.212+0000 7f64c58e6700 1 -- 172.21.15.92:0/1322132752 wait complete. 2023-05-16T07:02:34.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2023-05-16T07:02:34.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-05-16T07:02:34.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.338+0000 7f4ae6c0a700 1 Processor -- start 2023-05-16T07:02:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.339+0000 7f4ae6c0a700 1 -- start start 2023-05-16T07:02:34.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.339+0000 7f4ae6c0a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4ae0067f60 0x7f4ae00bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.339+0000 7f4ae6c0a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 0x7f4ae00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.339+0000 7f4ae6c0a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae00691c0 con 0x7f4ae0153ca0 2023-05-16T07:02:34.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.339+0000 7f4ae6c0a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae00683c0 con 0x7f4ae0067f60 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.339+0000 7f4ae6c0a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0069d60 con 0x7f4ae00c8800 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae5407700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4ae0067f60 0x7f4ae00bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 0x7f4ae00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae5407700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4ae0067f60 0x7f4ae00bf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44422/0 (socket says 172.21.15.92:44422) 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 0x7f4ae00c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:33360/0 (socket says 172.21.15.92:33360) 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae5407700 1 -- 172.21.15.92:0/2765839852 learned_addr learned my addr 172.21.15.92:0/2765839852 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae5c08700 1 --1- 172.21.15.92:0/2765839852 >> v1:172.21.15.158:6789/0 conn(0x7f4ae0153ca0 0x7f4ae0069650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:55132/0 (socket says 172.21.15.92:55132) 2023-05-16T07:02:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 -- 172.21.15.92:0/2765839852 >> v2:172.21.15.158:3300/0 conn(0x7f4ae0067f60 msgr2=0x7f4ae00bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 --2- 172.21.15.92:0/2765839852 >> v2:172.21.15.158:3300/0 conn(0x7f4ae0067f60 0x7f4ae00bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 -- 172.21.15.92:0/2765839852 >> v1:172.21.15.158:6789/0 conn(0x7f4ae0153ca0 legacy=0x7f4ae0069650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 -- 172.21.15.92:0/2765839852 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00c9140 con 0x7f4ae00c8800 2023-05-16T07:02:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae6409700 1 --2- 172.21.15.92:0/2765839852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 0x7f4ae00c8c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f4ad8009760 tx=0x7f4ad8009060).ready entity=mon.2 client_cookie=98a45363362e6d61 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae4c06700 1 -- 172.21.15.92:0/2765839852 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800f510 con 0x7f4ae00c8800 2023-05-16T07:02:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.340+0000 7f4ae4c06700 1 -- 172.21.15.92:0/2765839852 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad800f670 con 0x7f4ae00c8800 2023-05-16T07:02:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae4c06700 1 -- 172.21.15.92:0/2765839852 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800f9f0 con 0x7f4ae00c8800 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2765839852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 msgr2=0x7f4ae00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2765839852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 0x7f4ae00c8c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f4ad8009760 tx=0x7f4ad8009060).stop 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2765839852 shutdown_connections 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2765839852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae00c8800 0x7f4ae00c8c50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2765839852 >> v2:172.21.15.158:3300/0 conn(0x7f4ae0067f60 0x7f4ae00bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2765839852 >> 172.21.15.92:0/2765839852 conn(0x7f4ae00622b0 msgr2=0x7f4ae00626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2765839852 shutdown_connections 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.341+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2765839852 wait complete. 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.342+0000 7f4ae6c0a700 1 Processor -- start 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.342+0000 7f4ae6c0a700 1 -- start start 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6c0a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae0067f60 0x7f4ae0148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6c0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 0x7f4ae01675d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6c0a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ae0153ca0 0x7f4ae0167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6c0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0167fb0 con 0x7f4ae00c8800 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6c0a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae00a9f10 con 0x7f4ae0153ca0 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6c0a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae00aa1f0 con 0x7f4ae0067f60 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 0x7f4ae01675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 0x7f4ae01675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40792/0 (socket says 172.21.15.92:40792) 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 -- 172.21.15.92:0/2086740308 learned_addr learned my addr 172.21.15.92:0/2086740308 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5c08700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae0067f60 0x7f4ae0148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae6409700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ae0153ca0 0x7f4ae0167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 -- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae0067f60 msgr2=0x7f4ae0148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae0067f60 0x7f4ae0148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 -- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ae0153ca0 msgr2=0x7f4ae0167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ae0153ca0 0x7f4ae0167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 -- 172.21.15.92:0/2086740308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80092a0 con 0x7f4ae00c8800 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.343+0000 7f4ae5407700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 0x7f4ae01675d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f4ad400e900 tx=0x7f4ad40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.344+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4009420 con 0x7f4ae00c8800 2023-05-16T07:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.344+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4012600 con 0x7f4ae00c8800 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.344+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae00aa4d0 con 0x7f4ae00c8800 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.344+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4012980 con 0x7f4ae00c8800 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.344+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae01491f0 con 0x7f4ae00c8800 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.345+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae00a58e0 con 0x7f4ae00c8800 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.345+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4ad4022070 con 0x7f4ae00c8800 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.345+0000 7f4addffb700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4acc05ed90 0x7f4acc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.345+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4ad4079d60 con 0x7f4ae00c8800 2023-05-16T07:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.345+0000 7f4ae5c08700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4acc05ed90 0x7f4acc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.346+0000 7f4ae5c08700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4acc05ed90 0x7f4acc061250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4ad0001a10 tx=0x7f4ad0001560).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.349+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4ae00a58e0 con 0x7f4ae00c8800 2023-05-16T07:02:34.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.515+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4ae00a58e0 con 0x7f4ae00c8800 2023-05-16T07:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.516+0000 7f4addffb700 1 -- 172.21.15.92:0/2086740308 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ae00a58e0 con 0x7f4ae00c8800 2023-05-16T07:02:34.520 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.517+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4acc05ed90 msgr2=0x7f4acc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.517+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4acc05ed90 0x7f4acc061250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4ad0001a10 tx=0x7f4ad0001560).stop 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.517+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 msgr2=0x7f4ae01675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.517+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 0x7f4ae01675d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f4ad400e900 tx=0x7f4ad40090f0).stop 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.517+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 shutdown_connections 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.517+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4acc05ed90 0x7f4acc061250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.518+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ae0067f60 0x7f4ae0148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.518+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ae0153ca0 0x7f4ae0167ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.518+0000 7f4ae6c0a700 1 --2- 172.21.15.92:0/2086740308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae00c8800 0x7f4ae01675d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.518+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 >> 172.21.15.92:0/2086740308 conn(0x7f4ae00622b0 msgr2=0x7f4ae0063bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.518+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 shutdown_connections 2023-05-16T07:02:34.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.518+0000 7f4ae6c0a700 1 -- 172.21.15.92:0/2086740308 wait complete. 2023-05-16T07:02:34.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2023-05-16T07:02:34.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-05-16T07:02:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.641+0000 7fad0a78a700 1 Processor -- start 2023-05-16T07:02:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.641+0000 7fad0a78a700 1 -- start start 2023-05-16T07:02:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad0a78a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fad04063df0 0x7fad0415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad0a78a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad04154f20 con 0x7fad04151bb0 2023-05-16T07:02:34.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad0a78a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad04064580 con 0x7fad04067f60 2023-05-16T07:02:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad0a78a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad04151f90 con 0x7fad04063df0 2023-05-16T07:02:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad03fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad04151bb0 0x7fad04069a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51544/0 (socket says 172.21.15.92:51544) 2023-05-16T07:02:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad03fff700 1 -- 172.21.15.92:0/2611601063 learned_addr learned my addr 172.21.15.92:0/2611601063 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad08d27700 1 --2- 172.21.15.92:0/2611601063 >> v2:172.21.15.158:3300/0 conn(0x7fad04063df0 0x7fad0415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707402691 0 0) 0x7fad04154f20 con 0x7fad04151bb0 2023-05-16T07:02:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf00033f0 con 0x7fad04151bb0 2023-05-16T07:02:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387184192 0 0) 0x7facf00033f0 con 0x7fad04151bb0 2023-05-16T07:02:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 >> v2:172.21.15.158:3300/0 conn(0x7fad04063df0 msgr2=0x7fad0415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 --2- 172.21.15.92:0/2611601063 >> v2:172.21.15.158:3300/0 conn(0x7fad04063df0 0x7fad0415ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 >> v1:172.21.15.158:6789/0 conn(0x7fad04067f60 legacy=0x7fad040636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04154630 con 0x7fad04151bb0 2023-05-16T07:02:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7facf4001470 con 0x7fad04151bb0 2023-05-16T07:02:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 >> v1:172.21.15.92:6789/0 conn(0x7fad04151bb0 legacy=0x7fad04069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facf00038e0 0x7facf0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf0004690 0x7facf0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0005000 0x7facf000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf000be80 con 0x7facf0004690 2023-05-16T07:02:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf000bff0 con 0x7facf0005000 2023-05-16T07:02:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf000c1b0 con 0x7facf00038e0 2023-05-16T07:02:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7facfbfff700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf0004690 0x7facf0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad03fff700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facf00038e0 0x7facf0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.642+0000 7fad08d27700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0005000 0x7facf000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7facfbfff700 1 -- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facf00038e0 msgr2=0x7facf0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7facfbfff700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facf00038e0 0x7facf0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7facfbfff700 1 -- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0005000 msgr2=0x7facf000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7facfbfff700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0005000 0x7facf000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad08d27700 1 -- 172.21.15.92:0/2611601063 reap_dead start 2023-05-16T07:02:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7facfbfff700 1 -- 172.21.15.92:0/2611601063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00033f0 con 0x7facf0004690 2023-05-16T07:02:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7facfbfff700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf0004690 0x7facf0004b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7facec001730 tx=0x7facec007bb0).ready entity=mon.0 client_cookie=d8065801b0e2680 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec007020 con 0x7facf0004690 2023-05-16T07:02:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec007180 con 0x7facf0004690 2023-05-16T07:02:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad037fe700 1 -- 172.21.15.92:0/2611601063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec007520 con 0x7facf0004690 2023-05-16T07:02:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 -- 172.21.15.92:0/2611601063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf0004690 msgr2=0x7facf0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf0004690 0x7facf0004b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7facec001730 tx=0x7facec007bb0).stop 2023-05-16T07:02:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 -- 172.21.15.92:0/2611601063 shutdown_connections 2023-05-16T07:02:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2611601063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf0004690 0x7facf0004b10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 -- 172.21.15.92:0/2611601063 >> 172.21.15.92:0/2611601063 conn(0x7fad040c8a90 msgr2=0x7fad040c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 -- 172.21.15.92:0/2611601063 shutdown_connections 2023-05-16T07:02:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.643+0000 7fad0a78a700 1 -- 172.21.15.92:0/2611601063 wait complete. 2023-05-16T07:02:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.644+0000 7fad0a78a700 1 Processor -- start 2023-05-16T07:02:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 -- start start 2023-05-16T07:02:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0004690 0x7fad040c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad04155bf0 0x7fad0414a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 0x7fad040bda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad040c3ff0 con 0x7fad040ac780 2023-05-16T07:02:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad040c3560 con 0x7facf0004690 2023-05-16T07:02:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad0a78a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad04151280 con 0x7fad04155bf0 2023-05-16T07:02:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad08d27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 0x7fad040bda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad08d27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 0x7fad040bda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40812/0 (socket says 172.21.15.92:40812) 2023-05-16T07:02:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad08d27700 1 -- 172.21.15.92:0/2403838098 learned_addr learned my addr 172.21.15.92:0/2403838098 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7fad03fff700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0004690 0x7fad040c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.645+0000 7facfbfff700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad04155bf0 0x7fad0414a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad08d27700 1 -- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad04155bf0 msgr2=0x7fad0414a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad08d27700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad04155bf0 0x7fad0414a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad08d27700 1 -- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0004690 msgr2=0x7fad040c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad08d27700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0004690 0x7fad040c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad08d27700 1 -- 172.21.15.92:0/2403838098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04157b50 con 0x7fad040ac780 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad08d27700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 0x7fad040bda80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7facfc009d70 tx=0x7facfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc00e040 con 0x7fad040ac780 2023-05-16T07:02:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc00f040 con 0x7fad040ac780 2023-05-16T07:02:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad040c3750 con 0x7fad040ac780 2023-05-16T07:02:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc00fd40 con 0x7fad040ac780 2023-05-16T07:02:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.646+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad040c3d10 con 0x7fad040ac780 2023-05-16T07:02:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.647+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad04157ce0 con 0x7fad040ac780 2023-05-16T07:02:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.647+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7facfc015660 con 0x7fad040ac780 2023-05-16T07:02:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.647+0000 7fad00ff9700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7face405ed90 0x7face4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.647+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7facfc07a6f0 con 0x7fad040ac780 2023-05-16T07:02:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.648+0000 7fad03fff700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7face405ed90 0x7face4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.648+0000 7fad03fff700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7face405ed90 0x7face4061250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7facf4001050 tx=0x7facf4001e50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.651+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7facfc04bc90 con 0x7fad040ac780 2023-05-16T07:02:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.815+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fad0406c350 con 0x7fad040ac780 2023-05-16T07:02:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.815+0000 7fad00ff9700 1 -- 172.21.15.92:0/2403838098 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7facfc00fea0 con 0x7fad040ac780 2023-05-16T07:02:34.819 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7face405ed90 msgr2=0x7face4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7face405ed90 0x7face4061250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7facf4001050 tx=0x7facf4001e50).stop 2023-05-16T07:02:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 msgr2=0x7fad040bda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 0x7fad040bda80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7facfc009d70 tx=0x7facfc00b040).stop 2023-05-16T07:02:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 shutdown_connections 2023-05-16T07:02:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7face405ed90 0x7face4061250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad040ac780 0x7fad040bda80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facf0004690 0x7fad040c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 --2- 172.21.15.92:0/2403838098 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad04155bf0 0x7fad0414a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 >> 172.21.15.92:0/2403838098 conn(0x7fad040c8a90 msgr2=0x7fad0414c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 shutdown_connections 2023-05-16T07:02:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.817+0000 7fad0a78a700 1 -- 172.21.15.92:0/2403838098 wait complete. 2023-05-16T07:02:34.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2023-05-16T07:02:34.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-05-16T07:02:34.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:02:34.928 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:02:34.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.942+0000 7fb342f5e700 1 Processor -- start 2023-05-16T07:02:34.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.942+0000 7fb342f5e700 1 -- start start 2023-05-16T07:02:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb342f5e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154170 0x7fb33c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb342f5e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb33c069db0 0x7fb33c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb342f5e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c0c0660 con 0x7fb33c154550 2023-05-16T07:02:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb342f5e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0c1b00 con 0x7fb33c069db0 2023-05-16T07:02:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb342f5e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0bfd30 con 0x7fb33c154170 2023-05-16T07:02:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb33c069db0 0x7fb33c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb33c069db0 0x7fb33c0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40820/0 (socket says 172.21.15.92:40820) 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 -- 172.21.15.92:0/254394216 learned_addr learned my addr 172.21.15.92:0/254394216 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb340cfa700 1 --2- 172.21.15.92:0/254394216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154170 0x7fb33c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 -- 172.21.15.92:0/254394216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154170 msgr2=0x7fb33c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 --2- 172.21.15.92:0/254394216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154170 0x7fb33c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 -- 172.21.15.92:0/254394216 >> v1:172.21.15.158:6789/0 conn(0x7fb33c154550 legacy=0x7fb33c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.943+0000 7fb3414fb700 1 -- 172.21.15.92:0/254394216 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0c0010 con 0x7fb33c069db0 2023-05-16T07:02:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb3414fb700 1 --2- 172.21.15.92:0/254394216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c069db0 0x7fb33c0c6160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb334009f20 tx=0x7fb334009840).ready entity=mon.0 client_cookie=196fff2a29f2dd88 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb33b7fe700 1 -- 172.21.15.92:0/254394216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400e070 con 0x7fb33c069db0 2023-05-16T07:02:34.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb33b7fe700 1 -- 172.21.15.92:0/254394216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33400b9d0 con 0x7fb33c069db0 2023-05-16T07:02:34.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb33b7fe700 1 -- 172.21.15.92:0/254394216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400bd50 con 0x7fb33c069db0 2023-05-16T07:02:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 -- 172.21.15.92:0/254394216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c069db0 msgr2=0x7fb33c0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 --2- 172.21.15.92:0/254394216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c069db0 0x7fb33c0c6160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb334009f20 tx=0x7fb334009840).stop 2023-05-16T07:02:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 -- 172.21.15.92:0/254394216 shutdown_connections 2023-05-16T07:02:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 --2- 172.21.15.92:0/254394216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c069db0 0x7fb33c0c6160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 --2- 172.21.15.92:0/254394216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154170 0x7fb33c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 -- 172.21.15.92:0/254394216 >> 172.21.15.92:0/254394216 conn(0x7fb33c0cb700 msgr2=0x7fb33c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 -- 172.21.15.92:0/254394216 shutdown_connections 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.944+0000 7fb342f5e700 1 -- 172.21.15.92:0/254394216 wait complete. 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.946+0000 7fb342f5e700 1 Processor -- start 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.946+0000 7fb342f5e700 1 -- start start 2023-05-16T07:02:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb342f5e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c069db0 0x7fb33c1648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb342f5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 0x7fb33c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb342f5e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154550 0x7fb33c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb342f5e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c157d90 con 0x7fb33c154170 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb342f5e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0c30f0 con 0x7fb33c069db0 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb342f5e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1574f0 con 0x7fb33c154550 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 0x7fb33c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 0x7fb33c156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40824/0 (socket says 172.21.15.92:40824) 2023-05-16T07:02:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 -- 172.21.15.92:0/3698747870 learned_addr learned my addr 172.21.15.92:0/3698747870 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb340cfa700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c069db0 0x7fb33c1648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb3414fb700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154550 0x7fb33c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 -- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154550 msgr2=0x7fb33c156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154550 0x7fb33c156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 -- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c069db0 msgr2=0x7fb33c1648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.947+0000 7fb33bfff700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c069db0 0x7fb33c1648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb33bfff700 1 -- 172.21.15.92:0/3698747870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb334009ae0 con 0x7fb33c154170 2023-05-16T07:02:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb33bfff700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 0x7fb33c156950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fb33c154930 tx=0x7fb328007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280123f0 con 0x7fb33c154170 2023-05-16T07:02:34.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328012550 con 0x7fb33c154170 2023-05-16T07:02:34.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c1577d0 con 0x7fb33c154170 2023-05-16T07:02:34.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280128d0 con 0x7fb33c154170 2023-05-16T07:02:34.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.948+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c164240 con 0x7fb33c154170 2023-05-16T07:02:34.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.949+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c0666c0 con 0x7fb33c154170 2023-05-16T07:02:34.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.949+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb328010070 con 0x7fb33c154170 2023-05-16T07:02:34.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.949+0000 7fb338ff9700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb32005ed90 0x7fb320061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:34.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.949+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb328078d70 con 0x7fb33c154170 2023-05-16T07:02:34.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.950+0000 7fb340cfa700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb32005ed90 0x7fb320061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:34.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.950+0000 7fb340cfa700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb32005ed90 0x7fb320061250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fb33c156740 tx=0x7fb330009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:34.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:34.953+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb3280478f0 con 0x7fb33c154170 2023-05-16T07:02:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.117+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb33c0c2ea0 con 0x7fb33c154170 2023-05-16T07:02:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.117+0000 7fb338ff9700 1 -- 172.21.15.92:0/3698747870 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb32804a310 con 0x7fb33c154170 2023-05-16T07:02:35.122 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:35.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb32005ed90 msgr2=0x7fb320061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb32005ed90 0x7fb320061250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fb33c156740 tx=0x7fb330009060).stop 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 msgr2=0x7fb33c156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 0x7fb33c156950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fb33c154930 tx=0x7fb328007d20).stop 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 shutdown_connections 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb32005ed90 0x7fb320061250 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c154550 0x7fb33c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c069db0 0x7fb33c1648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 --2- 172.21.15.92:0/3698747870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c154170 0x7fb33c156950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 >> 172.21.15.92:0/3698747870 conn(0x7fb33c0cb700 msgr2=0x7fb33c14d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:35.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.119+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 shutdown_connections 2023-05-16T07:02:35.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.120+0000 7fb342f5e700 1 -- 172.21.15.92:0/3698747870 wait complete. 2023-05-16T07:02:35.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2023-05-16T07:02:35.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-05-16T07:02:35.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.244+0000 7f4bfa701700 1 Processor -- start 2023-05-16T07:02:35.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.244+0000 7f4bfa701700 1 -- start start 2023-05-16T07:02:35.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bfa701700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 0x7f4bf415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bfa701700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40c0660 con 0x7f4bf4154170 2023-05-16T07:02:35.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bfa701700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40c1b00 con 0x7f4bf4154550 2023-05-16T07:02:35.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bfa701700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40bfd30 con 0x7f4bf4069db0 2023-05-16T07:02:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4bf4154170 0x7f4bf40c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51578/0 (socket says 172.21.15.92:51578) 2023-05-16T07:02:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf3fff700 1 -- 172.21.15.92:0/2871327113 learned_addr learned my addr 172.21.15.92:0/2871327113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118418924 0 0) 0x7f4bf40c0660 con 0x7f4bf4154170 2023-05-16T07:02:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf8c9e700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 0x7f4bf415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bdc0033f0 con 0x7f4bf4154170 2023-05-16T07:02:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2937106788 0 0) 0x7f4bdc0033f0 con 0x7f4bf4154170 2023-05-16T07:02:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 msgr2=0x7f4bf415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 0x7f4bf415f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 >> v1:172.21.15.158:6789/0 conn(0x7f4bf4154550 legacy=0x7f4bf40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40c0010 con 0x7f4bf4154170 2023-05-16T07:02:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4be4001470 con 0x7f4bf4154170 2023-05-16T07:02:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 >> v1:172.21.15.92:6789/0 conn(0x7f4bf4154170 legacy=0x7f4bf40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 0x7f4bdc003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bdc003ef0 0x7f4bdc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc004b80 0x7f4bdc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.245+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40bfd30 con 0x7f4bdc004b80 2023-05-16T07:02:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc00bb20 con 0x7f4bdc003ef0 2023-05-16T07:02:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc00bce0 con 0x7f4bf4069db0 2023-05-16T07:02:35.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc004b80 0x7f4bdc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf3fff700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 0x7f4bdc003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf37fe700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bdc003ef0 0x7f4bdc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 -- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 msgr2=0x7f4bdc003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf4069db0 0x7f4bdc003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 -- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bdc003ef0 msgr2=0x7f4bdc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bdc003ef0 0x7f4bdc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 -- 172.21.15.92:0/2871327113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0033f0 con 0x7f4bdc004b80 2023-05-16T07:02:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf8c9e700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc004b80 0x7f4bdc005000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4bec009fd0 tx=0x7f4bec009710).ready entity=mon.0 client_cookie=d893428ff5a45c44 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec009a40 con 0x7f4bdc004b80 2023-05-16T07:02:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec007cc0 con 0x7f4bdc004b80 2023-05-16T07:02:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.246+0000 7f4bf2ffd700 1 -- 172.21.15.92:0/2871327113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec015620 con 0x7f4bdc004b80 2023-05-16T07:02:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bfa701700 1 -- 172.21.15.92:0/2871327113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc004b80 msgr2=0x7f4bdc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bfa701700 1 --2- 172.21.15.92:0/2871327113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc004b80 0x7f4bdc005000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4bec009fd0 tx=0x7f4bec009710).stop 2023-05-16T07:02:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bf8c9e700 1 -- 172.21.15.92:0/2871327113 reap_dead start 2023-05-16T07:02:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bfa701700 1 -- 172.21.15.92:0/2871327113 shutdown_connections 2023-05-16T07:02:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bfa701700 1 -- 172.21.15.92:0/2871327113 >> 172.21.15.92:0/2871327113 conn(0x7f4bf40cb700 msgr2=0x7f4bf40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bfa701700 1 -- 172.21.15.92:0/2871327113 shutdown_connections 2023-05-16T07:02:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.247+0000 7f4bfa701700 1 -- 172.21.15.92:0/2871327113 wait complete. 2023-05-16T07:02:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.248+0000 7f4bfa701700 1 Processor -- start 2023-05-16T07:02:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.248+0000 7f4bfa701700 1 -- start start 2023-05-16T07:02:35.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 0x7f4bf40af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf40c1d40 0x7f4bf4156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf4152440 0x7f4bf41528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4159b10 con 0x7f4bf40aede0 2023-05-16T07:02:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40a9da0 con 0x7f4bf4152440 2023-05-16T07:02:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf41594c0 con 0x7f4bf40c1d40 2023-05-16T07:02:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 0x7f4bf40af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 0x7f4bf40af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40842/0 (socket says 172.21.15.92:40842) 2023-05-16T07:02:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 -- 172.21.15.92:0/694234851 learned_addr learned my addr 172.21.15.92:0/694234851 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf8c9e700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf4152440 0x7f4bf41528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf37fe700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf40c1d40 0x7f4bf4156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 -- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf40c1d40 msgr2=0x7f4bf4156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf40c1d40 0x7f4bf4156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 -- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf4152440 msgr2=0x7f4bf41528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf4152440 0x7f4bf41528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 -- 172.21.15.92:0/694234851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf41597a0 con 0x7f4bf40aede0 2023-05-16T07:02:35.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bf3fff700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 0x7f4bf40af1c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f4be40047f0 tx=0x7f4be4001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4000fa0 con 0x7f4bf40aede0 2023-05-16T07:02:35.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4011590 con 0x7f4bf40aede0 2023-05-16T07:02:35.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf4152d90 con 0x7f4bf40aede0 2023-05-16T07:02:35.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4011910 con 0x7f4bf40aede0 2023-05-16T07:02:35.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.249+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf4156fb0 con 0x7f4bf40aede0 2023-05-16T07:02:35.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.250+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf40666c0 con 0x7f4bf40aede0 2023-05-16T07:02:35.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.251+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4be401b030 con 0x7f4bf40aede0 2023-05-16T07:02:35.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.251+0000 7f4bdbfff700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be005edc0 0x7f4be0061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.251+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4be40793e0 con 0x7f4bf40aede0 2023-05-16T07:02:35.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.251+0000 7f4bf37fe700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be005edc0 0x7f4be0061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.251+0000 7f4bf37fe700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be005edc0 0x7f4be0061280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4be8000f60 tx=0x7f4be8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.255+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4be404e3b0 con 0x7f4bf40aede0 2023-05-16T07:02:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.420+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4bf406c350 con 0x7f4bf40aede0 2023-05-16T07:02:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.420+0000 7f4bdbfff700 1 -- 172.21.15.92:0/694234851 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4be4019030 con 0x7f4bf40aede0 2023-05-16T07:02:35.424 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-05-16T07:02:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be005edc0 msgr2=0x7f4be0061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be005edc0 0x7f4be0061280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4be8000f60 tx=0x7f4be8009040).stop 2023-05-16T07:02:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 msgr2=0x7f4bf40af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 0x7f4bf40af1c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f4be40047f0 tx=0x7f4be4001e50).stop 2023-05-16T07:02:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 shutdown_connections 2023-05-16T07:02:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be005edc0 0x7f4be0061280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf4152440 0x7f4bf41528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf40aede0 0x7f4bf40af1c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 --2- 172.21.15.92:0/694234851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf40c1d40 0x7f4bf4156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 >> 172.21.15.92:0/694234851 conn(0x7f4bf40cb700 msgr2=0x7f4bf40c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:35.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 shutdown_connections 2023-05-16T07:02:35.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.422+0000 7f4bfa701700 1 -- 172.21.15.92:0/694234851 wait complete. 2023-05-16T07:02:35.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2023-05-16T07:02:35.438 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-05-16T07:02:35.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:02:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.547+0000 7fe08057b700 1 Processor -- start 2023-05-16T07:02:35.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.547+0000 7fe08057b700 1 -- start start 2023-05-16T07:02:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe08057b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe078154170 0x7fe0780c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe08057b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe078154550 0x7fe0780c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe08057b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0780696a0 0x7fe078069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe08057b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780c0660 con 0x7fe078154550 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe08057b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07806a020 con 0x7fe078154170 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe08057b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780bfd30 con 0x7fe0780696a0 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe07db16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe078154550 0x7fe0780c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe07db16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe078154550 0x7fe0780c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40852/0 (socket says 172.21.15.92:40852) 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe07db16700 1 -- 172.21.15.92:0/732594162 learned_addr learned my addr 172.21.15.92:0/732594162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:35.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.548+0000 7fe07e317700 1 --2- 172.21.15.92:0/732594162 >> v2:172.21.15.158:3300/0 conn(0x7fe078154170 0x7fe0780c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07eb18700 1 --2- 172.21.15.92:0/732594162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0780696a0 0x7fe078069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07db16700 1 -- 172.21.15.92:0/732594162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0780696a0 msgr2=0x7fe078069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07db16700 1 --2- 172.21.15.92:0/732594162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0780696a0 0x7fe078069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07db16700 1 -- 172.21.15.92:0/732594162 >> v2:172.21.15.158:3300/0 conn(0x7fe078154170 msgr2=0x7fe0780c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07db16700 1 --2- 172.21.15.92:0/732594162 >> v2:172.21.15.158:3300/0 conn(0x7fe078154170 0x7fe0780c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07db16700 1 -- 172.21.15.92:0/732594162 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780c0010 con 0x7fe078154550 2023-05-16T07:02:35.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07db16700 1 --2- 172.21.15.92:0/732594162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe078154550 0x7fe0780c6160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe078083030 tx=0x7fe0680090f0).ready entity=mon.0 client_cookie=699108fcbb878304 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07d315700 1 -- 172.21.15.92:0/732594162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06800b600 con 0x7fe078154550 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07d315700 1 -- 172.21.15.92:0/732594162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06800b760 con 0x7fe078154550 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe07d315700 1 -- 172.21.15.92:0/732594162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06800bae0 con 0x7fe078154550 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 -- 172.21.15.92:0/732594162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe078154550 msgr2=0x7fe0780c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 --2- 172.21.15.92:0/732594162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe078154550 0x7fe0780c6160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe078083030 tx=0x7fe0680090f0).stop 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 -- 172.21.15.92:0/732594162 shutdown_connections 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 --2- 172.21.15.92:0/732594162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0780696a0 0x7fe078069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 --2- 172.21.15.92:0/732594162 >> v2:172.21.15.158:3300/0 conn(0x7fe078154170 0x7fe0780c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 --2- 172.21.15.92:0/732594162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe078154550 0x7fe0780c6160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 -- 172.21.15.92:0/732594162 >> 172.21.15.92:0/732594162 conn(0x7fe0780cb750 msgr2=0x7fe0780c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:35.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 -- 172.21.15.92:0/732594162 shutdown_connections 2023-05-16T07:02:35.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.549+0000 7fe08057b700 1 -- 172.21.15.92:0/732594162 wait complete. 2023-05-16T07:02:35.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.550+0000 7fe08057b700 1 Processor -- start 2023-05-16T07:02:35.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 -- start start 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 0x7fe07813c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe078154170 0x7fe0780c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe078154550 0x7fe078068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07813ccc0 con 0x7fe0780696a0 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe078158200 con 0x7fe078154550 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe08057b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0781575f0 con 0x7fe078154170 2023-05-16T07:02:35.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe07e317700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 0x7fe07813c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe07e317700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 0x7fe07813c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40856/0 (socket says 172.21.15.92:40856) 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe07e317700 1 -- 172.21.15.92:0/510533871 learned_addr learned my addr 172.21.15.92:0/510533871 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe07db16700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe078154170 0x7fe0780c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.551+0000 7fe07eb18700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe078154550 0x7fe078068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07e317700 1 -- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe078154170 msgr2=0x7fe0780c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07e317700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe078154170 0x7fe0780c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07e317700 1 -- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe078154550 msgr2=0x7fe078068090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07e317700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe078154550 0x7fe078068090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07e317700 1 -- 172.21.15.92:0/510533871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe068009370 con 0x7fe0780696a0 2023-05-16T07:02:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07db16700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe078154170 0x7fe0780c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe07e317700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 0x7fe07813c7d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe07400d890 tx=0x7fe074008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074011590 con 0x7fe0780696a0 2023-05-16T07:02:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0740116f0 con 0x7fe0780696a0 2023-05-16T07:02:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0781578d0 con 0x7fe0780696a0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074011a70 con 0x7fe0780696a0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.552+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe078157e90 con 0x7fe0780696a0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.553+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0781584e0 con 0x7fe0780696a0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.553+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe07401c460 con 0x7fe0780696a0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.554+0000 7fe06e7fc700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe05c05ed60 0x7fe05c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.554+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe0740799d0 con 0x7fe0780696a0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.554+0000 7fe07db16700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe05c05ed60 0x7fe05c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.554+0000 7fe07db16700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe05c05ed60 0x7fe05c061220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe078154930 tx=0x7fe068017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.558+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe07404af10 con 0x7fe0780696a0 2023-05-16T07:02:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.717+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0780666c0 con 0x7fe05c05ed60 2023-05-16T07:02:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.726+0000 7fe06e7fc700 1 -- 172.21.15.92:0/510533871 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336258 (secure 0 0 0) 0x7fe0780666c0 con 0x7fe05c05ed60 2023-05-16T07:02:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.729+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe05c05ed60 msgr2=0x7fe05c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe05c05ed60 0x7fe05c061220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe078154930 tx=0x7fe068017000).stop 2023-05-16T07:02:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 msgr2=0x7fe07813c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 0x7fe07813c7d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe07400d890 tx=0x7fe074008040).stop 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 shutdown_connections 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe05c05ed60 0x7fe05c061220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe078154550 0x7fe078068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0780696a0 0x7fe07813c7d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 --2- 172.21.15.92:0/510533871 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe078154170 0x7fe0780c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 >> 172.21.15.92:0/510533871 conn(0x7fe0780cb750 msgr2=0x7fe0780abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 shutdown_connections 2023-05-16T07:02:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.730+0000 7fe08057b700 1 -- 172.21.15.92:0/510533871 wait complete. 2023-05-16T07:02:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:02:35.745 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-05-16T07:02:35.454234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41536,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":754933184,"statfs":{"total":773094113280,"available":773051580416,"internally_reserved":0,"allocated":1900544,"data_stored":687308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002052"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272660+0000","last_change":"2023-05-16T07:02:26.266087+0000","last_active":"2023-05-16T07:02:26.272660+0000","last_peered":"2023-05-16T07:02:26.272660+0000","last_clean":"2023-05-16T07:02:26.272660+0000","last_became_active":"2023-05-16T07:02:26.265870+0000","last_became_peered":"2023-05-16T07:02:26.265870+0000","last_unstale":"2023-05-16T07:02:26.272660+0000","last_undegraded":"2023-05-16T07:02:26.272660+0000","last_fullsized":"2023-05-16T07:02:26.272660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260827+0000","last_change":"2023-05-16T07:02:25.902715+0000","last_active":"2023-05-16T07:02:26.260827+0000","last_peered":"2023-05-16T07:02:26.260827+0000","last_clean":"2023-05-16T07:02:26.260827+0000","last_became_active":"2023-05-16T07:02:25.902401+0000","last_became_peered":"2023-05-16T07:02:25.902401+0000","last_unstale":"2023-05-16T07:02:26.260827+0000","last_undegraded":"2023-05-16T07:02:26.260827+0000","last_fullsized":"2023-05-16T07:02:26.260827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270248+0000","last_change":"2023-05-16T07:02:26.269154+0000","last_active":"2023-05-16T07:02:26.270248+0000","last_peered":"2023-05-16T07:02:26.270248+0000","last_clean":"2023-05-16T07:02:26.270248+0000","last_became_active":"2023-05-16T07:02:26.268835+0000","last_became_peered":"2023-05-16T07:02:26.268835+0000","last_unstale":"2023-05-16T07:02:26.270248+0000","last_undegraded":"2023-05-16T07:02:26.270248+0000","last_fullsized":"2023-05-16T07:02:26.270248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248521+0000","last_change":"2023-05-16T07:02:25.899378+0000","last_active":"2023-05-16T07:02:26.248521+0000","last_peered":"2023-05-16T07:02:26.248521+0000","last_clean":"2023-05-16T07:02:26.248521+0000","last_became_active":"2023-05-16T07:02:25.899114+0000","last_became_peered":"2023-05-16T07:02:25.899114+0000","last_unstale":"2023-05-16T07:02:26.248521+0000","last_undegraded":"2023-05-16T07:02:26.248521+0000","last_fullsized":"2023-05-16T07:02:26.248521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.258180+0000","last_change":"2023-05-16T07:02:26.255359+0000","last_active":"2023-05-16T07:02:26.258180+0000","last_peered":"2023-05-16T07:02:26.258180+0000","last_clean":"2023-05-16T07:02:26.258180+0000","last_became_active":"2023-05-16T07:02:26.254336+0000","last_became_peered":"2023-05-16T07:02:26.254336+0000","last_unstale":"2023-05-16T07:02:26.258180+0000","last_undegraded":"2023-05-16T07:02:26.258180+0000","last_fullsized":"2023-05-16T07:02:26.258180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252195+0000","last_change":"2023-05-16T07:02:25.903697+0000","last_active":"2023-05-16T07:02:26.252195+0000","last_peered":"2023-05-16T07:02:26.252195+0000","last_clean":"2023-05-16T07:02:26.252195+0000","last_became_active":"2023-05-16T07:02:25.903431+0000","last_became_peered":"2023-05-16T07:02:25.903431+0000","last_unstale":"2023-05-16T07:02:26.252195+0000","last_undegraded":"2023-05-16T07:02:26.252195+0000","last_fullsized":"2023-05-16T07:02:26.252195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275541+0000","last_change":"2023-05-16T07:02:26.272040+0000","last_active":"2023-05-16T07:02:26.275541+0000","last_peered":"2023-05-16T07:02:26.275541+0000","last_clean":"2023-05-16T07:02:26.275541+0000","last_became_active":"2023-05-16T07:02:26.271774+0000","last_became_peered":"2023-05-16T07:02:26.271774+0000","last_unstale":"2023-05-16T07:02:26.275541+0000","last_undegraded":"2023-05-16T07:02:26.275541+0000","last_fullsized":"2023-05-16T07:02:26.275541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261304+0000","last_change":"2023-05-16T07:02:25.899639+0000","last_active":"2023-05-16T07:02:26.261304+0000","last_peered":"2023-05-16T07:02:26.261304+0000","last_clean":"2023-05-16T07:02:26.261304+0000","last_became_active":"2023-05-16T07:02:25.899189+0000","last_became_peered":"2023-05-16T07:02:25.899189+0000","last_unstale":"2023-05-16T07:02:26.261304+0000","last_undegraded":"2023-05-16T07:02:26.261304+0000","last_fullsized":"2023-05-16T07:02:26.261304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265676+0000","last_change":"2023-05-16T07:02:26.263614+0000","last_active":"2023-05-16T07:02:26.265676+0000","last_peered":"2023-05-16T07:02:26.265676+0000","last_clean":"2023-05-16T07:02:26.265676+0000","last_became_active":"2023-05-16T07:02:26.263333+0000","last_became_peered":"2023-05-16T07:02:26.263333+0000","last_unstale":"2023-05-16T07:02:26.265676+0000","last_undegraded":"2023-05-16T07:02:26.265676+0000","last_fullsized":"2023-05-16T07:02:26.265676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252139+0000","last_change":"2023-05-16T07:02:25.902127+0000","last_active":"2023-05-16T07:02:26.252139+0000","last_peered":"2023-05-16T07:02:26.252139+0000","last_clean":"2023-05-16T07:02:26.252139+0000","last_became_active":"2023-05-16T07:02:25.901841+0000","last_became_peered":"2023-05-16T07:02:25.901841+0000","last_unstale":"2023-05-16T07:02:26.252139+0000","last_undegraded":"2023-05-16T07:02:26.252139+0000","last_fullsized":"2023-05-16T07:02:26.252139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273327+0000","last_change":"2023-05-16T07:02:26.266427+0000","last_active":"2023-05-16T07:02:26.273327+0000","last_peered":"2023-05-16T07:02:26.273327+0000","last_clean":"2023-05-16T07:02:26.273327+0000","last_became_active":"2023-05-16T07:02:26.265962+0000","last_became_peered":"2023-05-16T07:02:26.265962+0000","last_unstale":"2023-05-16T07:02:26.273327+0000","last_undegraded":"2023-05-16T07:02:26.273327+0000","last_fullsized":"2023-05-16T07:02:26.273327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270470+0000","last_change":"2023-05-16T07:02:25.906258+0000","last_active":"2023-05-16T07:02:26.270470+0000","last_peered":"2023-05-16T07:02:26.270470+0000","last_clean":"2023-05-16T07:02:26.270470+0000","last_became_active":"2023-05-16T07:02:25.905996+0000","last_became_peered":"2023-05-16T07:02:25.905996+0000","last_unstale":"2023-05-16T07:02:26.270470+0000","last_undegraded":"2023-05-16T07:02:26.270470+0000","last_fullsized":"2023-05-16T07:02:26.270470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273736+0000","last_change":"2023-05-16T07:02:26.269375+0000","last_active":"2023-05-16T07:02:26.273736+0000","last_peered":"2023-05-16T07:02:26.273736+0000","last_clean":"2023-05-16T07:02:26.273736+0000","last_became_active":"2023-05-16T07:02:26.269086+0000","last_became_peered":"2023-05-16T07:02:26.269086+0000","last_unstale":"2023-05-16T07:02:26.273736+0000","last_undegraded":"2023-05-16T07:02:26.273736+0000","last_fullsized":"2023-05-16T07:02:26.273736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275465+0000","last_change":"2023-05-16T07:02:25.903795+0000","last_active":"2023-05-16T07:02:26.275465+0000","last_peered":"2023-05-16T07:02:26.275465+0000","last_clean":"2023-05-16T07:02:26.275465+0000","last_became_active":"2023-05-16T07:02:25.903251+0000","last_became_peered":"2023-05-16T07:02:25.903251+0000","last_unstale":"2023-05-16T07:02:26.275465+0000","last_undegraded":"2023-05-16T07:02:26.275465+0000","last_fullsized":"2023-05-16T07:02:26.275465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272707+0000","last_change":"2023-05-16T07:02:26.266963+0000","last_active":"2023-05-16T07:02:26.272707+0000","last_peered":"2023-05-16T07:02:26.272707+0000","last_clean":"2023-05-16T07:02:26.272707+0000","last_became_active":"2023-05-16T07:02:26.266311+0000","last_became_peered":"2023-05-16T07:02:26.266311+0000","last_unstale":"2023-05-16T07:02:26.272707+0000","last_undegraded":"2023-05-16T07:02:26.272707+0000","last_fullsized":"2023-05-16T07:02:26.272707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272749+0000","last_change":"2023-05-16T07:02:25.909821+0000","last_active":"2023-05-16T07:02:26.272749+0000","last_peered":"2023-05-16T07:02:26.272749+0000","last_clean":"2023-05-16T07:02:26.272749+0000","last_became_active":"2023-05-16T07:02:25.909518+0000","last_became_peered":"2023-05-16T07:02:25.909518+0000","last_unstale":"2023-05-16T07:02:26.272749+0000","last_undegraded":"2023-05-16T07:02:26.272749+0000","last_fullsized":"2023-05-16T07:02:26.272749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266094+0000","last_change":"2023-05-16T07:02:26.263808+0000","last_active":"2023-05-16T07:02:26.266094+0000","last_peered":"2023-05-16T07:02:26.266094+0000","last_clean":"2023-05-16T07:02:26.266094+0000","last_became_active":"2023-05-16T07:02:26.263527+0000","last_became_peered":"2023-05-16T07:02:26.263527+0000","last_unstale":"2023-05-16T07:02:26.266094+0000","last_undegraded":"2023-05-16T07:02:26.266094+0000","last_fullsized":"2023-05-16T07:02:26.266094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254202+0000","last_change":"2023-05-16T07:02:25.901918+0000","last_active":"2023-05-16T07:02:26.254202+0000","last_peered":"2023-05-16T07:02:26.254202+0000","last_clean":"2023-05-16T07:02:26.254202+0000","last_became_active":"2023-05-16T07:02:25.901641+0000","last_became_peered":"2023-05-16T07:02:25.901641+0000","last_unstale":"2023-05-16T07:02:26.254202+0000","last_undegraded":"2023-05-16T07:02:26.254202+0000","last_fullsized":"2023-05-16T07:02:26.254202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269556+0000","last_change":"2023-05-16T07:02:26.267982+0000","last_active":"2023-05-16T07:02:26.269556+0000","last_peered":"2023-05-16T07:02:26.269556+0000","last_clean":"2023-05-16T07:02:26.269556+0000","last_became_active":"2023-05-16T07:02:26.267703+0000","last_became_peered":"2023-05-16T07:02:26.267703+0000","last_unstale":"2023-05-16T07:02:26.269556+0000","last_undegraded":"2023-05-16T07:02:26.269556+0000","last_fullsized":"2023-05-16T07:02:26.269556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254313+0000","last_change":"2023-05-16T07:02:25.909768+0000","last_active":"2023-05-16T07:02:26.254313+0000","last_peered":"2023-05-16T07:02:26.254313+0000","last_clean":"2023-05-16T07:02:26.254313+0000","last_became_active":"2023-05-16T07:02:25.909522+0000","last_became_peered":"2023-05-16T07:02:25.909522+0000","last_unstale":"2023-05-16T07:02:26.254313+0000","last_undegraded":"2023-05-16T07:02:26.254313+0000","last_fullsized":"2023-05-16T07:02:26.254313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272806+0000","last_change":"2023-05-16T07:02:26.268669+0000","last_active":"2023-05-16T07:02:26.272806+0000","last_peered":"2023-05-16T07:02:26.272806+0000","last_clean":"2023-05-16T07:02:26.272806+0000","last_became_active":"2023-05-16T07:02:26.268378+0000","last_became_peered":"2023-05-16T07:02:26.268378+0000","last_unstale":"2023-05-16T07:02:26.272806+0000","last_undegraded":"2023-05-16T07:02:26.272806+0000","last_fullsized":"2023-05-16T07:02:26.272806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269854+0000","last_change":"2023-05-16T07:02:25.901955+0000","last_active":"2023-05-16T07:02:26.269854+0000","last_peered":"2023-05-16T07:02:26.269854+0000","last_clean":"2023-05-16T07:02:26.269854+0000","last_became_active":"2023-05-16T07:02:25.901688+0000","last_became_peered":"2023-05-16T07:02:25.901688+0000","last_unstale":"2023-05-16T07:02:26.269854+0000","last_undegraded":"2023-05-16T07:02:26.269854+0000","last_fullsized":"2023-05-16T07:02:26.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273371+0000","last_change":"2023-05-16T07:02:26.270760+0000","last_active":"2023-05-16T07:02:26.273371+0000","last_peered":"2023-05-16T07:02:26.273371+0000","last_clean":"2023-05-16T07:02:26.273371+0000","last_became_active":"2023-05-16T07:02:26.270396+0000","last_became_peered":"2023-05-16T07:02:26.270396+0000","last_unstale":"2023-05-16T07:02:26.273371+0000","last_undegraded":"2023-05-16T07:02:26.273371+0000","last_fullsized":"2023-05-16T07:02:26.273371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274528+0000","last_change":"2023-05-16T07:02:25.909903+0000","last_active":"2023-05-16T07:02:26.274528+0000","last_peered":"2023-05-16T07:02:26.274528+0000","last_clean":"2023-05-16T07:02:26.274528+0000","last_became_active":"2023-05-16T07:02:25.909569+0000","last_became_peered":"2023-05-16T07:02:25.909569+0000","last_unstale":"2023-05-16T07:02:26.274528+0000","last_undegraded":"2023-05-16T07:02:26.274528+0000","last_fullsized":"2023-05-16T07:02:26.274528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273702+0000","last_change":"2023-05-16T07:02:26.268910+0000","last_active":"2023-05-16T07:02:26.273702+0000","last_peered":"2023-05-16T07:02:26.273702+0000","last_clean":"2023-05-16T07:02:26.273702+0000","last_became_active":"2023-05-16T07:02:26.268594+0000","last_became_peered":"2023-05-16T07:02:26.268594+0000","last_unstale":"2023-05-16T07:02:26.273702+0000","last_undegraded":"2023-05-16T07:02:26.273702+0000","last_fullsized":"2023-05-16T07:02:26.273702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273618+0000","last_change":"2023-05-16T07:02:25.911781+0000","last_active":"2023-05-16T07:02:26.273618+0000","last_peered":"2023-05-16T07:02:26.273618+0000","last_clean":"2023-05-16T07:02:26.273618+0000","last_became_active":"2023-05-16T07:02:25.911476+0000","last_became_peered":"2023-05-16T07:02:25.911476+0000","last_unstale":"2023-05-16T07:02:26.273618+0000","last_undegraded":"2023-05-16T07:02:26.273618+0000","last_fullsized":"2023-05-16T07:02:26.273618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273691+0000","last_change":"2023-05-16T07:02:26.271378+0000","last_active":"2023-05-16T07:02:26.273691+0000","last_peered":"2023-05-16T07:02:26.273691+0000","last_clean":"2023-05-16T07:02:26.273691+0000","last_became_active":"2023-05-16T07:02:26.271109+0000","last_became_peered":"2023-05-16T07:02:26.271109+0000","last_unstale":"2023-05-16T07:02:26.273691+0000","last_undegraded":"2023-05-16T07:02:26.273691+0000","last_fullsized":"2023-05-16T07:02:26.273691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255189+0000","last_change":"2023-05-16T07:02:25.908805+0000","last_active":"2023-05-16T07:02:26.255189+0000","last_peered":"2023-05-16T07:02:26.255189+0000","last_clean":"2023-05-16T07:02:26.255189+0000","last_became_active":"2023-05-16T07:02:25.906682+0000","last_became_peered":"2023-05-16T07:02:25.906682+0000","last_unstale":"2023-05-16T07:02:26.255189+0000","last_undegraded":"2023-05-16T07:02:26.255189+0000","last_fullsized":"2023-05-16T07:02:26.255189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269698+0000","last_change":"2023-05-16T07:02:26.267973+0000","last_active":"2023-05-16T07:02:26.269698+0000","last_peered":"2023-05-16T07:02:26.269698+0000","last_clean":"2023-05-16T07:02:26.269698+0000","last_became_active":"2023-05-16T07:02:26.267715+0000","last_became_peered":"2023-05-16T07:02:26.267715+0000","last_unstale":"2023-05-16T07:02:26.269698+0000","last_undegraded":"2023-05-16T07:02:26.269698+0000","last_fullsized":"2023-05-16T07:02:26.269698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252005+0000","last_change":"2023-05-16T07:02:25.902602+0000","last_active":"2023-05-16T07:02:26.252005+0000","last_peered":"2023-05-16T07:02:26.252005+0000","last_clean":"2023-05-16T07:02:26.252005+0000","last_became_active":"2023-05-16T07:02:25.901915+0000","last_became_peered":"2023-05-16T07:02:25.901915+0000","last_unstale":"2023-05-16T07:02:26.252005+0000","last_undegraded":"2023-05-16T07:02:26.252005+0000","last_fullsized":"2023-05-16T07:02:26.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274675+0000","last_change":"2023-05-16T07:02:26.272831+0000","last_active":"2023-05-16T07:02:26.274675+0000","last_peered":"2023-05-16T07:02:26.274675+0000","last_clean":"2023-05-16T07:02:26.274675+0000","last_became_active":"2023-05-16T07:02:26.270437+0000","last_became_peered":"2023-05-16T07:02:26.270437+0000","last_unstale":"2023-05-16T07:02:26.274675+0000","last_undegraded":"2023-05-16T07:02:26.274675+0000","last_fullsized":"2023-05-16T07:02:26.274675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266792+0000","last_change":"2023-05-16T07:02:25.905337+0000","last_active":"2023-05-16T07:02:26.266792+0000","last_peered":"2023-05-16T07:02:26.266792+0000","last_clean":"2023-05-16T07:02:26.266792+0000","last_became_active":"2023-05-16T07:02:25.904994+0000","last_became_peered":"2023-05-16T07:02:25.904994+0000","last_unstale":"2023-05-16T07:02:26.266792+0000","last_undegraded":"2023-05-16T07:02:26.266792+0000","last_fullsized":"2023-05-16T07:02:26.266792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273663+0000","last_change":"2023-05-16T07:02:26.269107+0000","last_active":"2023-05-16T07:02:26.273663+0000","last_peered":"2023-05-16T07:02:26.273663+0000","last_clean":"2023-05-16T07:02:26.273663+0000","last_became_active":"2023-05-16T07:02:26.268131+0000","last_became_peered":"2023-05-16T07:02:26.268131+0000","last_unstale":"2023-05-16T07:02:26.273663+0000","last_undegraded":"2023-05-16T07:02:26.273663+0000","last_fullsized":"2023-05-16T07:02:26.273663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250472+0000","last_change":"2023-05-16T07:02:25.897709+0000","last_active":"2023-05-16T07:02:26.250472+0000","last_peered":"2023-05-16T07:02:26.250472+0000","last_clean":"2023-05-16T07:02:26.250472+0000","last_became_active":"2023-05-16T07:02:25.897436+0000","last_became_peered":"2023-05-16T07:02:25.897436+0000","last_unstale":"2023-05-16T07:02:26.250472+0000","last_undegraded":"2023-05-16T07:02:26.250472+0000","last_fullsized":"2023-05-16T07:02:26.250472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273849+0000","last_change":"2023-05-16T07:02:26.269529+0000","last_active":"2023-05-16T07:02:26.273849+0000","last_peered":"2023-05-16T07:02:26.273849+0000","last_clean":"2023-05-16T07:02:26.273849+0000","last_became_active":"2023-05-16T07:02:26.269233+0000","last_became_peered":"2023-05-16T07:02:26.269233+0000","last_unstale":"2023-05-16T07:02:26.273849+0000","last_undegraded":"2023-05-16T07:02:26.273849+0000","last_fullsized":"2023-05-16T07:02:26.273849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.257675+0000","last_change":"2023-05-16T07:02:25.901600+0000","last_active":"2023-05-16T07:02:26.257675+0000","last_peered":"2023-05-16T07:02:26.257675+0000","last_clean":"2023-05-16T07:02:26.257675+0000","last_became_active":"2023-05-16T07:02:25.901168+0000","last_became_peered":"2023-05-16T07:02:25.901168+0000","last_unstale":"2023-05-16T07:02:26.257675+0000","last_undegraded":"2023-05-16T07:02:26.257675+0000","last_fullsized":"2023-05-16T07:02:26.257675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269725+0000","last_change":"2023-05-16T07:02:26.263820+0000","last_active":"2023-05-16T07:02:26.269725+0000","last_peered":"2023-05-16T07:02:26.269725+0000","last_clean":"2023-05-16T07:02:26.269725+0000","last_became_active":"2023-05-16T07:02:26.263188+0000","last_became_peered":"2023-05-16T07:02:26.263188+0000","last_unstale":"2023-05-16T07:02:26.269725+0000","last_undegraded":"2023-05-16T07:02:26.269725+0000","last_fullsized":"2023-05-16T07:02:26.269725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252302+0000","last_change":"2023-05-16T07:02:25.903683+0000","last_active":"2023-05-16T07:02:26.252302+0000","last_peered":"2023-05-16T07:02:26.252302+0000","last_clean":"2023-05-16T07:02:26.252302+0000","last_became_active":"2023-05-16T07:02:25.903358+0000","last_became_peered":"2023-05-16T07:02:25.903358+0000","last_unstale":"2023-05-16T07:02:26.252302+0000","last_undegraded":"2023-05-16T07:02:26.252302+0000","last_fullsized":"2023-05-16T07:02:26.252302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248865+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:02:26.248865+0000","last_peered":"2023-05-16T07:02:26.248865+0000","last_clean":"2023-05-16T07:02:26.248865+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:02:26.248865+0000","last_undegraded":"2023-05-16T07:02:26.248865+0000","last_fullsized":"2023-05-16T07:02:26.248865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252187+0000","last_change":"2023-05-16T07:02:25.903947+0000","last_active":"2023-05-16T07:02:26.252187+0000","last_peered":"2023-05-16T07:02:26.252187+0000","last_clean":"2023-05-16T07:02:26.252187+0000","last_became_active":"2023-05-16T07:02:25.903647+0000","last_became_peered":"2023-05-16T07:02:25.903647+0000","last_unstale":"2023-05-16T07:02:26.252187+0000","last_undegraded":"2023-05-16T07:02:26.252187+0000","last_fullsized":"2023-05-16T07:02:26.252187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270805+0000","last_change":"2023-05-16T07:02:26.260004+0000","last_active":"2023-05-16T07:02:26.270805+0000","last_peered":"2023-05-16T07:02:26.270805+0000","last_clean":"2023-05-16T07:02:26.270805+0000","last_became_active":"2023-05-16T07:02:26.256118+0000","last_became_peered":"2023-05-16T07:02:26.256118+0000","last_unstale":"2023-05-16T07:02:26.270805+0000","last_undegraded":"2023-05-16T07:02:26.270805+0000","last_fullsized":"2023-05-16T07:02:26.270805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252201+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:02:26.252201+0000","last_peered":"2023-05-16T07:02:26.252201+0000","last_clean":"2023-05-16T07:02:26.252201+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:02:26.252201+0000","last_undegraded":"2023-05-16T07:02:26.252201+0000","last_fullsized":"2023-05-16T07:02:26.252201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260286+0000","last_change":"2023-05-16T07:02:25.902850+0000","last_active":"2023-05-16T07:02:26.260286+0000","last_peered":"2023-05-16T07:02:26.260286+0000","last_clean":"2023-05-16T07:02:26.260286+0000","last_became_active":"2023-05-16T07:02:25.902557+0000","last_became_peered":"2023-05-16T07:02:25.902557+0000","last_unstale":"2023-05-16T07:02:26.260286+0000","last_undegraded":"2023-05-16T07:02:26.260286+0000","last_fullsized":"2023-05-16T07:02:26.260286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274107+0000","last_change":"2023-05-16T07:02:26.265589+0000","last_active":"2023-05-16T07:02:26.274107+0000","last_peered":"2023-05-16T07:02:26.274107+0000","last_clean":"2023-05-16T07:02:26.274107+0000","last_became_active":"2023-05-16T07:02:26.264804+0000","last_became_peered":"2023-05-16T07:02:26.264804+0000","last_unstale":"2023-05-16T07:02:26.274107+0000","last_undegraded":"2023-05-16T07:02:26.274107+0000","last_fullsized":"2023-05-16T07:02:26.274107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269704+0000","last_change":"2023-05-16T07:02:26.263668+0000","last_active":"2023-05-16T07:02:26.269704+0000","last_peered":"2023-05-16T07:02:26.269704+0000","last_clean":"2023-05-16T07:02:26.269704+0000","last_became_active":"2023-05-16T07:02:26.262937+0000","last_became_peered":"2023-05-16T07:02:26.262937+0000","last_unstale":"2023-05-16T07:02:26.269704+0000","last_undegraded":"2023-05-16T07:02:26.269704+0000","last_fullsized":"2023-05-16T07:02:26.269704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252944+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:02:26.252944+0000","last_peered":"2023-05-16T07:02:26.252944+0000","last_clean":"2023-05-16T07:02:26.252944+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:02:26.252944+0000","last_undegraded":"2023-05-16T07:02:26.252944+0000","last_fullsized":"2023-05-16T07:02:26.252944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248559+0000","last_change":"2023-05-16T07:02:25.900932+0000","last_active":"2023-05-16T07:02:26.248559+0000","last_peered":"2023-05-16T07:02:26.248559+0000","last_clean":"2023-05-16T07:02:26.248559+0000","last_became_active":"2023-05-16T07:02:25.899982+0000","last_became_peered":"2023-05-16T07:02:25.899982+0000","last_unstale":"2023-05-16T07:02:26.248559+0000","last_undegraded":"2023-05-16T07:02:26.248559+0000","last_fullsized":"2023-05-16T07:02:26.248559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274431+0000","last_change":"2023-05-16T07:02:26.271648+0000","last_active":"2023-05-16T07:02:26.274431+0000","last_peered":"2023-05-16T07:02:26.274431+0000","last_clean":"2023-05-16T07:02:26.274431+0000","last_became_active":"2023-05-16T07:02:26.271391+0000","last_became_peered":"2023-05-16T07:02:26.271391+0000","last_unstale":"2023-05-16T07:02:26.274431+0000","last_undegraded":"2023-05-16T07:02:26.274431+0000","last_fullsized":"2023-05-16T07:02:26.274431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251974+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:02:26.251974+0000","last_peered":"2023-05-16T07:02:26.251974+0000","last_clean":"2023-05-16T07:02:26.251974+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:02:26.251974+0000","last_undegraded":"2023-05-16T07:02:26.251974+0000","last_fullsized":"2023-05-16T07:02:26.251974+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255213+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:02:26.255213+0000","last_peered":"2023-05-16T07:02:26.255213+0000","last_clean":"2023-05-16T07:02:26.255213+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:02:26.255213+0000","last_undegraded":"2023-05-16T07:02:26.255213+0000","last_fullsized":"2023-05-16T07:02:26.255213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251830+0000","last_change":"2023-05-16T07:02:25.906617+0000","last_active":"2023-05-16T07:02:26.251830+0000","last_peered":"2023-05-16T07:02:26.251830+0000","last_clean":"2023-05-16T07:02:26.251830+0000","last_became_active":"2023-05-16T07:02:25.906269+0000","last_became_peered":"2023-05-16T07:02:25.906269+0000","last_unstale":"2023-05-16T07:02:26.251830+0000","last_undegraded":"2023-05-16T07:02:26.251830+0000","last_fullsized":"2023-05-16T07:02:26.251830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274440+0000","last_change":"2023-05-16T07:02:26.273104+0000","last_active":"2023-05-16T07:02:26.274440+0000","last_peered":"2023-05-16T07:02:26.274440+0000","last_clean":"2023-05-16T07:02:26.274440+0000","last_became_active":"2023-05-16T07:02:26.272816+0000","last_became_peered":"2023-05-16T07:02:26.272816+0000","last_unstale":"2023-05-16T07:02:26.274440+0000","last_undegraded":"2023-05-16T07:02:26.274440+0000","last_fullsized":"2023-05-16T07:02:26.274440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.257669+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:02:26.257669+0000","last_peered":"2023-05-16T07:02:26.257669+0000","last_clean":"2023-05-16T07:02:26.257669+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:02:26.257669+0000","last_undegraded":"2023-05-16T07:02:26.257669+0000","last_fullsized":"2023-05-16T07:02:26.257669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251964+0000","last_change":"2023-05-16T07:02:25.902199+0000","last_active":"2023-05-16T07:02:26.251964+0000","last_peered":"2023-05-16T07:02:26.251964+0000","last_clean":"2023-05-16T07:02:26.251964+0000","last_became_active":"2023-05-16T07:02:25.901913+0000","last_became_peered":"2023-05-16T07:02:25.901913+0000","last_unstale":"2023-05-16T07:02:26.251964+0000","last_undegraded":"2023-05-16T07:02:26.251964+0000","last_fullsized":"2023-05-16T07:02:26.251964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255055+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:02:26.255055+0000","last_peered":"2023-05-16T07:02:26.255055+0000","last_clean":"2023-05-16T07:02:26.255055+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:02:26.255055+0000","last_undegraded":"2023-05-16T07:02:26.255055+0000","last_fullsized":"2023-05-16T07:02:26.255055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248963+0000","last_change":"2023-05-16T07:02:25.900748+0000","last_active":"2023-05-16T07:02:26.248963+0000","last_peered":"2023-05-16T07:02:26.248963+0000","last_clean":"2023-05-16T07:02:26.248963+0000","last_became_active":"2023-05-16T07:02:25.900481+0000","last_became_peered":"2023-05-16T07:02:25.900481+0000","last_unstale":"2023-05-16T07:02:26.248963+0000","last_undegraded":"2023-05-16T07:02:26.248963+0000","last_fullsized":"2023-05-16T07:02:26.248963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265039+0000","last_change":"2023-05-16T07:02:26.260513+0000","last_active":"2023-05-16T07:02:26.265039+0000","last_peered":"2023-05-16T07:02:26.265039+0000","last_clean":"2023-05-16T07:02:26.265039+0000","last_became_active":"2023-05-16T07:02:26.260265+0000","last_became_peered":"2023-05-16T07:02:26.260265+0000","last_unstale":"2023-05-16T07:02:26.265039+0000","last_undegraded":"2023-05-16T07:02:26.265039+0000","last_fullsized":"2023-05-16T07:02:26.265039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271487+0000","last_change":"2023-05-16T07:02:25.900313+0000","last_active":"2023-05-16T07:02:26.271487+0000","last_peered":"2023-05-16T07:02:26.271487+0000","last_clean":"2023-05-16T07:02:26.271487+0000","last_became_active":"2023-05-16T07:02:25.900056+0000","last_became_peered":"2023-05-16T07:02:25.900056+0000","last_unstale":"2023-05-16T07:02:26.271487+0000","last_undegraded":"2023-05-16T07:02:26.271487+0000","last_fullsized":"2023-05-16T07:02:26.271487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.249182+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:02:26.249182+0000","last_peered":"2023-05-16T07:02:26.249182+0000","last_clean":"2023-05-16T07:02:26.249182+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:02:26.249182+0000","last_undegraded":"2023-05-16T07:02:26.249182+0000","last_fullsized":"2023-05-16T07:02:26.249182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274618+0000","last_change":"2023-05-16T07:02:26.270645+0000","last_active":"2023-05-16T07:02:26.274618+0000","last_peered":"2023-05-16T07:02:26.274618+0000","last_clean":"2023-05-16T07:02:26.274618+0000","last_became_active":"2023-05-16T07:02:26.270312+0000","last_became_peered":"2023-05-16T07:02:26.270312+0000","last_unstale":"2023-05-16T07:02:26.274618+0000","last_undegraded":"2023-05-16T07:02:26.274618+0000","last_fullsized":"2023-05-16T07:02:26.274618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250955+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:02:26.250955+0000","last_peered":"2023-05-16T07:02:26.250955+0000","last_clean":"2023-05-16T07:02:26.250955+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:02:26.250955+0000","last_undegraded":"2023-05-16T07:02:26.250955+0000","last_fullsized":"2023-05-16T07:02:26.250955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.258392+0000","last_change":"2023-05-16T07:02:25.902709+0000","last_active":"2023-05-16T07:02:26.258392+0000","last_peered":"2023-05-16T07:02:26.258392+0000","last_clean":"2023-05-16T07:02:26.258392+0000","last_became_active":"2023-05-16T07:02:25.902421+0000","last_became_peered":"2023-05-16T07:02:25.902421+0000","last_unstale":"2023-05-16T07:02:26.258392+0000","last_undegraded":"2023-05-16T07:02:26.258392+0000","last_fullsized":"2023-05-16T07:02:26.258392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272520+0000","last_change":"2023-05-16T07:02:26.263603+0000","last_active":"2023-05-16T07:02:26.272520+0000","last_peered":"2023-05-16T07:02:26.272520+0000","last_clean":"2023-05-16T07:02:26.272520+0000","last_became_active":"2023-05-16T07:02:26.260959+0000","last_became_peered":"2023-05-16T07:02:26.260959+0000","last_unstale":"2023-05-16T07:02:26.272520+0000","last_undegraded":"2023-05-16T07:02:26.272520+0000","last_fullsized":"2023-05-16T07:02:26.272520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273598+0000","last_change":"2023-05-16T07:02:26.272233+0000","last_active":"2023-05-16T07:02:26.273598+0000","last_peered":"2023-05-16T07:02:26.273598+0000","last_clean":"2023-05-16T07:02:26.273598+0000","last_became_active":"2023-05-16T07:02:26.271964+0000","last_became_peered":"2023-05-16T07:02:26.271964+0000","last_unstale":"2023-05-16T07:02:26.273598+0000","last_undegraded":"2023-05-16T07:02:26.273598+0000","last_fullsized":"2023-05-16T07:02:26.273598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253633+0000","last_change":"2023-05-16T07:02:25.902264+0000","last_active":"2023-05-16T07:02:26.253633+0000","last_peered":"2023-05-16T07:02:26.253633+0000","last_clean":"2023-05-16T07:02:26.253633+0000","last_became_active":"2023-05-16T07:02:25.901192+0000","last_became_peered":"2023-05-16T07:02:25.901192+0000","last_unstale":"2023-05-16T07:02:26.253633+0000","last_undegraded":"2023-05-16T07:02:26.253633+0000","last_fullsized":"2023-05-16T07:02:26.253633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270652+0000","last_change":"2023-05-16T07:02:26.264892+0000","last_active":"2023-05-16T07:02:26.270652+0000","last_peered":"2023-05-16T07:02:26.270652+0000","last_clean":"2023-05-16T07:02:26.270652+0000","last_became_active":"2023-05-16T07:02:26.264659+0000","last_became_peered":"2023-05-16T07:02:26.264659+0000","last_unstale":"2023-05-16T07:02:26.270652+0000","last_undegraded":"2023-05-16T07:02:26.270652+0000","last_fullsized":"2023-05-16T07:02:26.270652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252103+0000","last_change":"2023-05-16T07:02:25.899001+0000","last_active":"2023-05-16T07:02:26.252103+0000","last_peered":"2023-05-16T07:02:26.252103+0000","last_clean":"2023-05-16T07:02:26.252103+0000","last_became_active":"2023-05-16T07:02:25.898724+0000","last_became_peered":"2023-05-16T07:02:25.898724+0000","last_unstale":"2023-05-16T07:02:26.252103+0000","last_undegraded":"2023-05-16T07:02:26.252103+0000","last_fullsized":"2023-05-16T07:02:26.252103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274782+0000","last_change":"2023-05-16T07:02:26.270764+0000","last_active":"2023-05-16T07:02:26.274782+0000","last_peered":"2023-05-16T07:02:26.274782+0000","last_clean":"2023-05-16T07:02:26.274782+0000","last_became_active":"2023-05-16T07:02:26.270171+0000","last_became_peered":"2023-05-16T07:02:26.270171+0000","last_unstale":"2023-05-16T07:02:26.274782+0000","last_undegraded":"2023-05-16T07:02:26.274782+0000","last_fullsized":"2023-05-16T07:02:26.274782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270540+0000","last_change":"2023-05-16T07:02:25.907500+0000","last_active":"2023-05-16T07:02:26.270540+0000","last_peered":"2023-05-16T07:02:26.270540+0000","last_clean":"2023-05-16T07:02:26.270540+0000","last_became_active":"2023-05-16T07:02:25.907202+0000","last_became_peered":"2023-05-16T07:02:25.907202+0000","last_unstale":"2023-05-16T07:02:26.270540+0000","last_undegraded":"2023-05-16T07:02:26.270540+0000","last_fullsized":"2023-05-16T07:02:26.270540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.264264+0000","last_change":"2023-05-16T07:02:26.262010+0000","last_active":"2023-05-16T07:02:26.264264+0000","last_peered":"2023-05-16T07:02:26.264264+0000","last_clean":"2023-05-16T07:02:26.264264+0000","last_became_active":"2023-05-16T07:02:26.261641+0000","last_became_peered":"2023-05-16T07:02:26.261641+0000","last_unstale":"2023-05-16T07:02:26.264264+0000","last_undegraded":"2023-05-16T07:02:26.264264+0000","last_fullsized":"2023-05-16T07:02:26.264264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266384+0000","last_change":"2023-05-16T07:02:25.900614+0000","last_active":"2023-05-16T07:02:26.266384+0000","last_peered":"2023-05-16T07:02:26.266384+0000","last_clean":"2023-05-16T07:02:26.266384+0000","last_became_active":"2023-05-16T07:02:25.898929+0000","last_became_peered":"2023-05-16T07:02:25.898929+0000","last_unstale":"2023-05-16T07:02:26.266384+0000","last_undegraded":"2023-05-16T07:02:26.266384+0000","last_fullsized":"2023-05-16T07:02:26.266384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273729+0000","last_change":"2023-05-16T07:02:26.269314+0000","last_active":"2023-05-16T07:02:26.273729+0000","last_peered":"2023-05-16T07:02:26.273729+0000","last_clean":"2023-05-16T07:02:26.273729+0000","last_became_active":"2023-05-16T07:02:26.269002+0000","last_became_peered":"2023-05-16T07:02:26.269002+0000","last_unstale":"2023-05-16T07:02:26.273729+0000","last_undegraded":"2023-05-16T07:02:26.273729+0000","last_fullsized":"2023-05-16T07:02:26.273729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.267696+0000","last_change":"2023-05-16T07:02:25.899453+0000","last_active":"2023-05-16T07:02:26.267696+0000","last_peered":"2023-05-16T07:02:26.267696+0000","last_clean":"2023-05-16T07:02:26.267696+0000","last_became_active":"2023-05-16T07:02:25.898915+0000","last_became_peered":"2023-05-16T07:02:25.898915+0000","last_unstale":"2023-05-16T07:02:26.267696+0000","last_undegraded":"2023-05-16T07:02:26.267696+0000","last_fullsized":"2023-05-16T07:02:26.267696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269745+0000","last_change":"2023-05-16T07:02:26.263984+0000","last_active":"2023-05-16T07:02:26.269745+0000","last_peered":"2023-05-16T07:02:26.269745+0000","last_clean":"2023-05-16T07:02:26.269745+0000","last_became_active":"2023-05-16T07:02:26.263440+0000","last_became_peered":"2023-05-16T07:02:26.263440+0000","last_unstale":"2023-05-16T07:02:26.269745+0000","last_undegraded":"2023-05-16T07:02:26.269745+0000","last_fullsized":"2023-05-16T07:02:26.269745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251262+0000","last_change":"2023-05-16T07:02:25.903155+0000","last_active":"2023-05-16T07:02:26.251262+0000","last_peered":"2023-05-16T07:02:26.251262+0000","last_clean":"2023-05-16T07:02:26.251262+0000","last_became_active":"2023-05-16T07:02:25.901362+0000","last_became_peered":"2023-05-16T07:02:25.901362+0000","last_unstale":"2023-05-16T07:02:26.251262+0000","last_undegraded":"2023-05-16T07:02:26.251262+0000","last_fullsized":"2023-05-16T07:02:26.251262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266261+0000","last_change":"2023-05-16T07:02:26.262016+0000","last_active":"2023-05-16T07:02:26.266261+0000","last_peered":"2023-05-16T07:02:26.266261+0000","last_clean":"2023-05-16T07:02:26.266261+0000","last_became_active":"2023-05-16T07:02:26.261803+0000","last_became_peered":"2023-05-16T07:02:26.261803+0000","last_unstale":"2023-05-16T07:02:26.266261+0000","last_undegraded":"2023-05-16T07:02:26.266261+0000","last_fullsized":"2023-05-16T07:02:26.266261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253045+0000","last_change":"2023-05-16T07:02:25.904392+0000","last_active":"2023-05-16T07:02:26.253045+0000","last_peered":"2023-05-16T07:02:26.253045+0000","last_clean":"2023-05-16T07:02:26.253045+0000","last_became_active":"2023-05-16T07:02:25.904025+0000","last_became_peered":"2023-05-16T07:02:25.904025+0000","last_unstale":"2023-05-16T07:02:26.253045+0000","last_undegraded":"2023-05-16T07:02:26.253045+0000","last_fullsized":"2023-05-16T07:02:26.253045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266065+0000","last_change":"2023-05-16T07:02:26.263661+0000","last_active":"2023-05-16T07:02:26.266065+0000","last_peered":"2023-05-16T07:02:26.266065+0000","last_clean":"2023-05-16T07:02:26.266065+0000","last_became_active":"2023-05-16T07:02:26.263328+0000","last_became_peered":"2023-05-16T07:02:26.263328+0000","last_unstale":"2023-05-16T07:02:26.266065+0000","last_undegraded":"2023-05-16T07:02:26.266065+0000","last_fullsized":"2023-05-16T07:02:26.266065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252146+0000","last_change":"2023-05-16T07:02:25.905914+0000","last_active":"2023-05-16T07:02:26.252146+0000","last_peered":"2023-05-16T07:02:26.252146+0000","last_clean":"2023-05-16T07:02:26.252146+0000","last_became_active":"2023-05-16T07:02:25.905585+0000","last_became_peered":"2023-05-16T07:02:25.905585+0000","last_unstale":"2023-05-16T07:02:26.252146+0000","last_undegraded":"2023-05-16T07:02:26.252146+0000","last_fullsized":"2023-05-16T07:02:26.252146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274654+0000","last_change":"2023-05-16T07:02:26.271450+0000","last_active":"2023-05-16T07:02:26.274654+0000","last_peered":"2023-05-16T07:02:26.274654+0000","last_clean":"2023-05-16T07:02:26.274654+0000","last_became_active":"2023-05-16T07:02:26.269701+0000","last_became_peered":"2023-05-16T07:02:26.269701+0000","last_unstale":"2023-05-16T07:02:26.274654+0000","last_undegraded":"2023-05-16T07:02:26.274654+0000","last_fullsized":"2023-05-16T07:02:26.274654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253701+0000","last_change":"2023-05-16T07:02:25.908650+0000","last_active":"2023-05-16T07:02:26.253701+0000","last_peered":"2023-05-16T07:02:26.253701+0000","last_clean":"2023-05-16T07:02:26.253701+0000","last_became_active":"2023-05-16T07:02:25.908385+0000","last_became_peered":"2023-05-16T07:02:25.908385+0000","last_unstale":"2023-05-16T07:02:26.253701+0000","last_undegraded":"2023-05-16T07:02:26.253701+0000","last_fullsized":"2023-05-16T07:02:26.253701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275550+0000","last_change":"2023-05-16T07:02:26.272118+0000","last_active":"2023-05-16T07:02:26.275550+0000","last_peered":"2023-05-16T07:02:26.275550+0000","last_clean":"2023-05-16T07:02:26.275550+0000","last_became_active":"2023-05-16T07:02:26.271865+0000","last_became_peered":"2023-05-16T07:02:26.271865+0000","last_unstale":"2023-05-16T07:02:26.275550+0000","last_undegraded":"2023-05-16T07:02:26.275550+0000","last_fullsized":"2023-05-16T07:02:26.275550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252765+0000","last_change":"2023-05-16T07:02:25.905216+0000","last_active":"2023-05-16T07:02:26.252765+0000","last_peered":"2023-05-16T07:02:26.252765+0000","last_clean":"2023-05-16T07:02:26.252765+0000","last_became_active":"2023-05-16T07:02:25.904954+0000","last_became_peered":"2023-05-16T07:02:25.904954+0000","last_unstale":"2023-05-16T07:02:26.252765+0000","last_undegraded":"2023-05-16T07:02:26.252765+0000","last_fullsized":"2023-05-16T07:02:26.252765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274803+0000","last_change":"2023-05-16T07:02:26.270884+0000","last_active":"2023-05-16T07:02:26.274803+0000","last_peered":"2023-05-16T07:02:26.274803+0000","last_clean":"2023-05-16T07:02:26.274803+0000","last_became_active":"2023-05-16T07:02:26.270473+0000","last_became_peered":"2023-05-16T07:02:26.270473+0000","last_unstale":"2023-05-16T07:02:26.274803+0000","last_undegraded":"2023-05-16T07:02:26.274803+0000","last_fullsized":"2023-05-16T07:02:26.274803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248720+0000","last_change":"2023-05-16T07:02:25.900138+0000","last_active":"2023-05-16T07:02:26.248720+0000","last_peered":"2023-05-16T07:02:26.248720+0000","last_clean":"2023-05-16T07:02:26.248720+0000","last_became_active":"2023-05-16T07:02:25.899831+0000","last_became_peered":"2023-05-16T07:02:25.899831+0000","last_unstale":"2023-05-16T07:02:26.248720+0000","last_undegraded":"2023-05-16T07:02:26.248720+0000","last_fullsized":"2023-05-16T07:02:26.248720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275578+0000","last_change":"2023-05-16T07:02:26.273190+0000","last_active":"2023-05-16T07:02:26.275578+0000","last_peered":"2023-05-16T07:02:26.275578+0000","last_clean":"2023-05-16T07:02:26.275578+0000","last_became_active":"2023-05-16T07:02:26.272912+0000","last_became_peered":"2023-05-16T07:02:26.272912+0000","last_unstale":"2023-05-16T07:02:26.275578+0000","last_undegraded":"2023-05-16T07:02:26.275578+0000","last_fullsized":"2023-05-16T07:02:26.275578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-05-16T07:02:27.253747+0000","last_change":"2023-05-16T07:02:25.902881+0000","last_active":"2023-05-16T07:02:27.253747+0000","last_peered":"2023-05-16T07:02:27.253747+0000","last_clean":"2023-05-16T07:02:27.253747+0000","last_became_active":"2023-05-16T07:02:25.902555+0000","last_became_peered":"2023-05-16T07:02:25.902555+0000","last_unstale":"2023-05-16T07:02:27.253747+0000","last_undegraded":"2023-05-16T07:02:27.253747+0000","last_fullsized":"2023-05-16T07:02:27.253747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266078+0000","last_change":"2023-05-16T07:02:26.262732+0000","last_active":"2023-05-16T07:02:26.266078+0000","last_peered":"2023-05-16T07:02:26.266078+0000","last_clean":"2023-05-16T07:02:26.266078+0000","last_became_active":"2023-05-16T07:02:26.258080+0000","last_became_peered":"2023-05-16T07:02:26.258080+0000","last_unstale":"2023-05-16T07:02:26.266078+0000","last_undegraded":"2023-05-16T07:02:26.266078+0000","last_fullsized":"2023-05-16T07:02:26.266078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253611+0000","last_change":"2023-05-16T07:02:25.899623+0000","last_active":"2023-05-16T07:02:26.253611+0000","last_peered":"2023-05-16T07:02:26.253611+0000","last_clean":"2023-05-16T07:02:26.253611+0000","last_became_active":"2023-05-16T07:02:25.899225+0000","last_became_peered":"2023-05-16T07:02:25.899225+0000","last_unstale":"2023-05-16T07:02:26.253611+0000","last_undegraded":"2023-05-16T07:02:26.253611+0000","last_fullsized":"2023-05-16T07:02:26.253611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269443+0000","last_change":"2023-05-16T07:02:26.267767+0000","last_active":"2023-05-16T07:02:26.269443+0000","last_peered":"2023-05-16T07:02:26.269443+0000","last_clean":"2023-05-16T07:02:26.269443+0000","last_became_active":"2023-05-16T07:02:26.267496+0000","last_became_peered":"2023-05-16T07:02:26.267496+0000","last_unstale":"2023-05-16T07:02:26.269443+0000","last_undegraded":"2023-05-16T07:02:26.269443+0000","last_fullsized":"2023-05-16T07:02:26.269443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251732+0000","last_change":"2023-05-16T07:02:25.905706+0000","last_active":"2023-05-16T07:02:26.251732+0000","last_peered":"2023-05-16T07:02:26.251732+0000","last_clean":"2023-05-16T07:02:26.251732+0000","last_became_active":"2023-05-16T07:02:25.905387+0000","last_became_peered":"2023-05-16T07:02:25.905387+0000","last_unstale":"2023-05-16T07:02:26.251732+0000","last_undegraded":"2023-05-16T07:02:26.251732+0000","last_fullsized":"2023-05-16T07:02:26.251732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273026+0000","last_change":"2023-05-16T07:02:26.264777+0000","last_active":"2023-05-16T07:02:26.273026+0000","last_peered":"2023-05-16T07:02:26.273026+0000","last_clean":"2023-05-16T07:02:26.273026+0000","last_became_active":"2023-05-16T07:02:26.264481+0000","last_became_peered":"2023-05-16T07:02:26.264481+0000","last_unstale":"2023-05-16T07:02:26.273026+0000","last_undegraded":"2023-05-16T07:02:26.273026+0000","last_fullsized":"2023-05-16T07:02:26.273026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252159+0000","last_change":"2023-05-16T07:02:25.907961+0000","last_active":"2023-05-16T07:02:26.252159+0000","last_peered":"2023-05-16T07:02:26.252159+0000","last_clean":"2023-05-16T07:02:26.252159+0000","last_became_active":"2023-05-16T07:02:25.907665+0000","last_became_peered":"2023-05-16T07:02:25.907665+0000","last_unstale":"2023-05-16T07:02:26.252159+0000","last_undegraded":"2023-05-16T07:02:26.252159+0000","last_fullsized":"2023-05-16T07:02:26.252159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261333+0000","last_change":"2023-05-16T07:02:26.257095+0000","last_active":"2023-05-16T07:02:26.261333+0000","last_peered":"2023-05-16T07:02:26.261333+0000","last_clean":"2023-05-16T07:02:26.261333+0000","last_became_active":"2023-05-16T07:02:26.256845+0000","last_became_peered":"2023-05-16T07:02:26.256845+0000","last_unstale":"2023-05-16T07:02:26.261333+0000","last_undegraded":"2023-05-16T07:02:26.261333+0000","last_fullsized":"2023-05-16T07:02:26.261333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252260+0000","last_change":"2023-05-16T07:02:25.907092+0000","last_active":"2023-05-16T07:02:26.252260+0000","last_peered":"2023-05-16T07:02:26.252260+0000","last_clean":"2023-05-16T07:02:26.252260+0000","last_became_active":"2023-05-16T07:02:25.906488+0000","last_became_peered":"2023-05-16T07:02:25.906488+0000","last_unstale":"2023-05-16T07:02:26.252260+0000","last_undegraded":"2023-05-16T07:02:26.252260+0000","last_fullsized":"2023-05-16T07:02:26.252260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274666+0000","last_change":"2023-05-16T07:02:26.272727+0000","last_active":"2023-05-16T07:02:26.274666+0000","last_peered":"2023-05-16T07:02:26.274666+0000","last_clean":"2023-05-16T07:02:26.274666+0000","last_became_active":"2023-05-16T07:02:26.270729+0000","last_became_peered":"2023-05-16T07:02:26.270729+0000","last_unstale":"2023-05-16T07:02:26.274666+0000","last_undegraded":"2023-05-16T07:02:26.274666+0000","last_fullsized":"2023-05-16T07:02:26.274666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270991+0000","last_change":"2023-05-16T07:02:25.902010+0000","last_active":"2023-05-16T07:02:26.270991+0000","last_peered":"2023-05-16T07:02:26.270991+0000","last_clean":"2023-05-16T07:02:26.270991+0000","last_became_active":"2023-05-16T07:02:25.900891+0000","last_became_peered":"2023-05-16T07:02:25.900891+0000","last_unstale":"2023-05-16T07:02:26.270991+0000","last_undegraded":"2023-05-16T07:02:26.270991+0000","last_fullsized":"2023-05-16T07:02:26.270991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271884+0000","last_change":"2023-05-16T07:02:26.268192+0000","last_active":"2023-05-16T07:02:26.271884+0000","last_peered":"2023-05-16T07:02:26.271884+0000","last_clean":"2023-05-16T07:02:26.271884+0000","last_became_active":"2023-05-16T07:02:26.263201+0000","last_became_peered":"2023-05-16T07:02:26.263201+0000","last_unstale":"2023-05-16T07:02:26.271884+0000","last_undegraded":"2023-05-16T07:02:26.271884+0000","last_fullsized":"2023-05-16T07:02:26.271884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250995+0000","last_change":"2023-05-16T07:02:25.899500+0000","last_active":"2023-05-16T07:02:26.250995+0000","last_peered":"2023-05-16T07:02:26.250995+0000","last_clean":"2023-05-16T07:02:26.250995+0000","last_became_active":"2023-05-16T07:02:25.898726+0000","last_became_peered":"2023-05-16T07:02:25.898726+0000","last_unstale":"2023-05-16T07:02:26.250995+0000","last_undegraded":"2023-05-16T07:02:26.250995+0000","last_fullsized":"2023-05-16T07:02:26.250995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274483+0000","last_change":"2023-05-16T07:02:26.273388+0000","last_active":"2023-05-16T07:02:26.274483+0000","last_peered":"2023-05-16T07:02:26.274483+0000","last_clean":"2023-05-16T07:02:26.274483+0000","last_became_active":"2023-05-16T07:02:26.273112+0000","last_became_peered":"2023-05-16T07:02:26.273112+0000","last_unstale":"2023-05-16T07:02:26.274483+0000","last_undegraded":"2023-05-16T07:02:26.274483+0000","last_fullsized":"2023-05-16T07:02:26.274483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270551+0000","last_change":"2023-05-16T07:02:25.905564+0000","last_active":"2023-05-16T07:02:26.270551+0000","last_peered":"2023-05-16T07:02:26.270551+0000","last_clean":"2023-05-16T07:02:26.270551+0000","last_became_active":"2023-05-16T07:02:25.905277+0000","last_became_peered":"2023-05-16T07:02:25.905277+0000","last_unstale":"2023-05-16T07:02:26.270551+0000","last_undegraded":"2023-05-16T07:02:26.270551+0000","last_fullsized":"2023-05-16T07:02:26.270551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273245+0000","last_change":"2023-05-16T07:02:26.265761+0000","last_active":"2023-05-16T07:02:26.273245+0000","last_peered":"2023-05-16T07:02:26.273245+0000","last_clean":"2023-05-16T07:02:26.273245+0000","last_became_active":"2023-05-16T07:02:26.265515+0000","last_became_peered":"2023-05-16T07:02:26.265515+0000","last_unstale":"2023-05-16T07:02:26.273245+0000","last_undegraded":"2023-05-16T07:02:26.273245+0000","last_fullsized":"2023-05-16T07:02:26.273245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254410+0000","last_change":"2023-05-16T07:02:25.900333+0000","last_active":"2023-05-16T07:02:26.254410+0000","last_peered":"2023-05-16T07:02:26.254410+0000","last_clean":"2023-05-16T07:02:26.254410+0000","last_became_active":"2023-05-16T07:02:25.900071+0000","last_became_peered":"2023-05-16T07:02:25.900071+0000","last_unstale":"2023-05-16T07:02:26.254410+0000","last_undegraded":"2023-05-16T07:02:26.254410+0000","last_fullsized":"2023-05-16T07:02:26.254410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271814+0000","last_change":"2023-05-16T07:02:26.269160+0000","last_active":"2023-05-16T07:02:26.271814+0000","last_peered":"2023-05-16T07:02:26.271814+0000","last_clean":"2023-05-16T07:02:26.271814+0000","last_became_active":"2023-05-16T07:02:26.268896+0000","last_became_peered":"2023-05-16T07:02:26.268896+0000","last_unstale":"2023-05-16T07:02:26.271814+0000","last_undegraded":"2023-05-16T07:02:26.271814+0000","last_fullsized":"2023-05-16T07:02:26.271814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253776+0000","last_change":"2023-05-16T07:02:25.902998+0000","last_active":"2023-05-16T07:02:26.253776+0000","last_peered":"2023-05-16T07:02:26.253776+0000","last_clean":"2023-05-16T07:02:26.253776+0000","last_became_active":"2023-05-16T07:02:25.900472+0000","last_became_peered":"2023-05-16T07:02:25.900472+0000","last_unstale":"2023-05-16T07:02:26.253776+0000","last_undegraded":"2023-05-16T07:02:26.253776+0000","last_fullsized":"2023-05-16T07:02:26.253776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269460+0000","last_change":"2023-05-16T07:02:26.268175+0000","last_active":"2023-05-16T07:02:26.269460+0000","last_peered":"2023-05-16T07:02:26.269460+0000","last_clean":"2023-05-16T07:02:26.269460+0000","last_became_active":"2023-05-16T07:02:26.267874+0000","last_became_peered":"2023-05-16T07:02:26.267874+0000","last_unstale":"2023-05-16T07:02:26.269460+0000","last_undegraded":"2023-05-16T07:02:26.269460+0000","last_fullsized":"2023-05-16T07:02:26.269460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265758+0000","last_change":"2023-05-16T07:02:25.903192+0000","last_active":"2023-05-16T07:02:26.265758+0000","last_peered":"2023-05-16T07:02:26.265758+0000","last_clean":"2023-05-16T07:02:26.265758+0000","last_became_active":"2023-05-16T07:02:25.902859+0000","last_became_peered":"2023-05-16T07:02:25.902859+0000","last_unstale":"2023-05-16T07:02:26.265758+0000","last_undegraded":"2023-05-16T07:02:26.265758+0000","last_fullsized":"2023-05-16T07:02:26.265758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269471+0000","last_change":"2023-05-16T07:02:26.267628+0000","last_active":"2023-05-16T07:02:26.269471+0000","last_peered":"2023-05-16T07:02:26.269471+0000","last_clean":"2023-05-16T07:02:26.269471+0000","last_became_active":"2023-05-16T07:02:26.267196+0000","last_became_peered":"2023-05-16T07:02:26.267196+0000","last_unstale":"2023-05-16T07:02:26.269471+0000","last_undegraded":"2023-05-16T07:02:26.269471+0000","last_fullsized":"2023-05-16T07:02:26.269471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248618+0000","last_change":"2023-05-16T07:02:25.890842+0000","last_active":"2023-05-16T07:02:26.248618+0000","last_peered":"2023-05-16T07:02:26.248618+0000","last_clean":"2023-05-16T07:02:26.248618+0000","last_became_active":"2023-05-16T07:02:25.890548+0000","last_became_peered":"2023-05-16T07:02:25.890548+0000","last_unstale":"2023-05-16T07:02:26.248618+0000","last_undegraded":"2023-05-16T07:02:26.248618+0000","last_fullsized":"2023-05-16T07:02:26.248618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273817+0000","last_change":"2023-05-16T07:02:26.269745+0000","last_active":"2023-05-16T07:02:26.273817+0000","last_peered":"2023-05-16T07:02:26.273817+0000","last_clean":"2023-05-16T07:02:26.273817+0000","last_became_active":"2023-05-16T07:02:26.269454+0000","last_became_peered":"2023-05-16T07:02:26.269454+0000","last_unstale":"2023-05-16T07:02:26.273817+0000","last_undegraded":"2023-05-16T07:02:26.273817+0000","last_fullsized":"2023-05-16T07:02:26.273817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254294+0000","last_change":"2023-05-16T07:02:25.892810+0000","last_active":"2023-05-16T07:02:26.254294+0000","last_peered":"2023-05-16T07:02:26.254294+0000","last_clean":"2023-05-16T07:02:26.254294+0000","last_became_active":"2023-05-16T07:02:25.892538+0000","last_became_peered":"2023-05-16T07:02:25.892538+0000","last_unstale":"2023-05-16T07:02:26.254294+0000","last_undegraded":"2023-05-16T07:02:26.254294+0000","last_fullsized":"2023-05-16T07:02:26.254294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272472+0000","last_change":"2023-05-16T07:02:26.271699+0000","last_active":"2023-05-16T07:02:26.272472+0000","last_peered":"2023-05-16T07:02:26.272472+0000","last_clean":"2023-05-16T07:02:26.272472+0000","last_became_active":"2023-05-16T07:02:26.271437+0000","last_became_peered":"2023-05-16T07:02:26.271437+0000","last_unstale":"2023-05-16T07:02:26.272472+0000","last_undegraded":"2023-05-16T07:02:26.272472+0000","last_fullsized":"2023-05-16T07:02:26.272472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263471+0000","last_change":"2023-05-16T07:02:25.899544+0000","last_active":"2023-05-16T07:02:26.263471+0000","last_peered":"2023-05-16T07:02:26.263471+0000","last_clean":"2023-05-16T07:02:26.263471+0000","last_became_active":"2023-05-16T07:02:25.898789+0000","last_became_peered":"2023-05-16T07:02:25.898789+0000","last_unstale":"2023-05-16T07:02:26.263471+0000","last_undegraded":"2023-05-16T07:02:26.263471+0000","last_fullsized":"2023-05-16T07:02:26.263471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.264508+0000","last_change":"2023-05-16T07:02:26.262702+0000","last_active":"2023-05-16T07:02:26.264508+0000","last_peered":"2023-05-16T07:02:26.264508+0000","last_clean":"2023-05-16T07:02:26.264508+0000","last_became_active":"2023-05-16T07:02:26.262411+0000","last_became_peered":"2023-05-16T07:02:26.262411+0000","last_unstale":"2023-05-16T07:02:26.264508+0000","last_undegraded":"2023-05-16T07:02:26.264508+0000","last_fullsized":"2023-05-16T07:02:26.264508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.276391+0000","last_change":"2023-05-16T07:02:25.898185+0000","last_active":"2023-05-16T07:02:26.276391+0000","last_peered":"2023-05-16T07:02:26.276391+0000","last_clean":"2023-05-16T07:02:26.276391+0000","last_became_active":"2023-05-16T07:02:25.897879+0000","last_became_peered":"2023-05-16T07:02:25.897879+0000","last_unstale":"2023-05-16T07:02:26.276391+0000","last_undegraded":"2023-05-16T07:02:26.276391+0000","last_fullsized":"2023-05-16T07:02:26.276391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261293+0000","last_change":"2023-05-16T07:02:26.259483+0000","last_active":"2023-05-16T07:02:26.261293+0000","last_peered":"2023-05-16T07:02:26.261293+0000","last_clean":"2023-05-16T07:02:26.261293+0000","last_became_active":"2023-05-16T07:02:26.259204+0000","last_became_peered":"2023-05-16T07:02:26.259204+0000","last_unstale":"2023-05-16T07:02:26.261293+0000","last_undegraded":"2023-05-16T07:02:26.261293+0000","last_fullsized":"2023-05-16T07:02:26.261293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251063+0000","last_change":"2023-05-16T07:02:25.904881+0000","last_active":"2023-05-16T07:02:26.251063+0000","last_peered":"2023-05-16T07:02:26.251063+0000","last_clean":"2023-05-16T07:02:26.251063+0000","last_became_active":"2023-05-16T07:02:25.904608+0000","last_became_peered":"2023-05-16T07:02:25.904608+0000","last_unstale":"2023-05-16T07:02:26.251063+0000","last_undegraded":"2023-05-16T07:02:26.251063+0000","last_fullsized":"2023-05-16T07:02:26.251063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271560+0000","last_change":"2023-05-16T07:02:26.267010+0000","last_active":"2023-05-16T07:02:26.271560+0000","last_peered":"2023-05-16T07:02:26.271560+0000","last_clean":"2023-05-16T07:02:26.271560+0000","last_became_active":"2023-05-16T07:02:26.266536+0000","last_became_peered":"2023-05-16T07:02:26.266536+0000","last_unstale":"2023-05-16T07:02:26.271560+0000","last_undegraded":"2023-05-16T07:02:26.271560+0000","last_fullsized":"2023-05-16T07:02:26.271560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250515+0000","last_change":"2023-05-16T07:02:25.904051+0000","last_active":"2023-05-16T07:02:26.250515+0000","last_peered":"2023-05-16T07:02:26.250515+0000","last_clean":"2023-05-16T07:02:26.250515+0000","last_became_active":"2023-05-16T07:02:25.903808+0000","last_became_peered":"2023-05-16T07:02:25.903808+0000","last_unstale":"2023-05-16T07:02:26.250515+0000","last_undegraded":"2023-05-16T07:02:26.250515+0000","last_fullsized":"2023-05-16T07:02:26.250515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270685+0000","last_change":"2023-05-16T07:02:26.264443+0000","last_active":"2023-05-16T07:02:26.270685+0000","last_peered":"2023-05-16T07:02:26.270685+0000","last_clean":"2023-05-16T07:02:26.270685+0000","last_became_active":"2023-05-16T07:02:26.263678+0000","last_became_peered":"2023-05-16T07:02:26.263678+0000","last_unstale":"2023-05-16T07:02:26.270685+0000","last_undegraded":"2023-05-16T07:02:26.270685+0000","last_fullsized":"2023-05-16T07:02:26.270685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254540+0000","last_change":"2023-05-16T07:02:25.908234+0000","last_active":"2023-05-16T07:02:26.254540+0000","last_peered":"2023-05-16T07:02:26.254540+0000","last_clean":"2023-05-16T07:02:26.254540+0000","last_became_active":"2023-05-16T07:02:25.907873+0000","last_became_peered":"2023-05-16T07:02:25.907873+0000","last_unstale":"2023-05-16T07:02:26.254540+0000","last_undegraded":"2023-05-16T07:02:26.254540+0000","last_fullsized":"2023-05-16T07:02:26.254540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266820+0000","last_change":"2023-05-16T07:02:26.264091+0000","last_active":"2023-05-16T07:02:26.266820+0000","last_peered":"2023-05-16T07:02:26.266820+0000","last_clean":"2023-05-16T07:02:26.266820+0000","last_became_active":"2023-05-16T07:02:26.263428+0000","last_became_peered":"2023-05-16T07:02:26.263428+0000","last_unstale":"2023-05-16T07:02:26.266820+0000","last_undegraded":"2023-05-16T07:02:26.266820+0000","last_fullsized":"2023-05-16T07:02:26.266820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254528+0000","last_change":"2023-05-16T07:02:25.906021+0000","last_active":"2023-05-16T07:02:26.254528+0000","last_peered":"2023-05-16T07:02:26.254528+0000","last_clean":"2023-05-16T07:02:26.254528+0000","last_became_active":"2023-05-16T07:02:25.905737+0000","last_became_peered":"2023-05-16T07:02:25.905737+0000","last_unstale":"2023-05-16T07:02:26.254528+0000","last_undegraded":"2023-05-16T07:02:26.254528+0000","last_fullsized":"2023-05-16T07:02:26.254528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269451+0000","last_change":"2023-05-16T07:02:26.267451+0000","last_active":"2023-05-16T07:02:26.269451+0000","last_peered":"2023-05-16T07:02:26.269451+0000","last_clean":"2023-05-16T07:02:26.269451+0000","last_became_active":"2023-05-16T07:02:26.266832+0000","last_became_peered":"2023-05-16T07:02:26.266832+0000","last_unstale":"2023-05-16T07:02:26.269451+0000","last_undegraded":"2023-05-16T07:02:26.269451+0000","last_fullsized":"2023-05-16T07:02:26.269451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254418+0000","last_change":"2023-05-16T07:02:25.908040+0000","last_active":"2023-05-16T07:02:26.254418+0000","last_peered":"2023-05-16T07:02:26.254418+0000","last_clean":"2023-05-16T07:02:26.254418+0000","last_became_active":"2023-05-16T07:02:25.907139+0000","last_became_peered":"2023-05-16T07:02:25.907139+0000","last_unstale":"2023-05-16T07:02:26.254418+0000","last_undegraded":"2023-05-16T07:02:26.254418+0000","last_fullsized":"2023-05-16T07:02:26.254418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273639+0000","last_change":"2023-05-16T07:02:26.268915+0000","last_active":"2023-05-16T07:02:26.273639+0000","last_peered":"2023-05-16T07:02:26.273639+0000","last_clean":"2023-05-16T07:02:26.273639+0000","last_became_active":"2023-05-16T07:02:26.263749+0000","last_became_peered":"2023-05-16T07:02:26.263749+0000","last_unstale":"2023-05-16T07:02:26.273639+0000","last_undegraded":"2023-05-16T07:02:26.273639+0000","last_fullsized":"2023-05-16T07:02:26.273639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261994+0000","last_change":"2023-05-16T07:02:25.904081+0000","last_active":"2023-05-16T07:02:26.261994+0000","last_peered":"2023-05-16T07:02:26.261994+0000","last_clean":"2023-05-16T07:02:26.261994+0000","last_became_active":"2023-05-16T07:02:25.903797+0000","last_became_peered":"2023-05-16T07:02:25.903797+0000","last_unstale":"2023-05-16T07:02:26.261994+0000","last_undegraded":"2023-05-16T07:02:26.261994+0000","last_fullsized":"2023-05-16T07:02:26.261994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261837+0000","last_change":"2023-05-16T07:02:26.258411+0000","last_active":"2023-05-16T07:02:26.261837+0000","last_peered":"2023-05-16T07:02:26.261837+0000","last_clean":"2023-05-16T07:02:26.261837+0000","last_became_active":"2023-05-16T07:02:26.258108+0000","last_became_peered":"2023-05-16T07:02:26.258108+0000","last_unstale":"2023-05-16T07:02:26.261837+0000","last_undegraded":"2023-05-16T07:02:26.261837+0000","last_fullsized":"2023-05-16T07:02:26.261837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.268905+0000","last_change":"2023-05-16T07:02:25.900588+0000","last_active":"2023-05-16T07:02:26.268905+0000","last_peered":"2023-05-16T07:02:26.268905+0000","last_clean":"2023-05-16T07:02:26.268905+0000","last_became_active":"2023-05-16T07:02:25.900336+0000","last_became_peered":"2023-05-16T07:02:25.900336+0000","last_unstale":"2023-05-16T07:02:26.268905+0000","last_undegraded":"2023-05-16T07:02:26.268905+0000","last_fullsized":"2023-05-16T07:02:26.268905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269680+0000","last_change":"2023-05-16T07:02:26.267380+0000","last_active":"2023-05-16T07:02:26.269680+0000","last_peered":"2023-05-16T07:02:26.269680+0000","last_clean":"2023-05-16T07:02:26.269680+0000","last_became_active":"2023-05-16T07:02:26.267111+0000","last_became_peered":"2023-05-16T07:02:26.267111+0000","last_unstale":"2023-05-16T07:02:26.269680+0000","last_undegraded":"2023-05-16T07:02:26.269680+0000","last_fullsized":"2023-05-16T07:02:26.269680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263166+0000","last_change":"2023-05-16T07:02:25.902771+0000","last_active":"2023-05-16T07:02:26.263166+0000","last_peered":"2023-05-16T07:02:26.263166+0000","last_clean":"2023-05-16T07:02:26.263166+0000","last_became_active":"2023-05-16T07:02:25.902327+0000","last_became_peered":"2023-05-16T07:02:25.902327+0000","last_unstale":"2023-05-16T07:02:26.263166+0000","last_undegraded":"2023-05-16T07:02:26.263166+0000","last_fullsized":"2023-05-16T07:02:26.263166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275129+0000","last_change":"2023-05-16T07:02:26.270942+0000","last_active":"2023-05-16T07:02:26.275129+0000","last_peered":"2023-05-16T07:02:26.275129+0000","last_clean":"2023-05-16T07:02:26.275129+0000","last_became_active":"2023-05-16T07:02:26.270608+0000","last_became_peered":"2023-05-16T07:02:26.270608+0000","last_unstale":"2023-05-16T07:02:26.275129+0000","last_undegraded":"2023-05-16T07:02:26.275129+0000","last_fullsized":"2023-05-16T07:02:26.275129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251411+0000","last_change":"2023-05-16T07:02:25.901785+0000","last_active":"2023-05-16T07:02:26.251411+0000","last_peered":"2023-05-16T07:02:26.251411+0000","last_clean":"2023-05-16T07:02:26.251411+0000","last_became_active":"2023-05-16T07:02:25.901529+0000","last_became_peered":"2023-05-16T07:02:25.901529+0000","last_unstale":"2023-05-16T07:02:26.251411+0000","last_undegraded":"2023-05-16T07:02:26.251411+0000","last_fullsized":"2023-05-16T07:02:26.251411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271585+0000","last_change":"2023-05-16T07:02:26.267468+0000","last_active":"2023-05-16T07:02:26.271585+0000","last_peered":"2023-05-16T07:02:26.271585+0000","last_clean":"2023-05-16T07:02:26.271585+0000","last_became_active":"2023-05-16T07:02:26.266500+0000","last_became_peered":"2023-05-16T07:02:26.266500+0000","last_unstale":"2023-05-16T07:02:26.271585+0000","last_undegraded":"2023-05-16T07:02:26.271585+0000","last_fullsized":"2023-05-16T07:02:26.271585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250458+0000","last_change":"2023-05-16T07:02:25.904335+0000","last_active":"2023-05-16T07:02:26.250458+0000","last_peered":"2023-05-16T07:02:26.250458+0000","last_clean":"2023-05-16T07:02:26.250458+0000","last_became_active":"2023-05-16T07:02:25.903977+0000","last_became_peered":"2023-05-16T07:02:25.903977+0000","last_unstale":"2023-05-16T07:02:26.250458+0000","last_undegraded":"2023-05-16T07:02:26.250458+0000","last_fullsized":"2023-05-16T07:02:26.250458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266029+0000","last_change":"2023-05-16T07:02:26.264992+0000","last_active":"2023-05-16T07:02:26.266029+0000","last_peered":"2023-05-16T07:02:26.266029+0000","last_clean":"2023-05-16T07:02:26.266029+0000","last_became_active":"2023-05-16T07:02:26.264703+0000","last_became_peered":"2023-05-16T07:02:26.264703+0000","last_unstale":"2023-05-16T07:02:26.266029+0000","last_undegraded":"2023-05-16T07:02:26.266029+0000","last_fullsized":"2023-05-16T07:02:26.266029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271717+0000","last_change":"2023-05-16T07:02:25.899178+0000","last_active":"2023-05-16T07:02:26.271717+0000","last_peered":"2023-05-16T07:02:26.271717+0000","last_clean":"2023-05-16T07:02:26.271717+0000","last_became_active":"2023-05-16T07:02:25.898835+0000","last_became_peered":"2023-05-16T07:02:25.898835+0000","last_unstale":"2023-05-16T07:02:26.271717+0000","last_undegraded":"2023-05-16T07:02:26.271717+0000","last_fullsized":"2023-05-16T07:02:26.271717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5252,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366588,"statfs":{"total":96636764160,"available":96631386112,"internally_reserved":0,"allocated":258048,"data_stored":86751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5168,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366672,"statfs":{"total":96636764160,"available":96631472128,"internally_reserved":0,"allocated":237568,"data_stored":86783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5160,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366680,"statfs":{"total":96636764160,"available":96631480320,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":233472,"data_stored":85407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5228,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366612,"statfs":{"total":96636764160,"available":96631410688,"internally_reserved":0,"allocated":233472,"data_stored":85427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5244,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366596,"statfs":{"total":96636764160,"available":96631394304,"internally_reserved":0,"allocated":249856,"data_stored":86929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5160,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366680,"statfs":{"total":96636764160,"available":96631480320,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:02:35.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:02:35.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.892+0000 7ffac8628700 1 Processor -- start 2023-05-16T07:02:35.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.892+0000 7ffac8628700 1 -- start start 2023-05-16T07:02:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac8628700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffac0154170 0x7ffac00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac8628700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ffac0154550 0x7ffac00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac8628700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac00c0660 con 0x7ffac00696a0 2023-05-16T07:02:35.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac8628700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00c1b00 con 0x7ffac0154170 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac8628700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00bfd30 con 0x7ffac0154550 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac63c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffac0154170 0x7ffac00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac6bc5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffac00696a0 0x7ffac0157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51614/0 (socket says 172.21.15.92:51614) 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac63c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffac0154170 0x7ffac00c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40864/0 (socket says 172.21.15.92:40864) 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac6bc5700 1 -- 172.21.15.92:0/236709080 learned_addr learned my addr 172.21.15.92:0/236709080 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac5bc3700 1 --2- 172.21.15.92:0/236709080 >> v2:172.21.15.158:3300/0 conn(0x7ffac0154550 0x7ffac00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac63c4700 1 -- 172.21.15.92:0/236709080 >> v2:172.21.15.158:3300/0 conn(0x7ffac0154550 msgr2=0x7ffac00c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac63c4700 1 --2- 172.21.15.92:0/236709080 >> v2:172.21.15.158:3300/0 conn(0x7ffac0154550 0x7ffac00c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac63c4700 1 -- 172.21.15.92:0/236709080 >> v1:172.21.15.92:6789/0 conn(0x7ffac00696a0 legacy=0x7ffac0157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac63c4700 1 -- 172.21.15.92:0/236709080 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00c0010 con 0x7ffac0154170 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.893+0000 7ffac53c2700 1 -- 172.21.15.92:0/236709080 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193463508 0 0) 0x7ffac00c0660 con 0x7ffac00696a0 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac63c4700 1 --2- 172.21.15.92:0/236709080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c5c70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ffac00844e0 tx=0x7ffab0009780).ready entity=mon.0 client_cookie=99fb9acf42f557f9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac53c2700 1 -- 172.21.15.92:0/236709080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab000e070 con 0x7ffac0154170 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac53c2700 1 -- 172.21.15.92:0/236709080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab000bc30 con 0x7ffac0154170 2023-05-16T07:02:35.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac53c2700 1 -- 172.21.15.92:0/236709080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0012410 con 0x7ffac0154170 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 -- 172.21.15.92:0/236709080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 msgr2=0x7ffac00c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 --2- 172.21.15.92:0/236709080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c5c70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ffac00844e0 tx=0x7ffab0009780).stop 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 -- 172.21.15.92:0/236709080 shutdown_connections 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 --2- 172.21.15.92:0/236709080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c5c70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 --2- 172.21.15.92:0/236709080 >> v2:172.21.15.158:3300/0 conn(0x7ffac0154550 0x7ffac00c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 -- 172.21.15.92:0/236709080 >> 172.21.15.92:0/236709080 conn(0x7ffac00cb700 msgr2=0x7ffac00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:35.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 -- 172.21.15.92:0/236709080 shutdown_connections 2023-05-16T07:02:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.894+0000 7ffac8628700 1 -- 172.21.15.92:0/236709080 wait complete. 2023-05-16T07:02:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.895+0000 7ffac8628700 1 Processor -- start 2023-05-16T07:02:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.895+0000 7ffac8628700 1 -- start start 2023-05-16T07:02:35.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac8628700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffac00696a0 0x7ffac01552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac8628700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac8628700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffac0154550 0x7ffac00c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac8628700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00c9870 con 0x7ffac0154170 2023-05-16T07:02:35.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac8628700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac013e100 con 0x7ffac00696a0 2023-05-16T07:02:35.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac8628700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac013e3e0 con 0x7ffac0154550 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac5bc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac5bc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40872/0 (socket says 172.21.15.92:40872) 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac5bc3700 1 -- 172.21.15.92:0/222987573 learned_addr learned my addr 172.21.15.92:0/222987573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac63c4700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffac00696a0 0x7ffac01552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.896+0000 7ffac6bc5700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffac0154550 0x7ffac00c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac5bc3700 1 -- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffac0154550 msgr2=0x7ffac00c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac5bc3700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffac0154550 0x7ffac00c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac5bc3700 1 -- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffac00696a0 msgr2=0x7ffac01552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac5bc3700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffac00696a0 0x7ffac01552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac5bc3700 1 -- 172.21.15.92:0/222987573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0009a20 con 0x7ffac0154170 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac5bc3700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c8e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ffab4013db0 tx=0x7ffab4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4018070 con 0x7ffac0154170 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4015a30 con 0x7ffac0154170 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac01557a0 con 0x7ffac0154170 2023-05-16T07:02:35.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4015db0 con 0x7ffac0154170 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.897+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac01580e0 con 0x7ffac0154170 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.898+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac006c350 con 0x7ffac0154170 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.899+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ffab4024030 con 0x7ffac0154170 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.899+0000 7ffabe7fc700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffaac05ed90 0x7ffaac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.899+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ffab4081d40 con 0x7ffac0154170 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.899+0000 7ffac63c4700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffaac05ed90 0x7ffaac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.899+0000 7ffac63c4700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffaac05ed90 0x7ffaac061250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ffab0009f90 tx=0x7ffab0009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:35.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:35.902+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ffab4053360 con 0x7ffac0154170 2023-05-16T07:02:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.059+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffac00c02f0 con 0x7ffaac05ed90 2023-05-16T07:02:36.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.069+0000 7ffabe7fc700 1 -- 172.21.15.92:0/222987573 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336258 (secure 0 0 0) 0x7ffac00c02f0 con 0x7ffaac05ed90 2023-05-16T07:02:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:36.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffaac05ed90 msgr2=0x7ffaac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffaac05ed90 0x7ffaac061250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ffab0009f90 tx=0x7ffab0009780).stop 2023-05-16T07:02:36.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 msgr2=0x7ffac00c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c8e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ffab4013db0 tx=0x7ffab4008040).stop 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 shutdown_connections 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffaac05ed90 0x7ffaac061250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffac0154550 0x7ffac00c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffac00696a0 0x7ffac01552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 --2- 172.21.15.92:0/222987573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac0154170 0x7ffac00c8e90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 >> 172.21.15.92:0/222987573 conn(0x7ffac00cb700 msgr2=0x7ffac00abc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 shutdown_connections 2023-05-16T07:02:36.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.072+0000 7ffac8628700 1 -- 172.21.15.92:0/222987573 wait complete. 2023-05-16T07:02:36.080 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:02:36.088 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-05-16T07:02:35.454234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41536,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":754933184,"statfs":{"total":773094113280,"available":773051580416,"internally_reserved":0,"allocated":1900544,"data_stored":687308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002052"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272660+0000","last_change":"2023-05-16T07:02:26.266087+0000","last_active":"2023-05-16T07:02:26.272660+0000","last_peered":"2023-05-16T07:02:26.272660+0000","last_clean":"2023-05-16T07:02:26.272660+0000","last_became_active":"2023-05-16T07:02:26.265870+0000","last_became_peered":"2023-05-16T07:02:26.265870+0000","last_unstale":"2023-05-16T07:02:26.272660+0000","last_undegraded":"2023-05-16T07:02:26.272660+0000","last_fullsized":"2023-05-16T07:02:26.272660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260827+0000","last_change":"2023-05-16T07:02:25.902715+0000","last_active":"2023-05-16T07:02:26.260827+0000","last_peered":"2023-05-16T07:02:26.260827+0000","last_clean":"2023-05-16T07:02:26.260827+0000","last_became_active":"2023-05-16T07:02:25.902401+0000","last_became_peered":"2023-05-16T07:02:25.902401+0000","last_unstale":"2023-05-16T07:02:26.260827+0000","last_undegraded":"2023-05-16T07:02:26.260827+0000","last_fullsized":"2023-05-16T07:02:26.260827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270248+0000","last_change":"2023-05-16T07:02:26.269154+0000","last_active":"2023-05-16T07:02:26.270248+0000","last_peered":"2023-05-16T07:02:26.270248+0000","last_clean":"2023-05-16T07:02:26.270248+0000","last_became_active":"2023-05-16T07:02:26.268835+0000","last_became_peered":"2023-05-16T07:02:26.268835+0000","last_unstale":"2023-05-16T07:02:26.270248+0000","last_undegraded":"2023-05-16T07:02:26.270248+0000","last_fullsized":"2023-05-16T07:02:26.270248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248521+0000","last_change":"2023-05-16T07:02:25.899378+0000","last_active":"2023-05-16T07:02:26.248521+0000","last_peered":"2023-05-16T07:02:26.248521+0000","last_clean":"2023-05-16T07:02:26.248521+0000","last_became_active":"2023-05-16T07:02:25.899114+0000","last_became_peered":"2023-05-16T07:02:25.899114+0000","last_unstale":"2023-05-16T07:02:26.248521+0000","last_undegraded":"2023-05-16T07:02:26.248521+0000","last_fullsized":"2023-05-16T07:02:26.248521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.258180+0000","last_change":"2023-05-16T07:02:26.255359+0000","last_active":"2023-05-16T07:02:26.258180+0000","last_peered":"2023-05-16T07:02:26.258180+0000","last_clean":"2023-05-16T07:02:26.258180+0000","last_became_active":"2023-05-16T07:02:26.254336+0000","last_became_peered":"2023-05-16T07:02:26.254336+0000","last_unstale":"2023-05-16T07:02:26.258180+0000","last_undegraded":"2023-05-16T07:02:26.258180+0000","last_fullsized":"2023-05-16T07:02:26.258180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252195+0000","last_change":"2023-05-16T07:02:25.903697+0000","last_active":"2023-05-16T07:02:26.252195+0000","last_peered":"2023-05-16T07:02:26.252195+0000","last_clean":"2023-05-16T07:02:26.252195+0000","last_became_active":"2023-05-16T07:02:25.903431+0000","last_became_peered":"2023-05-16T07:02:25.903431+0000","last_unstale":"2023-05-16T07:02:26.252195+0000","last_undegraded":"2023-05-16T07:02:26.252195+0000","last_fullsized":"2023-05-16T07:02:26.252195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275541+0000","last_change":"2023-05-16T07:02:26.272040+0000","last_active":"2023-05-16T07:02:26.275541+0000","last_peered":"2023-05-16T07:02:26.275541+0000","last_clean":"2023-05-16T07:02:26.275541+0000","last_became_active":"2023-05-16T07:02:26.271774+0000","last_became_peered":"2023-05-16T07:02:26.271774+0000","last_unstale":"2023-05-16T07:02:26.275541+0000","last_undegraded":"2023-05-16T07:02:26.275541+0000","last_fullsized":"2023-05-16T07:02:26.275541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261304+0000","last_change":"2023-05-16T07:02:25.899639+0000","last_active":"2023-05-16T07:02:26.261304+0000","last_peered":"2023-05-16T07:02:26.261304+0000","last_clean":"2023-05-16T07:02:26.261304+0000","last_became_active":"2023-05-16T07:02:25.899189+0000","last_became_peered":"2023-05-16T07:02:25.899189+0000","last_unstale":"2023-05-16T07:02:26.261304+0000","last_undegraded":"2023-05-16T07:02:26.261304+0000","last_fullsized":"2023-05-16T07:02:26.261304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265676+0000","last_change":"2023-05-16T07:02:26.263614+0000","last_active":"2023-05-16T07:02:26.265676+0000","last_peered":"2023-05-16T07:02:26.265676+0000","last_clean":"2023-05-16T07:02:26.265676+0000","last_became_active":"2023-05-16T07:02:26.263333+0000","last_became_peered":"2023-05-16T07:02:26.263333+0000","last_unstale":"2023-05-16T07:02:26.265676+0000","last_undegraded":"2023-05-16T07:02:26.265676+0000","last_fullsized":"2023-05-16T07:02:26.265676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252139+0000","last_change":"2023-05-16T07:02:25.902127+0000","last_active":"2023-05-16T07:02:26.252139+0000","last_peered":"2023-05-16T07:02:26.252139+0000","last_clean":"2023-05-16T07:02:26.252139+0000","last_became_active":"2023-05-16T07:02:25.901841+0000","last_became_peered":"2023-05-16T07:02:25.901841+0000","last_unstale":"2023-05-16T07:02:26.252139+0000","last_undegraded":"2023-05-16T07:02:26.252139+0000","last_fullsized":"2023-05-16T07:02:26.252139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273327+0000","last_change":"2023-05-16T07:02:26.266427+0000","last_active":"2023-05-16T07:02:26.273327+0000","last_peered":"2023-05-16T07:02:26.273327+0000","last_clean":"2023-05-16T07:02:26.273327+0000","last_became_active":"2023-05-16T07:02:26.265962+0000","last_became_peered":"2023-05-16T07:02:26.265962+0000","last_unstale":"2023-05-16T07:02:26.273327+0000","last_undegraded":"2023-05-16T07:02:26.273327+0000","last_fullsized":"2023-05-16T07:02:26.273327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270470+0000","last_change":"2023-05-16T07:02:25.906258+0000","last_active":"2023-05-16T07:02:26.270470+0000","last_peered":"2023-05-16T07:02:26.270470+0000","last_clean":"2023-05-16T07:02:26.270470+0000","last_became_active":"2023-05-16T07:02:25.905996+0000","last_became_peered":"2023-05-16T07:02:25.905996+0000","last_unstale":"2023-05-16T07:02:26.270470+0000","last_undegraded":"2023-05-16T07:02:26.270470+0000","last_fullsized":"2023-05-16T07:02:26.270470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273736+0000","last_change":"2023-05-16T07:02:26.269375+0000","last_active":"2023-05-16T07:02:26.273736+0000","last_peered":"2023-05-16T07:02:26.273736+0000","last_clean":"2023-05-16T07:02:26.273736+0000","last_became_active":"2023-05-16T07:02:26.269086+0000","last_became_peered":"2023-05-16T07:02:26.269086+0000","last_unstale":"2023-05-16T07:02:26.273736+0000","last_undegraded":"2023-05-16T07:02:26.273736+0000","last_fullsized":"2023-05-16T07:02:26.273736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275465+0000","last_change":"2023-05-16T07:02:25.903795+0000","last_active":"2023-05-16T07:02:26.275465+0000","last_peered":"2023-05-16T07:02:26.275465+0000","last_clean":"2023-05-16T07:02:26.275465+0000","last_became_active":"2023-05-16T07:02:25.903251+0000","last_became_peered":"2023-05-16T07:02:25.903251+0000","last_unstale":"2023-05-16T07:02:26.275465+0000","last_undegraded":"2023-05-16T07:02:26.275465+0000","last_fullsized":"2023-05-16T07:02:26.275465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272707+0000","last_change":"2023-05-16T07:02:26.266963+0000","last_active":"2023-05-16T07:02:26.272707+0000","last_peered":"2023-05-16T07:02:26.272707+0000","last_clean":"2023-05-16T07:02:26.272707+0000","last_became_active":"2023-05-16T07:02:26.266311+0000","last_became_peered":"2023-05-16T07:02:26.266311+0000","last_unstale":"2023-05-16T07:02:26.272707+0000","last_undegraded":"2023-05-16T07:02:26.272707+0000","last_fullsized":"2023-05-16T07:02:26.272707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272749+0000","last_change":"2023-05-16T07:02:25.909821+0000","last_active":"2023-05-16T07:02:26.272749+0000","last_peered":"2023-05-16T07:02:26.272749+0000","last_clean":"2023-05-16T07:02:26.272749+0000","last_became_active":"2023-05-16T07:02:25.909518+0000","last_became_peered":"2023-05-16T07:02:25.909518+0000","last_unstale":"2023-05-16T07:02:26.272749+0000","last_undegraded":"2023-05-16T07:02:26.272749+0000","last_fullsized":"2023-05-16T07:02:26.272749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266094+0000","last_change":"2023-05-16T07:02:26.263808+0000","last_active":"2023-05-16T07:02:26.266094+0000","last_peered":"2023-05-16T07:02:26.266094+0000","last_clean":"2023-05-16T07:02:26.266094+0000","last_became_active":"2023-05-16T07:02:26.263527+0000","last_became_peered":"2023-05-16T07:02:26.263527+0000","last_unstale":"2023-05-16T07:02:26.266094+0000","last_undegraded":"2023-05-16T07:02:26.266094+0000","last_fullsized":"2023-05-16T07:02:26.266094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254202+0000","last_change":"2023-05-16T07:02:25.901918+0000","last_active":"2023-05-16T07:02:26.254202+0000","last_peered":"2023-05-16T07:02:26.254202+0000","last_clean":"2023-05-16T07:02:26.254202+0000","last_became_active":"2023-05-16T07:02:25.901641+0000","last_became_peered":"2023-05-16T07:02:25.901641+0000","last_unstale":"2023-05-16T07:02:26.254202+0000","last_undegraded":"2023-05-16T07:02:26.254202+0000","last_fullsized":"2023-05-16T07:02:26.254202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269556+0000","last_change":"2023-05-16T07:02:26.267982+0000","last_active":"2023-05-16T07:02:26.269556+0000","last_peered":"2023-05-16T07:02:26.269556+0000","last_clean":"2023-05-16T07:02:26.269556+0000","last_became_active":"2023-05-16T07:02:26.267703+0000","last_became_peered":"2023-05-16T07:02:26.267703+0000","last_unstale":"2023-05-16T07:02:26.269556+0000","last_undegraded":"2023-05-16T07:02:26.269556+0000","last_fullsized":"2023-05-16T07:02:26.269556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254313+0000","last_change":"2023-05-16T07:02:25.909768+0000","last_active":"2023-05-16T07:02:26.254313+0000","last_peered":"2023-05-16T07:02:26.254313+0000","last_clean":"2023-05-16T07:02:26.254313+0000","last_became_active":"2023-05-16T07:02:25.909522+0000","last_became_peered":"2023-05-16T07:02:25.909522+0000","last_unstale":"2023-05-16T07:02:26.254313+0000","last_undegraded":"2023-05-16T07:02:26.254313+0000","last_fullsized":"2023-05-16T07:02:26.254313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272806+0000","last_change":"2023-05-16T07:02:26.268669+0000","last_active":"2023-05-16T07:02:26.272806+0000","last_peered":"2023-05-16T07:02:26.272806+0000","last_clean":"2023-05-16T07:02:26.272806+0000","last_became_active":"2023-05-16T07:02:26.268378+0000","last_became_peered":"2023-05-16T07:02:26.268378+0000","last_unstale":"2023-05-16T07:02:26.272806+0000","last_undegraded":"2023-05-16T07:02:26.272806+0000","last_fullsized":"2023-05-16T07:02:26.272806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269854+0000","last_change":"2023-05-16T07:02:25.901955+0000","last_active":"2023-05-16T07:02:26.269854+0000","last_peered":"2023-05-16T07:02:26.269854+0000","last_clean":"2023-05-16T07:02:26.269854+0000","last_became_active":"2023-05-16T07:02:25.901688+0000","last_became_peered":"2023-05-16T07:02:25.901688+0000","last_unstale":"2023-05-16T07:02:26.269854+0000","last_undegraded":"2023-05-16T07:02:26.269854+0000","last_fullsized":"2023-05-16T07:02:26.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273371+0000","last_change":"2023-05-16T07:02:26.270760+0000","last_active":"2023-05-16T07:02:26.273371+0000","last_peered":"2023-05-16T07:02:26.273371+0000","last_clean":"2023-05-16T07:02:26.273371+0000","last_became_active":"2023-05-16T07:02:26.270396+0000","last_became_peered":"2023-05-16T07:02:26.270396+0000","last_unstale":"2023-05-16T07:02:26.273371+0000","last_undegraded":"2023-05-16T07:02:26.273371+0000","last_fullsized":"2023-05-16T07:02:26.273371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274528+0000","last_change":"2023-05-16T07:02:25.909903+0000","last_active":"2023-05-16T07:02:26.274528+0000","last_peered":"2023-05-16T07:02:26.274528+0000","last_clean":"2023-05-16T07:02:26.274528+0000","last_became_active":"2023-05-16T07:02:25.909569+0000","last_became_peered":"2023-05-16T07:02:25.909569+0000","last_unstale":"2023-05-16T07:02:26.274528+0000","last_undegraded":"2023-05-16T07:02:26.274528+0000","last_fullsized":"2023-05-16T07:02:26.274528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273702+0000","last_change":"2023-05-16T07:02:26.268910+0000","last_active":"2023-05-16T07:02:26.273702+0000","last_peered":"2023-05-16T07:02:26.273702+0000","last_clean":"2023-05-16T07:02:26.273702+0000","last_became_active":"2023-05-16T07:02:26.268594+0000","last_became_peered":"2023-05-16T07:02:26.268594+0000","last_unstale":"2023-05-16T07:02:26.273702+0000","last_undegraded":"2023-05-16T07:02:26.273702+0000","last_fullsized":"2023-05-16T07:02:26.273702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273618+0000","last_change":"2023-05-16T07:02:25.911781+0000","last_active":"2023-05-16T07:02:26.273618+0000","last_peered":"2023-05-16T07:02:26.273618+0000","last_clean":"2023-05-16T07:02:26.273618+0000","last_became_active":"2023-05-16T07:02:25.911476+0000","last_became_peered":"2023-05-16T07:02:25.911476+0000","last_unstale":"2023-05-16T07:02:26.273618+0000","last_undegraded":"2023-05-16T07:02:26.273618+0000","last_fullsized":"2023-05-16T07:02:26.273618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273691+0000","last_change":"2023-05-16T07:02:26.271378+0000","last_active":"2023-05-16T07:02:26.273691+0000","last_peered":"2023-05-16T07:02:26.273691+0000","last_clean":"2023-05-16T07:02:26.273691+0000","last_became_active":"2023-05-16T07:02:26.271109+0000","last_became_peered":"2023-05-16T07:02:26.271109+0000","last_unstale":"2023-05-16T07:02:26.273691+0000","last_undegraded":"2023-05-16T07:02:26.273691+0000","last_fullsized":"2023-05-16T07:02:26.273691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255189+0000","last_change":"2023-05-16T07:02:25.908805+0000","last_active":"2023-05-16T07:02:26.255189+0000","last_peered":"2023-05-16T07:02:26.255189+0000","last_clean":"2023-05-16T07:02:26.255189+0000","last_became_active":"2023-05-16T07:02:25.906682+0000","last_became_peered":"2023-05-16T07:02:25.906682+0000","last_unstale":"2023-05-16T07:02:26.255189+0000","last_undegraded":"2023-05-16T07:02:26.255189+0000","last_fullsized":"2023-05-16T07:02:26.255189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269698+0000","last_change":"2023-05-16T07:02:26.267973+0000","last_active":"2023-05-16T07:02:26.269698+0000","last_peered":"2023-05-16T07:02:26.269698+0000","last_clean":"2023-05-16T07:02:26.269698+0000","last_became_active":"2023-05-16T07:02:26.267715+0000","last_became_peered":"2023-05-16T07:02:26.267715+0000","last_unstale":"2023-05-16T07:02:26.269698+0000","last_undegraded":"2023-05-16T07:02:26.269698+0000","last_fullsized":"2023-05-16T07:02:26.269698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252005+0000","last_change":"2023-05-16T07:02:25.902602+0000","last_active":"2023-05-16T07:02:26.252005+0000","last_peered":"2023-05-16T07:02:26.252005+0000","last_clean":"2023-05-16T07:02:26.252005+0000","last_became_active":"2023-05-16T07:02:25.901915+0000","last_became_peered":"2023-05-16T07:02:25.901915+0000","last_unstale":"2023-05-16T07:02:26.252005+0000","last_undegraded":"2023-05-16T07:02:26.252005+0000","last_fullsized":"2023-05-16T07:02:26.252005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274675+0000","last_change":"2023-05-16T07:02:26.272831+0000","last_active":"2023-05-16T07:02:26.274675+0000","last_peered":"2023-05-16T07:02:26.274675+0000","last_clean":"2023-05-16T07:02:26.274675+0000","last_became_active":"2023-05-16T07:02:26.270437+0000","last_became_peered":"2023-05-16T07:02:26.270437+0000","last_unstale":"2023-05-16T07:02:26.274675+0000","last_undegraded":"2023-05-16T07:02:26.274675+0000","last_fullsized":"2023-05-16T07:02:26.274675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266792+0000","last_change":"2023-05-16T07:02:25.905337+0000","last_active":"2023-05-16T07:02:26.266792+0000","last_peered":"2023-05-16T07:02:26.266792+0000","last_clean":"2023-05-16T07:02:26.266792+0000","last_became_active":"2023-05-16T07:02:25.904994+0000","last_became_peered":"2023-05-16T07:02:25.904994+0000","last_unstale":"2023-05-16T07:02:26.266792+0000","last_undegraded":"2023-05-16T07:02:26.266792+0000","last_fullsized":"2023-05-16T07:02:26.266792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273663+0000","last_change":"2023-05-16T07:02:26.269107+0000","last_active":"2023-05-16T07:02:26.273663+0000","last_peered":"2023-05-16T07:02:26.273663+0000","last_clean":"2023-05-16T07:02:26.273663+0000","last_became_active":"2023-05-16T07:02:26.268131+0000","last_became_peered":"2023-05-16T07:02:26.268131+0000","last_unstale":"2023-05-16T07:02:26.273663+0000","last_undegraded":"2023-05-16T07:02:26.273663+0000","last_fullsized":"2023-05-16T07:02:26.273663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250472+0000","last_change":"2023-05-16T07:02:25.897709+0000","last_active":"2023-05-16T07:02:26.250472+0000","last_peered":"2023-05-16T07:02:26.250472+0000","last_clean":"2023-05-16T07:02:26.250472+0000","last_became_active":"2023-05-16T07:02:25.897436+0000","last_became_peered":"2023-05-16T07:02:25.897436+0000","last_unstale":"2023-05-16T07:02:26.250472+0000","last_undegraded":"2023-05-16T07:02:26.250472+0000","last_fullsized":"2023-05-16T07:02:26.250472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273849+0000","last_change":"2023-05-16T07:02:26.269529+0000","last_active":"2023-05-16T07:02:26.273849+0000","last_peered":"2023-05-16T07:02:26.273849+0000","last_clean":"2023-05-16T07:02:26.273849+0000","last_became_active":"2023-05-16T07:02:26.269233+0000","last_became_peered":"2023-05-16T07:02:26.269233+0000","last_unstale":"2023-05-16T07:02:26.273849+0000","last_undegraded":"2023-05-16T07:02:26.273849+0000","last_fullsized":"2023-05-16T07:02:26.273849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.257675+0000","last_change":"2023-05-16T07:02:25.901600+0000","last_active":"2023-05-16T07:02:26.257675+0000","last_peered":"2023-05-16T07:02:26.257675+0000","last_clean":"2023-05-16T07:02:26.257675+0000","last_became_active":"2023-05-16T07:02:25.901168+0000","last_became_peered":"2023-05-16T07:02:25.901168+0000","last_unstale":"2023-05-16T07:02:26.257675+0000","last_undegraded":"2023-05-16T07:02:26.257675+0000","last_fullsized":"2023-05-16T07:02:26.257675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269725+0000","last_change":"2023-05-16T07:02:26.263820+0000","last_active":"2023-05-16T07:02:26.269725+0000","last_peered":"2023-05-16T07:02:26.269725+0000","last_clean":"2023-05-16T07:02:26.269725+0000","last_became_active":"2023-05-16T07:02:26.263188+0000","last_became_peered":"2023-05-16T07:02:26.263188+0000","last_unstale":"2023-05-16T07:02:26.269725+0000","last_undegraded":"2023-05-16T07:02:26.269725+0000","last_fullsized":"2023-05-16T07:02:26.269725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252302+0000","last_change":"2023-05-16T07:02:25.903683+0000","last_active":"2023-05-16T07:02:26.252302+0000","last_peered":"2023-05-16T07:02:26.252302+0000","last_clean":"2023-05-16T07:02:26.252302+0000","last_became_active":"2023-05-16T07:02:25.903358+0000","last_became_peered":"2023-05-16T07:02:25.903358+0000","last_unstale":"2023-05-16T07:02:26.252302+0000","last_undegraded":"2023-05-16T07:02:26.252302+0000","last_fullsized":"2023-05-16T07:02:26.252302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248865+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:02:26.248865+0000","last_peered":"2023-05-16T07:02:26.248865+0000","last_clean":"2023-05-16T07:02:26.248865+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:02:26.248865+0000","last_undegraded":"2023-05-16T07:02:26.248865+0000","last_fullsized":"2023-05-16T07:02:26.248865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252187+0000","last_change":"2023-05-16T07:02:25.903947+0000","last_active":"2023-05-16T07:02:26.252187+0000","last_peered":"2023-05-16T07:02:26.252187+0000","last_clean":"2023-05-16T07:02:26.252187+0000","last_became_active":"2023-05-16T07:02:25.903647+0000","last_became_peered":"2023-05-16T07:02:25.903647+0000","last_unstale":"2023-05-16T07:02:26.252187+0000","last_undegraded":"2023-05-16T07:02:26.252187+0000","last_fullsized":"2023-05-16T07:02:26.252187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270805+0000","last_change":"2023-05-16T07:02:26.260004+0000","last_active":"2023-05-16T07:02:26.270805+0000","last_peered":"2023-05-16T07:02:26.270805+0000","last_clean":"2023-05-16T07:02:26.270805+0000","last_became_active":"2023-05-16T07:02:26.256118+0000","last_became_peered":"2023-05-16T07:02:26.256118+0000","last_unstale":"2023-05-16T07:02:26.270805+0000","last_undegraded":"2023-05-16T07:02:26.270805+0000","last_fullsized":"2023-05-16T07:02:26.270805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252201+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:02:26.252201+0000","last_peered":"2023-05-16T07:02:26.252201+0000","last_clean":"2023-05-16T07:02:26.252201+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:02:26.252201+0000","last_undegraded":"2023-05-16T07:02:26.252201+0000","last_fullsized":"2023-05-16T07:02:26.252201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.260286+0000","last_change":"2023-05-16T07:02:25.902850+0000","last_active":"2023-05-16T07:02:26.260286+0000","last_peered":"2023-05-16T07:02:26.260286+0000","last_clean":"2023-05-16T07:02:26.260286+0000","last_became_active":"2023-05-16T07:02:25.902557+0000","last_became_peered":"2023-05-16T07:02:25.902557+0000","last_unstale":"2023-05-16T07:02:26.260286+0000","last_undegraded":"2023-05-16T07:02:26.260286+0000","last_fullsized":"2023-05-16T07:02:26.260286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274107+0000","last_change":"2023-05-16T07:02:26.265589+0000","last_active":"2023-05-16T07:02:26.274107+0000","last_peered":"2023-05-16T07:02:26.274107+0000","last_clean":"2023-05-16T07:02:26.274107+0000","last_became_active":"2023-05-16T07:02:26.264804+0000","last_became_peered":"2023-05-16T07:02:26.264804+0000","last_unstale":"2023-05-16T07:02:26.274107+0000","last_undegraded":"2023-05-16T07:02:26.274107+0000","last_fullsized":"2023-05-16T07:02:26.274107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269704+0000","last_change":"2023-05-16T07:02:26.263668+0000","last_active":"2023-05-16T07:02:26.269704+0000","last_peered":"2023-05-16T07:02:26.269704+0000","last_clean":"2023-05-16T07:02:26.269704+0000","last_became_active":"2023-05-16T07:02:26.262937+0000","last_became_peered":"2023-05-16T07:02:26.262937+0000","last_unstale":"2023-05-16T07:02:26.269704+0000","last_undegraded":"2023-05-16T07:02:26.269704+0000","last_fullsized":"2023-05-16T07:02:26.269704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252944+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:02:26.252944+0000","last_peered":"2023-05-16T07:02:26.252944+0000","last_clean":"2023-05-16T07:02:26.252944+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:02:26.252944+0000","last_undegraded":"2023-05-16T07:02:26.252944+0000","last_fullsized":"2023-05-16T07:02:26.252944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248559+0000","last_change":"2023-05-16T07:02:25.900932+0000","last_active":"2023-05-16T07:02:26.248559+0000","last_peered":"2023-05-16T07:02:26.248559+0000","last_clean":"2023-05-16T07:02:26.248559+0000","last_became_active":"2023-05-16T07:02:25.899982+0000","last_became_peered":"2023-05-16T07:02:25.899982+0000","last_unstale":"2023-05-16T07:02:26.248559+0000","last_undegraded":"2023-05-16T07:02:26.248559+0000","last_fullsized":"2023-05-16T07:02:26.248559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274431+0000","last_change":"2023-05-16T07:02:26.271648+0000","last_active":"2023-05-16T07:02:26.274431+0000","last_peered":"2023-05-16T07:02:26.274431+0000","last_clean":"2023-05-16T07:02:26.274431+0000","last_became_active":"2023-05-16T07:02:26.271391+0000","last_became_peered":"2023-05-16T07:02:26.271391+0000","last_unstale":"2023-05-16T07:02:26.274431+0000","last_undegraded":"2023-05-16T07:02:26.274431+0000","last_fullsized":"2023-05-16T07:02:26.274431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251974+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:02:26.251974+0000","last_peered":"2023-05-16T07:02:26.251974+0000","last_clean":"2023-05-16T07:02:26.251974+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:02:26.251974+0000","last_undegraded":"2023-05-16T07:02:26.251974+0000","last_fullsized":"2023-05-16T07:02:26.251974+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255213+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:02:26.255213+0000","last_peered":"2023-05-16T07:02:26.255213+0000","last_clean":"2023-05-16T07:02:26.255213+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:02:26.255213+0000","last_undegraded":"2023-05-16T07:02:26.255213+0000","last_fullsized":"2023-05-16T07:02:26.255213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251830+0000","last_change":"2023-05-16T07:02:25.906617+0000","last_active":"2023-05-16T07:02:26.251830+0000","last_peered":"2023-05-16T07:02:26.251830+0000","last_clean":"2023-05-16T07:02:26.251830+0000","last_became_active":"2023-05-16T07:02:25.906269+0000","last_became_peered":"2023-05-16T07:02:25.906269+0000","last_unstale":"2023-05-16T07:02:26.251830+0000","last_undegraded":"2023-05-16T07:02:26.251830+0000","last_fullsized":"2023-05-16T07:02:26.251830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274440+0000","last_change":"2023-05-16T07:02:26.273104+0000","last_active":"2023-05-16T07:02:26.274440+0000","last_peered":"2023-05-16T07:02:26.274440+0000","last_clean":"2023-05-16T07:02:26.274440+0000","last_became_active":"2023-05-16T07:02:26.272816+0000","last_became_peered":"2023-05-16T07:02:26.272816+0000","last_unstale":"2023-05-16T07:02:26.274440+0000","last_undegraded":"2023-05-16T07:02:26.274440+0000","last_fullsized":"2023-05-16T07:02:26.274440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.257669+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:02:26.257669+0000","last_peered":"2023-05-16T07:02:26.257669+0000","last_clean":"2023-05-16T07:02:26.257669+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:02:26.257669+0000","last_undegraded":"2023-05-16T07:02:26.257669+0000","last_fullsized":"2023-05-16T07:02:26.257669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251964+0000","last_change":"2023-05-16T07:02:25.902199+0000","last_active":"2023-05-16T07:02:26.251964+0000","last_peered":"2023-05-16T07:02:26.251964+0000","last_clean":"2023-05-16T07:02:26.251964+0000","last_became_active":"2023-05-16T07:02:25.901913+0000","last_became_peered":"2023-05-16T07:02:25.901913+0000","last_unstale":"2023-05-16T07:02:26.251964+0000","last_undegraded":"2023-05-16T07:02:26.251964+0000","last_fullsized":"2023-05-16T07:02:26.251964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.255055+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:02:26.255055+0000","last_peered":"2023-05-16T07:02:26.255055+0000","last_clean":"2023-05-16T07:02:26.255055+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:02:26.255055+0000","last_undegraded":"2023-05-16T07:02:26.255055+0000","last_fullsized":"2023-05-16T07:02:26.255055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248963+0000","last_change":"2023-05-16T07:02:25.900748+0000","last_active":"2023-05-16T07:02:26.248963+0000","last_peered":"2023-05-16T07:02:26.248963+0000","last_clean":"2023-05-16T07:02:26.248963+0000","last_became_active":"2023-05-16T07:02:25.900481+0000","last_became_peered":"2023-05-16T07:02:25.900481+0000","last_unstale":"2023-05-16T07:02:26.248963+0000","last_undegraded":"2023-05-16T07:02:26.248963+0000","last_fullsized":"2023-05-16T07:02:26.248963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265039+0000","last_change":"2023-05-16T07:02:26.260513+0000","last_active":"2023-05-16T07:02:26.265039+0000","last_peered":"2023-05-16T07:02:26.265039+0000","last_clean":"2023-05-16T07:02:26.265039+0000","last_became_active":"2023-05-16T07:02:26.260265+0000","last_became_peered":"2023-05-16T07:02:26.260265+0000","last_unstale":"2023-05-16T07:02:26.265039+0000","last_undegraded":"2023-05-16T07:02:26.265039+0000","last_fullsized":"2023-05-16T07:02:26.265039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271487+0000","last_change":"2023-05-16T07:02:25.900313+0000","last_active":"2023-05-16T07:02:26.271487+0000","last_peered":"2023-05-16T07:02:26.271487+0000","last_clean":"2023-05-16T07:02:26.271487+0000","last_became_active":"2023-05-16T07:02:25.900056+0000","last_became_peered":"2023-05-16T07:02:25.900056+0000","last_unstale":"2023-05-16T07:02:26.271487+0000","last_undegraded":"2023-05-16T07:02:26.271487+0000","last_fullsized":"2023-05-16T07:02:26.271487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.249182+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:02:26.249182+0000","last_peered":"2023-05-16T07:02:26.249182+0000","last_clean":"2023-05-16T07:02:26.249182+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:02:26.249182+0000","last_undegraded":"2023-05-16T07:02:26.249182+0000","last_fullsized":"2023-05-16T07:02:26.249182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274618+0000","last_change":"2023-05-16T07:02:26.270645+0000","last_active":"2023-05-16T07:02:26.274618+0000","last_peered":"2023-05-16T07:02:26.274618+0000","last_clean":"2023-05-16T07:02:26.274618+0000","last_became_active":"2023-05-16T07:02:26.270312+0000","last_became_peered":"2023-05-16T07:02:26.270312+0000","last_unstale":"2023-05-16T07:02:26.274618+0000","last_undegraded":"2023-05-16T07:02:26.274618+0000","last_fullsized":"2023-05-16T07:02:26.274618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250955+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:02:26.250955+0000","last_peered":"2023-05-16T07:02:26.250955+0000","last_clean":"2023-05-16T07:02:26.250955+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:02:26.250955+0000","last_undegraded":"2023-05-16T07:02:26.250955+0000","last_fullsized":"2023-05-16T07:02:26.250955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.258392+0000","last_change":"2023-05-16T07:02:25.902709+0000","last_active":"2023-05-16T07:02:26.258392+0000","last_peered":"2023-05-16T07:02:26.258392+0000","last_clean":"2023-05-16T07:02:26.258392+0000","last_became_active":"2023-05-16T07:02:25.902421+0000","last_became_peered":"2023-05-16T07:02:25.902421+0000","last_unstale":"2023-05-16T07:02:26.258392+0000","last_undegraded":"2023-05-16T07:02:26.258392+0000","last_fullsized":"2023-05-16T07:02:26.258392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272520+0000","last_change":"2023-05-16T07:02:26.263603+0000","last_active":"2023-05-16T07:02:26.272520+0000","last_peered":"2023-05-16T07:02:26.272520+0000","last_clean":"2023-05-16T07:02:26.272520+0000","last_became_active":"2023-05-16T07:02:26.260959+0000","last_became_peered":"2023-05-16T07:02:26.260959+0000","last_unstale":"2023-05-16T07:02:26.272520+0000","last_undegraded":"2023-05-16T07:02:26.272520+0000","last_fullsized":"2023-05-16T07:02:26.272520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273598+0000","last_change":"2023-05-16T07:02:26.272233+0000","last_active":"2023-05-16T07:02:26.273598+0000","last_peered":"2023-05-16T07:02:26.273598+0000","last_clean":"2023-05-16T07:02:26.273598+0000","last_became_active":"2023-05-16T07:02:26.271964+0000","last_became_peered":"2023-05-16T07:02:26.271964+0000","last_unstale":"2023-05-16T07:02:26.273598+0000","last_undegraded":"2023-05-16T07:02:26.273598+0000","last_fullsized":"2023-05-16T07:02:26.273598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253633+0000","last_change":"2023-05-16T07:02:25.902264+0000","last_active":"2023-05-16T07:02:26.253633+0000","last_peered":"2023-05-16T07:02:26.253633+0000","last_clean":"2023-05-16T07:02:26.253633+0000","last_became_active":"2023-05-16T07:02:25.901192+0000","last_became_peered":"2023-05-16T07:02:25.901192+0000","last_unstale":"2023-05-16T07:02:26.253633+0000","last_undegraded":"2023-05-16T07:02:26.253633+0000","last_fullsized":"2023-05-16T07:02:26.253633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270652+0000","last_change":"2023-05-16T07:02:26.264892+0000","last_active":"2023-05-16T07:02:26.270652+0000","last_peered":"2023-05-16T07:02:26.270652+0000","last_clean":"2023-05-16T07:02:26.270652+0000","last_became_active":"2023-05-16T07:02:26.264659+0000","last_became_peered":"2023-05-16T07:02:26.264659+0000","last_unstale":"2023-05-16T07:02:26.270652+0000","last_undegraded":"2023-05-16T07:02:26.270652+0000","last_fullsized":"2023-05-16T07:02:26.270652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252103+0000","last_change":"2023-05-16T07:02:25.899001+0000","last_active":"2023-05-16T07:02:26.252103+0000","last_peered":"2023-05-16T07:02:26.252103+0000","last_clean":"2023-05-16T07:02:26.252103+0000","last_became_active":"2023-05-16T07:02:25.898724+0000","last_became_peered":"2023-05-16T07:02:25.898724+0000","last_unstale":"2023-05-16T07:02:26.252103+0000","last_undegraded":"2023-05-16T07:02:26.252103+0000","last_fullsized":"2023-05-16T07:02:26.252103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274782+0000","last_change":"2023-05-16T07:02:26.270764+0000","last_active":"2023-05-16T07:02:26.274782+0000","last_peered":"2023-05-16T07:02:26.274782+0000","last_clean":"2023-05-16T07:02:26.274782+0000","last_became_active":"2023-05-16T07:02:26.270171+0000","last_became_peered":"2023-05-16T07:02:26.270171+0000","last_unstale":"2023-05-16T07:02:26.274782+0000","last_undegraded":"2023-05-16T07:02:26.274782+0000","last_fullsized":"2023-05-16T07:02:26.274782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270540+0000","last_change":"2023-05-16T07:02:25.907500+0000","last_active":"2023-05-16T07:02:26.270540+0000","last_peered":"2023-05-16T07:02:26.270540+0000","last_clean":"2023-05-16T07:02:26.270540+0000","last_became_active":"2023-05-16T07:02:25.907202+0000","last_became_peered":"2023-05-16T07:02:25.907202+0000","last_unstale":"2023-05-16T07:02:26.270540+0000","last_undegraded":"2023-05-16T07:02:26.270540+0000","last_fullsized":"2023-05-16T07:02:26.270540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.264264+0000","last_change":"2023-05-16T07:02:26.262010+0000","last_active":"2023-05-16T07:02:26.264264+0000","last_peered":"2023-05-16T07:02:26.264264+0000","last_clean":"2023-05-16T07:02:26.264264+0000","last_became_active":"2023-05-16T07:02:26.261641+0000","last_became_peered":"2023-05-16T07:02:26.261641+0000","last_unstale":"2023-05-16T07:02:26.264264+0000","last_undegraded":"2023-05-16T07:02:26.264264+0000","last_fullsized":"2023-05-16T07:02:26.264264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266384+0000","last_change":"2023-05-16T07:02:25.900614+0000","last_active":"2023-05-16T07:02:26.266384+0000","last_peered":"2023-05-16T07:02:26.266384+0000","last_clean":"2023-05-16T07:02:26.266384+0000","last_became_active":"2023-05-16T07:02:25.898929+0000","last_became_peered":"2023-05-16T07:02:25.898929+0000","last_unstale":"2023-05-16T07:02:26.266384+0000","last_undegraded":"2023-05-16T07:02:26.266384+0000","last_fullsized":"2023-05-16T07:02:26.266384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273729+0000","last_change":"2023-05-16T07:02:26.269314+0000","last_active":"2023-05-16T07:02:26.273729+0000","last_peered":"2023-05-16T07:02:26.273729+0000","last_clean":"2023-05-16T07:02:26.273729+0000","last_became_active":"2023-05-16T07:02:26.269002+0000","last_became_peered":"2023-05-16T07:02:26.269002+0000","last_unstale":"2023-05-16T07:02:26.273729+0000","last_undegraded":"2023-05-16T07:02:26.273729+0000","last_fullsized":"2023-05-16T07:02:26.273729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.267696+0000","last_change":"2023-05-16T07:02:25.899453+0000","last_active":"2023-05-16T07:02:26.267696+0000","last_peered":"2023-05-16T07:02:26.267696+0000","last_clean":"2023-05-16T07:02:26.267696+0000","last_became_active":"2023-05-16T07:02:25.898915+0000","last_became_peered":"2023-05-16T07:02:25.898915+0000","last_unstale":"2023-05-16T07:02:26.267696+0000","last_undegraded":"2023-05-16T07:02:26.267696+0000","last_fullsized":"2023-05-16T07:02:26.267696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269745+0000","last_change":"2023-05-16T07:02:26.263984+0000","last_active":"2023-05-16T07:02:26.269745+0000","last_peered":"2023-05-16T07:02:26.269745+0000","last_clean":"2023-05-16T07:02:26.269745+0000","last_became_active":"2023-05-16T07:02:26.263440+0000","last_became_peered":"2023-05-16T07:02:26.263440+0000","last_unstale":"2023-05-16T07:02:26.269745+0000","last_undegraded":"2023-05-16T07:02:26.269745+0000","last_fullsized":"2023-05-16T07:02:26.269745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251262+0000","last_change":"2023-05-16T07:02:25.903155+0000","last_active":"2023-05-16T07:02:26.251262+0000","last_peered":"2023-05-16T07:02:26.251262+0000","last_clean":"2023-05-16T07:02:26.251262+0000","last_became_active":"2023-05-16T07:02:25.901362+0000","last_became_peered":"2023-05-16T07:02:25.901362+0000","last_unstale":"2023-05-16T07:02:26.251262+0000","last_undegraded":"2023-05-16T07:02:26.251262+0000","last_fullsized":"2023-05-16T07:02:26.251262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266261+0000","last_change":"2023-05-16T07:02:26.262016+0000","last_active":"2023-05-16T07:02:26.266261+0000","last_peered":"2023-05-16T07:02:26.266261+0000","last_clean":"2023-05-16T07:02:26.266261+0000","last_became_active":"2023-05-16T07:02:26.261803+0000","last_became_peered":"2023-05-16T07:02:26.261803+0000","last_unstale":"2023-05-16T07:02:26.266261+0000","last_undegraded":"2023-05-16T07:02:26.266261+0000","last_fullsized":"2023-05-16T07:02:26.266261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253045+0000","last_change":"2023-05-16T07:02:25.904392+0000","last_active":"2023-05-16T07:02:26.253045+0000","last_peered":"2023-05-16T07:02:26.253045+0000","last_clean":"2023-05-16T07:02:26.253045+0000","last_became_active":"2023-05-16T07:02:25.904025+0000","last_became_peered":"2023-05-16T07:02:25.904025+0000","last_unstale":"2023-05-16T07:02:26.253045+0000","last_undegraded":"2023-05-16T07:02:26.253045+0000","last_fullsized":"2023-05-16T07:02:26.253045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266065+0000","last_change":"2023-05-16T07:02:26.263661+0000","last_active":"2023-05-16T07:02:26.266065+0000","last_peered":"2023-05-16T07:02:26.266065+0000","last_clean":"2023-05-16T07:02:26.266065+0000","last_became_active":"2023-05-16T07:02:26.263328+0000","last_became_peered":"2023-05-16T07:02:26.263328+0000","last_unstale":"2023-05-16T07:02:26.266065+0000","last_undegraded":"2023-05-16T07:02:26.266065+0000","last_fullsized":"2023-05-16T07:02:26.266065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252146+0000","last_change":"2023-05-16T07:02:25.905914+0000","last_active":"2023-05-16T07:02:26.252146+0000","last_peered":"2023-05-16T07:02:26.252146+0000","last_clean":"2023-05-16T07:02:26.252146+0000","last_became_active":"2023-05-16T07:02:25.905585+0000","last_became_peered":"2023-05-16T07:02:25.905585+0000","last_unstale":"2023-05-16T07:02:26.252146+0000","last_undegraded":"2023-05-16T07:02:26.252146+0000","last_fullsized":"2023-05-16T07:02:26.252146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274654+0000","last_change":"2023-05-16T07:02:26.271450+0000","last_active":"2023-05-16T07:02:26.274654+0000","last_peered":"2023-05-16T07:02:26.274654+0000","last_clean":"2023-05-16T07:02:26.274654+0000","last_became_active":"2023-05-16T07:02:26.269701+0000","last_became_peered":"2023-05-16T07:02:26.269701+0000","last_unstale":"2023-05-16T07:02:26.274654+0000","last_undegraded":"2023-05-16T07:02:26.274654+0000","last_fullsized":"2023-05-16T07:02:26.274654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253701+0000","last_change":"2023-05-16T07:02:25.908650+0000","last_active":"2023-05-16T07:02:26.253701+0000","last_peered":"2023-05-16T07:02:26.253701+0000","last_clean":"2023-05-16T07:02:26.253701+0000","last_became_active":"2023-05-16T07:02:25.908385+0000","last_became_peered":"2023-05-16T07:02:25.908385+0000","last_unstale":"2023-05-16T07:02:26.253701+0000","last_undegraded":"2023-05-16T07:02:26.253701+0000","last_fullsized":"2023-05-16T07:02:26.253701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275550+0000","last_change":"2023-05-16T07:02:26.272118+0000","last_active":"2023-05-16T07:02:26.275550+0000","last_peered":"2023-05-16T07:02:26.275550+0000","last_clean":"2023-05-16T07:02:26.275550+0000","last_became_active":"2023-05-16T07:02:26.271865+0000","last_became_peered":"2023-05-16T07:02:26.271865+0000","last_unstale":"2023-05-16T07:02:26.275550+0000","last_undegraded":"2023-05-16T07:02:26.275550+0000","last_fullsized":"2023-05-16T07:02:26.275550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252765+0000","last_change":"2023-05-16T07:02:25.905216+0000","last_active":"2023-05-16T07:02:26.252765+0000","last_peered":"2023-05-16T07:02:26.252765+0000","last_clean":"2023-05-16T07:02:26.252765+0000","last_became_active":"2023-05-16T07:02:25.904954+0000","last_became_peered":"2023-05-16T07:02:25.904954+0000","last_unstale":"2023-05-16T07:02:26.252765+0000","last_undegraded":"2023-05-16T07:02:26.252765+0000","last_fullsized":"2023-05-16T07:02:26.252765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274803+0000","last_change":"2023-05-16T07:02:26.270884+0000","last_active":"2023-05-16T07:02:26.274803+0000","last_peered":"2023-05-16T07:02:26.274803+0000","last_clean":"2023-05-16T07:02:26.274803+0000","last_became_active":"2023-05-16T07:02:26.270473+0000","last_became_peered":"2023-05-16T07:02:26.270473+0000","last_unstale":"2023-05-16T07:02:26.274803+0000","last_undegraded":"2023-05-16T07:02:26.274803+0000","last_fullsized":"2023-05-16T07:02:26.274803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248720+0000","last_change":"2023-05-16T07:02:25.900138+0000","last_active":"2023-05-16T07:02:26.248720+0000","last_peered":"2023-05-16T07:02:26.248720+0000","last_clean":"2023-05-16T07:02:26.248720+0000","last_became_active":"2023-05-16T07:02:25.899831+0000","last_became_peered":"2023-05-16T07:02:25.899831+0000","last_unstale":"2023-05-16T07:02:26.248720+0000","last_undegraded":"2023-05-16T07:02:26.248720+0000","last_fullsized":"2023-05-16T07:02:26.248720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275578+0000","last_change":"2023-05-16T07:02:26.273190+0000","last_active":"2023-05-16T07:02:26.275578+0000","last_peered":"2023-05-16T07:02:26.275578+0000","last_clean":"2023-05-16T07:02:26.275578+0000","last_became_active":"2023-05-16T07:02:26.272912+0000","last_became_peered":"2023-05-16T07:02:26.272912+0000","last_unstale":"2023-05-16T07:02:26.275578+0000","last_undegraded":"2023-05-16T07:02:26.275578+0000","last_fullsized":"2023-05-16T07:02:26.275578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-05-16T07:02:27.253747+0000","last_change":"2023-05-16T07:02:25.902881+0000","last_active":"2023-05-16T07:02:27.253747+0000","last_peered":"2023-05-16T07:02:27.253747+0000","last_clean":"2023-05-16T07:02:27.253747+0000","last_became_active":"2023-05-16T07:02:25.902555+0000","last_became_peered":"2023-05-16T07:02:25.902555+0000","last_unstale":"2023-05-16T07:02:27.253747+0000","last_undegraded":"2023-05-16T07:02:27.253747+0000","last_fullsized":"2023-05-16T07:02:27.253747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266078+0000","last_change":"2023-05-16T07:02:26.262732+0000","last_active":"2023-05-16T07:02:26.266078+0000","last_peered":"2023-05-16T07:02:26.266078+0000","last_clean":"2023-05-16T07:02:26.266078+0000","last_became_active":"2023-05-16T07:02:26.258080+0000","last_became_peered":"2023-05-16T07:02:26.258080+0000","last_unstale":"2023-05-16T07:02:26.266078+0000","last_undegraded":"2023-05-16T07:02:26.266078+0000","last_fullsized":"2023-05-16T07:02:26.266078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253611+0000","last_change":"2023-05-16T07:02:25.899623+0000","last_active":"2023-05-16T07:02:26.253611+0000","last_peered":"2023-05-16T07:02:26.253611+0000","last_clean":"2023-05-16T07:02:26.253611+0000","last_became_active":"2023-05-16T07:02:25.899225+0000","last_became_peered":"2023-05-16T07:02:25.899225+0000","last_unstale":"2023-05-16T07:02:26.253611+0000","last_undegraded":"2023-05-16T07:02:26.253611+0000","last_fullsized":"2023-05-16T07:02:26.253611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269443+0000","last_change":"2023-05-16T07:02:26.267767+0000","last_active":"2023-05-16T07:02:26.269443+0000","last_peered":"2023-05-16T07:02:26.269443+0000","last_clean":"2023-05-16T07:02:26.269443+0000","last_became_active":"2023-05-16T07:02:26.267496+0000","last_became_peered":"2023-05-16T07:02:26.267496+0000","last_unstale":"2023-05-16T07:02:26.269443+0000","last_undegraded":"2023-05-16T07:02:26.269443+0000","last_fullsized":"2023-05-16T07:02:26.269443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251732+0000","last_change":"2023-05-16T07:02:25.905706+0000","last_active":"2023-05-16T07:02:26.251732+0000","last_peered":"2023-05-16T07:02:26.251732+0000","last_clean":"2023-05-16T07:02:26.251732+0000","last_became_active":"2023-05-16T07:02:25.905387+0000","last_became_peered":"2023-05-16T07:02:25.905387+0000","last_unstale":"2023-05-16T07:02:26.251732+0000","last_undegraded":"2023-05-16T07:02:26.251732+0000","last_fullsized":"2023-05-16T07:02:26.251732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273026+0000","last_change":"2023-05-16T07:02:26.264777+0000","last_active":"2023-05-16T07:02:26.273026+0000","last_peered":"2023-05-16T07:02:26.273026+0000","last_clean":"2023-05-16T07:02:26.273026+0000","last_became_active":"2023-05-16T07:02:26.264481+0000","last_became_peered":"2023-05-16T07:02:26.264481+0000","last_unstale":"2023-05-16T07:02:26.273026+0000","last_undegraded":"2023-05-16T07:02:26.273026+0000","last_fullsized":"2023-05-16T07:02:26.273026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252159+0000","last_change":"2023-05-16T07:02:25.907961+0000","last_active":"2023-05-16T07:02:26.252159+0000","last_peered":"2023-05-16T07:02:26.252159+0000","last_clean":"2023-05-16T07:02:26.252159+0000","last_became_active":"2023-05-16T07:02:25.907665+0000","last_became_peered":"2023-05-16T07:02:25.907665+0000","last_unstale":"2023-05-16T07:02:26.252159+0000","last_undegraded":"2023-05-16T07:02:26.252159+0000","last_fullsized":"2023-05-16T07:02:26.252159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261333+0000","last_change":"2023-05-16T07:02:26.257095+0000","last_active":"2023-05-16T07:02:26.261333+0000","last_peered":"2023-05-16T07:02:26.261333+0000","last_clean":"2023-05-16T07:02:26.261333+0000","last_became_active":"2023-05-16T07:02:26.256845+0000","last_became_peered":"2023-05-16T07:02:26.256845+0000","last_unstale":"2023-05-16T07:02:26.261333+0000","last_undegraded":"2023-05-16T07:02:26.261333+0000","last_fullsized":"2023-05-16T07:02:26.261333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.252260+0000","last_change":"2023-05-16T07:02:25.907092+0000","last_active":"2023-05-16T07:02:26.252260+0000","last_peered":"2023-05-16T07:02:26.252260+0000","last_clean":"2023-05-16T07:02:26.252260+0000","last_became_active":"2023-05-16T07:02:25.906488+0000","last_became_peered":"2023-05-16T07:02:25.906488+0000","last_unstale":"2023-05-16T07:02:26.252260+0000","last_undegraded":"2023-05-16T07:02:26.252260+0000","last_fullsized":"2023-05-16T07:02:26.252260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274666+0000","last_change":"2023-05-16T07:02:26.272727+0000","last_active":"2023-05-16T07:02:26.274666+0000","last_peered":"2023-05-16T07:02:26.274666+0000","last_clean":"2023-05-16T07:02:26.274666+0000","last_became_active":"2023-05-16T07:02:26.270729+0000","last_became_peered":"2023-05-16T07:02:26.270729+0000","last_unstale":"2023-05-16T07:02:26.274666+0000","last_undegraded":"2023-05-16T07:02:26.274666+0000","last_fullsized":"2023-05-16T07:02:26.274666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270991+0000","last_change":"2023-05-16T07:02:25.902010+0000","last_active":"2023-05-16T07:02:26.270991+0000","last_peered":"2023-05-16T07:02:26.270991+0000","last_clean":"2023-05-16T07:02:26.270991+0000","last_became_active":"2023-05-16T07:02:25.900891+0000","last_became_peered":"2023-05-16T07:02:25.900891+0000","last_unstale":"2023-05-16T07:02:26.270991+0000","last_undegraded":"2023-05-16T07:02:26.270991+0000","last_fullsized":"2023-05-16T07:02:26.270991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271884+0000","last_change":"2023-05-16T07:02:26.268192+0000","last_active":"2023-05-16T07:02:26.271884+0000","last_peered":"2023-05-16T07:02:26.271884+0000","last_clean":"2023-05-16T07:02:26.271884+0000","last_became_active":"2023-05-16T07:02:26.263201+0000","last_became_peered":"2023-05-16T07:02:26.263201+0000","last_unstale":"2023-05-16T07:02:26.271884+0000","last_undegraded":"2023-05-16T07:02:26.271884+0000","last_fullsized":"2023-05-16T07:02:26.271884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250995+0000","last_change":"2023-05-16T07:02:25.899500+0000","last_active":"2023-05-16T07:02:26.250995+0000","last_peered":"2023-05-16T07:02:26.250995+0000","last_clean":"2023-05-16T07:02:26.250995+0000","last_became_active":"2023-05-16T07:02:25.898726+0000","last_became_peered":"2023-05-16T07:02:25.898726+0000","last_unstale":"2023-05-16T07:02:26.250995+0000","last_undegraded":"2023-05-16T07:02:26.250995+0000","last_fullsized":"2023-05-16T07:02:26.250995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.274483+0000","last_change":"2023-05-16T07:02:26.273388+0000","last_active":"2023-05-16T07:02:26.274483+0000","last_peered":"2023-05-16T07:02:26.274483+0000","last_clean":"2023-05-16T07:02:26.274483+0000","last_became_active":"2023-05-16T07:02:26.273112+0000","last_became_peered":"2023-05-16T07:02:26.273112+0000","last_unstale":"2023-05-16T07:02:26.274483+0000","last_undegraded":"2023-05-16T07:02:26.274483+0000","last_fullsized":"2023-05-16T07:02:26.274483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270551+0000","last_change":"2023-05-16T07:02:25.905564+0000","last_active":"2023-05-16T07:02:26.270551+0000","last_peered":"2023-05-16T07:02:26.270551+0000","last_clean":"2023-05-16T07:02:26.270551+0000","last_became_active":"2023-05-16T07:02:25.905277+0000","last_became_peered":"2023-05-16T07:02:25.905277+0000","last_unstale":"2023-05-16T07:02:26.270551+0000","last_undegraded":"2023-05-16T07:02:26.270551+0000","last_fullsized":"2023-05-16T07:02:26.270551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273245+0000","last_change":"2023-05-16T07:02:26.265761+0000","last_active":"2023-05-16T07:02:26.273245+0000","last_peered":"2023-05-16T07:02:26.273245+0000","last_clean":"2023-05-16T07:02:26.273245+0000","last_became_active":"2023-05-16T07:02:26.265515+0000","last_became_peered":"2023-05-16T07:02:26.265515+0000","last_unstale":"2023-05-16T07:02:26.273245+0000","last_undegraded":"2023-05-16T07:02:26.273245+0000","last_fullsized":"2023-05-16T07:02:26.273245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254410+0000","last_change":"2023-05-16T07:02:25.900333+0000","last_active":"2023-05-16T07:02:26.254410+0000","last_peered":"2023-05-16T07:02:26.254410+0000","last_clean":"2023-05-16T07:02:26.254410+0000","last_became_active":"2023-05-16T07:02:25.900071+0000","last_became_peered":"2023-05-16T07:02:25.900071+0000","last_unstale":"2023-05-16T07:02:26.254410+0000","last_undegraded":"2023-05-16T07:02:26.254410+0000","last_fullsized":"2023-05-16T07:02:26.254410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271814+0000","last_change":"2023-05-16T07:02:26.269160+0000","last_active":"2023-05-16T07:02:26.271814+0000","last_peered":"2023-05-16T07:02:26.271814+0000","last_clean":"2023-05-16T07:02:26.271814+0000","last_became_active":"2023-05-16T07:02:26.268896+0000","last_became_peered":"2023-05-16T07:02:26.268896+0000","last_unstale":"2023-05-16T07:02:26.271814+0000","last_undegraded":"2023-05-16T07:02:26.271814+0000","last_fullsized":"2023-05-16T07:02:26.271814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.253776+0000","last_change":"2023-05-16T07:02:25.902998+0000","last_active":"2023-05-16T07:02:26.253776+0000","last_peered":"2023-05-16T07:02:26.253776+0000","last_clean":"2023-05-16T07:02:26.253776+0000","last_became_active":"2023-05-16T07:02:25.900472+0000","last_became_peered":"2023-05-16T07:02:25.900472+0000","last_unstale":"2023-05-16T07:02:26.253776+0000","last_undegraded":"2023-05-16T07:02:26.253776+0000","last_fullsized":"2023-05-16T07:02:26.253776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269460+0000","last_change":"2023-05-16T07:02:26.268175+0000","last_active":"2023-05-16T07:02:26.269460+0000","last_peered":"2023-05-16T07:02:26.269460+0000","last_clean":"2023-05-16T07:02:26.269460+0000","last_became_active":"2023-05-16T07:02:26.267874+0000","last_became_peered":"2023-05-16T07:02:26.267874+0000","last_unstale":"2023-05-16T07:02:26.269460+0000","last_undegraded":"2023-05-16T07:02:26.269460+0000","last_fullsized":"2023-05-16T07:02:26.269460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.265758+0000","last_change":"2023-05-16T07:02:25.903192+0000","last_active":"2023-05-16T07:02:26.265758+0000","last_peered":"2023-05-16T07:02:26.265758+0000","last_clean":"2023-05-16T07:02:26.265758+0000","last_became_active":"2023-05-16T07:02:25.902859+0000","last_became_peered":"2023-05-16T07:02:25.902859+0000","last_unstale":"2023-05-16T07:02:26.265758+0000","last_undegraded":"2023-05-16T07:02:26.265758+0000","last_fullsized":"2023-05-16T07:02:26.265758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269471+0000","last_change":"2023-05-16T07:02:26.267628+0000","last_active":"2023-05-16T07:02:26.269471+0000","last_peered":"2023-05-16T07:02:26.269471+0000","last_clean":"2023-05-16T07:02:26.269471+0000","last_became_active":"2023-05-16T07:02:26.267196+0000","last_became_peered":"2023-05-16T07:02:26.267196+0000","last_unstale":"2023-05-16T07:02:26.269471+0000","last_undegraded":"2023-05-16T07:02:26.269471+0000","last_fullsized":"2023-05-16T07:02:26.269471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.248618+0000","last_change":"2023-05-16T07:02:25.890842+0000","last_active":"2023-05-16T07:02:26.248618+0000","last_peered":"2023-05-16T07:02:26.248618+0000","last_clean":"2023-05-16T07:02:26.248618+0000","last_became_active":"2023-05-16T07:02:25.890548+0000","last_became_peered":"2023-05-16T07:02:25.890548+0000","last_unstale":"2023-05-16T07:02:26.248618+0000","last_undegraded":"2023-05-16T07:02:26.248618+0000","last_fullsized":"2023-05-16T07:02:26.248618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273817+0000","last_change":"2023-05-16T07:02:26.269745+0000","last_active":"2023-05-16T07:02:26.273817+0000","last_peered":"2023-05-16T07:02:26.273817+0000","last_clean":"2023-05-16T07:02:26.273817+0000","last_became_active":"2023-05-16T07:02:26.269454+0000","last_became_peered":"2023-05-16T07:02:26.269454+0000","last_unstale":"2023-05-16T07:02:26.273817+0000","last_undegraded":"2023-05-16T07:02:26.273817+0000","last_fullsized":"2023-05-16T07:02:26.273817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254294+0000","last_change":"2023-05-16T07:02:25.892810+0000","last_active":"2023-05-16T07:02:26.254294+0000","last_peered":"2023-05-16T07:02:26.254294+0000","last_clean":"2023-05-16T07:02:26.254294+0000","last_became_active":"2023-05-16T07:02:25.892538+0000","last_became_peered":"2023-05-16T07:02:25.892538+0000","last_unstale":"2023-05-16T07:02:26.254294+0000","last_undegraded":"2023-05-16T07:02:26.254294+0000","last_fullsized":"2023-05-16T07:02:26.254294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.272472+0000","last_change":"2023-05-16T07:02:26.271699+0000","last_active":"2023-05-16T07:02:26.272472+0000","last_peered":"2023-05-16T07:02:26.272472+0000","last_clean":"2023-05-16T07:02:26.272472+0000","last_became_active":"2023-05-16T07:02:26.271437+0000","last_became_peered":"2023-05-16T07:02:26.271437+0000","last_unstale":"2023-05-16T07:02:26.272472+0000","last_undegraded":"2023-05-16T07:02:26.272472+0000","last_fullsized":"2023-05-16T07:02:26.272472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263471+0000","last_change":"2023-05-16T07:02:25.899544+0000","last_active":"2023-05-16T07:02:26.263471+0000","last_peered":"2023-05-16T07:02:26.263471+0000","last_clean":"2023-05-16T07:02:26.263471+0000","last_became_active":"2023-05-16T07:02:25.898789+0000","last_became_peered":"2023-05-16T07:02:25.898789+0000","last_unstale":"2023-05-16T07:02:26.263471+0000","last_undegraded":"2023-05-16T07:02:26.263471+0000","last_fullsized":"2023-05-16T07:02:26.263471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.264508+0000","last_change":"2023-05-16T07:02:26.262702+0000","last_active":"2023-05-16T07:02:26.264508+0000","last_peered":"2023-05-16T07:02:26.264508+0000","last_clean":"2023-05-16T07:02:26.264508+0000","last_became_active":"2023-05-16T07:02:26.262411+0000","last_became_peered":"2023-05-16T07:02:26.262411+0000","last_unstale":"2023-05-16T07:02:26.264508+0000","last_undegraded":"2023-05-16T07:02:26.264508+0000","last_fullsized":"2023-05-16T07:02:26.264508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.276391+0000","last_change":"2023-05-16T07:02:25.898185+0000","last_active":"2023-05-16T07:02:26.276391+0000","last_peered":"2023-05-16T07:02:26.276391+0000","last_clean":"2023-05-16T07:02:26.276391+0000","last_became_active":"2023-05-16T07:02:25.897879+0000","last_became_peered":"2023-05-16T07:02:25.897879+0000","last_unstale":"2023-05-16T07:02:26.276391+0000","last_undegraded":"2023-05-16T07:02:26.276391+0000","last_fullsized":"2023-05-16T07:02:26.276391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261293+0000","last_change":"2023-05-16T07:02:26.259483+0000","last_active":"2023-05-16T07:02:26.261293+0000","last_peered":"2023-05-16T07:02:26.261293+0000","last_clean":"2023-05-16T07:02:26.261293+0000","last_became_active":"2023-05-16T07:02:26.259204+0000","last_became_peered":"2023-05-16T07:02:26.259204+0000","last_unstale":"2023-05-16T07:02:26.261293+0000","last_undegraded":"2023-05-16T07:02:26.261293+0000","last_fullsized":"2023-05-16T07:02:26.261293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251063+0000","last_change":"2023-05-16T07:02:25.904881+0000","last_active":"2023-05-16T07:02:26.251063+0000","last_peered":"2023-05-16T07:02:26.251063+0000","last_clean":"2023-05-16T07:02:26.251063+0000","last_became_active":"2023-05-16T07:02:25.904608+0000","last_became_peered":"2023-05-16T07:02:25.904608+0000","last_unstale":"2023-05-16T07:02:26.251063+0000","last_undegraded":"2023-05-16T07:02:26.251063+0000","last_fullsized":"2023-05-16T07:02:26.251063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271560+0000","last_change":"2023-05-16T07:02:26.267010+0000","last_active":"2023-05-16T07:02:26.271560+0000","last_peered":"2023-05-16T07:02:26.271560+0000","last_clean":"2023-05-16T07:02:26.271560+0000","last_became_active":"2023-05-16T07:02:26.266536+0000","last_became_peered":"2023-05-16T07:02:26.266536+0000","last_unstale":"2023-05-16T07:02:26.271560+0000","last_undegraded":"2023-05-16T07:02:26.271560+0000","last_fullsized":"2023-05-16T07:02:26.271560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250515+0000","last_change":"2023-05-16T07:02:25.904051+0000","last_active":"2023-05-16T07:02:26.250515+0000","last_peered":"2023-05-16T07:02:26.250515+0000","last_clean":"2023-05-16T07:02:26.250515+0000","last_became_active":"2023-05-16T07:02:25.903808+0000","last_became_peered":"2023-05-16T07:02:25.903808+0000","last_unstale":"2023-05-16T07:02:26.250515+0000","last_undegraded":"2023-05-16T07:02:26.250515+0000","last_fullsized":"2023-05-16T07:02:26.250515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.270685+0000","last_change":"2023-05-16T07:02:26.264443+0000","last_active":"2023-05-16T07:02:26.270685+0000","last_peered":"2023-05-16T07:02:26.270685+0000","last_clean":"2023-05-16T07:02:26.270685+0000","last_became_active":"2023-05-16T07:02:26.263678+0000","last_became_peered":"2023-05-16T07:02:26.263678+0000","last_unstale":"2023-05-16T07:02:26.270685+0000","last_undegraded":"2023-05-16T07:02:26.270685+0000","last_fullsized":"2023-05-16T07:02:26.270685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254540+0000","last_change":"2023-05-16T07:02:25.908234+0000","last_active":"2023-05-16T07:02:26.254540+0000","last_peered":"2023-05-16T07:02:26.254540+0000","last_clean":"2023-05-16T07:02:26.254540+0000","last_became_active":"2023-05-16T07:02:25.907873+0000","last_became_peered":"2023-05-16T07:02:25.907873+0000","last_unstale":"2023-05-16T07:02:26.254540+0000","last_undegraded":"2023-05-16T07:02:26.254540+0000","last_fullsized":"2023-05-16T07:02:26.254540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266820+0000","last_change":"2023-05-16T07:02:26.264091+0000","last_active":"2023-05-16T07:02:26.266820+0000","last_peered":"2023-05-16T07:02:26.266820+0000","last_clean":"2023-05-16T07:02:26.266820+0000","last_became_active":"2023-05-16T07:02:26.263428+0000","last_became_peered":"2023-05-16T07:02:26.263428+0000","last_unstale":"2023-05-16T07:02:26.266820+0000","last_undegraded":"2023-05-16T07:02:26.266820+0000","last_fullsized":"2023-05-16T07:02:26.266820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254528+0000","last_change":"2023-05-16T07:02:25.906021+0000","last_active":"2023-05-16T07:02:26.254528+0000","last_peered":"2023-05-16T07:02:26.254528+0000","last_clean":"2023-05-16T07:02:26.254528+0000","last_became_active":"2023-05-16T07:02:25.905737+0000","last_became_peered":"2023-05-16T07:02:25.905737+0000","last_unstale":"2023-05-16T07:02:26.254528+0000","last_undegraded":"2023-05-16T07:02:26.254528+0000","last_fullsized":"2023-05-16T07:02:26.254528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269451+0000","last_change":"2023-05-16T07:02:26.267451+0000","last_active":"2023-05-16T07:02:26.269451+0000","last_peered":"2023-05-16T07:02:26.269451+0000","last_clean":"2023-05-16T07:02:26.269451+0000","last_became_active":"2023-05-16T07:02:26.266832+0000","last_became_peered":"2023-05-16T07:02:26.266832+0000","last_unstale":"2023-05-16T07:02:26.269451+0000","last_undegraded":"2023-05-16T07:02:26.269451+0000","last_fullsized":"2023-05-16T07:02:26.269451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.254418+0000","last_change":"2023-05-16T07:02:25.908040+0000","last_active":"2023-05-16T07:02:26.254418+0000","last_peered":"2023-05-16T07:02:26.254418+0000","last_clean":"2023-05-16T07:02:26.254418+0000","last_became_active":"2023-05-16T07:02:25.907139+0000","last_became_peered":"2023-05-16T07:02:25.907139+0000","last_unstale":"2023-05-16T07:02:26.254418+0000","last_undegraded":"2023-05-16T07:02:26.254418+0000","last_fullsized":"2023-05-16T07:02:26.254418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.273639+0000","last_change":"2023-05-16T07:02:26.268915+0000","last_active":"2023-05-16T07:02:26.273639+0000","last_peered":"2023-05-16T07:02:26.273639+0000","last_clean":"2023-05-16T07:02:26.273639+0000","last_became_active":"2023-05-16T07:02:26.263749+0000","last_became_peered":"2023-05-16T07:02:26.263749+0000","last_unstale":"2023-05-16T07:02:26.273639+0000","last_undegraded":"2023-05-16T07:02:26.273639+0000","last_fullsized":"2023-05-16T07:02:26.273639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261994+0000","last_change":"2023-05-16T07:02:25.904081+0000","last_active":"2023-05-16T07:02:26.261994+0000","last_peered":"2023-05-16T07:02:26.261994+0000","last_clean":"2023-05-16T07:02:26.261994+0000","last_became_active":"2023-05-16T07:02:25.903797+0000","last_became_peered":"2023-05-16T07:02:25.903797+0000","last_unstale":"2023-05-16T07:02:26.261994+0000","last_undegraded":"2023-05-16T07:02:26.261994+0000","last_fullsized":"2023-05-16T07:02:26.261994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.261837+0000","last_change":"2023-05-16T07:02:26.258411+0000","last_active":"2023-05-16T07:02:26.261837+0000","last_peered":"2023-05-16T07:02:26.261837+0000","last_clean":"2023-05-16T07:02:26.261837+0000","last_became_active":"2023-05-16T07:02:26.258108+0000","last_became_peered":"2023-05-16T07:02:26.258108+0000","last_unstale":"2023-05-16T07:02:26.261837+0000","last_undegraded":"2023-05-16T07:02:26.261837+0000","last_fullsized":"2023-05-16T07:02:26.261837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.268905+0000","last_change":"2023-05-16T07:02:25.900588+0000","last_active":"2023-05-16T07:02:26.268905+0000","last_peered":"2023-05-16T07:02:26.268905+0000","last_clean":"2023-05-16T07:02:26.268905+0000","last_became_active":"2023-05-16T07:02:25.900336+0000","last_became_peered":"2023-05-16T07:02:25.900336+0000","last_unstale":"2023-05-16T07:02:26.268905+0000","last_undegraded":"2023-05-16T07:02:26.268905+0000","last_fullsized":"2023-05-16T07:02:26.268905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.269680+0000","last_change":"2023-05-16T07:02:26.267380+0000","last_active":"2023-05-16T07:02:26.269680+0000","last_peered":"2023-05-16T07:02:26.269680+0000","last_clean":"2023-05-16T07:02:26.269680+0000","last_became_active":"2023-05-16T07:02:26.267111+0000","last_became_peered":"2023-05-16T07:02:26.267111+0000","last_unstale":"2023-05-16T07:02:26.269680+0000","last_undegraded":"2023-05-16T07:02:26.269680+0000","last_fullsized":"2023-05-16T07:02:26.269680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.263166+0000","last_change":"2023-05-16T07:02:25.902771+0000","last_active":"2023-05-16T07:02:26.263166+0000","last_peered":"2023-05-16T07:02:26.263166+0000","last_clean":"2023-05-16T07:02:26.263166+0000","last_became_active":"2023-05-16T07:02:25.902327+0000","last_became_peered":"2023-05-16T07:02:25.902327+0000","last_unstale":"2023-05-16T07:02:26.263166+0000","last_undegraded":"2023-05-16T07:02:26.263166+0000","last_fullsized":"2023-05-16T07:02:26.263166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.275129+0000","last_change":"2023-05-16T07:02:26.270942+0000","last_active":"2023-05-16T07:02:26.275129+0000","last_peered":"2023-05-16T07:02:26.275129+0000","last_clean":"2023-05-16T07:02:26.275129+0000","last_became_active":"2023-05-16T07:02:26.270608+0000","last_became_peered":"2023-05-16T07:02:26.270608+0000","last_unstale":"2023-05-16T07:02:26.275129+0000","last_undegraded":"2023-05-16T07:02:26.275129+0000","last_fullsized":"2023-05-16T07:02:26.275129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.251411+0000","last_change":"2023-05-16T07:02:25.901785+0000","last_active":"2023-05-16T07:02:26.251411+0000","last_peered":"2023-05-16T07:02:26.251411+0000","last_clean":"2023-05-16T07:02:26.251411+0000","last_became_active":"2023-05-16T07:02:25.901529+0000","last_became_peered":"2023-05-16T07:02:25.901529+0000","last_unstale":"2023-05-16T07:02:26.251411+0000","last_undegraded":"2023-05-16T07:02:26.251411+0000","last_fullsized":"2023-05-16T07:02:26.251411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271585+0000","last_change":"2023-05-16T07:02:26.267468+0000","last_active":"2023-05-16T07:02:26.271585+0000","last_peered":"2023-05-16T07:02:26.271585+0000","last_clean":"2023-05-16T07:02:26.271585+0000","last_became_active":"2023-05-16T07:02:26.266500+0000","last_became_peered":"2023-05-16T07:02:26.266500+0000","last_unstale":"2023-05-16T07:02:26.271585+0000","last_undegraded":"2023-05-16T07:02:26.271585+0000","last_fullsized":"2023-05-16T07:02:26.271585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.250458+0000","last_change":"2023-05-16T07:02:25.904335+0000","last_active":"2023-05-16T07:02:26.250458+0000","last_peered":"2023-05-16T07:02:26.250458+0000","last_clean":"2023-05-16T07:02:26.250458+0000","last_became_active":"2023-05-16T07:02:25.903977+0000","last_became_peered":"2023-05-16T07:02:25.903977+0000","last_unstale":"2023-05-16T07:02:26.250458+0000","last_undegraded":"2023-05-16T07:02:26.250458+0000","last_fullsized":"2023-05-16T07:02:26.250458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.266029+0000","last_change":"2023-05-16T07:02:26.264992+0000","last_active":"2023-05-16T07:02:26.266029+0000","last_peered":"2023-05-16T07:02:26.266029+0000","last_clean":"2023-05-16T07:02:26.266029+0000","last_became_active":"2023-05-16T07:02:26.264703+0000","last_became_peered":"2023-05-16T07:02:26.264703+0000","last_unstale":"2023-05-16T07:02:26.266029+0000","last_undegraded":"2023-05-16T07:02:26.266029+0000","last_fullsized":"2023-05-16T07:02:26.266029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:25.880105+0000","last_clean_scrub_stamp":"2023-05-16T07:02:25.880105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-05-16T07:02:26.271717+0000","last_change":"2023-05-16T07:02:25.899178+0000","last_active":"2023-05-16T07:02:26.271717+0000","last_peered":"2023-05-16T07:02:26.271717+0000","last_clean":"2023-05-16T07:02:26.271717+0000","last_became_active":"2023-05-16T07:02:25.898835+0000","last_became_peered":"2023-05-16T07:02:25.898835+0000","last_unstale":"2023-05-16T07:02:26.271717+0000","last_undegraded":"2023-05-16T07:02:26.271717+0000","last_fullsized":"2023-05-16T07:02:26.271717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:24.878069+0000","last_clean_scrub_stamp":"2023-05-16T07:02:24.878069+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5252,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366588,"statfs":{"total":96636764160,"available":96631386112,"internally_reserved":0,"allocated":258048,"data_stored":86751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5168,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366672,"statfs":{"total":96636764160,"available":96631472128,"internally_reserved":0,"allocated":237568,"data_stored":86783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5160,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366680,"statfs":{"total":96636764160,"available":96631480320,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":233472,"data_stored":85407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5228,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366612,"statfs":{"total":96636764160,"available":96631410688,"internally_reserved":0,"allocated":233472,"data_stored":85427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5244,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366596,"statfs":{"total":96636764160,"available":96631394304,"internally_reserved":0,"allocated":249856,"data_stored":86929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5160,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366680,"statfs":{"total":96636764160,"available":96631480320,"internally_reserved":0,"allocated":229376,"data_stored":85337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:02:36.095 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-05-16T07:02:36.095 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-05-16T07:02:36.095 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-05-16T07:02:36.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-05-16T07:02:36.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.233+0000 7fb4266ad700 1 Processor -- start 2023-05-16T07:02:36.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.233+0000 7fb4266ad700 1 -- start start 2023-05-16T07:02:36.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4266ad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb4200c7380 0x7fb420154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4266ad700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb4200c7760 0x7fb4200acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4266ad700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb42014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4266ad700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb420158970 con 0x7fb4200c7380 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4266ad700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb420155340 con 0x7fb4200c7760 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4266ad700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb420157d60 con 0x7fb4200ad040 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb4200c7380 0x7fb420154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb4200c7380 0x7fb420154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40876/0 (socket says 172.21.15.92:40876) 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 -- 172.21.15.92:0/748008241 learned_addr learned my addr 172.21.15.92:0/748008241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 -- 172.21.15.92:0/748008241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 msgr2=0x7fb42014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 --2- 172.21.15.92:0/748008241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb42014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 -- 172.21.15.92:0/748008241 >> v2:172.21.15.158:3300/0 conn(0x7fb4200c7760 msgr2=0x7fb4200acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb424eaa700 1 --2- 172.21.15.92:0/748008241 >> v2:172.21.15.158:3300/0 conn(0x7fb4200c7760 0x7fb4200acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 --2- 172.21.15.92:0/748008241 >> v2:172.21.15.158:3300/0 conn(0x7fb4200c7760 0x7fb4200acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.234+0000 7fb4256ab700 1 -- 172.21.15.92:0/748008241 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420158040 con 0x7fb4200c7380 2023-05-16T07:02:36.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4256ab700 1 --2- 172.21.15.92:0/748008241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7380 0x7fb420154e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fb420084350 tx=0x7fb41c0090f0).ready entity=mon.0 client_cookie=8560a939848bab7a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb417fff700 1 -- 172.21.15.92:0/748008241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00b600 con 0x7fb4200c7380 2023-05-16T07:02:36.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb417fff700 1 -- 172.21.15.92:0/748008241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c00b760 con 0x7fb4200c7380 2023-05-16T07:02:36.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb417fff700 1 -- 172.21.15.92:0/748008241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00bae0 con 0x7fb4200c7380 2023-05-16T07:02:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 -- 172.21.15.92:0/748008241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7380 msgr2=0x7fb420154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 --2- 172.21.15.92:0/748008241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7380 0x7fb420154e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fb420084350 tx=0x7fb41c0090f0).stop 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 -- 172.21.15.92:0/748008241 shutdown_connections 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 --2- 172.21.15.92:0/748008241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb42014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 --2- 172.21.15.92:0/748008241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7380 0x7fb420154e50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 --2- 172.21.15.92:0/748008241 >> v2:172.21.15.158:3300/0 conn(0x7fb4200c7760 0x7fb4200acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 -- 172.21.15.92:0/748008241 >> 172.21.15.92:0/748008241 conn(0x7fb420068330 msgr2=0x7fb420068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 -- 172.21.15.92:0/748008241 shutdown_connections 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.235+0000 7fb4266ad700 1 -- 172.21.15.92:0/748008241 wait complete. 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.236+0000 7fb4266ad700 1 Processor -- start 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.236+0000 7fb4266ad700 1 -- start start 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4266ad700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb4200c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4266ad700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4200c7380 0x7fb420152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4266ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 0x7fb420152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4266ad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb420152a00 con 0x7fb4200c7760 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4266ad700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb420168830 con 0x7fb4200c7380 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4266ad700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4201683a0 con 0x7fb4200ad040 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb425eac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 0x7fb420152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb425eac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 0x7fb420152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40886/0 (socket says 172.21.15.92:40886) 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb425eac700 1 -- 172.21.15.92:0/2506797301 learned_addr learned my addr 172.21.15.92:0/2506797301 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb4256ab700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb4200c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.237+0000 7fb424eaa700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4200c7380 0x7fb420152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb425eac700 1 -- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 msgr2=0x7fb4200c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb425eac700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb4200c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb425eac700 1 -- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4200c7380 msgr2=0x7fb420152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb425eac700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4200c7380 0x7fb420152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb425eac700 1 -- 172.21.15.92:0/2506797301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009370 con 0x7fb4200c7760 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb424eaa700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4200c7380 0x7fb420152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:36.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb425eac700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 0x7fb420152510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fb41800bc60 tx=0x7fb418009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800c6e0 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb418010070 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4200c48b0 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800c860 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.238+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb420167d50 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.239+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb420047150 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.239+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb41801c070 con 0x7fb4200c7760 2023-05-16T07:02:36.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.239+0000 7fb4157fa700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb40405ed90 0x7fb404061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.239+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb418079230 con 0x7fb4200c7760 2023-05-16T07:02:36.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.239+0000 7fb4256ab700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb40405ed90 0x7fb404061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.240+0000 7fb4256ab700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb40405ed90 0x7fb404061250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb420167850 tx=0x7fb41c017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.243+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb41804a7d0 con 0x7fb4200c7760 2023-05-16T07:02:36.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.441+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb420158320 con 0x7fb4200c7760 2023-05-16T07:02:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.442+0000 7fb4157fa700 1 -- 172.21.15.92:0/2506797301 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fb41801a020 con 0x7fb4200c7760 2023-05-16T07:02:36.511 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.443+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb40405ed90 msgr2=0x7fb404061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.443+0000 7fb4266ad700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb40405ed90 0x7fb404061250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb420167850 tx=0x7fb41c017000).stop 2023-05-16T07:02:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.443+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 msgr2=0x7fb420152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.443+0000 7fb4266ad700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 0x7fb420152510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fb41800bc60 tx=0x7fb418009840).stop 2023-05-16T07:02:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 shutdown_connections 2023-05-16T07:02:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb40405ed90 0x7fb404061250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb4200ad040 0x7fb4200c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4200c7760 0x7fb420152510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 --2- 172.21.15.92:0/2506797301 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4200c7380 0x7fb420152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 >> 172.21.15.92:0/2506797301 conn(0x7fb420068330 msgr2=0x7fb420150b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 shutdown_connections 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.444+0000 7fb4266ad700 1 -- 172.21.15.92:0/2506797301 wait complete. 2023-05-16T07:02:36.514 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-05-16T07:02:36.515 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-05-16T07:02:36.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 Processor -- start 2023-05-16T07:02:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 -- start start 2023-05-16T07:02:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154170 0x7f489c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f489c154550 0x7f489c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f489c0696a0 0x7f489c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c0c0660 con 0x7f489c154550 2023-05-16T07:02:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c06a020 con 0x7f489c0696a0 2023-05-16T07:02:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.622+0000 7f48a1a83700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c0bfd30 con 0x7f489c154170 2023-05-16T07:02:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f489c154550 0x7f489c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f489c154550 0x7f489c0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40892/0 (socket says 172.21.15.92:40892) 2023-05-16T07:02:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 -- 172.21.15.92:0/3934764252 learned_addr learned my addr 172.21.15.92:0/3934764252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489bfff700 1 --2- 172.21.15.92:0/3934764252 >> v2:172.21.15.158:3300/0 conn(0x7f489c0696a0 0x7f489c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489b7fe700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154170 0x7f489c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 -- 172.21.15.92:0/3934764252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154170 msgr2=0x7f489c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154170 0x7f489c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 -- 172.21.15.92:0/3934764252 >> v2:172.21.15.158:3300/0 conn(0x7f489c0696a0 msgr2=0x7f489c069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 --2- 172.21.15.92:0/3934764252 >> v2:172.21.15.158:3300/0 conn(0x7f489c0696a0 0x7f489c069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 -- 172.21.15.92:0/3934764252 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0c0010 con 0x7f489c154550 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489b7fe700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154170 0x7f489c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489bfff700 1 --2- 172.21.15.92:0/3934764252 >> v2:172.21.15.158:3300/0 conn(0x7f489c0696a0 0x7f489c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489affd700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c154550 0x7f489c0c6160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f489c083be0 tx=0x7f48840090f0).ready entity=mon.0 client_cookie=3a970992b98fbce4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489a7fc700 1 -- 172.21.15.92:0/3934764252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400b600 con 0x7f489c154550 2023-05-16T07:02:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489a7fc700 1 -- 172.21.15.92:0/3934764252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b760 con 0x7f489c154550 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.623+0000 7f489a7fc700 1 -- 172.21.15.92:0/3934764252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bb00 con 0x7f489c154550 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 -- 172.21.15.92:0/3934764252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c154550 msgr2=0x7f489c0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c154550 0x7f489c0c6160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f489c083be0 tx=0x7f48840090f0).stop 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 -- 172.21.15.92:0/3934764252 shutdown_connections 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 --2- 172.21.15.92:0/3934764252 >> v2:172.21.15.158:3300/0 conn(0x7f489c0696a0 0x7f489c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154170 0x7f489c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 --2- 172.21.15.92:0/3934764252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c154550 0x7f489c0c6160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 -- 172.21.15.92:0/3934764252 >> 172.21.15.92:0/3934764252 conn(0x7f489c0cb700 msgr2=0x7f489c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 -- 172.21.15.92:0/3934764252 shutdown_connections 2023-05-16T07:02:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.624+0000 7f48a1a83700 1 -- 172.21.15.92:0/3934764252 wait complete. 2023-05-16T07:02:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.625+0000 7f48a1a83700 1 Processor -- start 2023-05-16T07:02:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.625+0000 7f48a1a83700 1 -- start start 2023-05-16T07:02:36.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 0x7f489c0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f489c154170 0x7f489c0c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154550 0x7f489c0c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c0c4ad0 con 0x7f489c0696a0 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c0c3ec0 con 0x7f489c154170 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c0c41a0 con 0x7f489c154550 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 0x7f489c0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 0x7f489c0c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40896/0 (socket says 172.21.15.92:40896) 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 -- 172.21.15.92:0/2897205277 learned_addr learned my addr 172.21.15.92:0/2897205277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:36.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 -- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154550 msgr2=0x7f489c0c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154550 0x7f489c0c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 -- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f489c154170 msgr2=0x7f489c0c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f489c154170 0x7f489c0c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 -- 172.21.15.92:0/2897205277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884009370 con 0x7f489c0696a0 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f489b7fe700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 0x7f489c0c8ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f489400a400 tx=0x7f4894007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400d650 con 0x7f489c0696a0 2023-05-16T07:02:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489400d7b0 con 0x7f489c0696a0 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c0c4480 con 0x7f489c0696a0 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400db30 con 0x7f489c0696a0 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.626+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c0c4810 con 0x7f489c0696a0 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.627+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c0c4610 con 0x7f489c0696a0 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.627+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4894016470 con 0x7f489c0696a0 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.628+0000 7f48937fe700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f487c05ed90 0x7f487c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.628+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f489407aab0 con 0x7f489c0696a0 2023-05-16T07:02:36.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.628+0000 7f489affd700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f487c05ed90 0x7f487c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.628+0000 7f489affd700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f487c05ed90 0x7f487c061250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f489c083be0 tx=0x7f4884017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.632+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f489404bff0 con 0x7f489c0696a0 2023-05-16T07:02:36.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.797+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f489c0666c0 con 0x7f489c0696a0 2023-05-16T07:02:36.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.798+0000 7f48937fe700 1 -- 172.21.15.92:0/2897205277 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f4894007a50 con 0x7f489c0696a0 2023-05-16T07:02:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:36.802 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:36.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f487c05ed90 msgr2=0x7f487c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f487c05ed90 0x7f487c061250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f489c083be0 tx=0x7f4884017000).stop 2023-05-16T07:02:36.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 msgr2=0x7f489c0c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 0x7f489c0c8ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f489400a400 tx=0x7f4894007910).stop 2023-05-16T07:02:36.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 shutdown_connections 2023-05-16T07:02:36.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f487c05ed90 0x7f487c061250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f489c154550 0x7f489c0c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489c0696a0 0x7f489c0c8ff0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 --2- 172.21.15.92:0/2897205277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f489c154170 0x7f489c0c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 >> 172.21.15.92:0/2897205277 conn(0x7f489c0cb700 msgr2=0x7f489c068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 shutdown_connections 2023-05-16T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.800+0000 7f48a1a83700 1 -- 172.21.15.92:0/2897205277 wait complete. 2023-05-16T07:02:36.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:36.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:36.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.927+0000 7fa58d5d2700 1 Processor -- start 2023-05-16T07:02:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.927+0000 7fa58d5d2700 1 -- start start 2023-05-16T07:02:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa58d5d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 0x7fa588150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa58d5d2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa588151520 con 0x7fa588150920 2023-05-16T07:02:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa58d5d2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5881511f0 con 0x7fa588062980 2023-05-16T07:02:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa58d5d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5881520c0 con 0x7fa5880a75b0 2023-05-16T07:02:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa587fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa588150920 0x7fa5881519b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51652/0 (socket says 172.21.15.92:51652) 2023-05-16T07:02:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa587fff700 1 -- 172.21.15.92:0/152721729 learned_addr learned my addr 172.21.15.92:0/152721729 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743524269 0 0) 0x7fa588151520 con 0x7fa588150920 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5740033f0 con 0x7fa588150920 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa57ffff700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 0x7fa588150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169716479 0 0) 0x7fa5740033f0 con 0x7fa588150920 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 msgr2=0x7fa588150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 0x7fa588150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 >> v1:172.21.15.158:6789/0 conn(0x7fa588062980 legacy=0x7fa588154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880a7a30 con 0x7fa588150920 2023-05-16T07:02:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fa578001470 con 0x7fa588150920 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 >> v1:172.21.15.92:6789/0 conn(0x7fa588150920 legacy=0x7fa5881519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa574003910 0x7fa574003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa574004690 0x7fa574004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 0x7fa574005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5881520c0 con 0x7fa574004690 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57400b910 con 0x7fa574003910 2023-05-16T07:02:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57400bad0 con 0x7fa5880a75b0 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa57ffff700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa574004690 0x7fa574004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa587fff700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa574003910 0x7fa574003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.928+0000 7fa58cdd1700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 0x7fa574005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa57ffff700 1 -- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 msgr2=0x7fa574005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa57ffff700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa5880a75b0 0x7fa574005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa57ffff700 1 -- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa574003910 msgr2=0x7fa574003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa57ffff700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa574003910 0x7fa574003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa57ffff700 1 -- 172.21.15.92:0/152721729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740033f0 con 0x7fa574004690 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa57ffff700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa574004690 0x7fa574004b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fa570009a60 tx=0x7fa570009060).ready entity=mon.0 client_cookie=19a28a4c15c139ea server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570009ea0 con 0x7fa574004690 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa570009430 con 0x7fa574004690 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa5877fe700 1 -- 172.21.15.92:0/152721729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000e870 con 0x7fa574004690 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa58d5d2700 1 -- 172.21.15.92:0/152721729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa574004690 msgr2=0x7fa574004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/152721729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa574004690 0x7fa574004b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fa570009a60 tx=0x7fa570009060).stop 2023-05-16T07:02:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa58cdd1700 1 -- 172.21.15.92:0/152721729 reap_dead start 2023-05-16T07:02:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa58d5d2700 1 -- 172.21.15.92:0/152721729 shutdown_connections 2023-05-16T07:02:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa58d5d2700 1 -- 172.21.15.92:0/152721729 >> 172.21.15.92:0/152721729 conn(0x7fa588068a30 msgr2=0x7fa5880c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.929+0000 7fa58d5d2700 1 -- 172.21.15.92:0/152721729 shutdown_connections 2023-05-16T07:02:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.930+0000 7fa58d5d2700 1 -- 172.21.15.92:0/152721729 wait complete. 2023-05-16T07:02:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 Processor -- start 2023-05-16T07:02:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 -- start start 2023-05-16T07:02:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa574004690 0x7fa5880633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa5880a6480 0x7fa58814d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 0x7fa5880c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880638b0 con 0x7fa5881620a0 2023-05-16T07:02:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880a40d0 con 0x7fa5880a6480 2023-05-16T07:02:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58d5d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880a69f0 con 0x7fa574004690 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58cdd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 0x7fa5880c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58cdd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 0x7fa5880c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40920/0 (socket says 172.21.15.92:40920) 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa58cdd1700 1 -- 172.21.15.92:0/1055750699 learned_addr learned my addr 172.21.15.92:0/1055750699 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa587fff700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa574004690 0x7fa5880633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.931+0000 7fa57ffff700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa5880a6480 0x7fa58814d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58cdd1700 1 -- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa574004690 msgr2=0x7fa5880633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58cdd1700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa574004690 0x7fa5880633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58cdd1700 1 -- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa5880a6480 msgr2=0x7fa58814d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58cdd1700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa5880a6480 0x7fa58814d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58cdd1700 1 -- 172.21.15.92:0/1055750699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880a6cd0 con 0x7fa5881620a0 2023-05-16T07:02:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58cdd1700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 0x7fa5880c6a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa5880c8910 tx=0x7fa580007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580006c30 con 0x7fa5881620a0 2023-05-16T07:02:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa580006d90 con 0x7fa5881620a0 2023-05-16T07:02:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5880a6fb0 con 0x7fa5881620a0 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580007260 con 0x7fa5881620a0 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.932+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa588062fc0 con 0x7fa5881620a0 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.933+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa588047150 con 0x7fa5881620a0 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.933+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa580007650 con 0x7fa5881620a0 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.933+0000 7fa584ff9700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa56805ed90 0x7fa568061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.934+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa5800793d0 con 0x7fa5881620a0 2023-05-16T07:02:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.934+0000 7fa587fff700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa56805ed90 0x7fa568061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.934+0000 7fa587fff700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa56805ed90 0x7fa568061250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fa578000e50 tx=0x7fa578001e50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:36.937+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa58004aa20 con 0x7fa5881620a0 2023-05-16T07:02:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.098+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa5880a8480 con 0x7fa5881620a0 2023-05-16T07:02:37.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.099+0000 7fa584ff9700 1 -- 172.21.15.92:0/1055750699 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa58004e040 con 0x7fa5881620a0 2023-05-16T07:02:37.103 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.100+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa56805ed90 msgr2=0x7fa568061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.100+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa56805ed90 0x7fa568061250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fa578000e50 tx=0x7fa578001e50).stop 2023-05-16T07:02:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.100+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 msgr2=0x7fa5880c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.100+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 0x7fa5880c6a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa5880c8910 tx=0x7fa580007f30).stop 2023-05-16T07:02:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 shutdown_connections 2023-05-16T07:02:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa56805ed90 0x7fa568061250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa574004690 0x7fa5880633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5881620a0 0x7fa5880c6a90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 --2- 172.21.15.92:0/1055750699 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa5880a6480 0x7fa58814d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 >> 172.21.15.92:0/1055750699 conn(0x7fa588068a30 msgr2=0x7fa588153b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 shutdown_connections 2023-05-16T07:02:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.101+0000 7fa58d5d2700 1 -- 172.21.15.92:0/1055750699 wait complete. 2023-05-16T07:02:37.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.226+0000 7f998f59e700 1 Processor -- start 2023-05-16T07:02:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.226+0000 7f998f59e700 1 -- start start 2023-05-16T07:02:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.227+0000 7f998f59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f99901545b0 0x7f999015b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.227+0000 7f998f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9990065aa0 con 0x7f9990150250 2023-05-16T07:02:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.227+0000 7f998f59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99900c2720 con 0x7f9990064920 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.227+0000 7f998f59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9990064e90 con 0x7f99901545b0 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9990150250 0x7f99900c2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51672/0 (socket says 172.21.15.92:51672) 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998e59c700 1 -- 172.21.15.92:0/2801183953 learned_addr learned my addr 172.21.15.92:0/2801183953 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 --2- 172.21.15.92:0/2801183953 >> v2:172.21.15.158:3300/0 conn(0x7f99901545b0 0x7f999015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501672857 0 0) 0x7f9990065aa0 con 0x7f9990150250 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99780033f0 con 0x7f9990150250 2023-05-16T07:02:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916420521 0 0) 0x7f99780033f0 con 0x7f9990150250 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 >> v2:172.21.15.158:3300/0 conn(0x7f99901545b0 msgr2=0x7f999015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 --2- 172.21.15.92:0/2801183953 >> v2:172.21.15.158:3300/0 conn(0x7f99901545b0 0x7f999015b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 >> v1:172.21.15.158:6789/0 conn(0x7f9990064920 legacy=0x7f9990153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990065170 con 0x7f9990150250 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f9980001470 con 0x7f9990150250 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 >> v1:172.21.15.92:6789/0 conn(0x7f9990150250 legacy=0x7f99900c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99780038e0 0x7f9978003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978004690 0x7f9978004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978005000 0x7f997800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9990064e90 con 0x7f9978005000 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997800beb0 con 0x7f99780038e0 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997800c070 con 0x7f9978004690 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978005000 0x7f997800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998e59c700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99780038e0 0x7f9978003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998dd9b700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978004690 0x7f9978004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 -- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978004690 msgr2=0x7f9978004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978004690 0x7f9978004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 -- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99780038e0 msgr2=0x7f9978003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99780038e0 0x7f9978003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.228+0000 7f998ed9d700 1 -- 172.21.15.92:0/2801183953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780033f0 con 0x7f9978005000 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998ed9d700 1 -- 172.21.15.92:0/2801183953 reap_dead start 2023-05-16T07:02:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998ed9d700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978005000 0x7f997800b8e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f998800e1c0 tx=0x7f998800b040).ready entity=mon.0 client_cookie=caf5cb1271fa8b86 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988017070 con 0x7f9978005000 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998800f040 con 0x7f9978005000 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998d59a700 1 -- 172.21.15.92:0/2801183953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998800eca0 con 0x7f9978005000 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 -- 172.21.15.92:0/2801183953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978005000 msgr2=0x7f997800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978005000 0x7f997800b8e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f998800e1c0 tx=0x7f998800b040).stop 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 -- 172.21.15.92:0/2801183953 shutdown_connections 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 --2- 172.21.15.92:0/2801183953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978005000 0x7f997800b8e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 -- 172.21.15.92:0/2801183953 >> 172.21.15.92:0/2801183953 conn(0x7f99900c7a80 msgr2=0x7f99900c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 -- 172.21.15.92:0/2801183953 shutdown_connections 2023-05-16T07:02:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.229+0000 7f998f59e700 1 -- 172.21.15.92:0/2801183953 wait complete. 2023-05-16T07:02:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.230+0000 7f998f59e700 1 Processor -- start 2023-05-16T07:02:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.230+0000 7f998f59e700 1 -- start start 2023-05-16T07:02:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978005000 0x7f9990147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 0x7f9990152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998f59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99900c62c0 0x7f99900c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99901526f0 con 0x7f99900ab0c0 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998f59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9990151ae0 con 0x7f99900c62c0 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9990151dc0 con 0x7f9978005000 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 0x7f9990152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 0x7f9990152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40938/0 (socket says 172.21.15.92:40938) 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 -- 172.21.15.92:0/2720556733 learned_addr learned my addr 172.21.15.92:0/2720556733 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998ed9d700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99900c62c0 0x7f99900c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998e59c700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978005000 0x7f9990147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 -- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978005000 msgr2=0x7f9990147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978005000 0x7f9990147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 -- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99900c62c0 msgr2=0x7f99900c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99900c62c0 0x7f99900c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.231+0000 7f998dd9b700 1 -- 172.21.15.92:0/2720556733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99901520a0 con 0x7f99900ab0c0 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.232+0000 7f998dd9b700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 0x7f9990152b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f99840026b0 tx=0x7f9984007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.232+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840114c0 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.232+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9984011620 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.232+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9990152380 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.232+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840119a0 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.232+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99900ac810 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.233+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99901528a0 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.233+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f998401a030 con 0x7f99900ab0c0 2023-05-16T07:02:37.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.233+0000 7f997e7fc700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f997005ed90 0x7f9970061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.233+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9984079380 con 0x7f99900ab0c0 2023-05-16T07:02:37.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.233+0000 7f998e59c700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f997005ed90 0x7f9970061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.234+0000 7f998e59c700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f997005ed90 0x7f9970061250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9980001050 tx=0x7f9980001e50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.237+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f99840498f0 con 0x7f99900ab0c0 2023-05-16T07:02:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.399+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9990047150 con 0x7f99900ab0c0 2023-05-16T07:02:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.400+0000 7f997e7fc700 1 -- 172.21.15.92:0/2720556733 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99901528a0 con 0x7f99900ab0c0 2023-05-16T07:02:37.404 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f997005ed90 msgr2=0x7f9970061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f997005ed90 0x7f9970061250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9980001050 tx=0x7f9980001e50).stop 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 msgr2=0x7f9990152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 0x7f9990152b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f99840026b0 tx=0x7f9984007bb0).stop 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 shutdown_connections 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f997005ed90 0x7f9970061250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9978005000 0x7f9990147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99900c62c0 0x7f99900c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 --2- 172.21.15.92:0/2720556733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99900ab0c0 0x7f9990152b80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 >> 172.21.15.92:0/2720556733 conn(0x7f99900c7a80 msgr2=0x7f99900bc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 shutdown_connections 2023-05-16T07:02:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.401+0000 7f998f59e700 1 -- 172.21.15.92:0/2720556733 wait complete. 2023-05-16T07:02:37.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.527+0000 7fb555b2b700 1 Processor -- start 2023-05-16T07:02:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.527+0000 7fb555b2b700 1 -- start start 2023-05-16T07:02:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb555b2b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 0x7fb5500aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb555b2b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb550068160 0x7fb550068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb555b2b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5500c94d0 con 0x7fb550154470 2023-05-16T07:02:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb555b2b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5500cb7e0 con 0x7fb550068160 2023-05-16T07:02:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb555b2b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb550068a70 con 0x7fb550154090 2023-05-16T07:02:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb54effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb550154470 0x7fb5500a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51696/0 (socket says 172.21.15.92:51696) 2023-05-16T07:02:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb54effd700 1 -- 172.21.15.92:0/1404542177 learned_addr learned my addr 172.21.15.92:0/1404542177 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519817907 0 0) 0x7fb5500c94d0 con 0x7fb550154470 2023-05-16T07:02:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb54ffff700 1 --2- 172.21.15.92:0/1404542177 >> v2:172.21.15.158:3300/0 conn(0x7fb550068160 0x7fb550068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb53c0033f0 con 0x7fb550154470 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.528+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 0x7fb5500aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646158588 0 0) 0x7fb53c0033f0 con 0x7fb550154470 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 msgr2=0x7fb5500aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 0x7fb5500aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 >> v2:172.21.15.158:3300/0 conn(0x7fb550068160 msgr2=0x7fb550068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 --2- 172.21.15.92:0/1404542177 >> v2:172.21.15.158:3300/0 conn(0x7fb550068160 0x7fb550068580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5500c88c0 con 0x7fb550154470 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fb538001470 con 0x7fb550154470 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 >> v1:172.21.15.92:6789/0 conn(0x7fb550154470 legacy=0x7fb5500a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb53c0038e0 0x7fb53c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53c004690 0x7fb53c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 0x7fb53c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500cb7e0 con 0x7fb53c004690 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb550068a70 con 0x7fb53c0038e0 2023-05-16T07:02:37.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c00b8e0 con 0x7fb550154090 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53c004690 0x7fb53c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb53c0038e0 0x7fb53c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54ffff700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 0x7fb53c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 -- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 msgr2=0x7fb53c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb550154090 0x7fb53c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 -- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb53c0038e0 msgr2=0x7fb53c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb53c0038e0 0x7fb53c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 -- 172.21.15.92:0/1404542177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0033f0 con 0x7fb53c004690 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb53c0038e0 0x7fb53c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54effd700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53c004690 0x7fb53c004b10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb550084320 tx=0x7fb5380077c0).ready entity=mon.0 client_cookie=5f0339749d56e2e1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53800cab0 con 0x7fb53c004690 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53800cc10 con 0x7fb53c004690 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.529+0000 7fb54e7fc700 1 -- 172.21.15.92:0/1404542177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380070d0 con 0x7fb53c004690 2023-05-16T07:02:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb555b2b700 1 -- 172.21.15.92:0/1404542177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53c004690 msgr2=0x7fb53c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1404542177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53c004690 0x7fb53c004b10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb550084320 tx=0x7fb5380077c0).stop 2023-05-16T07:02:37.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb54ffff700 1 -- 172.21.15.92:0/1404542177 reap_dead start 2023-05-16T07:02:37.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb555b2b700 1 -- 172.21.15.92:0/1404542177 shutdown_connections 2023-05-16T07:02:37.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb555b2b700 1 -- 172.21.15.92:0/1404542177 >> 172.21.15.92:0/1404542177 conn(0x7fb550067920 msgr2=0x7fb5500cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:37.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb555b2b700 1 -- 172.21.15.92:0/1404542177 shutdown_connections 2023-05-16T07:02:37.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.530+0000 7fb555b2b700 1 -- 172.21.15.92:0/1404542177 wait complete. 2023-05-16T07:02:37.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.531+0000 7fb555b2b700 1 Processor -- start 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.531+0000 7fb555b2b700 1 -- start start 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb555b2b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb53c004690 0x7fb5500c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb555b2b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 0x7fb55014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb555b2b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb55014f7f0 0x7fb550152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb555b2b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500c2f20 con 0x7fb550151300 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb555b2b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb550156710 con 0x7fb55014f7f0 2023-05-16T07:02:37.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb555b2b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb550155820 con 0x7fb53c004690 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 0x7fb55014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 0x7fb55014f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40958/0 (socket says 172.21.15.92:40958) 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 -- 172.21.15.92:0/1358380413 learned_addr learned my addr 172.21.15.92:0/1358380413 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb53c004690 0x7fb5500c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54ffff700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb55014f7f0 0x7fb550152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 -- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb53c004690 msgr2=0x7fb5500c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb53c004690 0x7fb5500c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 -- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb55014f7f0 msgr2=0x7fb550152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb55014f7f0 0x7fb550152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 -- 172.21.15.92:0/1358380413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550155b00 con 0x7fb550151300 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb53c004690 0x7fb5500c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb54effd700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 0x7fb55014f300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fb550067e30 tx=0x7fb538015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.532+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538001810 con 0x7fb550151300 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.533+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380046b0 con 0x7fb550151300 2023-05-16T07:02:37.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.533+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538004a30 con 0x7fb550151300 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.533+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb550155de0 con 0x7fb550151300 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.533+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5501563a0 con 0x7fb550151300 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.533+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb550066730 con 0x7fb550151300 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.534+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb53801d070 con 0x7fb550151300 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.534+0000 7fb5477fe700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb53005ed90 0x7fb530061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.534+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb53807aa50 con 0x7fb550151300 2023-05-16T07:02:37.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.534+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb53005ed90 0x7fb530061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.535+0000 7fb54f7fe700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb53005ed90 0x7fb530061250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fb5400094d0 tx=0x7fb54000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.538+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb53804bff0 con 0x7fb550151300 2023-05-16T07:02:37.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.699+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb550156850 con 0x7fb550151300 2023-05-16T07:02:37.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.699+0000 7fb5477fe700 1 -- 172.21.15.92:0/1358380413 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb538025020 con 0x7fb550151300 2023-05-16T07:02:37.704 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb53005ed90 msgr2=0x7fb530061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb53005ed90 0x7fb530061250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fb5400094d0 tx=0x7fb54000b040).stop 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 msgr2=0x7fb55014f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 0x7fb55014f300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fb550067e30 tx=0x7fb538015000).stop 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 shutdown_connections 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb53005ed90 0x7fb530061250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb53c004690 0x7fb5500c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb55014f7f0 0x7fb550152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 --2- 172.21.15.92:0/1358380413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb550151300 0x7fb55014f300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 >> 172.21.15.92:0/1358380413 conn(0x7fb550067920 msgr2=0x7fb5500c10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 shutdown_connections 2023-05-16T07:02:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.701+0000 7fb555b2b700 1 -- 172.21.15.92:0/1358380413 wait complete. 2023-05-16T07:02:37.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:37.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.827+0000 7f2268f2f700 1 Processor -- start 2023-05-16T07:02:37.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.827+0000 7f2268f2f700 1 -- start start 2023-05-16T07:02:37.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.827+0000 7f2268f2f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2264154090 0x7f22640aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2268f2f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22640685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2268f2f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22640c94d0 con 0x7f2264154470 2023-05-16T07:02:37.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2268f2f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22640cb7e0 con 0x7f2264154090 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2268f2f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2264068aa0 con 0x7f2264068160 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2261d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2264154470 0x7f22640a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51714/0 (socket says 172.21.15.92:51714) 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2261d9b700 1 -- 172.21.15.92:0/653824827 learned_addr learned my addr 172.21.15.92:0/653824827 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226259c700 1 --2- 172.21.15.92:0/653824827 >> v2:172.21.15.158:3300/0 conn(0x7f2264154090 0x7f22640aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2262d9d700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22640685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224829326 0 0) 0x7f22640c94d0 con 0x7f2264154470 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22500033f0 con 0x7f2264154470 2023-05-16T07:02:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2353521500 0 0) 0x7f22500033f0 con 0x7f2264154470 2023-05-16T07:02:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 msgr2=0x7f22640685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22640685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 >> v2:172.21.15.158:3300/0 conn(0x7f2264154090 msgr2=0x7f22640aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 --2- 172.21.15.92:0/653824827 >> v2:172.21.15.158:3300/0 conn(0x7f2264154090 0x7f22640aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22640c88c0 con 0x7f2264154470 2023-05-16T07:02:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f224c001470 con 0x7f2264154470 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2262d9d700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22640685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226259c700 1 --2- 172.21.15.92:0/653824827 >> v2:172.21.15.158:3300/0 conn(0x7f2264154090 0x7f22640aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 >> v1:172.21.15.92:6789/0 conn(0x7f2264154470 legacy=0x7f22640a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22500038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2250003dd0 0x7f2250004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2250004b80 0x7f2250005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22640cb7e0 con 0x7f2250004b80 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2264068aa0 con 0x7f2250003dd0 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225000b8e0 con 0x7f2264068160 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2262d9d700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2250004b80 0x7f2250005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f226259c700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22500038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.828+0000 7f2261d9b700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2250003dd0 0x7f2250004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 -- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 msgr2=0x7f22500038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2264068160 0x7f22500038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 -- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2250003dd0 msgr2=0x7f2250004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2250003dd0 0x7f2250004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 -- 172.21.15.92:0/653824827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c0092a0 con 0x7f2250004b80 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2250004b80 0x7f2250005000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f225c009c70 tx=0x7f225c009060).ready entity=mon.0 client_cookie=5c734675b662258a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0095c0 con 0x7f2250004b80 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c00e690 con 0x7f2250004b80 2023-05-16T07:02:37.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f226159a700 1 -- 172.21.15.92:0/653824827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00ea10 con 0x7f2250004b80 2023-05-16T07:02:37.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2268f2f700 1 -- 172.21.15.92:0/653824827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2250004b80 msgr2=0x7f2250005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2268f2f700 1 --2- 172.21.15.92:0/653824827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2250004b80 0x7f2250005000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f225c009c70 tx=0x7f225c009060).stop 2023-05-16T07:02:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2262d9d700 1 -- 172.21.15.92:0/653824827 reap_dead start 2023-05-16T07:02:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2268f2f700 1 -- 172.21.15.92:0/653824827 shutdown_connections 2023-05-16T07:02:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.829+0000 7f2268f2f700 1 -- 172.21.15.92:0/653824827 >> 172.21.15.92:0/653824827 conn(0x7f2264067920 msgr2=0x7f22640cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.830+0000 7f2268f2f700 1 -- 172.21.15.92:0/653824827 shutdown_connections 2023-05-16T07:02:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.830+0000 7f2268f2f700 1 -- 172.21.15.92:0/653824827 wait complete. 2023-05-16T07:02:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 Processor -- start 2023-05-16T07:02:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 -- start start 2023-05-16T07:02:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2250004b80 0x7f22641449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2264156e30 0x7f22641534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 0x7f22640c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22640c3160 con 0x7f22640c3660 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226414fa30 con 0x7f2264156e30 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2268f2f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2264144ed0 con 0x7f2250004b80 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2262d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 0x7f22640c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2262d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 0x7f22640c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40980/0 (socket says 172.21.15.92:40980) 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2262d9d700 1 -- 172.21.15.92:0/1188065728 learned_addr learned my addr 172.21.15.92:0/1188065728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.831+0000 7f2261d9b700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2264156e30 0x7f22641534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f226259c700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2250004b80 0x7f22641449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2262d9d700 1 -- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2250004b80 msgr2=0x7f22641449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2262d9d700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2250004b80 0x7f22641449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2262d9d700 1 -- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2264156e30 msgr2=0x7f22641534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2262d9d700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2264156e30 0x7f22641534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2262d9d700 1 -- 172.21.15.92:0/1188065728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22641569a0 con 0x7f22640c3660 2023-05-16T07:02:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f226259c700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2250004b80 0x7f22641449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2262d9d700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 0x7f22640c3ae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f225c00e4b0 tx=0x7f225c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00e800 con 0x7f22640c3660 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c00e960 con 0x7f22640c3660 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22640c1d70 con 0x7f22640c3660 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c026650 con 0x7f22640c3660 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.832+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22640c1fe0 con 0x7f22640c3660 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.833+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2264066730 con 0x7f22640c3660 2023-05-16T07:02:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.833+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f225c026a40 con 0x7f22640c3660 2023-05-16T07:02:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.833+0000 7f225a7fc700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f224405edc0 0x7f2244061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.833+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f225c07ba90 con 0x7f22640c3660 2023-05-16T07:02:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.834+0000 7f226259c700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f224405edc0 0x7f2244061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.834+0000 7f226259c700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f224405edc0 0x7f2244061280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f22540094a0 tx=0x7f225400b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:37.837+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f225c04cf20 con 0x7f22640c3660 2023-05-16T07:02:38.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.002+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2264153990 con 0x7f22640c3660 2023-05-16T07:02:38.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.003+0000 7f225a7fc700 1 -- 172.21.15.92:0/1188065728 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f225c016030 con 0x7f22640c3660 2023-05-16T07:02:38.007 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:38.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f224405edc0 msgr2=0x7f2244061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f224405edc0 0x7f2244061280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f22540094a0 tx=0x7f225400b040).stop 2023-05-16T07:02:38.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 msgr2=0x7f22640c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 0x7f22640c3ae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f225c00e4b0 tx=0x7f225c01a000).stop 2023-05-16T07:02:38.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 shutdown_connections 2023-05-16T07:02:38.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f224405edc0 0x7f2244061280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2250004b80 0x7f22641449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22640c3660 0x7f22640c3ae0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 --2- 172.21.15.92:0/1188065728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2264156e30 0x7f22641534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 >> 172.21.15.92:0/1188065728 conn(0x7f2264067920 msgr2=0x7f22640c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 shutdown_connections 2023-05-16T07:02:38.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.005+0000 7f2268f2f700 1 -- 172.21.15.92:0/1188065728 wait complete. 2023-05-16T07:02:38.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.126+0000 7f95b9107700 1 Processor -- start 2023-05-16T07:02:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.126+0000 7f95b9107700 1 -- start start 2023-05-16T07:02:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b9107700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95b4154550 0x7f95b40696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b9107700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4069b90 0x7f95b40c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b9107700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b40c0660 con 0x7f95b4154170 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b9107700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b406a000 con 0x7f95b4154550 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b9107700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40bfd30 con 0x7f95b4069b90 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95b4154550 0x7f95b40696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95b4154550 0x7f95b40696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40986/0 (socket says 172.21.15.92:40986) 2023-05-16T07:02:38.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.127+0000 7f95b259c700 1 -- 172.21.15.92:0/3016044796 learned_addr learned my addr 172.21.15.92:0/3016044796 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b359e700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4069b90 0x7f95b40c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b259c700 1 -- 172.21.15.92:0/3016044796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4069b90 msgr2=0x7f95b40c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b259c700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4069b90 0x7f95b40c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b259c700 1 -- 172.21.15.92:0/3016044796 >> v1:172.21.15.158:6789/0 conn(0x7f95b4154170 legacy=0x7f95b40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b259c700 1 -- 172.21.15.92:0/3016044796 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b40c0010 con 0x7f95b4154550 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b359e700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4069b90 0x7f95b40c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:38.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b259c700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154550 0x7f95b40696a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f959c006fd0 tx=0x7f959c0090f0).ready entity=mon.0 client_cookie=319c6d1c630dd5ed server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b1d9b700 1 -- 172.21.15.92:0/3016044796 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00b600 con 0x7f95b4154550 2023-05-16T07:02:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b1d9b700 1 -- 172.21.15.92:0/3016044796 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00b760 con 0x7f95b4154550 2023-05-16T07:02:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b1d9b700 1 -- 172.21.15.92:0/3016044796 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00bb00 con 0x7f95b4154550 2023-05-16T07:02:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b9107700 1 -- 172.21.15.92:0/3016044796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154550 msgr2=0x7f95b40696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b9107700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154550 0x7f95b40696a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f959c006fd0 tx=0x7f959c0090f0).stop 2023-05-16T07:02:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b9107700 1 -- 172.21.15.92:0/3016044796 shutdown_connections 2023-05-16T07:02:38.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b9107700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4069b90 0x7f95b40c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b9107700 1 --2- 172.21.15.92:0/3016044796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154550 0x7f95b40696a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.128+0000 7f95b9107700 1 -- 172.21.15.92:0/3016044796 >> 172.21.15.92:0/3016044796 conn(0x7f95b40cb700 msgr2=0x7f95b40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.129+0000 7f95b9107700 1 -- 172.21.15.92:0/3016044796 shutdown_connections 2023-05-16T07:02:38.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.129+0000 7f95b9107700 1 -- 172.21.15.92:0/3016044796 wait complete. 2023-05-16T07:02:38.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 Processor -- start 2023-05-16T07:02:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 -- start start 2023-05-16T07:02:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95b4069b90 0x7f95b4150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 0x7f95b4151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4154550 0x7f95b41680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40adc30 con 0x7f95b4154170 2023-05-16T07:02:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4150c30 con 0x7f95b4069b90 2023-05-16T07:02:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.130+0000 7f95b9107700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4150f10 con 0x7f95b4154550 2023-05-16T07:02:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 0x7f95b4151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 0x7f95b4151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40992/0 (socket says 172.21.15.92:40992) 2023-05-16T07:02:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 -- 172.21.15.92:0/1174664982 learned_addr learned my addr 172.21.15.92:0/1174664982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b359e700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4154550 0x7f95b41680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b2d9d700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95b4069b90 0x7f95b4150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 -- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4154550 msgr2=0x7f95b41680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4154550 0x7f95b41680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 -- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95b4069b90 msgr2=0x7f95b4150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95b4069b90 0x7f95b4150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 -- 172.21.15.92:0/1174664982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009370 con 0x7f95b4154170 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b2d9d700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95b4069b90 0x7f95b4150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b259c700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 0x7f95b4151c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f959c011000 tx=0x7f959c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c006930 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c002f30 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b41511f0 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c024470 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.131+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b41508f0 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.132+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b40666c0 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.133+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f959c00f070 con 0x7f95b4154170 2023-05-16T07:02:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.133+0000 7f95aaffd700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f959405ed90 0x7f9594061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.133+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f959c07bab0 con 0x7f95b4154170 2023-05-16T07:02:38.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.133+0000 7f95b2d9d700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f959405ed90 0x7f9594061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.134+0000 7f95b2d9d700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f959405ed90 0x7f9594061250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f95a4007fd0 tx=0x7f95a4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.137+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f959c04d110 con 0x7f95b4154170 2023-05-16T07:02:38.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.339+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95b406c350 con 0x7f95b4154170 2023-05-16T07:02:38.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.339+0000 7f95aaffd700 1 -- 172.21.15.92:0/1174664982 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f959c0506f0 con 0x7f95b4154170 2023-05-16T07:02:38.343 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:38.343 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:38.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f959405ed90 msgr2=0x7f9594061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f959405ed90 0x7f9594061250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f95a4007fd0 tx=0x7f95a4009040).stop 2023-05-16T07:02:38.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 msgr2=0x7f95b4151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 0x7f95b4151c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f959c011000 tx=0x7f959c017000).stop 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 shutdown_connections 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f959405ed90 0x7f9594061250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95b4154550 0x7f95b41680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95b4069b90 0x7f95b4150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 --2- 172.21.15.92:0/1174664982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95b4154170 0x7f95b4151c30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 >> 172.21.15.92:0/1174664982 conn(0x7f95b40cb700 msgr2=0x7f95b40c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 shutdown_connections 2023-05-16T07:02:38.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.341+0000 7f95b9107700 1 -- 172.21.15.92:0/1174664982 wait complete. 2023-05-16T07:02:38.347 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:38.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:38.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.464+0000 7fc74c698700 1 Processor -- start 2023-05-16T07:02:38.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.464+0000 7fc74c698700 1 -- start start 2023-05-16T07:02:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74c698700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc744154170 0x7fc7440c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74c698700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc7440c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74c698700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440c0660 con 0x7fc7440696a0 2023-05-16T07:02:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74c698700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7440c1b00 con 0x7fc744154170 2023-05-16T07:02:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74c698700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440bfd30 con 0x7fc744154550 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74ac35700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7440696a0 0x7fc744157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51748/0 (socket says 172.21.15.92:51748) 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74ac35700 1 -- 172.21.15.92:0/996683728 learned_addr learned my addr 172.21.15.92:0/996683728 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc749c33700 1 --2- 172.21.15.92:0/996683728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc7440c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.465+0000 7fc74a434700 1 --2- 172.21.15.92:0/996683728 >> v2:172.21.15.158:3300/0 conn(0x7fc744154170 0x7fc7440c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749432700 1 -- 172.21.15.92:0/996683728 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459519658 0 0) 0x7fc7440c0660 con 0x7fc7440696a0 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749432700 1 -- 172.21.15.92:0/996683728 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc72c0033f0 con 0x7fc7440696a0 2023-05-16T07:02:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749c33700 1 -- 172.21.15.92:0/996683728 >> v2:172.21.15.158:3300/0 conn(0x7fc744154170 msgr2=0x7fc7440c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749c33700 1 --2- 172.21.15.92:0/996683728 >> v2:172.21.15.158:3300/0 conn(0x7fc744154170 0x7fc7440c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749c33700 1 -- 172.21.15.92:0/996683728 >> v1:172.21.15.92:6789/0 conn(0x7fc7440696a0 legacy=0x7fc744157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749c33700 1 -- 172.21.15.92:0/996683728 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440c0010 con 0x7fc744154550 2023-05-16T07:02:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc74a434700 1 --2- 172.21.15.92:0/996683728 >> v2:172.21.15.158:3300/0 conn(0x7fc744154170 0x7fc7440c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749c33700 1 --2- 172.21.15.92:0/996683728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc7440c6160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc738009760 tx=0x7fc738009060).ready entity=mon.2 client_cookie=278e1865a58fa9b6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.466+0000 7fc749432700 1 -- 172.21.15.92:0/996683728 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380090d0 con 0x7fc744154550 2023-05-16T07:02:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.468+0000 7fc749432700 1 -- 172.21.15.92:0/996683728 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73800d4a0 con 0x7fc744154550 2023-05-16T07:02:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.468+0000 7fc749432700 1 -- 172.21.15.92:0/996683728 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800d820 con 0x7fc744154550 2023-05-16T07:02:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 -- 172.21.15.92:0/996683728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 msgr2=0x7fc7440c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 --2- 172.21.15.92:0/996683728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc7440c6160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc738009760 tx=0x7fc738009060).stop 2023-05-16T07:02:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 -- 172.21.15.92:0/996683728 shutdown_connections 2023-05-16T07:02:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 --2- 172.21.15.92:0/996683728 >> v2:172.21.15.158:3300/0 conn(0x7fc744154170 0x7fc7440c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 --2- 172.21.15.92:0/996683728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc7440c6160 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 -- 172.21.15.92:0/996683728 >> 172.21.15.92:0/996683728 conn(0x7fc7440cb700 msgr2=0x7fc7440c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 -- 172.21.15.92:0/996683728 shutdown_connections 2023-05-16T07:02:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.469+0000 7fc74c698700 1 -- 172.21.15.92:0/996683728 wait complete. 2023-05-16T07:02:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.470+0000 7fc74c698700 1 Processor -- start 2023-05-16T07:02:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.470+0000 7fc74c698700 1 -- start start 2023-05-16T07:02:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7440696a0 0x7fc7440a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 0x7fc7440c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc74414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441505f0 con 0x7fc744154170 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441508d0 con 0x7fc7440696a0 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc744165140 con 0x7fc744154550 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 0x7fc7440c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 0x7fc7440c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41004/0 (socket says 172.21.15.92:41004) 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 -- 172.21.15.92:0/4103360683 learned_addr learned my addr 172.21.15.92:0/4103360683 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74ac35700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc74414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74a434700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7440696a0 0x7fc7440a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 -- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 msgr2=0x7fc74414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc74414fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 -- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7440696a0 msgr2=0x7fc7440a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7440696a0 0x7fc7440a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 -- 172.21.15.92:0/4103360683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734009a20 con 0x7fc744154170 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc749c33700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 0x7fc7440c1d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fc73800d260 tx=0x7fc738016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738011860 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7380119c0 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7380092a0 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738011d40 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.471+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc744165700 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.472+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc744158c50 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.472+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc73800f070 con 0x7fc744154170 2023-05-16T07:02:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.473+0000 7fc7427fc700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc73005ed90 0x7fc730061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.473+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc73807b1b0 con 0x7fc744154170 2023-05-16T07:02:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.473+0000 7fc74a434700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc73005ed90 0x7fc730061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.473+0000 7fc74a434700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc73005ed90 0x7fc730061250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fc7340099f0 tx=0x7fc73400b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.477+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc73804c750 con 0x7fc744154170 2023-05-16T07:02:38.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.638+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7440666c0 con 0x7fc744154170 2023-05-16T07:02:38.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.639+0000 7fc7427fc700 1 -- 172.21.15.92:0/4103360683 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fc73804fd70 con 0x7fc744154170 2023-05-16T07:02:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:38.644 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc73005ed90 msgr2=0x7fc730061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc73005ed90 0x7fc730061250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fc7340099f0 tx=0x7fc73400b040).stop 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 msgr2=0x7fc7440c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 0x7fc7440c1d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fc73800d260 tx=0x7fc738016000).stop 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 shutdown_connections 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc73005ed90 0x7fc730061250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc744154550 0x7fc74414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7440696a0 0x7fc7440a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 --2- 172.21.15.92:0/4103360683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc744154170 0x7fc7440c1d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 >> 172.21.15.92:0/4103360683 conn(0x7fc7440cb700 msgr2=0x7fc7440c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 shutdown_connections 2023-05-16T07:02:38.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.641+0000 7fc74c698700 1 -- 172.21.15.92:0/4103360683 wait complete. 2023-05-16T07:02:38.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:38.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:38.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.767+0000 7fc9994d4700 1 Processor -- start 2023-05-16T07:02:38.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.767+0000 7fc9994d4700 1 -- start start 2023-05-16T07:02:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.767+0000 7fc9994d4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 0x7fc9940c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.767+0000 7fc9994d4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9940669d0 con 0x7fc994154d60 2023-05-16T07:02:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc9994d4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9940c2770 con 0x7fc994150dc0 2023-05-16T07:02:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc9994d4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940666f0 con 0x7fc994154980 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc9937fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc994154d60 0x7fc9941506b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51760/0 (socket says 172.21.15.92:51760) 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc9937fe700 1 -- 172.21.15.92:0/1925562454 learned_addr learned my addr 172.21.15.92:0/1925562454 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc993fff700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 0x7fc9940c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006983762 0 0) 0x7fc9940669d0 con 0x7fc994154d60 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc97c0033f0 con 0x7fc994154d60 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2394492877 0 0) 0x7fc97c0033f0 con 0x7fc994154d60 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 msgr2=0x7fc9940c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 0x7fc9940c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 >> v1:172.21.15.158:6789/0 conn(0x7fc994150dc0 legacy=0x7fc9940bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940c2120 con 0x7fc994154d60 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc9840015f0 con 0x7fc994154d60 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 >> v1:172.21.15.92:6789/0 conn(0x7fc994154d60 legacy=0x7fc9941506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c003910 0x7fc97c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 0x7fc97c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c004b80 0x7fc97c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940666f0 con 0x7fc97c003910 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c00b910 con 0x7fc97c004b80 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c00bad0 con 0x7fc994154980 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc993fff700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c003910 0x7fc97c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc9937fe700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 0x7fc97c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc998cd3700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c004b80 0x7fc97c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc993fff700 1 -- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 msgr2=0x7fc97c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.768+0000 7fc993fff700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994154980 0x7fc97c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc993fff700 1 -- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c004b80 msgr2=0x7fc97c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc993fff700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c004b80 0x7fc97c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc993fff700 1 -- 172.21.15.92:0/1925562454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0033f0 con 0x7fc97c003910 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc998cd3700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c004b80 0x7fc97c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc993fff700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c003910 0x7fc97c003d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fc988009a60 tx=0x7fc988009060).ready entity=mon.0 client_cookie=4671dbd7a06a21a6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988009ea0 con 0x7fc97c003910 2023-05-16T07:02:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988009430 con 0x7fc97c003910 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc992ffd700 1 -- 172.21.15.92:0/1925562454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800e870 con 0x7fc97c003910 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc9994d4700 1 -- 172.21.15.92:0/1925562454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c003910 msgr2=0x7fc97c003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc9994d4700 1 --2- 172.21.15.92:0/1925562454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc97c003910 0x7fc97c003d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fc988009a60 tx=0x7fc988009060).stop 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc998cd3700 1 -- 172.21.15.92:0/1925562454 reap_dead start 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc9994d4700 1 -- 172.21.15.92:0/1925562454 shutdown_connections 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc9994d4700 1 -- 172.21.15.92:0/1925562454 >> 172.21.15.92:0/1925562454 conn(0x7fc9940c7a40 msgr2=0x7fc9940c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc9994d4700 1 -- 172.21.15.92:0/1925562454 shutdown_connections 2023-05-16T07:02:38.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.769+0000 7fc9994d4700 1 -- 172.21.15.92:0/1925562454 wait complete. 2023-05-16T07:02:38.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.770+0000 7fc9994d4700 1 Processor -- start 2023-05-16T07:02:38.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.770+0000 7fc9994d4700 1 -- start start 2023-05-16T07:02:38.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9994d4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c003910 0x7fc9940beae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9994d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 0x7fc9941537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9994d4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994153cd0 0x7fc9940abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9994d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994154150 con 0x7fc9941522b0 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9994d4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994153350 con 0x7fc97c003910 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9994d4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc994152740 con 0x7fc994153cd0 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 0x7fc9941537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 0x7fc9941537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41024/0 (socket says 172.21.15.92:41024) 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc9937fe700 1 -- 172.21.15.92:0/194709365 learned_addr learned my addr 172.21.15.92:0/194709365 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc998cd3700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994153cd0 0x7fc9940abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.771+0000 7fc993fff700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c003910 0x7fc9940beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9937fe700 1 -- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994153cd0 msgr2=0x7fc9940abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9937fe700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994153cd0 0x7fc9940abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9937fe700 1 -- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c003910 msgr2=0x7fc9940beae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9937fe700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c003910 0x7fc9940beae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9937fe700 1 -- 172.21.15.92:0/194709365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994152a20 con 0x7fc9941522b0 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc993fff700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c003910 0x7fc9940beae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9937fe700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 0x7fc9941537e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fc98400e840 tx=0x7fc984009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984000f70 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984007320 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994152d00 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984005880 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.772+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9940befd0 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.773+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994153500 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.773+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc984005bf0 con 0x7fc9941522b0 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.773+0000 7fc97bfff700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc98005ed90 0x7fc980061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:38.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.774+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc984079820 con 0x7fc9941522b0 2023-05-16T07:02:38.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.774+0000 7fc993fff700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc98005ed90 0x7fc980061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:38.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.774+0000 7fc993fff700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc98005ed90 0x7fc980061250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc988006d50 tx=0x7fc98801a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:38.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.777+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc9840498f0 con 0x7fc9941522b0 2023-05-16T07:02:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.939+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc994047150 con 0x7fc9941522b0 2023-05-16T07:02:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.939+0000 7fc97bfff700 1 -- 172.21.15.92:0/194709365 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc994153500 con 0x7fc9941522b0 2023-05-16T07:02:38.943 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc98005ed90 msgr2=0x7fc980061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc98005ed90 0x7fc980061250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc988006d50 tx=0x7fc98801a000).stop 2023-05-16T07:02:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 msgr2=0x7fc9941537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 0x7fc9941537e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fc98400e840 tx=0x7fc984009040).stop 2023-05-16T07:02:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 shutdown_connections 2023-05-16T07:02:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc98005ed90 0x7fc980061250 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc994153cd0 0x7fc9940abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc97c003910 0x7fc9940beae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 --2- 172.21.15.92:0/194709365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9941522b0 0x7fc9941537e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 >> 172.21.15.92:0/194709365 conn(0x7fc9940c7a40 msgr2=0x7fc9940c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 shutdown_connections 2023-05-16T07:02:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:38.941+0000 7fc9994d4700 1 -- 172.21.15.92:0/194709365 wait complete. 2023-05-16T07:02:38.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.065+0000 7f0924eb8700 1 Processor -- start 2023-05-16T07:02:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.065+0000 7f0924eb8700 1 -- start start 2023-05-16T07:02:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f0924eb8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0920154090 0x7f09200aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f0924eb8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0920154470 0x7f09200a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f0924eb8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09200aa150 0x7f0920068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f0924eb8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200aa570 con 0x7f09200aa150 2023-05-16T07:02:39.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f0924eb8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200c94d0 con 0x7f0920154470 2023-05-16T07:02:39.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f0924eb8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200cb7e0 con 0x7f0920154090 2023-05-16T07:02:39.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09200aa150 0x7f0920068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09200aa150 0x7f0920068160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41032/0 (socket says 172.21.15.92:41032) 2023-05-16T07:02:39.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 -- 172.21.15.92:0/3199689947 learned_addr learned my addr 172.21.15.92:0/3199689947 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091dd9b700 1 --2- 172.21.15.92:0/3199689947 >> v2:172.21.15.158:3300/0 conn(0x7f0920154470 0x7f09200a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091e59c700 1 --2- 172.21.15.92:0/3199689947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0920154090 0x7f09200aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 -- 172.21.15.92:0/3199689947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0920154090 msgr2=0x7f09200aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 --2- 172.21.15.92:0/3199689947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0920154090 0x7f09200aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 -- 172.21.15.92:0/3199689947 >> v2:172.21.15.158:3300/0 conn(0x7f0920154470 msgr2=0x7f09200a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 --2- 172.21.15.92:0/3199689947 >> v2:172.21.15.158:3300/0 conn(0x7f0920154470 0x7f09200a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.066+0000 7f091ed9d700 1 -- 172.21.15.92:0/3199689947 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200c88c0 con 0x7f09200aa150 2023-05-16T07:02:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f091ed9d700 1 --2- 172.21.15.92:0/3199689947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09200aa150 0x7f0920068160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f0920083c50 tx=0x7f09180090f0).ready entity=mon.0 client_cookie=bfdf094e9841806 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f091d59a700 1 -- 172.21.15.92:0/3199689947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800b600 con 0x7f09200aa150 2023-05-16T07:02:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f091d59a700 1 -- 172.21.15.92:0/3199689947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091800b760 con 0x7f09200aa150 2023-05-16T07:02:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f091d59a700 1 -- 172.21.15.92:0/3199689947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800bae0 con 0x7f09200aa150 2023-05-16T07:02:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 -- 172.21.15.92:0/3199689947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09200aa150 msgr2=0x7f0920068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 --2- 172.21.15.92:0/3199689947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09200aa150 0x7f0920068160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f0920083c50 tx=0x7f09180090f0).stop 2023-05-16T07:02:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 -- 172.21.15.92:0/3199689947 shutdown_connections 2023-05-16T07:02:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 --2- 172.21.15.92:0/3199689947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09200aa150 0x7f0920068160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 --2- 172.21.15.92:0/3199689947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0920154090 0x7f09200aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 --2- 172.21.15.92:0/3199689947 >> v2:172.21.15.158:3300/0 conn(0x7f0920154470 0x7f09200a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 -- 172.21.15.92:0/3199689947 >> 172.21.15.92:0/3199689947 conn(0x7f0920067920 msgr2=0x7f09200cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 -- 172.21.15.92:0/3199689947 shutdown_connections 2023-05-16T07:02:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.067+0000 7f0924eb8700 1 -- 172.21.15.92:0/3199689947 wait complete. 2023-05-16T07:02:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.068+0000 7f0924eb8700 1 Processor -- start 2023-05-16T07:02:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.068+0000 7f0924eb8700 1 -- start start 2023-05-16T07:02:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f0924eb8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09200aa150 0x7f092014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f0924eb8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 0x7f0920152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f0924eb8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0920154470 0x7f09201522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f0924eb8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0920153070 con 0x7f0920154090 2023-05-16T07:02:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f0924eb8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0920152790 con 0x7f0920154470 2023-05-16T07:02:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f0924eb8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200aef00 con 0x7f09200aa150 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f091dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 0x7f0920152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f091dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 0x7f0920152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41036/0 (socket says 172.21.15.92:41036) 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f091dd9b700 1 -- 172.21.15.92:0/2138198023 learned_addr learned my addr 172.21.15.92:0/2138198023 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f091ed9d700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0920154470 0x7f09201522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.069+0000 7f091e59c700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09200aa150 0x7f092014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f091dd9b700 1 -- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09200aa150 msgr2=0x7f092014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f091dd9b700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09200aa150 0x7f092014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f091dd9b700 1 -- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0920154470 msgr2=0x7f09201522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f091dd9b700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0920154470 0x7f09201522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f091dd9b700 1 -- 172.21.15.92:0/2138198023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918009370 con 0x7f0920154090 2023-05-16T07:02:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f091dd9b700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 0x7f0920152b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f090800d8c0 tx=0x7f0908008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908011590 con 0x7f0920154090 2023-05-16T07:02:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09080116f0 con 0x7f0920154090 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09200ae5f0 con 0x7f0920154090 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908011a70 con 0x7f0920154090 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.070+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09200ae8b0 con 0x7f0920154090 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.071+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0920066730 con 0x7f0920154090 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.071+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f090801c460 con 0x7f0920154090 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.071+0000 7f09167fc700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090005ed90 0x7f0900061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.072+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f09080798c0 con 0x7f0920154090 2023-05-16T07:02:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.072+0000 7f091e59c700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090005ed90 0x7f0900061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.072+0000 7f091e59c700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090005ed90 0x7f0900061250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f09100075a0 tx=0x7f09100097e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.076+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f090804ae00 con 0x7f0920154090 2023-05-16T07:02:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.238+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09200c53f0 con 0x7f0920154090 2023-05-16T07:02:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.239+0000 7f09167fc700 1 -- 172.21.15.92:0/2138198023 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f090804e420 con 0x7f0920154090 2023-05-16T07:02:39.243 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090005ed90 msgr2=0x7f0900061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090005ed90 0x7f0900061250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f09100075a0 tx=0x7f09100097e0).stop 2023-05-16T07:02:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 msgr2=0x7f0920152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 0x7f0920152b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f090800d8c0 tx=0x7f0908008040).stop 2023-05-16T07:02:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 shutdown_connections 2023-05-16T07:02:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090005ed90 0x7f0900061250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09200aa150 0x7f092014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0920154470 0x7f09201522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 --2- 172.21.15.92:0/2138198023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0920154090 0x7f0920152b80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 >> 172.21.15.92:0/2138198023 conn(0x7f0920067920 msgr2=0x7f09200ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.240+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 shutdown_connections 2023-05-16T07:02:39.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.241+0000 7f0924eb8700 1 -- 172.21.15.92:0/2138198023 wait complete. 2023-05-16T07:02:39.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.365+0000 7fcb11fd9700 1 Processor -- start 2023-05-16T07:02:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.365+0000 7fcb11fd9700 1 -- start start 2023-05-16T07:02:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb11fd9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 0x7fcb0c0622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb11fd9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c14ead0 con 0x7fcb0c0627a0 2023-05-16T07:02:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb11fd9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c154cb0 con 0x7fcb0c14e6f0 2023-05-16T07:02:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb11fd9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c062c10 con 0x7fcb0c155140 2023-05-16T07:02:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb0bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcb0c0627a0 0x7fcb0c063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51794/0 (socket says 172.21.15.92:51794) 2023-05-16T07:02:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb0bfff700 1 -- 172.21.15.92:0/1461094087 learned_addr learned my addr 172.21.15.92:0/1461094087 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb0affd700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 0x7fcb0c0622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619692034 0 0) 0x7fcb0c14ead0 con 0x7fcb0c0627a0 2023-05-16T07:02:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.366+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf40033f0 con 0x7fcb0c0627a0 2023-05-16T07:02:39.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1930499981 0 0) 0x7fcaf40033f0 con 0x7fcb0c0627a0 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 msgr2=0x7fcb0c0622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 0x7fcb0c0622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 >> v1:172.21.15.158:6789/0 conn(0x7fcb0c14e6f0 legacy=0x7fcb0c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0c6020 con 0x7fcb0c0627a0 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fcb040017f0 con 0x7fcb0c0627a0 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 >> v1:172.21.15.92:6789/0 conn(0x7fcb0c0627a0 legacy=0x7fcb0c063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcaf4003910 0x7fcaf4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 0x7fcaf4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf4004b80 0x7fcaf4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c062c10 con 0x7fcaf4004b80 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf400b940 con 0x7fcaf4003910 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf400bb00 con 0x7fcb0c155140 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf4004b80 0x7fcaf4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0b7fe700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcaf4003910 0x7fcaf4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0affd700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 0x7fcaf4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 -- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 msgr2=0x7fcaf4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb0c155140 0x7fcaf4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 -- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcaf4003910 msgr2=0x7fcaf4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcaf4003910 0x7fcaf4003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 -- 172.21.15.92:0/1461094087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf40033f0 con 0x7fcaf4004b80 2023-05-16T07:02:39.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0bfff700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf4004b80 0x7fcaf4005000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fcb0400c480 tx=0x7fcb0400c790).ready entity=mon.0 client_cookie=f2e8e5e364f9a990 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04002030 con 0x7fcaf4004b80 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400ce20 con 0x7fcaf4004b80 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.367+0000 7fcb0a7fc700 1 -- 172.21.15.92:0/1461094087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005d30 con 0x7fcaf4004b80 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb11fd9700 1 -- 172.21.15.92:0/1461094087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf4004b80 msgr2=0x7fcaf4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/1461094087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf4004b80 0x7fcaf4005000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fcb0400c480 tx=0x7fcb0400c790).stop 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb0bfff700 1 -- 172.21.15.92:0/1461094087 reap_dead start 2023-05-16T07:02:39.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb11fd9700 1 -- 172.21.15.92:0/1461094087 shutdown_connections 2023-05-16T07:02:39.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb11fd9700 1 -- 172.21.15.92:0/1461094087 >> 172.21.15.92:0/1461094087 conn(0x7fcb0c05e820 msgr2=0x7fcb0c0c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb11fd9700 1 -- 172.21.15.92:0/1461094087 shutdown_connections 2023-05-16T07:02:39.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.368+0000 7fcb11fd9700 1 -- 172.21.15.92:0/1461094087 wait complete. 2023-05-16T07:02:39.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.369+0000 7fcb11fd9700 1 Processor -- start 2023-05-16T07:02:39.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.369+0000 7fcb11fd9700 1 -- start start 2023-05-16T07:02:39.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb11fd9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcaf4004b80 0x7fcb0c0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb11fd9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 0x7fcb0c14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb11fd9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb0c0adcd0 0x7fcb0c0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb11fd9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c14faa0 con 0x7fcb0c164940 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb11fd9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0ac780 con 0x7fcb0c0adcd0 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb11fd9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c165210 con 0x7fcaf4004b80 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 0x7fcb0c14cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 0x7fcb0c14cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41058/0 (socket says 172.21.15.92:41058) 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 -- 172.21.15.92:0/3283771763 learned_addr learned my addr 172.21.15.92:0/3283771763 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0b7fe700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcaf4004b80 0x7fcb0c0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0bfff700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb0c0adcd0 0x7fcb0c0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 -- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcaf4004b80 msgr2=0x7fcb0c0ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcaf4004b80 0x7fcb0c0ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 -- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb0c0adcd0 msgr2=0x7fcb0c0ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb0c0adcd0 0x7fcb0c0ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 -- 172.21.15.92:0/3283771763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c1654f0 con 0x7fcb0c164940 2023-05-16T07:02:39.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0bfff700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb0c0adcd0 0x7fcb0c0ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:39.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcb0affd700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 0x7fcb0c14cd20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fcb000119f0 tx=0x7fcb00009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00011e80 con 0x7fcb0c164940 2023-05-16T07:02:39.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.370+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0000f8e0 con 0x7fcb0c164940 2023-05-16T07:02:39.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.371+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1657d0 con 0x7fcb0c164940 2023-05-16T07:02:39.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.371+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000fca0 con 0x7fcb0c164940 2023-05-16T07:02:39.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.371+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c0ae640 con 0x7fcb0c164940 2023-05-16T07:02:39.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.371+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c14feb0 con 0x7fcb0c164940 2023-05-16T07:02:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.372+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcb00024070 con 0x7fcb0c164940 2023-05-16T07:02:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.372+0000 7fcaf37fe700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcaf805ed90 0x7fcaf8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.372+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcb0007a5a0 con 0x7fcb0c164940 2023-05-16T07:02:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.372+0000 7fcb0b7fe700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcaf805ed90 0x7fcaf8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.373+0000 7fcb0b7fe700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcaf805ed90 0x7fcaf8061250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fcafc007fd0 tx=0x7fcafc007bb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.376+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcb0004bb40 con 0x7fcb0c164940 2023-05-16T07:02:39.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.543+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb0c150060 con 0x7fcb0c164940 2023-05-16T07:02:39.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.543+0000 7fcaf37fe700 1 -- 172.21.15.92:0/3283771763 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb0004f160 con 0x7fcb0c164940 2023-05-16T07:02:39.548 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:39.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcaf805ed90 msgr2=0x7fcaf8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcaf805ed90 0x7fcaf8061250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fcafc007fd0 tx=0x7fcafc007bb0).stop 2023-05-16T07:02:39.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 msgr2=0x7fcb0c14cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 0x7fcb0c14cd20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fcb000119f0 tx=0x7fcb00009060).stop 2023-05-16T07:02:39.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 shutdown_connections 2023-05-16T07:02:39.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcaf805ed90 0x7fcaf8061250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcaf4004b80 0x7fcb0c0ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb0c0adcd0 0x7fcb0c0ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 --2- 172.21.15.92:0/3283771763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c164940 0x7fcb0c14cd20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 >> 172.21.15.92:0/3283771763 conn(0x7fcb0c05e820 msgr2=0x7fcb0c067d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 shutdown_connections 2023-05-16T07:02:39.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.545+0000 7fcb11fd9700 1 -- 172.21.15.92:0/3283771763 wait complete. 2023-05-16T07:02:39.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:39.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.670+0000 7fb94bd04700 1 Processor -- start 2023-05-16T07:02:39.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.670+0000 7fb94bd04700 1 -- start start 2023-05-16T07:02:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94bd04700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb944154090 0x7fb9440aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94bd04700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 0x7fb9440685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94bd04700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440c94d0 con 0x7fb944154470 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94bd04700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440cb7e0 con 0x7fb944154090 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94bd04700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb944068aa0 con 0x7fb944068160 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94a2a1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 0x7fb9440685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb949aa0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb944154090 0x7fb9440aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94a2a1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 0x7fb9440685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:33640/0 (socket says 172.21.15.92:33640) 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb949aa0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb944154090 0x7fb9440aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44700/0 (socket says 172.21.15.92:44700) 2023-05-16T07:02:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.671+0000 7fb94a2a1700 1 -- 172.21.15.92:0/983161125 learned_addr learned my addr 172.21.15.92:0/983161125 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb949aa0700 1 -- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 msgr2=0x7fb9440685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb949aa0700 1 --2- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 0x7fb9440685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb949aa0700 1 -- 172.21.15.92:0/983161125 >> v1:172.21.15.158:6789/0 conn(0x7fb944154470 legacy=0x7fb9440a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb948a9e700 1 -- 172.21.15.92:0/983161125 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139291329 0 0) 0x7fb9440c94d0 con 0x7fb944154470 2023-05-16T07:02:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb949aa0700 1 -- 172.21.15.92:0/983161125 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440c88c0 con 0x7fb944154090 2023-05-16T07:02:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb94a2a1700 1 --2- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 0x7fb9440685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb949aa0700 1 --2- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944154090 0x7fb9440aaca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb934009e60 tx=0x7fb934009780).ready entity=mon.1 client_cookie=43297657a18c4748 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb948a9e700 1 -- 172.21.15.92:0/983161125 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93400e070 con 0x7fb944154090 2023-05-16T07:02:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb948a9e700 1 -- 172.21.15.92:0/983161125 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93400bb60 con 0x7fb944154090 2023-05-16T07:02:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb948a9e700 1 -- 172.21.15.92:0/983161125 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934012410 con 0x7fb944154090 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb94bd04700 1 -- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944154090 msgr2=0x7fb9440aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.672+0000 7fb94bd04700 1 --2- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944154090 0x7fb9440aaca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb934009e60 tx=0x7fb934009780).stop 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.673+0000 7fb94bd04700 1 -- 172.21.15.92:0/983161125 shutdown_connections 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.673+0000 7fb94bd04700 1 --2- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944154090 0x7fb9440aaca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.673+0000 7fb94bd04700 1 --2- 172.21.15.92:0/983161125 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944068160 0x7fb9440685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.673+0000 7fb94bd04700 1 -- 172.21.15.92:0/983161125 >> 172.21.15.92:0/983161125 conn(0x7fb944067920 msgr2=0x7fb9440cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.673+0000 7fb94bd04700 1 -- 172.21.15.92:0/983161125 shutdown_connections 2023-05-16T07:02:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.673+0000 7fb94bd04700 1 -- 172.21.15.92:0/983161125 wait complete. 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 Processor -- start 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 -- start start 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944068160 0x7fb9441549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944154090 0x7fb944141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 0x7fb9440c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944155bf0 con 0x7fb944154470 2023-05-16T07:02:39.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944154fe0 con 0x7fb944068160 2023-05-16T07:02:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94bd04700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9441552c0 con 0x7fb944154090 2023-05-16T07:02:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.674+0000 7fb94a2a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 0x7fb9440c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 0x7fb9440c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41074/0 (socket says 172.21.15.92:41074) 2023-05-16T07:02:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 -- 172.21.15.92:0/2247946868 learned_addr learned my addr 172.21.15.92:0/2247946868 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb949aa0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944068160 0x7fb9441549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94929f700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944154090 0x7fb944141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 -- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944154090 msgr2=0x7fb944141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944154090 0x7fb944141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 -- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944068160 msgr2=0x7fb9441549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944068160 0x7fb9441549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 -- 172.21.15.92:0/2247946868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c009370 con 0x7fb944154470 2023-05-16T07:02:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94a2a1700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 0x7fb9440c4530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fb93c00e950 tx=0x7fb93c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0095c0 con 0x7fb944154470 2023-05-16T07:02:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c012600 con 0x7fb944154470 2023-05-16T07:02:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb934009a20 con 0x7fb944154470 2023-05-16T07:02:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0129a0 con 0x7fb944154470 2023-05-16T07:02:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.675+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb944155880 con 0x7fb944154470 2023-05-16T07:02:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.676+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb944066730 con 0x7fb944154470 2023-05-16T07:02:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.676+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb93c022070 con 0x7fb944154470 2023-05-16T07:02:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.677+0000 7fb941ffb700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb93005ed90 0x7fb930061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.677+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb93c079e60 con 0x7fb944154470 2023-05-16T07:02:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.677+0000 7fb949aa0700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb93005ed90 0x7fb930061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.677+0000 7fb949aa0700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb93005ed90 0x7fb930061250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb934009e60 tx=0x7fb934009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.681+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb93c04ee30 con 0x7fb944154470 2023-05-16T07:02:39.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.842+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb94406bd40 con 0x7fb944154470 2023-05-16T07:02:39.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.842+0000 7fb941ffb700 1 -- 172.21.15.92:0/2247946868 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb93c016020 con 0x7fb944154470 2023-05-16T07:02:39.846 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:39.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb93005ed90 msgr2=0x7fb930061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb93005ed90 0x7fb930061250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb934009e60 tx=0x7fb934009780).stop 2023-05-16T07:02:39.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 msgr2=0x7fb9440c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 0x7fb9440c4530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fb93c00e950 tx=0x7fb93c0090f0).stop 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 shutdown_connections 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb93005ed90 0x7fb930061250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb944154470 0x7fb9440c4530 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb944068160 0x7fb9441549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 --2- 172.21.15.92:0/2247946868 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb944154090 0x7fb944141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 >> 172.21.15.92:0/2247946868 conn(0x7fb944067920 msgr2=0x7fb94414fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 shutdown_connections 2023-05-16T07:02:39.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.844+0000 7fb94bd04700 1 -- 172.21.15.92:0/2247946868 wait complete. 2023-05-16T07:02:39.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:39.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.967+0000 7f6192961700 1 Processor -- start 2023-05-16T07:02:39.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.967+0000 7f6192961700 1 -- start start 2023-05-16T07:02:39.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6192961700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f618c153da0 0x7f618c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6192961700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f618c154180 0x7f618c063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6192961700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6192961700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c064100 con 0x7f618c154180 2023-05-16T07:02:39.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6192961700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c068620 con 0x7f618c153da0 2023-05-16T07:02:39.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6192961700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0622b0 con 0x7f618c062fa0 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f618b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f618c154180 0x7f618c063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f618b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f618c154180 0x7f618c063720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41080/0 (socket says 172.21.15.92:41080) 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f618b7fe700 1 -- 172.21.15.92:0/2801070745 learned_addr learned my addr 172.21.15.92:0/2801070745 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f6190efe700 1 --2- 172.21.15.92:0/2801070745 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f618bfff700 1 --2- 172.21.15.92:0/2801070745 >> v2:172.21.15.158:3300/0 conn(0x7f618c153da0 0x7f618c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f618b7fe700 1 -- 172.21.15.92:0/2801070745 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 msgr2=0x7f618c063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.968+0000 7f618b7fe700 1 --2- 172.21.15.92:0/2801070745 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618b7fe700 1 -- 172.21.15.92:0/2801070745 >> v2:172.21.15.158:3300/0 conn(0x7f618c153da0 msgr2=0x7f618c069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618b7fe700 1 --2- 172.21.15.92:0/2801070745 >> v2:172.21.15.158:3300/0 conn(0x7f618c153da0 0x7f618c069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618b7fe700 1 -- 172.21.15.92:0/2801070745 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c062b10 con 0x7f618c154180 2023-05-16T07:02:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618bfff700 1 --2- 172.21.15.92:0/2801070745 >> v2:172.21.15.158:3300/0 conn(0x7f618c153da0 0x7f618c069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618b7fe700 1 --2- 172.21.15.92:0/2801070745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c063720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f618c067ae0 tx=0x7f61800090f0).ready entity=mon.0 client_cookie=72d322e3bbe47fef server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618affd700 1 -- 172.21.15.92:0/2801070745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000b600 con 0x7f618c154180 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618affd700 1 -- 172.21.15.92:0/2801070745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000b760 con 0x7f618c154180 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f618affd700 1 -- 172.21.15.92:0/2801070745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000bae0 con 0x7f618c154180 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 -- 172.21.15.92:0/2801070745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 msgr2=0x7f618c063720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 --2- 172.21.15.92:0/2801070745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c063720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f618c067ae0 tx=0x7f61800090f0).stop 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 -- 172.21.15.92:0/2801070745 shutdown_connections 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 --2- 172.21.15.92:0/2801070745 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 --2- 172.21.15.92:0/2801070745 >> v2:172.21.15.158:3300/0 conn(0x7f618c153da0 0x7f618c069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 --2- 172.21.15.92:0/2801070745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c063720 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 -- 172.21.15.92:0/2801070745 >> 172.21.15.92:0/2801070745 conn(0x7f618c0c8aa0 msgr2=0x7f618c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 -- 172.21.15.92:0/2801070745 shutdown_connections 2023-05-16T07:02:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.969+0000 7f6192961700 1 -- 172.21.15.92:0/2801070745 wait complete. 2023-05-16T07:02:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.970+0000 7f6192961700 1 Processor -- start 2023-05-16T07:02:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 -- start start 2023-05-16T07:02:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c0c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f618c153da0 0x7f618c150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c1473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1514f0 con 0x7f618c154180 2023-05-16T07:02:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1517d0 con 0x7f618c153da0 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6192961700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0bfb40 con 0x7f618c062fa0 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6190efe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c1473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6190efe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c1473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41088/0 (socket says 172.21.15.92:41088) 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.971+0000 7f6190efe700 1 -- 172.21.15.92:0/404440433 learned_addr learned my addr 172.21.15.92:0/404440433 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f618b7fe700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f618c153da0 0x7f618c150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f618bfff700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c0c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6190efe700 1 -- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 msgr2=0x7f618c0c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6190efe700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c0c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6190efe700 1 -- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f618c153da0 msgr2=0x7f618c150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6190efe700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f618c153da0 0x7f618c150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6190efe700 1 -- 172.21.15.92:0/404440433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009370 con 0x7f618c154180 2023-05-16T07:02:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6190efe700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c1473c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f6184009fa0 tx=0x7f6184009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618400ed20 con 0x7f618c154180 2023-05-16T07:02:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618400ee80 con 0x7f618c154180 2023-05-16T07:02:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c0bfe20 con 0x7f618c154180 2023-05-16T07:02:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184014610 con 0x7f618c154180 2023-05-16T07:02:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.972+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c151190 con 0x7f618c154180 2023-05-16T07:02:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.973+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c06c350 con 0x7f618c154180 2023-05-16T07:02:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.973+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f618401b030 con 0x7f618c154180 2023-05-16T07:02:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.973+0000 7f6173fff700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617805ed90 0x7f6178061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.973+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6184079fb0 con 0x7f618c154180 2023-05-16T07:02:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.974+0000 7f618bfff700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617805ed90 0x7f6178061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.974+0000 7f618bfff700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617805ed90 0x7f6178061250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f617c006d80 tx=0x7f617c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:39.977+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f618404b550 con 0x7f618c154180 2023-05-16T07:02:40.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.180+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f618c068340 con 0x7f618c154180 2023-05-16T07:02:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.180+0000 7f6173fff700 1 -- 172.21.15.92:0/404440433 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f618404eb70 con 0x7f618c154180 2023-05-16T07:02:40.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:40.186 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:40.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617805ed90 msgr2=0x7f6178061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617805ed90 0x7f6178061250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f617c006d80 tx=0x7f617c008040).stop 2023-05-16T07:02:40.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 msgr2=0x7f618c1473c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c1473c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f6184009fa0 tx=0x7f6184009900).stop 2023-05-16T07:02:40.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 shutdown_connections 2023-05-16T07:02:40.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617805ed90 0x7f6178061250 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f618c062fa0 0x7f618c0c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618c154180 0x7f618c1473c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 --2- 172.21.15.92:0/404440433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f618c153da0 0x7f618c150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.182+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 >> 172.21.15.92:0/404440433 conn(0x7f618c0c8aa0 msgr2=0x7f618c0c4870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:40.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.183+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 shutdown_connections 2023-05-16T07:02:40.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.183+0000 7f6192961700 1 -- 172.21.15.92:0/404440433 wait complete. 2023-05-16T07:02:40.189 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:40.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:40.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 Processor -- start 2023-05-16T07:02:40.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 -- start start 2023-05-16T07:02:40.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154170 0x7f90680c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9068069db0 0x7f90680c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90680c0660 con 0x7f9068154550 2023-05-16T07:02:40.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90680c1b00 con 0x7f9068069db0 2023-05-16T07:02:40.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.305+0000 7f906d016700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90680bfd30 con 0x7f9068154170 2023-05-16T07:02:40.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f906659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9068154550 0x7f90680696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51842/0 (socket says 172.21.15.92:51842) 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f906659c700 1 -- 172.21.15.92:0/1264935124 learned_addr learned my addr 172.21.15.92:0/1264935124 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9066d9d700 1 --2- 172.21.15.92:0/1264935124 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154170 0x7f90680c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f906759e700 1 --2- 172.21.15.92:0/1264935124 >> v2:172.21.15.158:3300/0 conn(0x7f9068069db0 0x7f90680c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9065d9b700 1 -- 172.21.15.92:0/1264935124 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498604291 0 0) 0x7f90680c0660 con 0x7f9068154550 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9065d9b700 1 -- 172.21.15.92:0/1264935124 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90500033f0 con 0x7f9068154550 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9065d9b700 1 -- 172.21.15.92:0/1264935124 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347243144 0 0) 0x7f90500033f0 con 0x7f9068154550 2023-05-16T07:02:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9066d9d700 1 -- 172.21.15.92:0/1264935124 >> v2:172.21.15.158:3300/0 conn(0x7f9068069db0 msgr2=0x7f90680c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9066d9d700 1 --2- 172.21.15.92:0/1264935124 >> v2:172.21.15.158:3300/0 conn(0x7f9068069db0 0x7f90680c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9066d9d700 1 -- 172.21.15.92:0/1264935124 >> v1:172.21.15.92:6789/0 conn(0x7f9068154550 legacy=0x7f90680696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9066d9d700 1 -- 172.21.15.92:0/1264935124 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680c0010 con 0x7f9068154170 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9066d9d700 1 --2- 172.21.15.92:0/1264935124 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154170 0x7f90680c5c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9058009e60 tx=0x7f9058009780).ready entity=mon.2 client_cookie=9cfe1753e76526dc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9065d9b700 1 -- 172.21.15.92:0/1264935124 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905800e070 con 0x7f9068154170 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9065d9b700 1 -- 172.21.15.92:0/1264935124 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905800b7a0 con 0x7f9068154170 2023-05-16T07:02:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.306+0000 7f9065d9b700 1 -- 172.21.15.92:0/1264935124 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c001470 con 0x7f9068154170 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 -- 172.21.15.92:0/1264935124 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154170 msgr2=0x7f90680c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 --2- 172.21.15.92:0/1264935124 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154170 0x7f90680c5c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9058009e60 tx=0x7f9058009780).stop 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 -- 172.21.15.92:0/1264935124 shutdown_connections 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 --2- 172.21.15.92:0/1264935124 >> v2:172.21.15.158:3300/0 conn(0x7f9068069db0 0x7f90680c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 --2- 172.21.15.92:0/1264935124 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154170 0x7f90680c5c70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 -- 172.21.15.92:0/1264935124 >> 172.21.15.92:0/1264935124 conn(0x7f90680cb700 msgr2=0x7f90680c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 -- 172.21.15.92:0/1264935124 shutdown_connections 2023-05-16T07:02:40.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.307+0000 7f906d016700 1 -- 172.21.15.92:0/1264935124 wait complete. 2023-05-16T07:02:40.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.308+0000 7f906d016700 1 Processor -- start 2023-05-16T07:02:40.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.308+0000 7f906d016700 1 -- start start 2023-05-16T07:02:40.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 0x7f9068165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9068154170 0x7f906814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154550 0x7f90681651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90680c1d40 con 0x7f9068069db0 2023-05-16T07:02:40.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90680c25e0 con 0x7f9068154170 2023-05-16T07:02:40.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90680c4590 con 0x7f9068154550 2023-05-16T07:02:40.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 0x7f9068165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 0x7f9068165990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41098/0 (socket says 172.21.15.92:41098) 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 -- 172.21.15.92:0/890397538 learned_addr learned my addr 172.21.15.92:0/890397538 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906759e700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154550 0x7f90681651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906659c700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9068154170 0x7f906814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 -- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154550 msgr2=0x7f90681651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154550 0x7f90681651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 -- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9068154170 msgr2=0x7f906814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9068154170 0x7f906814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 -- 172.21.15.92:0/890397538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009a20 con 0x7f9068069db0 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906759e700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154550 0x7f90681651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906659c700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9068154170 0x7f906814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f9066d9d700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 0x7f9068165990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9058013da0 tx=0x7f9058009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058021070 con 0x7f9068069db0 2023-05-16T07:02:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058007040 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90680c1f90 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905801d670 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.309+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90681656e0 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.310+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90680666c0 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.311+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f905801da20 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.311+0000 7f904effd700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f905405ed60 0x7f9054061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.311+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f905807b1c0 con 0x7f9068069db0 2023-05-16T07:02:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.311+0000 7f906659c700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f905405ed60 0x7f9054061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.311+0000 7f906659c700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f905405ed60 0x7f9054061220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9068158b30 tx=0x7f905c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.315+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f905804c760 con 0x7f9068069db0 2023-05-16T07:02:40.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.479+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f906806c350 con 0x7f9068069db0 2023-05-16T07:02:40.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.479+0000 7f904effd700 1 -- 172.21.15.92:0/890397538 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f905801dcb0 con 0x7f9068069db0 2023-05-16T07:02:40.484 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:40.484 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:40.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.481+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f905405ed60 msgr2=0x7f9054061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.481+0000 7f906d016700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f905405ed60 0x7f9054061220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9068158b30 tx=0x7f905c009040).stop 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.481+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 msgr2=0x7f9068165990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.481+0000 7f906d016700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 0x7f9068165990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9058013da0 tx=0x7f9058009780).stop 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 shutdown_connections 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f905405ed60 0x7f9054061220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9068154550 0x7f90681651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9068069db0 0x7f9068165990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 --2- 172.21.15.92:0/890397538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9068154170 0x7f906814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 >> 172.21.15.92:0/890397538 conn(0x7f90680cb700 msgr2=0x7f90680ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:40.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 shutdown_connections 2023-05-16T07:02:40.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.482+0000 7f906d016700 1 -- 172.21.15.92:0/890397538 wait complete. 2023-05-16T07:02:40.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:40.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.604+0000 7f59c1ec4700 1 Processor -- start 2023-05-16T07:02:40.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.604+0000 7f59c1ec4700 1 -- start start 2023-05-16T07:02:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59c1ec4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc154180 0x7f59bc063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59c1ec4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f59bc062fa0 0x7f59bc063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59c1ec4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc062b10 con 0x7f59bc153da0 2023-05-16T07:02:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59c1ec4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc064100 con 0x7f59bc062fa0 2023-05-16T07:02:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59c1ec4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc068620 con 0x7f59bc154180 2023-05-16T07:02:40.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59baffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc154180 0x7f59bc063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59bbfff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f59bc062fa0 0x7f59bc063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59baffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc154180 0x7f59bc063720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:33682/0 (socket says 172.21.15.92:33682) 2023-05-16T07:02:40.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59baffd700 1 -- 172.21.15.92:0/420504882 learned_addr learned my addr 172.21.15.92:0/420504882 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.605+0000 7f59bbfff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f59bc062fa0 0x7f59bc063c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44744/0 (socket says 172.21.15.92:44744) 2023-05-16T07:02:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59bbfff700 1 -- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc154180 msgr2=0x7f59bc063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59bbfff700 1 --2- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc154180 0x7f59bc063720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59bbfff700 1 -- 172.21.15.92:0/420504882 >> v1:172.21.15.158:6789/0 conn(0x7f59bc153da0 legacy=0x7f59bc069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59ba7fc700 1 -- 172.21.15.92:0/420504882 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912627534 0 0) 0x7f59bc062b10 con 0x7f59bc153da0 2023-05-16T07:02:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59bbfff700 1 -- 172.21.15.92:0/420504882 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc068340 con 0x7f59bc062fa0 2023-05-16T07:02:40.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59bbfff700 1 --2- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc062fa0 0x7f59bc063c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f59b4009760 tx=0x7f59b4009060).ready entity=mon.1 client_cookie=514505864fc8c657 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59ba7fc700 1 -- 172.21.15.92:0/420504882 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400f510 con 0x7f59bc062fa0 2023-05-16T07:02:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59ba7fc700 1 -- 172.21.15.92:0/420504882 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b400f670 con 0x7f59bc062fa0 2023-05-16T07:02:40.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59ba7fc700 1 -- 172.21.15.92:0/420504882 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400f9f0 con 0x7f59bc062fa0 2023-05-16T07:02:40.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59c1ec4700 1 -- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc062fa0 msgr2=0x7f59bc063c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.606+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc062fa0 0x7f59bc063c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f59b4009760 tx=0x7f59b4009060).stop 2023-05-16T07:02:40.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.607+0000 7f59c1ec4700 1 -- 172.21.15.92:0/420504882 shutdown_connections 2023-05-16T07:02:40.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.607+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc062fa0 0x7f59bc063c10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.607+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/420504882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc154180 0x7f59bc063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.607+0000 7f59c1ec4700 1 -- 172.21.15.92:0/420504882 >> 172.21.15.92:0/420504882 conn(0x7f59bc0c8aa0 msgr2=0x7f59bc0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:40.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.607+0000 7f59c1ec4700 1 -- 172.21.15.92:0/420504882 shutdown_connections 2023-05-16T07:02:40.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.607+0000 7f59c1ec4700 1 -- 172.21.15.92:0/420504882 wait complete. 2023-05-16T07:02:40.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.608+0000 7f59c1ec4700 1 Processor -- start 2023-05-16T07:02:40.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.608+0000 7f59c1ec4700 1 -- start start 2023-05-16T07:02:40.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.608+0000 7f59c1ec4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 0x7f59bc167620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.608+0000 7f59c1ec4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc153da0 0x7f59bc167b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59c1ec4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc154180 0x7f59bc1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59c1ec4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc168000 con 0x7f59bc062fa0 2023-05-16T07:02:40.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59c1ec4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc13ce50 con 0x7f59bc154180 2023-05-16T07:02:40.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59c1ec4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc13c280 con 0x7f59bc153da0 2023-05-16T07:02:40.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 0x7f59bc167620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 0x7f59bc167620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41112/0 (socket says 172.21.15.92:41112) 2023-05-16T07:02:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 -- 172.21.15.92:0/2298167902 learned_addr learned my addr 172.21.15.92:0/2298167902 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bbfff700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc154180 0x7f59bc1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59baffd700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc153da0 0x7f59bc167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 -- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc153da0 msgr2=0x7f59bc167b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc153da0 0x7f59bc167b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 -- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc154180 msgr2=0x7f59bc1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc154180 0x7f59bc1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 -- 172.21.15.92:0/2298167902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40092a0 con 0x7f59bc062fa0 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59bb7fe700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 0x7f59bc167620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f59ac00c930 tx=0x7f59ac001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:40.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00a520 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac00a6a0 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc13c520 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00aa60 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.609+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc13cae0 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.610+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc13d130 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.610+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f59ac010070 con 0x7f59bc062fa0 2023-05-16T07:02:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.611+0000 7f59a37fe700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59a805ed90 0x7f59a8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.611+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f59ac079910 con 0x7f59bc062fa0 2023-05-16T07:02:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.611+0000 7f59baffd700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59a805ed90 0x7f59a8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.611+0000 7f59baffd700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59a805ed90 0x7f59a8061250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f59bc14f6d0 tx=0x7f59b00090f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.615+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f59ac04aeb0 con 0x7f59bc062fa0 2023-05-16T07:02:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.817+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59bc06c350 con 0x7f59bc062fa0 2023-05-16T07:02:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.817+0000 7f59a37fe700 1 -- 172.21.15.92:0/2298167902 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f59ac04e4d0 con 0x7f59bc062fa0 2023-05-16T07:02:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:40.822 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:40.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59a805ed90 msgr2=0x7f59a8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59a805ed90 0x7f59a8061250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f59bc14f6d0 tx=0x7f59b00090f0).stop 2023-05-16T07:02:40.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 msgr2=0x7f59bc167620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 0x7f59bc167620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f59ac00c930 tx=0x7f59ac001630).stop 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 shutdown_connections 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59a805ed90 0x7f59a8061250 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59bc154180 0x7f59bc1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc062fa0 0x7f59bc167620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 --2- 172.21.15.92:0/2298167902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59bc153da0 0x7f59bc167b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.819+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 >> 172.21.15.92:0/2298167902 conn(0x7f59bc0c8aa0 msgr2=0x7f59bc063820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.820+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 shutdown_connections 2023-05-16T07:02:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:40.820+0000 7f59c1ec4700 1 -- 172.21.15.92:0/2298167902 wait complete. 2023-05-16T07:02:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:41.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:02:26.244736+0000', 'modified': '2023-05-16T07:02:28.250832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/240924417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 240924417}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 240924417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:02:41.238 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:02:41.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-16T07:02:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.348+0000 7fd9c7557700 1 Processor -- start 2023-05-16T07:02:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.348+0000 7fd9c7557700 1 -- start start 2023-05-16T07:02:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c7557700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9c00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c7557700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00c0660 con 0x7fd9c0069db0 2023-05-16T07:02:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c7557700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00bfd30 con 0x7fd9c0154550 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c7557700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00c0010 con 0x7fd9c0154170 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c5af4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd9c0069db0 0x7fd9c00c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51872/0 (socket says 172.21.15.92:51872) 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c5af4700 1 -- 172.21.15.92:0/285919988 learned_addr learned my addr 172.21.15.92:0/285919988 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9c00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.349+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653353668 0 0) 0x7fd9c00c0660 con 0x7fd9c0069db0 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ac0033f0 con 0x7fd9c0069db0 2023-05-16T07:02:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255761400 0 0) 0x7fd9ac0033f0 con 0x7fd9c0069db0 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 msgr2=0x7fd9c00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9c00c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 >> v1:172.21.15.158:6789/0 conn(0x7fd9c0154550 legacy=0x7fd9c00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00c02f0 con 0x7fd9c0069db0 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd9b80017f0 con 0x7fd9c0069db0 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 >> v1:172.21.15.92:6789/0 conn(0x7fd9c0069db0 legacy=0x7fd9c00c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ac003910 0x7fd9ac003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9ac004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9ac004b80 0x7fd9ac005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00c0010 con 0x7fd9ac003910 2023-05-16T07:02:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac00b9c0 con 0x7fd9ac004b80 2023-05-16T07:02:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac00bb80 con 0x7fd9c0154170 2023-05-16T07:02:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ac003910 0x7fd9ac003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9ac004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9ac004b80 0x7fd9ac005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 -- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 msgr2=0x7fd9ac004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9ac004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 -- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9ac004b80 msgr2=0x7fd9ac005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9ac004b80 0x7fd9ac005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 -- 172.21.15.92:0/285919988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0033f0 con 0x7fd9ac003910 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9c0154170 0x7fd9ac004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ac003910 0x7fd9ac003d30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fd9bc006d80 tx=0x7fd9bc009060).ready entity=mon.0 client_cookie=c611dd46656823e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc009430 con 0x7fd9ac003910 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc009e70 con 0x7fd9ac003910 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.350+0000 7fd9b7fff700 1 -- 172.21.15.92:0/285919988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00e850 con 0x7fd9ac003910 2023-05-16T07:02:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c7557700 1 -- 172.21.15.92:0/285919988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ac003910 msgr2=0x7fd9ac003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c7557700 1 --2- 172.21.15.92:0/285919988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ac003910 0x7fd9ac003d30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fd9bc006d80 tx=0x7fd9bc009060).stop 2023-05-16T07:02:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c5af4700 1 -- 172.21.15.92:0/285919988 reap_dead start 2023-05-16T07:02:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c7557700 1 -- 172.21.15.92:0/285919988 shutdown_connections 2023-05-16T07:02:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c7557700 1 -- 172.21.15.92:0/285919988 >> 172.21.15.92:0/285919988 conn(0x7fd9c00cb700 msgr2=0x7fd9c00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c7557700 1 -- 172.21.15.92:0/285919988 shutdown_connections 2023-05-16T07:02:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.351+0000 7fd9c7557700 1 -- 172.21.15.92:0/285919988 wait complete. 2023-05-16T07:02:41.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.352+0000 7fd9c7557700 1 Processor -- start 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.352+0000 7fd9c7557700 1 -- start start 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c7557700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9ac003910 0x7fd9c01651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c7557700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9c00b0200 0x7fd9c014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c7557700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 0x7fd9c00c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c7557700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00afd40 con 0x7fd9c014fac0 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c7557700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c014ff20 con 0x7fd9c00b0200 2023-05-16T07:02:41.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c7557700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01531d0 con 0x7fd9ac003910 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 0x7fd9c00c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 0x7fd9c00c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41136/0 (socket says 172.21.15.92:41136) 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 -- 172.21.15.92:0/3425050312 learned_addr learned my addr 172.21.15.92:0/3425050312 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9ac003910 0x7fd9c01651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9c00b0200 0x7fd9c014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 -- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9ac003910 msgr2=0x7fd9c01651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9ac003910 0x7fd9c01651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 -- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9c00b0200 msgr2=0x7fd9c014f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9c00b0200 0x7fd9c014f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 -- 172.21.15.92:0/3425050312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c01534b0 con 0x7fd9c014fac0 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9c00b0200 0x7fd9c014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.353+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 0x7fd9c00c3d60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd9c0083aa0 tx=0x7fd9b8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.354+0000 7fd9b57fa700 1 -- 172.21.15.92:0/3425050312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8010520 con 0x7fd9c014fac0 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.354+0000 7fd9b57fa700 1 -- 172.21.15.92:0/3425050312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b80106a0 con 0x7fd9c014fac0 2023-05-16T07:02:41.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.354+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c01649b0 con 0x7fd9c014fac0 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.354+0000 7fd9b57fa700 1 -- 172.21.15.92:0/3425050312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8010a60 con 0x7fd9c014fac0 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.354+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0164660 con 0x7fd9c014fac0 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.357+0000 7fd9d8ee9b80 1 Processor -- start 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.358+0000 7fd9b57fa700 1 -- 172.21.15.92:0/3425050312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd9b8014430 con 0x7fd9c014fac0 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.358+0000 7fd9d8ee9b80 1 -- start start 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.358+0000 7fd9b57fa700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd9a405ed90 0x7fd9a4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.358+0000 7fd9b57fa700 1 -- 172.21.15.92:0/3425050312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd9b80791c0 con 0x7fd9c014fac0 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.358+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd9a405ed90 0x7fd9a4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8c26c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8c43940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8bf9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8bc45e0 con 0x5625f8c48480 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8b3b8a0 con 0x5625f8c43560 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f8b50900 con 0x5625f8bff300 2023-05-16T07:02:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8c43940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8c43940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41142/0 (socket says 172.21.15.92:41142) 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 -- 172.21.15.92:0/1947373478 learned_addr learned my addr 172.21.15.92:0/1947373478 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8c26c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8bf9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd9a405ed90 0x7fd9a4061250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd9c00ad720 tx=0x7fd9b0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 -- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 msgr2=0x5625f8bf9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8bf9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 -- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 msgr2=0x5625f8c26c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8c26c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 -- 172.21.15.92:0/1947373478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f8bc4450 con 0x5625f8c48480 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8c26c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8c43940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd9b0014960 tx=0x7fd9b000a000).ready entity=mon.0 client_cookie=42d6c1d4b7451015 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9aa7fc700 1 -- 172.21.15.92:0/1947373478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001d070 con 0x5625f8c48480 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9aa7fc700 1 -- 172.21.15.92:0/1947373478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0014ea0 con 0x5625f8c48480 2023-05-16T07:02:41.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.359+0000 7fd9aa7fc700 1 -- 172.21.15.92:0/1947373478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0018a30 con 0x5625f8c48480 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/1947373478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 msgr2=0x5625f8c43940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8c43940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd9b0014960 tx=0x7fd9b000a000).stop 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/1947373478 shutdown_connections 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8bf9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8c26c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/1947373478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8c43940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/1947373478 >> 172.21.15.92:0/1947373478 conn(0x5625f89bbe40 msgr2=0x5625f8c479d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/1947373478 shutdown_connections 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/1947373478 wait complete. 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.360+0000 7fd9d8ee9b80 1 -- start start 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 Processor -- start 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8c57170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 0x5625f8c2cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c48480 0x5625f8c2d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8b17ed0 con 0x5625f8c43560 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8c2d650 con 0x5625f8c48480 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f8c4bbd0 con 0x5625f8bff300 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9d8ee9b80 10 client.0 ll_register_callbacks cb 0x5625f8bbe1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9c4af2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 0x5625f8c2cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9c4af2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 0x5625f8c2cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41148/0 (socket says 172.21.15.92:41148) 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9c4af2700 1 -- 172.21.15.92:0/3794693396 learned_addr learned my addr 172.21.15.92:0/3794693396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8c57170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.361+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c48480 0x5625f8c2d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c4af2700 1 -- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 msgr2=0x5625f8c57170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8c57170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c4af2700 1 -- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c48480 msgr2=0x5625f8c2d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c48480 0x5625f8c2d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c4af2700 1 -- 172.21.15.92:0/3794693396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b000f040 con 0x5625f8c43560 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 0x5625f8c2cc70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd9b0013000 tx=0x7fd9b0021000).ready entity=mon.0 client_cookie=e881a7430e1562ff server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd99f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd99f7fe700 1 -- 172.21.15.92:0/3794693396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0028070 con 0x5625f8c43560 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd99f7fe700 1 -- 172.21.15.92:0/3794693396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd9b0022620 con 0x5625f8c43560 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625f8bf2460 con 0x5625f8c43560 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd99f7fe700 1 -- 172.21.15.92:0/3794693396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b00229f0 con 0x5625f8c43560 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 10 client.4653 fetch_fsmap learned FSMap version 6 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5625f89f3310 con 0x5625f8c43560 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd99f7fe700 1 -- 172.21.15.92:0/3794693396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625f8bf2460 con 0x5625f8c43560 2023-05-16T07:02:41.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd99f7fe700 1 -- 172.21.15.92:0/3794693396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd9b0022b50 con 0x5625f8c43560 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 10 client.4653 fetch_fsmap finished waiting for FSMap version 6 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 10 client.4653 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 20 client.4653 populate_metadata read hostname 'smithi092' 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c28630 0x5625f8c61030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 4 client.4653 mds_command: new command op to 4278 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 --> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625f89b7ca0 con 0x5625f8c28630 2023-05-16T07:02:41.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.362+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c28630 0x5625f8c61030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd99f7fe700 10 client.4653 ms_handle_connect on v2:172.21.15.92:6834/240924417 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c28630 0x5625f8c61030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=650aff59d5da4c42 server_cookie=70d5bfc9feef8a60 in_seq=0 out_seq=0 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd99f7fe700 1 -- 172.21.15.92:0/3794693396 <== mds.0 v2:172.21.15.92:6834/240924417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5625f89b7ca0 con 0x5625f8c28630 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd99f7fe700 10 client.4653 handle_command_reply: tid=0 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 client.4653 shutdown 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 msgr2=0x5625f8c2cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 0x5625f8c2cc70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd9b0013000 tx=0x7fd9b0021000).stop 2023-05-16T07:02:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 shutdown_connections 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c28630 0x5625f8c61030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8bff300 0x5625f8c57170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c48480 0x5625f8c2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3794693396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c43560 0x5625f8c2cc70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd99f7fe700 0 client.4653 ms_handle_reset on v2:172.21.15.92:6834/240924417 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.363+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 >> 172.21.15.92:0/3794693396 conn(0x5625f89bbe40 msgr2=0x5625f8bfe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.364+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 shutdown_connections 2023-05-16T07:02:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.364+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3794693396 wait complete. 2023-05-16T07:02:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.364+0000 7fd9d8ee9b80 20 client.4653 trim_cache size 0 max 16384 2023-05-16T07:02:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 Processor -- start 2023-05-16T07:02:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 -- start start 2023-05-16T07:02:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 0x5625f8c56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8bfe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c48480 0x5625f8ca3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8bf4b00 con 0x5625f8bff300 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8c4aef0 con 0x5625f8c43560 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f8c4a3c0 con 0x5625f8c48480 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9c52f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 0x5625f8c56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9c52f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 0x5625f8c56080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41154/0 (socket says 172.21.15.92:41154) 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.380+0000 7fd9c52f3700 1 -- 172.21.15.92:0/3826525909 learned_addr learned my addr 172.21.15.92:0/3826525909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8bfe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c48480 0x5625f8ca3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c52f3700 1 -- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c48480 msgr2=0x5625f8ca3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c48480 0x5625f8ca3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c52f3700 1 -- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 msgr2=0x5625f8bfe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8bfe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c52f3700 1 -- 172.21.15.92:0/3826525909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f8b49880 con 0x5625f8bff300 2023-05-16T07:02:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 0x5625f8c56080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd9bc014000 tx=0x7fd9bc01d040).ready entity=mon.0 client_cookie=f053a8ed47a425ce server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9aaffd700 1 -- 172.21.15.92:0/3826525909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc020040 con 0x5625f8bff300 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9aaffd700 1 -- 172.21.15.92:0/3826525909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc00e640 con 0x5625f8bff300 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9aaffd700 1 -- 172.21.15.92:0/3826525909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00e7b0 con 0x5625f8bff300 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3826525909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 msgr2=0x5625f8c56080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 0x5625f8c56080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd9bc014000 tx=0x7fd9bc01d040).stop 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3826525909 shutdown_connections 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c48480 0x5625f8ca3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8bff300 0x5625f8c56080 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/3826525909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8c43560 0x5625f8bfe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3826525909 >> 172.21.15.92:0/3826525909 conn(0x5625f89bbe40 msgr2=0x5625f8c65050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3826525909 shutdown_connections 2023-05-16T07:02:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.381+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/3826525909 wait complete. 2023-05-16T07:02:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.382+0000 7fd9d8ee9b80 1 -- start start 2023-05-16T07:02:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.382+0000 7fd9d8ee9b80 1 Processor -- start 2023-05-16T07:02:41.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8bff300 0x5625f8ca9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c43560 0x5625f8ca3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8ca34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8c56b50 con 0x5625f8c48480 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f8ca9300 con 0x5625f8bff300 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f8ca98b0 con 0x5625f8c43560 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9d8ee9b80 10 client.0 ll_register_callbacks cb 0x5625f8bbe1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8ca34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8ca34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41164/0 (socket says 172.21.15.92:41164) 2023-05-16T07:02:41.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 -- 172.21.15.92:0/2294264464 learned_addr learned my addr 172.21.15.92:0/2294264464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8bff300 0x5625f8ca9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c4af2700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c43560 0x5625f8ca3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 -- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c43560 msgr2=0x5625f8ca3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c43560 0x5625f8ca3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 -- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8bff300 msgr2=0x5625f8ca9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8bff300 0x5625f8ca9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 -- 172.21.15.92:0/2294264464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac0033f0 con 0x5625f8c48480 2023-05-16T07:02:41.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd9c5af4700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8ca34f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd9b804e2f0 tx=0x7fd9b8002210).ready entity=mon.0 client_cookie=d47e0fc7ee875244 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.383+0000 7fd99f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 1 -- 172.21.15.92:0/2294264464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8077440 con 0x5625f8c48480 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 1 -- 172.21.15.92:0/2294264464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd9b80775a0 con 0x5625f8c48480 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9b0014ea0 con 0x5625f8c48480 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 1 -- 172.21.15.92:0/2294264464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b804d8b0 con 0x5625f8c48480 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 10 client.4659 fetch_fsmap learned FSMap version 6 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5625f8b45070 con 0x5625f8c48480 2023-05-16T07:02:41.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 1 -- 172.21.15.92:0/2294264464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0014ea0 con 0x5625f8c48480 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 1 -- 172.21.15.92:0/2294264464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd9b804a1b0 con 0x5625f8c48480 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 10 client.4659 fetch_fsmap finished waiting for FSMap version 6 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 10 client.4659 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 20 client.4659 populate_metadata read hostname 'smithi092' 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c1f840 0x5625f8c1fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 4 client.4659 mds_command: new command op to 4278 tid=0[{"prefix": "status"}] 2023-05-16T07:02:41.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 --> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5625f89eeca0 con 0x5625f8c1f840 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c1f840 0x5625f8c1fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 10 client.4659 ms_handle_connect on v2:172.21.15.92:6834/240924417 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9c52f3700 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c1f840 0x5625f8c1fc20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a91b5768dfa4ba01 server_cookie=627eb4264ed13242 in_seq=0 out_seq=0 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 1 -- 172.21.15.92:0/2294264464 <== mds.0 v2:172.21.15.92:6834/240924417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5625f89eeca0 con 0x5625f8c1f840 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd99f7fe700 10 client.4659 handle_command_reply: tid=0 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.384+0000 7fd9d8ee9b80 1 client.4659 shutdown 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 msgr2=0x5625f8ca34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8ca34f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd9b804e2f0 tx=0x7fd9b8002210).stop 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 shutdown_connections 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:6834/240924417,v1:172.21.15.92:6835/240924417] conn(0x5625f8c1f840 0x5625f8c1fc20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625f8c48480 0x5625f8ca34f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd99f7fe700 0 client.4659 ms_handle_reset on v2:172.21.15.92:6834/240924417 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5625f8bff300 0x5625f8ca9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 --2- 172.21.15.92:0/2294264464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5625f8c43560 0x5625f8ca3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 >> 172.21.15.92:0/2294264464 conn(0x5625f89bbe40 msgr2=0x5625f8c29a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 shutdown_connections 2023-05-16T07:02:41.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 1 -- 172.21.15.92:0/2294264464 wait complete. 2023-05-16T07:02:41.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9d8ee9b80 20 client.4659 trim_cache size 0 max 16384 2023-05-16T07:02:41.400 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:02:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:02:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4278, 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 15.139127001, 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 6, 2023-05-16T07:02:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 22, 2023-05-16T07:02:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 22, 2023-05-16T07:02:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 19.517984851000001 2023-05-16T07:02:41.402 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:02:41.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd9a405ed90 msgr2=0x7fd9a4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd9a405ed90 0x7fd9a4061250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd9c00ad720 tx=0x7fd9b0009040).stop 2023-05-16T07:02:41.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 msgr2=0x7fd9c00c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 0x7fd9c00c3d60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd9c0083aa0 tx=0x7fd9b8007dc0).stop 2023-05-16T07:02:41.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 shutdown_connections 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd9a405ed90 0x7fd9a4061250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd9ac003910 0x7fd9c01651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c014fac0 0x7fd9c00c3d60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 --2- 172.21.15.92:0/3425050312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9c00b0200 0x7fd9c014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 >> 172.21.15.92:0/3425050312 conn(0x7fd9c00cb700 msgr2=0x7fd9c00ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 shutdown_connections 2023-05-16T07:02:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.385+0000 7fd9c7557700 1 -- 172.21.15.92:0/3425050312 wait complete. 2023-05-16T07:02:41.405 INFO:teuthology.run_tasks:Running task kclient... 2023-05-16T07:02:41.428 INFO:tasks.kclient:Mounting kernel clients... 2023-05-16T07:02:41.429 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-05-16T07:02:41.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.430 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:02:41.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-05-16T07:02:41.509 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-05-16T07:02:41.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:02:41.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.511 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:02:41.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-05-16T07:02:41.629 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-05-16T07:02:41.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:02:41.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.631 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list 2023-05-16T07:02:41.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.650 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link delete ceph-brx 2023-05-16T07:02:41.720 INFO:teuthology.orchestra.run.smithi158.stderr:Cannot find device "ceph-brx" 2023-05-16T07:02:41.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:02:41.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.723 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list 2023-05-16T07:02:41.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:41.779 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link delete ceph-brx 2023-05-16T07:02:41.845 INFO:teuthology.orchestra.run.smithi158.stderr:Cannot find device "ceph-brx" 2023-05-16T07:02:41.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:02:41.847 INFO:tasks.kclient:client.0 config is {} 2023-05-16T07:02:41.847 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:02:41.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.945+0000 7f8df9edc700 1 Processor -- start 2023-05-16T07:02:41.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.945+0000 7f8df9edc700 1 -- start start 2023-05-16T07:02:41.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df9edc700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8df40c1460 0x7f8df40a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df9edc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8df41545b0 0x7f8df4064a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df9edc700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df4154a60 con 0x7f8df40aa100 2023-05-16T07:02:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df9edc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df4154d40 con 0x7f8df41545b0 2023-05-16T07:02:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df9edc700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df4155020 con 0x7f8df40c1460 2023-05-16T07:02:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8df41545b0 0x7f8df4064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8df41545b0 0x7f8df4064a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41172/0 (socket says 172.21.15.92:41172) 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 -- 172.21.15.92:0/91975868 learned_addr learned my addr 172.21.15.92:0/91975868 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df8eda700 1 --2- 172.21.15.92:0/91975868 >> v2:172.21.15.158:3300/0 conn(0x7f8df40c1460 0x7f8df40a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 -- 172.21.15.92:0/91975868 >> v2:172.21.15.158:3300/0 conn(0x7f8df40c1460 msgr2=0x7f8df40a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 --2- 172.21.15.92:0/91975868 >> v2:172.21.15.158:3300/0 conn(0x7f8df40c1460 0x7f8df40a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 -- 172.21.15.92:0/91975868 >> v1:172.21.15.158:6789/0 conn(0x7f8df40aa100 legacy=0x7f8df4157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 -- 172.21.15.92:0/91975868 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4155300 con 0x7f8df41545b0 2023-05-16T07:02:41.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df3fff700 1 --2- 172.21.15.92:0/91975868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4064a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f8df4083ee0 tx=0x7f8de4009840).ready entity=mon.0 client_cookie=55e2dcf0eaa48f5c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df37fe700 1 -- 172.21.15.92:0/91975868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de400e070 con 0x7f8df41545b0 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df37fe700 1 -- 172.21.15.92:0/91975868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de400ba10 con 0x7f8df41545b0 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.946+0000 7f8df37fe700 1 -- 172.21.15.92:0/91975868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de400bd90 con 0x7f8df41545b0 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 -- 172.21.15.92:0/91975868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 msgr2=0x7f8df4064a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 --2- 172.21.15.92:0/91975868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4064a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f8df4083ee0 tx=0x7f8de4009840).stop 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 -- 172.21.15.92:0/91975868 shutdown_connections 2023-05-16T07:02:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 --2- 172.21.15.92:0/91975868 >> v2:172.21.15.158:3300/0 conn(0x7f8df40c1460 0x7f8df40a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 --2- 172.21.15.92:0/91975868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4064a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 -- 172.21.15.92:0/91975868 >> 172.21.15.92:0/91975868 conn(0x7f8df40cb600 msgr2=0x7f8df40c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 -- 172.21.15.92:0/91975868 shutdown_connections 2023-05-16T07:02:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.947+0000 7f8df9edc700 1 -- 172.21.15.92:0/91975868 wait complete. 2023-05-16T07:02:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.948+0000 7f8df9edc700 1 Processor -- start 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.948+0000 7f8df9edc700 1 -- start start 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df9edc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8df40aa100 0x7f8df40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df9edc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8df40c1460 0x7f8df4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df9edc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df9edc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40a9800 con 0x7f8df41545b0 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df9edc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40af5f0 con 0x7f8df40c1460 2023-05-16T07:02:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df9edc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40aecc0 con 0x7f8df40aa100 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41176/0 (socket says 172.21.15.92:41176) 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 -- 172.21.15.92:0/1184205510 learned_addr learned my addr 172.21.15.92:0/1184205510 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df3fff700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8df40c1460 0x7f8df4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df8eda700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8df40aa100 0x7f8df40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 -- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8df40aa100 msgr2=0x7f8df40b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8df40aa100 0x7f8df40b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 -- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8df40c1460 msgr2=0x7f8df4156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8df40c1460 0x7f8df4156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 -- 172.21.15.92:0/1184205510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4009ae0 con 0x7f8df41545b0 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.949+0000 7f8df96db700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4150cd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8ddc00c490 tx=0x7f8ddc007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.950+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc002ea0 con 0x7f8df41545b0 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.950+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc0123f0 con 0x7f8df41545b0 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.950+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df40aefa0 con 0x7f8df41545b0 2023-05-16T07:02:41.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.950+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc012790 con 0x7f8df41545b0 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.950+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df40c20d0 con 0x7f8df41545b0 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.951+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df40b0c60 con 0x7f8df41545b0 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.951+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8ddc010070 con 0x7f8df41545b0 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.951+0000 7f8df0ff9700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8dd405ed90 0x7f8dd4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.951+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8ddc078f00 con 0x7f8df41545b0 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.951+0000 7f8df8eda700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8dd405ed90 0x7f8dd4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:41.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.952+0000 7f8df8eda700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8dd405ed90 0x7f8dd4061250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f8dec00bbc0 tx=0x7f8dec008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:41.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:41.955+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8ddc0478f0 con 0x7f8df41545b0 2023-05-16T07:02:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.159+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8df40af760 con 0x7f8df41545b0 2023-05-16T07:02:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.160+0000 7f8df0ff9700 1 -- 172.21.15.92:0/1184205510 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8ddc04a4a0 con 0x7f8df41545b0 2023-05-16T07:02:42.164 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:02:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8dd405ed90 msgr2=0x7f8dd4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8dd405ed90 0x7f8dd4061250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f8dec00bbc0 tx=0x7f8dec008040).stop 2023-05-16T07:02:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 msgr2=0x7f8df4150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4150cd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8ddc00c490 tx=0x7f8ddc007de0).stop 2023-05-16T07:02:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 shutdown_connections 2023-05-16T07:02:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8dd405ed90 0x7f8dd4061250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8df40aa100 0x7f8df40b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df41545b0 0x7f8df4150cd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 --2- 172.21.15.92:0/1184205510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8df40c1460 0x7f8df4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.161+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 >> 172.21.15.92:0/1184205510 conn(0x7f8df40cb600 msgr2=0x7f8df40aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.162+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 shutdown_connections 2023-05-16T07:02:42.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:42.162+0000 7f8df9edc700 1 -- 172.21.15.92:0/1184205510 wait complete. 2023-05-16T07:02:42.176 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:02:42.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:02:42.176 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:02:42.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:02:42.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:02:42.177 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:02:42.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-16T07:02:42.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:02:42.178 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:02:42.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.178 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:02:42.193 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:42.194 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41993sec preferred_lft 41993sec 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:02:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:42.196 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-05-16T07:02:42.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.197 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:42.198 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:42.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-05-16T07:02:42.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-05-16T07:02:42.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-05-16T07:02:42.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-05-16T07:02:42.199 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:42.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.342 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:02:42.380 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:02:42.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.381 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:02:42.449 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:02:42.449 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:02:42.449 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:02:42.450 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:02:42.450 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:02:42.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.451 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:42.451 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:42.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:02:42.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:02:42.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:02:42.452 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:42.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.699 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:02:42.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.714 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-05-16T07:02:42.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:42.768 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:42.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:02:42.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-05-16T07:02:42.769 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:42.855 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-05-16T07:02:42.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:42.856 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:42.856 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:42.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-05-16T07:02:42.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-05-16T07:02:42.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-05-16T07:02:42.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-05-16T07:02:42.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-05-16T07:02:42.858 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:43.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.035 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:43.035 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:43.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:02:43.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-05-16T07:02:43.036 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:43.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.126 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:02:43.140 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:02:43.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:43.233 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:43.233 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:02:43.340 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:43.341 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:02:43.342 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:02:43.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:02:43.366 INFO:tasks.kclient:client.2 config is {} 2023-05-16T07:02:43.367 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:02:43.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.503+0000 7f11bd9a7700 1 Processor -- start 2023-05-16T07:02:43.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.503+0000 7f11bd9a7700 1 -- start start 2023-05-16T07:02:43.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.503+0000 7f11bd9a7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 0x7f11b815eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.503+0000 7f11bd9a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b80aaf40 con 0x7f11b8153dc0 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.503+0000 7f11bd9a7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b80c1450 con 0x7f11b81541a0 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.503+0000 7f11bd9a7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80c1730 con 0x7f11b8063d00 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f11b8153dc0 0x7f11b80c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51936/0 (socket says 172.21.15.92:51936) 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b6ffd700 1 -- 172.21.15.92:0/3542098380 learned_addr learned my addr 172.21.15.92:0/3542098380 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b77fe700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 0x7f11b815eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076628242 0 0) 0x7f11b80aaf40 con 0x7f11b8153dc0 2023-05-16T07:02:43.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a40033f0 con 0x7f11b8153dc0 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2450224163 0 0) 0x7f11a40033f0 con 0x7f11b8153dc0 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 msgr2=0x7f11b815eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 0x7f11b815eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 >> v1:172.21.15.158:6789/0 conn(0x7f11b81541a0 legacy=0x7f11b80635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80c1a10 con 0x7f11b8153dc0 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f11a80017f0 con 0x7f11b8153dc0 2023-05-16T07:02:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 >> v1:172.21.15.92:6789/0 conn(0x7f11b8153dc0 legacy=0x7f11b80c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 0x7f11a4003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11a4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11a4004b80 0x7f11a4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80c1730 con 0x7f11a4003e40 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a400b990 con 0x7f11a4004b80 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a400bb50 con 0x7f11b8063d00 2023-05-16T07:02:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b67fc700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11a4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b6ffd700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 0x7f11a4003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b77fe700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11a4004b80 0x7f11a4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b67fc700 1 -- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 msgr2=0x7f11a4003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b67fc700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8063d00 0x7f11a4003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b67fc700 1 -- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11a4004b80 msgr2=0x7f11a4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.504+0000 7f11b67fc700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11a4004b80 0x7f11a4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11b67fc700 1 -- 172.21.15.92:0/3542098380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40033f0 con 0x7f11a4003e40 2023-05-16T07:02:43.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11b67fc700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11a4004690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f11a0001700 tx=0x7f11a0007bb0).ready entity=mon.0 client_cookie=44ea0a9aaf156d72 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0007020 con 0x7f11a4003e40 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a0007180 con 0x7f11a4003e40 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11b5ffb700 1 -- 172.21.15.92:0/3542098380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0007500 con 0x7f11a4003e40 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11bd9a7700 1 -- 172.21.15.92:0/3542098380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 msgr2=0x7f11a4004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/3542098380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11a4004690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f11a0001700 tx=0x7f11a0007bb0).stop 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11b77fe700 1 -- 172.21.15.92:0/3542098380 reap_dead start 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11bd9a7700 1 -- 172.21.15.92:0/3542098380 shutdown_connections 2023-05-16T07:02:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11bd9a7700 1 -- 172.21.15.92:0/3542098380 >> 172.21.15.92:0/3542098380 conn(0x7f11b80caca0 msgr2=0x7f11b80cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:43.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11bd9a7700 1 -- 172.21.15.92:0/3542098380 shutdown_connections 2023-05-16T07:02:43.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.505+0000 7f11bd9a7700 1 -- 172.21.15.92:0/3542098380 wait complete. 2023-05-16T07:02:43.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 Processor -- start 2023-05-16T07:02:43.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 -- start start 2023-05-16T07:02:43.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11b814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8157470 0x7f11b814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11b80ae640 0x7f11b80aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b814f550 con 0x7f11a4003e40 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80b0330 con 0x7f11b80ae640 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11bd9a7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80c4c00 con 0x7f11b8157470 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.507+0000 7f11b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11b814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11b814eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41202/0 (socket says 172.21.15.92:41202) 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 -- 172.21.15.92:0/346846832 learned_addr learned my addr 172.21.15.92:0/346846832 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:43.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b67fc700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8157470 0x7f11b814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b77fe700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11b80ae640 0x7f11b80aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 -- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8157470 msgr2=0x7f11b814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8157470 0x7f11b814f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 -- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11b80ae640 msgr2=0x7f11b80aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11b80ae640 0x7f11b80aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 -- 172.21.15.92:0/346846832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80afea0 con 0x7f11a4003e40 2023-05-16T07:02:43.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11b6ffd700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11b814eb70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f11a8007750 tx=0x7f11a8001060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a800f8d0 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800fa30 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b80af290 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a800fdb0 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.508+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b80af850 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.509+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b805efa0 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.509+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f11a801b030 con 0x7f11a4003e40 2023-05-16T07:02:43.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.510+0000 7f11aeffd700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f119805edc0 0x7f1198061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:43.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.510+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f11a8079490 con 0x7f11a4003e40 2023-05-16T07:02:43.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.510+0000 7f11b67fc700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f119805edc0 0x7f1198061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:43.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.510+0000 7f11b67fc700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f119805edc0 0x7f1198061280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f11a0001010 tx=0x7f11a0016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:43.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.514+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f11a804aae0 con 0x7f11a4003e40 2023-05-16T07:02:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.715+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11b80aabd0 con 0x7f11a4003e40 2023-05-16T07:02:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.716+0000 7f11aeffd700 1 -- 172.21.15.92:0/346846832 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f11a8019030 con 0x7f11a4003e40 2023-05-16T07:02:43.720 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:02:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.717+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f119805edc0 msgr2=0x7f1198061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.717+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f119805edc0 0x7f1198061280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f11a0001010 tx=0x7f11a0016000).stop 2023-05-16T07:02:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.717+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 msgr2=0x7f11b814eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.717+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11b814eb70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f11a8007750 tx=0x7f11a8001060).stop 2023-05-16T07:02:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.717+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 shutdown_connections 2023-05-16T07:02:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f119805edc0 0x7f1198061280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f11b80ae640 0x7f11b80aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11a4003e40 0x7f11b814eb70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 --2- 172.21.15.92:0/346846832 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f11b8157470 0x7f11b814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 >> 172.21.15.92:0/346846832 conn(0x7f11b80caca0 msgr2=0x7f11b8065190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 shutdown_connections 2023-05-16T07:02:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:43.718+0000 7f11bd9a7700 1 -- 172.21.15.92:0/346846832 wait complete. 2023-05-16T07:02:43.732 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:02:43.732 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:02:43.733 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:02:43.733 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-05-16T07:02:43.733 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-05-16T07:02:43.733 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:02:43.733 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-16T07:02:43.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:02:43.734 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-05-16T07:02:43.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.734 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:02:43.749 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:43.750 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:43.750 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:43.750 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:43.751 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:43.751 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:43.751 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:43.751 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:43.752 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:43.752 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:43.752 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:43.752 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:43.753 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:02:43.753 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:43.753 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:02:43.754 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41991sec preferred_lft 41991sec 2023-05-16T07:02:43.754 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:02:43.754 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:43.754 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:43.755 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:43.755 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:02:43.755 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:43.755 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link tentative 2023-05-16T07:02:43.756 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:43.756 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:02:43.756 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:02:43.756 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link tentative 2023-05-16T07:02:43.757 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:43.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.758 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:02:43.811 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:02:43.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.814 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:02:43.868 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:02:43.868 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:02:43.868 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:02:43.868 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:02:43.868 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:02:43.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.869 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:43.869 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:43.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:02:43.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:02:43.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:02:43.871 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:43.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.966 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:02:43.980 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:02:43.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:43.981 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-05-16T07:02:44.033 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-05-16T07:02:44.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.034 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:44.035 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:44.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:02:44.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-05-16T07:02:44.035 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:44.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-05-16T07:02:44.155 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:02:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether ba:0c:c0:f9:05:82 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-16T07:02:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-16T07:02:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b80c:c0ff:fef9:582/64 scope link tentative 2023-05-16T07:02:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-05-16T07:02:44.240 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:02:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether ba:0c:c0:f9:05:82 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-16T07:02:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-16T07:02:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::b80c:c0ff:fef9:582/64 scope link tentative 2023-05-16T07:02:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:44.252 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-05-16T07:02:44.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.252 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:44.253 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:44.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-05-16T07:02:44.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-05-16T07:02:44.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-05-16T07:02:44.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-05-16T07:02:44.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-05-16T07:02:44.254 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:44.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.466 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:02:44.467 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:02:44.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2023-05-16T07:02:44.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2023-05-16T07:02:44.467 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:02:44.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.545 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.2 2023-05-16T07:02:44.559 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:02:44.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:44.637 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:44.637 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2023-05-16T07:02:44.679 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:44.680 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2023-05-16T07:02:44.681 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:02:44.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:44.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-05-16T07:02:44.745 INFO:tasks.kclient:client.1 config is {} 2023-05-16T07:02:44.746 DEBUG:teuthology.orchestra.run.smithi158:> ceph fs ls 2023-05-16T07:02:44.858 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.854+0000 7f7def150700 1 Processor -- start 2023-05-16T07:02:44.859 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.854+0000 7f7def150700 1 -- start start 2023-05-16T07:02:44.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.854+0000 7f7def150700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7de813b570 0x7f7de80be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:44.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7def150700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 0x7f7de8066070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:44.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7def150700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de80c3ba0 con 0x7f7de80beb40 2023-05-16T07:02:44.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7def150700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de80bf0c0 con 0x7f7de813b570 2023-05-16T07:02:44.861 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7def150700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de80bf3a0 con 0x7f7de813b950 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7de813b570 0x7f7de80be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 0x7f7de8066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7ded6ed700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f7de80beb40 0x7f7de8155bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:45256/0 (socket says 172.21.15.158:45256) 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7de813b570 0x7f7de80be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:54208/0 (socket says 172.21.15.158:54208) 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 0x7f7de8066070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41476/0 (socket says 172.21.15.158:41476) 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7ded6ed700 1 -- 172.21.15.158:0/3219611315 learned_addr learned my addr 172.21.15.158:0/3219611315 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 -- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 msgr2=0x7f7de8066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:44.862 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 --2- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 0x7f7de8066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 -- 172.21.15.158:0/3219611315 >> v1:172.21.15.158:6789/0 conn(0x7f7de80beb40 legacy=0x7f7de8155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de77fe700 1 -- 172.21.15.158:0/3219611315 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556537318 0 0) 0x7f7de80c3ba0 con 0x7f7de80beb40 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 -- 172.21.15.158:0/3219611315 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80c3550 con 0x7f7de813b570 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de7fff700 1 --2- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 0x7f7de8066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7deceec700 1 --2- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b570 0x7f7de80be650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7ddc009760 tx=0x7f7ddc009060).ready entity=mon.1 client_cookie=116ee862cafa2927 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de77fe700 1 -- 172.21.15.158:0/3219611315 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00f510 con 0x7f7de813b570 2023-05-16T07:02:44.863 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de77fe700 1 -- 172.21.15.158:0/3219611315 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc00f670 con 0x7f7de813b570 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.855+0000 7f7de77fe700 1 -- 172.21.15.158:0/3219611315 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00f9f0 con 0x7f7de813b570 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 -- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b570 msgr2=0x7f7de80be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 --2- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b570 0x7f7de80be650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7ddc009760 tx=0x7f7ddc009060).stop 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 -- 172.21.15.158:0/3219611315 shutdown_connections 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 --2- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b570 0x7f7de80be650 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 --2- 172.21.15.158:0/3219611315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de813b950 0x7f7de8066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 -- 172.21.15.158:0/3219611315 >> 172.21.15.158:0/3219611315 conn(0x7f7de80c8b10 msgr2=0x7f7de80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:44.864 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 -- 172.21.15.158:0/3219611315 shutdown_connections 2023-05-16T07:02:44.865 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.856+0000 7f7def150700 1 -- 172.21.15.158:0/3219611315 wait complete. 2023-05-16T07:02:44.866 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.857+0000 7f7def150700 1 Processor -- start 2023-05-16T07:02:44.867 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.857+0000 7f7def150700 1 -- start start 2023-05-16T07:02:44.867 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7def150700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 0x7f7de8151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:44.867 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7def150700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de813b570 0x7f7de80c2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:44.867 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7def150700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b950 0x7f7de8095e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:44.867 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7def150700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de81654d0 con 0x7f7de813b570 2023-05-16T07:02:44.867 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7def150700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de81648c0 con 0x7f7de813b950 2023-05-16T07:02:44.868 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7def150700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de8164ba0 con 0x7f7de80beb40 2023-05-16T07:02:44.868 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 0x7f7de8151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:44.868 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 0x7f7de8151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.158:41478/0 (socket says 172.21.15.158:41478) 2023-05-16T07:02:44.868 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 -- 172.21.15.158:0/3654353763 learned_addr learned my addr 172.21.15.158:0/3654353763 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:44.868 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7ded6ed700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b950 0x7f7de8095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:44.868 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 -- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b950 msgr2=0x7f7de8095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:44.869 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b950 0x7f7de8095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:44.869 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 -- 172.21.15.158:0/3654353763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de813b570 msgr2=0x7f7de80c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:44.869 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de813b570 0x7f7de80c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:44.869 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.858+0000 7f7deceec700 1 -- 172.21.15.158:0/3654353763 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40092a0 con 0x7f7de80beb40 2023-05-16T07:02:44.869 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.859+0000 7f7deceec700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 0x7f7de8151cc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f7de80c8a40 tx=0x7f7ddc017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.859+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00f7e0 con 0x7f7de80beb40 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.859+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc00f940 con 0x7f7de80beb40 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.859+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc0092a0 con 0x7f7de80beb40 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.859+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc01b660 con 0x7f7de80beb40 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.859+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de8165160 con 0x7f7de80beb40 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.860+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de805efa0 con 0x7f7de80beb40 2023-05-16T07:02:44.870 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.860+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7ddc00f070 con 0x7f7de80beb40 2023-05-16T07:02:44.871 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.860+0000 7f7de4ff9700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7dcc05ed90 0x7f7dcc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:44.871 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.860+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ddc07b3c0 con 0x7f7de80beb40 2023-05-16T07:02:44.871 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.860+0000 7f7de7fff700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7dcc05ed90 0x7f7dcc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:44.871 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.861+0000 7f7de7fff700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7dcc05ed90 0x7f7dcc061250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7de8096a30 tx=0x7f7dd4009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:44.871 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:44.865+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7ddc04c960 con 0x7f7de80beb40 2023-05-16T07:02:45.078 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.073+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7de8164ce0 con 0x7f7de80beb40 2023-05-16T07:02:45.078 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.073+0000 7f7de4ff9700 1 -- 172.21.15.158:0/3654353763 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7de8164ce0 con 0x7f7de80beb40 2023-05-16T07:02:45.079 INFO:teuthology.orchestra.run.smithi158.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:02:45.080 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7dcc05ed90 msgr2=0x7f7dcc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:45.081 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7dcc05ed90 0x7f7dcc061250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7de8096a30 tx=0x7f7dd4009060).stop 2023-05-16T07:02:45.081 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 msgr2=0x7f7de8151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:45.081 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 0x7f7de8151cc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f7de80c8a40 tx=0x7f7ddc017000).stop 2023-05-16T07:02:45.082 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 shutdown_connections 2023-05-16T07:02:45.082 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7dcc05ed90 0x7f7dcc061250 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:45.082 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7de80beb40 0x7f7de8151cc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:45.082 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7de813b950 0x7f7de8095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:45.083 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 --2- 172.21.15.158:0/3654353763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de813b570 0x7f7de80c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:45.083 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 >> 172.21.15.158:0/3654353763 conn(0x7f7de80c8b10 msgr2=0x7f7de80c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:45.083 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 shutdown_connections 2023-05-16T07:02:45.084 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:45.075+0000 7f7def150700 1 -- 172.21.15.158:0/3654353763 wait complete. 2023-05-16T07:02:45.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:02:45.091 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:02:45.091 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi158.front.sepia.ceph.com 2023-05-16T07:02:45.091 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-05-16T07:02:45.092 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-05-16T07:02:45.092 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:02:45.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-16T07:02:45.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:02:45.094 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-05-16T07:02:45.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.095 DEBUG:teuthology.orchestra.run.smithi158:> ip addr 2023-05-16T07:02:45.109 INFO:teuthology.orchestra.run.smithi158.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:45.110 INFO:teuthology.orchestra.run.smithi158.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:45.110 INFO:teuthology.orchestra.run.smithi158.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:45.110 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:45.110 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:45.110 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:da:6b:d4 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:da:6b:d5 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:88:72:6e brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:45.111 INFO:teuthology.orchestra.run.smithi158.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:02:45.112 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:88:72:6f brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:45.112 INFO:teuthology.orchestra.run.smithi158.stdout: inet 172.21.15.158/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:02:45.112 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft 42020sec preferred_lft 42020sec 2023-05-16T07:02:45.112 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 fe80::ec4:7aff:fe88:726f/64 scope link 2023-05-16T07:02:45.112 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:45.112 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-05-16T07:02:45.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.113 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:45.113 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:45.114 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link add name ceph-brx type bridge 2023-05-16T07:02:45.114 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip addr flush dev ceph-brx 2023-05-16T07:02:45.114 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link set ceph-brx up 2023-05-16T07:02:45.114 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-05-16T07:02:45.115 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:45.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.269 DEBUG:teuthology.orchestra.run.smithi158:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:02:45.306 INFO:teuthology.orchestra.run.smithi158.stdout:1 2023-05-16T07:02:45.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.308 DEBUG:teuthology.orchestra.run.smithi158:> route 2023-05-16T07:02:45.377 INFO:teuthology.orchestra.run.smithi158.stdout:Kernel IP routing table 2023-05-16T07:02:45.378 INFO:teuthology.orchestra.run.smithi158.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:02:45.378 INFO:teuthology.orchestra.run.smithi158.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:02:45.378 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:02:45.378 INFO:teuthology.orchestra.run.smithi158.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:02:45.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.380 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:45.380 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:45.381 DEBUG:teuthology.orchestra.run.smithi158:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:02:45.381 DEBUG:teuthology.orchestra.run.smithi158:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:02:45.382 DEBUG:teuthology.orchestra.run.smithi158:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:02:45.382 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:45.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.621 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list 2023-05-16T07:02:45.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.637 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list-id 2023-05-16T07:02:45.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.691 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:45.691 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:45.691 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-05-16T07:02:45.691 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-05-16T07:02:45.692 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:45.777 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-05-16T07:02:45.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.778 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:45.779 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:45.779 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-05-16T07:02:45.779 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-05-16T07:02:45.779 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-05-16T07:02:45.779 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-05-16T07:02:45.780 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-05-16T07:02:45.780 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:45.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:45.958 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:45.958 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:45.959 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link set brx.0 up 2023-05-16T07:02:45.959 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link set dev brx.0 master ceph-brx 2023-05-16T07:02:45.960 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:46.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.051 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p /home/ubuntu/cephtest/mnt.1 2023-05-16T07:02:46.065 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:02:46.065 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:46.154 INFO:teuthology.orchestra.run.smithi158.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:46.154 INFO:teuthology.orchestra.run.smithi158.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2023-05-16T07:02:46.262 INFO:teuthology.orchestra.run.smithi158.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:46.262 INFO:teuthology.orchestra.run.smithi158.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2023-05-16T07:02:46.265 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:02:46.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.265 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-05-16T07:02:46.288 INFO:tasks.kclient:client.3 config is {} 2023-05-16T07:02:46.289 DEBUG:teuthology.orchestra.run.smithi158:> ceph fs ls 2023-05-16T07:02:46.427 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.423+0000 7f6301709700 1 Processor -- start 2023-05-16T07:02:46.428 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.423+0000 7f6301709700 1 -- start start 2023-05-16T07:02:46.429 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.423+0000 7f6301709700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62fc0a9f30 0x7f62fc0bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:46.430 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.423+0000 7f6301709700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc063240 con 0x7f62fc134ca0 2023-05-16T07:02:46.430 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.423+0000 7f6301709700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc062630 con 0x7f62fc135080 2023-05-16T07:02:46.430 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.423+0000 7f6301709700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc0bce10 con 0x7f62fc0a9f30 2023-05-16T07:02:46.430 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fa7fc700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f62fc135080 0x7f62fc0a9820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.158:45276/0 (socket says 172.21.15.158:45276) 2023-05-16T07:02:46.430 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fa7fc700 1 -- 172.21.15.158:0/2502362687 learned_addr learned my addr 172.21.15.158:0/2502362687 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-05-16T07:02:46.430 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fb7fe700 1 --2- 172.21.15.158:0/2502362687 >> v2:172.21.15.92:3300/0 conn(0x7f62fc0a9f30 0x7f62fc0bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457255692 0 0) 0x7f62fc062630 con 0x7f62fc135080 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e80033f0 con 0x7f62fc135080 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523291721 0 0) 0x7f62fc063240 con 0x7f62fc134ca0 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62fc062630 con 0x7f62fc134ca0 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 <== mon.1 v1:172.21.15.158:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4233258784 0 0) 0x7f62e80033f0 con 0x7f62fc135080 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fb7fe700 1 -- 172.21.15.158:0/2502362687 >> v1:172.21.15.158:6789/0 conn(0x7f62fc135080 legacy=0x7f62fc0a9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.431 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fb7fe700 1 -- 172.21.15.158:0/2502362687 >> v1:172.21.15.92:6789/0 conn(0x7f62fc134ca0 legacy=0x7f62fc069330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fb7fe700 1 -- 172.21.15.158:0/2502362687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc062910 con 0x7f62fc0a9f30 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62fb7fe700 1 --2- 172.21.15.158:0/2502362687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0a9f30 0x7f62fc0bc920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f62f40096e0 tx=0x7f62f4009060).ready entity=mon.0 client_cookie=7389fd160267291e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.424+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40013f0 con 0x7f62fc0a9f30 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4011400 con 0x7f62fc0a9f30 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f62f9ffb700 1 -- 172.21.15.158:0/2502362687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4011780 con 0x7f62fc0a9f30 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 -- 172.21.15.158:0/2502362687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0a9f30 msgr2=0x7f62fc0bc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.432 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 --2- 172.21.15.158:0/2502362687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0a9f30 0x7f62fc0bc920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f62f40096e0 tx=0x7f62f4009060).stop 2023-05-16T07:02:46.433 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 -- 172.21.15.158:0/2502362687 shutdown_connections 2023-05-16T07:02:46.433 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 --2- 172.21.15.158:0/2502362687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0a9f30 0x7f62fc0bc920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.433 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 -- 172.21.15.158:0/2502362687 >> 172.21.15.158:0/2502362687 conn(0x7f62fc063e70 msgr2=0x7f62fc0648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:46.433 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 -- 172.21.15.158:0/2502362687 shutdown_connections 2023-05-16T07:02:46.433 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.425+0000 7f6301709700 1 -- 172.21.15.158:0/2502362687 wait complete. 2023-05-16T07:02:46.435 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.426+0000 7f6301709700 1 Processor -- start 2023-05-16T07:02:46.435 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 -- start start 2023-05-16T07:02:46.435 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 0x7f62fc14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62fc134ca0 0x7f62fc0bf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc135080 0x7f62fc0c0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc14f310 con 0x7f62fc135080 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc14e700 con 0x7f62fc0a9f30 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f6301709700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc14e9e0 con 0x7f62fc134ca0 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f62faffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 0x7f62fc14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:46.436 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f62fa7fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62fc134ca0 0x7f62fc0bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f62faffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 0x7f62fc14e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:54230/0 (socket says 172.21.15.158:54230) 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.427+0000 7f62faffd700 1 -- 172.21.15.158:0/3969094053 learned_addr learned my addr 172.21.15.158:0/3969094053 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fb7fe700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc135080 0x7f62fc0c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fa7fc700 1 -- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 msgr2=0x7f62fc14e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fa7fc700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 0x7f62fc14e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fa7fc700 1 -- 172.21.15.158:0/3969094053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc135080 msgr2=0x7f62fc0c0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.437 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fa7fc700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc135080 0x7f62fc0c0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fa7fc700 1 -- 172.21.15.158:0/3969094053 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40092a0 con 0x7f62fc134ca0 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62faffd700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 0x7f62fc14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62fa7fc700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62fc134ca0 0x7f62fc0bf940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f62e400c5e0 tx=0x7f62e40015a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4007170 con 0x7f62fc134ca0 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e40072f0 con 0x7f62fc134ca0 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62fc14ecc0 con 0x7f62fc134ca0 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40048c0 con 0x7f62fc134ca0 2023-05-16T07:02:46.438 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.428+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc0bfe30 con 0x7f62fc134ca0 2023-05-16T07:02:46.439 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.429+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc14f6a0 con 0x7f62fc134ca0 2023-05-16T07:02:46.439 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.429+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f62e4004c70 con 0x7f62fc134ca0 2023-05-16T07:02:46.439 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.429+0000 7f62f2ffd700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62dc05ed90 0x7f62dc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:46.439 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.430+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f62e40795f0 con 0x7f62fc134ca0 2023-05-16T07:02:46.439 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.430+0000 7f62faffd700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62dc05ed90 0x7f62dc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:46.439 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.430+0000 7f62faffd700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62dc05ed90 0x7f62dc061250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f62ec00f420 tx=0x7f62ec002740).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:46.440 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.434+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f62fc14f6a0 con 0x7f62fc134ca0 2023-05-16T07:02:46.643 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.639+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62fc14f6a0 con 0x7f62fc134ca0 2023-05-16T07:02:46.644 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.639+0000 7f62f2ffd700 1 -- 172.21.15.158:0/3969094053 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62fc14f6a0 con 0x7f62fc134ca0 2023-05-16T07:02:46.644 INFO:teuthology.orchestra.run.smithi158.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:02:46.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62dc05ed90 msgr2=0x7f62dc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62dc05ed90 0x7f62dc061250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f62ec00f420 tx=0x7f62ec002740).stop 2023-05-16T07:02:46.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62fc134ca0 msgr2=0x7f62fc0bf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:46.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62fc134ca0 0x7f62fc0bf940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f62e400c5e0 tx=0x7f62e40015a0).stop 2023-05-16T07:02:46.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 shutdown_connections 2023-05-16T07:02:46.646 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62dc05ed90 0x7f62dc061250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc135080 0x7f62fc0c0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62fc0a9f30 0x7f62fc14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 --2- 172.21.15.158:0/3969094053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62fc134ca0 0x7f62fc0bf940 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:46.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 >> 172.21.15.158:0/3969094053 conn(0x7f62fc063e70 msgr2=0x7f62fc0c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:46.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 shutdown_connections 2023-05-16T07:02:46.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-05-16T07:02:46.641+0000 7f6301709700 1 -- 172.21.15.158:0/3969094053 wait complete. 2023-05-16T07:02:46.656 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:02:46.656 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:02:46.657 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi158.front.sepia.ceph.com 2023-05-16T07:02:46.657 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-05-16T07:02:46.657 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-05-16T07:02:46.657 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:02:46.657 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-16T07:02:46.657 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:02:46.658 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-05-16T07:02:46.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.658 DEBUG:teuthology.orchestra.run.smithi158:> ip addr 2023-05-16T07:02:46.672 INFO:teuthology.orchestra.run.smithi158.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:46.673 INFO:teuthology.orchestra.run.smithi158.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:46.673 INFO:teuthology.orchestra.run.smithi158.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:46.673 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:46.673 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:46.673 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:46.673 INFO:teuthology.orchestra.run.smithi158.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:da:6b:d4 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:da:6b:d5 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:88:72:6e brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 0c:c4:7a:88:72:6f brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:46.674 INFO:teuthology.orchestra.run.smithi158.stdout: inet 172.21.15.158/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft 42019sec preferred_lft 42019sec 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 fe80::ec4:7aff:fe88:726f/64 scope link 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether fa:59:9b:a6:be:c7 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:02:46.675 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:46.676 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 fe80::f092:87ff:fef3:d6ec/64 scope link tentative 2023-05-16T07:02:46.676 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:46.676 INFO:teuthology.orchestra.run.smithi158.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:02:46.676 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether fa:59:9b:a6:be:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-05-16T07:02:46.676 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 fe80::f859:9bff:fea6:bec7/64 scope link tentative 2023-05-16T07:02:46.676 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:46.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.677 DEBUG:teuthology.orchestra.run.smithi158:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:02:46.734 INFO:teuthology.orchestra.run.smithi158.stdout:1 2023-05-16T07:02:46.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.736 DEBUG:teuthology.orchestra.run.smithi158:> route 2023-05-16T07:02:46.790 INFO:teuthology.orchestra.run.smithi158.stdout:Kernel IP routing table 2023-05-16T07:02:46.791 INFO:teuthology.orchestra.run.smithi158.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:02:46.791 INFO:teuthology.orchestra.run.smithi158.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:02:46.791 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:02:46.791 INFO:teuthology.orchestra.run.smithi158.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:02:46.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.792 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:46.792 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:46.793 DEBUG:teuthology.orchestra.run.smithi158:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:02:46.793 DEBUG:teuthology.orchestra.run.smithi158:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:02:46.794 DEBUG:teuthology.orchestra.run.smithi158:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:02:46.794 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:46.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.888 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list 2023-05-16T07:02:46.902 INFO:teuthology.orchestra.run.smithi158.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-05-16T07:02:46.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.903 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list-id 2023-05-16T07:02:46.956 INFO:teuthology.orchestra.run.smithi158.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-05-16T07:02:46.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:46.957 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:46.957 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:46.957 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-05-16T07:02:46.957 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-05-16T07:02:46.958 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:47.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:47.043 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-05-16T07:02:47.083 INFO:teuthology.orchestra.run.smithi158.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:47.083 INFO:teuthology.orchestra.run.smithi158.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.084 INFO:teuthology.orchestra.run.smithi158.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:02:47.084 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 3a:2b:5b:2f:c1:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-16T07:02:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-16T07:02:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 fe80::382b:5bff:fe2f:c12a/64 scope link tentative 2023-05-16T07:02:47.085 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:47.100 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-05-16T07:02:47.176 INFO:teuthology.orchestra.run.smithi158.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:02:47.176 INFO:teuthology.orchestra.run.smithi158.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:02:47.176 INFO:teuthology.orchestra.run.smithi158.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 ::1/128 scope host 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout: link/ether 3a:2b:5b:2f:c1:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-16T07:02:47.177 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.178 INFO:teuthology.orchestra.run.smithi158.stdout: inet6 fe80::382b:5bff:fe2f:c12a/64 scope link tentative 2023-05-16T07:02:47.178 INFO:teuthology.orchestra.run.smithi158.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:02:47.185 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-05-16T07:02:47.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:47.186 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:47.186 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:47.187 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-05-16T07:02:47.187 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-05-16T07:02:47.187 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-05-16T07:02:47.187 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-05-16T07:02:47.188 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-05-16T07:02:47.188 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:47.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:47.402 DEBUG:teuthology.orchestra.run.smithi158:> (cd / && exec bash -c ' 2023-05-16T07:02:47.403 DEBUG:teuthology.orchestra.run.smithi158:> set -e 2023-05-16T07:02:47.403 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link set brx.1 up 2023-05-16T07:02:47.403 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link set dev brx.1 master ceph-brx 2023-05-16T07:02:47.403 DEBUG:teuthology.orchestra.run.smithi158:> ') 2023-05-16T07:02:47.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:47.481 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p /home/ubuntu/cephtest/mnt.3 2023-05-16T07:02:47.495 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:02:47.495 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:47.573 INFO:teuthology.orchestra.run.smithi158.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:47.574 INFO:teuthology.orchestra.run.smithi158.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2023-05-16T07:02:47.616 INFO:teuthology.orchestra.run.smithi158.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:02:47.616 INFO:teuthology.orchestra.run.smithi158.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2023-05-16T07:02:47.617 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:02:47.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:47.618 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-05-16T07:02:47.681 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-05-16T07:02:47.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.822+0000 7fc502212700 1 Processor -- start 2023-05-16T07:02:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.822+0000 7fc502212700 1 -- start start 2023-05-16T07:02:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc502212700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4fc0c7380 0x7fc4fc154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc502212700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc4fc0c7760 0x7fc4fc0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc502212700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc158970 con 0x7fc4fc0ad040 2023-05-16T07:02:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc502212700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc155340 con 0x7fc4fc0c7380 2023-05-16T07:02:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc502212700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc157d60 con 0x7fc4fc0c7760 2023-05-16T07:02:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc501210700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4fc0c7380 0x7fc4fc154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc501210700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4fc0c7380 0x7fc4fc154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41214/0 (socket says 172.21.15.92:41214) 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc501210700 1 -- 172.21.15.92:0/2096497619 learned_addr learned my addr 172.21.15.92:0/2096497619 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.823+0000 7fc500a0f700 1 --2- 172.21.15.92:0/2096497619 >> v2:172.21.15.158:3300/0 conn(0x7fc4fc0c7760 0x7fc4fc0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc501210700 1 -- 172.21.15.92:0/2096497619 >> v2:172.21.15.158:3300/0 conn(0x7fc4fc0c7760 msgr2=0x7fc4fc0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc501210700 1 --2- 172.21.15.92:0/2096497619 >> v2:172.21.15.158:3300/0 conn(0x7fc4fc0c7760 0x7fc4fc0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc501210700 1 -- 172.21.15.92:0/2096497619 >> v1:172.21.15.158:6789/0 conn(0x7fc4fc0ad040 legacy=0x7fc4fc14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc501210700 1 -- 172.21.15.92:0/2096497619 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc158040 con 0x7fc4fc0c7380 2023-05-16T07:02:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc501210700 1 --2- 172.21.15.92:0/2096497619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc154e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc4fc084350 tx=0x7fc4ec009840).ready entity=mon.0 client_cookie=63a47d45f9a8b6b0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:47.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc4fbfff700 1 -- 172.21.15.92:0/2096497619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00e070 con 0x7fc4fc0c7380 2023-05-16T07:02:47.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc4fbfff700 1 -- 172.21.15.92:0/2096497619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec00ba20 con 0x7fc4fc0c7380 2023-05-16T07:02:47.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc4fbfff700 1 -- 172.21.15.92:0/2096497619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00bda0 con 0x7fc4fc0c7380 2023-05-16T07:02:47.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc502212700 1 -- 172.21.15.92:0/2096497619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 msgr2=0x7fc4fc154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:47.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc502212700 1 --2- 172.21.15.92:0/2096497619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc154e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc4fc084350 tx=0x7fc4ec009840).stop 2023-05-16T07:02:47.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc502212700 1 -- 172.21.15.92:0/2096497619 shutdown_connections 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc502212700 1 --2- 172.21.15.92:0/2096497619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc154e50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc502212700 1 --2- 172.21.15.92:0/2096497619 >> v2:172.21.15.158:3300/0 conn(0x7fc4fc0c7760 0x7fc4fc0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.824+0000 7fc502212700 1 -- 172.21.15.92:0/2096497619 >> 172.21.15.92:0/2096497619 conn(0x7fc4fc068330 msgr2=0x7fc4fc068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.825+0000 7fc502212700 1 -- 172.21.15.92:0/2096497619 shutdown_connections 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.825+0000 7fc502212700 1 -- 172.21.15.92:0/2096497619 wait complete. 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.826+0000 7fc502212700 1 Processor -- start 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.826+0000 7fc502212700 1 -- start start 2023-05-16T07:02:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc4fc0ad040 0x7fc4fc0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc4fc0c7760 0x7fc4fc0c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0c3bd0 con 0x7fc4fc0c7380 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc14eca0 con 0x7fc4fc0c7760 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0c2fc0 con 0x7fc4fc0ad040 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc0adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41220/0 (socket says 172.21.15.92:41220) 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 -- 172.21.15.92:0/1115010174 learned_addr learned my addr 172.21.15.92:0/1115010174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc501210700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc4fc0ad040 0x7fc4fc0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc501a11700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc4fc0c7760 0x7fc4fc0c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 -- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc4fc0ad040 msgr2=0x7fc4fc0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc4fc0ad040 0x7fc4fc0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 -- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc4fc0c7760 msgr2=0x7fc4fc0c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc4fc0c7760 0x7fc4fc0c1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 -- 172.21.15.92:0/1115010174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec009ae0 con 0x7fc4fc0c7380 2023-05-16T07:02:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc501a11700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc4fc0c7760 0x7fc4fc0c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc500a0f700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc0adaa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fc4f000e900 tx=0x7fc4f0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0012600 con 0x7fc4fc0c7380 2023-05-16T07:02:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f0012760 con 0x7fc4fc0c7380 2023-05-16T07:02:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc0c32a0 con 0x7fc4fc0c7380 2023-05-16T07:02:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0012ae0 con 0x7fc4fc0c7380 2023-05-16T07:02:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.827+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc0c3860 con 0x7fc4fc0c7380 2023-05-16T07:02:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.828+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc0c3d40 con 0x7fc4fc0c7380 2023-05-16T07:02:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.829+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc4f0022070 con 0x7fc4fc0c7380 2023-05-16T07:02:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.829+0000 7fc4f97fa700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc4e805ed90 0x7fc4e8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.829+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc4f0079cf0 con 0x7fc4fc0c7380 2023-05-16T07:02:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.829+0000 7fc501210700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc4e805ed90 0x7fc4e8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.830+0000 7fc501210700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc4e805ed90 0x7fc4e8061250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fc4ec00c010 tx=0x7fc4ec009840).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.833+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc4f004b290 con 0x7fc4fc0c7380 2023-05-16T07:02:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.997+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4fc047150 con 0x7fc4fc0c7380 2023-05-16T07:02:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.998+0000 7fc4f97fa700 1 -- 172.21.15.92:0/1115010174 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fc4f004e8b0 con 0x7fc4fc0c7380 2023-05-16T07:02:48.002 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:48.003 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc4e805ed90 msgr2=0x7fc4e8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc4e805ed90 0x7fc4e8061250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fc4ec00c010 tx=0x7fc4ec009840).stop 2023-05-16T07:02:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 msgr2=0x7fc4fc0adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc0adaa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fc4f000e900 tx=0x7fc4f0009060).stop 2023-05-16T07:02:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 shutdown_connections 2023-05-16T07:02:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc4e805ed90 0x7fc4e8061250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc4fc0ad040 0x7fc4fc0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc4fc0c7760 0x7fc4fc0c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 --2- 172.21.15.92:0/1115010174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc0c7380 0x7fc4fc0adaa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 >> 172.21.15.92:0/1115010174 conn(0x7fc4fc068330 msgr2=0x7fc4fc14f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 shutdown_connections 2023-05-16T07:02:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:47.999+0000 7fc502212700 1 -- 172.21.15.92:0/1115010174 wait complete. 2023-05-16T07:02:48.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:48.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:48.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.128+0000 7f3270523700 1 Processor -- start 2023-05-16T07:02:48.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.128+0000 7f3270523700 1 -- start start 2023-05-16T07:02:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f3270523700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f3270523700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32680629c0 0x7f3268063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f3270523700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326814ead0 con 0x7f32680c4d40 2023-05-16T07:02:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f3270523700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3268154cb0 con 0x7f32680629c0 2023-05-16T07:02:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f3270523700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32680c6020 con 0x7f326814e6f0 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32680629c0 0x7f3268063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32680629c0 0x7f3268063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41228/0 (socket says 172.21.15.92:41228) 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 -- 172.21.15.92:0/3523588697 learned_addr learned my addr 172.21.15.92:0/3523588697 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326e2bf700 1 --2- 172.21.15.92:0/3523588697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 -- 172.21.15.92:0/3523588697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 msgr2=0x7f3268155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 --2- 172.21.15.92:0/3523588697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 -- 172.21.15.92:0/3523588697 >> v1:172.21.15.158:6789/0 conn(0x7f32680c4d40 legacy=0x7f32680622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 -- 172.21.15.92:0/3523588697 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268154660 con 0x7f32680629c0 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326eac0700 1 --2- 172.21.15.92:0/3523588697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680629c0 0x7f3268063600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f3260009f20 tx=0x7f3260009840).ready entity=mon.0 client_cookie=62025dba0aa6f28b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326d2bd700 1 -- 172.21.15.92:0/3523588697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000e070 con 0x7f32680629c0 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326d2bd700 1 -- 172.21.15.92:0/3523588697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000b9d0 con 0x7f32680629c0 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.129+0000 7f326d2bd700 1 -- 172.21.15.92:0/3523588697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000bd50 con 0x7f32680629c0 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 -- 172.21.15.92:0/3523588697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680629c0 msgr2=0x7f3268063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 --2- 172.21.15.92:0/3523588697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680629c0 0x7f3268063600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f3260009f20 tx=0x7f3260009840).stop 2023-05-16T07:02:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 -- 172.21.15.92:0/3523588697 shutdown_connections 2023-05-16T07:02:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 --2- 172.21.15.92:0/3523588697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680629c0 0x7f3268063600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 --2- 172.21.15.92:0/3523588697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 -- 172.21.15.92:0/3523588697 >> 172.21.15.92:0/3523588697 conn(0x7f326805e820 msgr2=0x7f32680c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 -- 172.21.15.92:0/3523588697 shutdown_connections 2023-05-16T07:02:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.130+0000 7f3270523700 1 -- 172.21.15.92:0/3523588697 wait complete. 2023-05-16T07:02:48.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.131+0000 7f3270523700 1 Processor -- start 2023-05-16T07:02:48.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.131+0000 7f3270523700 1 -- start start 2023-05-16T07:02:48.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f3270523700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32680629c0 0x7f326814ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f3270523700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 0x7f3268150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f3270523700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f3270523700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268151110 con 0x7f32680c4d40 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f3270523700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268153cc0 con 0x7f32680629c0 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f3270523700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326814ee20 con 0x7f326814e6f0 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 0x7f3268150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 0x7f3268150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41230/0 (socket says 172.21.15.92:41230) 2023-05-16T07:02:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 -- 172.21.15.92:0/1314686158 learned_addr learned my addr 172.21.15.92:0/1314686158 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326e2bf700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32680629c0 0x7f326814ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326eac0700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 -- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 msgr2=0x7f3268150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 -- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32680629c0 msgr2=0x7f326814ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32680629c0 0x7f326814ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 -- 172.21.15.92:0/1314686158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009ae0 con 0x7f32680c4d40 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f326dabe700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 0x7f3268150730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f325800c4c0 tx=0x7f3258007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800ce10 con 0x7f32680c4d40 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.132+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258007020 con 0x7f32680c4d40 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.133+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326814cf80 con 0x7f32680c4d40 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.133+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580073c0 con 0x7f32680c4d40 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.133+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3268153420 con 0x7f32680c4d40 2023-05-16T07:02:48.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.133+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326814ef90 con 0x7f32680c4d40 2023-05-16T07:02:48.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.134+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f3258010070 con 0x7f32680c4d40 2023-05-16T07:02:48.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.134+0000 7f325e7fc700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324c05ed90 0x7f324c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.134+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3258078c70 con 0x7f32680c4d40 2023-05-16T07:02:48.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.134+0000 7f326e2bf700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324c05ed90 0x7f324c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.135+0000 7f326e2bf700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324c05ed90 0x7f324c061250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3264009b90 tx=0x7f3264009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.138+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f32580478f0 con 0x7f32680c4d40 2023-05-16T07:02:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.301+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f326806c350 con 0x7f32680c4d40 2023-05-16T07:02:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.301+0000 7f325e7fc700 1 -- 172.21.15.92:0/1314686158 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f325804a210 con 0x7f32680c4d40 2023-05-16T07:02:48.306 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324c05ed90 msgr2=0x7f324c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324c05ed90 0x7f324c061250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3264009b90 tx=0x7f3264009060).stop 2023-05-16T07:02:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 msgr2=0x7f3268150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 0x7f3268150730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f325800c4c0 tx=0x7f3258007dd0).stop 2023-05-16T07:02:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 shutdown_connections 2023-05-16T07:02:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324c05ed90 0x7f324c061250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f326814e6f0 0x7f3268150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f32680629c0 0x7f326814ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 --2- 172.21.15.92:0/1314686158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c4d40 0x7f3268150730 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 >> 172.21.15.92:0/1314686158 conn(0x7f326805e820 msgr2=0x7f3268068970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 shutdown_connections 2023-05-16T07:02:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.303+0000 7f3270523700 1 -- 172.21.15.92:0/1314686158 wait complete. 2023-05-16T07:02:48.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.428+0000 7fddc1eb2700 1 Processor -- start 2023-05-16T07:02:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.428+0000 7fddc1eb2700 1 -- start start 2023-05-16T07:02:48.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddc1eb2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154470 0x7fddbc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddc1eb2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fddbc068160 0x7fddbc0685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddc1eb2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc0c94d0 con 0x7fddbc154090 2023-05-16T07:02:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddc1eb2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc0cb7e0 con 0x7fddbc068160 2023-05-16T07:02:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddc1eb2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc068ac0 con 0x7fddbc154470 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fddbc068160 0x7fddbc0685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fddbc068160 0x7fddbc0685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41242/0 (socket says 172.21.15.92:41242) 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 -- 172.21.15.92:0/2390961614 learned_addr learned my addr 172.21.15.92:0/2390961614 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbaffd700 1 --2- 172.21.15.92:0/2390961614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154470 0x7fddbc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 -- 172.21.15.92:0/2390961614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154470 msgr2=0x7fddbc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 --2- 172.21.15.92:0/2390961614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154470 0x7fddbc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 -- 172.21.15.92:0/2390961614 >> v1:172.21.15.158:6789/0 conn(0x7fddbc154090 legacy=0x7fddbc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 -- 172.21.15.92:0/2390961614 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc0c88c0 con 0x7fddbc068160 2023-05-16T07:02:48.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddbbfff700 1 --2- 172.21.15.92:0/2390961614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc0685d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fddb4009f20 tx=0x7fddb4009840).ready entity=mon.0 client_cookie=1c312d44d061c384 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.429+0000 7fddba7fc700 1 -- 172.21.15.92:0/2390961614 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400e070 con 0x7fddbc068160 2023-05-16T07:02:48.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddba7fc700 1 -- 172.21.15.92:0/2390961614 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400ba80 con 0x7fddbc068160 2023-05-16T07:02:48.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddba7fc700 1 -- 172.21.15.92:0/2390961614 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400be00 con 0x7fddbc068160 2023-05-16T07:02:48.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 -- 172.21.15.92:0/2390961614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 msgr2=0x7fddbc0685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/2390961614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc0685d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fddb4009f20 tx=0x7fddb4009840).stop 2023-05-16T07:02:48.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 -- 172.21.15.92:0/2390961614 shutdown_connections 2023-05-16T07:02:48.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/2390961614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc0685d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/2390961614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154470 0x7fddbc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 -- 172.21.15.92:0/2390961614 >> 172.21.15.92:0/2390961614 conn(0x7fddbc067920 msgr2=0x7fddbc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 -- 172.21.15.92:0/2390961614 shutdown_connections 2023-05-16T07:02:48.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.430+0000 7fddc1eb2700 1 -- 172.21.15.92:0/2390961614 wait complete. 2023-05-16T07:02:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.431+0000 7fddc1eb2700 1 Processor -- start 2023-05-16T07:02:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.431+0000 7fddc1eb2700 1 -- start start 2023-05-16T07:02:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddc1eb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddc1eb2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154090 0x7fddbc14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddc1eb2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fddbc154470 0x7fddbc0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddc1eb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc1561f0 con 0x7fddbc068160 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddc1eb2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc1508c0 con 0x7fddbc154470 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddc1eb2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc150ba0 con 0x7fddbc154090 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc1564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41244/0 (socket says 172.21.15.92:41244) 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 -- 172.21.15.92:0/266185235 learned_addr learned my addr 172.21.15.92:0/266185235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbaffd700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154090 0x7fddbc14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbbfff700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fddbc154470 0x7fddbc0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 -- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154090 msgr2=0x7fddbc14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154090 0x7fddbc14ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 -- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fddbc154470 msgr2=0x7fddbc0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fddbc154470 0x7fddbc0c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.432+0000 7fddbb7fe700 1 -- 172.21.15.92:0/266185235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4009ae0 con 0x7fddbc068160 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fddbb7fe700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc1564d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fddac00c4c0 tx=0x7fddac007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac00ce70 con 0x7fddbc068160 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac001e90 con 0x7fddbc068160 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc150e80 con 0x7fddbc068160 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac007250 con 0x7fddbc068160 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc0ae5c0 con 0x7fddbc068160 2023-05-16T07:02:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.433+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc066730 con 0x7fddbc068160 2023-05-16T07:02:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.434+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fddac010070 con 0x7fddbc068160 2023-05-16T07:02:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.434+0000 7fdda37fe700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdda805ed90 0x7fdda8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.434+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fddac0793e0 con 0x7fddbc068160 2023-05-16T07:02:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.434+0000 7fddbaffd700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdda805ed90 0x7fdda8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.435+0000 7fddbaffd700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdda805ed90 0x7fdda8061250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fddbc167740 tx=0x7fddb0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.438+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fddac04a870 con 0x7fddbc068160 2023-05-16T07:02:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.601+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fddbc0c8ba0 con 0x7fddbc068160 2023-05-16T07:02:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.601+0000 7fdda37fe700 1 -- 172.21.15.92:0/266185235 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fddac04de90 con 0x7fddbc068160 2023-05-16T07:02:48.606 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdda805ed90 msgr2=0x7fdda8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdda805ed90 0x7fdda8061250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fddbc167740 tx=0x7fddb0008040).stop 2023-05-16T07:02:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 msgr2=0x7fddbc1564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc1564d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fddac00c4c0 tx=0x7fddac007e00).stop 2023-05-16T07:02:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 shutdown_connections 2023-05-16T07:02:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdda805ed90 0x7fdda8061250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fddbc154470 0x7fddbc0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddbc068160 0x7fddbc1564d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 --2- 172.21.15.92:0/266185235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fddbc154090 0x7fddbc14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 >> 172.21.15.92:0/266185235 conn(0x7fddbc067920 msgr2=0x7fddbc0bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 shutdown_connections 2023-05-16T07:02:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.603+0000 7fddc1eb2700 1 -- 172.21.15.92:0/266185235 wait complete. 2023-05-16T07:02:48.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:48.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.728+0000 7f700ed3c700 1 Processor -- start 2023-05-16T07:02:48.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.728+0000 7f700ed3c700 1 -- start start 2023-05-16T07:02:48.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f700ed3c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7008154090 0x7f70080aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f700ed3c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008068160 0x7f70080685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f700ed3c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080c94d0 con 0x7f7008154470 2023-05-16T07:02:48.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f700ed3c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70080cb7e0 con 0x7f7008154090 2023-05-16T07:02:48.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f700ed3c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7008068aa0 con 0x7f7008068160 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f7007fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7008154470 0x7f70080a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52002/0 (socket says 172.21.15.92:52002) 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.729+0000 7f7007fff700 1 -- 172.21.15.92:0/723990900 learned_addr learned my addr 172.21.15.92:0/723990900 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700d2d9700 1 --2- 172.21.15.92:0/723990900 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008068160 0x7f70080685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700cad8700 1 --2- 172.21.15.92:0/723990900 >> v2:172.21.15.158:3300/0 conn(0x7f7008154090 0x7f70080aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f70077fe700 1 -- 172.21.15.92:0/723990900 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213994742 0 0) 0x7f70080c94d0 con 0x7f7008154470 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f70077fe700 1 -- 172.21.15.92:0/723990900 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff00033f0 con 0x7f7008154470 2023-05-16T07:02:48.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700d2d9700 1 -- 172.21.15.92:0/723990900 >> v2:172.21.15.158:3300/0 conn(0x7f7008154090 msgr2=0x7f70080aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700d2d9700 1 --2- 172.21.15.92:0/723990900 >> v2:172.21.15.158:3300/0 conn(0x7f7008154090 0x7f70080aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700d2d9700 1 -- 172.21.15.92:0/723990900 >> v1:172.21.15.92:6789/0 conn(0x7f7008154470 legacy=0x7f70080a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f70077fe700 1 -- 172.21.15.92:0/723990900 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098939100 0 0) 0x7f6ff00033f0 con 0x7f7008154470 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700d2d9700 1 -- 172.21.15.92:0/723990900 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080c88c0 con 0x7f7008068160 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700cad8700 1 --2- 172.21.15.92:0/723990900 >> v2:172.21.15.158:3300/0 conn(0x7f7008154090 0x7f70080aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f700d2d9700 1 --2- 172.21.15.92:0/723990900 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008068160 0x7f70080685b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7000009e60 tx=0x7f7000009780).ready entity=mon.2 client_cookie=a4954daa79e968cc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f70077fe700 1 -- 172.21.15.92:0/723990900 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000e070 con 0x7f7008068160 2023-05-16T07:02:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f70077fe700 1 -- 172.21.15.92:0/723990900 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700000b7a0 con 0x7f7008068160 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.730+0000 7f70077fe700 1 -- 172.21.15.92:0/723990900 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc001470 con 0x7f7008068160 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 -- 172.21.15.92:0/723990900 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008068160 msgr2=0x7f70080685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 --2- 172.21.15.92:0/723990900 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008068160 0x7f70080685b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7000009e60 tx=0x7f7000009780).stop 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 -- 172.21.15.92:0/723990900 shutdown_connections 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 --2- 172.21.15.92:0/723990900 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008068160 0x7f70080685b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 --2- 172.21.15.92:0/723990900 >> v2:172.21.15.158:3300/0 conn(0x7f7008154090 0x7f70080aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 -- 172.21.15.92:0/723990900 >> 172.21.15.92:0/723990900 conn(0x7f7008067920 msgr2=0x7f70080cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 -- 172.21.15.92:0/723990900 shutdown_connections 2023-05-16T07:02:48.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.731+0000 7f700ed3c700 1 -- 172.21.15.92:0/723990900 wait complete. 2023-05-16T07:02:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.732+0000 7f700ed3c700 1 Processor -- start 2023-05-16T07:02:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.732+0000 7f700ed3c700 1 -- start start 2023-05-16T07:02:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7008068160 0x7f700814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008154090 0x7f7008164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 0x7f7008154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008151a20 con 0x7f7008154470 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008164b50 con 0x7f7008068160 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7008164e30 con 0x7f7008154090 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 0x7f7008154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 0x7f7008154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41260/0 (socket says 172.21.15.92:41260) 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 -- 172.21.15.92:0/3136022303 learned_addr learned my addr 172.21.15.92:0/3136022303 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700cad8700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7008068160 0x7f700814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f7007fff700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008154090 0x7f7008164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 -- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008154090 msgr2=0x7f7008164260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008154090 0x7f7008164260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 -- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7008068160 msgr2=0x7f700814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7008068160 0x7f700814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 -- 172.21.15.92:0/3136022303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80092a0 con 0x7f7008154470 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700d2d9700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 0x7f7008154850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f700000b610 tx=0x7f7000009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000009440 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700001d420 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7000009a20 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700001d7e0 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.733+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70081515b0 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.734+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008151860 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.735+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f700001dbd0 con 0x7f7008154470 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.735+0000 7f7004ff9700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ff405ed90 0x7f6ff4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.735+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f700007ba90 con 0x7f7008154470 2023-05-16T07:02:48.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.735+0000 7f700cad8700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ff405ed90 0x7f6ff4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:48.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.736+0000 7f700cad8700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ff405ed90 0x7f6ff4061250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6ff8009dc0 tx=0x7f6ff8009060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:48.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.739+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7008151860 con 0x7f7008154470 2023-05-16T07:02:48.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.900+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7008151830 con 0x7f7008154470 2023-05-16T07:02:48.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.900+0000 7f7004ff9700 1 -- 172.21.15.92:0/3136022303 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f700001de60 con 0x7f7008154470 2023-05-16T07:02:48.904 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:48.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ff405ed90 msgr2=0x7f6ff4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ff405ed90 0x7f6ff4061250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6ff8009dc0 tx=0x7f6ff8009060).stop 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 msgr2=0x7f7008154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 0x7f7008154850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f700000b610 tx=0x7f7000009780).stop 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 shutdown_connections 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ff405ed90 0x7f6ff4061250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008154470 0x7f7008154850 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7008068160 0x7f700814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 --2- 172.21.15.92:0/3136022303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7008154090 0x7f7008164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:48.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 >> 172.21.15.92:0/3136022303 conn(0x7f7008067920 msgr2=0x7f700814fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:48.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 shutdown_connections 2023-05-16T07:02:48.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:48.902+0000 7f700ed3c700 1 -- 172.21.15.92:0/3136022303 wait complete. 2023-05-16T07:02:48.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.026+0000 7fbfa8420700 1 Processor -- start 2023-05-16T07:02:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.026+0000 7fbfa8420700 1 -- start start 2023-05-16T07:02:49.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa8420700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbfa014e6f0 0x7fbfa0155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa8420700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa014ead0 con 0x7fbfa00c4d40 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa8420700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa0154cb0 con 0x7fbfa00629c0 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa8420700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa00c6020 con 0x7fbfa014e6f0 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa59bb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbfa00c4d40 0x7fbfa00622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52014/0 (socket says 172.21.15.92:52014) 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa59bb700 1 -- 172.21.15.92:0/1519536797 learned_addr learned my addr 172.21.15.92:0/1519536797 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490766679 0 0) 0x7fbfa014ead0 con 0x7fbfa00c4d40 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf8c0033f0 con 0x7fbfa00c4d40 2023-05-16T07:02:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.027+0000 7fbfa61bc700 1 --2- 172.21.15.92:0/1519536797 >> v2:172.21.15.158:3300/0 conn(0x7fbfa014e6f0 0x7fbfa0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584001967 0 0) 0x7fbf8c0033f0 con 0x7fbfa00c4d40 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 >> v2:172.21.15.158:3300/0 conn(0x7fbfa014e6f0 msgr2=0x7fbfa0155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 --2- 172.21.15.92:0/1519536797 >> v2:172.21.15.158:3300/0 conn(0x7fbfa014e6f0 0x7fbfa0155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 >> v1:172.21.15.158:6789/0 conn(0x7fbfa00629c0 legacy=0x7fbfa0063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0154660 con 0x7fbfa00c4d40 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fbf90001470 con 0x7fbfa00c4d40 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 >> v1:172.21.15.92:6789/0 conn(0x7fbfa00c4d40 legacy=0x7fbfa00622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c0038e0 0x7fbf8c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf8c004690 0x7fbf8c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf8c005000 0x7fbf8c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00c6020 con 0x7fbf8c004690 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c00beb0 con 0x7fbf8c0038e0 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c00c070 con 0x7fbf8c005000 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf8c004690 0x7fbf8c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa61bc700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c0038e0 0x7fbf8c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa69bd700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf8c005000 0x7fbf8c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 -- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf8c005000 msgr2=0x7fbf8c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf8c005000 0x7fbf8c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 -- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c0038e0 msgr2=0x7fbf8c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c0038e0 0x7fbf8c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 -- 172.21.15.92:0/1519536797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0033f0 con 0x7fbf8c004690 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa69bd700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf8c005000 0x7fbf8c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa61bc700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c0038e0 0x7fbf8c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa69bd700 1 -- 172.21.15.92:0/1519536797 reap_dead start 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf8c004690 0x7fbf8c004b10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fbf9000c190 tx=0x7fbf90001e50).ready entity=mon.0 client_cookie=65261a653c870031 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90007a20 con 0x7fbf8c004690 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9000cac0 con 0x7fbf8c004690 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.028+0000 7fbfa51ba700 1 -- 172.21.15.92:0/1519536797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000ce40 con 0x7fbf8c004690 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 -- 172.21.15.92:0/1519536797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf8c004690 msgr2=0x7fbf8c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf8c004690 0x7fbf8c004b10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fbf9000c190 tx=0x7fbf90001e50).stop 2023-05-16T07:02:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 -- 172.21.15.92:0/1519536797 shutdown_connections 2023-05-16T07:02:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 --2- 172.21.15.92:0/1519536797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf8c004690 0x7fbf8c004b10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 -- 172.21.15.92:0/1519536797 >> 172.21.15.92:0/1519536797 conn(0x7fbfa005e820 msgr2=0x7fbfa00c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 -- 172.21.15.92:0/1519536797 shutdown_connections 2023-05-16T07:02:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.029+0000 7fbfa8420700 1 -- 172.21.15.92:0/1519536797 wait complete. 2023-05-16T07:02:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.030+0000 7fbfa8420700 1 Processor -- start 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.030+0000 7fbfa8420700 1 -- start start 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c004690 0x7fbfa0165660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 0x7fbfa00aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfa00abf30 0x7fbfa00c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00ad160 con 0x7fbfa00ad580 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00ac4e0 con 0x7fbf8c004690 2023-05-16T07:02:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00ac7c0 con 0x7fbfa00abf30 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 0x7fbfa00aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 0x7fbfa00aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41278/0 (socket says 172.21.15.92:41278) 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 -- 172.21.15.92:0/3806919973 learned_addr learned my addr 172.21.15.92:0/3806919973 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa69bd700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfa00abf30 0x7fbfa00c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 -- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfa00abf30 msgr2=0x7fbfa00c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfa00abf30 0x7fbfa00c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa61bc700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c004690 0x7fbfa0165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 -- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c004690 msgr2=0x7fbfa0165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c004690 0x7fbfa0165660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 -- 172.21.15.92:0/3806919973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa00acaa0 con 0x7fbfa00ad580 2023-05-16T07:02:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa59bb700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 0x7fbfa00aba40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fbf90001ed0 tx=0x7fbf90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90007430 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf90007590 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa00acd80 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900048e0 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.031+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0164e90 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.032+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa00ad2a0 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.033+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbf90010070 con 0x7fbfa00ad580 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.033+0000 7fbf967fc700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf8405ed90 0x7fbf84061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.033+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbf9007a380 con 0x7fbfa00ad580 2023-05-16T07:02:49.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.033+0000 7fbfa61bc700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf8405ed90 0x7fbf84061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.034+0000 7fbfa61bc700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf8405ed90 0x7fbf84061250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbf98009be0 tx=0x7fbf98008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.037+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbfa00ad2a0 con 0x7fbfa00ad580 2023-05-16T07:02:49.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.200+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfa00ad2a0 con 0x7fbfa00ad580 2023-05-16T07:02:49.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.200+0000 7fbf967fc700 1 -- 172.21.15.92:0/3806919973 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfa00ad2a0 con 0x7fbfa00ad580 2023-05-16T07:02:49.204 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:49.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf8405ed90 msgr2=0x7fbf84061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf8405ed90 0x7fbf84061250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbf98009be0 tx=0x7fbf98008040).stop 2023-05-16T07:02:49.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 msgr2=0x7fbfa00aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 0x7fbfa00aba40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fbf90001ed0 tx=0x7fbf90015000).stop 2023-05-16T07:02:49.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 shutdown_connections 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf8405ed90 0x7fbf84061250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfa00abf30 0x7fbfa00c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf8c004690 0x7fbfa0165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 --2- 172.21.15.92:0/3806919973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00ad580 0x7fbfa00aba40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 >> 172.21.15.92:0/3806919973 conn(0x7fbfa005e820 msgr2=0x7fbfa00c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 shutdown_connections 2023-05-16T07:02:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.202+0000 7fbfa8420700 1 -- 172.21.15.92:0/3806919973 wait complete. 2023-05-16T07:02:49.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.324+0000 7fe4556bd700 1 Processor -- start 2023-05-16T07:02:49.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.324+0000 7fe4556bd700 1 -- start start 2023-05-16T07:02:49.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe4556bd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154550 0x7fe4500696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe4556bd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe450069b90 0x7fe4500c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe4556bd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4500c0660 con 0x7fe450154170 2023-05-16T07:02:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe4556bd700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45006a020 con 0x7fe450069b90 2023-05-16T07:02:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe4556bd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500bfd30 con 0x7fe450154550 2023-05-16T07:02:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe44e7fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154550 0x7fe4500696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe44f7fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe450069b90 0x7fe4500c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe44f7fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe450069b90 0x7fe4500c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:44924/0 (socket says 172.21.15.92:44924) 2023-05-16T07:02:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.325+0000 7fe44f7fe700 1 -- 172.21.15.92:0/4203127603 learned_addr learned my addr 172.21.15.92:0/4203127603 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44f7fe700 1 -- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154550 msgr2=0x7fe4500696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44f7fe700 1 --2- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154550 0x7fe4500696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44f7fe700 1 -- 172.21.15.92:0/4203127603 >> v1:172.21.15.158:6789/0 conn(0x7fe450154170 legacy=0x7fe4500c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44dffb700 1 -- 172.21.15.92:0/4203127603 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276405935 0 0) 0x7fe4500c0660 con 0x7fe450154170 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44f7fe700 1 -- 172.21.15.92:0/4203127603 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500c0010 con 0x7fe450069b90 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44f7fe700 1 --2- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450069b90 0x7fe4500c0af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe448006fd0 tx=0x7fe4480090f0).ready entity=mon.1 client_cookie=5c98a4bce06dbb36 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44dffb700 1 -- 172.21.15.92:0/4203127603 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800b580 con 0x7fe450069b90 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44dffb700 1 -- 172.21.15.92:0/4203127603 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44800b6e0 con 0x7fe450069b90 2023-05-16T07:02:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.326+0000 7fe44dffb700 1 -- 172.21.15.92:0/4203127603 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800ba80 con 0x7fe450069b90 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 -- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450069b90 msgr2=0x7fe4500c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 --2- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450069b90 0x7fe4500c0af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe448006fd0 tx=0x7fe4480090f0).stop 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 -- 172.21.15.92:0/4203127603 shutdown_connections 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 --2- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450069b90 0x7fe4500c0af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 --2- 172.21.15.92:0/4203127603 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154550 0x7fe4500696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 -- 172.21.15.92:0/4203127603 >> 172.21.15.92:0/4203127603 conn(0x7fe4500cb700 msgr2=0x7fe4500c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 -- 172.21.15.92:0/4203127603 shutdown_connections 2023-05-16T07:02:49.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.327+0000 7fe4556bd700 1 -- 172.21.15.92:0/4203127603 wait complete. 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.328+0000 7fe4556bd700 1 Processor -- start 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.328+0000 7fe4556bd700 1 -- start start 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 0x7fe450148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154170 0x7fe4500c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450154550 0x7fe450068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450153580 con 0x7fe450069b90 2023-05-16T07:02:49.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450150dd0 con 0x7fe450154550 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe450148780 con 0x7fe450154170 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 0x7fe450148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 0x7fe450148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41290/0 (socket says 172.21.15.92:41290) 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 -- 172.21.15.92:0/816031255 learned_addr learned my addr 172.21.15.92:0/816031255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44e7fc700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154170 0x7fe4500c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44f7fe700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450154550 0x7fe450068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 -- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154170 msgr2=0x7fe4500c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154170 0x7fe4500c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 -- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450154550 msgr2=0x7fe450068090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450154550 0x7fe450068090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 -- 172.21.15.92:0/816031255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448009370 con 0x7fe450069b90 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44e7fc700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154170 0x7fe4500c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe44effd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 0x7fe450148290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fe44000c930 tx=0x7fe440001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000a780 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000a900 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4500c4300 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000acc0 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.329+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe450150a60 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.330+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4501510b0 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.331+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe440010070 con 0x7fe450069b90 2023-05-16T07:02:49.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.331+0000 7fe446ffd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe43005ed90 0x7fe430061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.331+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe440079a40 con 0x7fe450069b90 2023-05-16T07:02:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.331+0000 7fe44e7fc700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe43005ed90 0x7fe430061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.332+0000 7fe44e7fc700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe43005ed90 0x7fe430061250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe438006d80 tx=0x7fe438008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.335+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe44004afe0 con 0x7fe450069b90 2023-05-16T07:02:49.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.498+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4500666c0 con 0x7fe450069b90 2023-05-16T07:02:49.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.499+0000 7fe446ffd700 1 -- 172.21.15.92:0/816031255 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fe44004e600 con 0x7fe450069b90 2023-05-16T07:02:49.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:49.504 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:49.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe43005ed90 msgr2=0x7fe430061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe43005ed90 0x7fe430061250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe438006d80 tx=0x7fe438008040).stop 2023-05-16T07:02:49.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 msgr2=0x7fe450148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 0x7fe450148290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fe44000c930 tx=0x7fe440001630).stop 2023-05-16T07:02:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 shutdown_connections 2023-05-16T07:02:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe43005ed90 0x7fe430061250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe450154550 0x7fe450068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe450069b90 0x7fe450148290 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 --2- 172.21.15.92:0/816031255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe450154170 0x7fe4500c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 >> 172.21.15.92:0/816031255 conn(0x7fe4500cb700 msgr2=0x7fe4500ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 shutdown_connections 2023-05-16T07:02:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.501+0000 7fe4556bd700 1 -- 172.21.15.92:0/816031255 wait complete. 2023-05-16T07:02:49.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:49.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.628+0000 7fc958c9d700 1 Processor -- start 2023-05-16T07:02:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.628+0000 7fc958c9d700 1 -- start start 2023-05-16T07:02:49.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc958c9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc954154090 0x7fc9540aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc958c9d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9540c94d0 con 0x7fc954068160 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc958c9d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9540cb7e0 con 0x7fc954154470 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc958c9d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9540c88c0 con 0x7fc954154090 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc952d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc954068160 0x7fc9540c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52048/0 (socket says 172.21.15.92:52048) 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc952d9d700 1 -- 172.21.15.92:0/2944678235 learned_addr learned my addr 172.21.15.92:0/2944678235 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc95259c700 1 --2- 172.21.15.92:0/2944678235 >> v2:172.21.15.158:3300/0 conn(0x7fc954154090 0x7fc9540aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360613472 0 0) 0x7fc9540c94d0 con 0x7fc954068160 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9400033f0 con 0x7fc954068160 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 985227149 0 0) 0x7fc9400033f0 con 0x7fc954068160 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 >> v2:172.21.15.158:3300/0 conn(0x7fc954154090 msgr2=0x7fc9540aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 --2- 172.21.15.92:0/2944678235 >> v2:172.21.15.158:3300/0 conn(0x7fc954154090 0x7fc9540aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 >> v1:172.21.15.158:6789/0 conn(0x7fc954154470 legacy=0x7fc9540a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540c8ba0 con 0x7fc954068160 2023-05-16T07:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.629+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc94c0017f0 con 0x7fc954068160 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 >> v1:172.21.15.92:6789/0 conn(0x7fc954068160 legacy=0x7fc9540c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9400038e0 0x7fc940003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc940004690 0x7fc940004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc940005000 0x7fc94000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540c88c0 con 0x7fc9400038e0 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94000bf60 con 0x7fc940004690 2023-05-16T07:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94000c0f0 con 0x7fc940005000 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9400038e0 0x7fc940003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc94bfff700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc940004690 0x7fc940004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc952d9d700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc940005000 0x7fc94000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 -- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc940005000 msgr2=0x7fc94000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc940005000 0x7fc94000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 -- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc940004690 msgr2=0x7fc940004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc940004690 0x7fc940004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 -- 172.21.15.92:0/2944678235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400033f0 con 0x7fc9400038e0 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc952d9d700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc940005000 0x7fc94000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc952d9d700 1 -- 172.21.15.92:0/2944678235 reap_dead start 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc95259c700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9400038e0 0x7fc940003d00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fc944009a80 tx=0x7fc944009060).ready entity=mon.0 client_cookie=2682919af241129b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944009430 con 0x7fc9400038e0 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94400e640 con 0x7fc9400038e0 2023-05-16T07:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.630+0000 7fc951d9b700 1 -- 172.21.15.92:0/2944678235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400e9c0 con 0x7fc9400038e0 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 -- 172.21.15.92:0/2944678235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9400038e0 msgr2=0x7fc940003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9400038e0 0x7fc940003d00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fc944009a80 tx=0x7fc944009060).stop 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 -- 172.21.15.92:0/2944678235 shutdown_connections 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2944678235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9400038e0 0x7fc940003d00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 -- 172.21.15.92:0/2944678235 >> 172.21.15.92:0/2944678235 conn(0x7fc954067920 msgr2=0x7fc9540cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 -- 172.21.15.92:0/2944678235 shutdown_connections 2023-05-16T07:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.631+0000 7fc958c9d700 1 -- 172.21.15.92:0/2944678235 wait complete. 2023-05-16T07:02:49.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 Processor -- start 2023-05-16T07:02:49.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 -- start start 2023-05-16T07:02:49.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc9400038e0 0x7fc9540c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 0x7fc9540ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9540ae550 0x7fc95414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954168260 con 0x7fc95414c790 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954168540 con 0x7fc9540ae550 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.632+0000 7fc958c9d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc954168820 con 0x7fc9400038e0 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 0x7fc9540ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 0x7fc9540ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41312/0 (socket says 172.21.15.92:41312) 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 -- 172.21.15.92:0/2871580499 learned_addr learned my addr 172.21.15.92:0/2871580499 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:49.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc95259c700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc9400038e0 0x7fc9540c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc952d9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9540ae550 0x7fc95414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 -- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc9400038e0 msgr2=0x7fc9540c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc9400038e0 0x7fc9540c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 -- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9540ae550 msgr2=0x7fc95414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9540ae550 0x7fc95414dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 -- 172.21.15.92:0/2871580499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc954168b00 con 0x7fc95414c790 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc95259c700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc9400038e0 0x7fc9540c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94bfff700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 0x7fc9540ae060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fc93c0028c0 tx=0x7fc93c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00f6c0 con 0x7fc95414c790 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c00f820 con 0x7fc95414c790 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9540c37c0 con 0x7fc95414c790 2023-05-16T07:02:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00fba0 con 0x7fc95414c790 2023-05-16T07:02:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.633+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954167fb0 con 0x7fc95414c790 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.634+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9540c9cd0 con 0x7fc95414c790 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.634+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc93c01b030 con 0x7fc95414c790 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.635+0000 7fc94a7fc700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc93405ed90 0x7fc934061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.635+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc93c079650 con 0x7fc95414c790 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.635+0000 7fc95259c700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc93405ed90 0x7fc934061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.635+0000 7fc95259c700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc93405ed90 0x7fc934061250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc944006d50 tx=0x7fc94401a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.638+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc93c04aca0 con 0x7fc95414c790 2023-05-16T07:02:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.799+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc95406c350 con 0x7fc95414c790 2023-05-16T07:02:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.799+0000 7fc94a7fc700 1 -- 172.21.15.92:0/2871580499 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc93c04e2c0 con 0x7fc95414c790 2023-05-16T07:02:49.804 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:49.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc93405ed90 msgr2=0x7fc934061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc93405ed90 0x7fc934061250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc944006d50 tx=0x7fc94401a000).stop 2023-05-16T07:02:49.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 msgr2=0x7fc9540ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 0x7fc9540ae060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fc93c0028c0 tx=0x7fc93c007bb0).stop 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 shutdown_connections 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc93405ed90 0x7fc934061250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc9400038e0 0x7fc9540c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9540ae550 0x7fc95414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 --2- 172.21.15.92:0/2871580499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95414c790 0x7fc9540ae060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 >> 172.21.15.92:0/2871580499 conn(0x7fc954067920 msgr2=0x7fc9540bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 shutdown_connections 2023-05-16T07:02:49.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.801+0000 7fc958c9d700 1 -- 172.21.15.92:0/2871580499 wait complete. 2023-05-16T07:02:49.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:49.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.929+0000 7fccba873700 1 Processor -- start 2023-05-16T07:02:49.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.930+0000 7fccba873700 1 -- start start 2023-05-16T07:02:49.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.930+0000 7fccba873700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fccb40aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.930+0000 7fccba873700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb4069440 con 0x7fccb4062b40 2023-05-16T07:02:49.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.930+0000 7fccba873700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb4069fe0 con 0x7fccb40aa100 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.930+0000 7fccba873700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4069110 con 0x7fccb4062f20 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb9871700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fccb4062b40 0x7fccb40a75b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52066/0 (socket says 172.21.15.92:52066) 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb9871700 1 -- 172.21.15.92:0/1329580675 learned_addr learned my addr 172.21.15.92:0/1329580675 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccba072700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fccb40aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551668612 0 0) 0x7fccb4069440 con 0x7fccb4062b40 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc9c0033f0 con 0x7fccb4062b40 2023-05-16T07:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725075674 0 0) 0x7fccb4069fe0 con 0x7fccb40aa100 2023-05-16T07:02:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb4069440 con 0x7fccb40aa100 2023-05-16T07:02:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4148060737 0 0) 0x7fcc9c0033f0 con 0x7fccb4062b40 2023-05-16T07:02:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 msgr2=0x7fccb40aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fccb40aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 >> v1:172.21.15.158:6789/0 conn(0x7fccb40aa100 legacy=0x7fccb40698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb4065bb0 con 0x7fccb4062b40 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fcca40017f0 con 0x7fccb4062b40 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccba072700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fccb40aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 >> v1:172.21.15.92:6789/0 conn(0x7fccb4062b40 legacy=0x7fccb40a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc9c003aa0 0x7fcc9c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc9c004690 0x7fcc9c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fcc9c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4069110 con 0x7fcc9c003aa0 2023-05-16T07:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c00b8e0 con 0x7fcc9c004690 2023-05-16T07:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c00baa0 con 0x7fccb4062f20 2023-05-16T07:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb9871700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc9c003aa0 0x7fcc9c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccba072700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fcc9c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.931+0000 7fccb9070700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc9c004690 0x7fcc9c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9871700 1 -- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 msgr2=0x7fcc9c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9871700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fcc9c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9871700 1 -- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc9c004690 msgr2=0x7fcc9c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9871700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc9c004690 0x7fcc9c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9871700 1 -- 172.21.15.92:0/1329580675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0092a0 con 0x7fcc9c003aa0 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9070700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc9c004690 0x7fcc9c004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba072700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccb4062f20 0x7fcc9c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb9871700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc9c003aa0 0x7fcc9c003ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fcca400c5c0 tx=0x7fcca4007f40).ready entity=mon.0 client_cookie=d575f87a9243ee5b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400cdc0 con 0x7fcc9c003aa0 2023-05-16T07:02:49.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4005b20 con 0x7fcc9c003aa0 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccb886f700 1 -- 172.21.15.92:0/1329580675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40070d0 con 0x7fcc9c003aa0 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba873700 1 -- 172.21.15.92:0/1329580675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc9c003aa0 msgr2=0x7fcc9c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba873700 1 --2- 172.21.15.92:0/1329580675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc9c003aa0 0x7fcc9c003ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fcca400c5c0 tx=0x7fcca4007f40).stop 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba072700 1 -- 172.21.15.92:0/1329580675 reap_dead start 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba873700 1 -- 172.21.15.92:0/1329580675 shutdown_connections 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba873700 1 -- 172.21.15.92:0/1329580675 >> 172.21.15.92:0/1329580675 conn(0x7fccb4068a30 msgr2=0x7fccb40c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba873700 1 -- 172.21.15.92:0/1329580675 shutdown_connections 2023-05-16T07:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.932+0000 7fccba873700 1 -- 172.21.15.92:0/1329580675 wait complete. 2023-05-16T07:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.933+0000 7fccba873700 1 Processor -- start 2023-05-16T07:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.933+0000 7fccba873700 1 -- start start 2023-05-16T07:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba873700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc9c003aa0 0x7fccb414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba873700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 0x7fccb4152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba873700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccb413cca0 0x7fccb4066e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba873700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4153070 con 0x7fccb40abcc0 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba873700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb414ad90 con 0x7fccb413cca0 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba873700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb40ac1c0 con 0x7fcc9c003aa0 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccb9070700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 0x7fccb4152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccb9070700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 0x7fccb4152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41332/0 (socket says 172.21.15.92:41332) 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccb9070700 1 -- 172.21.15.92:0/1587398977 learned_addr learned my addr 172.21.15.92:0/1587398977 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccba072700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccb413cca0 0x7fccb4066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.934+0000 7fccb9871700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc9c003aa0 0x7fccb414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9070700 1 -- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc9c003aa0 msgr2=0x7fccb414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9070700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc9c003aa0 0x7fccb414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9070700 1 -- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccb413cca0 msgr2=0x7fccb4066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9070700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccb413cca0 0x7fccb4066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9070700 1 -- 172.21.15.92:0/1587398977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb40ac4a0 con 0x7fccb40abcc0 2023-05-16T07:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9871700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc9c003aa0 0x7fccb414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb9070700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 0x7fccb4152780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fcca800e810 tx=0x7fcca8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8003030 con 0x7fccb40abcc0 2023-05-16T07:02:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca8003190 con 0x7fccb40abcc0 2023-05-16T07:02:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb40ac780 con 0x7fccb40abcc0 2023-05-16T07:02:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca80017c0 con 0x7fccb40abcc0 2023-05-16T07:02:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.935+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb4152c70 con 0x7fccb40abcc0 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.936+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb40a7830 con 0x7fccb40abcc0 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.936+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcca8001bb0 con 0x7fccb40abcc0 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.937+0000 7fccb1ffb700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca005ed90 0x7fcca0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.937+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcca807a050 con 0x7fccb40abcc0 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.937+0000 7fccb9871700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca005ed90 0x7fcca0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.937+0000 7fccb9871700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca005ed90 0x7fcca0061250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fcca4001df0 tx=0x7fcca4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:49.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:49.941+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcca804b5f0 con 0x7fccb40abcc0 2023-05-16T07:02:50.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.102+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fccb4047150 con 0x7fccb40abcc0 2023-05-16T07:02:50.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.102+0000 7fccb1ffb700 1 -- 172.21.15.92:0/1587398977 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcca8003300 con 0x7fccb40abcc0 2023-05-16T07:02:50.107 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:50.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca005ed90 msgr2=0x7fcca0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca005ed90 0x7fcca0061250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fcca4001df0 tx=0x7fcca4015000).stop 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 msgr2=0x7fccb4152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 0x7fccb4152780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fcca800e810 tx=0x7fcca8009040).stop 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 shutdown_connections 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca005ed90 0x7fcca0061250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc9c003aa0 0x7fccb414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccb413cca0 0x7fccb4066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 --2- 172.21.15.92:0/1587398977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb40abcc0 0x7fccb4152780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 >> 172.21.15.92:0/1587398977 conn(0x7fccb4068a30 msgr2=0x7fccb414fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 shutdown_connections 2023-05-16T07:02:50.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.104+0000 7fccba873700 1 -- 172.21.15.92:0/1587398977 wait complete. 2023-05-16T07:02:50.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.229+0000 7fd414842700 1 Processor -- start 2023-05-16T07:02:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.229+0000 7fd414842700 1 -- start start 2023-05-16T07:02:50.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd414842700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153bb0 0x7fd4100698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd414842700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd410153f90 0x7fd410067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd414842700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd4100bd810 0x7fd4100bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd414842700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41005e760 con 0x7fd410153f90 2023-05-16T07:02:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd414842700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd410069ef0 con 0x7fd4100bd810 2023-05-16T07:02:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd414842700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100bf540 con 0x7fd410153bb0 2023-05-16T07:02:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd40e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd410153f90 0x7fd410067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd40e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd410153f90 0x7fd410067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41340/0 (socket says 172.21.15.92:41340) 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.230+0000 7fd40e59c700 1 -- 172.21.15.92:0/3702694490 learned_addr learned my addr 172.21.15.92:0/3702694490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40ed9d700 1 --2- 172.21.15.92:0/3702694490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153bb0 0x7fd4100698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40f59e700 1 --2- 172.21.15.92:0/3702694490 >> v2:172.21.15.158:3300/0 conn(0x7fd4100bd810 0x7fd4100bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40e59c700 1 -- 172.21.15.92:0/3702694490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153bb0 msgr2=0x7fd4100698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40e59c700 1 --2- 172.21.15.92:0/3702694490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153bb0 0x7fd4100698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40e59c700 1 -- 172.21.15.92:0/3702694490 >> v2:172.21.15.158:3300/0 conn(0x7fd4100bd810 msgr2=0x7fd4100bdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40e59c700 1 --2- 172.21.15.92:0/3702694490 >> v2:172.21.15.158:3300/0 conn(0x7fd4100bd810 0x7fd4100bdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40e59c700 1 -- 172.21.15.92:0/3702694490 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100be170 con 0x7fd410153f90 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40e59c700 1 --2- 172.21.15.92:0/3702694490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd410153f90 0x7fd410067f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fd4100c2e40 tx=0x7fd3f80090f0).ready entity=mon.0 client_cookie=a25745c915f2bad0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40dd9b700 1 -- 172.21.15.92:0/3702694490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800b600 con 0x7fd410153f90 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40dd9b700 1 -- 172.21.15.92:0/3702694490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f800b760 con 0x7fd410153f90 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd40dd9b700 1 -- 172.21.15.92:0/3702694490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800bae0 con 0x7fd410153f90 2023-05-16T07:02:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd414842700 1 -- 172.21.15.92:0/3702694490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd410153f90 msgr2=0x7fd410067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.231+0000 7fd414842700 1 --2- 172.21.15.92:0/3702694490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd410153f90 0x7fd410067f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fd4100c2e40 tx=0x7fd3f80090f0).stop 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 -- 172.21.15.92:0/3702694490 shutdown_connections 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 --2- 172.21.15.92:0/3702694490 >> v2:172.21.15.158:3300/0 conn(0x7fd4100bd810 0x7fd4100bdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 --2- 172.21.15.92:0/3702694490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153bb0 0x7fd4100698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 --2- 172.21.15.92:0/3702694490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd410153f90 0x7fd410067f60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 -- 172.21.15.92:0/3702694490 >> 172.21.15.92:0/3702694490 conn(0x7fd410062d80 msgr2=0x7fd41005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 -- 172.21.15.92:0/3702694490 shutdown_connections 2023-05-16T07:02:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.232+0000 7fd414842700 1 -- 172.21.15.92:0/3702694490 wait complete. 2023-05-16T07:02:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.233+0000 7fd414842700 1 Processor -- start 2023-05-16T07:02:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.233+0000 7fd414842700 1 -- start start 2023-05-16T07:02:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.233+0000 7fd414842700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 0x7fd410154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.233+0000 7fd414842700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd410153bb0 0x7fd4101475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd414842700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153f90 0x7fd410156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd414842700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100bfd30 con 0x7fd4100bd810 2023-05-16T07:02:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd414842700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410144cd0 con 0x7fd410153bb0 2023-05-16T07:02:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd414842700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410155620 con 0x7fd410153f90 2023-05-16T07:02:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 0x7fd410154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 0x7fd410154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41344/0 (socket says 172.21.15.92:41344) 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 -- 172.21.15.92:0/2231150405 learned_addr learned my addr 172.21.15.92:0/2231150405 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40e59c700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd410153bb0 0x7fd4101475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40f59e700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153f90 0x7fd410156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 -- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153f90 msgr2=0x7fd410156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153f90 0x7fd410156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 -- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd410153bb0 msgr2=0x7fd4101475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd410153bb0 0x7fd4101475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 -- 172.21.15.92:0/2231150405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8009370 con 0x7fd4100bd810 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40e59c700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd410153bb0 0x7fd4101475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd40ed9d700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 0x7fd410154dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd400007f60 tx=0x7fd400009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000ecf0 con 0x7fd4100bd810 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40000ee50 con 0x7fd4100bd810 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd410155900 con 0x7fd4100bd810 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400014610 con 0x7fd4100bd810 2023-05-16T07:02:50.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.234+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4100bf950 con 0x7fd4100bd810 2023-05-16T07:02:50.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.235+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd410047150 con 0x7fd4100bd810 2023-05-16T07:02:50.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.236+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd40001b030 con 0x7fd4100bd810 2023-05-16T07:02:50.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.236+0000 7fd406ffd700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd3f005ed60 0x7fd3f0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.236+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd40007a050 con 0x7fd4100bd810 2023-05-16T07:02:50.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.236+0000 7fd40e59c700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd3f005ed60 0x7fd3f0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.236+0000 7fd40e59c700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd3f005ed60 0x7fd3f0061220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd4100c2e40 tx=0x7fd3f8017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.240+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd40004b5f0 con 0x7fd4100bd810 2023-05-16T07:02:50.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.404+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4100bf1f0 con 0x7fd4100bd810 2023-05-16T07:02:50.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.405+0000 7fd406ffd700 1 -- 172.21.15.92:0/2231150405 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd400019020 con 0x7fd4100bd810 2023-05-16T07:02:50.409 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:50.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd3f005ed60 msgr2=0x7fd3f0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd3f005ed60 0x7fd3f0061220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd4100c2e40 tx=0x7fd3f8017000).stop 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 msgr2=0x7fd410154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 0x7fd410154dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd400007f60 tx=0x7fd400009860).stop 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 shutdown_connections 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd3f005ed60 0x7fd3f0061220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd410153f90 0x7fd410156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4100bd810 0x7fd410154dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 --2- 172.21.15.92:0/2231150405 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd410153bb0 0x7fd4101475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 >> 172.21.15.92:0/2231150405 conn(0x7fd410062d80 msgr2=0x7fd410157b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:50.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 shutdown_connections 2023-05-16T07:02:50.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.406+0000 7fd414842700 1 -- 172.21.15.92:0/2231150405 wait complete. 2023-05-16T07:02:50.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:50.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.533+0000 7fd13f9e8700 1 Processor -- start 2023-05-16T07:02:50.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.533+0000 7fd13f9e8700 1 -- start start 2023-05-16T07:02:50.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13f9e8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd138154090 0x7fd1380aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13f9e8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd138154470 0x7fd1380a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13f9e8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1380c94d0 con 0x7fd1380aa150 2023-05-16T07:02:50.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13f9e8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1380aa5a0 con 0x7fd138154090 2023-05-16T07:02:50.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13f9e8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1380cb7e0 con 0x7fd138154470 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd138154090 0x7fd1380aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13df85700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd1380aa150 0x7fd138068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52102/0 (socket says 172.21.15.92:52102) 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13df85700 1 -- 172.21.15.92:0/698433373 learned_addr learned my addr 172.21.15.92:0/698433373 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd138154090 0x7fd1380aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41352/0 (socket says 172.21.15.92:41352) 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 -- 172.21.15.92:0/698433373 >> v2:172.21.15.158:3300/0 conn(0x7fd138154470 msgr2=0x7fd1380a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 --2- 172.21.15.92:0/698433373 >> v2:172.21.15.158:3300/0 conn(0x7fd138154470 0x7fd1380a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:50.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 -- 172.21.15.92:0/698433373 >> v1:172.21.15.92:6789/0 conn(0x7fd1380aa150 legacy=0x7fd138068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd12ffff700 1 -- 172.21.15.92:0/698433373 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949965328 0 0) 0x7fd1380c94d0 con 0x7fd1380aa150 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 -- 172.21.15.92:0/698433373 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1380c88c0 con 0x7fd138154090 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd13d784700 1 --2- 172.21.15.92:0/698433373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd138154090 0x7fd1380aaca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd138084320 tx=0x7fd130009780).ready entity=mon.0 client_cookie=6d86fdc13ce0d561 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd12ffff700 1 -- 172.21.15.92:0/698433373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13000e070 con 0x7fd138154090 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd12ffff700 1 -- 172.21.15.92:0/698433373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13000bc30 con 0x7fd138154090 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.534+0000 7fd12ffff700 1 -- 172.21.15.92:0/698433373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130012410 con 0x7fd138154090 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 -- 172.21.15.92:0/698433373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd138154090 msgr2=0x7fd1380aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/698433373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd138154090 0x7fd1380aaca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd138084320 tx=0x7fd130009780).stop 2023-05-16T07:02:50.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 -- 172.21.15.92:0/698433373 shutdown_connections 2023-05-16T07:02:50.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/698433373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd138154090 0x7fd1380aaca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/698433373 >> v2:172.21.15.158:3300/0 conn(0x7fd138154470 0x7fd1380a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:50.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 -- 172.21.15.92:0/698433373 >> 172.21.15.92:0/698433373 conn(0x7fd138067920 msgr2=0x7fd1380cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:50.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 -- 172.21.15.92:0/698433373 shutdown_connections 2023-05-16T07:02:50.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.535+0000 7fd13f9e8700 1 -- 172.21.15.92:0/698433373 wait complete. 2023-05-16T07:02:50.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.536+0000 7fd13f9e8700 1 Processor -- start 2023-05-16T07:02:50.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.536+0000 7fd13f9e8700 1 -- start start 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13f9e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 0x7fd1380b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13f9e8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd138154090 0x7fd1380adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13f9e8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd138154470 0x7fd13814e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13f9e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1380b07a0 con 0x7fd1380aa150 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13f9e8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1380b0a80 con 0x7fd138154470 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13f9e8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd138151fe0 con 0x7fd138154090 2023-05-16T07:02:50.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 0x7fd1380b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 0x7fd1380b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41358/0 (socket says 172.21.15.92:41358) 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 -- 172.21.15.92:0/700548212 learned_addr learned my addr 172.21.15.92:0/700548212 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13cf83700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd138154090 0x7fd1380adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13df85700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd138154470 0x7fd13814e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 -- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd138154090 msgr2=0x7fd1380adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd138154090 0x7fd1380adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 -- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd138154470 msgr2=0x7fd13814e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd138154470 0x7fd13814e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 -- 172.21.15.92:0/700548212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130009a20 con 0x7fd1380aa150 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.537+0000 7fd13d784700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 0x7fd1380b01c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd13806bb30 tx=0x7fd130009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.538+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130001090 con 0x7fd1380aa150 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.538+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13001d630 con 0x7fd1380aa150 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.538+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1381522c0 con 0x7fd1380aa150 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.538+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13001d9f0 con 0x7fd1380aa150 2023-05-16T07:02:50.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.538+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd138151cd0 con 0x7fd1380aa150 2023-05-16T07:02:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.538+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd138152480 con 0x7fd1380aa150 2023-05-16T07:02:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.539+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd13001dd60 con 0x7fd1380aa150 2023-05-16T07:02:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.539+0000 7fd12d7fa700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd11c05ed90 0x7fd11c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.539+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd13007baf0 con 0x7fd1380aa150 2023-05-16T07:02:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.539+0000 7fd13cf83700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd11c05ed90 0x7fd11c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.540+0000 7fd13cf83700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd11c05ed90 0x7fd11c061250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd12800b870 tx=0x7fd128008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.543+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd13004d090 con 0x7fd1380aa150 2023-05-16T07:02:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.705+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd138066730 con 0x7fd1380aa150 2023-05-16T07:02:50.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.705+0000 7fd12d7fa700 1 -- 172.21.15.92:0/700548212 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1300506b0 con 0x7fd1380aa150 2023-05-16T07:02:50.709 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd11c05ed90 msgr2=0x7fd11c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd11c05ed90 0x7fd11c061250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd12800b870 tx=0x7fd128008040).stop 2023-05-16T07:02:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 msgr2=0x7fd1380b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 0x7fd1380b01c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd13806bb30 tx=0x7fd130009780).stop 2023-05-16T07:02:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 shutdown_connections 2023-05-16T07:02:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd11c05ed90 0x7fd11c061250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd138154470 0x7fd13814e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1380aa150 0x7fd1380b01c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 --2- 172.21.15.92:0/700548212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd138154090 0x7fd1380adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 >> 172.21.15.92:0/700548212 conn(0x7fd138067920 msgr2=0x7fd1380c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 shutdown_connections 2023-05-16T07:02:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.707+0000 7fd13f9e8700 1 -- 172.21.15.92:0/700548212 wait complete. 2023-05-16T07:02:50.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.828+0000 7f123359e700 1 Processor -- start 2023-05-16T07:02:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.828+0000 7f123359e700 1 -- start start 2023-05-16T07:02:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f123359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f12340acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f123359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1234158970 con 0x7f12340c7380 2023-05-16T07:02:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f123359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1234157d60 con 0x7f12340ad040 2023-05-16T07:02:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f123359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1234158040 con 0x7f1234154e50 2023-05-16T07:02:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f123259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f12340c7380 0x7f12340c7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52114/0 (socket says 172.21.15.92:52114) 2023-05-16T07:02:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f123259c700 1 -- 172.21.15.92:0/578524787 learned_addr learned my addr 172.21.15.92:0/578524787 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f122bfff700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f12340acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779994082 0 0) 0x7f1234158970 con 0x7f12340c7380 2023-05-16T07:02:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12200033f0 con 0x7f12340c7380 2023-05-16T07:02:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2629792065 0 0) 0x7f12200033f0 con 0x7f12340c7380 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 msgr2=0x7f12340acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f12340acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 >> v1:172.21.15.158:6789/0 conn(0x7f12340ad040 legacy=0x7f123414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1234158320 con 0x7f12340c7380 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f1224001470 con 0x7f12340c7380 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 >> v1:172.21.15.92:6789/0 conn(0x7f12340c7380 legacy=0x7f12340c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1220003910 0x7f1220003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220004690 0x7f1220004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f1220005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234158040 con 0x7f1220003910 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122000b910 con 0x7f1220004690 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.829+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122000bad0 con 0x7f1234154e50 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1220003910 0x7f1220003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f122bfff700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220004690 0x7f1220004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f1232d9d700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f1220005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 -- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 msgr2=0x7f1220005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f1220005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 -- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220004690 msgr2=0x7f1220004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220004690 0x7f1220004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 -- 172.21.15.92:0/578524787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200033f0 con 0x7f1220003910 2023-05-16T07:02:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f1232d9d700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234154e50 0x7f1220005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f123259c700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1220003910 0x7f1220003d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f122400c160 tx=0x7f12341588e0).ready entity=mon.0 client_cookie=a10898f372c3419c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400ca90 con 0x7f1220003910 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400cbf0 con 0x7f1220003910 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.830+0000 7f1231d9b700 1 -- 172.21.15.92:0/578524787 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240070d0 con 0x7f1220003910 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f123359e700 1 -- 172.21.15.92:0/578524787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1220003910 msgr2=0x7f1220003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f123359e700 1 --2- 172.21.15.92:0/578524787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1220003910 0x7f1220003d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f122400c160 tx=0x7f12341588e0).stop 2023-05-16T07:02:50.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f1232d9d700 1 -- 172.21.15.92:0/578524787 reap_dead start 2023-05-16T07:02:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f123359e700 1 -- 172.21.15.92:0/578524787 shutdown_connections 2023-05-16T07:02:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f123359e700 1 -- 172.21.15.92:0/578524787 >> 172.21.15.92:0/578524787 conn(0x7f1234068330 msgr2=0x7f1234068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f123359e700 1 -- 172.21.15.92:0/578524787 shutdown_connections 2023-05-16T07:02:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.831+0000 7f123359e700 1 -- 172.21.15.92:0/578524787 wait complete. 2023-05-16T07:02:50.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 Processor -- start 2023-05-16T07:02:50.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 -- start start 2023-05-16T07:02:50.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220003910 0x7f12340c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 0x7f1234164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234164a70 0x7f1234164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234145b60 con 0x7f12340c26c0 2023-05-16T07:02:50.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12341653e0 con 0x7f1220003910 2023-05-16T07:02:50.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.832+0000 7f123359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12340c2e90 con 0x7f1234164a70 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 0x7f1234164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 0x7f1234164580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41380/0 (socket says 172.21.15.92:41380) 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 -- 172.21.15.92:0/2793707360 learned_addr learned my addr 172.21.15.92:0/2793707360 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f123259c700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220003910 0x7f12340c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 -- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234164a70 msgr2=0x7f1234164ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f1232d9d700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234164a70 0x7f1234164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234164a70 0x7f1234164ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 -- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220003910 msgr2=0x7f12340c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220003910 0x7f12340c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 -- 172.21.15.92:0/2793707360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340c3170 con 0x7f12340c26c0 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122bfff700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 0x7f1234164580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f121c0119f0 tx=0x7f121c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00f940 con 0x7f12340c26c0 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c00faa0 con 0x7f12340c26c0 2023-05-16T07:02:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00fe20 con 0x7f12340c26c0 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12340c3450 con 0x7f12340c26c0 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.833+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12340c2b40 con 0x7f12340c26c0 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.834+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1234047150 con 0x7f12340c26c0 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.834+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f121c024070 con 0x7f12340c26c0 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.835+0000 7f122a7fc700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121405ed60 0x7f1214061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.835+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f121c07a5a0 con 0x7f12340c26c0 2023-05-16T07:02:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.835+0000 7f123259c700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121405ed60 0x7f1214061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:50.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.835+0000 7f123259c700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121405ed60 0x7f1214061220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f122400c160 tx=0x7f1224015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:50.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:50.839+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f121c04bb40 con 0x7f12340c26c0 2023-05-16T07:02:51.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.001+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12340ad650 con 0x7f12340c26c0 2023-05-16T07:02:51.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.002+0000 7f122a7fc700 1 -- 172.21.15.92:0/2793707360 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f121c04f160 con 0x7f12340c26c0 2023-05-16T07:02:51.007 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:51.007 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:51.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121405ed60 msgr2=0x7f1214061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121405ed60 0x7f1214061220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f122400c160 tx=0x7f1224015000).stop 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 msgr2=0x7f1234164580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 0x7f1234164580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f121c0119f0 tx=0x7f121c009060).stop 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 shutdown_connections 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f121405ed60 0x7f1214061220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1234164a70 0x7f1234164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1220003910 0x7f12340c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 --2- 172.21.15.92:0/2793707360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12340c26c0 0x7f1234164580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 >> 172.21.15.92:0/2793707360 conn(0x7f1234068330 msgr2=0x7f12341538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 shutdown_connections 2023-05-16T07:02:51.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.003+0000 7f123359e700 1 -- 172.21.15.92:0/2793707360 wait complete. 2023-05-16T07:02:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:51.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.129+0000 7fae28ddf700 1 Processor -- start 2023-05-16T07:02:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.129+0000 7fae28ddf700 1 -- start start 2023-05-16T07:02:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae28ddf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae2414f990 0x7fae240c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae28ddf700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae24062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae28ddf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae24063690 con 0x7fae24152e40 2023-05-16T07:02:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae28ddf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae24062c30 con 0x7fae2414f990 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae28ddf700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2414fd70 con 0x7fae240622b0 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae2259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae2414f990 0x7fae240c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae21d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fae24152e40 0x7fae240c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52138/0 (socket says 172.21.15.92:52138) 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae21d9b700 1 -- 172.21.15.92:0/716202642 learned_addr learned my addr 172.21.15.92:0/716202642 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae22d9d700 1 --2- 172.21.15.92:0/716202642 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae24062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae2259c700 1 -- 172.21.15.92:0/716202642 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 msgr2=0x7fae24062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae2259c700 1 --2- 172.21.15.92:0/716202642 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae24062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae2259c700 1 -- 172.21.15.92:0/716202642 >> v1:172.21.15.92:6789/0 conn(0x7fae24152e40 legacy=0x7fae240c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae2259c700 1 -- 172.21.15.92:0/716202642 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240c52f0 con 0x7fae2414f990 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.130+0000 7fae2159a700 1 -- 172.21.15.92:0/716202642 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321533168 0 0) 0x7fae24063690 con 0x7fae24152e40 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae2259c700 1 --2- 172.21.15.92:0/716202642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae2414f990 0x7fae240c4c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fae14009e60 tx=0x7fae14009780).ready entity=mon.0 client_cookie=aa977938d2d3b001 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae2159a700 1 -- 172.21.15.92:0/716202642 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400e070 con 0x7fae2414f990 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae2159a700 1 -- 172.21.15.92:0/716202642 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1400bc30 con 0x7fae2414f990 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae2159a700 1 -- 172.21.15.92:0/716202642 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14012410 con 0x7fae2414f990 2023-05-16T07:02:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 -- 172.21.15.92:0/716202642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae2414f990 msgr2=0x7fae240c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 --2- 172.21.15.92:0/716202642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae2414f990 0x7fae240c4c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fae14009e60 tx=0x7fae14009780).stop 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 -- 172.21.15.92:0/716202642 shutdown_connections 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 --2- 172.21.15.92:0/716202642 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae24062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 --2- 172.21.15.92:0/716202642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae2414f990 0x7fae240c4c80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 -- 172.21.15.92:0/716202642 >> 172.21.15.92:0/716202642 conn(0x7fae2405e820 msgr2=0x7fae240c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 -- 172.21.15.92:0/716202642 shutdown_connections 2023-05-16T07:02:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.131+0000 7fae28ddf700 1 -- 172.21.15.92:0/716202642 wait complete. 2023-05-16T07:02:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.132+0000 7fae28ddf700 1 Processor -- start 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.132+0000 7fae28ddf700 1 -- start start 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae28ddf700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae2414e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae28ddf700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae2414f990 0x7fae24164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae28ddf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 0x7fae241653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae28ddf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae240c2d80 con 0x7fae24152e40 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae28ddf700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae240c2170 con 0x7fae2414f990 2023-05-16T07:02:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae28ddf700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae240c2450 con 0x7fae240622b0 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 0x7fae241653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 0x7fae241653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41396/0 (socket says 172.21.15.92:41396) 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 -- 172.21.15.92:0/617009486 learned_addr learned my addr 172.21.15.92:0/617009486 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae21d9b700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae2414f990 0x7fae24164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae2259c700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae2414e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 -- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 msgr2=0x7fae2414e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae2414e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 -- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae2414f990 msgr2=0x7fae24164ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae2414f990 0x7fae24164ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae22d9d700 1 -- 172.21.15.92:0/617009486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14009a20 con 0x7fae24152e40 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae2259c700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae2414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.133+0000 7fae21d9b700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae2414f990 0x7fae24164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.134+0000 7fae22d9d700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 0x7fae241653e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fae1c00d890 tx=0x7fae1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.134+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0125f0 con 0x7fae24152e40 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.134+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c012750 con 0x7fae24152e40 2023-05-16T07:02:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.134+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae240c2730 con 0x7fae24152e40 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.134+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c012ad0 con 0x7fae24152e40 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.134+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae240c1d50 con 0x7fae24152e40 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.135+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae24154300 con 0x7fae24152e40 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.135+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fae1c013480 con 0x7fae24152e40 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.135+0000 7fae1a7fc700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae0405ed90 0x7fae04061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.135+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fae1c0799f0 con 0x7fae24152e40 2023-05-16T07:02:51.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.136+0000 7fae2259c700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae0405ed90 0x7fae04061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.136+0000 7fae2259c700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae0405ed90 0x7fae04061250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fae1400b9f0 tx=0x7fae14009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.139+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fae24154300 con 0x7fae24152e40 2023-05-16T07:02:51.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.303+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fae24154300 con 0x7fae24152e40 2023-05-16T07:02:51.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.303+0000 7fae1a7fc700 1 -- 172.21.15.92:0/617009486 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fae24154300 con 0x7fae24152e40 2023-05-16T07:02:51.309 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae0405ed90 msgr2=0x7fae04061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae0405ed90 0x7fae04061250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fae1400b9f0 tx=0x7fae14009780).stop 2023-05-16T07:02:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 msgr2=0x7fae241653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 0x7fae241653e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fae1c00d890 tx=0x7fae1c008040).stop 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 shutdown_connections 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fae0405ed90 0x7fae04061250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae240622b0 0x7fae2414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24152e40 0x7fae241653e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 --2- 172.21.15.92:0/617009486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae2414f990 0x7fae24164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 >> 172.21.15.92:0/617009486 conn(0x7fae2405e820 msgr2=0x7fae240c3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 shutdown_connections 2023-05-16T07:02:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.305+0000 7fae28ddf700 1 -- 172.21.15.92:0/617009486 wait complete. 2023-05-16T07:02:51.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:51.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.430+0000 7fb5e9a72700 1 Processor -- start 2023-05-16T07:02:51.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.430+0000 7fb5e9a72700 1 -- start start 2023-05-16T07:02:51.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.430+0000 7fb5e9a72700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e414e620 0x7fb5e40c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.430+0000 7fb5e9a72700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e4065bf0 con 0x7fb5e414ea00 2023-05-16T07:02:51.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.430+0000 7fb5e9a72700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e4065ed0 con 0x7fb5e40c4610 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.430+0000 7fb5e9a72700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40661b0 con 0x7fb5e414e620 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5e414ea00 0x7fb5e4154760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52150/0 (socket says 172.21.15.92:52150) 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e3fff700 1 -- 172.21.15.92:0/385949916 learned_addr learned my addr 172.21.15.92:0/385949916 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e8a70700 1 --2- 172.21.15.92:0/385949916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e414e620 0x7fb5e40c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e37fe700 1 -- 172.21.15.92:0/385949916 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675037144 0 0) 0x7fb5e4065bf0 con 0x7fb5e414ea00 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e37fe700 1 -- 172.21.15.92:0/385949916 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d00033f0 con 0x7fb5e414ea00 2023-05-16T07:02:51.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e8a70700 1 -- 172.21.15.92:0/385949916 >> v1:172.21.15.158:6789/0 conn(0x7fb5e40c4610 legacy=0x7fb5e40c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e37fe700 1 -- 172.21.15.92:0/385949916 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836015026 0 0) 0x7fb5d00033f0 con 0x7fb5e414ea00 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e8a70700 1 -- 172.21.15.92:0/385949916 >> v1:172.21.15.92:6789/0 conn(0x7fb5e414ea00 legacy=0x7fb5e4154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e8a70700 1 -- 172.21.15.92:0/385949916 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e4066490 con 0x7fb5e414e620 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e8a70700 1 --2- 172.21.15.92:0/385949916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e414e620 0x7fb5e40c4120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb5d40097c0 tx=0x7fb5d4009060).ready entity=mon.2 client_cookie=4fb4dc1016e524fb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e37fe700 1 -- 172.21.15.92:0/385949916 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc001470 con 0x7fb5e414e620 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.431+0000 7fb5e37fe700 1 -- 172.21.15.92:0/385949916 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d4012410 con 0x7fb5e414e620 2023-05-16T07:02:51.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e37fe700 1 -- 172.21.15.92:0/385949916 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4012790 con 0x7fb5e414e620 2023-05-16T07:02:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 -- 172.21.15.92:0/385949916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e414e620 msgr2=0x7fb5e40c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/385949916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e414e620 0x7fb5e40c4120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb5d40097c0 tx=0x7fb5d4009060).stop 2023-05-16T07:02:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 -- 172.21.15.92:0/385949916 shutdown_connections 2023-05-16T07:02:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/385949916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e414e620 0x7fb5e40c4120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 -- 172.21.15.92:0/385949916 >> 172.21.15.92:0/385949916 conn(0x7fb5e40c7a80 msgr2=0x7fb5e40c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 -- 172.21.15.92:0/385949916 shutdown_connections 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.432+0000 7fb5e9a72700 1 -- 172.21.15.92:0/385949916 wait complete. 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.433+0000 7fb5e9a72700 1 Processor -- start 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.433+0000 7fb5e9a72700 1 -- start start 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9a72700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e40c4610 0x7fb5e40c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9a72700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5e414e620 0x7fb5e4163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9a72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 0x7fb5e40bec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9a72700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40c0eb0 con 0x7fb5e414ea00 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9a72700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e4163080 con 0x7fb5e414e620 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9a72700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40be790 con 0x7fb5e40c4610 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 0x7fb5e40bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 0x7fb5e40bec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41410/0 (socket says 172.21.15.92:41410) 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 -- 172.21.15.92:0/2355132601 learned_addr learned my addr 172.21.15.92:0/2355132601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e8a70700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e40c4610 0x7fb5e40c09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e3fff700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5e414e620 0x7fb5e4163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 -- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e40c4610 msgr2=0x7fb5e40c09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e40c4610 0x7fb5e40c09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 -- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5e414e620 msgr2=0x7fb5e4163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5e414e620 0x7fb5e4163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e9271700 1 -- 172.21.15.92:0/2355132601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40092a0 con 0x7fb5e414ea00 2023-05-16T07:02:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.434+0000 7fb5e3fff700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5e414e620 0x7fb5e4163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.435+0000 7fb5e9271700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 0x7fb5e40bec20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fb5dc001560 tx=0x7fb5dc001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.435+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc010510 con 0x7fb5e414ea00 2023-05-16T07:02:51.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.435+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc010670 con 0x7fb5e414ea00 2023-05-16T07:02:51.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.435+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc010a10 con 0x7fb5e414ea00 2023-05-16T07:02:51.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.435+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e4162da0 con 0x7fb5e414ea00 2023-05-16T07:02:51.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.435+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e40be420 con 0x7fb5e414ea00 2023-05-16T07:02:51.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.436+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e40bea70 con 0x7fb5e414ea00 2023-05-16T07:02:51.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.436+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb5dc003020 con 0x7fb5e414ea00 2023-05-16T07:02:51.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.436+0000 7fb5e0ff9700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5c405ed90 0x7fb5c4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.436+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb5dc0798e0 con 0x7fb5e414ea00 2023-05-16T07:02:51.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.436+0000 7fb5e8a70700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5c405ed90 0x7fb5c4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.437+0000 7fb5e8a70700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5c405ed90 0x7fb5c4061250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb5e4146830 tx=0x7fb5d4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.440+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb5dc04ae80 con 0x7fb5e414ea00 2023-05-16T07:02:51.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.605+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb5e405e180 con 0x7fb5e414ea00 2023-05-16T07:02:51.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.605+0000 7fb5e0ff9700 1 -- 172.21.15.92:0/2355132601 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb5dc04e4a0 con 0x7fb5e414ea00 2023-05-16T07:02:51.610 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:51.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5c405ed90 msgr2=0x7fb5c4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5c405ed90 0x7fb5c4061250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb5e4146830 tx=0x7fb5d4015000).stop 2023-05-16T07:02:51.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 msgr2=0x7fb5e40bec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 0x7fb5e40bec20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fb5dc001560 tx=0x7fb5dc001630).stop 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 shutdown_connections 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5c405ed90 0x7fb5c4061250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5e40c4610 0x7fb5e40c09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e414ea00 0x7fb5e40bec20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 --2- 172.21.15.92:0/2355132601 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5e414e620 0x7fb5e4163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 >> 172.21.15.92:0/2355132601 conn(0x7fb5e40c7a80 msgr2=0x7fb5e40bbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 shutdown_connections 2023-05-16T07:02:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.607+0000 7fb5e9a72700 1 -- 172.21.15.92:0/2355132601 wait complete. 2023-05-16T07:02:51.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:51.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.732+0000 7fc8d4b4f700 1 Processor -- start 2023-05-16T07:02:51.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.733+0000 7fc8d4b4f700 1 -- start start 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.733+0000 7fc8d4b4f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d0153f90 0x7fc8d00a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.733+0000 7fc8d4b4f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d006a140 con 0x7fc8d0155380 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.733+0000 7fc8d4b4f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d006a420 con 0x7fc8d00c5d00 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.733+0000 7fc8d4b4f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d0069f60 con 0x7fc8d0153f90 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8cffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc8d0155380 0x7fc8d00c0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52164/0 (socket says 172.21.15.92:52164) 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8cffff700 1 -- 172.21.15.92:0/1978042695 learned_addr learned my addr 172.21.15.92:0/1978042695 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d0153f90 0x7fc8d00a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579750520 0 0) 0x7fc8d006a140 con 0x7fc8d0155380 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8b80033f0 con 0x7fc8d0155380 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764716717 0 0) 0x7fc8d006a420 con 0x7fc8d00c5d00 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8d006a140 con 0x7fc8d00c5d00 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380762807 0 0) 0x7fc8b80033f0 con 0x7fc8d0155380 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d0153f90 msgr2=0x7fc8d00a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:51.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d0153f90 0x7fc8d00a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 >> v1:172.21.15.158:6789/0 conn(0x7fc8d00c5d00 legacy=0x7fc8d00c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d006a700 con 0x7fc8d0155380 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc8c8001470 con 0x7fc8d0155380 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 >> v1:172.21.15.92:6789/0 conn(0x7fc8d0155380 legacy=0x7fc8d00c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8003aa0 0x7fc8b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8b8004690 0x7fc8b8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8005000 0x7fc8b800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d0069f60 con 0x7fc8b8003aa0 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b800bdd0 con 0x7fc8b8005000 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b800bf90 con 0x7fc8b8004690 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d0153f90 0x7fc8d00a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8003aa0 0x7fc8b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8cffff700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8005000 0x7fc8b800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.734+0000 7fc8ceffd700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8b8004690 0x7fc8b8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cf7fe700 1 -- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8b8004690 msgr2=0x7fc8b8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8b8004690 0x7fc8b8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cf7fe700 1 -- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8005000 msgr2=0x7fc8b800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8005000 0x7fc8b800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cf7fe700 1 -- 172.21.15.92:0/1978042695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80033f0 con 0x7fc8b8003aa0 2023-05-16T07:02:51.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8003aa0 0x7fc8b8003ea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc8d0083ed0 tx=0x7fc8c000b040).ready entity=mon.0 client_cookie=c2106995fec99ee9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000eaa0 con 0x7fc8b8003aa0 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c000f040 con 0x7fc8b8003aa0 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8ce7fc700 1 -- 172.21.15.92:0/1978042695 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000ee30 con 0x7fc8b8003aa0 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1978042695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8003aa0 msgr2=0x7fc8b8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1978042695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8003aa0 0x7fc8b8003ea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc8d0083ed0 tx=0x7fc8c000b040).stop 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8cffff700 1 -- 172.21.15.92:0/1978042695 reap_dead start 2023-05-16T07:02:51.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1978042695 shutdown_connections 2023-05-16T07:02:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1978042695 >> 172.21.15.92:0/1978042695 conn(0x7fc8d00cb600 msgr2=0x7fc8d00c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1978042695 shutdown_connections 2023-05-16T07:02:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.735+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1978042695 wait complete. 2023-05-16T07:02:51.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 Processor -- start 2023-05-16T07:02:51.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 -- start start 2023-05-16T07:02:51.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8003aa0 0x7fc8d014e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00adae0 0x7fc8d00b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 0x7fc8d00ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00c41a0 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00aed40 con 0x7fc8b8003aa0 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.737+0000 7fc8d4b4f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d014e5f0 con 0x7fc8d00adae0 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 0x7fc8d00ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 0x7fc8d00ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41430/0 (socket says 172.21.15.92:41430) 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 -- 172.21.15.92:0/1153041969 learned_addr learned my addr 172.21.15.92:0/1153041969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8ceffd700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00adae0 0x7fc8d00b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8003aa0 0x7fc8d014e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 -- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00adae0 msgr2=0x7fc8d00b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00adae0 0x7fc8d00b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 -- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8003aa0 msgr2=0x7fc8d014e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8003aa0 0x7fc8d014e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 -- 172.21.15.92:0/1153041969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00adfe0 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8cffff700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 0x7fc8d00ae850 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fc8d00c1b50 tx=0x7fc8c8001e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011590 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8011710 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011ad0 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d00c3870 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.738+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d00c3e30 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.739+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d0151e00 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.739+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc8c801b030 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.740+0000 7fc8b77fe700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8bc05ed90 0x7fc8bc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.740+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc8c8079210 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.740+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8bc05ed90 0x7fc8bc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.740+0000 7fc8cf7fe700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8bc05ed90 0x7fc8bc061250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc8d00c1da0 tx=0x7fc8c0017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.744+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc8d0151e00 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.907+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8d0151e00 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.907+0000 7fc8b77fe700 1 -- 172.21.15.92:0/1153041969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8d0151e00 con 0x7fc8d00ae3d0 2023-05-16T07:02:51.912 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:51.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8bc05ed90 msgr2=0x7fc8bc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8bc05ed90 0x7fc8bc061250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc8d00c1da0 tx=0x7fc8c0017000).stop 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 msgr2=0x7fc8d00ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 0x7fc8d00ae850 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fc8d00c1b50 tx=0x7fc8c8001e10).stop 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 shutdown_connections 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8bc05ed90 0x7fc8bc061250 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d00ae3d0 0x7fc8d00ae850 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b8003aa0 0x7fc8d014e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 --2- 172.21.15.92:0/1153041969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00adae0 0x7fc8d00b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:51.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 >> 172.21.15.92:0/1153041969 conn(0x7fc8d00cb600 msgr2=0x7fc8d00a9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:51.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 shutdown_connections 2023-05-16T07:02:51.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:51.909+0000 7fc8d4b4f700 1 -- 172.21.15.92:0/1153041969 wait complete. 2023-05-16T07:02:51.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:52.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.034+0000 7fb3940e0700 1 Processor -- start 2023-05-16T07:02:52.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.034+0000 7fb3940e0700 1 -- start start 2023-05-16T07:02:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb3940e0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb38c154090 0x7fb38c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb3940e0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb38c068160 0x7fb38c0685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb3940e0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0c94d0 con 0x7fb38c154470 2023-05-16T07:02:52.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb3940e0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0cb7e0 con 0x7fb38c068160 2023-05-16T07:02:52.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb3940e0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c068aa0 con 0x7fb38c154090 2023-05-16T07:02:52.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb39267d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb38c068160 0x7fb38c0685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb39167b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb38c154470 0x7fb38c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52184/0 (socket says 172.21.15.92:52184) 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.035+0000 7fb39167b700 1 -- 172.21.15.92:0/3713551479 learned_addr learned my addr 172.21.15.92:0/3713551479 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb391e7c700 1 --2- 172.21.15.92:0/3713551479 >> v2:172.21.15.158:3300/0 conn(0x7fb38c154090 0x7fb38c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb39267d700 1 -- 172.21.15.92:0/3713551479 >> v2:172.21.15.158:3300/0 conn(0x7fb38c154090 msgr2=0x7fb38c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb39267d700 1 --2- 172.21.15.92:0/3713551479 >> v2:172.21.15.158:3300/0 conn(0x7fb38c154090 0x7fb38c0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb39267d700 1 -- 172.21.15.92:0/3713551479 >> v1:172.21.15.92:6789/0 conn(0x7fb38c154470 legacy=0x7fb38c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb390e7a700 1 -- 172.21.15.92:0/3713551479 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492664728 0 0) 0x7fb38c0c94d0 con 0x7fb38c154470 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb39267d700 1 -- 172.21.15.92:0/3713551479 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0c88c0 con 0x7fb38c068160 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb391e7c700 1 --2- 172.21.15.92:0/3713551479 >> v2:172.21.15.158:3300/0 conn(0x7fb38c154090 0x7fb38c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb39267d700 1 --2- 172.21.15.92:0/3713551479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c068160 0x7fb38c0685b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb38c06bd10 tx=0x7fb384009060).ready entity=mon.0 client_cookie=6cf69046ab1b7fe4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb390e7a700 1 -- 172.21.15.92:0/3713551479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400b580 con 0x7fb38c068160 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb390e7a700 1 -- 172.21.15.92:0/3713551479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38400b6e0 con 0x7fb38c068160 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb390e7a700 1 -- 172.21.15.92:0/3713551479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400ba60 con 0x7fb38c068160 2023-05-16T07:02:52.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb3940e0700 1 -- 172.21.15.92:0/3713551479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c068160 msgr2=0x7fb38c0685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb3940e0700 1 --2- 172.21.15.92:0/3713551479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c068160 0x7fb38c0685b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb38c06bd10 tx=0x7fb384009060).stop 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb3940e0700 1 -- 172.21.15.92:0/3713551479 shutdown_connections 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb3940e0700 1 --2- 172.21.15.92:0/3713551479 >> v2:172.21.15.158:3300/0 conn(0x7fb38c154090 0x7fb38c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb3940e0700 1 --2- 172.21.15.92:0/3713551479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c068160 0x7fb38c0685b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.036+0000 7fb3940e0700 1 -- 172.21.15.92:0/3713551479 >> 172.21.15.92:0/3713551479 conn(0x7fb38c067920 msgr2=0x7fb38c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.037+0000 7fb3940e0700 1 -- 172.21.15.92:0/3713551479 shutdown_connections 2023-05-16T07:02:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.037+0000 7fb3940e0700 1 -- 172.21.15.92:0/3713551479 wait complete. 2023-05-16T07:02:52.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 Processor -- start 2023-05-16T07:02:52.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 -- start start 2023-05-16T07:02:52.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb38c068160 0x7fb38c14f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb38c154090 0x7fb38c14e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 0x7fb38c167560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c14fe70 con 0x7fb38c154470 2023-05-16T07:02:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c14f1f0 con 0x7fb38c154090 2023-05-16T07:02:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb3940e0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c159570 con 0x7fb38c068160 2023-05-16T07:02:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb39267d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 0x7fb38c167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb39267d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 0x7fb38c167560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41442/0 (socket says 172.21.15.92:41442) 2023-05-16T07:02:52.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.038+0000 7fb39267d700 1 -- 172.21.15.92:0/1048830825 learned_addr learned my addr 172.21.15.92:0/1048830825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39167b700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb38c154090 0x7fb38c14e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb391e7c700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb38c068160 0x7fb38c14f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39267d700 1 -- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb38c068160 msgr2=0x7fb38c14f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39267d700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb38c068160 0x7fb38c14f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39267d700 1 -- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb38c154090 msgr2=0x7fb38c14e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39267d700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb38c154090 0x7fb38c14e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39267d700 1 -- 172.21.15.92:0/1048830825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840092a0 con 0x7fb38c154470 2023-05-16T07:02:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb391e7c700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb38c068160 0x7fb38c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb39267d700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 0x7fb38c167560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fb38c06bb30 tx=0x7fb384017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400b850 con 0x7fb38c154470 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38400b9b0 con 0x7fb38c154470 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38401b660 con 0x7fb38c154470 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c168420 con 0x7fb38c154470 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.039+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c168140 con 0x7fb38c154470 2023-05-16T07:02:52.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.040+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c066730 con 0x7fb38c154470 2023-05-16T07:02:52.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.040+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb38400f070 con 0x7fb38c154470 2023-05-16T07:02:52.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.041+0000 7fb389ffb700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb37805ed90 0x7fb378061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.041+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb38407b8f0 con 0x7fb38c154470 2023-05-16T07:02:52.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.041+0000 7fb391e7c700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb37805ed90 0x7fb378061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.041+0000 7fb391e7c700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb37805ed90 0x7fb378061250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb38c1504e0 tx=0x7fb37c009910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.045+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb38404ce90 con 0x7fb38c154470 2023-05-16T07:02:52.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.205+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb38c0c8c00 con 0x7fb38c154470 2023-05-16T07:02:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.206+0000 7fb389ffb700 1 -- 172.21.15.92:0/1048830825 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb384028070 con 0x7fb38c154470 2023-05-16T07:02:52.211 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb37805ed90 msgr2=0x7fb378061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb37805ed90 0x7fb378061250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb38c1504e0 tx=0x7fb37c009910).stop 2023-05-16T07:02:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 msgr2=0x7fb38c167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 0x7fb38c167560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fb38c06bb30 tx=0x7fb384017000).stop 2023-05-16T07:02:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 shutdown_connections 2023-05-16T07:02:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb37805ed90 0x7fb378061250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb38c068160 0x7fb38c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb38c154470 0x7fb38c167560 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 --2- 172.21.15.92:0/1048830825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb38c154090 0x7fb38c14e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 >> 172.21.15.92:0/1048830825 conn(0x7fb38c067920 msgr2=0x7fb38c0ab9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 shutdown_connections 2023-05-16T07:02:52.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.208+0000 7fb3940e0700 1 -- 172.21.15.92:0/1048830825 wait complete. 2023-05-16T07:02:52.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:52.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.341+0000 7f104774b700 1 Processor -- start 2023-05-16T07:02:52.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.342+0000 7f104774b700 1 -- start start 2023-05-16T07:02:52.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f104774b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1040154170 0x7f10400c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f104774b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1040154550 0x7f10400c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f104774b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f10400696a0 0x7f1040069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f104774b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10400c0660 con 0x7f1040154170 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f104774b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104006a020 con 0x7f1040154550 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f104774b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400bfd30 con 0x7f10400696a0 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1040154170 0x7f10400c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1040154170 0x7f10400c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41446/0 (socket says 172.21.15.92:41446) 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 -- 172.21.15.92:0/3587283144 learned_addr learned my addr 172.21.15.92:0/3587283144 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f1044ce6700 1 --2- 172.21.15.92:0/3587283144 >> v2:172.21.15.158:3300/0 conn(0x7f1040154550 0x7f10400c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f1045ce8700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f10400696a0 0x7f1040069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 -- 172.21.15.92:0/3587283144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f10400696a0 msgr2=0x7f1040069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f10400696a0 0x7f1040069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 -- 172.21.15.92:0/3587283144 >> v2:172.21.15.158:3300/0 conn(0x7f1040154550 msgr2=0x7f10400c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 --2- 172.21.15.92:0/3587283144 >> v2:172.21.15.158:3300/0 conn(0x7f1040154550 0x7f10400c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 -- 172.21.15.92:0/3587283144 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400c0010 con 0x7f1040154170 2023-05-16T07:02:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f1045ce8700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f10400696a0 0x7f1040069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f10454e7700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154170 0x7f10400c5c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f10400844e0 tx=0x7f1030009840).ready entity=mon.0 client_cookie=ca2352c001918b55 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f103ffff700 1 -- 172.21.15.92:0/3587283144 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000e070 con 0x7f1040154170 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f103ffff700 1 -- 172.21.15.92:0/3587283144 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000ba50 con 0x7f1040154170 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.343+0000 7f103ffff700 1 -- 172.21.15.92:0/3587283144 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000bdd0 con 0x7f1040154170 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 -- 172.21.15.92:0/3587283144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154170 msgr2=0x7f10400c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154170 0x7f10400c5c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f10400844e0 tx=0x7f1030009840).stop 2023-05-16T07:02:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 -- 172.21.15.92:0/3587283144 shutdown_connections 2023-05-16T07:02:52.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f10400696a0 0x7f1040069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 --2- 172.21.15.92:0/3587283144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154170 0x7f10400c5c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 --2- 172.21.15.92:0/3587283144 >> v2:172.21.15.158:3300/0 conn(0x7f1040154550 0x7f10400c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 -- 172.21.15.92:0/3587283144 >> 172.21.15.92:0/3587283144 conn(0x7f10400cb700 msgr2=0x7f10400c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 -- 172.21.15.92:0/3587283144 shutdown_connections 2023-05-16T07:02:52.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.344+0000 7f104774b700 1 -- 172.21.15.92:0/3587283144 wait complete. 2023-05-16T07:02:52.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.345+0000 7f104774b700 1 Processor -- start 2023-05-16T07:02:52.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.345+0000 7f104774b700 1 -- start start 2023-05-16T07:02:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f104774b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10400696a0 0x7f10401449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f104774b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1040154170 0x7f10400aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f104774b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 0x7f10400af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f104774b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400af600 con 0x7f1040154550 2023-05-16T07:02:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f104774b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10401643c0 con 0x7f10400696a0 2023-05-16T07:02:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f104774b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1040164940 con 0x7f1040154170 2023-05-16T07:02:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 0x7f10400af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 0x7f10400af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41454/0 (socket says 172.21.15.92:41454) 2023-05-16T07:02:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 -- 172.21.15.92:0/3302173846 learned_addr learned my addr 172.21.15.92:0/3302173846 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1044ce6700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1040154170 0x7f10400aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f10454e7700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10400696a0 0x7f10401449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 -- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1040154170 msgr2=0x7f10400aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1040154170 0x7f10400aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 -- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10400696a0 msgr2=0x7f10401449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10400696a0 0x7f10401449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 -- 172.21.15.92:0/3302173846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030009ae0 con 0x7f1040154550 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.346+0000 7f1045ce8700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 0x7f10400af110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f103800c650 tx=0x7f1038008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.347+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038010580 con 0x7f1040154550 2023-05-16T07:02:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.347+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10380106e0 con 0x7f1040154550 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.347+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1040148280 con 0x7f1040154550 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.347+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038010a60 con 0x7f1040154550 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.347+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10401645d0 con 0x7f1040154550 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.347+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10401487a0 con 0x7f1040154550 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.348+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f103801b460 con 0x7f1040154550 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.348+0000 7f103d7fa700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f102c05ed90 0x7f102c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.348+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1038079a30 con 0x7f1040154550 2023-05-16T07:02:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.348+0000 7f10454e7700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f102c05ed90 0x7f102c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.349+0000 7f10454e7700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f102c05ed90 0x7f102c061250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1030009930 tx=0x7f1030009840).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.352+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f10401487a0 con 0x7f1040154550 2023-05-16T07:02:52.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.515+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10401487a0 con 0x7f1040154550 2023-05-16T07:02:52.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.516+0000 7f103d7fa700 1 -- 172.21.15.92:0/3302173846 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f103804ee60 con 0x7f1040154550 2023-05-16T07:02:52.521 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:52.521 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f102c05ed90 msgr2=0x7f102c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f102c05ed90 0x7f102c061250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1030009930 tx=0x7f1030009840).stop 2023-05-16T07:02:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 msgr2=0x7f10400af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 0x7f10400af110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f103800c650 tx=0x7f1038008040).stop 2023-05-16T07:02:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 shutdown_connections 2023-05-16T07:02:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f102c05ed90 0x7f102c061250 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1040154550 0x7f10400af110 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f10400696a0 0x7f10401449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 --2- 172.21.15.92:0/3302173846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1040154170 0x7f10400aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 >> 172.21.15.92:0/3302173846 conn(0x7f10400cb700 msgr2=0x7f104014e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 shutdown_connections 2023-05-16T07:02:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.518+0000 7f104774b700 1 -- 172.21.15.92:0/3302173846 wait complete. 2023-05-16T07:02:52.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:52.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:52.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.644+0000 7fc85ed05700 1 Processor -- start 2023-05-16T07:02:52.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.644+0000 7fc85ed05700 1 -- start start 2023-05-16T07:02:52.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.645+0000 7fc85ed05700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc858152e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.645+0000 7fc85ed05700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc858063690 con 0x7fc8580622b0 2023-05-16T07:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.645+0000 7fc85ed05700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc858153330 con 0x7fc85814f990 2023-05-16T07:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.645+0000 7fc85ed05700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580c5100 con 0x7fc8580c4c80 2023-05-16T07:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.645+0000 7fc85d2a2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc8580622b0 0x7fc858063b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52210/0 (socket says 172.21.15.92:52210) 2023-05-16T07:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.645+0000 7fc85d2a2700 1 -- 172.21.15.92:0/4208762177 learned_addr learned my addr 172.21.15.92:0/4208762177 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc857fff700 1 --2- 172.21.15.92:0/4208762177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc858152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc857fff700 1 -- 172.21.15.92:0/4208762177 >> v1:172.21.15.158:6789/0 conn(0x7fc85814f990 legacy=0x7fc8580c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc857fff700 1 -- 172.21.15.92:0/4208762177 >> v1:172.21.15.92:6789/0 conn(0x7fc8580622b0 legacy=0x7fc858063b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:52.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc8577fe700 1 -- 172.21.15.92:0/4208762177 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827366865 0 0) 0x7fc858153330 con 0x7fc85814f990 2023-05-16T07:02:52.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc857fff700 1 -- 172.21.15.92:0/4208762177 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85814fd70 con 0x7fc8580c4c80 2023-05-16T07:02:52.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc857fff700 1 --2- 172.21.15.92:0/4208762177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc858152e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc84c009790 tx=0x7fc84c009060).ready entity=mon.2 client_cookie=48c56e12106865b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc8577fe700 1 -- 172.21.15.92:0/4208762177 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00b580 con 0x7fc8580c4c80 2023-05-16T07:02:52.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc8577fe700 1 -- 172.21.15.92:0/4208762177 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00b6e0 con 0x7fc8580c4c80 2023-05-16T07:02:52.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.646+0000 7fc8577fe700 1 -- 172.21.15.92:0/4208762177 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00ba60 con 0x7fc8580c4c80 2023-05-16T07:02:52.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 -- 172.21.15.92:0/4208762177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 msgr2=0x7fc858152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 --2- 172.21.15.92:0/4208762177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc858152e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc84c009790 tx=0x7fc84c009060).stop 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 -- 172.21.15.92:0/4208762177 shutdown_connections 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 --2- 172.21.15.92:0/4208762177 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc858152e40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 -- 172.21.15.92:0/4208762177 >> 172.21.15.92:0/4208762177 conn(0x7fc85805e820 msgr2=0x7fc8580c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 -- 172.21.15.92:0/4208762177 shutdown_connections 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.647+0000 7fc85ed05700 1 -- 172.21.15.92:0/4208762177 wait complete. 2023-05-16T07:02:52.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.648+0000 7fc85ed05700 1 Processor -- start 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.648+0000 7fc85ed05700 1 -- start start 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85ed05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 0x7fc85814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85ed05700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc8580abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85ed05700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc85814f990 0x7fc8580ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85ed05700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580ac6a0 con 0x7fc8580622b0 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85ed05700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858155400 con 0x7fc85814f990 2023-05-16T07:02:52.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85ed05700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581547f0 con 0x7fc8580c4c80 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 0x7fc85814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 0x7fc85814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41464/0 (socket says 172.21.15.92:41464) 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 -- 172.21.15.92:0/2935213866 learned_addr learned my addr 172.21.15.92:0/2935213866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc857fff700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc8580abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85d2a2700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc85814f990 0x7fc8580ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 -- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 msgr2=0x7fc8580abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc8580abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 -- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc85814f990 msgr2=0x7fc8580ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.649+0000 7fc85caa1700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc85814f990 0x7fc8580ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc85caa1700 1 -- 172.21.15.92:0/2935213866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c0092a0 con 0x7fc8580622b0 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc85d2a2700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc85814f990 0x7fc8580ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc85caa1700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 0x7fc85814e5c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fc84800c410 tx=0x7fc848007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848007020 con 0x7fc8580622b0 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8480071a0 con 0x7fc8580622b0 2023-05-16T07:02:52.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc858154ad0 con 0x7fc8580622b0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848007560 con 0x7fc8580622b0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.650+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858155090 con 0x7fc8580622b0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.651+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85814f3c0 con 0x7fc8580622b0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.651+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc848010070 con 0x7fc8580622b0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.651+0000 7fc854ff9700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc84405ed90 0x7fc844061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.651+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc848078e60 con 0x7fc8580622b0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.651+0000 7fc857fff700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc84405ed90 0x7fc844061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.652+0000 7fc857fff700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc84405ed90 0x7fc844061250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc84c0096e0 tx=0x7fc84c017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.655+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc8480478f0 con 0x7fc8580622b0 2023-05-16T07:02:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.817+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc8580c52f0 con 0x7fc8580622b0 2023-05-16T07:02:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.818+0000 7fc854ff9700 1 -- 172.21.15.92:0/2935213866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc84804a400 con 0x7fc8580622b0 2023-05-16T07:02:52.824 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.819+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc84405ed90 msgr2=0x7fc844061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.819+0000 7fc85ed05700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc84405ed90 0x7fc844061250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc84c0096e0 tx=0x7fc84c017000).stop 2023-05-16T07:02:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.819+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 msgr2=0x7fc85814e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.819+0000 7fc85ed05700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 0x7fc85814e5c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fc84800c410 tx=0x7fc848007e00).stop 2023-05-16T07:02:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 shutdown_connections 2023-05-16T07:02:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc84405ed90 0x7fc844061250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc85814f990 0x7fc8580ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8580622b0 0x7fc85814e5c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 --2- 172.21.15.92:0/2935213866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8580c4c80 0x7fc8580abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 >> 172.21.15.92:0/2935213866 conn(0x7fc85805e820 msgr2=0x7fc858063b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 shutdown_connections 2023-05-16T07:02:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.820+0000 7fc85ed05700 1 -- 172.21.15.92:0/2935213866 wait complete. 2023-05-16T07:02:52.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:52.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.945+0000 7fe928851700 1 Processor -- start 2023-05-16T07:02:52.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.945+0000 7fe928851700 1 -- start start 2023-05-16T07:02:52.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe928851700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe924153f90 0x7fe9240a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe928851700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe928851700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9240c6670 con 0x7fe9240c0150 2023-05-16T07:02:52.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe928851700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe924069f60 con 0x7fe924153f90 2023-05-16T07:02:52.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe928851700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92406a9e0 con 0x7fe9240c5d00 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe9237fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe924153f90 0x7fe9240a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.946+0000 7fe922ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe924153f90 0x7fe9240a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:45108/0 (socket says 172.21.15.92:45108) 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 -- 172.21.15.92:0/2049309486 learned_addr learned my addr 172.21.15.92:0/2049309486 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe923fff700 1 --1- 172.21.15.92:0/2049309486 >> v1:172.21.15.158:6789/0 conn(0x7fe9240c0150 0x7fe9240c05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:55820/0 (socket says 172.21.15.92:55820) 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 -- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 msgr2=0x7fe9240c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 --2- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 -- 172.21.15.92:0/2049309486 >> v1:172.21.15.158:6789/0 conn(0x7fe9240c0150 legacy=0x7fe9240c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 -- 172.21.15.92:0/2049309486 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92406a140 con 0x7fe924153f90 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9227fc700 1 -- 172.21.15.92:0/2049309486 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560055841 0 0) 0x7fe9240c6670 con 0x7fe9240c0150 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe922ffd700 1 --2- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9237fe700 1 --2- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a9d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe914009f20 tx=0x7fe914009840).ready entity=mon.1 client_cookie=aecbda5457a91a6a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9227fc700 1 -- 172.21.15.92:0/2049309486 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91400e070 con 0x7fe924153f90 2023-05-16T07:02:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9227fc700 1 -- 172.21.15.92:0/2049309486 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91400b9d0 con 0x7fe924153f90 2023-05-16T07:02:52.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.947+0000 7fe9227fc700 1 -- 172.21.15.92:0/2049309486 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91400bd50 con 0x7fe924153f90 2023-05-16T07:02:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 -- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 msgr2=0x7fe9240a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 --2- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a9d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe914009f20 tx=0x7fe914009840).stop 2023-05-16T07:02:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 -- 172.21.15.92:0/2049309486 shutdown_connections 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 --2- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a9d20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 --2- 172.21.15.92:0/2049309486 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 -- 172.21.15.92:0/2049309486 >> 172.21.15.92:0/2049309486 conn(0x7fe9240cb600 msgr2=0x7fe9240c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 -- 172.21.15.92:0/2049309486 shutdown_connections 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.948+0000 7fe928851700 1 -- 172.21.15.92:0/2049309486 wait complete. 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.949+0000 7fe928851700 1 Processor -- start 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.949+0000 7fe928851700 1 -- start start 2023-05-16T07:02:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 0x7fe924144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9241425b0 con 0x7fe9240c0150 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240a22b0 con 0x7fe924153f90 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe924142360 con 0x7fe9240c5d00 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 0x7fe924144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 0x7fe924144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41476/0 (socket says 172.21.15.92:41476) 2023-05-16T07:02:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 -- 172.21.15.92:0/289286215 learned_addr learned my addr 172.21.15.92:0/289286215 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe922ffd700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe923fff700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 -- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 msgr2=0x7fe9240a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 -- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 msgr2=0x7fe9240a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 -- 172.21.15.92:0/289286215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180092a0 con 0x7fe9240c0150 2023-05-16T07:02:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe923fff700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe9237fe700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 0x7fe924144ee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fe914013fd0 tx=0x7fe9240aa7a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91400bb40 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91400bca0 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91401d670 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe914009ae0 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.950+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9240af620 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.951+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe924083120 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.952+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe914024030 con 0x7fe9240c0150 2023-05-16T07:02:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.952+0000 7fe90b7fe700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe91005ed90 0x7fe910061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.952+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe91407ae80 con 0x7fe9240c0150 2023-05-16T07:02:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.952+0000 7fe922ffd700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe91005ed90 0x7fe910061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.952+0000 7fe922ffd700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe91005ed90 0x7fe910061250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe918009270 tx=0x7fe918008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:52.956+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe91404c4d0 con 0x7fe9240c0150 2023-05-16T07:02:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.117+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe924047150 con 0x7fe9240c0150 2023-05-16T07:02:53.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.117+0000 7fe90b7fe700 1 -- 172.21.15.92:0/289286215 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe914022030 con 0x7fe9240c0150 2023-05-16T07:02:53.122 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe91005ed90 msgr2=0x7fe910061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe91005ed90 0x7fe910061250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe918009270 tx=0x7fe918008040).stop 2023-05-16T07:02:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 msgr2=0x7fe924144ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 0x7fe924144ee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fe914013fd0 tx=0x7fe9240aa7a0).stop 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 shutdown_connections 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe91005ed90 0x7fe910061250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe924153f90 0x7fe9240a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9240c0150 0x7fe924144ee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 --2- 172.21.15.92:0/289286215 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9240c5d00 0x7fe9240a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 >> 172.21.15.92:0/289286215 conn(0x7fe9240cb600 msgr2=0x7fe9240c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 shutdown_connections 2023-05-16T07:02:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.119+0000 7fe928851700 1 -- 172.21.15.92:0/289286215 wait complete. 2023-05-16T07:02:53.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:53.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.244+0000 7fe86cf25700 1 Processor -- start 2023-05-16T07:02:53.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.244+0000 7fe86cf25700 1 -- start start 2023-05-16T07:02:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86cf25700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe868154090 0x7fe8680aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86cf25700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe868154470 0x7fe8680a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86cf25700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8680aa570 con 0x7fe8680aa150 2023-05-16T07:02:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86cf25700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680c94d0 con 0x7fe868154470 2023-05-16T07:02:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86cf25700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680cb7e0 con 0x7fe868154090 2023-05-16T07:02:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe868154470 0x7fe8680a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe868154470 0x7fe8680a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41488/0 (socket says 172.21.15.92:41488) 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 -- 172.21.15.92:0/1114513835 learned_addr learned my addr 172.21.15.92:0/1114513835 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86659c700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe868154090 0x7fe8680aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 -- 172.21.15.92:0/1114513835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe868154090 msgr2=0x7fe8680aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe868154090 0x7fe8680aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 -- 172.21.15.92:0/1114513835 >> v1:172.21.15.158:6789/0 conn(0x7fe8680aa150 legacy=0x7fe868068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 -- 172.21.15.92:0/1114513835 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680c88c0 con 0x7fe868154470 2023-05-16T07:02:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86659c700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe868154090 0x7fe8680aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe865d9b700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680a9c60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fe85c009f20 tx=0x7fe85c009840).ready entity=mon.0 client_cookie=3191ee8f0142d87b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86559a700 1 -- 172.21.15.92:0/1114513835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00e070 con 0x7fe868154470 2023-05-16T07:02:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86559a700 1 -- 172.21.15.92:0/1114513835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c00ba80 con 0x7fe868154470 2023-05-16T07:02:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.245+0000 7fe86559a700 1 -- 172.21.15.92:0/1114513835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00be00 con 0x7fe868154470 2023-05-16T07:02:53.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 -- 172.21.15.92:0/1114513835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 msgr2=0x7fe8680a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680a9c60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fe85c009f20 tx=0x7fe85c009840).stop 2023-05-16T07:02:53.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 -- 172.21.15.92:0/1114513835 shutdown_connections 2023-05-16T07:02:53.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe868154090 0x7fe8680aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1114513835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680a9c60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 -- 172.21.15.92:0/1114513835 >> 172.21.15.92:0/1114513835 conn(0x7fe868067920 msgr2=0x7fe8680cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:53.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 -- 172.21.15.92:0/1114513835 shutdown_connections 2023-05-16T07:02:53.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.246+0000 7fe86cf25700 1 -- 172.21.15.92:0/1114513835 wait complete. 2023-05-16T07:02:53.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.247+0000 7fe86cf25700 1 Processor -- start 2023-05-16T07:02:53.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.247+0000 7fe86cf25700 1 -- start start 2023-05-16T07:02:53.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86cf25700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe8680aa150 0x7fe86814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86cf25700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe868154090 0x7fe868153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86cf25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86cf25700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680aecb0 con 0x7fe868154470 2023-05-16T07:02:53.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86cf25700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868156c00 con 0x7fe868154090 2023-05-16T07:02:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86cf25700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe868156ea0 con 0x7fe8680aa150 2023-05-16T07:02:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41494/0 (socket says 172.21.15.92:41494) 2023-05-16T07:02:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 -- 172.21.15.92:0/1830440271 learned_addr learned my addr 172.21.15.92:0/1830440271 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe86659c700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe8680aa150 0x7fe86814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe865d9b700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe868154090 0x7fe868153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 -- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe8680aa150 msgr2=0x7fe86814cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe8680aa150 0x7fe86814cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 -- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe868154090 msgr2=0x7fe868153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe868154090 0x7fe868153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 -- 172.21.15.92:0/1830440271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c009ae0 con 0x7fe868154470 2023-05-16T07:02:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.248+0000 7fe866d9d700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680c43c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fe85800c440 tx=0x7fe858007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.249+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580123f0 con 0x7fe868154470 2023-05-16T07:02:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.249+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858001e90 con 0x7fe868154470 2023-05-16T07:02:53.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.249+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe868157180 con 0x7fe868154470 2023-05-16T07:02:53.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.249+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858012760 con 0x7fe868154470 2023-05-16T07:02:53.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.249+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86814d400 con 0x7fe868154470 2023-05-16T07:02:53.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.250+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe868066730 con 0x7fe868154470 2023-05-16T07:02:53.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.250+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe858010070 con 0x7fe868154470 2023-05-16T07:02:53.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.250+0000 7fe8567fc700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe84805edc0 0x7fe848061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.250+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe858078b20 con 0x7fe868154470 2023-05-16T07:02:53.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.250+0000 7fe86659c700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe84805edc0 0x7fe848061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.251+0000 7fe86659c700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe84805edc0 0x7fe848061280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fe860006d80 tx=0x7fe860008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.255+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe8580478f0 con 0x7fe868154470 2023-05-16T07:02:53.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.415+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8680aa550 con 0x7fe868154470 2023-05-16T07:02:53.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.415+0000 7fe8567fc700 1 -- 172.21.15.92:0/1830440271 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe85804a0c0 con 0x7fe868154470 2023-05-16T07:02:53.421 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:53.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe84805edc0 msgr2=0x7fe848061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe84805edc0 0x7fe848061280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fe860006d80 tx=0x7fe860008040).stop 2023-05-16T07:02:53.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 msgr2=0x7fe8680c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680c43c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fe85800c440 tx=0x7fe858007d70).stop 2023-05-16T07:02:53.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 shutdown_connections 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe84805edc0 0x7fe848061280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe8680aa150 0x7fe86814cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe868154470 0x7fe8680c43c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 --2- 172.21.15.92:0/1830440271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe868154090 0x7fe868153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 >> 172.21.15.92:0/1830440271 conn(0x7fe868067920 msgr2=0x7fe8680c0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 shutdown_connections 2023-05-16T07:02:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.417+0000 7fe86cf25700 1 -- 172.21.15.92:0/1830440271 wait complete. 2023-05-16T07:02:53.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:53.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.544+0000 7f465d3d0700 1 Processor -- start 2023-05-16T07:02:53.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.544+0000 7f465d3d0700 1 -- start start 2023-05-16T07:02:53.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.544+0000 7f465d3d0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f46581509f0 0x7f46580a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.544+0000 7f465d3d0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4658151780 con 0x7f4658069820 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.544+0000 7f465d3d0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4658151a60 con 0x7f4658150dd0 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.544+0000 7f465d3d0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4658065570 con 0x7f46581509f0 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46577fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4658069820 0x7f46580659c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52248/0 (socket says 172.21.15.92:52248) 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46577fe700 1 -- 172.21.15.92:0/3699498502 learned_addr learned my addr 172.21.15.92:0/3699498502 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696418129 0 0) 0x7f4658151780 con 0x7f4658069820 2023-05-16T07:02:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4656ffd700 1 --2- 172.21.15.92:0/3699498502 >> v2:172.21.15.158:3300/0 conn(0x7f46581509f0 0x7f46580a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46440033f0 con 0x7f4658069820 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586872642 0 0) 0x7f46440033f0 con 0x7f4658069820 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 >> v2:172.21.15.158:3300/0 conn(0x7f46581509f0 msgr2=0x7f46580a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 --2- 172.21.15.92:0/3699498502 >> v2:172.21.15.158:3300/0 conn(0x7f46581509f0 0x7f46580a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 >> v1:172.21.15.158:6789/0 conn(0x7f4658150dd0 legacy=0x7f4658069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658151d40 con 0x7f4658069820 2023-05-16T07:02:53.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4650001470 con 0x7f4658069820 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 >> v1:172.21.15.92:6789/0 conn(0x7f4658069820 legacy=0x7f46580659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46440038e0 0x7f4644003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f4644004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4644005000 0x7f464400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4658065570 con 0x7f4644004690 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464400beb0 con 0x7f4644005000 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464400c070 con 0x7f46440038e0 2023-05-16T07:02:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46567fc700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f4644004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46577fe700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4644005000 0x7f464400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f4656ffd700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46440038e0 0x7f4644003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46567fc700 1 -- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46440038e0 msgr2=0x7f4644003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46567fc700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46440038e0 0x7f4644003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46567fc700 1 -- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4644005000 msgr2=0x7f464400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46567fc700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4644005000 0x7f464400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46577fe700 1 -- 172.21.15.92:0/3699498502 reap_dead start 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.545+0000 7f46567fc700 1 -- 172.21.15.92:0/3699498502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46440033f0 con 0x7f4644004690 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f4656ffd700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46440038e0 0x7f4644003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f46567fc700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f4644004b10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f4640001730 tx=0x7f4640007bb0).ready entity=mon.0 client_cookie=947ccbf8c875c4d5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640007020 con 0x7f4644004690 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4640007180 con 0x7f4644004690 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f4655ffb700 1 -- 172.21.15.92:0/3699498502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640007520 con 0x7f4644004690 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 -- 172.21.15.92:0/3699498502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 msgr2=0x7f4644004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f4644004b10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f4640001730 tx=0x7f4640007bb0).stop 2023-05-16T07:02:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 -- 172.21.15.92:0/3699498502 shutdown_connections 2023-05-16T07:02:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 --2- 172.21.15.92:0/3699498502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f4644004b10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 -- 172.21.15.92:0/3699498502 >> 172.21.15.92:0/3699498502 conn(0x7f46580c8270 msgr2=0x7f46580c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 -- 172.21.15.92:0/3699498502 shutdown_connections 2023-05-16T07:02:53.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.546+0000 7f465d3d0700 1 -- 172.21.15.92:0/3699498502 wait complete. 2023-05-16T07:02:53.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 Processor -- start 2023-05-16T07:02:53.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 -- start start 2023-05-16T07:02:53.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f46581552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4658165410 0x7f4658153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4658153c80 0x7f46580c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4658067d40 con 0x7f4644004690 2023-05-16T07:02:53.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465813eef0 con 0x7f4658165410 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f465d3d0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4658161e60 con 0x7f4658153c80 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f4656ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f46581552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f4656ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f46581552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41510/0 (socket says 172.21.15.92:41510) 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f4656ffd700 1 -- 172.21.15.92:0/2785150600 learned_addr learned my addr 172.21.15.92:0/2785150600 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f46577fe700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4658153c80 0x7f46580c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.548+0000 7f46567fc700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4658165410 0x7f4658153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f4656ffd700 1 -- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4658153c80 msgr2=0x7f46580c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f4656ffd700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4658153c80 0x7f46580c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f4656ffd700 1 -- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4658165410 msgr2=0x7f4658153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f4656ffd700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4658165410 0x7f4658153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f4656ffd700 1 -- 172.21.15.92:0/2785150600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46580671c0 con 0x7f4644004690 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f46567fc700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4658165410 0x7f4658153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f4656ffd700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f46581552f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f46480101c0 tx=0x7f464800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464800e030 con 0x7f4644004690 2023-05-16T07:02:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464800f040 con 0x7f4644004690 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46580674a0 con 0x7f4644004690 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648015690 con 0x7f4644004690 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.549+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4658067a60 con 0x7f4644004690 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.550+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4658047150 con 0x7f4644004690 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.550+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4648015ac0 con 0x7f4644004690 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.551+0000 7f464effd700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f463805ed60 0x7f4638061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.551+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f464807a8a0 con 0x7f4644004690 2023-05-16T07:02:53.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.551+0000 7f46567fc700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f463805ed60 0x7f4638061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.551+0000 7f46567fc700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f463805ed60 0x7f4638061220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f4640001010 tx=0x7f4640015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.555+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f464804be40 con 0x7f4644004690 2023-05-16T07:02:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.719+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46580c73e0 con 0x7f4644004690 2023-05-16T07:02:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.719+0000 7f464effd700 1 -- 172.21.15.92:0/2785150600 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f464804f460 con 0x7f4644004690 2023-05-16T07:02:53.725 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f463805ed60 msgr2=0x7f4638061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f463805ed60 0x7f4638061220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f4640001010 tx=0x7f4640015000).stop 2023-05-16T07:02:53.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 msgr2=0x7f46581552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f46581552f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f46480101c0 tx=0x7f464800b040).stop 2023-05-16T07:02:53.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 shutdown_connections 2023-05-16T07:02:53.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f463805ed60 0x7f4638061220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4658153c80 0x7f46580c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4644004690 0x7f46581552f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 --2- 172.21.15.92:0/2785150600 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4658165410 0x7f4658153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 >> 172.21.15.92:0/2785150600 conn(0x7f46580c8270 msgr2=0x7f46581565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:53.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 shutdown_connections 2023-05-16T07:02:53.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.721+0000 7f465d3d0700 1 -- 172.21.15.92:0/2785150600 wait complete. 2023-05-16T07:02:53.768 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-05-16T07:02:53.769 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:02:53.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-05-16T07:02:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.878+0000 7f51a4a33700 1 Processor -- start 2023-05-16T07:02:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.878+0000 7f51a4a33700 1 -- start start 2023-05-16T07:02:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f51a4a33700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f51a0154550 0x7f51a00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f51a4a33700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a00c0660 con 0x7f51a0154170 2023-05-16T07:02:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f51a4a33700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a0069fc0 con 0x7f51a0069b90 2023-05-16T07:02:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f51a4a33700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a00bfd30 con 0x7f51a0154550 2023-05-16T07:02:53.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51a0154170 0x7f51a00c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52266/0 (socket says 172.21.15.92:52266) 2023-05-16T07:02:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519e59c700 1 -- 172.21.15.92:0/3416680524 learned_addr learned my addr 172.21.15.92:0/3416680524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519dd9b700 1 --2- 172.21.15.92:0/3416680524 >> v2:172.21.15.158:3300/0 conn(0x7f51a0154550 0x7f51a00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897804948 0 0) 0x7f51a00c0660 con 0x7f51a0154170 2023-05-16T07:02:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51880033f0 con 0x7f51a0154170 2023-05-16T07:02:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281291384 0 0) 0x7f51880033f0 con 0x7f51a0154170 2023-05-16T07:02:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 >> v2:172.21.15.158:3300/0 conn(0x7f51a0154550 msgr2=0x7f51a00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 --2- 172.21.15.92:0/3416680524 >> v2:172.21.15.158:3300/0 conn(0x7f51a0154550 0x7f51a00696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 >> v1:172.21.15.158:6789/0 conn(0x7f51a0069b90 legacy=0x7f51a00c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00c0010 con 0x7f51a0154170 2023-05-16T07:02:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f51900017f0 con 0x7f51a0154170 2023-05-16T07:02:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 >> v1:172.21.15.92:6789/0 conn(0x7f51a0154170 legacy=0x7f51a00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188003930 0x7f5188003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188004690 0x7f5188004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5188005120 0x7f518800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00bfd30 con 0x7f5188003930 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518800c030 con 0x7f5188005120 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518800c1f0 con 0x7f5188004690 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519e59c700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188003930 0x7f5188003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.879+0000 7f519ed9d700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5188005120 0x7f518800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519dd9b700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188004690 0x7f5188004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519e59c700 1 -- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188004690 msgr2=0x7f5188004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519e59c700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188004690 0x7f5188004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519e59c700 1 -- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5188005120 msgr2=0x7f518800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519e59c700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5188005120 0x7f518800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519e59c700 1 -- 172.21.15.92:0/3416680524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880033f0 con 0x7f5188003930 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519ed9d700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5188005120 0x7f518800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519ed9d700 1 -- 172.21.15.92:0/3416680524 reap_dead start 2023-05-16T07:02:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519e59c700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188003930 0x7f5188003d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f519000c3c0 tx=0x7f519000c6d0).ready entity=mon.0 client_cookie=a56e0ff3a3fcedc4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190001f60 con 0x7f5188003930 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000cd60 con 0x7f5188003930 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f519d59a700 1 -- 172.21.15.92:0/3416680524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190005d30 con 0x7f5188003930 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 -- 172.21.15.92:0/3416680524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188003930 msgr2=0x7f5188003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188003930 0x7f5188003d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f519000c3c0 tx=0x7f519000c6d0).stop 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 -- 172.21.15.92:0/3416680524 shutdown_connections 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 --2- 172.21.15.92:0/3416680524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188003930 0x7f5188003d50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 -- 172.21.15.92:0/3416680524 >> 172.21.15.92:0/3416680524 conn(0x7f51a00cb700 msgr2=0x7f51a00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 -- 172.21.15.92:0/3416680524 shutdown_connections 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.880+0000 7f51a4a33700 1 -- 172.21.15.92:0/3416680524 wait complete. 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.881+0000 7f51a4a33700 1 Processor -- start 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 -- start start 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188003930 0x7f51a00adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51a00a9f70 0x7f51a0164a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 0x7f51a01515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00ae120 con 0x7f51a014d870 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0165340 con 0x7f51a00a9f70 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f51a4a33700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0165620 con 0x7f5188003930 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f519ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 0x7f51a01515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f519ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 0x7f51a01515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41534/0 (socket says 172.21.15.92:41534) 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f519ed9d700 1 -- 172.21.15.92:0/4048603041 learned_addr learned my addr 172.21.15.92:0/4048603041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:53.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f519e59c700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188003930 0x7f51a00adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.882+0000 7f519dd9b700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51a00a9f70 0x7f51a0164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f519ed9d700 1 -- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188003930 msgr2=0x7f51a00adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f519ed9d700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188003930 0x7f51a00adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f519ed9d700 1 -- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51a00a9f70 msgr2=0x7f51a0164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:53.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f519ed9d700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51a00a9f70 0x7f51a0164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:53.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f519ed9d700 1 -- 172.21.15.92:0/4048603041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0165900 con 0x7f51a014d870 2023-05-16T07:02:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f519ed9d700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 0x7f51a01515a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f51a0083aa0 tx=0x7f5198009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800ebe0 con 0x7f51a014d870 2023-05-16T07:02:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800ed40 con 0x7f51a014d870 2023-05-16T07:02:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a014fa40 con 0x7f51a014d870 2023-05-16T07:02:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198014660 con 0x7f51a014d870 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.883+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a0164f70 con 0x7f51a014d870 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.884+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a014ff60 con 0x7f51a014d870 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.884+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f519801c060 con 0x7f51a014d870 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.884+0000 7f518e7fc700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f518005ed90 0x7f5180061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.885+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5198078f00 con 0x7f51a014d870 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.885+0000 7f519e59c700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f518005ed90 0x7f5180061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:53.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.885+0000 7f519e59c700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f518005ed90 0x7f5180061250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f519000c710 tx=0x7f5190015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:53.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:53.888+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f51a014ff60 con 0x7f51a014d870 2023-05-16T07:02:54.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.116+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f51a0153a90 con 0x7f51a014d870 2023-05-16T07:02:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.523+0000 7f518e7fc700 1 -- 172.21.15.92:0/4048603041 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f519804a4a0 con 0x7f51a014d870 2023-05-16T07:02:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f518005ed90 msgr2=0x7f5180061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f518005ed90 0x7f5180061250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f519000c710 tx=0x7f5190015000).stop 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 msgr2=0x7f51a01515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 0x7f51a01515a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f51a0083aa0 tx=0x7f5198009040).stop 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 shutdown_connections 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f518005ed90 0x7f5180061250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5188003930 0x7f51a00adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a014d870 0x7f51a01515a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 --2- 172.21.15.92:0/4048603041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51a00a9f70 0x7f51a0164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 >> 172.21.15.92:0/4048603041 conn(0x7f51a00cb700 msgr2=0x7f51a00ac300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 shutdown_connections 2023-05-16T07:02:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.525+0000 7f51a4a33700 1 -- 172.21.15.92:0/4048603041 wait complete. 2023-05-16T07:02:54.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.651+0000 7f51b02c7700 1 Processor -- start 2023-05-16T07:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.651+0000 7f51b02c7700 1 -- start start 2023-05-16T07:02:54.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.652+0000 7f51b02c7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f51a8062cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.652+0000 7f51b02c7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a8155870 con 0x7f51a8153ee0 2023-05-16T07:02:54.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.652+0000 7f51b02c7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a81548b0 con 0x7f51a80be280 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.652+0000 7f51b02c7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8155670 con 0x7f51a81542c0 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.652+0000 7f51ae063700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51a8153ee0 0x7f51a80bdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52286/0 (socket says 172.21.15.92:52286) 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.652+0000 7f51ae063700 1 -- 172.21.15.92:0/170833640 learned_addr learned my addr 172.21.15.92:0/170833640 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451768283 0 0) 0x7f51a8155870 con 0x7f51a8153ee0 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae864700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f51a8062cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51900033f0 con 0x7f51a8153ee0 2023-05-16T07:02:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623729159 0 0) 0x7f51900033f0 con 0x7f51a8153ee0 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 msgr2=0x7f51a8062cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f51a8062cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 >> v1:172.21.15.158:6789/0 conn(0x7f51a80be280 legacy=0x7f51a8064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8154b90 con 0x7f51a8153ee0 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f51980017f0 con 0x7f51a8153ee0 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 >> v1:172.21.15.92:6789/0 conn(0x7f51a8153ee0 legacy=0x7f51a80bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5190003910 0x7f5190003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f5190004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190004b80 0x7f5190005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8155670 con 0x7f5190003910 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519000b940 con 0x7f5190004b80 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519000bb00 con 0x7f51a81542c0 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5190003910 0x7f5190003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae864700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190004b80 0x7f5190005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad862700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f5190004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 -- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 msgr2=0x7f5190004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f5190004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 -- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190004b80 msgr2=0x7f5190005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190004b80 0x7f5190005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 -- 172.21.15.92:0/170833640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900033f0 con 0x7f5190003910 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae864700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190004b80 0x7f5190005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ad862700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a81542c0 0x7f5190004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:54.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.653+0000 7f51ae063700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5190003910 0x7f5190003d30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f519800c480 tx=0x7f519800c790).ready entity=mon.0 client_cookie=2d1893e21cc816e2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198002030 con 0x7f5190003910 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800ce20 con 0x7f5190003910 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51ad061700 1 -- 172.21.15.92:0/170833640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198005d30 con 0x7f5190003910 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51b02c7700 1 -- 172.21.15.92:0/170833640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5190003910 msgr2=0x7f5190003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51b02c7700 1 --2- 172.21.15.92:0/170833640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5190003910 0x7f5190003d30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f519800c480 tx=0x7f519800c790).stop 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51ae864700 1 -- 172.21.15.92:0/170833640 reap_dead start 2023-05-16T07:02:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51b02c7700 1 -- 172.21.15.92:0/170833640 shutdown_connections 2023-05-16T07:02:54.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51b02c7700 1 -- 172.21.15.92:0/170833640 >> 172.21.15.92:0/170833640 conn(0x7f51a80c8420 msgr2=0x7f51a80c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:54.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51b02c7700 1 -- 172.21.15.92:0/170833640 shutdown_connections 2023-05-16T07:02:54.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.654+0000 7f51b02c7700 1 -- 172.21.15.92:0/170833640 wait complete. 2023-05-16T07:02:54.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.655+0000 7f51b02c7700 1 Processor -- start 2023-05-16T07:02:54.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.655+0000 7f51b02c7700 1 -- start start 2023-05-16T07:02:54.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51b02c7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190003910 0x7f51a8150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51b02c7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a80bfc10 0x7f51a80c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51b02c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 0x7f51a80a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51b02c7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8168780 con 0x7f51a80c3560 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51b02c7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c39c0 con 0x7f5190003910 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51b02c7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8150550 con 0x7f51a80bfc10 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 0x7f51a80a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 0x7f51a80a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41554/0 (socket says 172.21.15.92:41554) 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 -- 172.21.15.92:0/2951352446 learned_addr learned my addr 172.21.15.92:0/2951352446 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ad862700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a80bfc10 0x7f51a80c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae063700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190003910 0x7f51a8150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 -- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a80bfc10 msgr2=0x7f51a80c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a80bfc10 0x7f51a80c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 -- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190003910 msgr2=0x7f51a8150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190003910 0x7f51a8150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 -- 172.21.15.92:0/2951352446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8167b00 con 0x7f51a80c3560 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.656+0000 7f51ae864700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 0x7f51a80a4c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f51a805e870 tx=0x7f51a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.657+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0015440 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.657+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00155a0 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.657+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a8167de0 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.657+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0015920 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.657+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a81683a0 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.657+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a81688c0 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.658+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f51a000f070 con 0x7f51a80c3560 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.658+0000 7f51a67fc700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f519405ed90 0x7f5194061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.658+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f51a0079af0 con 0x7f51a80c3560 2023-05-16T07:02:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.658+0000 7f51ae063700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f519405ed90 0x7f5194061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.659+0000 7f51ae063700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f519405ed90 0x7f5194061250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f519800c7d0 tx=0x7f5198015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.662+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f51a81688c0 con 0x7f51a80c3560 2023-05-16T07:02:54.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.825+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f51a81688c0 con 0x7f51a80c3560 2023-05-16T07:02:54.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.828+0000 7f51a67fc700 1 -- 172.21.15.92:0/2951352446 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51a81688c0 con 0x7f51a80c3560 2023-05-16T07:02:54.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f519405ed90 msgr2=0x7f5194061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f519405ed90 0x7f5194061250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f519800c7d0 tx=0x7f5198015000).stop 2023-05-16T07:02:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 msgr2=0x7f51a80a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 0x7f51a80a4c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f51a805e870 tx=0x7f51a0008040).stop 2023-05-16T07:02:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 shutdown_connections 2023-05-16T07:02:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f519405ed90 0x7f5194061250 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51a80c3560 0x7f51a80a4c00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5190003910 0x7f51a8150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 --2- 172.21.15.92:0/2951352446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51a80bfc10 0x7f51a80c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 >> 172.21.15.92:0/2951352446 conn(0x7f51a80c8420 msgr2=0x7f51a81525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 shutdown_connections 2023-05-16T07:02:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:54.830+0000 7f51b02c7700 1 -- 172.21.15.92:0/2951352446 wait complete. 2023-05-16T07:02:54.847 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:02:54.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:02:54.874 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:02:54.906 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:02:54.907 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:02:54.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:02:54.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-05-16T07:02:54.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:02:54.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:54.950 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:02:55.004 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-05-16T07:02:55.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:02:55.006 DEBUG:teuthology.orchestra.run.smithi158:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-05-16T07:02:55.052 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:02:55.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:55.053 DEBUG:teuthology.orchestra.run.smithi158:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-05-16T07:02:55.070 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-05-16T07:02:55.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:02:55.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-05-16T07:02:55.105 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:02:55.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:55.106 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-05-16T07:02:55.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-05-16T07:02:55.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:02:55.161 DEBUG:teuthology.orchestra.run.smithi158:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-05-16T07:02:55.196 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:02:55.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:02:55.197 DEBUG:teuthology.orchestra.run.smithi158:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-05-16T07:02:55.252 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:02:55.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:55.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.359+0000 7f9ffbee1700 1 Processor -- start 2023-05-16T07:02:55.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.359+0000 7f9ffbee1700 1 -- start start 2023-05-16T07:02:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffbee1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4154170 0x7f9ff40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffbee1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ff4069db0 0x7f9ff40c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffbee1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40c0660 con 0x7f9ff4154550 2023-05-16T07:02:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffbee1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40c1b00 con 0x7f9ff4069db0 2023-05-16T07:02:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffbee1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40bfd30 con 0x7f9ff4154170 2023-05-16T07:02:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffa47e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ff4069db0 0x7f9ff40c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffa47e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ff4069db0 0x7f9ff40c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41560/0 (socket says 172.21.15.92:41560) 2023-05-16T07:02:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.360+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3725178841 learned_addr learned my addr 172.21.15.92:0/3725178841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:55.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ff9c7d700 1 --2- 172.21.15.92:0/3725178841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4154170 0x7f9ff40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3725178841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4154170 msgr2=0x7f9ff40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:55.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffa47e700 1 --2- 172.21.15.92:0/3725178841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4154170 0x7f9ff40c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3725178841 >> v1:172.21.15.92:6789/0 conn(0x7f9ff4154550 legacy=0x7f9ff40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3725178841 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40c0010 con 0x7f9ff4069db0 2023-05-16T07:02:55.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ff8c7b700 1 -- 172.21.15.92:0/3725178841 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803428107 0 0) 0x7f9ff40c0660 con 0x7f9ff4154550 2023-05-16T07:02:55.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffa47e700 1 --2- 172.21.15.92:0/3725178841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4069db0 0x7f9ff40c6160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9fec009f20 tx=0x7f9fec009840).ready entity=mon.0 client_cookie=9742fc4e007b3f3f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:55.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ff8c7b700 1 -- 172.21.15.92:0/3725178841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00e070 con 0x7f9ff4069db0 2023-05-16T07:02:55.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ff8c7b700 1 -- 172.21.15.92:0/3725178841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00ba80 con 0x7f9ff4069db0 2023-05-16T07:02:55.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ff8c7b700 1 -- 172.21.15.92:0/3725178841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00be00 con 0x7f9ff4069db0 2023-05-16T07:02:55.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3725178841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4069db0 msgr2=0x7f9ff40c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.361+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3725178841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4069db0 0x7f9ff40c6160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9fec009f20 tx=0x7f9fec009840).stop 2023-05-16T07:02:55.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.362+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3725178841 shutdown_connections 2023-05-16T07:02:55.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.362+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3725178841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4154170 0x7f9ff40c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.362+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3725178841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4069db0 0x7f9ff40c6160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.362+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3725178841 >> 172.21.15.92:0/3725178841 conn(0x7f9ff40cb700 msgr2=0x7f9ff40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:55.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.362+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3725178841 shutdown_connections 2023-05-16T07:02:55.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.362+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3725178841 wait complete. 2023-05-16T07:02:55.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 Processor -- start 2023-05-16T07:02:55.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 -- start start 2023-05-16T07:02:55.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4069db0 0x7f9ff4152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ff4154170 0x7f9ff41445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 0x7f9ff4144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4147600 con 0x7f9ff4154550 2023-05-16T07:02:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4144fa0 con 0x7f9ff4154170 2023-05-16T07:02:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.363+0000 7f9ffbee1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40ae160 con 0x7f9ff4069db0 2023-05-16T07:02:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 0x7f9ff4144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 0x7f9ff4144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41568/0 (socket says 172.21.15.92:41568) 2023-05-16T07:02:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3983082721 learned_addr learned my addr 172.21.15.92:0/3983082721 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ff947c700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ff4154170 0x7f9ff41445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ff9c7d700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4069db0 0x7f9ff4152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4069db0 msgr2=0x7f9ff4152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4069db0 0x7f9ff4152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ff4154170 msgr2=0x7f9ff41445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ff4154170 0x7f9ff41445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 -- 172.21.15.92:0/3983082721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009ae0 con 0x7f9ff4154550 2023-05-16T07:02:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffa47e700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 0x7f9ff4144ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f9ff4083060 tx=0x7f9fec009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00b6f0 con 0x7f9ff4154550 2023-05-16T07:02:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec013e60 con 0x7f9ff4154550 2023-05-16T07:02:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff4147040 con 0x7f9ff4154550 2023-05-16T07:02:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec01e6a0 con 0x7f9ff4154550 2023-05-16T07:02:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.364+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff40addf0 con 0x7f9ff4154550 2023-05-16T07:02:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.365+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff40666c0 con 0x7f9ff4154550 2023-05-16T07:02:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.366+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9fec025030 con 0x7f9ff4154550 2023-05-16T07:02:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.366+0000 7f9ff1ffb700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ed90 0x7f9fe0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.366+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9fec07b0f0 con 0x7f9ff4154550 2023-05-16T07:02:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.366+0000 7f9ff9c7d700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ed90 0x7f9fe0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.366+0000 7f9ff9c7d700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ed90 0x7f9fe0061250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f9fe400bba0 tx=0x7f9fe4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.370+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9fec04c740 con 0x7f9ff4154550 2023-05-16T07:02:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.575+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ff406c350 con 0x7f9ff4154550 2023-05-16T07:02:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.576+0000 7f9ff1ffb700 1 -- 172.21.15.92:0/3983082721 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f9fec023070 con 0x7f9ff4154550 2023-05-16T07:02:55.581 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:55.581 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ed90 msgr2=0x7f9fe0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ed90 0x7f9fe0061250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f9fe400bba0 tx=0x7f9fe4008040).stop 2023-05-16T07:02:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 msgr2=0x7f9ff4144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 0x7f9ff4144ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f9ff4083060 tx=0x7f9fec009840).stop 2023-05-16T07:02:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 shutdown_connections 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ed90 0x7f9fe0061250 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4069db0 0x7f9ff4152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff4154550 0x7f9ff4144ab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 --2- 172.21.15.92:0/3983082721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ff4154170 0x7f9ff41445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 >> 172.21.15.92:0/3983082721 conn(0x7f9ff40cb700 msgr2=0x7f9ff414f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 shutdown_connections 2023-05-16T07:02:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.578+0000 7f9ffbee1700 1 -- 172.21.15.92:0/3983082721 wait complete. 2023-05-16T07:02:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:55.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.700+0000 7f90c7fff700 1 Processor -- start 2023-05-16T07:02:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.700+0000 7f90c7fff700 1 -- start start 2023-05-16T07:02:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 0x7f90c8154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c7fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c8158970 con 0x7f90c80ad260 2023-05-16T07:02:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c7fff700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c8155340 con 0x7f90c80c7760 2023-05-16T07:02:55.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c7fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80ad6e0 con 0x7f90c80c7380 2023-05-16T07:02:55.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f90c80ad260 0x7f90c814dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52320/0 (socket says 172.21.15.92:52320) 2023-05-16T07:02:55.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c77fe700 1 -- 172.21.15.92:0/1241934466 learned_addr learned my addr 172.21.15.92:0/1241934466 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:55.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 0x7f90c8154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946967409 0 0) 0x7f90c8158970 con 0x7f90c80ad260 2023-05-16T07:02:55.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b40033f0 con 0x7f90c80ad260 2023-05-16T07:02:55.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335516559 0 0) 0x7f90c8155340 con 0x7f90c80c7760 2023-05-16T07:02:55.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c8158970 con 0x7f90c80c7760 2023-05-16T07:02:55.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1551943549 0 0) 0x7f90b40033f0 con 0x7f90c80ad260 2023-05-16T07:02:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 msgr2=0x7f90c8154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 0x7f90c8154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 >> v1:172.21.15.158:6789/0 conn(0x7f90c80c7760 legacy=0x7f90c80acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8157d60 con 0x7f90c80ad260 2023-05-16T07:02:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f90c0001470 con 0x7f90c80ad260 2023-05-16T07:02:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 >> v1:172.21.15.92:6789/0 conn(0x7f90c80ad260 legacy=0x7f90c814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90b4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 0x7f90b4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90b4004b80 0x7f90b4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80ad6e0 con 0x7f90b4003aa0 2023-05-16T07:02:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b400b910 con 0x7f90b4004b80 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b400bad0 con 0x7f90c80c7380 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90b4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c67fc700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 0x7f90b4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c77fe700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90b4004b80 0x7f90b4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 -- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 msgr2=0x7f90b4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c80c7380 0x7f90b4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 -- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90b4004b80 msgr2=0x7f90b4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90b4004b80 0x7f90b4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 -- 172.21.15.92:0/1241934466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40033f0 con 0x7f90b4003aa0 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90b4003ea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f90b800e1c0 tx=0x7f90b800b040).ready entity=mon.0 client_cookie=bb3afb18cd8d2d78 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8009ac0 con 0x7f90b4003aa0 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800f040 con 0x7f90b4003aa0 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.702+0000 7f90c5ffb700 1 -- 172.21.15.92:0/1241934466 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800eca0 con 0x7f90b4003aa0 2023-05-16T07:02:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c7fff700 1 -- 172.21.15.92:0/1241934466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 msgr2=0x7f90b4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c7fff700 1 --2- 172.21.15.92:0/1241934466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90b4003ea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f90b800e1c0 tx=0x7f90b800b040).stop 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c77fe700 1 -- 172.21.15.92:0/1241934466 reap_dead start 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c7fff700 1 -- 172.21.15.92:0/1241934466 shutdown_connections 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c7fff700 1 -- 172.21.15.92:0/1241934466 >> 172.21.15.92:0/1241934466 conn(0x7f90c8068330 msgr2=0x7f90c8068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c7fff700 1 -- 172.21.15.92:0/1241934466 shutdown_connections 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.703+0000 7f90c7fff700 1 -- 172.21.15.92:0/1241934466 wait complete. 2023-05-16T07:02:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.704+0000 7f90c7fff700 1 Processor -- start 2023-05-16T07:02:55.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.704+0000 7f90c7fff700 1 -- start start 2023-05-16T07:02:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90c80ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c8167f10 0x7f90c81494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c80afff0 0x7f90c80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80c9ef0 con 0x7f90b4003aa0 2023-05-16T07:02:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80c9570 con 0x7f90c80afff0 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8168390 con 0x7f90c8167f10 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90c80ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90c80ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41584/0 (socket says 172.21.15.92:41584) 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 -- 172.21.15.92:0/2231313433 learned_addr learned my addr 172.21.15.92:0/2231313433 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c67fc700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c8167f10 0x7f90c81494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 -- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c8167f10 msgr2=0x7f90c81494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c8167f10 0x7f90c81494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 -- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c80afff0 msgr2=0x7f90c80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c80afff0 0x7f90c80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 -- 172.21.15.92:0/2231313433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80c97e0 con 0x7f90b4003aa0 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c6ffd700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90c80ae440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f90b8013a30 tx=0x7f90b801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8027420 con 0x7f90b4003aa0 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b8027580 con 0x7f90b4003aa0 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c80c9ac0 con 0x7f90b4003aa0 2023-05-16T07:02:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.705+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8027730 con 0x7f90b4003aa0 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.706+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c80aea20 con 0x7f90b4003aa0 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.706+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c8047150 con 0x7f90b4003aa0 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.707+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f90b802e070 con 0x7f90b4003aa0 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.707+0000 7f90beffd700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a805ed90 0x7f90a8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.707+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f90b807be10 con 0x7f90b4003aa0 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.707+0000 7f90c67fc700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a805ed90 0x7f90a8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:55.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.708+0000 7f90c67fc700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a805ed90 0x7f90a8061250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f90b00010e0 tx=0x7f90b0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:55.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.712+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f90b804d3b0 con 0x7f90b4003aa0 2023-05-16T07:02:55.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.875+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90c80bfb90 con 0x7f90b4003aa0 2023-05-16T07:02:55.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.876+0000 7f90beffd700 1 -- 172.21.15.92:0/2231313433 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f90b802c030 con 0x7f90b4003aa0 2023-05-16T07:02:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:55.882 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:55.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a805ed90 msgr2=0x7f90a8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a805ed90 0x7f90a8061250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f90b00010e0 tx=0x7f90b0009040).stop 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 msgr2=0x7f90c80ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90c80ae440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f90b8013a30 tx=0x7f90b801a000).stop 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 shutdown_connections 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a805ed90 0x7f90a8061250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c80afff0 0x7f90c80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b4003aa0 0x7f90c80ae440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 --2- 172.21.15.92:0/2231313433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c8167f10 0x7f90c81494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:55.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 >> 172.21.15.92:0/2231313433 conn(0x7f90c8068330 msgr2=0x7f90c8068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:55.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 shutdown_connections 2023-05-16T07:02:55.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:55.878+0000 7f90c7fff700 1 -- 172.21.15.92:0/2231313433 wait complete. 2023-05-16T07:02:55.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:02:24.569243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:02:25.217735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:02:55.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:02:56.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.004+0000 7f1f69c53700 1 Processor -- start 2023-05-16T07:02:56.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.004+0000 7f1f69c53700 1 -- start start 2023-05-16T07:02:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f640aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1f64154470 0x7f1f640a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640aa570 con 0x7f1f640aa150 2023-05-16T07:02:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640c94d0 con 0x7f1f64154470 2023-05-16T07:02:56.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640cb7e0 con 0x7f1f64154090 2023-05-16T07:02:56.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1f64154470 0x7f1f640a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1f64154470 0x7f1f640a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41594/0 (socket says 172.21.15.92:41594) 2023-05-16T07:02:56.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 -- 172.21.15.92:0/3536729498 learned_addr learned my addr 172.21.15.92:0/3536729498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:56.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f637fe700 1 --2- 172.21.15.92:0/3536729498 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f640aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 -- 172.21.15.92:0/3536729498 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 msgr2=0x7f1f640aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:56.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 --2- 172.21.15.92:0/3536729498 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f640aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 -- 172.21.15.92:0/3536729498 >> v1:172.21.15.158:6789/0 conn(0x7f1f640aa150 legacy=0x7f1f64068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 -- 172.21.15.92:0/3536729498 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640c88c0 con 0x7f1f64154470 2023-05-16T07:02:56.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f62ffd700 1 --2- 172.21.15.92:0/3536729498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f64154470 0x7f1f640a9c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f1f58006fd0 tx=0x7f1f580090f0).ready entity=mon.0 client_cookie=644748297689c79a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f627fc700 1 -- 172.21.15.92:0/3536729498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800b600 con 0x7f1f64154470 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f627fc700 1 -- 172.21.15.92:0/3536729498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5800b760 con 0x7f1f64154470 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f627fc700 1 -- 172.21.15.92:0/3536729498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800bae0 con 0x7f1f64154470 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- 172.21.15.92:0/3536729498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f64154470 msgr2=0x7f1f640a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 --2- 172.21.15.92:0/3536729498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f64154470 0x7f1f640a9c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f1f58006fd0 tx=0x7f1f580090f0).stop 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- 172.21.15.92:0/3536729498 shutdown_connections 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 --2- 172.21.15.92:0/3536729498 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f640aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 --2- 172.21.15.92:0/3536729498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f64154470 0x7f1f640a9c60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- 172.21.15.92:0/3536729498 >> 172.21.15.92:0/3536729498 conn(0x7f1f64067920 msgr2=0x7f1f640cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- 172.21.15.92:0/3536729498 shutdown_connections 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.006+0000 7f1f69c53700 1 -- 172.21.15.92:0/3536729498 wait complete. 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.007+0000 7f1f69c53700 1 Processor -- start 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.007+0000 7f1f69c53700 1 -- start start 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f69c53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 0x7f1f6414de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f69c53700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f64156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f69c53700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f64154470 0x7f1f64156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f69c53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641576e0 con 0x7f1f640aa150 2023-05-16T07:02:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f69c53700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c24c0 con 0x7f1f64154470 2023-05-16T07:02:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f69c53700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64157400 con 0x7f1f64154090 2023-05-16T07:02:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 0x7f1f6414de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 0x7f1f6414de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41602/0 (socket says 172.21.15.92:41602) 2023-05-16T07:02:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f637fe700 1 -- 172.21.15.92:0/1114304121 learned_addr learned my addr 172.21.15.92:0/1114304121 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f62ffd700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f64156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.008+0000 7f1f63fff700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f64154470 0x7f1f64156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f637fe700 1 -- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 msgr2=0x7f1f64156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f637fe700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f64156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f637fe700 1 -- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f64154470 msgr2=0x7f1f64156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f637fe700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f64154470 0x7f1f64156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f637fe700 1 -- 172.21.15.92:0/1114304121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f58009370 con 0x7f1f640aa150 2023-05-16T07:02:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f637fe700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 0x7f1f6414de70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1f5400e190 tx=0x7f1f54009920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400ecd0 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5400ee30 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640c1e70 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54014610 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.009+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f64168cc0 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.010+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f64066730 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.010+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1f5401b030 con 0x7f1f640aa150 2023-05-16T07:02:56.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.011+0000 7f1f4b7fe700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1f5005ed60 0x7f1f50061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.011+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1f54079e10 con 0x7f1f640aa150 2023-05-16T07:02:56.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.011+0000 7f1f62ffd700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1f5005ed60 0x7f1f50061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.011+0000 7f1f62ffd700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1f5005ed60 0x7f1f50061220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f1f64158ec0 tx=0x7f1f58017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.015+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1f5404b3b0 con 0x7f1f640aa150 2023-05-16T07:02:56.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.176+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1f640c1cb0 con 0x7f1f640aa150 2023-05-16T07:02:56.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.177+0000 7f1f4b7fe700 1 -- 172.21.15.92:0/1114304121 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1f5404e9d0 con 0x7f1f640aa150 2023-05-16T07:02:56.182 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:02:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1f5005ed60 msgr2=0x7f1f50061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1f5005ed60 0x7f1f50061220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f1f64158ec0 tx=0x7f1f58017000).stop 2023-05-16T07:02:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 msgr2=0x7f1f6414de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 0x7f1f6414de70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1f5400e190 tx=0x7f1f54009920).stop 2023-05-16T07:02:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 shutdown_connections 2023-05-16T07:02:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1f5005ed60 0x7f1f50061220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f64154470 0x7f1f64156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f640aa150 0x7f1f6414de70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 --2- 172.21.15.92:0/1114304121 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f64154090 0x7f1f64156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.178+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 >> 172.21.15.92:0/1114304121 conn(0x7f1f64067920 msgr2=0x7f1f640c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.179+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 shutdown_connections 2023-05-16T07:02:56.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.179+0000 7f1f69c53700 1 -- 172.21.15.92:0/1114304121 wait complete. 2023-05-16T07:02:56.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:02:56.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.304+0000 7f024c0a8700 1 Processor -- start 2023-05-16T07:02:56.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.304+0000 7f024c0a8700 1 -- start start 2023-05-16T07:02:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024c0a8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0244154090 0x7f02440aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024c0a8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0244154470 0x7f02440a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024c0a8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02440c94d0 con 0x7f02440aa150 2023-05-16T07:02:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024c0a8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02440aa5a0 con 0x7f0244154090 2023-05-16T07:02:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024c0a8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02440cb7e0 con 0x7f0244154470 2023-05-16T07:02:56.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f0249e44700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0244154090 0x7f02440aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024a645700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f02440aa150 0x7f0244068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52358/0 (socket says 172.21.15.92:52358) 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f0249e44700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0244154090 0x7f02440aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41606/0 (socket says 172.21.15.92:41606) 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f024a645700 1 -- 172.21.15.92:0/3201597014 learned_addr learned my addr 172.21.15.92:0/3201597014 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.305+0000 7f0249643700 1 --2- 172.21.15.92:0/3201597014 >> v2:172.21.15.158:3300/0 conn(0x7f0244154470 0x7f02440a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0249e44700 1 -- 172.21.15.92:0/3201597014 >> v2:172.21.15.158:3300/0 conn(0x7f0244154470 msgr2=0x7f02440a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0249e44700 1 --2- 172.21.15.92:0/3201597014 >> v2:172.21.15.158:3300/0 conn(0x7f0244154470 0x7f02440a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0249e44700 1 -- 172.21.15.92:0/3201597014 >> v1:172.21.15.92:6789/0 conn(0x7f02440aa150 legacy=0x7f0244068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0249e44700 1 -- 172.21.15.92:0/3201597014 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440c88c0 con 0x7f0244154090 2023-05-16T07:02:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0248e42700 1 -- 172.21.15.92:0/3201597014 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042927524 0 0) 0x7f02440c94d0 con 0x7f02440aa150 2023-05-16T07:02:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0249e44700 1 --2- 172.21.15.92:0/3201597014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154090 0x7f02440aaca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f0244084320 tx=0x7f0234009780).ready entity=mon.0 client_cookie=225c46ba26de464f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0248e42700 1 -- 172.21.15.92:0/3201597014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023400e070 con 0x7f0244154090 2023-05-16T07:02:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0248e42700 1 -- 172.21.15.92:0/3201597014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023400bc30 con 0x7f0244154090 2023-05-16T07:02:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f0248e42700 1 -- 172.21.15.92:0/3201597014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234012410 con 0x7f0244154090 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 -- 172.21.15.92:0/3201597014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154090 msgr2=0x7f02440aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 --2- 172.21.15.92:0/3201597014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154090 0x7f02440aaca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f0244084320 tx=0x7f0234009780).stop 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 -- 172.21.15.92:0/3201597014 shutdown_connections 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 --2- 172.21.15.92:0/3201597014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154090 0x7f02440aaca0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 --2- 172.21.15.92:0/3201597014 >> v2:172.21.15.158:3300/0 conn(0x7f0244154470 0x7f02440a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 -- 172.21.15.92:0/3201597014 >> 172.21.15.92:0/3201597014 conn(0x7f0244067920 msgr2=0x7f02440cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 -- 172.21.15.92:0/3201597014 shutdown_connections 2023-05-16T07:02:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.306+0000 7f024c0a8700 1 -- 172.21.15.92:0/3201597014 wait complete. 2023-05-16T07:02:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.307+0000 7f024c0a8700 1 Processor -- start 2023-05-16T07:02:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 -- start start 2023-05-16T07:02:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02440aa150 0x7f02440b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0244154090 0x7f024414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 0x7f02440c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02440add10 con 0x7f0244154470 2023-05-16T07:02:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024414ef70 con 0x7f0244154090 2023-05-16T07:02:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024c0a8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02440c29c0 con 0x7f02440aa150 2023-05-16T07:02:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024a645700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 0x7f02440c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024a645700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 0x7f02440c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41616/0 (socket says 172.21.15.92:41616) 2023-05-16T07:02:56.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f024a645700 1 -- 172.21.15.92:0/1650329112 learned_addr learned my addr 172.21.15.92:0/1650329112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:56.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f0249e44700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02440aa150 0x7f02440b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.308+0000 7f0249643700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0244154090 0x7f024414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024a645700 1 -- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02440aa150 msgr2=0x7f02440b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024a645700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02440aa150 0x7f02440b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024a645700 1 -- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0244154090 msgr2=0x7f024414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024a645700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0244154090 0x7f024414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024a645700 1 -- 172.21.15.92:0/1650329112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234009a20 con 0x7f0244154470 2023-05-16T07:02:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024a645700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 0x7f02440c3700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f023c00c490 tx=0x7f023c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c00ce00 con 0x7f0244154470 2023-05-16T07:02:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c0072e0 con 0x7f0244154470 2023-05-16T07:02:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02440c2ca0 con 0x7f0244154470 2023-05-16T07:02:56.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c007660 con 0x7f0244154470 2023-05-16T07:02:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.309+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02440a9e50 con 0x7f0244154470 2023-05-16T07:02:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.310+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0244066730 con 0x7f0244154470 2023-05-16T07:02:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.310+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f023c010070 con 0x7f0244154470 2023-05-16T07:02:56.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.311+0000 7f0241ffb700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f023005ed90 0x7f0230061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.311+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f023c0790a0 con 0x7f0244154470 2023-05-16T07:02:56.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.311+0000 7f0249e44700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f023005ed90 0x7f0230061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.311+0000 7f0249e44700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f023005ed90 0x7f0230061250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0234009f90 tx=0x7f0234009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.315+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f023c04a690 con 0x7f0244154470 2023-05-16T07:02:56.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.477+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f024406c350 con 0x7f0244154470 2023-05-16T07:02:56.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.477+0000 7f0241ffb700 1 -- 172.21.15.92:0/1650329112 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f023c04dcb0 con 0x7f0244154470 2023-05-16T07:02:56.482 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:02:56.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f023005ed90 msgr2=0x7f0230061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f023005ed90 0x7f0230061250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0234009f90 tx=0x7f0234009780).stop 2023-05-16T07:02:56.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 msgr2=0x7f02440c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 0x7f02440c3700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f023c00c490 tx=0x7f023c007c10).stop 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 shutdown_connections 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f023005ed90 0x7f0230061250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02440aa150 0x7f02440b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0244154470 0x7f02440c3700 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 --2- 172.21.15.92:0/1650329112 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0244154090 0x7f024414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 >> 172.21.15.92:0/1650329112 conn(0x7f0244067920 msgr2=0x7f02440ab410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 shutdown_connections 2023-05-16T07:02:56.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.479+0000 7f024c0a8700 1 -- 172.21.15.92:0/1650329112 wait complete. 2023-05-16T07:02:56.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:02:56.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.605+0000 7f25248ef700 1 Processor -- start 2023-05-16T07:02:56.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.605+0000 7f25248ef700 1 -- start start 2023-05-16T07:02:56.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.606+0000 7f25248ef700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2520150250 0x7f25200c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.606+0000 7f25248ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f2520066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.606+0000 7f25248ef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2520065aa0 con 0x7f25200c2500 2023-05-16T07:02:56.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.606+0000 7f25248ef700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2520064e90 con 0x7f2520150250 2023-05-16T07:02:56.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.606+0000 7f25248ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2520065170 con 0x7f25201545b0 2023-05-16T07:02:56.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.606+0000 7f251effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f25200c2500 0x7f2520153ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52372/0 (socket says 172.21.15.92:52372) 2023-05-16T07:02:56.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251effd700 1 -- 172.21.15.92:0/3623861812 learned_addr learned my addr 172.21.15.92:0/3623861812 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251ffff700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f2520066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251f7fe700 1 --2- 172.21.15.92:0/3623861812 >> v2:172.21.15.158:3300/0 conn(0x7f2520150250 0x7f25200c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734258049 0 0) 0x7f2520065aa0 con 0x7f25200c2500 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2508004500 con 0x7f25200c2500 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1573897067 0 0) 0x7f2508004500 con 0x7f25200c2500 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 msgr2=0x7f2520066ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:56.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f2520066ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 >> v2:172.21.15.158:3300/0 conn(0x7f2520150250 msgr2=0x7f25200c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 --2- 172.21.15.92:0/3623861812 >> v2:172.21.15.158:3300/0 conn(0x7f2520150250 0x7f25200c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520065450 con 0x7f25200c2500 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f2514001470 con 0x7f25200c2500 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 >> v1:172.21.15.92:6789/0 conn(0x7f25200c2500 legacy=0x7f2520153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25080049f0 0x7f2508004e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f25080057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2508005c90 0x7f2508006110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520064e90 con 0x7f2508005c90 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520065170 con 0x7f25080049f0 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250800c9f0 con 0x7f25201545b0 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251ffff700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2508005c90 0x7f2508006110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251f7fe700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25080049f0 0x7f2508004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251effd700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f25080057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251ffff700 1 -- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 msgr2=0x7f25080057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251ffff700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f25080057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251ffff700 1 -- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25080049f0 msgr2=0x7f2508004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.607+0000 7f251ffff700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25080049f0 0x7f2508004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251ffff700 1 -- 172.21.15.92:0/3623861812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508004500 con 0x7f2508005c90 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251effd700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25201545b0 0x7f25080057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251ffff700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2508005c90 0x7f2508006110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f25200646e0 tx=0x7f2518009060).ready entity=mon.0 client_cookie=f174a36c7b5b8cac server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518009430 con 0x7f2508005c90 2023-05-16T07:02:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518009e70 con 0x7f2508005c90 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251e7fc700 1 -- 172.21.15.92:0/3623861812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251800e850 con 0x7f2508005c90 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f25248ef700 1 -- 172.21.15.92:0/3623861812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2508005c90 msgr2=0x7f2508006110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f25248ef700 1 --2- 172.21.15.92:0/3623861812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2508005c90 0x7f2508006110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f25200646e0 tx=0x7f2518009060).stop 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f251ffff700 1 -- 172.21.15.92:0/3623861812 reap_dead start 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f25248ef700 1 -- 172.21.15.92:0/3623861812 shutdown_connections 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f25248ef700 1 -- 172.21.15.92:0/3623861812 >> 172.21.15.92:0/3623861812 conn(0x7f25200c7a80 msgr2=0x7f25200c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f25248ef700 1 -- 172.21.15.92:0/3623861812 shutdown_connections 2023-05-16T07:02:56.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.608+0000 7f25248ef700 1 -- 172.21.15.92:0/3623861812 wait complete. 2023-05-16T07:02:56.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 Processor -- start 2023-05-16T07:02:56.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 -- start start 2023-05-16T07:02:56.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 0x7f25200ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25200a7460 0x7f252014a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25200a78e0 0x7f252014ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520166570 con 0x7f25200ac310 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252014b100 con 0x7f25200a7460 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f25248ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25201659f0 con 0x7f25200a78e0 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f251f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 0x7f25200ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f251f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 0x7f25200ac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41632/0 (socket says 172.21.15.92:41632) 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f251f7fe700 1 -- 172.21.15.92:0/3773772313 learned_addr learned my addr 172.21.15.92:0/3773772313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f251ffff700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25200a78e0 0x7f252014ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.610+0000 7f251effd700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25200a7460 0x7f252014a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251f7fe700 1 -- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25200a78e0 msgr2=0x7f252014ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251f7fe700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25200a78e0 0x7f252014ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251f7fe700 1 -- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25200a7460 msgr2=0x7f252014a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251f7fe700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25200a7460 0x7f252014a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251f7fe700 1 -- 172.21.15.92:0/3773772313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520165cd0 con 0x7f25200ac310 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251ffff700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25200a78e0 0x7f252014ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251effd700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25200a7460 0x7f252014a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:56.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f251f7fe700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 0x7f25200ac6f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f2510009fd0 tx=0x7f251000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000e040 con 0x7f25200ac310 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251000f040 con 0x7f25200ac310 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2520165fb0 con 0x7f25200ac310 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510015450 con 0x7f25200ac310 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.611+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2520165630 con 0x7f25200ac310 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.612+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25200bf120 con 0x7f25200ac310 2023-05-16T07:02:56.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.612+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2510015880 con 0x7f25200ac310 2023-05-16T07:02:56.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.613+0000 7f25077fe700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f250c05ed60 0x7f250c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.613+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f251007ac60 con 0x7f25200ac310 2023-05-16T07:02:56.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.613+0000 7f251effd700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f250c05ed60 0x7f250c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.613+0000 7f251effd700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f250c05ed60 0x7f250c061220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f2514000e10 tx=0x7f2514009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.617+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f25200bf120 con 0x7f25200ac310 2023-05-16T07:02:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.779+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25200bf120 con 0x7f25200ac310 2023-05-16T07:02:56.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.780+0000 7f25077fe700 1 -- 172.21.15.92:0/3773772313 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25200bf120 con 0x7f25200ac310 2023-05-16T07:02:56.784 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:56.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f250c05ed60 msgr2=0x7f250c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f250c05ed60 0x7f250c061220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f2514000e10 tx=0x7f2514009040).stop 2023-05-16T07:02:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 msgr2=0x7f25200ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 0x7f25200ac6f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f2510009fd0 tx=0x7f251000b040).stop 2023-05-16T07:02:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 shutdown_connections 2023-05-16T07:02:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f250c05ed60 0x7f250c061220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25200a78e0 0x7f252014ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25200ac310 0x7f25200ac6f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 --2- 172.21.15.92:0/3773772313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25200a7460 0x7f252014a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 >> 172.21.15.92:0/3773772313 conn(0x7f25200c7a80 msgr2=0x7f2520153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 shutdown_connections 2023-05-16T07:02:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.781+0000 7f25248ef700 1 -- 172.21.15.92:0/3773772313 wait complete. 2023-05-16T07:02:56.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:02:56.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.907+0000 7f8e851d4700 1 Processor -- start 2023-05-16T07:02:56.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.907+0000 7f8e851d4700 1 -- start start 2023-05-16T07:02:56.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.907+0000 7f8e851d4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8e800c6410 0x7f8e80155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.907+0000 7f8e851d4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8006a140 con 0x7f8e80153f90 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.907+0000 7f8e851d4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8006a420 con 0x7f8e800c1160 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.907+0000 7f8e851d4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e80069f60 con 0x7f8e800c6410 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8e80153f90 0x7f8e800a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52388/0 (socket says 172.21.15.92:52388) 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7ffff700 1 -- 172.21.15.92:0/1458873995 learned_addr learned my addr 172.21.15.92:0/1458873995 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e849d3700 1 --2- 172.21.15.92:0/1458873995 >> v2:172.21.15.158:3300/0 conn(0x7f8e800c6410 0x7f8e80155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7f7fe700 1 -- 172.21.15.92:0/1458873995 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253379111 0 0) 0x7f8e8006a140 con 0x7f8e80153f90 2023-05-16T07:02:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7f7fe700 1 -- 172.21.15.92:0/1458873995 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e6c0033f0 con 0x7f8e80153f90 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7f7fe700 1 -- 172.21.15.92:0/1458873995 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3810612428 0 0) 0x7f8e6c0033f0 con 0x7f8e80153f90 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e849d3700 1 -- 172.21.15.92:0/1458873995 >> v1:172.21.15.158:6789/0 conn(0x7f8e800c1160 legacy=0x7f8e800c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e849d3700 1 -- 172.21.15.92:0/1458873995 >> v1:172.21.15.92:6789/0 conn(0x7f8e80153f90 legacy=0x7f8e800a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e849d3700 1 -- 172.21.15.92:0/1458873995 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8006a700 con 0x7f8e800c6410 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e849d3700 1 --2- 172.21.15.92:0/1458873995 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c6410 0x7f8e80155380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f8e78009e60 tx=0x7f8e78009780).ready entity=mon.1 client_cookie=882523d8dc167d46 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7f7fe700 1 -- 172.21.15.92:0/1458873995 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800e070 con 0x7f8e800c6410 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7f7fe700 1 -- 172.21.15.92:0/1458873995 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7800b850 con 0x7f8e800c6410 2023-05-16T07:02:56.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.908+0000 7f8e7f7fe700 1 -- 172.21.15.92:0/1458873995 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70001470 con 0x7f8e800c6410 2023-05-16T07:02:56.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 -- 172.21.15.92:0/1458873995 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c6410 msgr2=0x7f8e80155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 --2- 172.21.15.92:0/1458873995 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c6410 0x7f8e80155380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f8e78009e60 tx=0x7f8e78009780).stop 2023-05-16T07:02:56.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 -- 172.21.15.92:0/1458873995 shutdown_connections 2023-05-16T07:02:56.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 --2- 172.21.15.92:0/1458873995 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c6410 0x7f8e80155380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 -- 172.21.15.92:0/1458873995 >> 172.21.15.92:0/1458873995 conn(0x7f8e800cb600 msgr2=0x7f8e800c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:56.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 -- 172.21.15.92:0/1458873995 shutdown_connections 2023-05-16T07:02:56.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.909+0000 7f8e851d4700 1 -- 172.21.15.92:0/1458873995 wait complete. 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.910+0000 7f8e851d4700 1 Processor -- start 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.910+0000 7f8e851d4700 1 -- start start 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c1160 0x7f8e8014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 0x7f8e800b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8e80153f90 0x7f8e800c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800aff60 con 0x7f8e800c6410 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800af350 con 0x7f8e800c1160 2023-05-16T07:02:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800af630 con 0x7f8e80153f90 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 0x7f8e800b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 0x7f8e800b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41648/0 (socket says 172.21.15.92:41648) 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 -- 172.21.15.92:0/3173716854 learned_addr learned my addr 172.21.15.92:0/3173716854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e849d3700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8e80153f90 0x7f8e800c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e7ffff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c1160 0x7f8e8014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 -- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8e80153f90 msgr2=0x7f8e800c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8e80153f90 0x7f8e800c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 -- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c1160 msgr2=0x7f8e8014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c1160 0x7f8e8014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 -- 172.21.15.92:0/3173716854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009a20 con 0x7f8e800c6410 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e849d3700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8e80153f90 0x7f8e800c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e7ffff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c1160 0x7f8e8014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e77fff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 0x7f8e800b03f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f8e6800c4c0 tx=0x7f8e68007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007020 con 0x7f8e800c6410 2023-05-16T07:02:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e68007180 con 0x7f8e800c6410 2023-05-16T07:02:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e800af910 con 0x7f8e800c6410 2023-05-16T07:02:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.911+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007520 con 0x7f8e800c6410 2023-05-16T07:02:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.912+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8014e120 con 0x7f8e800c6410 2023-05-16T07:02:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.912+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e80083120 con 0x7f8e800c6410 2023-05-16T07:02:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.913+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8e68010070 con 0x7f8e800c6410 2023-05-16T07:02:56.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.913+0000 7f8e7cff9700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8e6005ed90 0x7f8e60061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:56.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.913+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8e68078e00 con 0x7f8e800c6410 2023-05-16T07:02:56.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.913+0000 7f8e7ffff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8e6005ed90 0x7f8e60061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:56.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.914+0000 7f8e7ffff700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8e6005ed90 0x7f8e60061250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f8e70007aa0 tx=0x7f8e70009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:56.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:56.917+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8e680478f0 con 0x7f8e800c6410 2023-05-16T07:02:57.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.078+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e80047150 con 0x7f8e800c6410 2023-05-16T07:02:57.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.079+0000 7f8e7cff9700 1 -- 172.21.15.92:0/3173716854 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e6804a3a0 con 0x7f8e800c6410 2023-05-16T07:02:57.083 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:02:57.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.080+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8e6005ed90 msgr2=0x7f8e60061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.080+0000 7f8e851d4700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8e6005ed90 0x7f8e60061250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f8e70007aa0 tx=0x7f8e70009040).stop 2023-05-16T07:02:57.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.080+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 msgr2=0x7f8e800b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.080+0000 7f8e851d4700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 0x7f8e800b03f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f8e6800c4c0 tx=0x7f8e68007dd0).stop 2023-05-16T07:02:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.080+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 shutdown_connections 2023-05-16T07:02:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.080+0000 7f8e851d4700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8e6005ed90 0x7f8e60061250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.081+0000 7f8e851d4700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8e80153f90 0x7f8e800c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.081+0000 7f8e851d4700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8e800c1160 0x7f8e8014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.081+0000 7f8e851d4700 1 --2- 172.21.15.92:0/3173716854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e800c6410 0x7f8e800b03f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.081+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 >> 172.21.15.92:0/3173716854 conn(0x7f8e800cb600 msgr2=0x7f8e8014e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.081+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 shutdown_connections 2023-05-16T07:02:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.081+0000 7f8e851d4700 1 -- 172.21.15.92:0/3173716854 wait complete. 2023-05-16T07:02:57.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:57.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.203+0000 7f150d428700 1 Processor -- start 2023-05-16T07:02:57.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.203+0000 7f150d428700 1 -- start start 2023-05-16T07:02:57.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150d428700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15080a7e00 0x7f1508156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150d428700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080c8750 con 0x7f1508151bd0 2023-05-16T07:02:57.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150d428700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1508155d70 con 0x7f15081517f0 2023-05-16T07:02:57.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150d428700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080c8470 con 0x7f15080a7e00 2023-05-16T07:02:57.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150cc27700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15080a7e00 0x7f1508156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f15077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1508151bd0 0x7f15080a76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52406/0 (socket says 172.21.15.92:52406) 2023-05-16T07:02:57.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150cc27700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15080a7e00 0x7f1508156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41656/0 (socket says 172.21.15.92:41656) 2023-05-16T07:02:57.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f15077fe700 1 -- 172.21.15.92:0/530774647 learned_addr learned my addr 172.21.15.92:0/530774647 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:57.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150cc27700 1 -- 172.21.15.92:0/530774647 >> v1:172.21.15.158:6789/0 conn(0x7f15081517f0 legacy=0x7f1508152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:57.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150cc27700 1 -- 172.21.15.92:0/530774647 >> v1:172.21.15.92:6789/0 conn(0x7f1508151bd0 legacy=0x7f15080a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f1506ffd700 1 -- 172.21.15.92:0/530774647 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693259453 0 0) 0x7f15080c8750 con 0x7f1508151bd0 2023-05-16T07:02:57.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150cc27700 1 -- 172.21.15.92:0/530774647 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508155720 con 0x7f15080a7e00 2023-05-16T07:02:57.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f150cc27700 1 --2- 172.21.15.92:0/530774647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15080a7e00 0x7f1508156200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1500009e60 tx=0x7f1500009780).ready entity=mon.0 client_cookie=3308e6344e6168a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f1506ffd700 1 -- 172.21.15.92:0/530774647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000e070 con 0x7f15080a7e00 2023-05-16T07:02:57.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f1506ffd700 1 -- 172.21.15.92:0/530774647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150000bc30 con 0x7f15080a7e00 2023-05-16T07:02:57.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.204+0000 7f1506ffd700 1 -- 172.21.15.92:0/530774647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500012410 con 0x7f15080a7e00 2023-05-16T07:02:57.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 -- 172.21.15.92:0/530774647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15080a7e00 msgr2=0x7f1508156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 --2- 172.21.15.92:0/530774647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15080a7e00 0x7f1508156200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1500009e60 tx=0x7f1500009780).stop 2023-05-16T07:02:57.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 -- 172.21.15.92:0/530774647 shutdown_connections 2023-05-16T07:02:57.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 --2- 172.21.15.92:0/530774647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15080a7e00 0x7f1508156200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 -- 172.21.15.92:0/530774647 >> 172.21.15.92:0/530774647 conn(0x7f150805e820 msgr2=0x7f15080622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:57.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 -- 172.21.15.92:0/530774647 shutdown_connections 2023-05-16T07:02:57.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.205+0000 7f150d428700 1 -- 172.21.15.92:0/530774647 wait complete. 2023-05-16T07:02:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.206+0000 7f150d428700 1 Processor -- start 2023-05-16T07:02:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.206+0000 7f150d428700 1 -- start start 2023-05-16T07:02:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150d428700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15080a7e00 0x7f15080c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150d428700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15081517f0 0x7f150814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150d428700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 0x7f15080ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150d428700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150814bdb0 con 0x7f1508151bd0 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150d428700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15080c2910 con 0x7f15080a7e00 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150d428700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508161ef0 con 0x7f15081517f0 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 0x7f15080ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 0x7f15080ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41664/0 (socket says 172.21.15.92:41664) 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 -- 172.21.15.92:0/2870156646 learned_addr learned my addr 172.21.15.92:0/2870156646 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f15077fe700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15081517f0 0x7f150814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f1507fff700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15080a7e00 0x7f15080c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 -- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15081517f0 msgr2=0x7f150814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15081517f0 0x7f150814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 -- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15080a7e00 msgr2=0x7f15080c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15080a7e00 0x7f15080c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 -- 172.21.15.92:0/2870156646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500009a20 con 0x7f1508151bd0 2023-05-16T07:02:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f1507fff700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15080a7e00 0x7f15080c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f15077fe700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15081517f0 0x7f150814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.207+0000 7f150cc27700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 0x7f15080ac040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f150805fd20 tx=0x7f1500009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.208+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000117d0 con 0x7f1508151bd0 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.208+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1500011930 con 0x7f1508151bd0 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.208+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150814e4d0 con 0x7f1508151bd0 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.208+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500011cf0 con 0x7f1508151bd0 2023-05-16T07:02:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.208+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150814df10 con 0x7f1508151bd0 2023-05-16T07:02:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.208+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508162080 con 0x7f1508151bd0 2023-05-16T07:02:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.209+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f150001d630 con 0x7f1508151bd0 2023-05-16T07:02:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.209+0000 7f14effff700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f14f405ed90 0x7f14f4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.209+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f150007c440 con 0x7f1508151bd0 2023-05-16T07:02:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.209+0000 7f1507fff700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f14f405ed90 0x7f14f4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.210+0000 7f1507fff700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f14f405ed90 0x7f14f4061250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f14f8001700 tx=0x7f14f8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.213+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f150004da60 con 0x7f1508151bd0 2023-05-16T07:02:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.419+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1508047150 con 0x7f1508151bd0 2023-05-16T07:02:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.420+0000 7f14effff700 1 -- 172.21.15.92:0/2870156646 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f1500051080 con 0x7f1508151bd0 2023-05-16T07:02:57.424 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:57.424 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:57.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f14f405ed90 msgr2=0x7f14f4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f14f405ed90 0x7f14f4061250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f14f8001700 tx=0x7f14f8009040).stop 2023-05-16T07:02:57.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 msgr2=0x7f15080ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 0x7f15080ac040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f150805fd20 tx=0x7f1500009780).stop 2023-05-16T07:02:57.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 shutdown_connections 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f14f405ed90 0x7f14f4061250 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1508151bd0 0x7f15080ac040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15080a7e00 0x7f15080c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 --2- 172.21.15.92:0/2870156646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15081517f0 0x7f150814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 >> 172.21.15.92:0/2870156646 conn(0x7f150805e820 msgr2=0x7f15080aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 shutdown_connections 2023-05-16T07:02:57.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.422+0000 7f150d428700 1 -- 172.21.15.92:0/2870156646 wait complete. 2023-05-16T07:02:57.428 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:57.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:57.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.545+0000 7f444b1d2700 1 Processor -- start 2023-05-16T07:02:57.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.545+0000 7f444b1d2700 1 -- start start 2023-05-16T07:02:57.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f444b1d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f44440acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f444b1d2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4444158970 con 0x7f44440ad040 2023-05-16T07:02:57.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f444b1d2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4444157d60 con 0x7f44440c7380 2023-05-16T07:02:57.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f444b1d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4444158040 con 0x7f4444154e50 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f444a9d1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44440ad040 0x7f444414dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52420/0 (socket says 172.21.15.92:52420) 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f444a9d1700 1 -- 172.21.15.92:0/1283416757 learned_addr learned my addr 172.21.15.92:0/1283416757 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133960145 0 0) 0x7f4444158970 con 0x7f44440ad040 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44300033f0 con 0x7f44440ad040 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f44499cf700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f44440acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090800263 0 0) 0x7f44300033f0 con 0x7f44440ad040 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.546+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 msgr2=0x7f44440acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f44440acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 >> v1:172.21.15.158:6789/0 conn(0x7f44440c7380 legacy=0x7f44440c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444158320 con 0x7f44440ad040 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4440001470 con 0x7f44440ad040 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44499cf700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f44440acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 >> v1:172.21.15.92:6789/0 conn(0x7f44440ad040 legacy=0x7f444414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430003910 0x7f4430003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f4430004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4430004b80 0x7f4430005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4444158040 con 0x7f4430004b80 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443000b990 con 0x7f4430003910 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443000bb50 con 0x7f4444154e50 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4430004b80 0x7f4430005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a1d0700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430003910 0x7f4430003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44499cf700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f4430004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 -- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 msgr2=0x7f4430004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4444154e50 0x7f4430004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 -- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430003910 msgr2=0x7f4430003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430003910 0x7f4430003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 -- 172.21.15.92:0/1283416757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300033f0 con 0x7f4430004b80 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f444a9d1700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4430004b80 0x7f4430005000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f4440001ed0 tx=0x7f44441588e0).ready entity=mon.0 client_cookie=2ca248c672274df6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000ca60 con 0x7f4430004b80 2023-05-16T07:02:57.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000cbc0 con 0x7f4430004b80 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.547+0000 7f44491ce700 1 -- 172.21.15.92:0/1283416757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400070d0 con 0x7f4430004b80 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444b1d2700 1 -- 172.21.15.92:0/1283416757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4430004b80 msgr2=0x7f4430005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444b1d2700 1 --2- 172.21.15.92:0/1283416757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4430004b80 0x7f4430005000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f4440001ed0 tx=0x7f44441588e0).stop 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444a9d1700 1 -- 172.21.15.92:0/1283416757 reap_dead start 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444b1d2700 1 -- 172.21.15.92:0/1283416757 shutdown_connections 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444b1d2700 1 -- 172.21.15.92:0/1283416757 >> 172.21.15.92:0/1283416757 conn(0x7f4444068330 msgr2=0x7f4444068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:57.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444b1d2700 1 -- 172.21.15.92:0/1283416757 shutdown_connections 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.548+0000 7f444b1d2700 1 -- 172.21.15.92:0/1283416757 wait complete. 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.549+0000 7f444b1d2700 1 Processor -- start 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.549+0000 7f444b1d2700 1 -- start start 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430004b80 0x7f444414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44440caf60 0x7f44440cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 0x7f4444165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444414e0f0 con 0x7f4444165700 2023-05-16T07:02:57.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440c4a60 con 0x7f4430004b80 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444414d200 con 0x7f44440caf60 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 0x7f4444165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 0x7f4444165b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41684/0 (socket says 172.21.15.92:41684) 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 -- 172.21.15.92:0/4019726735 learned_addr learned my addr 172.21.15.92:0/4019726735 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f44499cf700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44440caf60 0x7f44440cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a1d0700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430004b80 0x7f444414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 -- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44440caf60 msgr2=0x7f44440cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44440caf60 0x7f44440cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 -- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430004b80 msgr2=0x7f444414e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430004b80 0x7f444414e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 -- 172.21.15.92:0/4019726735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444414d4e0 con 0x7f4444165700 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444a9d1700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 0x7f4444165b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f4430005670 tx=0x7f4440015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440007650 con 0x7f4444165700 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44400046b0 con 0x7f4444165700 2023-05-16T07:02:57.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444414d7c0 con 0x7f4444165700 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440004a30 con 0x7f4444165700 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.550+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444414dd80 con 0x7f4444165700 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.551+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44441576f0 con 0x7f4444165700 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.552+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4440027070 con 0x7f4444165700 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.552+0000 7f443a7fc700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f442805ed90 0x7f4428061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.552+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f444007a320 con 0x7f4444165700 2023-05-16T07:02:57.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.552+0000 7f444a1d0700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f442805ed90 0x7f4428061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.553+0000 7f444a1d0700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f442805ed90 0x7f4428061250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f443c001600 tx=0x7f443c007f30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.556+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f444004b8c0 con 0x7f4444165700 2023-05-16T07:02:57.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.720+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4444047150 con 0x7f4444165700 2023-05-16T07:02:57.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.721+0000 7f443a7fc700 1 -- 172.21.15.92:0/4019726735 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f4440025070 con 0x7f4444165700 2023-05-16T07:02:57.725 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:57.725 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:57.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f442805ed90 msgr2=0x7f4428061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f442805ed90 0x7f4428061250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f443c001600 tx=0x7f443c007f30).stop 2023-05-16T07:02:57.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 msgr2=0x7f4444165b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 0x7f4444165b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f4430005670 tx=0x7f4440015000).stop 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 shutdown_connections 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f442805ed90 0x7f4428061250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4444165700 0x7f4444165b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4430004b80 0x7f444414e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 --2- 172.21.15.92:0/4019726735 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44440caf60 0x7f44440cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 >> 172.21.15.92:0/4019726735 conn(0x7f4444068330 msgr2=0x7f4444153670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:57.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 shutdown_connections 2023-05-16T07:02:57.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.723+0000 7f444b1d2700 1 -- 172.21.15.92:0/4019726735 wait complete. 2023-05-16T07:02:57.739 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:02:57.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.846+0000 7f67b6a4a700 1 Processor -- start 2023-05-16T07:02:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.846+0000 7f67b6a4a700 1 -- start start 2023-05-16T07:02:57.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.846+0000 7f67b6a4a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67b0154170 0x7f67b00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.846+0000 7f67b6a4a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154550 0x7f67b00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67b6a4a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b00c0660 con 0x7f67b00696a0 2023-05-16T07:02:57.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67b6a4a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00c1b00 con 0x7f67b0154170 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67b6a4a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00bfd30 con 0x7f67b0154550 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67b0154170 0x7f67b00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67b0154170 0x7f67b00c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41688/0 (socket says 172.21.15.92:41688) 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 -- 172.21.15.92:0/71115313 learned_addr learned my addr 172.21.15.92:0/71115313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67af7fe700 1 --2- 172.21.15.92:0/71115313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154550 0x7f67b00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 -- 172.21.15.92:0/71115313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154550 msgr2=0x7f67b00c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 --2- 172.21.15.92:0/71115313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154550 0x7f67b00c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 -- 172.21.15.92:0/71115313 >> v1:172.21.15.158:6789/0 conn(0x7f67b00696a0 legacy=0x7f67b0157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 -- 172.21.15.92:0/71115313 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00c0010 con 0x7f67b0154170 2023-05-16T07:02:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67affff700 1 --2- 172.21.15.92:0/71115313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154170 0x7f67b00c5c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f67a4006fd0 tx=0x7f67a40090f0).ready entity=mon.0 client_cookie=ec00ad2e90d49f28 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67aeffd700 1 -- 172.21.15.92:0/71115313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400b580 con 0x7f67b0154170 2023-05-16T07:02:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67aeffd700 1 -- 172.21.15.92:0/71115313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400b6e0 con 0x7f67b0154170 2023-05-16T07:02:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.847+0000 7f67aeffd700 1 -- 172.21.15.92:0/71115313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400ba80 con 0x7f67b0154170 2023-05-16T07:02:57.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 -- 172.21.15.92:0/71115313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154170 msgr2=0x7f67b00c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/71115313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154170 0x7f67b00c5c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f67a4006fd0 tx=0x7f67a40090f0).stop 2023-05-16T07:02:57.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 -- 172.21.15.92:0/71115313 shutdown_connections 2023-05-16T07:02:57.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/71115313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154170 0x7f67b00c5c70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/71115313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154550 0x7f67b00c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 -- 172.21.15.92:0/71115313 >> 172.21.15.92:0/71115313 conn(0x7f67b00cb700 msgr2=0x7f67b00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 -- 172.21.15.92:0/71115313 shutdown_connections 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.848+0000 7f67b6a4a700 1 -- 172.21.15.92:0/71115313 wait complete. 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.849+0000 7f67b6a4a700 1 Processor -- start 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.849+0000 7f67b6a4a700 1 -- start start 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b6a4a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f67b00696a0 0x7f67b0167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b6a4a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154170 0x7f67b0158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b6a4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 0x7f67b014e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b6a4a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0165560 con 0x7f67b0154550 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b6a4a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01649b0 con 0x7f67b00696a0 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b6a4a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0164c30 con 0x7f67b0154170 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 0x7f67b014e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 0x7f67b014e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41696/0 (socket says 172.21.15.92:41696) 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 -- 172.21.15.92:0/3036308083 learned_addr learned my addr 172.21.15.92:0/3036308083 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67af7fe700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154170 0x7f67b0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67affff700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f67b00696a0 0x7f67b0167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 -- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154170 msgr2=0x7f67b0158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154170 0x7f67b0158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 -- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f67b00696a0 msgr2=0x7f67b0167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f67b00696a0 0x7f67b0167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.850+0000 7f67b4fe7700 1 -- 172.21.15.92:0/3036308083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4009370 con 0x7f67b0154550 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.851+0000 7f67b4fe7700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 0x7f67b014e470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f67a8002230 tx=0x7f67a80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.851+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800e4a0 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.851+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800e600 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.851+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b0164f10 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.851+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800e980 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.851+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b0158760 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.852+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b00666c0 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.852+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f67a8019030 con 0x7f67b0154550 2023-05-16T07:02:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.852+0000 7f6797fff700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f679c05ed90 0x7f679c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.852+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f67a8079080 con 0x7f67b0154550 2023-05-16T07:02:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.852+0000 7f67affff700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f679c05ed90 0x7f679c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.853+0000 7f67affff700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f679c05ed90 0x7f679c061250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f67a4009fd0 tx=0x7f67a4017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:57.856+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f67a80488f0 con 0x7f67b0154550 2023-05-16T07:02:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.061+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67b006c350 con 0x7f67b0154550 2023-05-16T07:02:58.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.061+0000 7f6797fff700 1 -- 172.21.15.92:0/3036308083 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f67a804a6d0 con 0x7f67b0154550 2023-05-16T07:02:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:58.065 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f679c05ed90 msgr2=0x7f679c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f679c05ed90 0x7f679c061250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f67a4009fd0 tx=0x7f67a4017000).stop 2023-05-16T07:02:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 msgr2=0x7f67b014e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 0x7f67b014e470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f67a8002230 tx=0x7f67a80015e0).stop 2023-05-16T07:02:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 shutdown_connections 2023-05-16T07:02:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f679c05ed90 0x7f679c061250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67b0154550 0x7f67b014e470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f67b00696a0 0x7f67b0167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 --2- 172.21.15.92:0/3036308083 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f67b0154170 0x7f67b0158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 >> 172.21.15.92:0/3036308083 conn(0x7f67b00cb700 msgr2=0x7f67b00abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 shutdown_connections 2023-05-16T07:02:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.063+0000 7f67b6a4a700 1 -- 172.21.15.92:0/3036308083 wait complete. 2023-05-16T07:02:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:58.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:02:58.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.185+0000 7fc2594fe700 1 Processor -- start 2023-05-16T07:02:58.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.185+0000 7fc2594fe700 1 -- start start 2023-05-16T07:02:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc2594fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc254154170 0x7fc2540c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc2594fe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540c0660 con 0x7fc254069db0 2023-05-16T07:02:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc2594fe700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540bfd30 con 0x7fc254154550 2023-05-16T07:02:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc2594fe700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2540c0010 con 0x7fc254154170 2023-05-16T07:02:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc2537fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc254069db0 0x7fc2540c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52448/0 (socket says 172.21.15.92:52448) 2023-05-16T07:02:58.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc2537fe700 1 -- 172.21.15.92:0/291406304 learned_addr learned my addr 172.21.15.92:0/291406304 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:58.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc252ffd700 1 --2- 172.21.15.92:0/291406304 >> v2:172.21.15.158:3300/0 conn(0x7fc254154170 0x7fc2540c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc251ffb700 1 -- 172.21.15.92:0/291406304 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829352329 0 0) 0x7fc2540c0660 con 0x7fc254069db0 2023-05-16T07:02:58.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.186+0000 7fc251ffb700 1 -- 172.21.15.92:0/291406304 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2400033f0 con 0x7fc254069db0 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc252ffd700 1 -- 172.21.15.92:0/291406304 >> v1:172.21.15.158:6789/0 conn(0x7fc254154550 legacy=0x7fc2540696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc252ffd700 1 -- 172.21.15.92:0/291406304 >> v1:172.21.15.92:6789/0 conn(0x7fc254069db0 legacy=0x7fc2540c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc252ffd700 1 -- 172.21.15.92:0/291406304 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540c02f0 con 0x7fc254154170 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc252ffd700 1 --2- 172.21.15.92:0/291406304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254154170 0x7fc2540c5c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc244009e60 tx=0x7fc244009780).ready entity=mon.1 client_cookie=bba49364f50217dc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc251ffb700 1 -- 172.21.15.92:0/291406304 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24400e070 con 0x7fc254154170 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc251ffb700 1 -- 172.21.15.92:0/291406304 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24400bb60 con 0x7fc254154170 2023-05-16T07:02:58.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc251ffb700 1 -- 172.21.15.92:0/291406304 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244012410 con 0x7fc254154170 2023-05-16T07:02:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 -- 172.21.15.92:0/291406304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254154170 msgr2=0x7fc2540c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 --2- 172.21.15.92:0/291406304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254154170 0x7fc2540c5c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc244009e60 tx=0x7fc244009780).stop 2023-05-16T07:02:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 -- 172.21.15.92:0/291406304 shutdown_connections 2023-05-16T07:02:58.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 --2- 172.21.15.92:0/291406304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254154170 0x7fc2540c5c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 -- 172.21.15.92:0/291406304 >> 172.21.15.92:0/291406304 conn(0x7fc2540cb700 msgr2=0x7fc2540c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 -- 172.21.15.92:0/291406304 shutdown_connections 2023-05-16T07:02:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.187+0000 7fc2594fe700 1 -- 172.21.15.92:0/291406304 wait complete. 2023-05-16T07:02:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 Processor -- start 2023-05-16T07:02:58.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 -- start start 2023-05-16T07:02:58.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254069db0 0x7fc2540c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc254154170 0x7fc2540c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 0x7fc254154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c9040 con 0x7fc254154550 2023-05-16T07:02:58.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c92e0 con 0x7fc254069db0 2023-05-16T07:02:58.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2594fe700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c95c0 con 0x7fc254154170 2023-05-16T07:02:58.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 0x7fc254154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 0x7fc254154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41708/0 (socket says 172.21.15.92:41708) 2023-05-16T07:02:58.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.189+0000 7fc2537fe700 1 -- 172.21.15.92:0/2177126490 learned_addr learned my addr 172.21.15.92:0/2177126490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:58.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc252ffd700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254069db0 0x7fc2540c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2527fc700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc254154170 0x7fc2540c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2537fe700 1 -- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc254154170 msgr2=0x7fc2540c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2537fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc254154170 0x7fc2540c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2537fe700 1 -- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254069db0 msgr2=0x7fc2540c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2537fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254069db0 0x7fc2540c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2537fe700 1 -- 172.21.15.92:0/2177126490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc244009a20 con 0x7fc254154550 2023-05-16T07:02:58.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2537fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 0x7fc254154960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fc24c0010e0 tx=0x7fc24c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0070d0 con 0x7fc254154550 2023-05-16T07:02:58.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c007230 con 0x7fc254154550 2023-05-16T07:02:58.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2540c98a0 con 0x7fc254154550 2023-05-16T07:02:58.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0075b0 con 0x7fc254154550 2023-05-16T07:02:58.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.190+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2540c30f0 con 0x7fc254154550 2023-05-16T07:02:58.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.191+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2540666c0 con 0x7fc254154550 2023-05-16T07:02:58.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.191+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc24c01a060 con 0x7fc254154550 2023-05-16T07:02:58.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.192+0000 7fc24affd700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc23405ed90 0x7fc234061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.192+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc24c0793a0 con 0x7fc254154550 2023-05-16T07:02:58.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.192+0000 7fc252ffd700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc23405ed90 0x7fc234061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.192+0000 7fc252ffd700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc23405ed90 0x7fc234061250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fc244009e60 tx=0x7fc244009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.196+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc24c04a940 con 0x7fc254154550 2023-05-16T07:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.403+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2540c9ae0 con 0x7fc254154550 2023-05-16T07:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.404+0000 7fc24affd700 1 -- 172.21.15.92:0/2177126490 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7fc24c018020 con 0x7fc254154550 2023-05-16T07:02:58.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:58.408 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/837016991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":837016991},{"type":"v1","addr":"172.21.15.158:6835","nonce":837016991}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1353271689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1353271689},{"type":"v1","addr":"172.21.15.158:6837","nonce":1353271689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1236516128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1236516128},{"type":"v1","addr":"172.21.15.92:6833","nonce":1236516128}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:02:26.244736+0000","modified":"2023-05-16T07:02:28.250832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/240924417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":240924417},{"type":"v1","addr":"172.21.15.92:6835","nonce":240924417}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-05-16T07:02:58.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc23405ed90 msgr2=0x7fc234061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc23405ed90 0x7fc234061250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fc244009e60 tx=0x7fc244009780).stop 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 msgr2=0x7fc254154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 0x7fc254154960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fc24c0010e0 tx=0x7fc24c007c50).stop 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 shutdown_connections 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc23405ed90 0x7fc234061250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254154550 0x7fc254154960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc254069db0 0x7fc2540c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 --2- 172.21.15.92:0/2177126490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc254154170 0x7fc2540c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 >> 172.21.15.92:0/2177126490 conn(0x7fc2540cb700 msgr2=0x7fc2540ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 shutdown_connections 2023-05-16T07:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.405+0000 7fc2594fe700 1 -- 172.21.15.92:0/2177126490 wait complete. 2023-05-16T07:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2023-05-16T07:02:58.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:02:58.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.528+0000 7fdfab155700 1 Processor -- start 2023-05-16T07:02:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfab155700 1 -- start start 2023-05-16T07:02:58.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfab155700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfa4154170 0x7fdfa40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfab155700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4069db0 0x7fdfa40c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfab155700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa40c0660 con 0x7fdfa4154550 2023-05-16T07:02:58.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfab155700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa40c1b00 con 0x7fdfa4154170 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfab155700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40bfd30 con 0x7fdfa4069db0 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.529+0000 7fdfa8ef1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfa4154170 0x7fdfa40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfa4154170 0x7fdfa40c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41712/0 (socket says 172.21.15.92:41712) 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 -- 172.21.15.92:0/3504376893 learned_addr learned my addr 172.21.15.92:0/3504376893 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa96f2700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4069db0 0x7fdfa40c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 -- 172.21.15.92:0/3504376893 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4069db0 msgr2=0x7fdfa40c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4069db0 0x7fdfa40c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 -- 172.21.15.92:0/3504376893 >> v1:172.21.15.158:6789/0 conn(0x7fdfa4154550 legacy=0x7fdfa40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 -- 172.21.15.92:0/3504376893 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40c0010 con 0x7fdfa4154170 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa96f2700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4069db0 0x7fdfa40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa8ef1700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa40c5c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fdf94006fd0 tx=0x7fdf940090f0).ready entity=mon.0 client_cookie=aaf792129b526d68 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa37fe700 1 -- 172.21.15.92:0/3504376893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400b600 con 0x7fdfa4154170 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa37fe700 1 -- 172.21.15.92:0/3504376893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9400b760 con 0x7fdfa4154170 2023-05-16T07:02:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.530+0000 7fdfa37fe700 1 -- 172.21.15.92:0/3504376893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400bb00 con 0x7fdfa4154170 2023-05-16T07:02:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 -- 172.21.15.92:0/3504376893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 msgr2=0x7fdfa40c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa40c5c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fdf94006fd0 tx=0x7fdf940090f0).stop 2023-05-16T07:02:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 -- 172.21.15.92:0/3504376893 shutdown_connections 2023-05-16T07:02:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa40c5c70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 --2- 172.21.15.92:0/3504376893 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4069db0 0x7fdfa40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 -- 172.21.15.92:0/3504376893 >> 172.21.15.92:0/3504376893 conn(0x7fdfa40cb700 msgr2=0x7fdfa40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 -- 172.21.15.92:0/3504376893 shutdown_connections 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.531+0000 7fdfab155700 1 -- 172.21.15.92:0/3504376893 wait complete. 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.532+0000 7fdfab155700 1 Processor -- start 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.532+0000 7fdfab155700 1 -- start start 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfa4069db0 0x7fdfa41586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa4152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4154550 0x7fdfa40acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa414fed0 con 0x7fdfa4154170 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa41522d0 con 0x7fdfa4069db0 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa41516c0 con 0x7fdfa4154550 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa4152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa4152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41720/0 (socket says 172.21.15.92:41720) 2023-05-16T07:02:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 -- 172.21.15.92:0/83445258 learned_addr learned my addr 172.21.15.92:0/83445258 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa8ef1700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfa4069db0 0x7fdfa41586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa96f2700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4154550 0x7fdfa40acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 -- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4154550 msgr2=0x7fdfa40acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4154550 0x7fdfa40acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 -- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfa4069db0 msgr2=0x7fdfa41586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfa4069db0 0x7fdfa41586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 -- 172.21.15.92:0/83445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94009370 con 0x7fdfa4154170 2023-05-16T07:02:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa3fff700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa4152760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fdf8c00b810 tx=0x7fdf8c0092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00d500 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c00d680 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa41519a0 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00da40 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.533+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa4151f60 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.534+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa40666c0 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.535+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdf8c018030 con 0x7fdfa4154170 2023-05-16T07:02:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.535+0000 7fdfa0ff9700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf8405ed90 0x7fdf84061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.535+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdf8c079390 con 0x7fdfa4154170 2023-05-16T07:02:58.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.535+0000 7fdfa8ef1700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf8405ed90 0x7fdf84061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.536+0000 7fdfa8ef1700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf8405ed90 0x7fdf84061250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fdf940091e0 tx=0x7fdf94017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.539+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdf8c0478f0 con 0x7fdfa4154170 2023-05-16T07:02:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.701+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfa4152460 con 0x7fdfa4154170 2023-05-16T07:02:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.702+0000 7fdfa0ff9700 1 -- 172.21.15.92:0/83445258 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fdf8c04e410 con 0x7fdfa4154170 2023-05-16T07:02:58.706 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:02:58.706 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:02:27.246586+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:02:24.569243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-16T07:02:25.217735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:02:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.703+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf8405ed90 msgr2=0x7fdf84061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.703+0000 7fdfab155700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf8405ed90 0x7fdf84061250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fdf940091e0 tx=0x7fdf94017000).stop 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.703+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 msgr2=0x7fdfa4152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.703+0000 7fdfab155700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa4152760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fdf8c00b810 tx=0x7fdf8c0092d0).stop 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.703+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 shutdown_connections 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.703+0000 7fdfab155700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf8405ed90 0x7fdf84061250 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.704+0000 7fdfab155700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfa4154550 0x7fdfa40acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.704+0000 7fdfab155700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfa4069db0 0x7fdfa41586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.704+0000 7fdfab155700 1 --2- 172.21.15.92:0/83445258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfa4154170 0x7fdfa4152760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.704+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 >> 172.21.15.92:0/83445258 conn(0x7fdfa40cb700 msgr2=0x7fdfa40afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.704+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 shutdown_connections 2023-05-16T07:02:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.704+0000 7fdfab155700 1 -- 172.21.15.92:0/83445258 wait complete. 2023-05-16T07:02:58.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:02:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.826+0000 7fdf3f97e700 1 Processor -- start 2023-05-16T07:02:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.826+0000 7fdf3f97e700 1 -- start start 2023-05-16T07:02:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3f97e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdf38154550 0x7fdf380696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3f97e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf380c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3f97e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf380c0660 con 0x7fdf38154170 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3f97e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3806a000 con 0x7fdf38154550 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3f97e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380bfd30 con 0x7fdf38069b90 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3d71a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdf38154170 0x7fdf380c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52474/0 (socket says 172.21.15.92:52474) 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3d71a700 1 -- 172.21.15.92:0/2326982748 learned_addr learned my addr 172.21.15.92:0/2326982748 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3df1b700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf380c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.827+0000 7fdf3cf19700 1 --2- 172.21.15.92:0/2326982748 >> v2:172.21.15.158:3300/0 conn(0x7fdf38154550 0x7fdf380696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160229100 0 0) 0x7fdf380c0660 con 0x7fdf38154170 2023-05-16T07:02:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf200033f0 con 0x7fdf38154170 2023-05-16T07:02:58.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1814237005 0 0) 0x7fdf200033f0 con 0x7fdf38154170 2023-05-16T07:02:58.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 msgr2=0x7fdf380c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:58.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf380c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 >> v2:172.21.15.158:3300/0 conn(0x7fdf38154550 msgr2=0x7fdf380696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:58.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 --2- 172.21.15.92:0/2326982748 >> v2:172.21.15.158:3300/0 conn(0x7fdf38154550 0x7fdf380696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380c0010 con 0x7fdf38154170 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fdf280017f0 con 0x7fdf38154170 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3cf19700 1 --2- 172.21.15.92:0/2326982748 >> v2:172.21.15.158:3300/0 conn(0x7fdf38154550 0x7fdf380696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf380c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 >> v1:172.21.15.92:6789/0 conn(0x7fdf38154170 legacy=0x7fdf380c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf200038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf20003dd0 0x7fdf20004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf20004b80 0x7fdf20005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3806a000 con 0x7fdf20004b80 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380bfd30 con 0x7fdf20003dd0 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2000b8e0 con 0x7fdf38069b90 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf20004b80 0x7fdf20005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3cf19700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf20003dd0 0x7fdf20004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3d71a700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf200038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 -- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 msgr2=0x7fdf200038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf38069b90 0x7fdf200038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 -- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf20003dd0 msgr2=0x7fdf20004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf20003dd0 0x7fdf20004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 -- 172.21.15.92:0/2326982748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300092a0 con 0x7fdf20004b80 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf3df1b700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf20004b80 0x7fdf20005000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fdf30009c70 tx=0x7fdf30009060).ready entity=mon.0 client_cookie=85f8ec4a9ae0c200 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.828+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300095c0 con 0x7fdf20004b80 2023-05-16T07:02:58.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3000e690 con 0x7fdf20004b80 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf37fff700 1 -- 172.21.15.92:0/2326982748 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000ea10 con 0x7fdf20004b80 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3f97e700 1 -- 172.21.15.92:0/2326982748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf20004b80 msgr2=0x7fdf20005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/2326982748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf20004b80 0x7fdf20005000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fdf30009c70 tx=0x7fdf30009060).stop 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3df1b700 1 -- 172.21.15.92:0/2326982748 reap_dead start 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3f97e700 1 -- 172.21.15.92:0/2326982748 shutdown_connections 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3f97e700 1 -- 172.21.15.92:0/2326982748 >> 172.21.15.92:0/2326982748 conn(0x7fdf380cb700 msgr2=0x7fdf380c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3f97e700 1 -- 172.21.15.92:0/2326982748 shutdown_connections 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.829+0000 7fdf3f97e700 1 -- 172.21.15.92:0/2326982748 wait complete. 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.830+0000 7fdf3f97e700 1 Processor -- start 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.830+0000 7fdf3f97e700 1 -- start start 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3f97e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf380a9f20 0x7fdf380aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3f97e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 0x7fdf380aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3f97e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf380af1e0 0x7fdf380c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3f97e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf381492a0 con 0x7fdf380c3a30 2023-05-16T07:02:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3f97e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c35a0 con 0x7fdf380af1e0 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3f97e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380af660 con 0x7fdf380a9f20 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 0x7fdf380aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 0x7fdf380aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41738/0 (socket says 172.21.15.92:41738) 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 -- 172.21.15.92:0/1923286446 learned_addr learned my addr 172.21.15.92:0/1923286446 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 -- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf380a9f20 msgr2=0x7fdf380aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf380a9f20 0x7fdf380aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 -- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf380af1e0 msgr2=0x7fdf380c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf380af1e0 0x7fdf380c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 -- 172.21.15.92:0/1923286446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380c2990 con 0x7fdf380c3a30 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.831+0000 7fdf3cf19700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 0x7fdf380aecf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fdf2c011ad0 tx=0x7fdf2c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.832+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00fa10 con 0x7fdf380c3a30 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.832+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00fb70 con 0x7fdf380c3a30 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.832+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c014440 con 0x7fdf380c3a30 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.832+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380c2c70 con 0x7fdf380c3a30 2023-05-16T07:02:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.832+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf380c3230 con 0x7fdf380c3a30 2023-05-16T07:02:58.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.832+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf380666c0 con 0x7fdf380c3a30 2023-05-16T07:02:58.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.833+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdf2c024070 con 0x7fdf380c3a30 2023-05-16T07:02:58.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.833+0000 7fdf357fa700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2405ed90 0x7fdf24061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:58.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.833+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdf2c07a6b0 con 0x7fdf380c3a30 2023-05-16T07:02:58.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.833+0000 7fdf3d71a700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2405ed90 0x7fdf24061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:58.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.834+0000 7fdf3d71a700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2405ed90 0x7fdf24061250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fdf28007be0 tx=0x7fdf28002210).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:58.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:58.837+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdf2c04bc50 con 0x7fdf380c3a30 2023-05-16T07:02:59.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.042+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdf380c3730 con 0x7fdf380c3a30 2023-05-16T07:02:59.535 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:02:59.531+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:02:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.537+0000 7fdf357fa700 1 -- 172.21.15.92:0/1923286446 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdf2c018020 con 0x7fdf380c3a30 2023-05-16T07:02:59.543 INFO:tasks.ceph.mds.c.smithi092.stderr: -1500> 2023-05-16T07:02:28.878+0000 7f1859941700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:02:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2405ed90 msgr2=0x7fdf24061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2405ed90 0x7fdf24061250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fdf28007be0 tx=0x7fdf28002210).stop 2023-05-16T07:02:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 msgr2=0x7fdf380aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 0x7fdf380aecf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fdf2c011ad0 tx=0x7fdf2c009060).stop 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 shutdown_connections 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2405ed90 0x7fdf24061250 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf380a9f20 0x7fdf380aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf380af1e0 0x7fdf380c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 --2- 172.21.15.92:0/1923286446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf380c3a30 0x7fdf380aecf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 >> 172.21.15.92:0/1923286446 conn(0x7fdf380cb700 msgr2=0x7fdf380c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:59.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 shutdown_connections 2023-05-16T07:02:59.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.540+0000 7fdf3f97e700 1 -- 172.21.15.92:0/1923286446 wait complete. 2023-05-16T07:02:59.546 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:02:59.557 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:02:59.565 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:02:59.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.668+0000 7f0baae39700 1 Processor -- start 2023-05-16T07:02:59.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.668+0000 7f0baae39700 1 -- start start 2023-05-16T07:02:59.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0baae39700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ba4068870 0x7f0ba40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0baae39700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba40c94d0 con 0x7f0ba4154470 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0baae39700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba40cb7e0 con 0x7f0ba4154090 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0baae39700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba40c88c0 con 0x7f0ba4068870 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0ba3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0ba4154470 0x7f0ba4068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52506/0 (socket says 172.21.15.92:52506) 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0ba3fff700 1 -- 172.21.15.92:0/758824087 learned_addr learned my addr 172.21.15.92:0/758824087 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.669+0000 7f0ba93d6700 1 --2- 172.21.15.92:0/758824087 >> v2:172.21.15.158:3300/0 conn(0x7f0ba4068870 0x7f0ba40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907618320 0 0) 0x7f0ba40c94d0 con 0x7f0ba4154470 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b900033f0 con 0x7f0ba4154470 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340361460 0 0) 0x7f0b900033f0 con 0x7f0ba4154470 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 >> v2:172.21.15.158:3300/0 conn(0x7f0ba4068870 msgr2=0x7f0ba40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 --2- 172.21.15.92:0/758824087 >> v2:172.21.15.158:3300/0 conn(0x7f0ba4068870 0x7f0ba40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 >> v1:172.21.15.158:6789/0 conn(0x7f0ba4154090 legacy=0x7f0ba40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40c8ba0 con 0x7f0ba4154470 2023-05-16T07:02:59.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f0b8c0017f0 con 0x7f0ba4154470 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 >> v1:172.21.15.92:6789/0 conn(0x7f0ba4154470 legacy=0x7f0ba4068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b900038e0 0x7f0b90003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b90004690 0x7f0b90004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b90005000 0x7f0b9000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40c88c0 con 0x7f0b90004690 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9000beb0 con 0x7f0b900038e0 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9000c070 con 0x7f0b90005000 2023-05-16T07:02:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b90004690 0x7f0b90004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba8bd5700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b900038e0 0x7f0b90003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba93d6700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b90005000 0x7f0b9000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 -- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b90005000 msgr2=0x7f0b9000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b90005000 0x7f0b9000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 -- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b900038e0 msgr2=0x7f0b90003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b900038e0 0x7f0b90003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 -- 172.21.15.92:0/758824087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900033f0 con 0x7f0b90004690 2023-05-16T07:02:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba93d6700 1 -- 172.21.15.92:0/758824087 reap_dead start 2023-05-16T07:02:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b90004690 0x7f0b90004b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0b8c00c5f0 tx=0x7f0b8c001d10).ready entity=mon.0 client_cookie=a0cf118f71d9f279 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.670+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c001f40 con 0x7f0b90004690 2023-05-16T07:02:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c00cdf0 con 0x7f0b90004690 2023-05-16T07:02:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0ba37fe700 1 -- 172.21.15.92:0/758824087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c005d50 con 0x7f0b90004690 2023-05-16T07:02:59.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 -- 172.21.15.92:0/758824087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b90004690 msgr2=0x7f0b90004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b90004690 0x7f0b90004b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0b8c00c5f0 tx=0x7f0b8c001d10).stop 2023-05-16T07:02:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 -- 172.21.15.92:0/758824087 shutdown_connections 2023-05-16T07:02:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 --2- 172.21.15.92:0/758824087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b90004690 0x7f0b90004b10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 -- 172.21.15.92:0/758824087 >> 172.21.15.92:0/758824087 conn(0x7f0ba4067920 msgr2=0x7f0ba40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:02:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 -- 172.21.15.92:0/758824087 shutdown_connections 2023-05-16T07:02:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.671+0000 7f0baae39700 1 -- 172.21.15.92:0/758824087 wait complete. 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.672+0000 7f0baae39700 1 Processor -- start 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.672+0000 7f0baae39700 1 -- start start 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0baae39700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b90004690 0x7f0ba40c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0baae39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 0x7f0ba413c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0baae39700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ba4164f00 0x7f0ba4157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0baae39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba413c650 con 0x7f0ba40c4300 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0baae39700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba414fd20 con 0x7f0b90004690 2023-05-16T07:02:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0baae39700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40c3e70 con 0x7f0ba4164f00 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 0x7f0ba413c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 0x7f0ba413c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41768/0 (socket says 172.21.15.92:41768) 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 -- 172.21.15.92:0/2557510654 learned_addr learned my addr 172.21.15.92:0/2557510654 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba8bd5700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b90004690 0x7f0ba40c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba93d6700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ba4164f00 0x7f0ba4157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 -- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ba4164f00 msgr2=0x7f0ba4157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ba4164f00 0x7f0ba4157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 -- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b90004690 msgr2=0x7f0ba40c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b90004690 0x7f0ba40c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.673+0000 7f0ba3fff700 1 -- 172.21.15.92:0/2557510654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40c3260 con 0x7f0ba40c4300 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.674+0000 7f0ba3fff700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 0x7f0ba413c160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0b8c00cbe0 tx=0x7f0b8c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.674+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0017f0 con 0x7f0ba40c4300 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.674+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c001f40 con 0x7f0ba40c4300 2023-05-16T07:02:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.674+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba40c3540 con 0x7f0ba40c4300 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.674+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c004c80 con 0x7f0ba40c4300 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.674+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba40c3b00 con 0x7f0ba40c4300 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.675+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba4066730 con 0x7f0ba40c4300 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.675+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0b8c01d070 con 0x7f0ba40c4300 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.675+0000 7f0ba0ff9700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b8405ed90 0x7f0b84061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.675+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0b8c07a910 con 0x7f0ba40c4300 2023-05-16T07:02:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.676+0000 7f0ba8bd5700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b8405ed90 0x7f0b84061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:02:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.676+0000 7f0ba8bd5700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b8405ed90 0x7f0b84061250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f0ba40c4a40 tx=0x7f0b94007bb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:02:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.680+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0b8c04be60 con 0x7f0ba40c4300 2023-05-16T07:02:59.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:02:59.884+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0ba40c8e80 con 0x7f0ba40c4300 2023-05-16T07:03:00.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.540+0000 7f0ba0ff9700 1 -- 172.21.15.92:0/2557510654 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b8c024020 con 0x7f0ba40c4300 2023-05-16T07:03:00.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b8405ed90 msgr2=0x7f0b84061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:00.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b8405ed90 0x7f0b84061250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f0ba40c4a40 tx=0x7f0b94007bb0).stop 2023-05-16T07:03:00.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 msgr2=0x7f0ba413c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:00.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 0x7f0ba413c160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0b8c00cbe0 tx=0x7f0b8c015000).stop 2023-05-16T07:03:00.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 shutdown_connections 2023-05-16T07:03:00.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b8405ed90 0x7f0b84061250 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ba4164f00 0x7f0ba4157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b90004690 0x7f0ba40c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 --2- 172.21.15.92:0/2557510654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba40c4300 0x7f0ba413c160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 >> 172.21.15.92:0/2557510654 conn(0x7f0ba4067920 msgr2=0x7f0ba40c1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 shutdown_connections 2023-05-16T07:03:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.543+0000 7f0baae39700 1 -- 172.21.15.92:0/2557510654 wait complete. 2023-05-16T07:03:00.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:03:00.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.715+0000 7f243f59e700 1 Processor -- start 2023-05-16T07:03:00.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.716+0000 7f243f59e700 1 -- start start 2023-05-16T07:03:00.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.716+0000 7f243f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24400bdf20 0x7f24400698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:00.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.716+0000 7f243f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2440153f90 0x7f2440067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.716+0000 7f243f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244005e760 con 0x7f2440153bb0 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.716+0000 7f243f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2440069ef0 con 0x7f2440153f90 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.716+0000 7f243f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400bf540 con 0x7f24400bdf20 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2440153f90 0x7f2440067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2440153f90 0x7f2440067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41782/0 (socket says 172.21.15.92:41782) 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 -- 172.21.15.92:0/1079425510 learned_addr learned my addr 172.21.15.92:0/1079425510 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:00.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2440153bb0 0x7f24400bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52526/0 (socket says 172.21.15.92:52526) 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243dd9b700 1 --2- 172.21.15.92:0/1079425510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24400bdf20 0x7f24400698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 -- 172.21.15.92:0/1079425510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24400bdf20 msgr2=0x7f24400698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 --2- 172.21.15.92:0/1079425510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24400bdf20 0x7f24400698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 -- 172.21.15.92:0/1079425510 >> v1:172.21.15.92:6789/0 conn(0x7f2440153bb0 legacy=0x7f24400bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 -- 172.21.15.92:0/1079425510 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440068450 con 0x7f2440153f90 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243d59a700 1 -- 172.21.15.92:0/1079425510 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155403621 0 0) 0x7f244005e760 con 0x7f2440153bb0 2023-05-16T07:03:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243ed9d700 1 --2- 172.21.15.92:0/1079425510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2440153f90 0x7f2440067f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f2438006fd0 tx=0x7f24380090f0).ready entity=mon.0 client_cookie=a78cdc1a035ba692 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:00.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243d59a700 1 -- 172.21.15.92:0/1079425510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800b600 con 0x7f2440153f90 2023-05-16T07:03:00.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243d59a700 1 -- 172.21.15.92:0/1079425510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800b760 con 0x7f2440153f90 2023-05-16T07:03:00.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.717+0000 7f243d59a700 1 -- 172.21.15.92:0/1079425510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800bae0 con 0x7f2440153f90 2023-05-16T07:03:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 -- 172.21.15.92:0/1079425510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2440153f90 msgr2=0x7f2440067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 --2- 172.21.15.92:0/1079425510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2440153f90 0x7f2440067f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f2438006fd0 tx=0x7f24380090f0).stop 2023-05-16T07:03:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 -- 172.21.15.92:0/1079425510 shutdown_connections 2023-05-16T07:03:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 --2- 172.21.15.92:0/1079425510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2440153f90 0x7f2440067f60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 --2- 172.21.15.92:0/1079425510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24400bdf20 0x7f24400698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 -- 172.21.15.92:0/1079425510 >> 172.21.15.92:0/1079425510 conn(0x7f2440062d80 msgr2=0x7f244005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 -- 172.21.15.92:0/1079425510 shutdown_connections 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.718+0000 7f243f59e700 1 -- 172.21.15.92:0/1079425510 wait complete. 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.719+0000 7f243f59e700 1 Processor -- start 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.719+0000 7f243f59e700 1 -- start start 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 0x7f24400c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2440153bb0 0x7f2440152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2440153f90 0x7f24400adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400c2cc0 con 0x7f24400bdf20 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401545d0 con 0x7f2440153bb0 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440154860 con 0x7f2440153f90 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 0x7f24400c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 0x7f24400c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41784/0 (socket says 172.21.15.92:41784) 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 -- 172.21.15.92:0/2209648488 learned_addr learned my addr 172.21.15.92:0/2209648488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243ed9d700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2440153f90 0x7f24400adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243dd9b700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2440153bb0 0x7f2440152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 -- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2440153f90 msgr2=0x7f24400adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2440153f90 0x7f24400adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 -- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2440153bb0 msgr2=0x7f2440152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2440153bb0 0x7f2440152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 -- 172.21.15.92:0/2209648488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009370 con 0x7f24400bdf20 2023-05-16T07:03:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243e59c700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 0x7f24400c1c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f2430001560 tx=0x7f24300016c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005c70 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430005dd0 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400c2160 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430007320 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.720+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24400c2780 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.721+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400c2b00 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.721+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2430010030 con 0x7f24400bdf20 2023-05-16T07:03:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.722+0000 7f24367fc700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f242005ed90 0x7f2420061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.722+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2430079b80 con 0x7f24400bdf20 2023-05-16T07:03:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.722+0000 7f243dd9b700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f242005ed90 0x7f2420061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.722+0000 7f243dd9b700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f242005ed90 0x7f2420061250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f2428006fd0 tx=0x7f2428008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.726+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f243004b070 con 0x7f24400bdf20 2023-05-16T07:03:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:00.887+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2440047150 con 0x7f24400bdf20 2023-05-16T07:03:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.544+0000 7f24367fc700 1 -- 172.21.15.92:0/2209648488 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f243004e690 con 0x7f24400bdf20 2023-05-16T07:03:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f242005ed90 msgr2=0x7f2420061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f242005ed90 0x7f2420061250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f2428006fd0 tx=0x7f2428008040).stop 2023-05-16T07:03:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 msgr2=0x7f24400c1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 0x7f24400c1c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f2430001560 tx=0x7f24300016c0).stop 2023-05-16T07:03:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 shutdown_connections 2023-05-16T07:03:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f242005ed90 0x7f2420061250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2440153f90 0x7f24400adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400bdf20 0x7f24400c1c70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 --2- 172.21.15.92:0/2209648488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2440153bb0 0x7f2440152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 >> 172.21.15.92:0/2209648488 conn(0x7f2440062d80 msgr2=0x7f24400c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 shutdown_connections 2023-05-16T07:03:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.547+0000 7f243f59e700 1 -- 172.21.15.92:0/2209648488 wait complete. 2023-05-16T07:03:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:03:01.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:03:01.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.680+0000 7f6df3aec700 1 Processor -- start 2023-05-16T07:03:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.680+0000 7f6df3aec700 1 -- start start 2023-05-16T07:03:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.681+0000 7f6df3aec700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 0x7f6dec0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.681+0000 7f6df3aec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 0x7f6dec0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.681+0000 7f6df3aec700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dec0c94d0 con 0x7f6dec0aa150 2023-05-16T07:03:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.681+0000 7f6df3aec700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec0aa5c0 con 0x7f6dec154090 2023-05-16T07:03:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.681+0000 7f6df3aec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0cb7e0 con 0x7f6dec154470 2023-05-16T07:03:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 0x7f6dec0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1888700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 0x7f6dec0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1888700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 0x7f6dec0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:45428/0 (socket says 172.21.15.92:45428) 2023-05-16T07:03:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 0x7f6dec0a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:34366/0 (socket says 172.21.15.92:34366) 2023-05-16T07:03:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1888700 1 -- 172.21.15.92:0/3300717226 learned_addr learned my addr 172.21.15.92:0/3300717226 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df2089700 1 --1- 172.21.15.92:0/3300717226 >> v1:172.21.15.158:6789/0 conn(0x7f6dec0aa150 0x7f6dec068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:56142/0 (socket says 172.21.15.92:56142) 2023-05-16T07:03:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 -- 172.21.15.92:0/3300717226 >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 msgr2=0x7f6dec0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 --2- 172.21.15.92:0/3300717226 >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 0x7f6dec0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 -- 172.21.15.92:0/3300717226 >> v1:172.21.15.158:6789/0 conn(0x7f6dec0aa150 legacy=0x7f6dec068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 -- 172.21.15.92:0/3300717226 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0c88c0 con 0x7f6dec154470 2023-05-16T07:03:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1888700 1 --2- 172.21.15.92:0/3300717226 >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 0x7f6dec0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.682+0000 7f6df1087700 1 --2- 172.21.15.92:0/3300717226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 0x7f6dec0a9c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6de0009760 tx=0x7f6de0009060).ready entity=mon.2 client_cookie=55a1b2687f0a6a24 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df0886700 1 -- 172.21.15.92:0/3300717226 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000f510 con 0x7f6dec154470 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df0886700 1 -- 172.21.15.92:0/3300717226 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de000f670 con 0x7f6dec154470 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df0886700 1 -- 172.21.15.92:0/3300717226 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000f9f0 con 0x7f6dec154470 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 -- 172.21.15.92:0/3300717226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 msgr2=0x7f6dec0a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 --2- 172.21.15.92:0/3300717226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 0x7f6dec0a9c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6de0009760 tx=0x7f6de0009060).stop 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 -- 172.21.15.92:0/3300717226 shutdown_connections 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 --2- 172.21.15.92:0/3300717226 >> v2:172.21.15.158:3300/0 conn(0x7f6dec154090 0x7f6dec0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 --2- 172.21.15.92:0/3300717226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec154470 0x7f6dec0a9c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 -- 172.21.15.92:0/3300717226 >> 172.21.15.92:0/3300717226 conn(0x7f6dec067920 msgr2=0x7f6dec0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 -- 172.21.15.92:0/3300717226 shutdown_connections 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.683+0000 7f6df3aec700 1 -- 172.21.15.92:0/3300717226 wait complete. 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.684+0000 7f6df3aec700 1 Processor -- start 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.684+0000 7f6df3aec700 1 -- start start 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df3aec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec0aa150 0x7f6dec1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df3aec700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6dec154090 0x7f6dec152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df3aec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 0x7f6dec0ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df3aec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec150860 con 0x7f6dec154470 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df3aec700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec14fc50 con 0x7f6dec154090 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df3aec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec14ff30 con 0x7f6dec0aa150 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 0x7f6dec0ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 0x7f6dec0ab820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41800/0 (socket says 172.21.15.92:41800) 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 -- 172.21.15.92:0/1486802728 learned_addr learned my addr 172.21.15.92:0/1486802728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:01.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df1087700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6dec154090 0x7f6dec152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df1888700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec0aa150 0x7f6dec1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 -- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec0aa150 msgr2=0x7f6dec1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec0aa150 0x7f6dec1577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 -- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6dec154090 msgr2=0x7f6dec152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6dec154090 0x7f6dec152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 -- 172.21.15.92:0/1486802728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc009ae0 con 0x7f6dec154470 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df1087700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6dec154090 0x7f6dec152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:01.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.685+0000 7f6df2089700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 0x7f6dec0ab820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f6de400c410 tx=0x7f6de4007d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:01.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.686+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40072e0 con 0x7f6dec154470 2023-05-16T07:03:01.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.686+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4007440 con 0x7f6dec154470 2023-05-16T07:03:01.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.686+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de00092a0 con 0x7f6dec154470 2023-05-16T07:03:01.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.686+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40058f0 con 0x7f6dec154470 2023-05-16T07:03:01.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.686+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec1504f0 con 0x7f6dec154470 2023-05-16T07:03:01.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.687+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec066730 con 0x7f6dec154470 2023-05-16T07:03:01.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.687+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6de401a060 con 0x7f6dec154470 2023-05-16T07:03:01.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.687+0000 7f6de9ffb700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6dd805ed90 0x7f6dd8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:01.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.687+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5769+0+0 (secure 0 0 0) 0x7f6de4078c90 con 0x7f6dec154470 2023-05-16T07:03:01.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.688+0000 7f6df1888700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6dd805ed90 0x7f6dd8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:01.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.688+0000 7f6df1888700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6dd805ed90 0x7f6dd8061250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f6ddc009ab0 tx=0x7f6ddc00b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:01.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.691+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6de404a390 con 0x7f6dec154470 2023-05-16T07:03:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:01.855+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6dec1509a0 con 0x7f6dec154470 2023-05-16T07:03:02.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.546+0000 7f6de9ffb700 1 -- 172.21.15.92:0/1486802728 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6de4018070 con 0x7f6dec154470 2023-05-16T07:03:02.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.549+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6dd805ed90 msgr2=0x7f6dd8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:02.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.549+0000 7f6df3aec700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6dd805ed90 0x7f6dd8061250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f6ddc009ab0 tx=0x7f6ddc00b040).stop 2023-05-16T07:03:02.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.549+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 msgr2=0x7f6dec0ab820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:02.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.549+0000 7f6df3aec700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 0x7f6dec0ab820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f6de400c410 tx=0x7f6de4007d30).stop 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 shutdown_connections 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6dd805ed90 0x7f6dd8061250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6dec0aa150 0x7f6dec1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dec154470 0x7f6dec0ab820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 --2- 172.21.15.92:0/1486802728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6dec154090 0x7f6dec152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 >> 172.21.15.92:0/1486802728 conn(0x7f6dec067920 msgr2=0x7f6dec152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:02.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 shutdown_connections 2023-05-16T07:03:02.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.550+0000 7f6df3aec700 1 -- 172.21.15.92:0/1486802728 wait complete. 2023-05-16T07:03:02.556 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:03:02.568 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:03:02.568 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:03:02.569 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:03:02.569 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:03:02.569 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:03:02.569 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:03:02.570 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:03:02.570 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:03:02.570 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:03:02.570 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:03:02.570 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:03:02.571 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:03:02.571 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:03:02.571 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:03:02.571 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:03:02.571 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:03:02.577 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:03:02.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:03:02.579 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:03:02.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:03:02.581 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:03:02.581 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:03:02.581 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:03:02.582 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:03:02.620 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:03:02.621 DEBUG:teuthology.parallel:result is None 2023-05-16T07:03:02.622 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:03:02.623 DEBUG:teuthology.parallel:result is None 2023-05-16T07:03:02.623 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:03:02.624 DEBUG:teuthology.parallel:result is None 2023-05-16T07:03:02.624 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:03:02.625 DEBUG:teuthology.parallel:result is None 2023-05-16T07:03:02.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:03:02.693 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:03:02.694 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:03:02.706 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:03:02.711 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:03:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.750+0000 7f20388aa700 1 Processor -- start 2023-05-16T07:03:02.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.750+0000 7f20388aa700 1 -- start start 2023-05-16T07:03:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f20388aa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034153eb0 0x7f2034154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f20388aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2034068c70 0x7f2034066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f20388aa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20340bc9b0 con 0x7f2034065000 2023-05-16T07:03:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f20388aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20340bcc90 con 0x7f2034068c70 2023-05-16T07:03:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f20388aa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340bcf70 con 0x7f2034153eb0 2023-05-16T07:03:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2031d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2034068c70 0x7f2034066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2032d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2034065000 0x7f20340622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52578/0 (socket says 172.21.15.92:52578) 2023-05-16T07:03:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2031d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2034068c70 0x7f2034066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41832/0 (socket says 172.21.15.92:41832) 2023-05-16T07:03:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2032d9d700 1 -- 172.21.15.92:0/804483249 learned_addr learned my addr 172.21.15.92:0/804483249 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f203259c700 1 --2- 172.21.15.92:0/804483249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034153eb0 0x7f2034154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2031d9b700 1 -- 172.21.15.92:0/804483249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034153eb0 msgr2=0x7f2034154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2031d9b700 1 --2- 172.21.15.92:0/804483249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034153eb0 0x7f2034154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2031d9b700 1 -- 172.21.15.92:0/804483249 >> v1:172.21.15.92:6789/0 conn(0x7f2034065000 legacy=0x7f20340622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f203159a700 1 -- 172.21.15.92:0/804483249 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813891217 0 0) 0x7f20340bc9b0 con 0x7f2034065000 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.751+0000 7f2031d9b700 1 -- 172.21.15.92:0/804483249 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340bd250 con 0x7f2034068c70 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f2031d9b700 1 --2- 172.21.15.92:0/804483249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034068c70 0x7f2034066e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f201c009760 tx=0x7f201c009060).ready entity=mon.0 client_cookie=9e547983fe9c1a1b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f203159a700 1 -- 172.21.15.92:0/804483249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00b580 con 0x7f2034068c70 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f203159a700 1 -- 172.21.15.92:0/804483249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c00b6e0 con 0x7f2034068c70 2023-05-16T07:03:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f203159a700 1 -- 172.21.15.92:0/804483249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00ba60 con 0x7f2034068c70 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 -- 172.21.15.92:0/804483249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034068c70 msgr2=0x7f2034066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 --2- 172.21.15.92:0/804483249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034068c70 0x7f2034066e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f201c009760 tx=0x7f201c009060).stop 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 -- 172.21.15.92:0/804483249 shutdown_connections 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 --2- 172.21.15.92:0/804483249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034153eb0 0x7f2034154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 --2- 172.21.15.92:0/804483249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034068c70 0x7f2034066e50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 -- 172.21.15.92:0/804483249 >> 172.21.15.92:0/804483249 conn(0x7f20340c4430 msgr2=0x7f20340c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 -- 172.21.15.92:0/804483249 shutdown_connections 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.752+0000 7f20388aa700 1 -- 172.21.15.92:0/804483249 wait complete. 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.753+0000 7f20388aa700 1 Processor -- start 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.753+0000 7f20388aa700 1 -- start start 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f20388aa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034065000 0x7f2034163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f20388aa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2034153eb0 0x7f20340c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f20388aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 0x7f20340c2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f20388aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034164120 con 0x7f2034149d60 2023-05-16T07:03:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f20388aa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340c2820 con 0x7f2034153eb0 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f20388aa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340c0820 con 0x7f2034065000 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 0x7f20340c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 0x7f20340c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41838/0 (socket says 172.21.15.92:41838) 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 -- 172.21.15.92:0/1688832409 learned_addr learned my addr 172.21.15.92:0/1688832409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2031d9b700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2034153eb0 0x7f20340c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f203259c700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034065000 0x7f2034163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 -- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034065000 msgr2=0x7f2034163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034065000 0x7f2034163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 -- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2034153eb0 msgr2=0x7f20340c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2034153eb0 0x7f20340c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 -- 172.21.15.92:0/1688832409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c0092a0 con 0x7f2034149d60 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f203259c700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034065000 0x7f2034163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.754+0000 7f2032d9d700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 0x7f20340c2330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f202c00c730 tx=0x7f202c001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.755+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c005d30 con 0x7f2034149d60 2023-05-16T07:03:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.755+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c005e90 con 0x7f2034149d60 2023-05-16T07:03:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.755+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20340bff30 con 0x7f2034149d60 2023-05-16T07:03:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.755+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c007340 con 0x7f2034149d60 2023-05-16T07:03:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.755+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20340c04b0 con 0x7f2034149d60 2023-05-16T07:03:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.756+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203406c350 con 0x7f2034149d60 2023-05-16T07:03:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.756+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f202c010030 con 0x7f2034149d60 2023-05-16T07:03:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.756+0000 7f202a7fc700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f201405ed90 0x7f2014061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.757+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5355+0+0 (secure 0 0 0) 0x7f202c079610 con 0x7f2034149d60 2023-05-16T07:03:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.757+0000 7f203259c700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f201405ed90 0x7f2014061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.757+0000 7f203259c700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f201405ed90 0x7f2014061250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f202400aa10 tx=0x7f2024008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.760+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f202c04aeb0 con 0x7f2034149d60 2023-05-16T07:03:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:02.941+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f20340c0960 con 0x7f2034149d60 2023-05-16T07:03:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.733+0000 7f202a7fc700 1 -- 172.21.15.92:0/1688832409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f202c04e4d0 con 0x7f2034149d60 2023-05-16T07:03:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f201405ed90 msgr2=0x7f2014061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f201405ed90 0x7f2014061250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f202400aa10 tx=0x7f2024008040).stop 2023-05-16T07:03:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 msgr2=0x7f20340c2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 0x7f20340c2330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f202c00c730 tx=0x7f202c001520).stop 2023-05-16T07:03:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 shutdown_connections 2023-05-16T07:03:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f201405ed90 0x7f2014061250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2034065000 0x7f2034163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2034149d60 0x7f20340c2330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.735+0000 7f20388aa700 1 --2- 172.21.15.92:0/1688832409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2034153eb0 0x7f20340c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.736+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 >> 172.21.15.92:0/1688832409 conn(0x7f20340c4430 msgr2=0x7f20340c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.736+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 shutdown_connections 2023-05-16T07:03:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.736+0000 7f20388aa700 1 -- 172.21.15.92:0/1688832409 wait complete. 2023-05-16T07:03:03.742 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:03:03.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:03:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.864+0000 7fc0c74bc700 1 Processor -- start 2023-05-16T07:03:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.864+0000 7fc0c74bc700 1 -- start start 2023-05-16T07:03:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c74bc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 0x7fc0c015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c74bc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c00c0660 con 0x7fc0c0154170 2023-05-16T07:03:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c74bc700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c00c1b00 con 0x7fc0c0154550 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c74bc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00bfd30 con 0x7fc0c0069db0 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c5258700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc0c0154170 0x7fc0c00c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52594/0 (socket says 172.21.15.92:52594) 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c5258700 1 -- 172.21.15.92:0/2534939923 learned_addr learned my addr 172.21.15.92:0/2534939923 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0c5a59700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 0x7fc0c015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697710037 0 0) 0x7fc0c00c0660 con 0x7fc0c0154170 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.865+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a80033f0 con 0x7fc0c0154170 2023-05-16T07:03:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967717353 0 0) 0x7fc0c00c1b00 con 0x7fc0c0154550 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c00c0660 con 0x7fc0c0154550 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134653247 0 0) 0x7fc0a80033f0 con 0x7fc0c0154170 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 msgr2=0x7fc0c015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 0x7fc0c015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 >> v1:172.21.15.158:6789/0 conn(0x7fc0c0154550 legacy=0x7fc0c00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00c0010 con 0x7fc0c0154170 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc0b00017f0 con 0x7fc0c0154170 2023-05-16T07:03:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 >> v1:172.21.15.92:6789/0 conn(0x7fc0c0154170 legacy=0x7fc0c00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0a8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 0x7fc0a8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0a8004b80 0x7fc0a8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00bfd30 con 0x7fc0a8003aa0 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a800ba70 con 0x7fc0a8004b80 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a800bbb0 con 0x7fc0c0069db0 2023-05-16T07:03:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0a8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c4a57700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 0x7fc0a8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5a59700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0a8004b80 0x7fc0a8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 -- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 msgr2=0x7fc0a8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c0069db0 0x7fc0a8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 -- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0a8004b80 msgr2=0x7fc0a8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0a8004b80 0x7fc0a8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 -- 172.21.15.92:0/2534939923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a80033f0 con 0x7fc0a8003aa0 2023-05-16T07:03:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5a59700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0a8004b80 0x7fc0a8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0c5258700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0a8003ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc0b000c440 tx=0x7fc0b0007f40).ready entity=mon.0 client_cookie=42edd55b0cee888e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000cd30 con 0x7fc0a8003aa0 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b000ce90 con 0x7fc0a8003aa0 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.866+0000 7fc0bffff700 1 -- 172.21.15.92:0/2534939923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0005d70 con 0x7fc0a8003aa0 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c74bc700 1 -- 172.21.15.92:0/2534939923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 msgr2=0x7fc0a8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/2534939923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0a8003ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc0b000c440 tx=0x7fc0b0007f40).stop 2023-05-16T07:03:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c5a59700 1 -- 172.21.15.92:0/2534939923 reap_dead start 2023-05-16T07:03:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c74bc700 1 -- 172.21.15.92:0/2534939923 shutdown_connections 2023-05-16T07:03:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c74bc700 1 -- 172.21.15.92:0/2534939923 >> 172.21.15.92:0/2534939923 conn(0x7fc0c00cb700 msgr2=0x7fc0c00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c74bc700 1 -- 172.21.15.92:0/2534939923 shutdown_connections 2023-05-16T07:03:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.867+0000 7fc0c74bc700 1 -- 172.21.15.92:0/2534939923 wait complete. 2023-05-16T07:03:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.868+0000 7fc0c74bc700 1 Processor -- start 2023-05-16T07:03:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.868+0000 7fc0c74bc700 1 -- start start 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c74bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0c00c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c74bc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c01457c0 0x7fc0c01448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c74bc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c00c2c60 0x7fc0c00c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c74bc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00c35d0 con 0x7fc0a8003aa0 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c74bc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0147a00 con 0x7fc0c01457c0 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c74bc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0147ce0 con 0x7fc0c00c2c60 2023-05-16T07:03:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0c00c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0c00c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41858/0 (socket says 172.21.15.92:41858) 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 -- 172.21.15.92:0/1698699093 learned_addr learned my addr 172.21.15.92:0/1698699093 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c4a57700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c01457c0 0x7fc0c01448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 -- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c00c2c60 msgr2=0x7fc0c00c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c00c2c60 0x7fc0c00c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 -- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c01457c0 msgr2=0x7fc0c01448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c01457c0 0x7fc0c01448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c5258700 1 -- 172.21.15.92:0/1698699093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0147fc0 con 0x7fc0a8003aa0 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.869+0000 7fc0c4a57700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c01457c0 0x7fc0c01448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.870+0000 7fc0c5258700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0c00c2310 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fc0b0001d20 tx=0x7fc0b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.870+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0001a50 con 0x7fc0a8003aa0 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.870+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b00016f0 con 0x7fc0a8003aa0 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.870+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c01482a0 con 0x7fc0a8003aa0 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.870+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0004c60 con 0x7fc0a8003aa0 2023-05-16T07:03:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.870+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c00c2800 con 0x7fc0a8003aa0 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.871+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c00666c0 con 0x7fc0a8003aa0 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.871+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc0b0027070 con 0x7fc0a8003aa0 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.871+0000 7fc0bd7fa700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0ac05ed60 0x7fc0ac061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.871+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc0b007a520 con 0x7fc0a8003aa0 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.871+0000 7fc0c4a57700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0ac05ed60 0x7fc0ac061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.872+0000 7fc0c4a57700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0ac05ed60 0x7fc0ac061220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc0b4007a40 tx=0x7fc0b4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:03.875+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc0b004bdf0 con 0x7fc0a8003aa0 2023-05-16T07:03:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.098+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc0c00c29f0 con 0x7fc0a8003aa0 2023-05-16T07:03:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.099+0000 7fc0bd7fa700 1 -- 172.21.15.92:0/1698699093 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc0b0025020 con 0x7fc0a8003aa0 2023-05-16T07:03:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:04.104 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:03:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:03:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:03:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:03:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:03:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:03:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:03:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:03:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:03:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:03:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:03:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:03:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:03:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mds", 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBwKmNk8QBOKRAAQUhkcNSEzLs32ClVUkHbZA==", 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mds" 2023-05-16T07:03:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mgr", 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBwKmNkBQtOKRAALXM4p38q7K7Ali7uIsoBiw==", 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mgr" 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-osd", 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBwKmNkQRJOKRAA/jpzh8MLvAQkZW/dtck9vQ==", 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-osd" 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd", 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBwKmNkLhlOKRAAdXLkqWnqODDINgIGQVf37w==", 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd" 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBwKmNkUiBOKRAA+WstrkLIGcHRAKeM+QqMMg==", 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rgw", 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBwKmNk3SdOKRAAeG/VynJ+oTQlUoOysysPFg==", 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rgw" 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:03:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:03:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:03:04.137 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:03:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0ac05ed60 msgr2=0x7fc0ac061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0ac05ed60 0x7fc0ac061220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc0b4007a40 tx=0x7fc0b4009040).stop 2023-05-16T07:03:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 msgr2=0x7fc0c00c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0c00c2310 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fc0b0001d20 tx=0x7fc0b0015000).stop 2023-05-16T07:03:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 shutdown_connections 2023-05-16T07:03:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0ac05ed60 0x7fc0ac061220 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc0c00c2c60 0x7fc0c00c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a8003aa0 0x7fc0c00c2310 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 --2- 172.21.15.92:0/1698699093 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0c01457c0 0x7fc0c01448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 >> 172.21.15.92:0/1698699093 conn(0x7fc0c00cb700 msgr2=0x7fc0c00ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 shutdown_connections 2023-05-16T07:03:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.101+0000 7fc0c74bc700 1 -- 172.21.15.92:0/1698699093 wait complete. 2023-05-16T07:03:04.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-05-16T07:03:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.250+0000 7f0ca5d9a700 1 Processor -- start 2023-05-16T07:03:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.250+0000 7f0ca5d9a700 1 -- start start 2023-05-16T07:03:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0ca5d9a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0ca5d9a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ca0069db0 0x7f0ca00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0ca5d9a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca00c0660 con 0x7f0ca0154550 2023-05-16T07:03:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0ca5d9a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca00c1b00 con 0x7f0ca0069db0 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0ca5d9a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00bfd30 con 0x7f0ca0154170 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ca0069db0 0x7f0ca00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9f7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ca0069db0 0x7f0ca00c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:45506/0 (socket says 172.21.15.92:45506) 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9f7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:34438/0 (socket says 172.21.15.92:34438) 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9ffff700 1 -- 172.21.15.92:0/2082645941 learned_addr learned my addr 172.21.15.92:0/2082645941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9effd700 1 --1- 172.21.15.92:0/2082645941 >> v1:172.21.15.158:6789/0 conn(0x7f0ca0154550 0x7f0ca00696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:56216/0 (socket says 172.21.15.92:56216) 2023-05-16T07:03:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9f7fe700 1 -- 172.21.15.92:0/2082645941 >> v2:172.21.15.158:3300/0 conn(0x7f0ca0069db0 msgr2=0x7f0ca00c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9f7fe700 1 --2- 172.21.15.92:0/2082645941 >> v2:172.21.15.158:3300/0 conn(0x7f0ca0069db0 0x7f0ca00c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9f7fe700 1 -- 172.21.15.92:0/2082645941 >> v1:172.21.15.158:6789/0 conn(0x7f0ca0154550 legacy=0x7f0ca00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.251+0000 7f0c9f7fe700 1 -- 172.21.15.92:0/2082645941 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca00c0010 con 0x7f0ca0154170 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0c9f7fe700 1 --2- 172.21.15.92:0/2082645941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00c5c70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f0c90009760 tx=0x7f0c90009060).ready entity=mon.2 client_cookie=ed28c62feb379f47 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0c9e7fc700 1 -- 172.21.15.92:0/2082645941 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900090d0 con 0x7f0ca0154170 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0c9e7fc700 1 -- 172.21.15.92:0/2082645941 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000f510 con 0x7f0ca0154170 2023-05-16T07:03:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0c9e7fc700 1 -- 172.21.15.92:0/2082645941 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000f890 con 0x7f0ca0154170 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/2082645941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 msgr2=0x7f0ca00c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/2082645941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00c5c70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f0c90009760 tx=0x7f0c90009060).stop 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/2082645941 shutdown_connections 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/2082645941 >> v2:172.21.15.158:3300/0 conn(0x7f0ca0069db0 0x7f0ca00c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/2082645941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00c5c70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/2082645941 >> 172.21.15.92:0/2082645941 conn(0x7f0ca00cb700 msgr2=0x7f0ca00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/2082645941 shutdown_connections 2023-05-16T07:03:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.252+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/2082645941 wait complete. 2023-05-16T07:03:04.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.253+0000 7f0ca5d9a700 1 Processor -- start 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 -- start start 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ca0069db0 0x7f0ca0155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 0x7f0ca00aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca0155630 con 0x7f0ca0154550 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00b0710 con 0x7f0ca0069db0 2023-05-16T07:03:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0ca5d9a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01586f0 con 0x7f0ca0154170 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0c9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 0x7f0ca00aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0c9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 0x7f0ca00aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41876/0 (socket says 172.21.15.92:41876) 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0c9ffff700 1 -- 172.21.15.92:0/3771566525 learned_addr learned my addr 172.21.15.92:0/3771566525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0c9effd700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.254+0000 7f0c9f7fe700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ca0069db0 0x7f0ca0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c9ffff700 1 -- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 msgr2=0x7f0ca00b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c9ffff700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c9ffff700 1 -- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ca0069db0 msgr2=0x7f0ca0155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c9ffff700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ca0069db0 0x7f0ca0155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c9ffff700 1 -- 172.21.15.92:0/3771566525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900092a0 con 0x7f0ca0154550 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c9ffff700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 0x7f0ca00aef70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f0c98009e20 tx=0x7f0c9800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800c890 con 0x7f0ca0154550 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9800f040 con 0x7f0ca0154550 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca0155820 con 0x7f0ca0154550 2023-05-16T07:03:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800cc00 con 0x7f0ca0154550 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.255+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca0157c10 con 0x7f0ca0154550 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.256+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca00666c0 con 0x7f0ca0154550 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.256+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0c98014410 con 0x7f0ca0154550 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.256+0000 7f0c877fe700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8c05ed90 0x7f0c8c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.257+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0c98079a80 con 0x7f0ca0154550 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.257+0000 7f0c9f7fe700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8c05ed90 0x7f0c8c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.257+0000 7f0c9f7fe700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8c05ed90 0x7f0c8c061250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f0c90009760 tx=0x7f0c90017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.261+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0c980498f0 con 0x7f0ca0154550 2023-05-16T07:03:04.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.484+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f0ca006c350 con 0x7f0ca0154550 2023-05-16T07:03:04.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.487+0000 7f0c877fe700 1 -- 172.21.15.92:0/3771566525 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0c9804b200 con 0x7f0ca0154550 2023-05-16T07:03:04.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8c05ed90 msgr2=0x7f0c8c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8c05ed90 0x7f0c8c061250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f0c90009760 tx=0x7f0c90017000).stop 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 msgr2=0x7f0ca00aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 0x7f0ca00aef70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f0c98009e20 tx=0x7f0c9800b040).stop 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 shutdown_connections 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8c05ed90 0x7f0c8c061250 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca0154550 0x7f0ca00aef70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ca0069db0 0x7f0ca0155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 --2- 172.21.15.92:0/3771566525 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ca0154170 0x7f0ca00b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 >> 172.21.15.92:0/3771566525 conn(0x7f0ca00cb700 msgr2=0x7f0ca00ab270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 shutdown_connections 2023-05-16T07:03:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.489+0000 7f0ca5d9a700 1 -- 172.21.15.92:0/3771566525 wait complete. 2023-05-16T07:03:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:updated 2023-05-16T07:03:04.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-05-16T07:03:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.614+0000 7f2fa1bb2700 1 Processor -- start 2023-05-16T07:03:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.614+0000 7f2fa1bb2700 1 -- start start 2023-05-16T07:03:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2fa1bb2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f9c154170 0x7f2f9c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2fa1bb2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c069db0 0x7f2f9c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2fa1bb2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9c0c0660 con 0x7f2f9c154550 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2fa1bb2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9c0c1b00 con 0x7f2f9c154170 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2fa1bb2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0bfd30 con 0x7f2f9c069db0 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2f9b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f9c154170 0x7f2f9c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2f9b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f9c154170 0x7f2f9c0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41880/0 (socket says 172.21.15.92:41880) 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2f9affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2f9c154550 0x7f2f9c0696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52632/0 (socket says 172.21.15.92:52632) 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.615+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/1228337207 learned_addr learned my addr 172.21.15.92:0/1228337207 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9bfff700 1 --2- 172.21.15.92:0/1228337207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c069db0 0x7f2f9c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/1228337207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c069db0 msgr2=0x7f2f9c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9b7fe700 1 --2- 172.21.15.92:0/1228337207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c069db0 0x7f2f9c0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/1228337207 >> v1:172.21.15.92:6789/0 conn(0x7f2f9c154550 legacy=0x7f2f9c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/1228337207 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0c0010 con 0x7f2f9c154170 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9a7fc700 1 -- 172.21.15.92:0/1228337207 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622266909 0 0) 0x7f2f9c0c0660 con 0x7f2f9c154550 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9b7fe700 1 --2- 172.21.15.92:0/1228337207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c154170 0x7f2f9c0c5c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f2f8c009f20 tx=0x7f2f8c009840).ready entity=mon.0 client_cookie=94a8e5bb6663bebb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9a7fc700 1 -- 172.21.15.92:0/1228337207 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00e070 con 0x7f2f9c154170 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9a7fc700 1 -- 172.21.15.92:0/1228337207 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00ba80 con 0x7f2f9c154170 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2f9a7fc700 1 -- 172.21.15.92:0/1228337207 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00be00 con 0x7f2f9c154170 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/1228337207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c154170 msgr2=0x7f2f9c0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/1228337207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c154170 0x7f2f9c0c5c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f2f8c009f20 tx=0x7f2f8c009840).stop 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/1228337207 shutdown_connections 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/1228337207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c069db0 0x7f2f9c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/1228337207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c154170 0x7f2f9c0c5c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/1228337207 >> 172.21.15.92:0/1228337207 conn(0x7f2f9c0cb700 msgr2=0x7f2f9c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/1228337207 shutdown_connections 2023-05-16T07:03:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.616+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/1228337207 wait complete. 2023-05-16T07:03:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 Processor -- start 2023-05-16T07:03:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 -- start start 2023-05-16T07:03:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 0x7f2f9c0c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c154170 0x7f2f9c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f9c154550 0x7f2f9c0c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0c38d0 con 0x7f2f9c069db0 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0c2cc0 con 0x7f2f9c154550 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2fa1bb2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0c2fa0 con 0x7f2f9c154170 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2f9b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 0x7f2f9c0c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2f9b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 0x7f2f9c0c2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41886/0 (socket says 172.21.15.92:41886) 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.618+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/3834693128 learned_addr learned my addr 172.21.15.92:0/3834693128 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9affd700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c154170 0x7f2f9c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9bfff700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f9c154550 0x7f2f9c0c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c154170 msgr2=0x7f2f9c1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9b7fe700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c154170 0x7f2f9c1584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f9c154550 msgr2=0x7f2f9c0c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9b7fe700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f9c154550 0x7f2f9c0c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9b7fe700 1 -- 172.21.15.92:0/3834693128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c009ae0 con 0x7f2f9c069db0 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9bfff700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f9c154550 0x7f2f9c0c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f9b7fe700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 0x7f2f9c0c2700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f2f9c083060 tx=0x7f2f8c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c007080 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c0092e0 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9c0c3280 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c01d410 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.619+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c14d030 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.620+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c0666c0 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.620+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2f8c01b040 con 0x7f2f9c069db0 2023-05-16T07:03:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.620+0000 7f2f837fe700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f8805ed90 0x7f2f88061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.621+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f2f8c07b170 con 0x7f2f9c069db0 2023-05-16T07:03:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.621+0000 7f2f9affd700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f8805ed90 0x7f2f88061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.621+0000 7f2f9affd700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f8805ed90 0x7f2f88061250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f2f9c0c2bf0 tx=0x7f2f90007d70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.624+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2f8c04cae0 con 0x7f2f9c069db0 2023-05-16T07:03:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.849+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2f9c06c350 con 0x7f2f9c069db0 2023-05-16T07:03:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.852+0000 7f2f837fe700 1 -- 172.21.15.92:0/3834693128 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2f8c050100 con 0x7f2f9c069db0 2023-05-16T07:03:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f8805ed90 msgr2=0x7f2f88061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f8805ed90 0x7f2f88061250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f2f9c0c2bf0 tx=0x7f2f90007d70).stop 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 msgr2=0x7f2f9c0c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 0x7f2f9c0c2700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f2f9c083060 tx=0x7f2f8c009840).stop 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 shutdown_connections 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f8805ed90 0x7f2f88061250 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2f9c154550 0x7f2f9c0c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f9c069db0 0x7f2f9c0c2700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 --2- 172.21.15.92:0/3834693128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2f9c154170 0x7f2f9c1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 >> 172.21.15.92:0/3834693128 conn(0x7f2f9c0cb700 msgr2=0x7f2f9c0ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 shutdown_connections 2023-05-16T07:03:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.854+0000 7f2fa1bb2700 1 -- 172.21.15.92:0/3834693128 wait complete. 2023-05-16T07:03:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:updated 2023-05-16T07:03:04.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-05-16T07:03:04.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.977+0000 7fb53d867700 1 Processor -- start 2023-05-16T07:03:04.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.977+0000 7fb53d867700 1 -- start start 2023-05-16T07:03:04.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb53d867700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb538154170 0x7fb5380c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb53d867700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb538154550 0x7fb5380c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb53d867700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5380696a0 0x7fb538069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb53d867700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380c0660 con 0x7fb538154550 2023-05-16T07:03:04.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb53d867700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53806a020 con 0x7fb538154170 2023-05-16T07:03:04.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb53d867700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380bfd30 con 0x7fb5380696a0 2023-05-16T07:03:04.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb538154550 0x7fb5380c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb538154550 0x7fb5380c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41898/0 (socket says 172.21.15.92:41898) 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 -- 172.21.15.92:0/1714899153 learned_addr learned my addr 172.21.15.92:0/1714899153 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5377fe700 1 --2- 172.21.15.92:0/1714899153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5380696a0 0x7fb538069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb536ffd700 1 --2- 172.21.15.92:0/1714899153 >> v2:172.21.15.158:3300/0 conn(0x7fb538154170 0x7fb5380c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 -- 172.21.15.92:0/1714899153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5380696a0 msgr2=0x7fb538069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 --2- 172.21.15.92:0/1714899153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5380696a0 0x7fb538069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 -- 172.21.15.92:0/1714899153 >> v2:172.21.15.158:3300/0 conn(0x7fb538154170 msgr2=0x7fb5380c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 --2- 172.21.15.92:0/1714899153 >> v2:172.21.15.158:3300/0 conn(0x7fb538154170 0x7fb5380c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb5367fc700 1 -- 172.21.15.92:0/1714899153 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380c0010 con 0x7fb538154550 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.978+0000 7fb536ffd700 1 --2- 172.21.15.92:0/1714899153 >> v2:172.21.15.158:3300/0 conn(0x7fb538154170 0x7fb5380c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb5367fc700 1 --2- 172.21.15.92:0/1714899153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb538154550 0x7fb5380c6160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fb538083030 tx=0x7fb5200090f0).ready entity=mon.0 client_cookie=9e78fece395fc14a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb535ffb700 1 -- 172.21.15.92:0/1714899153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000b600 con 0x7fb538154550 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb535ffb700 1 -- 172.21.15.92:0/1714899153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000b760 con 0x7fb538154550 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb535ffb700 1 -- 172.21.15.92:0/1714899153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000bb00 con 0x7fb538154550 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 -- 172.21.15.92:0/1714899153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb538154550 msgr2=0x7fb5380c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 --2- 172.21.15.92:0/1714899153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb538154550 0x7fb5380c6160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fb538083030 tx=0x7fb5200090f0).stop 2023-05-16T07:03:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 -- 172.21.15.92:0/1714899153 shutdown_connections 2023-05-16T07:03:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 --2- 172.21.15.92:0/1714899153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5380696a0 0x7fb538069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 --2- 172.21.15.92:0/1714899153 >> v2:172.21.15.158:3300/0 conn(0x7fb538154170 0x7fb5380c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 --2- 172.21.15.92:0/1714899153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb538154550 0x7fb5380c6160 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 -- 172.21.15.92:0/1714899153 >> 172.21.15.92:0/1714899153 conn(0x7fb5380cb700 msgr2=0x7fb5380c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 -- 172.21.15.92:0/1714899153 shutdown_connections 2023-05-16T07:03:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.979+0000 7fb53d867700 1 -- 172.21.15.92:0/1714899153 wait complete. 2023-05-16T07:03:04.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.980+0000 7fb53d867700 1 Processor -- start 2023-05-16T07:03:04.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 -- start start 2023-05-16T07:03:04.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 0x7fb53814ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb538154170 0x7fb538155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb538154550 0x7fb5381559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb538157590 con 0x7fb5380696a0 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380c34b0 con 0x7fb538154170 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb53d867700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb538156070 con 0x7fb538154550 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb536ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 0x7fb53814ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb536ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 0x7fb53814ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41900/0 (socket says 172.21.15.92:41900) 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb536ffd700 1 -- 172.21.15.92:0/979424500 learned_addr learned my addr 172.21.15.92:0/979424500 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb5377fe700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb538154550 0x7fb5381559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.981+0000 7fb5367fc700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb538154170 0x7fb538155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb536ffd700 1 -- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb538154550 msgr2=0x7fb5381559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb536ffd700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb538154550 0x7fb5381559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb536ffd700 1 -- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb538154170 msgr2=0x7fb538155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb536ffd700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb538154170 0x7fb538155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb536ffd700 1 -- 172.21.15.92:0/979424500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009370 con 0x7fb5380696a0 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb5367fc700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb538154170 0x7fb538155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb536ffd700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 0x7fb53814ca70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fb52800a430 tx=0x7fb528007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800ae10 con 0x7fb5380696a0 2023-05-16T07:03:04.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800d650 con 0x7fb5380696a0 2023-05-16T07:03:04.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800d9d0 con 0x7fb5380696a0 2023-05-16T07:03:04.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5381577a0 con 0x7fb5380696a0 2023-05-16T07:03:04.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.982+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb538158040 con 0x7fb5380696a0 2023-05-16T07:03:04.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.983+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb538157990 con 0x7fb5380696a0 2023-05-16T07:03:04.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.983+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb528016470 con 0x7fb5380696a0 2023-05-16T07:03:04.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.983+0000 7fb52effd700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb51805ed60 0x7fb518061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:04.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.984+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb52807a620 con 0x7fb5380696a0 2023-05-16T07:03:04.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.984+0000 7fb5367fc700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb51805ed60 0x7fb518061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:04.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.984+0000 7fb5367fc700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb51805ed60 0x7fb518061220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fb538154930 tx=0x7fb520017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:04.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:03:04.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:04.987+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb52804bf60 con 0x7fb5380696a0 2023-05-16T07:03:04.998 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:03:05.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.216+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb538157b90 con 0x7fb5380696a0 2023-05-16T07:03:05.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.219+0000 7fb52effd700 1 -- 172.21.15.92:0/979424500 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb52804f580 con 0x7fb5380696a0 2023-05-16T07:03:05.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb51805ed60 msgr2=0x7fb518061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb51805ed60 0x7fb518061220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fb538154930 tx=0x7fb520017000).stop 2023-05-16T07:03:05.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 msgr2=0x7fb53814ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 0x7fb53814ca70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fb52800a430 tx=0x7fb528007910).stop 2023-05-16T07:03:05.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 shutdown_connections 2023-05-16T07:03:05.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb51805ed60 0x7fb518061220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb538154550 0x7fb5381559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5380696a0 0x7fb53814ca70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 --2- 172.21.15.92:0/979424500 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb538154170 0x7fb538155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 >> 172.21.15.92:0/979424500 conn(0x7fb5380cb700 msgr2=0x7fb53814f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 shutdown_connections 2023-05-16T07:03:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.221+0000 7fb53d867700 1 -- 172.21.15.92:0/979424500 wait complete. 2023-05-16T07:03:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:updated 2023-05-16T07:03:05.237 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-05-16T07:03:05.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.346+0000 7f05c1dde700 1 Processor -- start 2023-05-16T07:03:05.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.346+0000 7f05c1dde700 1 -- start start 2023-05-16T07:03:05.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05c1dde700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f05bc0c8f10 0x7f05bc0bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05c1dde700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0691c0 con 0x7f05bc067f60 2023-05-16T07:03:05.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05c1dde700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc069d60 con 0x7f05bc153ca0 2023-05-16T07:03:05.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05c1dde700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc0683c0 con 0x7f05bc0c8f10 2023-05-16T07:03:05.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f05bc067f60 0x7f05bc0c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52658/0 (socket says 172.21.15.92:52658) 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bbfff700 1 -- 172.21.15.92:0/3368717453 learned_addr learned my addr 172.21.15.92:0/3368717453 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632402749 0 0) 0x7f05bc0691c0 con 0x7f05bc067f60 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a40033f0 con 0x7f05bc067f60 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05c15dd700 1 --2- 172.21.15.92:0/3368717453 >> v2:172.21.15.158:3300/0 conn(0x7f05bc0c8f10 0x7f05bc0bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1747575149 0 0) 0x7f05a40033f0 con 0x7f05bc067f60 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 >> v2:172.21.15.158:3300/0 conn(0x7f05bc0c8f10 msgr2=0x7f05bc0bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:05.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 --2- 172.21.15.92:0/3368717453 >> v2:172.21.15.158:3300/0 conn(0x7f05bc0c8f10 0x7f05bc0bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 >> v1:172.21.15.158:6789/0 conn(0x7f05bc153ca0 legacy=0x7f05bc069650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0bf760 con 0x7f05bc067f60 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f05b0001470 con 0x7f05bc067f60 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 >> v1:172.21.15.92:6789/0 conn(0x7f05bc067f60 legacy=0x7f05bc0c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05a40038e0 0x7f05a4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05a4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05a4005000 0x7f05a400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a400be80 con 0x7f05a4004690 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a400bff0 con 0x7f05a40038e0 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a400c1b0 con 0x7f05a4005000 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.347+0000 7f05bbfff700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05a4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c0ddc700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05a40038e0 0x7f05a4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c15dd700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05a4005000 0x7f05a400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bbfff700 1 -- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05a4005000 msgr2=0x7f05a400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bbfff700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05a4005000 0x7f05a400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bbfff700 1 -- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05a40038e0 msgr2=0x7f05a4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bbfff700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05a40038e0 0x7f05a4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c15dd700 1 -- 172.21.15.92:0/3368717453 reap_dead start 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bbfff700 1 -- 172.21.15.92:0/3368717453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40033f0 con 0x7f05a4004690 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c0ddc700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05a40038e0 0x7f05a4003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bbfff700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05a4004b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f05b0007980 tx=0x7f05bc069130).ready entity=mon.0 client_cookie=da039f2adf82738c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0017070 con 0x7f05a4004690 2023-05-16T07:03:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b00070d0 con 0x7f05a4004690 2023-05-16T07:03:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05bb7fe700 1 -- 172.21.15.92:0/3368717453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0007450 con 0x7f05a4004690 2023-05-16T07:03:05.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c1dde700 1 -- 172.21.15.92:0/3368717453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 msgr2=0x7f05a4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c1dde700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05a4004b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f05b0007980 tx=0x7f05bc069130).stop 2023-05-16T07:03:05.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.348+0000 7f05c1dde700 1 -- 172.21.15.92:0/3368717453 shutdown_connections 2023-05-16T07:03:05.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.349+0000 7f05c1dde700 1 --2- 172.21.15.92:0/3368717453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05a4004b10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.349+0000 7f05c1dde700 1 -- 172.21.15.92:0/3368717453 >> 172.21.15.92:0/3368717453 conn(0x7f05bc0622b0 msgr2=0x7f05bc0626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:05.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.349+0000 7f05c1dde700 1 -- 172.21.15.92:0/3368717453 shutdown_connections 2023-05-16T07:03:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.349+0000 7f05c1dde700 1 -- 172.21.15.92:0/3368717453 wait complete. 2023-05-16T07:03:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.350+0000 7f05c1dde700 1 Processor -- start 2023-05-16T07:03:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.350+0000 7f05c1dde700 1 -- start start 2023-05-16T07:03:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05bc14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05bc0c82b0 0x7f05bc0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05bc15fce0 0x7f05bc160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc160a50 con 0x7f05a4004690 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc14ff70 con 0x7f05bc15fce0 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0ae2d0 con 0x7f05bc0c82b0 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05bc14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05bc14d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41920/0 (socket says 172.21.15.92:41920) 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 -- 172.21.15.92:0/362622911 learned_addr learned my addr 172.21.15.92:0/362622911 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c15dd700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05bc15fce0 0x7f05bc160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05bbfff700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05bc0c82b0 0x7f05bc0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 -- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05bc0c82b0 msgr2=0x7f05bc0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05bc0c82b0 0x7f05bc0c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 -- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05bc15fce0 msgr2=0x7f05bc160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05bc15fce0 0x7f05bc160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 -- 172.21.15.92:0/362622911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0c8c20 con 0x7f05a4004690 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05bbfff700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05bc0c82b0 0x7f05bc0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:05.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c0ddc700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05bc14d660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f05ac00e790 tx=0x7f05ac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00c5b0 con 0x7f05a4004690 2023-05-16T07:03:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac00c710 con 0x7f05a4004690 2023-05-16T07:03:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc0ae760 con 0x7f05a4004690 2023-05-16T07:03:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00ca90 con 0x7f05a4004690 2023-05-16T07:03:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.351+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0adc80 con 0x7f05a4004690 2023-05-16T07:03:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.352+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc047150 con 0x7f05a4004690 2023-05-16T07:03:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.353+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f05ac01b030 con 0x7f05a4004690 2023-05-16T07:03:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.353+0000 7f05b8ff9700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05a805ed90 0x7f05a8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.353+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f05ac079550 con 0x7f05a4004690 2023-05-16T07:03:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.353+0000 7f05bbfff700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05a805ed90 0x7f05a8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.354+0000 7f05bbfff700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05a805ed90 0x7f05a8061250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f05bc14f9e0 tx=0x7f05bc069130).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:05.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.357+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f05ac04e920 con 0x7f05a4004690 2023-05-16T07:03:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.584+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f05bc0685b0 con 0x7f05a4004690 2023-05-16T07:03:05.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.587+0000 7f05b8ff9700 1 -- 172.21.15.92:0/362622911 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f05ac0016f0 con 0x7f05a4004690 2023-05-16T07:03:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05a805ed90 msgr2=0x7f05a8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05a805ed90 0x7f05a8061250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f05bc14f9e0 tx=0x7f05bc069130).stop 2023-05-16T07:03:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 msgr2=0x7f05bc14d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05bc14d660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f05ac00e790 tx=0x7f05ac009040).stop 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 shutdown_connections 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05a805ed90 0x7f05a8061250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05bc15fce0 0x7f05bc160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a4004690 0x7f05bc14d660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 --2- 172.21.15.92:0/362622911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05bc0c82b0 0x7f05bc0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 >> 172.21.15.92:0/362622911 conn(0x7f05bc0622b0 msgr2=0x7f05bc0c1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 shutdown_connections 2023-05-16T07:03:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.589+0000 7f05c1dde700 1 -- 172.21.15.92:0/362622911 wait complete. 2023-05-16T07:03:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:updated 2023-05-16T07:03:05.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-05-16T07:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.712+0000 7f1cea962700 1 Processor -- start 2023-05-16T07:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.712+0000 7f1cea962700 1 -- start start 2023-05-16T07:03:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1cea962700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ce4154170 0x7f1ce40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1cea962700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1cea962700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce40c0660 con 0x7f1ce40696a0 2023-05-16T07:03:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1cea962700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce40c1b00 con 0x7f1ce4154170 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1cea962700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce40bfd30 con 0x7f1ce4154550 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1ce3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ce4154170 0x7f1ce40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1ce3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ce4154170 0x7f1ce40c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41928/0 (socket says 172.21.15.92:41928) 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1ce3fff700 1 -- 172.21.15.92:0/3777527937 learned_addr learned my addr 172.21.15.92:0/3777527937 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.713+0000 7f1ce37fe700 1 --2- 172.21.15.92:0/3777527937 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce3fff700 1 -- 172.21.15.92:0/3777527937 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 msgr2=0x7f1ce40c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce3fff700 1 --2- 172.21.15.92:0/3777527937 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce3fff700 1 -- 172.21.15.92:0/3777527937 >> v1:172.21.15.158:6789/0 conn(0x7f1ce40696a0 legacy=0x7f1ce4157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce3fff700 1 -- 172.21.15.92:0/3777527937 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce40c0010 con 0x7f1ce4154170 2023-05-16T07:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce3fff700 1 --2- 172.21.15.92:0/3777527937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce4154170 0x7f1ce40c5c70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f1cd4006fd0 tx=0x7f1cd40090f0).ready entity=mon.0 client_cookie=3857b118a8f9b952 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce2ffd700 1 -- 172.21.15.92:0/3777527937 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400b600 con 0x7f1ce4154170 2023-05-16T07:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce2ffd700 1 -- 172.21.15.92:0/3777527937 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd400b760 con 0x7f1ce4154170 2023-05-16T07:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1ce2ffd700 1 -- 172.21.15.92:0/3777527937 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400bb00 con 0x7f1ce4154170 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 -- 172.21.15.92:0/3777527937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce4154170 msgr2=0x7f1ce40c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 --2- 172.21.15.92:0/3777527937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce4154170 0x7f1ce40c5c70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f1cd4006fd0 tx=0x7f1cd40090f0).stop 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 -- 172.21.15.92:0/3777527937 shutdown_connections 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 --2- 172.21.15.92:0/3777527937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce4154170 0x7f1ce40c5c70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 --2- 172.21.15.92:0/3777527937 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 -- 172.21.15.92:0/3777527937 >> 172.21.15.92:0/3777527937 conn(0x7f1ce40cb700 msgr2=0x7f1ce40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 -- 172.21.15.92:0/3777527937 shutdown_connections 2023-05-16T07:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.714+0000 7f1cea962700 1 -- 172.21.15.92:0/3777527937 wait complete. 2023-05-16T07:03:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.715+0000 7f1cea962700 1 Processor -- start 2023-05-16T07:03:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 -- start start 2023-05-16T07:03:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 0x7f1ce4142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce4154170 0x7f1ce4150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41500b0 con 0x7f1ce40696a0 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4165690 con 0x7f1ce4154170 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1cea962700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41653b0 con 0x7f1ce4154550 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 0x7f1ce4142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 0x7f1ce4142830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41932/0 (socket says 172.21.15.92:41932) 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.716+0000 7f1ce3fff700 1 -- 172.21.15.92:0/1440549679 learned_addr learned my addr 172.21.15.92:0/1440549679 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce37fe700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce4154170 0x7f1ce4150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce8eff700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce3fff700 1 -- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 msgr2=0x7f1ce40c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce3fff700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce3fff700 1 -- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce4154170 msgr2=0x7f1ce4150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce3fff700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce4154170 0x7f1ce4150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce3fff700 1 -- 172.21.15.92:0/1440549679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4009370 con 0x7f1ce40696a0 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ce3fff700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 0x7f1ce4142830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1cd4009f50 tx=0x7f1cd4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400b8d0 con 0x7f1ce40696a0 2023-05-16T07:03:05.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd400ba30 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce414fa60 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40246c0 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.717+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce4142d20 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.718+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce40666c0 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.718+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1cd400f070 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.718+0000 7f1ccbfff700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cd005ed90 0x7f1cd0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.718+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1cd407b530 con 0x7f1ce40696a0 2023-05-16T07:03:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.719+0000 7f1ce37fe700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cd005ed90 0x7f1cd0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:05.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.719+0000 7f1ce37fe700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cd005ed90 0x7f1cd0061250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f1ce4153420 tx=0x7f1cd8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:05.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.722+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1cd404ce20 con 0x7f1ce40696a0 2023-05-16T07:03:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.950+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1ce406c350 con 0x7f1ce40696a0 2023-05-16T07:03:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.954+0000 7f1ccbfff700 1 -- 172.21.15.92:0/1440549679 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1cd4050440 con 0x7f1ce40696a0 2023-05-16T07:03:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.955+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cd005ed90 msgr2=0x7f1cd0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.955+0000 7f1cea962700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cd005ed90 0x7f1cd0061250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f1ce4153420 tx=0x7f1cd8008040).stop 2023-05-16T07:03:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 msgr2=0x7f1ce4142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 0x7f1ce4142830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1cd4009f50 tx=0x7f1cd4017000).stop 2023-05-16T07:03:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 shutdown_connections 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cd005ed90 0x7f1cd0061250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce4154550 0x7f1ce40c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce40696a0 0x7f1ce4142830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 --2- 172.21.15.92:0/1440549679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce4154170 0x7f1ce4150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 >> 172.21.15.92:0/1440549679 conn(0x7f1ce40cb700 msgr2=0x7f1ce40aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 shutdown_connections 2023-05-16T07:03:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:05.956+0000 7f1cea962700 1 -- 172.21.15.92:0/1440549679 wait complete. 2023-05-16T07:03:05.962 INFO:teuthology.orchestra.run.smithi092.stderr:updated 2023-05-16T07:03:05.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-05-16T07:03:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.081+0000 7f1ce5bfb700 1 Processor -- start 2023-05-16T07:03:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.081+0000 7f1ce5bfb700 1 -- start start 2023-05-16T07:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce5bfb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce00aab40 0x7f1ce0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce5bfb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ce0153b00 0x7f1ce0153f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce5bfb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce006a920 con 0x7f1ce0069750 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce5bfb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce00c65f0 con 0x7f1ce0153b00 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce5bfb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00bfa30 con 0x7f1ce00aab40 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce4bf9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1ce0069750 0x7f1ce00c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52688/0 (socket says 172.21.15.92:52688) 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce53fa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ce0153b00 0x7f1ce0153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce4bf9700 1 -- 172.21.15.92:0/3081384256 learned_addr learned my addr 172.21.15.92:0/3081384256 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1cdf7fe700 1 -- 172.21.15.92:0/3081384256 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247993406 0 0) 0x7f1ce006a920 con 0x7f1ce0069750 2023-05-16T07:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1cdf7fe700 1 -- 172.21.15.92:0/3081384256 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ccc0033f0 con 0x7f1ce0069750 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1cdffff700 1 --2- 172.21.15.92:0/3081384256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce00aab40 0x7f1ce0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce53fa700 1 -- 172.21.15.92:0/3081384256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce00aab40 msgr2=0x7f1ce0069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce53fa700 1 --2- 172.21.15.92:0/3081384256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce00aab40 0x7f1ce0069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce53fa700 1 -- 172.21.15.92:0/3081384256 >> v1:172.21.15.92:6789/0 conn(0x7f1ce0069750 legacy=0x7f1ce00c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce53fa700 1 -- 172.21.15.92:0/3081384256 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00bfc80 con 0x7f1ce0153b00 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1ce53fa700 1 --2- 172.21.15.92:0/3081384256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce0153b00 0x7f1ce0153f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f1cd8009760 tx=0x7f1cd8009060).ready entity=mon.0 client_cookie=e7154fb2b3f5205 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1cdf7fe700 1 -- 172.21.15.92:0/3081384256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800f510 con 0x7f1ce0153b00 2023-05-16T07:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1cdf7fe700 1 -- 172.21.15.92:0/3081384256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd800f670 con 0x7f1ce0153b00 2023-05-16T07:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.082+0000 7f1cdf7fe700 1 -- 172.21.15.92:0/3081384256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800f9f0 con 0x7f1ce0153b00 2023-05-16T07:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/3081384256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce0153b00 msgr2=0x7f1ce0153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/3081384256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce0153b00 0x7f1ce0153f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f1cd8009760 tx=0x7f1cd8009060).stop 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/3081384256 shutdown_connections 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/3081384256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce0153b00 0x7f1ce0153f70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/3081384256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce00aab40 0x7f1ce0069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/3081384256 >> 172.21.15.92:0/3081384256 conn(0x7f1ce00cb550 msgr2=0x7f1ce00c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/3081384256 shutdown_connections 2023-05-16T07:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.083+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/3081384256 wait complete. 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.084+0000 7f1ce5bfb700 1 Processor -- start 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.084+0000 7f1ce5bfb700 1 -- start start 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce5bfb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce0069750 0x7f1ce0164dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce5bfb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 0x7f1ce01647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce5bfb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce0153b00 0x7f1ce014e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce5bfb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce014e5d0 con 0x7f1ce00aab40 2023-05-16T07:03:06.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce5bfb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce014f0d0 con 0x7f1ce0069750 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce5bfb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce014e830 con 0x7f1ce0153b00 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 0x7f1ce01647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 0x7f1ce01647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41950/0 (socket says 172.21.15.92:41950) 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 -- 172.21.15.92:0/33782271 learned_addr learned my addr 172.21.15.92:0/33782271 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce4bf9700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce0069750 0x7f1ce0164dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce53fa700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce0153b00 0x7f1ce014e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 -- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce0153b00 msgr2=0x7f1ce014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce0153b00 0x7f1ce014e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 -- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce0069750 msgr2=0x7f1ce0164dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce0069750 0x7f1ce0164dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1cdffff700 1 -- 172.21.15.92:0/33782271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd80092a0 con 0x7f1ce00aab40 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce53fa700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce0153b00 0x7f1ce014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.085+0000 7f1ce4bf9700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce0069750 0x7f1ce0164dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.086+0000 7f1cdffff700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 0x7f1ce01647c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f1cc800c850 tx=0x7f1cc8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.086+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8010580 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.086+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc80106e0 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.086+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce014eb10 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.086+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8010a60 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.086+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce00af7b0 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.087+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce00af610 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.087+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1cc801b460 con 0x7f1ce00aab40 2023-05-16T07:03:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.087+0000 7f1cdcff9700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cc005ed90 0x7f1cc0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.087+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1cc80798c0 con 0x7f1ce00aab40 2023-05-16T07:03:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.088+0000 7f1ce4bf9700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cc005ed90 0x7f1cc0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.088+0000 7f1ce4bf9700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cc005ed90 0x7f1cc0061250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f1ce0151130 tx=0x7f1cd0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.091+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1ce00af610 con 0x7f1ce00aab40 2023-05-16T07:03:06.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.319+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f1ce00af610 con 0x7f1ce00aab40 2023-05-16T07:03:06.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.323+0000 7f1cdcff9700 1 -- 172.21.15.92:0/33782271 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1ce00af610 con 0x7f1ce00aab40 2023-05-16T07:03:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cc005ed90 msgr2=0x7f1cc0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cc005ed90 0x7f1cc0061250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f1ce0151130 tx=0x7f1cd0009040).stop 2023-05-16T07:03:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 msgr2=0x7f1ce01647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 0x7f1ce01647c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f1cc800c850 tx=0x7f1cc8008040).stop 2023-05-16T07:03:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 shutdown_connections 2023-05-16T07:03:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cc005ed90 0x7f1cc0061250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ce0153b00 0x7f1ce014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ce0069750 0x7f1ce0164dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 --2- 172.21.15.92:0/33782271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ce00aab40 0x7f1ce01647c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 >> 172.21.15.92:0/33782271 conn(0x7f1ce00cb550 msgr2=0x7f1ce00c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 shutdown_connections 2023-05-16T07:03:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.325+0000 7f1ce5bfb700 1 -- 172.21.15.92:0/33782271 wait complete. 2023-05-16T07:03:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:updated 2023-05-16T07:03:06.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:03:06.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.448+0000 7f396a415700 1 Processor -- start 2023-05-16T07:03:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.448+0000 7f396a415700 1 -- start start 2023-05-16T07:03:06.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f396a415700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3964154170 0x7f39640c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f396a415700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3964154550 0x7f39640c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f396a415700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39640c0660 con 0x7f39640696a0 2023-05-16T07:03:06.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f396a415700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39640c1b00 con 0x7f3964154170 2023-05-16T07:03:06.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f396a415700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39640bfd30 con 0x7f3964154550 2023-05-16T07:03:06.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f3963fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3964154170 0x7f39640c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f39689b2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f39640696a0 0x7f3964157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52704/0 (socket says 172.21.15.92:52704) 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.449+0000 7f39689b2700 1 -- 172.21.15.92:0/3511088682 learned_addr learned my addr 172.21.15.92:0/3511088682 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f39637fe700 1 --2- 172.21.15.92:0/3511088682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3964154550 0x7f39640c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3963fff700 1 -- 172.21.15.92:0/3511088682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3964154550 msgr2=0x7f39640c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3963fff700 1 --2- 172.21.15.92:0/3511088682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3964154550 0x7f39640c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3963fff700 1 -- 172.21.15.92:0/3511088682 >> v1:172.21.15.92:6789/0 conn(0x7f39640696a0 legacy=0x7f3964157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3963fff700 1 -- 172.21.15.92:0/3511088682 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39640c0010 con 0x7f3964154170 2023-05-16T07:03:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3962ffd700 1 -- 172.21.15.92:0/3511088682 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727276898 0 0) 0x7f39640c0660 con 0x7f39640696a0 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3963fff700 1 --2- 172.21.15.92:0/3511088682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f39640c5c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f3954009760 tx=0x7f3954009060).ready entity=mon.0 client_cookie=402d5cc9c39b861 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3962ffd700 1 -- 172.21.15.92:0/3511088682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39540090d0 con 0x7f3964154170 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3962ffd700 1 -- 172.21.15.92:0/3511088682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395400b580 con 0x7f3964154170 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f3962ffd700 1 -- 172.21.15.92:0/3511088682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395400b900 con 0x7f3964154170 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 -- 172.21.15.92:0/3511088682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 msgr2=0x7f39640c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 --2- 172.21.15.92:0/3511088682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f39640c5c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f3954009760 tx=0x7f3954009060).stop 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 -- 172.21.15.92:0/3511088682 shutdown_connections 2023-05-16T07:03:06.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 --2- 172.21.15.92:0/3511088682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f39640c5c70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 --2- 172.21.15.92:0/3511088682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3964154550 0x7f39640c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 -- 172.21.15.92:0/3511088682 >> 172.21.15.92:0/3511088682 conn(0x7f39640cb700 msgr2=0x7f39640c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:06.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 -- 172.21.15.92:0/3511088682 shutdown_connections 2023-05-16T07:03:06.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.450+0000 7f396a415700 1 -- 172.21.15.92:0/3511088682 wait complete. 2023-05-16T07:03:06.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.451+0000 7f396a415700 1 Processor -- start 2023-05-16T07:03:06.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 -- start start 2023-05-16T07:03:06.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39640696a0 0x7f39640c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f3964164600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3964154550 0x7f39641418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3964154a90 con 0x7f3964154170 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39641657b0 con 0x7f3964154550 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.452+0000 7f396a415700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396414dee0 con 0x7f39640696a0 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f3964164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f3964164600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41960/0 (socket says 172.21.15.92:41960) 2023-05-16T07:03:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 -- 172.21.15.92:0/1135390302 learned_addr learned my addr 172.21.15.92:0/1135390302 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f3963fff700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39640696a0 0x7f39640c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39689b2700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3964154550 0x7f39641418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 -- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39640696a0 msgr2=0x7f39640c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39640696a0 0x7f39640c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 -- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3964154550 msgr2=0x7f39641418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3964154550 0x7f39641418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 -- 172.21.15.92:0/1135390302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39540092a0 con 0x7f3964154170 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f3963fff700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39640696a0 0x7f39640c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39689b2700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3964154550 0x7f39641418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f39637fe700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f3964164600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f3958006980 tx=0x7f3958008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580125f0 con 0x7f3964154170 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3958012750 con 0x7f3964154170 2023-05-16T07:03:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3964164c90 con 0x7f3964154170 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958012af0 con 0x7f3964154170 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.453+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3964165250 con 0x7f3964154170 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.454+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3964154c00 con 0x7f3964154170 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.454+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f3958013480 con 0x7f3964154170 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.455+0000 7f394bfff700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f395005ed90 0x7f3950061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.455+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f3958079580 con 0x7f3964154170 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.455+0000 7f3963fff700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f395005ed90 0x7f3950061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.456+0000 7f3963fff700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f395005ed90 0x7f3950061250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f395400b340 tx=0x7f3954017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.459+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3964154c00 con 0x7f3964154170 2023-05-16T07:03:06.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.623+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39640666c0 con 0x7f3964154170 2023-05-16T07:03:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.623+0000 7f394bfff700 1 -- 172.21.15.92:0/1135390302 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13760 (secure 0 0 0) 0x7f395804ae10 con 0x7f3964154170 2023-05-16T07:03:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:06.628 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:03:03.727145+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:03:06.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f395005ed90 msgr2=0x7f3950061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f395005ed90 0x7f3950061250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f395400b340 tx=0x7f3954017000).stop 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 msgr2=0x7f3964164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f3964164600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f3958006980 tx=0x7f3958008040).stop 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 shutdown_connections 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f395005ed90 0x7f3950061250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39640696a0 0x7f39640c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3964154550 0x7f39641418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 --2- 172.21.15.92:0/1135390302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3964154170 0x7f3964164600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 >> 172.21.15.92:0/1135390302 conn(0x7f39640cb700 msgr2=0x7f39640acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 shutdown_connections 2023-05-16T07:03:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.625+0000 7f396a415700 1 -- 172.21.15.92:0/1135390302 wait complete. 2023-05-16T07:03:06.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:03:06.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:03:06.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.752+0000 7f77ed198700 1 Processor -- start 2023-05-16T07:03:06.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.752+0000 7f77ed198700 1 -- start start 2023-05-16T07:03:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77ed198700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 0x7f77e8155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77ed198700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 0x7f77e8063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77ed198700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e814ead0 con 0x7f77e80c4d40 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77ed198700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e8154cb0 con 0x7f77e814e6f0 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77ed198700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c6020 con 0x7f77e80629c0 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e6d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 0x7f77e8155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e759e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 0x7f77e8063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e6d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 0x7f77e8155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:45602/0 (socket says 172.21.15.92:45602) 2023-05-16T07:03:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e759e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 0x7f77e8063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:34542/0 (socket says 172.21.15.92:34542) 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e6d9d700 1 -- 172.21.15.92:0/543122042 learned_addr learned my addr 172.21.15.92:0/543122042 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e659c700 1 --1- 172.21.15.92:0/543122042 >> v1:172.21.15.158:6789/0 conn(0x7f77e80c4d40 0x7f77e80622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:56314/0 (socket says 172.21.15.92:56314) 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e759e700 1 -- 172.21.15.92:0/543122042 >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 msgr2=0x7f77e8155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.753+0000 7f77e759e700 1 --2- 172.21.15.92:0/543122042 >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 0x7f77e8155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e759e700 1 -- 172.21.15.92:0/543122042 >> v1:172.21.15.158:6789/0 conn(0x7f77e80c4d40 legacy=0x7f77e80622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e5d9b700 1 -- 172.21.15.92:0/543122042 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005991300 0 0) 0x7f77e814ead0 con 0x7f77e80c4d40 2023-05-16T07:03:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e759e700 1 -- 172.21.15.92:0/543122042 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8154660 con 0x7f77e80629c0 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e6d9d700 1 --2- 172.21.15.92:0/543122042 >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 0x7f77e8155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e759e700 1 --2- 172.21.15.92:0/543122042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 0x7f77e8063600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f77e0009760 tx=0x7f77e0009060).ready entity=mon.2 client_cookie=7612afdab165a7aa server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e5d9b700 1 -- 172.21.15.92:0/543122042 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000f510 con 0x7f77e80629c0 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e5d9b700 1 -- 172.21.15.92:0/543122042 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e000f670 con 0x7f77e80629c0 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77e5d9b700 1 -- 172.21.15.92:0/543122042 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000f9f0 con 0x7f77e80629c0 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 -- 172.21.15.92:0/543122042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 msgr2=0x7f77e8063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 --2- 172.21.15.92:0/543122042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 0x7f77e8063600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f77e0009760 tx=0x7f77e0009060).stop 2023-05-16T07:03:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 -- 172.21.15.92:0/543122042 shutdown_connections 2023-05-16T07:03:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 --2- 172.21.15.92:0/543122042 >> v2:172.21.15.158:3300/0 conn(0x7f77e814e6f0 0x7f77e8155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 --2- 172.21.15.92:0/543122042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e80629c0 0x7f77e8063600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 -- 172.21.15.92:0/543122042 >> 172.21.15.92:0/543122042 conn(0x7f77e805e820 msgr2=0x7f77e80c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 -- 172.21.15.92:0/543122042 shutdown_connections 2023-05-16T07:03:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.754+0000 7f77ed198700 1 -- 172.21.15.92:0/543122042 wait complete. 2023-05-16T07:03:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 Processor -- start 2023-05-16T07:03:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 -- start start 2023-05-16T07:03:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 0x7f77e81648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77e80c4d40 0x7f77e8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e814e6f0 0x7f77e80a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8164dc0 con 0x7f77e80629c0 2023-05-16T07:03:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8164fa0 con 0x7f77e80c4d40 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77ed198700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8165280 con 0x7f77e814e6f0 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 0x7f77e81648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 0x7f77e81648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41972/0 (socket says 172.21.15.92:41972) 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.756+0000 7f77e6d9d700 1 -- 172.21.15.92:0/3680282702 learned_addr learned my addr 172.21.15.92:0/3680282702 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e759e700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e814e6f0 0x7f77e80a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e659c700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77e80c4d40 0x7f77e8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e6d9d700 1 -- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e814e6f0 msgr2=0x7f77e80a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e6d9d700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e814e6f0 0x7f77e80a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e6d9d700 1 -- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77e80c4d40 msgr2=0x7f77e8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e6d9d700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77e80c4d40 0x7f77e8150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e6d9d700 1 -- 172.21.15.92:0/3680282702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8009ae0 con 0x7f77e80629c0 2023-05-16T07:03:06.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77e6d9d700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 0x7f77e81648d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f77d8009fc0 tx=0x7f77d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800c9c0 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d800f040 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800cd30 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e00092a0 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.757+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e81658c0 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.758+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e806c350 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.758+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f77d801b030 con 0x7f77e80629c0 2023-05-16T07:03:06.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.759+0000 7f77ceffd700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77d405ed90 0x7f77d4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:06.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.759+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f77d80795b0 con 0x7f77e80629c0 2023-05-16T07:03:06.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.759+0000 7f77e659c700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77d405ed90 0x7f77d4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:06.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.759+0000 7f77e659c700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77d405ed90 0x7f77d4061250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f77e81563d0 tx=0x7f77dc007bd0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:06.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.763+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f77d804af50 con 0x7f77e80629c0 2023-05-16T07:03:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.923+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f77e8064830 con 0x7f77e80629c0 2023-05-16T07:03:06.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.924+0000 7f77ceffd700 1 -- 172.21.15.92:0/3680282702 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v30) v1 ==== 110+0+10 (secure 0 0 0) 0x7f77d804e570 con 0x7f77e80629c0 2023-05-16T07:03:06.928 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:03:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77d405ed90 msgr2=0x7f77d4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77d405ed90 0x7f77d4061250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f77e81563d0 tx=0x7f77dc007bd0).stop 2023-05-16T07:03:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 msgr2=0x7f77e81648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 0x7f77e81648d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f77d8009fc0 tx=0x7f77d800b040).stop 2023-05-16T07:03:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 shutdown_connections 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77d405ed90 0x7f77d4061250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77e814e6f0 0x7f77e80a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e80629c0 0x7f77e81648d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 --2- 172.21.15.92:0/3680282702 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77e80c4d40 0x7f77e8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 >> 172.21.15.92:0/3680282702 conn(0x7f77e805e820 msgr2=0x7f77e814dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 shutdown_connections 2023-05-16T07:03:06.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:06.926+0000 7f77ed198700 1 -- 172.21.15.92:0/3680282702 wait complete. 2023-05-16T07:03:06.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:03:07.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.049+0000 7f972ba9b700 1 Processor -- start 2023-05-16T07:03:07.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.049+0000 7f972ba9b700 1 -- start start 2023-05-16T07:03:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f972ba9b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f972414fb70 0x7f972415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f972ba9b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724064260 con 0x7f9724156050 2023-05-16T07:03:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f972ba9b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97240683e0 con 0x7f9724155c70 2023-05-16T07:03:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f972ba9b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240689a0 con 0x7f972414fb70 2023-05-16T07:03:07.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f9729036700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9724156050 0x7f972414f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52730/0 (socket says 172.21.15.92:52730) 2023-05-16T07:03:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f9729036700 1 -- 172.21.15.92:0/941157175 learned_addr learned my addr 172.21.15.92:0/941157175 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.050+0000 7f972a038700 1 --2- 172.21.15.92:0/941157175 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f972414fb70 0x7f972415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f9728835700 1 -- 172.21.15.92:0/941157175 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697649563 0 0) 0x7f9724064260 con 0x7f9724156050 2023-05-16T07:03:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f9728835700 1 -- 172.21.15.92:0/941157175 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f970c0033f0 con 0x7f9724156050 2023-05-16T07:03:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f9728835700 1 -- 172.21.15.92:0/941157175 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1415983075 0 0) 0x7f970c0033f0 con 0x7f9724156050 2023-05-16T07:03:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f972a038700 1 -- 172.21.15.92:0/941157175 >> v1:172.21.15.158:6789/0 conn(0x7f9724155c70 legacy=0x7f9724069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f972a038700 1 -- 172.21.15.92:0/941157175 >> v1:172.21.15.92:6789/0 conn(0x7f9724156050 legacy=0x7f972414f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f972a038700 1 -- 172.21.15.92:0/941157175 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240686c0 con 0x7f972414fb70 2023-05-16T07:03:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f972a038700 1 --2- 172.21.15.92:0/941157175 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f972414fb70 0x7f972415ce50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f971c009790 tx=0x7f971c009060).ready entity=mon.2 client_cookie=697d66b1a9ce3842 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f9728835700 1 -- 172.21.15.92:0/941157175 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718001470 con 0x7f972414fb70 2023-05-16T07:03:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f9728835700 1 -- 172.21.15.92:0/941157175 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c012410 con 0x7f972414fb70 2023-05-16T07:03:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.051+0000 7f9728835700 1 -- 172.21.15.92:0/941157175 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c012790 con 0x7f972414fb70 2023-05-16T07:03:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 -- 172.21.15.92:0/941157175 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f972414fb70 msgr2=0x7f972415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 --2- 172.21.15.92:0/941157175 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f972414fb70 0x7f972415ce50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f971c009790 tx=0x7f971c009060).stop 2023-05-16T07:03:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 -- 172.21.15.92:0/941157175 shutdown_connections 2023-05-16T07:03:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 --2- 172.21.15.92:0/941157175 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f972414fb70 0x7f972415ce50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 -- 172.21.15.92:0/941157175 >> 172.21.15.92:0/941157175 conn(0x7f97240c3d90 msgr2=0x7f97240c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 -- 172.21.15.92:0/941157175 shutdown_connections 2023-05-16T07:03:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.052+0000 7f972ba9b700 1 -- 172.21.15.92:0/941157175 wait complete. 2023-05-16T07:03:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.053+0000 7f972ba9b700 1 Processor -- start 2023-05-16T07:03:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.053+0000 7f972ba9b700 1 -- start start 2023-05-16T07:03:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f972414fb70 0x7f9724140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 0x7f972414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9724156050 0x7f97240bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97241564b0 con 0x7f9724155c70 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972413ad80 con 0x7f972414fb70 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9724140d50 con 0x7f9724156050 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 0x7f972414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 0x7f972414bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41988/0 (socket says 172.21.15.92:41988) 2023-05-16T07:03:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 -- 172.21.15.92:0/937935674 learned_addr learned my addr 172.21.15.92:0/937935674 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972a038700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9724156050 0x7f97240bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729837700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f972414fb70 0x7f9724140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 -- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9724156050 msgr2=0x7f97240bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9724156050 0x7f97240bfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 -- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f972414fb70 msgr2=0x7f9724140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f972414fb70 0x7f9724140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 -- 172.21.15.92:0/937935674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0092a0 con 0x7f9724155c70 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9729036700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 0x7f972414bfe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f971800c810 tx=0x7f9718001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718007170 con 0x7f9724155c70 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97180072f0 con 0x7f9724155c70 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180048c0 con 0x7f9724155c70 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97240be9c0 con 0x7f9724155c70 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.054+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97240bef80 con 0x7f9724155c70 2023-05-16T07:03:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.055+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972406c350 con 0x7f9724155c70 2023-05-16T07:03:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.056+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9718010070 con 0x7f9724155c70 2023-05-16T07:03:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.056+0000 7f9721ffb700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f971005ed90 0x7f9710061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.056+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9718079110 con 0x7f9724155c70 2023-05-16T07:03:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.056+0000 7f9729837700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f971005ed90 0x7f9710061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.057+0000 7f9729837700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f971005ed90 0x7f9710061250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f9714007fd0 tx=0x7f9714009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.060+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f971804aa00 con 0x7f9724155c70 2023-05-16T07:03:07.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.221+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f97240c7240 con 0x7f9724155c70 2023-05-16T07:03:07.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.222+0000 7f9721ffb700 1 -- 172.21.15.92:0/937935674 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9718019030 con 0x7f9724155c70 2023-05-16T07:03:07.226 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:03:07.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.223+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f971005ed90 msgr2=0x7f9710061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.223+0000 7f972ba9b700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f971005ed90 0x7f9710061250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f9714007fd0 tx=0x7f9714009040).stop 2023-05-16T07:03:07.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.223+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 msgr2=0x7f972414bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.223+0000 7f972ba9b700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 0x7f972414bfe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f971800c810 tx=0x7f9718001ab0).stop 2023-05-16T07:03:07.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 shutdown_connections 2023-05-16T07:03:07.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f971005ed90 0x7f9710061250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9724156050 0x7f97240bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f972414fb70 0x7f9724140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 --2- 172.21.15.92:0/937935674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724155c70 0x7f972414bfe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 >> 172.21.15.92:0/937935674 conn(0x7f97240c3d90 msgr2=0x7f97240c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 shutdown_connections 2023-05-16T07:03:07.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.224+0000 7f972ba9b700 1 -- 172.21.15.92:0/937935674 wait complete. 2023-05-16T07:03:07.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:03:07.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.346+0000 7f1d0197c700 1 Processor -- start 2023-05-16T07:03:07.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.346+0000 7f1d0197c700 1 -- start start 2023-05-16T07:03:07.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1d0197c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1cfc154170 0x7f1cfc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1d0197c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154550 0x7f1cfc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1d0197c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cfc0696a0 0x7f1cfc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1d0197c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0c0660 con 0x7f1cfc0696a0 2023-05-16T07:03:07.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1d0197c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc06a020 con 0x7f1cfc154170 2023-05-16T07:03:07.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1d0197c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc0bfd30 con 0x7f1cfc154550 2023-05-16T07:03:07.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cfc0696a0 0x7f1cfc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cfc0696a0 0x7f1cfc069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41998/0 (socket says 172.21.15.92:41998) 2023-05-16T07:03:07.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/223492582 learned_addr learned my addr 172.21.15.92:0/223492582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfaffd700 1 --2- 172.21.15.92:0/223492582 >> v2:172.21.15.158:3300/0 conn(0x7f1cfc154170 0x7f1cfc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfa7fc700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154550 0x7f1cfc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/223492582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154550 msgr2=0x7f1cfc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154550 0x7f1cfc0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/223492582 >> v2:172.21.15.158:3300/0 conn(0x7f1cfc154170 msgr2=0x7f1cfc0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 --2- 172.21.15.92:0/223492582 >> v2:172.21.15.158:3300/0 conn(0x7f1cfc154170 0x7f1cfc0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/223492582 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0c0010 con 0x7f1cfc0696a0 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfaffd700 1 --2- 172.21.15.92:0/223492582 >> v2:172.21.15.158:3300/0 conn(0x7f1cfc154170 0x7f1cfc0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.347+0000 7f1cfa7fc700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154550 0x7f1cfc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1cfb7fe700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc069b30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f1cfc083030 tx=0x7f1cf40090f0).ready entity=mon.0 client_cookie=5b9f2a7a6ec05b7d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1cf9ffb700 1 -- 172.21.15.92:0/223492582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400b600 con 0x7f1cfc0696a0 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1cf9ffb700 1 -- 172.21.15.92:0/223492582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400b760 con 0x7f1cfc0696a0 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1cf9ffb700 1 -- 172.21.15.92:0/223492582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400bb00 con 0x7f1cfc0696a0 2023-05-16T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 -- 172.21.15.92:0/223492582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc0696a0 msgr2=0x7f1cfc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc069b30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f1cfc083030 tx=0x7f1cf40090f0).stop 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 -- 172.21.15.92:0/223492582 shutdown_connections 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc069b30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 --2- 172.21.15.92:0/223492582 >> v2:172.21.15.158:3300/0 conn(0x7f1cfc154170 0x7f1cfc0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 --2- 172.21.15.92:0/223492582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154550 0x7f1cfc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 -- 172.21.15.92:0/223492582 >> 172.21.15.92:0/223492582 conn(0x7f1cfc0cb700 msgr2=0x7f1cfc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 -- 172.21.15.92:0/223492582 shutdown_connections 2023-05-16T07:03:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.348+0000 7f1d0197c700 1 -- 172.21.15.92:0/223492582 wait complete. 2023-05-16T07:03:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 Processor -- start 2023-05-16T07:03:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 -- start start 2023-05-16T07:03:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154170 0x7f1cfc150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 0x7f1cfc152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc153880 con 0x7f1cfc154550 2023-05-16T07:03:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc152c70 con 0x7f1cfc0696a0 2023-05-16T07:03:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1d0197c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc152f50 con 0x7f1cfc154170 2023-05-16T07:03:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1cfb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 0x7f1cfc152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1cfb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 0x7f1cfc152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42002/0 (socket says 172.21.15.92:42002) 2023-05-16T07:03:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/76260107 learned_addr learned my addr 172.21.15.92:0/76260107 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1cfa7fc700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154170 0x7f1cfc150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.350+0000 7f1cfaffd700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154170 msgr2=0x7f1cfc150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cfb7fe700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154170 0x7f1cfc150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1cfc0696a0 msgr2=0x7f1cfc14cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cfb7fe700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc14cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cfb7fe700 1 -- 172.21.15.92:0/76260107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009370 con 0x7f1cfc154550 2023-05-16T07:03:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cfb7fe700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 0x7f1cfc152270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f1cf4006fd0 tx=0x7f1cf4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4006d90 con 0x7f1cfc154550 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400bec0 con 0x7f1cfc154550 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40246c0 con 0x7f1cfc154550 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc153230 con 0x7f1cfc154550 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.351+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc150d60 con 0x7f1cfc154550 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.352+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc0666c0 con 0x7f1cfc154550 2023-05-16T07:03:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.352+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1cf400f070 con 0x7f1cfc154550 2023-05-16T07:03:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.353+0000 7f1cf2ffd700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cdc05ed90 0x7f1cdc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.353+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1cf407b300 con 0x7f1cfc154550 2023-05-16T07:03:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.353+0000 7f1cfaffd700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cdc05ed90 0x7f1cdc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.353+0000 7f1cfaffd700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cdc05ed90 0x7f1cdc061250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f1cec007740 tx=0x7f1cec0099d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.357+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1cf404cbf0 con 0x7f1cfc154550 2023-05-16T07:03:07.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.521+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1cfc1539c0 con 0x7f1cfc154550 2023-05-16T07:03:07.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.522+0000 7f1cf2ffd700 1 -- 172.21.15.92:0/76260107 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1cfc1539c0 con 0x7f1cfc154550 2023-05-16T07:03:07.527 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:07.527 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:03:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:03:07.528 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:03:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:03:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:03:07.529 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:03:07.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:07.530 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:03:07.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cdc05ed90 msgr2=0x7f1cdc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cdc05ed90 0x7f1cdc061250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f1cec007740 tx=0x7f1cec0099d0).stop 2023-05-16T07:03:07.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 msgr2=0x7f1cfc152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 0x7f1cfc152270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f1cf4006fd0 tx=0x7f1cf4017000).stop 2023-05-16T07:03:07.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 shutdown_connections 2023-05-16T07:03:07.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1cdc05ed90 0x7f1cdc061250 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cfc154550 0x7f1cfc152270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1cfc0696a0 0x7f1cfc14cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 --2- 172.21.15.92:0/76260107 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1cfc154170 0x7f1cfc150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.523+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 >> 172.21.15.92:0/76260107 conn(0x7f1cfc0cb700 msgr2=0x7f1cfc0c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.524+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 shutdown_connections 2023-05-16T07:03:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.524+0000 7f1d0197c700 1 -- 172.21.15.92:0/76260107 wait complete. 2023-05-16T07:03:07.539 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:03:07.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:03:07.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.651+0000 7fc8cf828700 1 Processor -- start 2023-05-16T07:03:07.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.651+0000 7fc8cf828700 1 -- start start 2023-05-16T07:03:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cf828700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8c8154090 0x7fc8c80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cf828700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc8c8154470 0x7fc8c80a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cf828700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c80c94d0 con 0x7fc8c80aa150 2023-05-16T07:03:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cf828700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c80aa5a0 con 0x7fc8c8154090 2023-05-16T07:03:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cf828700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c80cb7e0 con 0x7fc8c8154470 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8c8154090 0x7fc8c80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8c8154090 0x7fc8c80aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42008/0 (socket says 172.21.15.92:42008) 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 -- 172.21.15.92:0/2289422959 learned_addr learned my addr 172.21.15.92:0/2289422959 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cddc5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc8c80aa150 0x7fc8c8068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52760/0 (socket says 172.21.15.92:52760) 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8ccdc3700 1 --2- 172.21.15.92:0/2289422959 >> v2:172.21.15.158:3300/0 conn(0x7fc8c8154470 0x7fc8c80a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 -- 172.21.15.92:0/2289422959 >> v2:172.21.15.158:3300/0 conn(0x7fc8c8154470 msgr2=0x7fc8c80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 --2- 172.21.15.92:0/2289422959 >> v2:172.21.15.158:3300/0 conn(0x7fc8c8154470 0x7fc8c80a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 -- 172.21.15.92:0/2289422959 >> v1:172.21.15.92:6789/0 conn(0x7fc8c80aa150 legacy=0x7fc8c8068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8cd5c4700 1 -- 172.21.15.92:0/2289422959 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80c88c0 con 0x7fc8c8154090 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.652+0000 7fc8c7fff700 1 -- 172.21.15.92:0/2289422959 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307663137 0 0) 0x7fc8c80c94d0 con 0x7fc8c80aa150 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cd5c4700 1 --2- 172.21.15.92:0/2289422959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c80aaca0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc8c8084320 tx=0x7fc8b8009840).ready entity=mon.0 client_cookie=196dab1ff84532fc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8c7fff700 1 -- 172.21.15.92:0/2289422959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800e070 con 0x7fc8c8154090 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8c7fff700 1 -- 172.21.15.92:0/2289422959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800ba50 con 0x7fc8c8154090 2023-05-16T07:03:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8c7fff700 1 -- 172.21.15.92:0/2289422959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800bdd0 con 0x7fc8c8154090 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 -- 172.21.15.92:0/2289422959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 msgr2=0x7fc8c80aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 --2- 172.21.15.92:0/2289422959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c80aaca0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc8c8084320 tx=0x7fc8b8009840).stop 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 -- 172.21.15.92:0/2289422959 shutdown_connections 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 --2- 172.21.15.92:0/2289422959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c80aaca0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 --2- 172.21.15.92:0/2289422959 >> v2:172.21.15.158:3300/0 conn(0x7fc8c8154470 0x7fc8c80a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 -- 172.21.15.92:0/2289422959 >> 172.21.15.92:0/2289422959 conn(0x7fc8c8067920 msgr2=0x7fc8c80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 -- 172.21.15.92:0/2289422959 shutdown_connections 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.653+0000 7fc8cf828700 1 -- 172.21.15.92:0/2289422959 wait complete. 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.654+0000 7fc8cf828700 1 Processor -- start 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.654+0000 7fc8cf828700 1 -- start start 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8cf828700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8c80aa150 0x7fc8c814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8cf828700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8cf828700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8c8154470 0x7fc8c80af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8cf828700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80ae5b0 con 0x7fc8c8154090 2023-05-16T07:03:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8cf828700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c8156af0 con 0x7fc8c8154470 2023-05-16T07:03:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8cf828700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c814c360 con 0x7fc8c80aa150 2023-05-16T07:03:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8ccdc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8ccdc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42016/0 (socket says 172.21.15.92:42016) 2023-05-16T07:03:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.655+0000 7fc8ccdc3700 1 -- 172.21.15.92:0/4289441481 learned_addr learned my addr 172.21.15.92:0/4289441481 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8cd5c4700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8c80aa150 0x7fc8c814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8cddc5700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8c8154470 0x7fc8c80af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8ccdc3700 1 -- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8c80aa150 msgr2=0x7fc8c814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8ccdc3700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8c80aa150 0x7fc8c814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8ccdc3700 1 -- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8c8154470 msgr2=0x7fc8c80af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8ccdc3700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8c8154470 0x7fc8c80af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8ccdc3700 1 -- 172.21.15.92:0/4289441481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8009ae0 con 0x7fc8c8154090 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8ccdc3700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c814be70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fc8bc00ae50 tx=0x7fc8bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007a20 con 0x7fc8c8154090 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc007b80 con 0x7fc8c8154090 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c80af0c0 con 0x7fc8c8154090 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc01d440 con 0x7fc8c8154090 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.656+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c80ae810 con 0x7fc8c8154090 2023-05-16T07:03:07.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.657+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c80af3a0 con 0x7fc8c8154090 2023-05-16T07:03:07.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.657+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc8bc01d870 con 0x7fc8c8154090 2023-05-16T07:03:07.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.658+0000 7fc8c57fa700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8b405ed90 0x7fc8b4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:07.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.658+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc8bc081fa0 con 0x7fc8c8154090 2023-05-16T07:03:07.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.658+0000 7fc8cd5c4700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8b405ed90 0x7fc8b4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:07.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.658+0000 7fc8cd5c4700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8b405ed90 0x7fc8b4061250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fc8b800ba20 tx=0x7fc8b8009840).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:07.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.662+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc8bc053890 con 0x7fc8c8154090 2023-05-16T07:03:07.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:07.824+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fc8c8066730 con 0x7fc8c8154090 2023-05-16T07:03:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.572+0000 7fc8c57fa700 1 -- 172.21.15.92:0/4289441481 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 152+0+0 (secure 0 0 0) 0x7fc8bc056eb0 con 0x7fc8c8154090 2023-05-16T07:03:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8b405ed90 msgr2=0x7fc8b4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8b405ed90 0x7fc8b4061250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fc8b800ba20 tx=0x7fc8b8009840).stop 2023-05-16T07:03:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 msgr2=0x7fc8c814be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c814be70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fc8bc00ae50 tx=0x7fc8bc008040).stop 2023-05-16T07:03:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 shutdown_connections 2023-05-16T07:03:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8b405ed90 0x7fc8b4061250 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0x7fc8b800ba20 tx=0x7fc8b8009840).stop 2023-05-16T07:03:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8c80aa150 0x7fc8c814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8c8154470 0x7fc8c80af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 --2- 172.21.15.92:0/4289441481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8154090 0x7fc8c814be70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.576+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 >> 172.21.15.92:0/4289441481 conn(0x7fc8c8067920 msgr2=0x7fc8c814e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.577+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 shutdown_connections 2023-05-16T07:03:08.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.577+0000 7fc8cf828700 1 -- 172.21.15.92:0/4289441481 wait complete. 2023-05-16T07:03:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:03:08.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:03:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.714+0000 7fa2da20c700 1 Processor -- start 2023-05-16T07:03:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.714+0000 7fa2da20c700 1 -- start start 2023-05-16T07:03:08.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2da20c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa2d40c1160 0x7fa2d40c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:08.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2da20c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d4154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:08.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2da20c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d40c6640 con 0x7fa2d4153f90 2023-05-16T07:03:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2da20c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d4069f60 con 0x7fa2d40c1160 2023-05-16T07:03:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2da20c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d406a9e0 con 0x7fa2d40c61f0 2023-05-16T07:03:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2d8a09700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa2d40c1160 0x7fa2d40c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2d9a0b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d4154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2d8a09700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa2d40c1160 0x7fa2d40c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:45662/0 (socket says 172.21.15.92:45662) 2023-05-16T07:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2d8a09700 1 -- 172.21.15.92:0/1057288706 learned_addr learned my addr 172.21.15.92:0/1057288706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2d9a0b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d4154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:34596/0 (socket says 172.21.15.92:34596) 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.715+0000 7fa2d920a700 1 --1- 172.21.15.92:0/1057288706 >> v1:172.21.15.158:6789/0 conn(0x7fa2d4153f90 0x7fa2d40a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:56370/0 (socket says 172.21.15.92:56370) 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d9a0b700 1 -- 172.21.15.92:0/1057288706 >> v2:172.21.15.158:3300/0 conn(0x7fa2d40c1160 msgr2=0x7fa2d40c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d9a0b700 1 --2- 172.21.15.92:0/1057288706 >> v2:172.21.15.158:3300/0 conn(0x7fa2d40c1160 0x7fa2d40c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d9a0b700 1 -- 172.21.15.92:0/1057288706 >> v1:172.21.15.158:6789/0 conn(0x7fa2d4153f90 legacy=0x7fa2d40a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d9a0b700 1 -- 172.21.15.92:0/1057288706 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d406a140 con 0x7fa2d40c61f0 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d9a0b700 1 --2- 172.21.15.92:0/1057288706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d4154370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa2cc009760 tx=0x7fa2cc009060).ready entity=mon.2 client_cookie=9515edf3511cc75b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d3fff700 1 -- 172.21.15.92:0/1057288706 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00f510 con 0x7fa2d40c61f0 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d3fff700 1 -- 172.21.15.92:0/1057288706 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00f670 con 0x7fa2d40c61f0 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2d3fff700 1 -- 172.21.15.92:0/1057288706 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00f9f0 con 0x7fa2d40c61f0 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2da20c700 1 -- 172.21.15.92:0/1057288706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 msgr2=0x7fa2d4154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.716+0000 7fa2da20c700 1 --2- 172.21.15.92:0/1057288706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d4154370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa2cc009760 tx=0x7fa2cc009060).stop 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.717+0000 7fa2da20c700 1 -- 172.21.15.92:0/1057288706 shutdown_connections 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.717+0000 7fa2da20c700 1 --2- 172.21.15.92:0/1057288706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d4154370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.717+0000 7fa2da20c700 1 --2- 172.21.15.92:0/1057288706 >> v2:172.21.15.158:3300/0 conn(0x7fa2d40c1160 0x7fa2d40c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.717+0000 7fa2da20c700 1 -- 172.21.15.92:0/1057288706 >> 172.21.15.92:0/1057288706 conn(0x7fa2d40cb600 msgr2=0x7fa2d40c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.717+0000 7fa2da20c700 1 -- 172.21.15.92:0/1057288706 shutdown_connections 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.717+0000 7fa2da20c700 1 -- 172.21.15.92:0/1057288706 wait complete. 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.718+0000 7fa2da20c700 1 Processor -- start 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.718+0000 7fa2da20c700 1 -- start start 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 0x7fa2d40c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d414db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d4153f90 0x7fa2d41477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40adae0 con 0x7fa2d40c1160 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40adcf0 con 0x7fa2d4153f90 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40adfd0 con 0x7fa2d40c61f0 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 0x7fa2d40c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 0x7fa2d40c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42028/0 (socket says 172.21.15.92:42028) 2023-05-16T07:03:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 -- 172.21.15.92:0/3280132182 learned_addr learned my addr 172.21.15.92:0/3280132182 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d8a09700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d9a0b700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d4153f90 0x7fa2d41477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 -- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 msgr2=0x7fa2d414db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d414db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 -- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d4153f90 msgr2=0x7fa2d41477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d4153f90 0x7fa2d41477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:08.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 -- 172.21.15.92:0/3280132182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0092a0 con 0x7fa2d40c1160 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d8a09700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d414db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d9a0b700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d4153f90 0x7fa2d41477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d920a700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 0x7fa2d40c4660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fa2c400c7c0 tx=0x7fa2c4001570).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4005c70 con 0x7fa2d40c1160 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4005dd0 con 0x7fa2d40c1160 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4007320 con 0x7fa2d40c1160 2023-05-16T07:03:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d40ae2b0 con 0x7fa2d40c1160 2023-05-16T07:03:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.719+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d414e050 con 0x7fa2d40c1160 2023-05-16T07:03:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.720+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d4083120 con 0x7fa2d40c1160 2023-05-16T07:03:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.721+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa2c4010030 con 0x7fa2d40c1160 2023-05-16T07:03:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.721+0000 7fa2d17fa700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c005edc0 0x7fa2c0061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.721+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fa2c4079860 con 0x7fa2d40c1160 2023-05-16T07:03:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.721+0000 7fa2d8a09700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c005edc0 0x7fa2c0061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.722+0000 7fa2d8a09700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c005edc0 0x7fa2c0061280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa2d40c4b50 tx=0x7fa2c8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.725+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa2c404afe0 con 0x7fa2d40c1160 2023-05-16T07:03:08.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:08.887+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa2d4047150 con 0x7fa2d40c1160 2023-05-16T07:03:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.576+0000 7fa2d17fa700 1 -- 172.21.15.92:0/3280132182 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa2c404e600 con 0x7fa2d40c1160 2023-05-16T07:03:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.585+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c005edc0 msgr2=0x7fa2c0061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.585+0000 7fa2da20c700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c005edc0 0x7fa2c0061280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa2d40c4b50 tx=0x7fa2c8008040).stop 2023-05-16T07:03:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.585+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 msgr2=0x7fa2d40c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.585+0000 7fa2da20c700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 0x7fa2d40c4660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fa2c400c7c0 tx=0x7fa2c4001570).stop 2023-05-16T07:03:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.586+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 shutdown_connections 2023-05-16T07:03:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.586+0000 7fa2da20c700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c005edc0 0x7fa2c0061280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.586+0000 7fa2da20c700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d4153f90 0x7fa2d41477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.586+0000 7fa2da20c700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d40c1160 0x7fa2d40c4660 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0x7fa2c400c7c0 tx=0x7fa2c4001570).stop 2023-05-16T07:03:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.586+0000 7fa2da20c700 1 --2- 172.21.15.92:0/3280132182 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d40c61f0 0x7fa2d414db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.586+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 >> 172.21.15.92:0/3280132182 conn(0x7fa2d40cb600 msgr2=0x7fa2d40ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:09.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.589+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 shutdown_connections 2023-05-16T07:03:09.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.591+0000 7fa2da20c700 1 -- 172.21.15.92:0/3280132182 wait complete. 2023-05-16T07:03:09.599 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:03:09.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:03:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.725+0000 7fd6a012f700 1 Processor -- start 2023-05-16T07:03:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.725+0000 7fd6a012f700 1 -- start start 2023-05-16T07:03:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd6a012f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd6a012f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd698154470 0x7fd6980a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd6a012f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980aa570 con 0x7fd6980aa150 2023-05-16T07:03:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd6a012f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980c94d0 con 0x7fd698154470 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd6a012f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980cb7e0 con 0x7fd698154090 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd698154470 0x7fd6980a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd698154470 0x7fd6980a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42040/0 (socket says 172.21.15.92:42040) 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 -- 172.21.15.92:0/2920619132 learned_addr learned my addr 172.21.15.92:0/2920619132 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 -- 172.21.15.92:0/2920619132 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 msgr2=0x7fd6980aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 --2- 172.21.15.92:0/2920619132 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 -- 172.21.15.92:0/2920619132 >> v1:172.21.15.158:6789/0 conn(0x7fd6980aa150 legacy=0x7fd698068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.726+0000 7fd69d6ca700 1 -- 172.21.15.92:0/2920619132 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980c88c0 con 0x7fd698154470 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd69d6ca700 1 --2- 172.21.15.92:0/2920619132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd698154470 0x7fd6980a9c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fd68c006fd0 tx=0x7fd68c0090f0).ready entity=mon.0 client_cookie=eedd73ccde492769 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd69cec9700 1 -- 172.21.15.92:0/2920619132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00b600 con 0x7fd698154470 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd69cec9700 1 -- 172.21.15.92:0/2920619132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00b760 con 0x7fd698154470 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd69cec9700 1 -- 172.21.15.92:0/2920619132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00bae0 con 0x7fd698154470 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 -- 172.21.15.92:0/2920619132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd698154470 msgr2=0x7fd6980a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 --2- 172.21.15.92:0/2920619132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd698154470 0x7fd6980a9c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fd68c006fd0 tx=0x7fd68c0090f0).stop 2023-05-16T07:03:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 -- 172.21.15.92:0/2920619132 shutdown_connections 2023-05-16T07:03:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 --2- 172.21.15.92:0/2920619132 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 --2- 172.21.15.92:0/2920619132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd698154470 0x7fd6980a9c60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 -- 172.21.15.92:0/2920619132 >> 172.21.15.92:0/2920619132 conn(0x7fd698067920 msgr2=0x7fd6980cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 -- 172.21.15.92:0/2920619132 shutdown_connections 2023-05-16T07:03:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.727+0000 7fd6a012f700 1 -- 172.21.15.92:0/2920619132 wait complete. 2023-05-16T07:03:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.728+0000 7fd6a012f700 1 Processor -- start 2023-05-16T07:03:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.728+0000 7fd6a012f700 1 -- start start 2023-05-16T07:03:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd6a012f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 0x7fd6980c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd6a012f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd6a012f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd698154470 0x7fd69814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd6a012f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c3ba0 con 0x7fd6980aa150 2023-05-16T07:03:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd6a012f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c2f90 con 0x7fd698154470 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd6a012f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c3270 con 0x7fd698154090 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 0x7fd6980c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 0x7fd6980c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42044/0 (socket says 172.21.15.92:42044) 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 -- 172.21.15.92:0/3720789519 learned_addr learned my addr 172.21.15.92:0/3720789519 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69d6ca700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69e6cc700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd698154470 0x7fd69814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 -- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 msgr2=0x7fd6980adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 -- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd698154470 msgr2=0x7fd69814c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd698154470 0x7fd69814c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69decb700 1 -- 172.21.15.92:0/3720789519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009370 con 0x7fd6980aa150 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.729+0000 7fd69e6cc700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd698154470 0x7fd69814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.730+0000 7fd69decb700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 0x7fd6980c47f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fd68800bc30 tx=0x7fd688009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.730+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800c6f0 con 0x7fd6980aa150 2023-05-16T07:03:09.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.730+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688010070 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.730+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980c3550 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.730+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800c8b0 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.730+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980c2730 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.731+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6980c3e80 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.731+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd68801c070 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.731+0000 7fd695ffb700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd68405ed90 0x7fd684061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.731+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fd688079200 con 0x7fd6980aa150 2023-05-16T07:03:09.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.732+0000 7fd69d6ca700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd68405ed90 0x7fd684061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:09.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.732+0000 7fd69d6ca700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd68405ed90 0x7fd684061250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fd698158ec0 tx=0x7fd68c017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:09.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.736+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd68804a800 con 0x7fd6980aa150 2023-05-16T07:03:09.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.941+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd698066730 con 0x7fd6980aa150 2023-05-16T07:03:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.948+0000 7fd695ffb700 1 -- 172.21.15.92:0/3720789519 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v10) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd68801a070 con 0x7fd6980aa150 2023-05-16T07:03:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.952+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd68405ed90 msgr2=0x7fd684061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.952+0000 7fd6a012f700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd68405ed90 0x7fd684061250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fd698158ec0 tx=0x7fd68c017000).stop 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.952+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 msgr2=0x7fd6980c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.952+0000 7fd6a012f700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 0x7fd6980c47f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fd68800bc30 tx=0x7fd688009840).stop 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.953+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 shutdown_connections 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.953+0000 7fd6a012f700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd68405ed90 0x7fd684061250 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.953+0000 7fd6a012f700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd698154470 0x7fd69814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.953+0000 7fd6a012f700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6980aa150 0x7fd6980c47f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.953+0000 7fd6a012f700 1 --2- 172.21.15.92:0/3720789519 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd698154090 0x7fd6980adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.953+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 >> 172.21.15.92:0/3720789519 conn(0x7fd698067920 msgr2=0x7fd6980c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.955+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 shutdown_connections 2023-05-16T07:03:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:09.955+0000 7fd6a012f700 1 -- 172.21.15.92:0/3720789519 wait complete. 2023-05-16T07:03:09.966 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 5 and data pool 6 2023-05-16T07:03:09.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:03:10.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.087+0000 7fb29f26a700 1 Processor -- start 2023-05-16T07:03:10.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.087+0000 7fb29f26a700 1 -- start start 2023-05-16T07:03:10.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29f26a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb298154170 0x7fb2980c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29f26a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb298069db0 0x7fb2980c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29f26a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2980c0660 con 0x7fb298154550 2023-05-16T07:03:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29f26a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980c1b00 con 0x7fb298069db0 2023-05-16T07:03:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29f26a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980bfd30 con 0x7fb298154170 2023-05-16T07:03:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb298069db0 0x7fb2980c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb298069db0 0x7fb2980c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42066/0 (socket says 172.21.15.92:42066) 2023-05-16T07:03:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 -- 172.21.15.92:0/3431256497 learned_addr learned my addr 172.21.15.92:0/3431256497 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d006700 1 --2- 172.21.15.92:0/3431256497 >> v2:172.21.15.158:3300/0 conn(0x7fb298154170 0x7fb2980c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 -- 172.21.15.92:0/3431256497 >> v2:172.21.15.158:3300/0 conn(0x7fb298154170 msgr2=0x7fb2980c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 --2- 172.21.15.92:0/3431256497 >> v2:172.21.15.158:3300/0 conn(0x7fb298154170 0x7fb2980c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 -- 172.21.15.92:0/3431256497 >> v1:172.21.15.158:6789/0 conn(0x7fb298154550 legacy=0x7fb2980696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 -- 172.21.15.92:0/3431256497 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980c0010 con 0x7fb298069db0 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.088+0000 7fb29d807700 1 --2- 172.21.15.92:0/3431256497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb2980c6160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fb298083030 tx=0x7fb294009840).ready entity=mon.0 client_cookie=24d4e0c6948f35f5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb28ffff700 1 -- 172.21.15.92:0/3431256497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400e070 con 0x7fb298069db0 2023-05-16T07:03:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb28ffff700 1 -- 172.21.15.92:0/3431256497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29400ba50 con 0x7fb298069db0 2023-05-16T07:03:10.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb28ffff700 1 -- 172.21.15.92:0/3431256497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400bdd0 con 0x7fb298069db0 2023-05-16T07:03:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 -- 172.21.15.92:0/3431256497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 msgr2=0x7fb2980c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3431256497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb2980c6160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fb298083030 tx=0x7fb294009840).stop 2023-05-16T07:03:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 -- 172.21.15.92:0/3431256497 shutdown_connections 2023-05-16T07:03:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3431256497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb2980c6160 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3431256497 >> v2:172.21.15.158:3300/0 conn(0x7fb298154170 0x7fb2980c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 -- 172.21.15.92:0/3431256497 >> 172.21.15.92:0/3431256497 conn(0x7fb2980cb700 msgr2=0x7fb2980c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 -- 172.21.15.92:0/3431256497 shutdown_connections 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.089+0000 7fb29f26a700 1 -- 172.21.15.92:0/3431256497 wait complete. 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.090+0000 7fb29f26a700 1 Processor -- start 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.090+0000 7fb29f26a700 1 -- start start 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29f26a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb298144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29f26a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb298154170 0x7fb2980adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29f26a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb298154550 0x7fb2981464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29f26a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298144970 con 0x7fb298069db0 2023-05-16T07:03:10.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29f26a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980b0d70 con 0x7fb298154550 2023-05-16T07:03:10.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29f26a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb298168da0 con 0x7fb298154170 2023-05-16T07:03:10.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29d006700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb298144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29d006700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb298144480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42070/0 (socket says 172.21.15.92:42070) 2023-05-16T07:03:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29d006700 1 -- 172.21.15.92:0/3565357178 learned_addr learned my addr 172.21.15.92:0/3565357178 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29d807700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb298154550 0x7fb2981464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29c805700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb298154170 0x7fb2980adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.091+0000 7fb29d006700 1 -- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb298154170 msgr2=0x7fb2980adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29d006700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb298154170 0x7fb2980adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29d006700 1 -- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb298154550 msgr2=0x7fb2981464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29d006700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb298154550 0x7fb2981464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29d006700 1 -- 172.21.15.92:0/3565357178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009ae0 con 0x7fb298069db0 2023-05-16T07:03:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29d807700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb298154550 0x7fb2981464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29d006700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb298144480 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb29000d890 tx=0x7fb290008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290011590 con 0x7fb298069db0 2023-05-16T07:03:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2900116f0 con 0x7fb298069db0 2023-05-16T07:03:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2980ae120 con 0x7fb298069db0 2023-05-16T07:03:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290011a70 con 0x7fb298069db0 2023-05-16T07:03:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.092+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb298168500 con 0x7fb298069db0 2023-05-16T07:03:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.093+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2980666c0 con 0x7fb298069db0 2023-05-16T07:03:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.093+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb290012420 con 0x7fb298069db0 2023-05-16T07:03:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.094+0000 7fb28d7fa700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb27c05ed90 0x7fb27c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.094+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb2900796c0 con 0x7fb298069db0 2023-05-16T07:03:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.094+0000 7fb29c805700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb27c05ed90 0x7fb27c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.094+0000 7fb29c805700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb27c05ed90 0x7fb27c061250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb2980af950 tx=0x7fb288007d20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.098+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb29004ac80 con 0x7fb298069db0 2023-05-16T07:03:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.263+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb29806c350 con 0x7fb298069db0 2023-05-16T07:03:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.263+0000 7fb28d7fa700 1 -- 172.21.15.92:0/3565357178 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16989 (secure 0 0 0) 0x7fb29004e2a0 con 0x7fb298069db0 2023-05-16T07:03:10.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:10.268 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:03:09.942932+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:03:07.846932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-05-16T07:03:08.910782+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:03:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.265+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb27c05ed90 msgr2=0x7fb27c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.265+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb27c05ed90 0x7fb27c061250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb2980af950 tx=0x7fb288007d20).stop 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.265+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 msgr2=0x7fb298144480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.265+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb298144480 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb29000d890 tx=0x7fb290008040).stop 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 shutdown_connections 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb27c05ed90 0x7fb27c061250 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb298154550 0x7fb2981464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298069db0 0x7fb298144480 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 --2- 172.21.15.92:0/3565357178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb298154170 0x7fb2980adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 >> 172.21.15.92:0/3565357178 conn(0x7fb2980cb700 msgr2=0x7fb29814e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 shutdown_connections 2023-05-16T07:03:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.266+0000 7fb29f26a700 1 -- 172.21.15.92:0/3565357178 wait complete. 2023-05-16T07:03:10.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:03:10.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.388+0000 7fd527227700 1 Processor -- start 2023-05-16T07:03:10.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.388+0000 7fd527227700 1 -- start start 2023-05-16T07:03:10.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd527227700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd520153db0 0x7fd520069b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd527227700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd520063fe0 con 0x7fd5200c6500 2023-05-16T07:03:10.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd527227700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd520069220 con 0x7fd5201539d0 2023-05-16T07:03:10.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd527227700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52006a000 con 0x7fd520153db0 2023-05-16T07:03:10.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd525a24700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd5200c6500 0x7fd5200c7e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52828/0 (socket says 172.21.15.92:52828) 2023-05-16T07:03:10.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd525a24700 1 -- 172.21.15.92:0/2635371091 learned_addr learned my addr 172.21.15.92:0/2635371091 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:10.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd526a26700 1 --2- 172.21.15.92:0/2635371091 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd520153db0 0x7fd520069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887719595 0 0) 0x7fd520063fe0 con 0x7fd5200c6500 2023-05-16T07:03:10.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5080033f0 con 0x7fd5200c6500 2023-05-16T07:03:10.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889088080 0 0) 0x7fd520069220 con 0x7fd5201539d0 2023-05-16T07:03:10.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.389+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd520063fe0 con 0x7fd5201539d0 2023-05-16T07:03:10.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd526a26700 1 -- 172.21.15.92:0/2635371091 >> v1:172.21.15.158:6789/0 conn(0x7fd5201539d0 legacy=0x7fd5200c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd526a26700 1 -- 172.21.15.92:0/2635371091 >> v1:172.21.15.92:6789/0 conn(0x7fd5200c6500 legacy=0x7fd5200c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4172715607 0 0) 0x7fd5080033f0 con 0x7fd5200c6500 2023-05-16T07:03:10.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd526a26700 1 -- 172.21.15.92:0/2635371091 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520064260 con 0x7fd520153db0 2023-05-16T07:03:10.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd526a26700 1 --2- 172.21.15.92:0/2635371091 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd520153db0 0x7fd520069b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd518009790 tx=0x7fd518009060).ready entity=mon.2 client_cookie=eca4f29a4fd6f509 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514001470 con 0x7fd520153db0 2023-05-16T07:03:10.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518011400 con 0x7fd520153db0 2023-05-16T07:03:10.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd525223700 1 -- 172.21.15.92:0/2635371091 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011780 con 0x7fd520153db0 2023-05-16T07:03:10.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 -- 172.21.15.92:0/2635371091 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd520153db0 msgr2=0x7fd520069b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 --2- 172.21.15.92:0/2635371091 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd520153db0 0x7fd520069b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd518009790 tx=0x7fd518009060).stop 2023-05-16T07:03:10.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 -- 172.21.15.92:0/2635371091 shutdown_connections 2023-05-16T07:03:10.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 --2- 172.21.15.92:0/2635371091 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd520153db0 0x7fd520069b10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 -- 172.21.15.92:0/2635371091 >> 172.21.15.92:0/2635371091 conn(0x7fd52005e820 msgr2=0x7fd5200622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:10.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 -- 172.21.15.92:0/2635371091 shutdown_connections 2023-05-16T07:03:10.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.390+0000 7fd527227700 1 -- 172.21.15.92:0/2635371091 wait complete. 2023-05-16T07:03:10.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.391+0000 7fd527227700 1 Processor -- start 2023-05-16T07:03:10.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 -- start start 2023-05-16T07:03:10.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5200c6500 0x7fd5201443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5201539d0 0x7fd5200a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 0x7fd520152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ac520 con 0x7fd520153db0 2023-05-16T07:03:10.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ab950 con 0x7fd5200c6500 2023-05-16T07:03:10.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd527227700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ac9b0 con 0x7fd5201539d0 2023-05-16T07:03:10.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd526a26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 0x7fd520152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd526a26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 0x7fd520152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42086/0 (socket says 172.21.15.92:42086) 2023-05-16T07:03:10.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.392+0000 7fd526a26700 1 -- 172.21.15.92:0/1685558421 learned_addr learned my addr 172.21.15.92:0/1685558421 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:10.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526225700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5200c6500 0x7fd5201443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd525a24700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5201539d0 0x7fd5200a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526a26700 1 -- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5201539d0 msgr2=0x7fd5200a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526a26700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5201539d0 0x7fd5200a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526a26700 1 -- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5200c6500 msgr2=0x7fd5201443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526a26700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5200c6500 0x7fd5201443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526a26700 1 -- 172.21.15.92:0/1685558421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180092a0 con 0x7fd520153db0 2023-05-16T07:03:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd526a26700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 0x7fd520152060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd52005fd20 tx=0x7fd518016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518021070 con 0x7fd520153db0 2023-05-16T07:03:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518010b80 con 0x7fd520153db0 2023-05-16T07:03:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801d430 con 0x7fd520153db0 2023-05-16T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5200abbf0 con 0x7fd520153db0 2023-05-16T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.393+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5200ac1b0 con 0x7fd520153db0 2023-05-16T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.394+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5200ac690 con 0x7fd520153db0 2023-05-16T07:03:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.394+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd51800f070 con 0x7fd520153db0 2023-05-16T07:03:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.395+0000 7fd51e7fc700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd50c05ed90 0x7fd50c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.395+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd51807add0 con 0x7fd520153db0 2023-05-16T07:03:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.395+0000 7fd526225700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd50c05ed90 0x7fd50c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.395+0000 7fd526225700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd50c05ed90 0x7fd50c061250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fd5100014f0 tx=0x7fd510009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.399+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd51804c370 con 0x7fd520153db0 2023-05-16T07:03:10.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.563+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd520062c50 con 0x7fd520153db0 2023-05-16T07:03:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.564+0000 7fd51e7fc700 1 -- 172.21.15.92:0/1685558421 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16989 (secure 0 0 0) 0x7fd51804f990 con 0x7fd520153db0 2023-05-16T07:03:10.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:10.569 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:03:09.942932+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:03:07.846932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-05-16T07:03:08.910782+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:03:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd50c05ed90 msgr2=0x7fd50c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd50c05ed90 0x7fd50c061250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fd5100014f0 tx=0x7fd510009040).stop 2023-05-16T07:03:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 msgr2=0x7fd520152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 0x7fd520152060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd52005fd20 tx=0x7fd518016000).stop 2023-05-16T07:03:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 shutdown_connections 2023-05-16T07:03:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd50c05ed90 0x7fd50c061250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd520153db0 0x7fd520152060 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5200c6500 0x7fd5201443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 --2- 172.21.15.92:0/1685558421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5201539d0 0x7fd5200a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 >> 172.21.15.92:0/1685558421 conn(0x7fd52005e820 msgr2=0x7fd5200bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 shutdown_connections 2023-05-16T07:03:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.565+0000 7fd527227700 1 -- 172.21.15.92:0/1685558421 wait complete. 2023-05-16T07:03:10.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:03:10.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.691+0000 7f5b50293700 1 Processor -- start 2023-05-16T07:03:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.691+0000 7f5b50293700 1 -- start start 2023-05-16T07:03:10.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b50293700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b50293700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b480c94d0 con 0x7f5b48154090 2023-05-16T07:03:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b50293700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b480cb7e0 con 0x7f5b48154470 2023-05-16T07:03:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b50293700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480c88c0 con 0x7f5b48068870 2023-05-16T07:03:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b4e02f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b48154090 0x7f5b480a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52840/0 (socket says 172.21.15.92:52840) 2023-05-16T07:03:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b4e02f700 1 -- 172.21.15.92:0/4137682048 learned_addr learned my addr 172.21.15.92:0/4137682048 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b4e830700 1 --2- 172.21.15.92:0/4137682048 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.692+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451549747 0 0) 0x7f5b480c94d0 con 0x7f5b48154090 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b300033f0 con 0x7f5b48154090 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513429683 0 0) 0x7f5b480cb7e0 con 0x7f5b48154470 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b480c94d0 con 0x7f5b48154470 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2372607084 0 0) 0x7f5b300033f0 con 0x7f5b48154090 2023-05-16T07:03:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4e830700 1 -- 172.21.15.92:0/4137682048 >> v1:172.21.15.158:6789/0 conn(0x7f5b48154470 legacy=0x7f5b48068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4e830700 1 -- 172.21.15.92:0/4137682048 >> v1:172.21.15.92:6789/0 conn(0x7f5b48154090 legacy=0x7f5b480a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4e830700 1 -- 172.21.15.92:0/4137682048 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480c8ba0 con 0x7f5b48068870 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4e830700 1 --2- 172.21.15.92:0/4137682048 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480aaca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5b40009790 tx=0x7f5b40009060).ready entity=mon.2 client_cookie=1c6a8f123e0431a6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b380017f0 con 0x7f5b48068870 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b40011400 con 0x7f5b48068870 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.693+0000 7f5b4d02d700 1 -- 172.21.15.92:0/4137682048 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40011780 con 0x7f5b48068870 2023-05-16T07:03:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 -- 172.21.15.92:0/4137682048 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 msgr2=0x7f5b480aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 --2- 172.21.15.92:0/4137682048 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480aaca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5b40009790 tx=0x7f5b40009060).stop 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 -- 172.21.15.92:0/4137682048 shutdown_connections 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 --2- 172.21.15.92:0/4137682048 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480aaca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 -- 172.21.15.92:0/4137682048 >> 172.21.15.92:0/4137682048 conn(0x7f5b48067920 msgr2=0x7f5b480cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 -- 172.21.15.92:0/4137682048 shutdown_connections 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.694+0000 7f5b50293700 1 -- 172.21.15.92:0/4137682048 wait complete. 2023-05-16T07:03:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.695+0000 7f5b50293700 1 Processor -- start 2023-05-16T07:03:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.695+0000 7f5b50293700 1 -- start start 2023-05-16T07:03:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.695+0000 7f5b50293700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.695+0000 7f5b50293700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 0x7f5b48156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.695+0000 7f5b50293700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b48154470 0x7f5b48157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.695+0000 7f5b50293700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48157750 con 0x7f5b48154090 2023-05-16T07:03:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b50293700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480aea60 con 0x7f5b48154470 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b50293700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480ae130 con 0x7f5b48068870 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 0x7f5b48156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 0x7f5b48156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42100/0 (socket says 172.21.15.92:42100) 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 -- 172.21.15.92:0/4240855792 learned_addr learned my addr 172.21.15.92:0/4240855792 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4e830700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b48154470 0x7f5b48157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4e02f700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 -- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 msgr2=0x7f5b480a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 -- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b48154470 msgr2=0x7f5b48157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b48154470 0x7f5b48157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 -- 172.21.15.92:0/4240855792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400092a0 con 0x7f5b48154090 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4e02f700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b4d82e700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 0x7f5b48156d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f5b3c00c490 tx=0x7f5b3c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00f470 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c00f5d0 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b480ae410 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00f970 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.696+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b480aa100 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.697+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b48066730 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.697+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5b3c019030 con 0x7f5b48154090 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.698+0000 7f5b467fc700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b3405ed90 0x7f5b34061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.698+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5b3c079050 con 0x7f5b48154090 2023-05-16T07:03:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.698+0000 7f5b4e02f700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b3405ed90 0x7f5b34061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.698+0000 7f5b4e02f700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b3405ed90 0x7f5b34061250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5b38000f80 tx=0x7f5b38009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.702+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5b3c04a5f0 con 0x7f5b48154090 2023-05-16T07:03:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.907+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5b480aebf0 con 0x7f5b48154090 2023-05-16T07:03:10.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.946+0000 7f5b467fc700 1 -- 172.21.15.92:0/4240855792 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5b3c04dc10 con 0x7f5b48154090 2023-05-16T07:03:10.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.948+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b3405ed90 msgr2=0x7f5b34061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.948+0000 7f5b50293700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b3405ed90 0x7f5b34061250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5b38000f80 tx=0x7f5b38009040).stop 2023-05-16T07:03:10.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.948+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 msgr2=0x7f5b48156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:10.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.948+0000 7f5b50293700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 0x7f5b48156d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f5b3c00c490 tx=0x7f5b3c007c90).stop 2023-05-16T07:03:10.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 shutdown_connections 2023-05-16T07:03:10.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b3405ed90 0x7f5b34061250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b48068870 0x7f5b480a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b48154470 0x7f5b48157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 --2- 172.21.15.92:0/4240855792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48154090 0x7f5b48156d70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:10.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 >> 172.21.15.92:0/4240855792 conn(0x7f5b48067920 msgr2=0x7f5b480c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:10.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 shutdown_connections 2023-05-16T07:03:10.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:10.949+0000 7f5b50293700 1 -- 172.21.15.92:0/4240855792 wait complete. 2023-05-16T07:03:10.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:03:11.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.075+0000 7f8416fb3700 1 Processor -- start 2023-05-16T07:03:11.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.075+0000 7f8416fb3700 1 -- start start 2023-05-16T07:03:11.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8416fb3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8410068870 0x7f84100aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:11.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8416fb3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100c94d0 con 0x7f8410154090 2023-05-16T07:03:11.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8416fb3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100cb7e0 con 0x7f8410154470 2023-05-16T07:03:11.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8416fb3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84100c88c0 con 0x7f8410068870 2023-05-16T07:03:11.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8414d4f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8410154090 0x7f84100a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52854/0 (socket says 172.21.15.92:52854) 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8414d4f700 1 -- 172.21.15.92:0/507936511 learned_addr learned my addr 172.21.15.92:0/507936511 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8415550700 1 --2- 172.21.15.92:0/507936511 >> v2:172.21.15.92:3300/0 conn(0x7f8410068870 0x7f84100aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f840f7fe700 1 -- 172.21.15.92:0/507936511 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170977998 0 0) 0x7f84100c94d0 con 0x7f8410154090 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f840f7fe700 1 -- 172.21.15.92:0/507936511 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83fc0033f0 con 0x7f8410154090 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8415550700 1 -- 172.21.15.92:0/507936511 >> v1:172.21.15.158:6789/0 conn(0x7f8410154470 legacy=0x7f8410068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8415550700 1 -- 172.21.15.92:0/507936511 >> v1:172.21.15.92:6789/0 conn(0x7f8410154090 legacy=0x7f84100a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8415550700 1 -- 172.21.15.92:0/507936511 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100c8ba0 con 0x7f8410068870 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.076+0000 7f8415550700 1 --2- 172.21.15.92:0/507936511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410068870 0x7f84100aaca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f8408009760 tx=0x7f8408009060).ready entity=mon.0 client_cookie=cd9e3f506a27e22c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f840f7fe700 1 -- 172.21.15.92:0/507936511 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080090d0 con 0x7f8410068870 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f840f7fe700 1 -- 172.21.15.92:0/507936511 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800f510 con 0x7f8410068870 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f840f7fe700 1 -- 172.21.15.92:0/507936511 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800f890 con 0x7f8410068870 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 -- 172.21.15.92:0/507936511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410068870 msgr2=0x7f84100aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 --2- 172.21.15.92:0/507936511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410068870 0x7f84100aaca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f8408009760 tx=0x7f8408009060).stop 2023-05-16T07:03:11.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 -- 172.21.15.92:0/507936511 shutdown_connections 2023-05-16T07:03:11.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 --2- 172.21.15.92:0/507936511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410068870 0x7f84100aaca0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 -- 172.21.15.92:0/507936511 >> 172.21.15.92:0/507936511 conn(0x7f8410067920 msgr2=0x7f84100cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:11.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 -- 172.21.15.92:0/507936511 shutdown_connections 2023-05-16T07:03:11.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.077+0000 7f8416fb3700 1 -- 172.21.15.92:0/507936511 wait complete. 2023-05-16T07:03:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.078+0000 7f8416fb3700 1 Processor -- start 2023-05-16T07:03:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.078+0000 7f8416fb3700 1 -- start start 2023-05-16T07:03:11.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8416fb3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8410068870 0x7f84100c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:11.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8416fb3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8410154090 0x7f8410166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:11.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8416fb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 0x7f841014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:11.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8416fb3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8410152c10 con 0x7f8410154470 2023-05-16T07:03:11.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8416fb3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c21e0 con 0x7f8410154090 2023-05-16T07:03:11.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8416fb3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8410152650 con 0x7f8410068870 2023-05-16T07:03:11.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 0x7f841014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:11.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 0x7f841014e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42116/0 (socket says 172.21.15.92:42116) 2023-05-16T07:03:11.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 -- 172.21.15.92:0/1534798425 learned_addr learned my addr 172.21.15.92:0/1534798425 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:11.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f840ffff700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8410154090 0x7f8410166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:11.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8414d4f700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8410068870 0x7f84100c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:11.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 -- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8410068870 msgr2=0x7f84100c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8410068870 0x7f84100c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 -- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8410154090 msgr2=0x7f8410166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.079+0000 7f8415550700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8410154090 0x7f8410166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f8415550700 1 -- 172.21.15.92:0/1534798425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080092a0 con 0x7f8410154470 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f8414d4f700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8410068870 0x7f84100c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f8415550700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 0x7f841014e650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f8408009e00 tx=0x7f8408017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:11.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800f890 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840801b410 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840801b7d0 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8410152930 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.080+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84100c4aa0 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.081+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8410066730 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.081+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f840802b070 con 0x7f8410154470 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.081+0000 7f840cff9700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f83f005ed90 0x7f83f0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:11.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.081+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f840807b690 con 0x7f8410154470 2023-05-16T07:03:11.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.082+0000 7f8414d4f700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f83f005ed90 0x7f83f0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:11.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.082+0000 7f8414d4f700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f83f005ed90 0x7f83f0061250 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f8400005ad0 tx=0x7f8400009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:11.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.086+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f840804cc30 con 0x7f8410154470 2023-05-16T07:03:11.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.289+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84100c1cb0 con 0x7f8410154470 2023-05-16T07:03:11.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.952+0000 7f840cff9700 1 -- 172.21.15.92:0/1534798425 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84080290c0 con 0x7f8410154470 2023-05-16T07:03:11.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f83f005ed90 msgr2=0x7f83f0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:11.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f83f005ed90 0x7f83f0061250 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f8400005ad0 tx=0x7f8400009040).stop 2023-05-16T07:03:11.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 msgr2=0x7f841014e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:11.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 0x7f841014e650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f8408009e00 tx=0x7f8408017000).stop 2023-05-16T07:03:11.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 shutdown_connections 2023-05-16T07:03:11.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f83f005ed90 0x7f83f0061250 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8410068870 0x7f84100c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8410154470 0x7f841014e650 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 --2- 172.21.15.92:0/1534798425 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8410154090 0x7f8410166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:11.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 >> 172.21.15.92:0/1534798425 conn(0x7f8410067920 msgr2=0x7f84100c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:11.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 shutdown_connections 2023-05-16T07:03:11.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:11.955+0000 7f8416fb3700 1 -- 172.21.15.92:0/1534798425 wait complete. 2023-05-16T07:03:11.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:03:12.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.083+0000 7f44872c7700 1 Processor -- start 2023-05-16T07:03:12.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.083+0000 7f44872c7700 1 -- start start 2023-05-16T07:03:12.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f44872c7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44800698b0 0x7f4480069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f44872c7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4480066e50 0x7f44800659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f44872c7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f448013a170 0x7f448013a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f44872c7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44801558f0 con 0x7f448013a170 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f44872c7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44800672b0 con 0x7f4480066e50 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f44872c7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4480065eb0 con 0x7f44800698b0 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f4485864700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f448013a170 0x7f448013a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f4485864700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f448013a170 0x7f448013a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42122/0 (socket says 172.21.15.92:42122) 2023-05-16T07:03:12.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.084+0000 7f4485864700 1 -- 172.21.15.92:0/2588333627 learned_addr learned my addr 172.21.15.92:0/2588333627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:12.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485063700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44800698b0 0x7f4480069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4484862700 1 --2- 172.21.15.92:0/2588333627 >> v2:172.21.15.158:3300/0 conn(0x7f4480066e50 0x7f44800659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485864700 1 -- 172.21.15.92:0/2588333627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44800698b0 msgr2=0x7f4480069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:12.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485864700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44800698b0 0x7f4480069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485864700 1 -- 172.21.15.92:0/2588333627 >> v2:172.21.15.158:3300/0 conn(0x7f4480066e50 msgr2=0x7f44800659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485864700 1 --2- 172.21.15.92:0/2588333627 >> v2:172.21.15.158:3300/0 conn(0x7f4480066e50 0x7f44800659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485864700 1 -- 172.21.15.92:0/2588333627 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448013ab10 con 0x7f448013a170 2023-05-16T07:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485063700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44800698b0 0x7f4480069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4484862700 1 --2- 172.21.15.92:0/2588333627 >> v2:172.21.15.158:3300/0 conn(0x7f4480066e50 0x7f44800659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f4485864700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f448013a620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4480066420 tx=0x7f44780090f0).ready entity=mon.0 client_cookie=d05119063742b3e7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f447ffff700 1 -- 172.21.15.92:0/2588333627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800b600 con 0x7f448013a170 2023-05-16T07:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f447ffff700 1 -- 172.21.15.92:0/2588333627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800b760 con 0x7f448013a170 2023-05-16T07:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f447ffff700 1 -- 172.21.15.92:0/2588333627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800bae0 con 0x7f448013a170 2023-05-16T07:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 -- 172.21.15.92:0/2588333627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 msgr2=0x7f448013a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f448013a620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4480066420 tx=0x7f44780090f0).stop 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 -- 172.21.15.92:0/2588333627 shutdown_connections 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f448013a620 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 --2- 172.21.15.92:0/2588333627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44800698b0 0x7f4480069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 --2- 172.21.15.92:0/2588333627 >> v2:172.21.15.158:3300/0 conn(0x7f4480066e50 0x7f44800659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.085+0000 7f44872c7700 1 -- 172.21.15.92:0/2588333627 >> 172.21.15.92:0/2588333627 conn(0x7f4480063630 msgr2=0x7f44800c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.086+0000 7f44872c7700 1 -- 172.21.15.92:0/2588333627 shutdown_connections 2023-05-16T07:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.086+0000 7f44872c7700 1 -- 172.21.15.92:0/2588333627 wait complete. 2023-05-16T07:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.087+0000 7f44872c7700 1 Processor -- start 2023-05-16T07:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.087+0000 7f44872c7700 1 -- start start 2023-05-16T07:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.087+0000 7f44872c7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4480066e50 0x7f44800a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44800698b0 0x7f44800ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f4480138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44801394b0 con 0x7f448013a170 2023-05-16T07:03:12.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480139970 con 0x7f44800698b0 2023-05-16T07:03:12.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44800a8ad0 con 0x7f4480066e50 2023-05-16T07:03:12.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f4480138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f4480138fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42130/0 (socket says 172.21.15.92:42130) 2023-05-16T07:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 -- 172.21.15.92:0/4007022844 learned_addr learned my addr 172.21.15.92:0/4007022844 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485063700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4480066e50 0x7f44800a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4484862700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44800698b0 0x7f44800ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 -- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4480066e50 msgr2=0x7f44800a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4480066e50 0x7f44800a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 -- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44800698b0 msgr2=0x7f44800ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44800698b0 0x7f44800ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 -- 172.21.15.92:0/4007022844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478009370 con 0x7f448013a170 2023-05-16T07:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4484862700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44800698b0 0x7f44800ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f4485864700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f4480138fc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f4478009130 tx=0x7f4478017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800bae0 con 0x7f448013a170 2023-05-16T07:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447801d430 con 0x7f448013a170 2023-05-16T07:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447801d7f0 con 0x7f448013a170 2023-05-16T07:03:12.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44800a7a30 con 0x7f448013a170 2023-05-16T07:03:12.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.088+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44800a7ff0 con 0x7f448013a170 2023-05-16T07:03:12.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.089+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448006c350 con 0x7f448013a170 2023-05-16T07:03:12.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.089+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4478023020 con 0x7f448013a170 2023-05-16T07:03:12.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.090+0000 7f447d7fa700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f446c05ed90 0x7f446c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.090+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f447807b7c0 con 0x7f448013a170 2023-05-16T07:03:12.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.090+0000 7f4485063700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f446c05ed90 0x7f446c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.090+0000 7f4485063700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f446c05ed90 0x7f446c061250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f44800a9220 tx=0x7f4470007850).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.094+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f447804ce10 con 0x7f448013a170 2023-05-16T07:03:12.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.294+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44800c8740 con 0x7f448013a170 2023-05-16T07:03:12.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.295+0000 7f447d7fa700 1 -- 172.21.15.92:0/4007022844 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f4478050430 con 0x7f448013a170 2023-05-16T07:03:12.299 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:12.300 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:03:09.943261+0000","modified":"2023-05-16T07:03:11.948935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2108552286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2108552286},{"type":"v1","addr":"172.21.15.92:6835","nonce":2108552286}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-05-16T07:03:12.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.296+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f446c05ed90 msgr2=0x7f446c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.296+0000 7f44872c7700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f446c05ed90 0x7f446c061250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f44800a9220 tx=0x7f4470007850).stop 2023-05-16T07:03:12.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.296+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 msgr2=0x7f4480138fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.296+0000 7f44872c7700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f4480138fc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f4478009130 tx=0x7f4478017000).stop 2023-05-16T07:03:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.296+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 shutdown_connections 2023-05-16T07:03:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f446c05ed90 0x7f446c061250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4480066e50 0x7f44800a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448013a170 0x7f4480138fc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 --2- 172.21.15.92:0/4007022844 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44800698b0 0x7f44800ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 >> 172.21.15.92:0/4007022844 conn(0x7f4480063630 msgr2=0x7f44800c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 shutdown_connections 2023-05-16T07:03:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.297+0000 7f44872c7700 1 -- 172.21.15.92:0/4007022844 wait complete. 2023-05-16T07:03:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2023-05-16T07:03:12.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:03:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.418+0000 7fa36124d700 1 Processor -- start 2023-05-16T07:03:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.418+0000 7fa36124d700 1 -- start start 2023-05-16T07:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa36124d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa35c154ec0 0x7fa35c0c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa36124d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa35c066280 0x7fa35c0666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa36124d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0c2240 con 0x7fa35c150ac0 2023-05-16T07:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa36124d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0c31a0 con 0x7fa35c066280 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa36124d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0c4680 con 0x7fa35c154ec0 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa35c066280 0x7fa35c0666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa35c066280 0x7fa35c0666f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42136/0 (socket says 172.21.15.92:42136) 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 -- 172.21.15.92:0/1107755826 learned_addr learned my addr 172.21.15.92:0/1107755826 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35a59c700 1 --2- 172.21.15.92:0/1107755826 >> v2:172.21.15.158:3300/0 conn(0x7fa35c154ec0 0x7fa35c0c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 -- 172.21.15.92:0/1107755826 >> v2:172.21.15.158:3300/0 conn(0x7fa35c154ec0 msgr2=0x7fa35c0c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 --2- 172.21.15.92:0/1107755826 >> v2:172.21.15.158:3300/0 conn(0x7fa35c154ec0 0x7fa35c0c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 -- 172.21.15.92:0/1107755826 >> v1:172.21.15.158:6789/0 conn(0x7fa35c150ac0 legacy=0x7fa35c1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 -- 172.21.15.92:0/1107755826 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0c24b0 con 0x7fa35c066280 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa35b59e700 1 --2- 172.21.15.92:0/1107755826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c066280 0x7fa35c0666f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fa35c063dd0 tx=0x7fa354009840).ready entity=mon.0 client_cookie=9ae2f08d66b45bd2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa359d9b700 1 -- 172.21.15.92:0/1107755826 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400e070 con 0x7fa35c066280 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa359d9b700 1 -- 172.21.15.92:0/1107755826 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35400ba10 con 0x7fa35c066280 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.419+0000 7fa359d9b700 1 -- 172.21.15.92:0/1107755826 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400bd90 con 0x7fa35c066280 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 -- 172.21.15.92:0/1107755826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c066280 msgr2=0x7fa35c0666f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 --2- 172.21.15.92:0/1107755826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c066280 0x7fa35c0666f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fa35c063dd0 tx=0x7fa354009840).stop 2023-05-16T07:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 -- 172.21.15.92:0/1107755826 shutdown_connections 2023-05-16T07:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 --2- 172.21.15.92:0/1107755826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c066280 0x7fa35c0666f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 --2- 172.21.15.92:0/1107755826 >> v2:172.21.15.158:3300/0 conn(0x7fa35c154ec0 0x7fa35c0c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 -- 172.21.15.92:0/1107755826 >> 172.21.15.92:0/1107755826 conn(0x7fa35c0c7bc0 msgr2=0x7fa35c0c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 -- 172.21.15.92:0/1107755826 shutdown_connections 2023-05-16T07:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.420+0000 7fa36124d700 1 -- 172.21.15.92:0/1107755826 wait complete. 2023-05-16T07:03:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.421+0000 7fa36124d700 1 Processor -- start 2023-05-16T07:03:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.421+0000 7fa36124d700 1 -- start start 2023-05-16T07:03:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa36124d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa35c066280 0x7fa35c161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa36124d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 0x7fa35c0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa36124d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa35c154ec0 0x7fa35c0ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa36124d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c162c90 con 0x7fa35c150ac0 2023-05-16T07:03:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa36124d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0acc70 con 0x7fa35c154ec0 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa36124d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1621e0 con 0x7fa35c066280 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 0x7fa35c0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 0x7fa35c0ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42142/0 (socket says 172.21.15.92:42142) 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 -- 172.21.15.92:0/3283326220 learned_addr learned my addr 172.21.15.92:0/3283326220 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35b59e700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa35c154ec0 0x7fa35c0ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 -- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa35c066280 msgr2=0x7fa35c161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35ad9d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa35c066280 0x7fa35c161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa35c066280 0x7fa35c161cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 -- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa35c154ec0 msgr2=0x7fa35c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa35c154ec0 0x7fa35c0ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 -- 172.21.15.92:0/3283326220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009ae0 con 0x7fa35c150ac0 2023-05-16T07:03:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35b59e700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa35c154ec0 0x7fa35c0ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:12.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35ad9d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa35c066280 0x7fa35c161cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:12.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.422+0000 7fa35a59c700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 0x7fa35c0ac290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fa35000c650 tx=0x7fa350008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.423+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350010580 con 0x7fa35c150ac0 2023-05-16T07:03:12.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.423+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3500106e0 con 0x7fa35c150ac0 2023-05-16T07:03:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.423+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350010a60 con 0x7fa35c150ac0 2023-05-16T07:03:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.423+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c1623f0 con 0x7fa35c150ac0 2023-05-16T07:03:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.423+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c1629b0 con 0x7fa35c150ac0 2023-05-16T07:03:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.424+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c14e430 con 0x7fa35c150ac0 2023-05-16T07:03:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.424+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa35001b460 con 0x7fa35c150ac0 2023-05-16T07:03:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.424+0000 7fa342ffd700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa34805ed90 0x7fa348061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.424+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa350079bb0 con 0x7fa35c150ac0 2023-05-16T07:03:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.424+0000 7fa35ad9d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa34805ed90 0x7fa348061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.425+0000 7fa35ad9d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa34805ed90 0x7fa348061250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fa34c00a8a0 tx=0x7fa34c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.428+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa35c14e430 con 0x7fa35c150ac0 2023-05-16T07:03:12.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.589+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa35c14e430 con 0x7fa35c150ac0 2023-05-16T07:03:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.591+0000 7fa342ffd700 1 -- 172.21.15.92:0/3283326220 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fa35c14e430 con 0x7fa35c150ac0 2023-05-16T07:03:12.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:12.596 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:03:10.945060+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:03:07.846932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-05-16T07:03:08.910782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:03:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa34805ed90 msgr2=0x7fa348061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa34805ed90 0x7fa348061250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fa34c00a8a0 tx=0x7fa34c008040).stop 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 msgr2=0x7fa35c0ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 0x7fa35c0ac290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fa35000c650 tx=0x7fa350008040).stop 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 shutdown_connections 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa34805ed90 0x7fa348061250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa35c066280 0x7fa35c161cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa35c154ec0 0x7fa35c0ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 --2- 172.21.15.92:0/3283326220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa35c150ac0 0x7fa35c0ac290 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 >> 172.21.15.92:0/3283326220 conn(0x7fa35c0c7bc0 msgr2=0x7fa35c0c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 shutdown_connections 2023-05-16T07:03:12.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.593+0000 7fa36124d700 1 -- 172.21.15.92:0/3283326220 wait complete. 2023-05-16T07:03:12.610 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:03:12.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:03:12.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.719+0000 7f2698c11700 1 Processor -- start 2023-05-16T07:03:12.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.719+0000 7f2698c11700 1 -- start start 2023-05-16T07:03:12.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2698c11700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26941386a0 0x7f26940c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2698c11700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f26940622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2698c11700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940c37f0 con 0x7f26940627a0 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2698c11700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2694138a80 con 0x7f26941386a0 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2698c11700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694062c30 con 0x7f26940c30a0 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f269259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26941386a0 0x7f26940c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2692d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f26940627a0 0x7f2694063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52896/0 (socket says 172.21.15.92:52896) 2023-05-16T07:03:12.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f269259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26941386a0 0x7f26940c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42148/0 (socket says 172.21.15.92:42148) 2023-05-16T07:03:12.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.720+0000 7f2692d9d700 1 -- 172.21.15.92:0/1585650811 learned_addr learned my addr 172.21.15.92:0/1585650811 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:12.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f2691d9b700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f26940622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269259c700 1 -- 172.21.15.92:0/1585650811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 msgr2=0x7f26940622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:12.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269259c700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f26940622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269259c700 1 -- 172.21.15.92:0/1585650811 >> v1:172.21.15.92:6789/0 conn(0x7f26940627a0 legacy=0x7f2694063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269259c700 1 -- 172.21.15.92:0/1585650811 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940bef90 con 0x7f26941386a0 2023-05-16T07:03:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269159a700 1 -- 172.21.15.92:0/1585650811 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276726036 0 0) 0x7f26940c37f0 con 0x7f26940627a0 2023-05-16T07:03:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f2691d9b700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f26940622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269259c700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c4ec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f2684009e60 tx=0x7f2684009780).ready entity=mon.0 client_cookie=52285319e22fae63 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269159a700 1 -- 172.21.15.92:0/1585650811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400e070 con 0x7f26941386a0 2023-05-16T07:03:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269159a700 1 -- 172.21.15.92:0/1585650811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268400bc30 con 0x7f26941386a0 2023-05-16T07:03:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.721+0000 7f269159a700 1 -- 172.21.15.92:0/1585650811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684012410 con 0x7f26941386a0 2023-05-16T07:03:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 -- 172.21.15.92:0/1585650811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 msgr2=0x7f26940c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c4ec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f2684009e60 tx=0x7f2684009780).stop 2023-05-16T07:03:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 -- 172.21.15.92:0/1585650811 shutdown_connections 2023-05-16T07:03:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c4ec0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 --2- 172.21.15.92:0/1585650811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f26940622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 -- 172.21.15.92:0/1585650811 >> 172.21.15.92:0/1585650811 conn(0x7f26940c8aa0 msgr2=0x7f26940c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 -- 172.21.15.92:0/1585650811 shutdown_connections 2023-05-16T07:03:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.722+0000 7f2698c11700 1 -- 172.21.15.92:0/1585650811 wait complete. 2023-05-16T07:03:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 Processor -- start 2023-05-16T07:03:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 -- start start 2023-05-16T07:03:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26940627a0 0x7f2694157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f2694155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694165da0 con 0x7f26941386a0 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694165120 con 0x7f26940627a0 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.723+0000 7f2698c11700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694165400 con 0x7f26940c30a0 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42158/0 (socket says 172.21.15.92:42158) 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 -- 172.21.15.92:0/3056427784 learned_addr learned my addr 172.21.15.92:0/3056427784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2691d9b700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f2694155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f269259c700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26940627a0 0x7f2694157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 -- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 msgr2=0x7f2694155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f2694155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 -- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26940627a0 msgr2=0x7f2694157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26940627a0 0x7f2694157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 -- 172.21.15.92:0/3056427784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684009a20 con 0x7f26941386a0 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f269259c700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26940627a0 0x7f2694157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2692d9d700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c68b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f268c001eb0 tx=0x7f268c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0072e0 con 0x7f26941386a0 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c007440 con 0x7f26941386a0 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26941656e0 con 0x7f26941386a0 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0057a0 con 0x7f26941386a0 2023-05-16T07:03:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.724+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2694157650 con 0x7f26941386a0 2023-05-16T07:03:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.725+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694165ee0 con 0x7f26941386a0 2023-05-16T07:03:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.725+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f268c01a060 con 0x7f26941386a0 2023-05-16T07:03:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.726+0000 7f268a7fc700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f267405ed90 0x7f2674061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.726+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f268c079280 con 0x7f26941386a0 2023-05-16T07:03:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.726+0000 7f269259c700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f267405ed90 0x7f2674061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.726+0000 7f269259c700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f267405ed90 0x7f2674061250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f268400b9f0 tx=0x7f2684009780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.730+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2694165ee0 con 0x7f26941386a0 2023-05-16T07:03:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.890+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2694165ee0 con 0x7f267405ed90 2023-05-16T07:03:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.900+0000 7f268a7fc700 1 -- 172.21.15.92:0/3056427784 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+327194 (secure 0 0 0) 0x7f2694165ee0 con 0x7f267405ed90 2023-05-16T07:03:12.908 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:12.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f267405ed90 msgr2=0x7f2674061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f267405ed90 0x7f2674061250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f268400b9f0 tx=0x7f2684009780).stop 2023-05-16T07:03:12.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 msgr2=0x7f26940c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:12.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c68b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f268c001eb0 tx=0x7f268c007bb0).stop 2023-05-16T07:03:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 shutdown_connections 2023-05-16T07:03:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f267405ed90 0x7f2674061250 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26941386a0 0x7f26940c68b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26940627a0 0x7f2694157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 --2- 172.21.15.92:0/3056427784 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26940c30a0 0x7f2694155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 >> 172.21.15.92:0/3056427784 conn(0x7f26940c8aa0 msgr2=0x7f2694137050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 shutdown_connections 2023-05-16T07:03:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:12.903+0000 7f2698c11700 1 -- 172.21.15.92:0/3056427784 wait complete. 2023-05-16T07:03:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:03:12.919 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2023-05-16T07:03:11.463457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":43532,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":40832,"kb_avail":754931188,"statfs":{"total":773094113280,"available":773049536512,"internally_reserved":0,"allocated":2699264,"data_stored":1061557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41811968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4317,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.085125"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575320+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579067+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580040+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578086+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579474+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575898+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577091+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579779+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580856+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578847+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578359+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576899+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581559+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576406+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574843+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574233+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580181+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575998+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577724+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575953+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579100+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574777+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574279+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576111+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579336+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577122+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575525+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579486+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578609+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575511+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581002+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574852+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581103+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580469+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.582010+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577226+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574558+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-05-16T07:03:02.905407+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:03:02.905407+0000","last_peered":"2023-05-16T07:03:02.905407+0000","last_clean":"2023-05-16T07:03:02.905407+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:03:02.905407+0000","last_undegraded":"2023-05-16T07:03:02.905407+0000","last_fullsized":"2023-05-16T07:03:02.905407+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574270+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195849+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:03:04.195849+0000","last_peered":"2023-05-16T07:03:04.195849+0000","last_clean":"2023-05-16T07:03:04.195849+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:03:04.195849+0000","last_undegraded":"2023-05-16T07:03:04.195849+0000","last_fullsized":"2023-05-16T07:03:04.195849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581564+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195420+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:03:04.195420+0000","last_peered":"2023-05-16T07:03:04.195420+0000","last_clean":"2023-05-16T07:03:04.195420+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:03:04.195420+0000","last_undegraded":"2023-05-16T07:03:04.195420+0000","last_fullsized":"2023-05-16T07:03:04.195420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576948+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195855+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:03:04.195855+0000","last_peered":"2023-05-16T07:03:04.195855+0000","last_clean":"2023-05-16T07:03:04.195855+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:03:04.195855+0000","last_undegraded":"2023-05-16T07:03:04.195855+0000","last_fullsized":"2023-05-16T07:03:04.195855+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195962+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:03:04.195962+0000","last_peered":"2023-05-16T07:03:04.195962+0000","last_clean":"2023-05-16T07:03:04.195962+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:03:04.195962+0000","last_undegraded":"2023-05-16T07:03:04.195962+0000","last_fullsized":"2023-05-16T07:03:04.195962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578664+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195958+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:03:04.195958+0000","last_peered":"2023-05-16T07:03:04.195958+0000","last_clean":"2023-05-16T07:03:04.195958+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:03:04.195958+0000","last_undegraded":"2023-05-16T07:03:04.195958+0000","last_fullsized":"2023-05-16T07:03:04.195958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574770+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195984+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:03:04.195984+0000","last_peered":"2023-05-16T07:03:04.195984+0000","last_clean":"2023-05-16T07:03:04.195984+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:03:04.195984+0000","last_undegraded":"2023-05-16T07:03:04.195984+0000","last_fullsized":"2023-05-16T07:03:04.195984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576596+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-05-16T07:03:02.906375+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:03:02.906375+0000","last_peered":"2023-05-16T07:03:02.906375+0000","last_clean":"2023-05-16T07:03:02.906375+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:03:02.906375+0000","last_undegraded":"2023-05-16T07:03:02.906375+0000","last_fullsized":"2023-05-16T07:03:02.906375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:03.735035+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:03:03.735035+0000","last_peered":"2023-05-16T07:03:03.735035+0000","last_clean":"2023-05-16T07:03:03.735035+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:03:03.735035+0000","last_undegraded":"2023-05-16T07:03:03.735035+0000","last_fullsized":"2023-05-16T07:03:03.735035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5392,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366448,"statfs":{"total":96636764160,"available":96631242752,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5328,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366512,"statfs":{"total":96636764160,"available":96631308288,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366320,"statfs":{"total":96636764160,"available":96631111680,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366320,"statfs":{"total":96636764160,"available":96631111680,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5404,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366436,"statfs":{"total":96636764160,"available":96631230464,"internally_reserved":0,"allocated":348160,"data_stored":138138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:03:12.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:03:13.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.065+0000 7f375ca81700 1 Processor -- start 2023-05-16T07:03:13.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.065+0000 7f375ca81700 1 -- start start 2023-05-16T07:03:13.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375ca81700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3758154170 0x7f37580c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375ca81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3758154550 0x7f37580c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375ca81700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580c0660 con 0x7f37580696a0 2023-05-16T07:03:13.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375ca81700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37580c1b00 con 0x7f3758154170 2023-05-16T07:03:13.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375ca81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580bfd30 con 0x7f3758154550 2023-05-16T07:03:13.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375659c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3758154170 0x7f37580c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375659c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3758154170 0x7f37580c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:45798/0 (socket says 172.21.15.92:45798) 2023-05-16T07:03:13.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f375659c700 1 -- 172.21.15.92:0/2330000283 learned_addr learned my addr 172.21.15.92:0/2330000283 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:13.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f3755d9b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3758154550 0x7f37580c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.066+0000 7f3756d9d700 1 --1- 172.21.15.92:0/2330000283 >> v1:172.21.15.158:6789/0 conn(0x7f37580696a0 0x7f3758157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:56512/0 (socket says 172.21.15.92:56512) 2023-05-16T07:03:13.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375659c700 1 -- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3758154550 msgr2=0x7f37580c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375659c700 1 --2- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3758154550 0x7f37580c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375659c700 1 -- 172.21.15.92:0/2330000283 >> v1:172.21.15.158:6789/0 conn(0x7f37580696a0 legacy=0x7f3758157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375659c700 1 -- 172.21.15.92:0/2330000283 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580c0010 con 0x7f3758154170 2023-05-16T07:03:13.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375659c700 1 --2- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f37580c5c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f3748009f20 tx=0x7f3748009840).ready entity=mon.1 client_cookie=ac71cdb7152cd9f9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375559a700 1 -- 172.21.15.92:0/2330000283 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374800e070 con 0x7f3758154170 2023-05-16T07:03:13.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375559a700 1 -- 172.21.15.92:0/2330000283 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374800b9d0 con 0x7f3758154170 2023-05-16T07:03:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.067+0000 7f375559a700 1 -- 172.21.15.92:0/2330000283 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374800bd50 con 0x7f3758154170 2023-05-16T07:03:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 -- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 msgr2=0x7f37580c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 --2- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f37580c5c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f3748009f20 tx=0x7f3748009840).stop 2023-05-16T07:03:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 -- 172.21.15.92:0/2330000283 shutdown_connections 2023-05-16T07:03:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 --2- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f37580c5c70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 --2- 172.21.15.92:0/2330000283 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3758154550 0x7f37580c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 -- 172.21.15.92:0/2330000283 >> 172.21.15.92:0/2330000283 conn(0x7f37580cb700 msgr2=0x7f37580c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 -- 172.21.15.92:0/2330000283 shutdown_connections 2023-05-16T07:03:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.068+0000 7f375ca81700 1 -- 172.21.15.92:0/2330000283 wait complete. 2023-05-16T07:03:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 Processor -- start 2023-05-16T07:03:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 -- start start 2023-05-16T07:03:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37580696a0 0x7f3758167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f3758158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 0x7f375814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758165560 con 0x7f3758154550 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37581649b0 con 0x7f3758154170 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.069+0000 7f375ca81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3758164c30 con 0x7f37580696a0 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 0x7f375814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 0x7f375814e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42170/0 (socket says 172.21.15.92:42170) 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 -- 172.21.15.92:0/4283543497 learned_addr learned my addr 172.21.15.92:0/4283543497 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f375659c700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37580696a0 0x7f3758167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3755d9b700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f3758158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 -- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37580696a0 msgr2=0x7f3758167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37580696a0 0x7f3758167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 -- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 msgr2=0x7f3758158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f3758158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 -- 172.21.15.92:0/4283543497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748009ae0 con 0x7f3758154550 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f3756d9d700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 0x7f375814e470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f375000c410 tx=0x7f3750007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500070d0 con 0x7f3758154550 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3750007230 con 0x7f3758154550 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3758164f10 con 0x7f3758154550 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500056e0 con 0x7f3758154550 2023-05-16T07:03:13.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.070+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3758158760 con 0x7f3758154550 2023-05-16T07:03:13.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.071+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37580666c0 con 0x7f3758154550 2023-05-16T07:03:13.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.072+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f375001a060 con 0x7f3758154550 2023-05-16T07:03:13.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.072+0000 7f374e7fc700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f373805ed90 0x7f3738061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.072+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f37500790c0 con 0x7f3758154550 2023-05-16T07:03:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.072+0000 7f375659c700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f373805ed90 0x7f3738061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.073+0000 7f375659c700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f373805ed90 0x7f3738061250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f3748009930 tx=0x7f3748009840).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.076+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f375004a660 con 0x7f3758154550 2023-05-16T07:03:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.235+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f375806c350 con 0x7f373805ed90 2023-05-16T07:03:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.245+0000 7f374e7fc700 1 -- 172.21.15.92:0/4283543497 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+327194 (secure 0 0 0) 0x7f375806c350 con 0x7f373805ed90 2023-05-16T07:03:13.253 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f373805ed90 msgr2=0x7f3738061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f373805ed90 0x7f3738061250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f3748009930 tx=0x7f3748009840).stop 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 msgr2=0x7f375814e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 0x7f375814e470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f375000c410 tx=0x7f3750007d70).stop 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 shutdown_connections 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f373805ed90 0x7f3738061250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37580696a0 0x7f3758167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3758154550 0x7f375814e470 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 --2- 172.21.15.92:0/4283543497 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3758154170 0x7f3758158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 >> 172.21.15.92:0/4283543497 conn(0x7f37580cb700 msgr2=0x7f37580abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 shutdown_connections 2023-05-16T07:03:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.248+0000 7f375ca81700 1 -- 172.21.15.92:0/4283543497 wait complete. 2023-05-16T07:03:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:03:13.263 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2023-05-16T07:03:11.463457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":43532,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":40832,"kb_avail":754931188,"statfs":{"total":773094113280,"available":773049536512,"internally_reserved":0,"allocated":2699264,"data_stored":1061557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41811968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4317,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.085125"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575320+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579067+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580040+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578086+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579474+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575898+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577091+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579779+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580856+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578847+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578359+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576899+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581559+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576406+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574843+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574233+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580181+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575998+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577724+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575953+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579100+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574777+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574279+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576111+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579336+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577122+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575525+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579486+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578609+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575511+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581002+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574852+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581103+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580469+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.582010+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577226+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574558+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-05-16T07:03:02.905407+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:03:02.905407+0000","last_peered":"2023-05-16T07:03:02.905407+0000","last_clean":"2023-05-16T07:03:02.905407+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:03:02.905407+0000","last_undegraded":"2023-05-16T07:03:02.905407+0000","last_fullsized":"2023-05-16T07:03:02.905407+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574270+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195849+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:03:04.195849+0000","last_peered":"2023-05-16T07:03:04.195849+0000","last_clean":"2023-05-16T07:03:04.195849+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:03:04.195849+0000","last_undegraded":"2023-05-16T07:03:04.195849+0000","last_fullsized":"2023-05-16T07:03:04.195849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581564+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195420+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:03:04.195420+0000","last_peered":"2023-05-16T07:03:04.195420+0000","last_clean":"2023-05-16T07:03:04.195420+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:03:04.195420+0000","last_undegraded":"2023-05-16T07:03:04.195420+0000","last_fullsized":"2023-05-16T07:03:04.195420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576948+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195855+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:03:04.195855+0000","last_peered":"2023-05-16T07:03:04.195855+0000","last_clean":"2023-05-16T07:03:04.195855+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:03:04.195855+0000","last_undegraded":"2023-05-16T07:03:04.195855+0000","last_fullsized":"2023-05-16T07:03:04.195855+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195962+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:03:04.195962+0000","last_peered":"2023-05-16T07:03:04.195962+0000","last_clean":"2023-05-16T07:03:04.195962+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:03:04.195962+0000","last_undegraded":"2023-05-16T07:03:04.195962+0000","last_fullsized":"2023-05-16T07:03:04.195962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578664+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195958+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:03:04.195958+0000","last_peered":"2023-05-16T07:03:04.195958+0000","last_clean":"2023-05-16T07:03:04.195958+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:03:04.195958+0000","last_undegraded":"2023-05-16T07:03:04.195958+0000","last_fullsized":"2023-05-16T07:03:04.195958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574770+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195984+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:03:04.195984+0000","last_peered":"2023-05-16T07:03:04.195984+0000","last_clean":"2023-05-16T07:03:04.195984+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:03:04.195984+0000","last_undegraded":"2023-05-16T07:03:04.195984+0000","last_fullsized":"2023-05-16T07:03:04.195984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576596+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-05-16T07:03:02.906375+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:03:02.906375+0000","last_peered":"2023-05-16T07:03:02.906375+0000","last_clean":"2023-05-16T07:03:02.906375+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:03:02.906375+0000","last_undegraded":"2023-05-16T07:03:02.906375+0000","last_fullsized":"2023-05-16T07:03:02.906375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.567973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:03.735035+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:03:03.735035+0000","last_peered":"2023-05-16T07:03:03.735035+0000","last_clean":"2023-05-16T07:03:03.735035+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:03:03.735035+0000","last_undegraded":"2023-05-16T07:03:03.735035+0000","last_fullsized":"2023-05-16T07:03:03.735035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5392,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366448,"statfs":{"total":96636764160,"available":96631242752,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5328,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366512,"statfs":{"total":96636764160,"available":96631308288,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366320,"statfs":{"total":96636764160,"available":96631111680,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366320,"statfs":{"total":96636764160,"available":96631111680,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5404,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366436,"statfs":{"total":96636764160,"available":96631230464,"internally_reserved":0,"allocated":348160,"data_stored":138138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:03:13.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.410+0000 7fc5dcba9700 1 Processor -- start 2023-05-16T07:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.410+0000 7fc5dcba9700 1 -- start start 2023-05-16T07:03:13.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.411+0000 7fc5dcba9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5d815f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.411+0000 7fc5dcba9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80c0660 con 0x7fc5d8154550 2023-05-16T07:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.411+0000 7fc5dcba9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80c1b00 con 0x7fc5d8154170 2023-05-16T07:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.411+0000 7fc5dcba9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80bfd30 con 0x7fc5d8069db0 2023-05-16T07:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d5d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5d8154550 0x7fc5d80696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52928/0 (socket says 172.21.15.92:52928) 2023-05-16T07:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d5d9b700 1 -- 172.21.15.92:0/1980575574 learned_addr learned my addr 172.21.15.92:0/1980575574 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211868063 0 0) 0x7fc5d80c0660 con 0x7fc5d8154550 2023-05-16T07:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d6d9d700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5d815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c00033f0 con 0x7fc5d8154550 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1662579172 0 0) 0x7fc5c00033f0 con 0x7fc5d8154550 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 msgr2=0x7fc5d815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5d815f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 >> v1:172.21.15.158:6789/0 conn(0x7fc5d8154170 legacy=0x7fc5d80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80c0010 con 0x7fc5d8154550 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc5cc001470 con 0x7fc5d8154550 2023-05-16T07:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 >> v1:172.21.15.92:6789/0 conn(0x7fc5d8154550 legacy=0x7fc5d80696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0003910 0x7fc5c0003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5c0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0004b80 0x7fc5c0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80bfd30 con 0x7fc5c0003910 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000b9c0 con 0x7fc5c0004b80 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000bb80 con 0x7fc5d8069db0 2023-05-16T07:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d659c700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0003910 0x7fc5c0003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d5d9b700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5c0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d6d9d700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0004b80 0x7fc5c0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d659c700 1 -- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 msgr2=0x7fc5c0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.412+0000 7fc5d659c700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5c0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d659c700 1 -- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0004b80 msgr2=0x7fc5c0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d659c700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0004b80 0x7fc5c0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d659c700 1 -- 172.21.15.92:0/1980575574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00033f0 con 0x7fc5c0003910 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d5d9b700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d8069db0 0x7fc5c0004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d659c700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0003910 0x7fc5c0003d30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fc5c8001e00 tx=0x7fc5c8007f30).ready entity=mon.0 client_cookie=ca78e19862659fc2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8006a20 con 0x7fc5c0003910 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8006b80 con 0x7fc5c0003910 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d559a700 1 -- 172.21.15.92:0/1980575574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800c5d0 con 0x7fc5c0003910 2023-05-16T07:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5dcba9700 1 -- 172.21.15.92:0/1980575574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0003910 msgr2=0x7fc5c0003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/1980575574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0003910 0x7fc5c0003d30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fc5c8001e00 tx=0x7fc5c8007f30).stop 2023-05-16T07:03:13.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5d6d9d700 1 -- 172.21.15.92:0/1980575574 reap_dead start 2023-05-16T07:03:13.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5dcba9700 1 -- 172.21.15.92:0/1980575574 shutdown_connections 2023-05-16T07:03:13.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5dcba9700 1 -- 172.21.15.92:0/1980575574 >> 172.21.15.92:0/1980575574 conn(0x7fc5d80cb700 msgr2=0x7fc5d80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:13.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5dcba9700 1 -- 172.21.15.92:0/1980575574 shutdown_connections 2023-05-16T07:03:13.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.413+0000 7fc5dcba9700 1 -- 172.21.15.92:0/1980575574 wait complete. 2023-05-16T07:03:13.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.414+0000 7fc5dcba9700 1 Processor -- start 2023-05-16T07:03:13.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.414+0000 7fc5dcba9700 1 -- start start 2023-05-16T07:03:13.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5dcba9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0003910 0x7fc5d80c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5dcba9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 0x7fc5d80c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5dcba9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d80c4060 0x7fc5d8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5dcba9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80c2230 con 0x7fc5d80c36f0 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5dcba9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80aef70 con 0x7fc5c0003910 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5dcba9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81532f0 con 0x7fc5d80c4060 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5d5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 0x7fc5d80c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5d5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 0x7fc5d80c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42190/0 (socket says 172.21.15.92:42190) 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5d5d9b700 1 -- 172.21.15.92:0/3408614916 learned_addr learned my addr 172.21.15.92:0/3408614916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5d659c700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0003910 0x7fc5d80c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.415+0000 7fc5d6d9d700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d80c4060 0x7fc5d8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5d5d9b700 1 -- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d80c4060 msgr2=0x7fc5d8150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5d5d9b700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d80c4060 0x7fc5d8150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5d5d9b700 1 -- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0003910 msgr2=0x7fc5d80c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5d5d9b700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0003910 0x7fc5d80c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5d5d9b700 1 -- 172.21.15.92:0/3408614916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80ae640 con 0x7fc5d80c36f0 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5d5d9b700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 0x7fc5d80c3b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fc5cc00e960 tx=0x7fc5cc001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc010520 con 0x7fc5d80c36f0 2023-05-16T07:03:13.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc010680 con 0x7fc5d80c36f0 2023-05-16T07:03:13.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d80ae920 con 0x7fc5d80c36f0 2023-05-16T07:03:13.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc010a00 con 0x7fc5d80c36f0 2023-05-16T07:03:13.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.416+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d8152f50 con 0x7fc5d80c36f0 2023-05-16T07:03:13.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.417+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d80666c0 con 0x7fc5d80c36f0 2023-05-16T07:03:13.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.417+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc5cc014430 con 0x7fc5d80c36f0 2023-05-16T07:03:13.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.417+0000 7fc5c67fc700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5b805ed60 0x7fc5b8061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.417+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc5cc0797c0 con 0x7fc5d80c36f0 2023-05-16T07:03:13.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.418+0000 7fc5d659c700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5b805ed60 0x7fc5b8061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.418+0000 7fc5d659c700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5b805ed60 0x7fc5b8061220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fc5c8001390 tx=0x7fc5c8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.421+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc5cc0498f0 con 0x7fc5d80c36f0 2023-05-16T07:03:13.562 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:03:13.558+0000 7fc316fde700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:03:13.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.581+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5d80af100 con 0x7fc5b805ed60 2023-05-16T07:03:13.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.591+0000 7fc5c67fc700 1 -- 172.21.15.92:0/3408614916 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+330144 (secure 0 0 0) 0x7fc5d80af100 con 0x7fc5b805ed60 2023-05-16T07:03:13.599 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5b805ed60 msgr2=0x7fc5b8061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5b805ed60 0x7fc5b8061220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fc5c8001390 tx=0x7fc5c8015000).stop 2023-05-16T07:03:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 msgr2=0x7fc5d80c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 0x7fc5d80c3b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fc5cc00e960 tx=0x7fc5cc001e90).stop 2023-05-16T07:03:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 shutdown_connections 2023-05-16T07:03:13.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5b805ed60 0x7fc5b8061220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5d80c4060 0x7fc5d8150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5c0003910 0x7fc5d80c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 --2- 172.21.15.92:0/3408614916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5d80c36f0 0x7fc5d80c3b70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 >> 172.21.15.92:0/3408614916 conn(0x7fc5d80cb700 msgr2=0x7fc5d80acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 shutdown_connections 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.595+0000 7fc5dcba9700 1 -- 172.21.15.92:0/3408614916 wait complete. 2023-05-16T07:03:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:03:13.611 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-05-16T07:03:13.463895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":748,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44040,"kb_used_data":2760,"kb_used_omap":0,"kb_used_meta":41216,"kb_avail":754930680,"statfs":{"total":773094113280,"available":773049016320,"internally_reserved":0,"allocated":2826240,"data_stored":1116923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42205184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4317,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.085125"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575320+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956551+0000","last_change":"2023-05-16T07:03:09.592121+0000","last_active":"2023-05-16T07:03:09.956551+0000","last_peered":"2023-05-16T07:03:09.956551+0000","last_clean":"2023-05-16T07:03:09.956551+0000","last_became_active":"2023-05-16T07:03:09.590810+0000","last_became_peered":"2023-05-16T07:03:09.590810+0000","last_unstale":"2023-05-16T07:03:09.956551+0000","last_undegraded":"2023-05-16T07:03:09.956551+0000","last_fullsized":"2023-05-16T07:03:09.956551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967924+0000","last_change":"2023-05-16T07:03:09.961688+0000","last_active":"2023-05-16T07:03:09.967924+0000","last_peered":"2023-05-16T07:03:09.967924+0000","last_clean":"2023-05-16T07:03:09.967924+0000","last_became_active":"2023-05-16T07:03:09.961428+0000","last_became_peered":"2023-05-16T07:03:09.961428+0000","last_unstale":"2023-05-16T07:03:09.967924+0000","last_undegraded":"2023-05-16T07:03:09.967924+0000","last_fullsized":"2023-05-16T07:03:09.967924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579067+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968927+0000","last_change":"2023-05-16T07:03:09.963887+0000","last_active":"2023-05-16T07:03:09.968927+0000","last_peered":"2023-05-16T07:03:09.968927+0000","last_clean":"2023-05-16T07:03:09.968927+0000","last_became_active":"2023-05-16T07:03:09.963611+0000","last_became_peered":"2023-05-16T07:03:09.963611+0000","last_unstale":"2023-05-16T07:03:09.968927+0000","last_undegraded":"2023-05-16T07:03:09.968927+0000","last_fullsized":"2023-05-16T07:03:09.968927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968997+0000","last_change":"2023-05-16T07:03:09.598293+0000","last_active":"2023-05-16T07:03:09.968997+0000","last_peered":"2023-05-16T07:03:09.968997+0000","last_clean":"2023-05-16T07:03:09.968997+0000","last_became_active":"2023-05-16T07:03:09.598000+0000","last_became_peered":"2023-05-16T07:03:09.598000+0000","last_unstale":"2023-05-16T07:03:09.968997+0000","last_undegraded":"2023-05-16T07:03:09.968997+0000","last_fullsized":"2023-05-16T07:03:09.968997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961523+0000","last_change":"2023-05-16T07:03:09.958946+0000","last_active":"2023-05-16T07:03:09.961523+0000","last_peered":"2023-05-16T07:03:09.961523+0000","last_clean":"2023-05-16T07:03:09.961523+0000","last_became_active":"2023-05-16T07:03:09.958685+0000","last_became_peered":"2023-05-16T07:03:09.958685+0000","last_unstale":"2023-05-16T07:03:09.961523+0000","last_undegraded":"2023-05-16T07:03:09.961523+0000","last_fullsized":"2023-05-16T07:03:09.961523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580040+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967380+0000","last_change":"2023-05-16T07:03:09.601574+0000","last_active":"2023-05-16T07:03:09.967380+0000","last_peered":"2023-05-16T07:03:09.967380+0000","last_clean":"2023-05-16T07:03:09.967380+0000","last_became_active":"2023-05-16T07:03:09.601284+0000","last_became_peered":"2023-05-16T07:03:09.601284+0000","last_unstale":"2023-05-16T07:03:09.967380+0000","last_undegraded":"2023-05-16T07:03:09.967380+0000","last_fullsized":"2023-05-16T07:03:09.967380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968186+0000","last_change":"2023-05-16T07:03:09.961726+0000","last_active":"2023-05-16T07:03:09.968186+0000","last_peered":"2023-05-16T07:03:09.968186+0000","last_clean":"2023-05-16T07:03:09.968186+0000","last_became_active":"2023-05-16T07:03:09.961457+0000","last_became_peered":"2023-05-16T07:03:09.961457+0000","last_unstale":"2023-05-16T07:03:09.968186+0000","last_undegraded":"2023-05-16T07:03:09.968186+0000","last_fullsized":"2023-05-16T07:03:09.968186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578086+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579474+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575898+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957527+0000","last_change":"2023-05-16T07:03:09.597577+0000","last_active":"2023-05-16T07:03:09.957527+0000","last_peered":"2023-05-16T07:03:09.957527+0000","last_clean":"2023-05-16T07:03:09.957527+0000","last_became_active":"2023-05-16T07:03:09.597292+0000","last_became_peered":"2023-05-16T07:03:09.597292+0000","last_unstale":"2023-05-16T07:03:09.957527+0000","last_undegraded":"2023-05-16T07:03:09.957527+0000","last_fullsized":"2023-05-16T07:03:09.957527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577091+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579779+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580856+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955244+0000","last_change":"2023-05-16T07:03:09.586854+0000","last_active":"2023-05-16T07:03:09.955244+0000","last_peered":"2023-05-16T07:03:09.955244+0000","last_clean":"2023-05-16T07:03:09.955244+0000","last_became_active":"2023-05-16T07:03:09.586601+0000","last_became_peered":"2023-05-16T07:03:09.586601+0000","last_unstale":"2023-05-16T07:03:09.955244+0000","last_undegraded":"2023-05-16T07:03:09.955244+0000","last_fullsized":"2023-05-16T07:03:09.955244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578847+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578359+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576899+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956453+0000","last_change":"2023-05-16T07:03:09.583122+0000","last_active":"2023-05-16T07:03:09.956453+0000","last_peered":"2023-05-16T07:03:09.956453+0000","last_clean":"2023-05-16T07:03:09.956453+0000","last_became_active":"2023-05-16T07:03:09.582863+0000","last_became_peered":"2023-05-16T07:03:09.582863+0000","last_unstale":"2023-05-16T07:03:09.956453+0000","last_undegraded":"2023-05-16T07:03:09.956453+0000","last_fullsized":"2023-05-16T07:03:09.956453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971194+0000","last_change":"2023-05-16T07:03:09.586289+0000","last_active":"2023-05-16T07:03:09.971194+0000","last_peered":"2023-05-16T07:03:09.971194+0000","last_clean":"2023-05-16T07:03:09.971194+0000","last_became_active":"2023-05-16T07:03:09.583887+0000","last_became_peered":"2023-05-16T07:03:09.583887+0000","last_unstale":"2023-05-16T07:03:09.971194+0000","last_undegraded":"2023-05-16T07:03:09.971194+0000","last_fullsized":"2023-05-16T07:03:09.971194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581559+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576406+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574843+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574233+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957660+0000","last_change":"2023-05-16T07:03:09.589985+0000","last_active":"2023-05-16T07:03:09.957660+0000","last_peered":"2023-05-16T07:03:09.957660+0000","last_clean":"2023-05-16T07:03:09.957660+0000","last_became_active":"2023-05-16T07:03:09.589723+0000","last_became_peered":"2023-05-16T07:03:09.589723+0000","last_unstale":"2023-05-16T07:03:09.957660+0000","last_undegraded":"2023-05-16T07:03:09.957660+0000","last_fullsized":"2023-05-16T07:03:09.957660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-05-16T07:03:10.950904+0000","last_change":"2023-05-16T07:03:09.592795+0000","last_active":"2023-05-16T07:03:10.950904+0000","last_peered":"2023-05-16T07:03:10.950904+0000","last_clean":"2023-05-16T07:03:10.950904+0000","last_became_active":"2023-05-16T07:03:09.592523+0000","last_became_peered":"2023-05-16T07:03:09.592523+0000","last_unstale":"2023-05-16T07:03:10.950904+0000","last_undegraded":"2023-05-16T07:03:10.950904+0000","last_fullsized":"2023-05-16T07:03:10.950904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971455+0000","last_change":"2023-05-16T07:03:09.968547+0000","last_active":"2023-05-16T07:03:09.971455+0000","last_peered":"2023-05-16T07:03:09.971455+0000","last_clean":"2023-05-16T07:03:09.971455+0000","last_became_active":"2023-05-16T07:03:09.968229+0000","last_became_peered":"2023-05-16T07:03:09.968229+0000","last_unstale":"2023-05-16T07:03:09.971455+0000","last_undegraded":"2023-05-16T07:03:09.971455+0000","last_fullsized":"2023-05-16T07:03:09.971455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961282+0000","last_change":"2023-05-16T07:03:09.597726+0000","last_active":"2023-05-16T07:03:09.961282+0000","last_peered":"2023-05-16T07:03:09.961282+0000","last_clean":"2023-05-16T07:03:09.961282+0000","last_became_active":"2023-05-16T07:03:09.591726+0000","last_became_peered":"2023-05-16T07:03:09.591726+0000","last_unstale":"2023-05-16T07:03:09.961282+0000","last_undegraded":"2023-05-16T07:03:09.961282+0000","last_fullsized":"2023-05-16T07:03:09.961282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580181+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970646+0000","last_change":"2023-05-16T07:03:09.964012+0000","last_active":"2023-05-16T07:03:09.970646+0000","last_peered":"2023-05-16T07:03:09.970646+0000","last_clean":"2023-05-16T07:03:09.970646+0000","last_became_active":"2023-05-16T07:03:09.961689+0000","last_became_peered":"2023-05-16T07:03:09.961689+0000","last_unstale":"2023-05-16T07:03:09.970646+0000","last_undegraded":"2023-05-16T07:03:09.970646+0000","last_fullsized":"2023-05-16T07:03:09.970646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575998+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577724+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575953+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579100+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574777+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971493+0000","last_change":"2023-05-16T07:03:09.969463+0000","last_active":"2023-05-16T07:03:09.971493+0000","last_peered":"2023-05-16T07:03:09.971493+0000","last_clean":"2023-05-16T07:03:09.971493+0000","last_became_active":"2023-05-16T07:03:09.968757+0000","last_became_peered":"2023-05-16T07:03:09.968757+0000","last_unstale":"2023-05-16T07:03:09.971493+0000","last_undegraded":"2023-05-16T07:03:09.971493+0000","last_fullsized":"2023-05-16T07:03:09.971493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574279+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970882+0000","last_change":"2023-05-16T07:03:09.591720+0000","last_active":"2023-05-16T07:03:09.970882+0000","last_peered":"2023-05-16T07:03:09.970882+0000","last_clean":"2023-05-16T07:03:09.970882+0000","last_became_active":"2023-05-16T07:03:09.591344+0000","last_became_peered":"2023-05-16T07:03:09.591344+0000","last_unstale":"2023-05-16T07:03:09.970882+0000","last_undegraded":"2023-05-16T07:03:09.970882+0000","last_fullsized":"2023-05-16T07:03:09.970882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970694+0000","last_change":"2023-05-16T07:03:09.968386+0000","last_active":"2023-05-16T07:03:09.970694+0000","last_peered":"2023-05-16T07:03:09.970694+0000","last_clean":"2023-05-16T07:03:09.970694+0000","last_became_active":"2023-05-16T07:03:09.968165+0000","last_became_peered":"2023-05-16T07:03:09.968165+0000","last_unstale":"2023-05-16T07:03:09.970694+0000","last_undegraded":"2023-05-16T07:03:09.970694+0000","last_fullsized":"2023-05-16T07:03:09.970694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576111+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579336+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577122+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955915+0000","last_change":"2023-05-16T07:03:09.599965+0000","last_active":"2023-05-16T07:03:09.955915+0000","last_peered":"2023-05-16T07:03:09.955915+0000","last_clean":"2023-05-16T07:03:09.955915+0000","last_became_active":"2023-05-16T07:03:09.599711+0000","last_became_peered":"2023-05-16T07:03:09.599711+0000","last_unstale":"2023-05-16T07:03:09.955915+0000","last_undegraded":"2023-05-16T07:03:09.955915+0000","last_fullsized":"2023-05-16T07:03:09.955915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.972055+0000","last_change":"2023-05-16T07:03:09.970741+0000","last_active":"2023-05-16T07:03:09.972055+0000","last_peered":"2023-05-16T07:03:09.972055+0000","last_clean":"2023-05-16T07:03:09.972055+0000","last_became_active":"2023-05-16T07:03:09.970463+0000","last_became_peered":"2023-05-16T07:03:09.970463+0000","last_unstale":"2023-05-16T07:03:09.972055+0000","last_undegraded":"2023-05-16T07:03:09.972055+0000","last_fullsized":"2023-05-16T07:03:09.972055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575525+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955985+0000","last_change":"2023-05-16T07:03:09.589805+0000","last_active":"2023-05-16T07:03:09.955985+0000","last_peered":"2023-05-16T07:03:09.955985+0000","last_clean":"2023-05-16T07:03:09.955985+0000","last_became_active":"2023-05-16T07:03:09.589562+0000","last_became_peered":"2023-05-16T07:03:09.589562+0000","last_unstale":"2023-05-16T07:03:09.955985+0000","last_undegraded":"2023-05-16T07:03:09.955985+0000","last_fullsized":"2023-05-16T07:03:09.955985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579486+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578609+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975811+0000","last_change":"2023-05-16T07:03:09.592715+0000","last_active":"2023-05-16T07:03:09.975811+0000","last_peered":"2023-05-16T07:03:09.975811+0000","last_clean":"2023-05-16T07:03:09.975811+0000","last_became_active":"2023-05-16T07:03:09.592445+0000","last_became_peered":"2023-05-16T07:03:09.592445+0000","last_unstale":"2023-05-16T07:03:09.975811+0000","last_undegraded":"2023-05-16T07:03:09.975811+0000","last_fullsized":"2023-05-16T07:03:09.975811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963495+0000","last_change":"2023-05-16T07:03:09.581278+0000","last_active":"2023-05-16T07:03:09.963495+0000","last_peered":"2023-05-16T07:03:09.963495+0000","last_clean":"2023-05-16T07:03:09.963495+0000","last_became_active":"2023-05-16T07:03:09.580950+0000","last_became_peered":"2023-05-16T07:03:09.580950+0000","last_unstale":"2023-05-16T07:03:09.963495+0000","last_undegraded":"2023-05-16T07:03:09.963495+0000","last_fullsized":"2023-05-16T07:03:09.963495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963155+0000","last_change":"2023-05-16T07:03:09.960677+0000","last_active":"2023-05-16T07:03:09.963155+0000","last_peered":"2023-05-16T07:03:09.963155+0000","last_clean":"2023-05-16T07:03:09.963155+0000","last_became_active":"2023-05-16T07:03:09.960434+0000","last_became_peered":"2023-05-16T07:03:09.960434+0000","last_unstale":"2023-05-16T07:03:09.963155+0000","last_undegraded":"2023-05-16T07:03:09.963155+0000","last_fullsized":"2023-05-16T07:03:09.963155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955745+0000","last_change":"2023-05-16T07:03:09.602043+0000","last_active":"2023-05-16T07:03:09.955745+0000","last_peered":"2023-05-16T07:03:09.955745+0000","last_clean":"2023-05-16T07:03:09.955745+0000","last_became_active":"2023-05-16T07:03:09.601724+0000","last_became_peered":"2023-05-16T07:03:09.601724+0000","last_unstale":"2023-05-16T07:03:09.955745+0000","last_undegraded":"2023-05-16T07:03:09.955745+0000","last_fullsized":"2023-05-16T07:03:09.955745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575511+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968676+0000","last_change":"2023-05-16T07:03:09.962406+0000","last_active":"2023-05-16T07:03:09.968676+0000","last_peered":"2023-05-16T07:03:09.968676+0000","last_clean":"2023-05-16T07:03:09.968676+0000","last_became_active":"2023-05-16T07:03:09.962107+0000","last_became_peered":"2023-05-16T07:03:09.962107+0000","last_unstale":"2023-05-16T07:03:09.968676+0000","last_undegraded":"2023-05-16T07:03:09.968676+0000","last_fullsized":"2023-05-16T07:03:09.968676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967878+0000","last_change":"2023-05-16T07:03:09.961168+0000","last_active":"2023-05-16T07:03:09.967878+0000","last_peered":"2023-05-16T07:03:09.967878+0000","last_clean":"2023-05-16T07:03:09.967878+0000","last_became_active":"2023-05-16T07:03:09.960908+0000","last_became_peered":"2023-05-16T07:03:09.960908+0000","last_unstale":"2023-05-16T07:03:09.967878+0000","last_undegraded":"2023-05-16T07:03:09.967878+0000","last_fullsized":"2023-05-16T07:03:09.967878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581002+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574852+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581103+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956116+0000","last_change":"2023-05-16T07:03:09.598598+0000","last_active":"2023-05-16T07:03:09.956116+0000","last_peered":"2023-05-16T07:03:09.956116+0000","last_clean":"2023-05-16T07:03:09.956116+0000","last_became_active":"2023-05-16T07:03:09.598382+0000","last_became_peered":"2023-05-16T07:03:09.598382+0000","last_unstale":"2023-05-16T07:03:09.956116+0000","last_undegraded":"2023-05-16T07:03:09.956116+0000","last_fullsized":"2023-05-16T07:03:09.956116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580469+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.582010+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577226+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.962297+0000","last_change":"2023-05-16T07:03:09.958211+0000","last_active":"2023-05-16T07:03:09.962297+0000","last_peered":"2023-05-16T07:03:09.962297+0000","last_clean":"2023-05-16T07:03:09.962297+0000","last_became_active":"2023-05-16T07:03:09.957915+0000","last_became_peered":"2023-05-16T07:03:09.957915+0000","last_unstale":"2023-05-16T07:03:09.962297+0000","last_undegraded":"2023-05-16T07:03:09.962297+0000","last_fullsized":"2023-05-16T07:03:09.962297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574558+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956188+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:03:09.956188+0000","last_peered":"2023-05-16T07:03:09.956188+0000","last_clean":"2023-05-16T07:03:09.956188+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:03:09.956188+0000","last_undegraded":"2023-05-16T07:03:09.956188+0000","last_fullsized":"2023-05-16T07:03:09.956188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968905+0000","last_change":"2023-05-16T07:03:09.964901+0000","last_active":"2023-05-16T07:03:09.968905+0000","last_peered":"2023-05-16T07:03:09.968905+0000","last_clean":"2023-05-16T07:03:09.968905+0000","last_became_active":"2023-05-16T07:03:09.964643+0000","last_became_peered":"2023-05-16T07:03:09.964643+0000","last_unstale":"2023-05-16T07:03:09.968905+0000","last_undegraded":"2023-05-16T07:03:09.968905+0000","last_fullsized":"2023-05-16T07:03:09.968905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574270+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195849+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:03:04.195849+0000","last_peered":"2023-05-16T07:03:04.195849+0000","last_clean":"2023-05-16T07:03:04.195849+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:03:04.195849+0000","last_undegraded":"2023-05-16T07:03:04.195849+0000","last_fullsized":"2023-05-16T07:03:04.195849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581564+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195420+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:03:04.195420+0000","last_peered":"2023-05-16T07:03:04.195420+0000","last_clean":"2023-05-16T07:03:04.195420+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:03:04.195420+0000","last_undegraded":"2023-05-16T07:03:04.195420+0000","last_fullsized":"2023-05-16T07:03:04.195420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576948+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195855+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:03:04.195855+0000","last_peered":"2023-05-16T07:03:04.195855+0000","last_clean":"2023-05-16T07:03:04.195855+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:03:04.195855+0000","last_undegraded":"2023-05-16T07:03:04.195855+0000","last_fullsized":"2023-05-16T07:03:04.195855+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195962+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:03:04.195962+0000","last_peered":"2023-05-16T07:03:04.195962+0000","last_clean":"2023-05-16T07:03:04.195962+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:03:04.195962+0000","last_undegraded":"2023-05-16T07:03:04.195962+0000","last_fullsized":"2023-05-16T07:03:04.195962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971163+0000","last_change":"2023-05-16T07:03:09.968710+0000","last_active":"2023-05-16T07:03:09.971163+0000","last_peered":"2023-05-16T07:03:09.971163+0000","last_clean":"2023-05-16T07:03:09.971163+0000","last_became_active":"2023-05-16T07:03:09.968451+0000","last_became_peered":"2023-05-16T07:03:09.968451+0000","last_unstale":"2023-05-16T07:03:09.971163+0000","last_undegraded":"2023-05-16T07:03:09.971163+0000","last_fullsized":"2023-05-16T07:03:09.971163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578664+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195958+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:03:04.195958+0000","last_peered":"2023-05-16T07:03:04.195958+0000","last_clean":"2023-05-16T07:03:04.195958+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:03:04.195958+0000","last_undegraded":"2023-05-16T07:03:04.195958+0000","last_fullsized":"2023-05-16T07:03:04.195958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970552+0000","last_change":"2023-05-16T07:03:09.964329+0000","last_active":"2023-05-16T07:03:09.970552+0000","last_peered":"2023-05-16T07:03:09.970552+0000","last_clean":"2023-05-16T07:03:09.970552+0000","last_became_active":"2023-05-16T07:03:09.960866+0000","last_became_peered":"2023-05-16T07:03:09.960866+0000","last_unstale":"2023-05-16T07:03:09.970552+0000","last_undegraded":"2023-05-16T07:03:09.970552+0000","last_fullsized":"2023-05-16T07:03:09.970552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574770+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195984+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:03:04.195984+0000","last_peered":"2023-05-16T07:03:04.195984+0000","last_clean":"2023-05-16T07:03:04.195984+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:03:04.195984+0000","last_undegraded":"2023-05-16T07:03:04.195984+0000","last_fullsized":"2023-05-16T07:03:04.195984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576596+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957557+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:03:09.957557+0000","last_peered":"2023-05-16T07:03:09.957557+0000","last_clean":"2023-05-16T07:03:09.957557+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:03:09.957557+0000","last_undegraded":"2023-05-16T07:03:09.957557+0000","last_fullsized":"2023-05-16T07:03:09.957557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.954982+0000","last_change":"2023-05-16T07:03:09.590083+0000","last_active":"2023-05-16T07:03:09.954982+0000","last_peered":"2023-05-16T07:03:09.954982+0000","last_clean":"2023-05-16T07:03:09.954982+0000","last_became_active":"2023-05-16T07:03:09.589801+0000","last_became_peered":"2023-05-16T07:03:09.589801+0000","last_unstale":"2023-05-16T07:03:09.954982+0000","last_undegraded":"2023-05-16T07:03:09.954982+0000","last_fullsized":"2023-05-16T07:03:09.954982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:03.735035+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:03:03.735035+0000","last_peered":"2023-05-16T07:03:03.735035+0000","last_clean":"2023-05-16T07:03:03.735035+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:03:03.735035+0000","last_undegraded":"2023-05-16T07:03:03.735035+0000","last_fullsized":"2023-05-16T07:03:03.735035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":2},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":748,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5640,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366200,"statfs":{"total":96636764160,"available":96630988800,"internally_reserved":0,"allocated":393216,"data_stored":158860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5328,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366512,"statfs":{"total":96636764160,"available":96631308288,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366060,"statfs":{"total":96636764160,"available":96630845440,"internally_reserved":0,"allocated":405504,"data_stored":160340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366320,"statfs":{"total":96636764160,"available":96631111680,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5404,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366436,"statfs":{"total":96636764160,"available":96631230464,"internally_reserved":0,"allocated":348160,"data_stored":138138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:03:13.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-05-16T07:03:13.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-05-16T07:03:13.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.575320+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-16T07:03:13.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-05-16T07:03:13.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-05-16T07:03:13.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579067+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-05-16T07:03:13.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.580040+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-05-16T07:03:13.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-05-16T07:03:13.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.578086+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-05-16T07:03:13.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-05-16T07:03:13.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579474+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-05-16T07:03:13.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-05-16T07:03:13.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.575898+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-05-16T07:03:13.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-05-16T07:03:13.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-05-16T07:03:13.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.577091+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-05-16T07:03:13.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-05-16T07:03:13.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579779+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-05-16T07:03:13.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-05-16T07:03:13.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.580856+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-05-16T07:03:13.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-05-16T07:03:13.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-05-16T07:03:13.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.578847+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-05-16T07:03:13.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-05-16T07:03:13.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.578359+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-05-16T07:03:13.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-05-16T07:03:13.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.576899+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-05-16T07:03:13.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-05-16T07:03:13.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-05-16T07:03:13.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-05-16T07:03:13.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.581559+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-05-16T07:03:13.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-05-16T07:03:13.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.576406+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-05-16T07:03:13.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-05-16T07:03:13.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574843+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-05-16T07:03:13.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-05-16T07:03:13.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574233+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-05-16T07:03:13.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-05-16T07:03:13.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-05-16T07:03:13.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579745+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-05-16T07:03:13.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.658 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-05-16T07:03:13.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-05-16T07:03:13.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-05-16T07:03:13.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.580181+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-05-16T07:03:13.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.575998+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-05-16T07:03:13.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-05-16T07:03:13.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.577724+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-05-16T07:03:13.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.663 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-05-16T07:03:13.663 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.575953+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-16T07:03:13.663 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-05-16T07:03:13.664 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.664 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-05-16T07:03:13.664 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579100+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-05-16T07:03:13.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-05-16T07:03:13.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574777+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-16T07:03:13.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-05-16T07:03:13.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574279+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-05-16T07:03:13.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-05-16T07:03:13.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-05-16T07:03:13.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.576111+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-05-16T07:03:13.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-05-16T07:03:13.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579336+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-05-16T07:03:13.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-05-16T07:03:13.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.577122+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-05-16T07:03:13.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-05-16T07:03:13.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.575525+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-05-16T07:03:13.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-05-16T07:03:13.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-05-16T07:03:13.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579486+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-05-16T07:03:13.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-05-16T07:03:13.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.578609+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-05-16T07:03:13.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-05-16T07:03:13.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-05-16T07:03:13.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-05-16T07:03:13.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.575511+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-16T07:03:13.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-05-16T07:03:13.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.578745+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-05-16T07:03:13.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.581002+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-05-16T07:03:13.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-05-16T07:03:13.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574852+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-05-16T07:03:13.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-05-16T07:03:13.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.581103+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-05-16T07:03:13.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-05-16T07:03:13.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-05-16T07:03:13.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.580469+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-05-16T07:03:13.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-05-16T07:03:13.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.582010+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-05-16T07:03:13.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-05-16T07:03:13.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.577226+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-05-16T07:03:13.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.579973+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-05-16T07:03:13.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-05-16T07:03:13.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574558+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-05-16T07:03:13.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-05-16T07:03:13.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574270+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-05-16T07:03:13.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-05-16T07:03:13.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-05-16T07:03:13.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.581564+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-16T07:03:13.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-05-16T07:03:13.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-05-16T07:03:13.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.576948+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-16T07:03:13.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-05-16T07:03:13.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.578664+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-16T07:03:13.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-05-16T07:03:13.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.574770+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-16T07:03:13.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-05-16T07:03:13.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-05-16T07:03:13.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-05-16T07:03:08.567973+0000', 'last_change': '2023-05-16T07:03:08.576596+0000', 'last_active': '2023-05-16T07:03:08.567973+0000', 'last_peered': '2023-05-16T07:03:08.567973+0000', 'last_clean': '2023-05-16T07:03:08.567973+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:08.567973+0000', 'last_undegraded': '2023-05-16T07:03:08.567973+0000', 'last_fullsized': '2023-05-16T07:03:08.567973+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:08.567973+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-16T07:03:13.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-05-16T07:03:13.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-16T07:03:09.571960+0000', 'last_change': '2023-05-16T07:03:09.571960+0000', 'last_active': '2023-05-16T07:03:09.571960+0000', 'last_peered': '2023-05-16T07:03:09.571960+0000', 'last_clean': '2023-05-16T07:03:09.571960+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-16T07:03:09.571960+0000', 'last_undegraded': '2023-05-16T07:03:09.571960+0000', 'last_fullsized': '2023-05-16T07:03:09.571960+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'last_clean_scrub_stamp': '2023-05-16T07:03:09.571960+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-16T07:03:13.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-05-16T07:03:13.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.807+0000 7fa4defb8700 1 Processor -- start 2023-05-16T07:03:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.807+0000 7fa4defb8700 1 -- start start 2023-05-16T07:03:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4defb8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa4d81542b0 0x7fa4d8154690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4defb8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d80c6040 con 0x7fa4d80a9ea0 2023-05-16T07:03:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4defb8700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d80c85d0 con 0x7fa4d8069e60 2023-05-16T07:03:13.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4defb8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d80c5750 con 0x7fa4d81542b0 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa4d80a9ea0 0x7fa4d8069750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52946/0 (socket says 172.21.15.92:52946) 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d7fff700 1 -- 172.21.15.92:0/761925548 learned_addr learned my addr 172.21.15.92:0/761925548 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309617594 0 0) 0x7fa4d80c6040 con 0x7fa4d80a9ea0 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4c40033f0 con 0x7fa4d80a9ea0 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/761925548 >> v2:172.21.15.158:3300/0 conn(0x7fa4d81542b0 0x7fa4d8154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2932944619 0 0) 0x7fa4c40033f0 con 0x7fa4d80a9ea0 2023-05-16T07:03:13.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 >> v2:172.21.15.158:3300/0 conn(0x7fa4d81542b0 msgr2=0x7fa4d8154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 --2- 172.21.15.92:0/761925548 >> v2:172.21.15.158:3300/0 conn(0x7fa4d81542b0 0x7fa4d8154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 >> v1:172.21.15.158:6789/0 conn(0x7fa4d8069e60 legacy=0x7fa4d80c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d80c59f0 con 0x7fa4d80a9ea0 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fa4c80017f0 con 0x7fa4d80a9ea0 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 >> v1:172.21.15.92:6789/0 conn(0x7fa4d80a9ea0 legacy=0x7fa4d8069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40038e0 0x7fa4c4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c4004690 0x7fa4c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c4005050 0x7fa4c400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80c5750 con 0x7fa4c40038e0 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c400bfe0 con 0x7fa4c4004690 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.808+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c400c1a0 con 0x7fa4c4005050 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/761925548 >> v2:172.21.15.158:3300/0 conn(0x7fa4d81542b0 0x7fa4d8154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40038e0 0x7fa4c4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4d7fff700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c4004690 0x7fa4c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dd555700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c4005050 0x7fa4c400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 -- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c4005050 msgr2=0x7fa4c400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c4005050 0x7fa4c400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 -- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c4004690 msgr2=0x7fa4c4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c4004690 0x7fa4c4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dd555700 1 -- 172.21.15.92:0/761925548 reap_dead start 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 -- 172.21.15.92:0/761925548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40033f0 con 0x7fa4c40038e0 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40038e0 0x7fa4c4003d00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fa4d000bab0 tx=0x7fa4d0008040).ready entity=mon.0 client_cookie=548eb367f70f7165 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0016070 con 0x7fa4c40038e0 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d000be90 con 0x7fa4c40038e0 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.809+0000 7fa4d77fe700 1 -- 172.21.15.92:0/761925548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000d7e0 con 0x7fa4c40038e0 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 -- 172.21.15.92:0/761925548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40038e0 msgr2=0x7fa4c4003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40038e0 0x7fa4c4003d00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fa4d000bab0 tx=0x7fa4d0008040).stop 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 -- 172.21.15.92:0/761925548 shutdown_connections 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 --2- 172.21.15.92:0/761925548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40038e0 0x7fa4c4003d00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 -- 172.21.15.92:0/761925548 >> 172.21.15.92:0/761925548 conn(0x7fa4d80cb750 msgr2=0x7fa4d80c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 -- 172.21.15.92:0/761925548 shutdown_connections 2023-05-16T07:03:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.810+0000 7fa4defb8700 1 -- 172.21.15.92:0/761925548 wait complete. 2023-05-16T07:03:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.811+0000 7fa4defb8700 1 Processor -- start 2023-05-16T07:03:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.811+0000 7fa4defb8700 1 -- start start 2023-05-16T07:03:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c40038e0 0x7fa4d80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4d8151350 0x7fa4d8167900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 0x7fa4d80c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d81688a0 con 0x7fa4d81517d0 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d8150ec0 con 0x7fa4d8151350 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d81682e0 con 0x7fa4c40038e0 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 0x7fa4d80c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 0x7fa4d80c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42212/0 (socket says 172.21.15.92:42212) 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 -- 172.21.15.92:0/1302862753 learned_addr learned my addr 172.21.15.92:0/1302862753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:13.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4d7fff700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4d8151350 0x7fa4d8167900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 -- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c40038e0 msgr2=0x7fa4d80a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c40038e0 0x7fa4d80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 -- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4d8151350 msgr2=0x7fa4d8167900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4d8151350 0x7fa4d8167900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 -- 172.21.15.92:0/1302862753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d81685c0 con 0x7fa4d81517d0 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4dd555700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 0x7fa4d80c6a90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fa4d8084480 tx=0x7fa4c0007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0002ee0 con 0x7fa4d81517d0 2023-05-16T07:03:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c0007240 con 0x7fa4d81517d0 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00075c0 con 0x7fa4d81517d0 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d8150b50 con 0x7fa4d81517d0 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.812+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d80a9770 con 0x7fa4d81517d0 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.813+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d81509b0 con 0x7fa4d81517d0 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.813+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa4c00068b0 con 0x7fa4d81517d0 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.814+0000 7fa4d4ff9700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4b805ed60 0x7fa4b8061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:13.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.814+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa4c0079240 con 0x7fa4d81517d0 2023-05-16T07:03:13.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.814+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4b805ed60 0x7fa4b8061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:13.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.815+0000 7fa4dcd54700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4b805ed60 0x7fa4b8061220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa4d0008010 tx=0x7fa4d0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:13.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:13.818+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa4d81509b0 con 0x7fa4d81517d0 2023-05-16T07:03:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.044+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa4d81509b0 con 0x7fa4d81517d0 2023-05-16T07:03:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.045+0000 7fa4d4ff9700 1 -- 172.21.15.92:0/1302862753 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1108 (secure 0 0 0) 0x7fa4d81509b0 con 0x7fa4d81517d0 2023-05-16T07:03:14.049 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:14.049 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":65,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1684220537,"num_in_osds":8,"osd_in_since":1684220531,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":48},{"state_name":"creating+peering","count":46},{"state_name":"active+clean","count":43}],"num_pgs":137,"num_pools":4,"num_objects":7,"data_bytes":748,"bytes_used":45096960,"bytes_avail":773049016320,"bytes_total":773094113280,"unknown_pgs_ratio":0.35036495327949524,"inactive_pgs_ratio":0.33576643466949463},"fsmap":{"epoch":12,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4935}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-05-16T07:02:08.693582+0000","services":{}},"progress_events":{}} 2023-05-16T07:03:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4b805ed60 msgr2=0x7fa4b8061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4b805ed60 0x7fa4b8061220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa4d0008010 tx=0x7fa4d0008040).stop 2023-05-16T07:03:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 msgr2=0x7fa4d80c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 0x7fa4d80c6a90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fa4d8084480 tx=0x7fa4c0007bb0).stop 2023-05-16T07:03:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 shutdown_connections 2023-05-16T07:03:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4b805ed60 0x7fa4b8061220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c40038e0 0x7fa4d80a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4d81517d0 0x7fa4d80c6a90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 --2- 172.21.15.92:0/1302862753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4d8151350 0x7fa4d8167900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 >> 172.21.15.92:0/1302862753 conn(0x7fa4d80cb750 msgr2=0x7fa4d80ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:14.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 shutdown_connections 2023-05-16T07:03:14.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.047+0000 7fa4defb8700 1 -- 172.21.15.92:0/1302862753 wait complete. 2023-05-16T07:03:14.062 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-05-16T07:03:14.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:03:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.169+0000 7f9724e07700 1 Processor -- start 2023-05-16T07:03:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.169+0000 7f9724e07700 1 -- start start 2023-05-16T07:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f9724e07700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9720154170 0x7f97200c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f9724e07700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97200c0660 con 0x7f9720154550 2023-05-16T07:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f9724e07700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97200bfd30 con 0x7f9720069db0 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f9724e07700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97200c0010 con 0x7f9720154170 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f971dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9720154550 0x7f97200696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52968/0 (socket says 172.21.15.92:52968) 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f971dd9b700 1 -- 172.21.15.92:0/510283450 learned_addr learned my addr 172.21.15.92:0/510283450 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f971e59c700 1 --2- 172.21.15.92:0/510283450 >> v2:172.21.15.158:3300/0 conn(0x7f9720154170 0x7f97200c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.170+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533777541 0 0) 0x7f97200c0660 con 0x7f9720154550 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f970c0033f0 con 0x7f9720154550 2023-05-16T07:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3211441122 0 0) 0x7f970c0033f0 con 0x7f9720154550 2023-05-16T07:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 >> v2:172.21.15.158:3300/0 conn(0x7f9720154170 msgr2=0x7f97200c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 --2- 172.21.15.92:0/510283450 >> v2:172.21.15.158:3300/0 conn(0x7f9720154170 0x7f97200c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 >> v1:172.21.15.158:6789/0 conn(0x7f9720069db0 legacy=0x7f97200c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97200c02f0 con 0x7f9720154550 2023-05-16T07:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f9708001470 con 0x7f9720154550 2023-05-16T07:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 >> v1:172.21.15.92:6789/0 conn(0x7f9720154550 legacy=0x7f97200696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f970c0038e0 0x7f970c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c004690 0x7f970c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f970c005050 0x7f970c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200c0010 con 0x7f970c0038e0 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c00bfe0 con 0x7f970c005050 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c00c1a0 con 0x7f970c004690 2023-05-16T07:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 --2- 172.21.15.92:0/510283450 >> v2:172.21.15.158:3300/0 conn(0x7f9720154170 0x7f97200c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f970c0038e0 0x7f970c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971dd9b700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c004690 0x7f970c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971ed9d700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f970c005050 0x7f970c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 -- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c004690 msgr2=0x7f970c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c004690 0x7f970c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 -- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f970c005050 msgr2=0x7f970c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f970c005050 0x7f970c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971ed9d700 1 -- 172.21.15.92:0/510283450 reap_dead start 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 -- 172.21.15.92:0/510283450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c0033f0 con 0x7f970c0038e0 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.171+0000 7f971e59c700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f970c0038e0 0x7f970c003d00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f97100094d0 tx=0x7f971000b040).ready entity=mon.0 client_cookie=cbea4240d1a1cb72 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710017070 con 0x7f970c0038e0 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971000f040 con 0x7f970c0038e0 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f971d59a700 1 -- 172.21.15.92:0/510283450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971000ee50 con 0x7f970c0038e0 2023-05-16T07:03:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 -- 172.21.15.92:0/510283450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f970c0038e0 msgr2=0x7f970c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f970c0038e0 0x7f970c003d00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f97100094d0 tx=0x7f971000b040).stop 2023-05-16T07:03:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 -- 172.21.15.92:0/510283450 shutdown_connections 2023-05-16T07:03:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 --2- 172.21.15.92:0/510283450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f970c0038e0 0x7f970c003d00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 -- 172.21.15.92:0/510283450 >> 172.21.15.92:0/510283450 conn(0x7f97200cb700 msgr2=0x7f97200c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 -- 172.21.15.92:0/510283450 shutdown_connections 2023-05-16T07:03:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.172+0000 7f9724e07700 1 -- 172.21.15.92:0/510283450 wait complete. 2023-05-16T07:03:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.173+0000 7f9724e07700 1 Processor -- start 2023-05-16T07:03:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.173+0000 7f9724e07700 1 -- start start 2023-05-16T07:03:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f9724e07700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c0038e0 0x7f9720153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f9724e07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 0x7f97200b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f9724e07700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f97201653a0 0x7f9720165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f9724e07700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200b0790 con 0x7f97200afe20 2023-05-16T07:03:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f9724e07700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972013e170 con 0x7f97201653a0 2023-05-16T07:03:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f9724e07700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97201521d0 con 0x7f970c0038e0 2023-05-16T07:03:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 0x7f97200b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 0x7f97200b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42230/0 (socket says 172.21.15.92:42230) 2023-05-16T07:03:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 -- 172.21.15.92:0/149397473 learned_addr learned my addr 172.21.15.92:0/149397473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971e59c700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c0038e0 0x7f9720153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971ed9d700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f97201653a0 0x7f9720165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 -- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c0038e0 msgr2=0x7f9720153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c0038e0 0x7f9720153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 -- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f97201653a0 msgr2=0x7f9720165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f97201653a0 0x7f9720165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 -- 172.21.15.92:0/149397473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97201524b0 con 0x7f97200afe20 2023-05-16T07:03:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.174+0000 7f971dd9b700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 0x7f97200b02a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f9708004a10 tx=0x7f9708001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.175+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970800f660 con 0x7f97200afe20 2023-05-16T07:03:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.175+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970800f7c0 con 0x7f97200afe20 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.175+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9720152790 con 0x7f97200afe20 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.175+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970800fb40 con 0x7f97200afe20 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.175+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9720152d50 con 0x7f97200afe20 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.175+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97200c1960 con 0x7f97200afe20 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.176+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f970801b030 con 0x7f97200afe20 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.176+0000 7f97167fc700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f970005ed60 0x7f9700061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.176+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9708079300 con 0x7f97200afe20 2023-05-16T07:03:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.176+0000 7f971e59c700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f970005ed60 0x7f9700061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.177+0000 7f971e59c700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f970005ed60 0x7f9700061220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f971000afd0 tx=0x7f9710019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.180+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f970804a950 con 0x7f97200afe20 2023-05-16T07:03:14.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.338+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97200666c0 con 0x7f970005ed60 2023-05-16T07:03:14.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.348+0000 7f97167fc700 1 -- 172.21.15.92:0/149397473 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+330144 (secure 0 0 0) 0x7f97200666c0 con 0x7f970005ed60 2023-05-16T07:03:14.355 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f970005ed60 msgr2=0x7f9700061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f970005ed60 0x7f9700061220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f971000afd0 tx=0x7f9710019000).stop 2023-05-16T07:03:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 msgr2=0x7f97200b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 0x7f97200b02a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f9708004a10 tx=0x7f9708001e50).stop 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 shutdown_connections 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f970005ed60 0x7f9700061220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f970c0038e0 0x7f9720153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f97201653a0 0x7f9720165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 --2- 172.21.15.92:0/149397473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97200afe20 0x7f97200b02a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 >> 172.21.15.92:0/149397473 conn(0x7f97200cb700 msgr2=0x7f97200ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 shutdown_connections 2023-05-16T07:03:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:14.351+0000 7f9724e07700 1 -- 172.21.15.92:0/149397473 wait complete. 2023-05-16T07:03:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:03:14.367 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-05-16T07:03:13.463895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":748,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44040,"kb_used_data":2760,"kb_used_omap":0,"kb_used_meta":41216,"kb_avail":754930680,"statfs":{"total":773094113280,"available":773049016320,"internally_reserved":0,"allocated":2826240,"data_stored":1116923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42205184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4317,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.085125"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575320+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956551+0000","last_change":"2023-05-16T07:03:09.592121+0000","last_active":"2023-05-16T07:03:09.956551+0000","last_peered":"2023-05-16T07:03:09.956551+0000","last_clean":"2023-05-16T07:03:09.956551+0000","last_became_active":"2023-05-16T07:03:09.590810+0000","last_became_peered":"2023-05-16T07:03:09.590810+0000","last_unstale":"2023-05-16T07:03:09.956551+0000","last_undegraded":"2023-05-16T07:03:09.956551+0000","last_fullsized":"2023-05-16T07:03:09.956551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967924+0000","last_change":"2023-05-16T07:03:09.961688+0000","last_active":"2023-05-16T07:03:09.967924+0000","last_peered":"2023-05-16T07:03:09.967924+0000","last_clean":"2023-05-16T07:03:09.967924+0000","last_became_active":"2023-05-16T07:03:09.961428+0000","last_became_peered":"2023-05-16T07:03:09.961428+0000","last_unstale":"2023-05-16T07:03:09.967924+0000","last_undegraded":"2023-05-16T07:03:09.967924+0000","last_fullsized":"2023-05-16T07:03:09.967924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579067+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968927+0000","last_change":"2023-05-16T07:03:09.963887+0000","last_active":"2023-05-16T07:03:09.968927+0000","last_peered":"2023-05-16T07:03:09.968927+0000","last_clean":"2023-05-16T07:03:09.968927+0000","last_became_active":"2023-05-16T07:03:09.963611+0000","last_became_peered":"2023-05-16T07:03:09.963611+0000","last_unstale":"2023-05-16T07:03:09.968927+0000","last_undegraded":"2023-05-16T07:03:09.968927+0000","last_fullsized":"2023-05-16T07:03:09.968927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968997+0000","last_change":"2023-05-16T07:03:09.598293+0000","last_active":"2023-05-16T07:03:09.968997+0000","last_peered":"2023-05-16T07:03:09.968997+0000","last_clean":"2023-05-16T07:03:09.968997+0000","last_became_active":"2023-05-16T07:03:09.598000+0000","last_became_peered":"2023-05-16T07:03:09.598000+0000","last_unstale":"2023-05-16T07:03:09.968997+0000","last_undegraded":"2023-05-16T07:03:09.968997+0000","last_fullsized":"2023-05-16T07:03:09.968997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961523+0000","last_change":"2023-05-16T07:03:09.958946+0000","last_active":"2023-05-16T07:03:09.961523+0000","last_peered":"2023-05-16T07:03:09.961523+0000","last_clean":"2023-05-16T07:03:09.961523+0000","last_became_active":"2023-05-16T07:03:09.958685+0000","last_became_peered":"2023-05-16T07:03:09.958685+0000","last_unstale":"2023-05-16T07:03:09.961523+0000","last_undegraded":"2023-05-16T07:03:09.961523+0000","last_fullsized":"2023-05-16T07:03:09.961523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580040+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967380+0000","last_change":"2023-05-16T07:03:09.601574+0000","last_active":"2023-05-16T07:03:09.967380+0000","last_peered":"2023-05-16T07:03:09.967380+0000","last_clean":"2023-05-16T07:03:09.967380+0000","last_became_active":"2023-05-16T07:03:09.601284+0000","last_became_peered":"2023-05-16T07:03:09.601284+0000","last_unstale":"2023-05-16T07:03:09.967380+0000","last_undegraded":"2023-05-16T07:03:09.967380+0000","last_fullsized":"2023-05-16T07:03:09.967380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968186+0000","last_change":"2023-05-16T07:03:09.961726+0000","last_active":"2023-05-16T07:03:09.968186+0000","last_peered":"2023-05-16T07:03:09.968186+0000","last_clean":"2023-05-16T07:03:09.968186+0000","last_became_active":"2023-05-16T07:03:09.961457+0000","last_became_peered":"2023-05-16T07:03:09.961457+0000","last_unstale":"2023-05-16T07:03:09.968186+0000","last_undegraded":"2023-05-16T07:03:09.968186+0000","last_fullsized":"2023-05-16T07:03:09.968186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578086+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579474+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575898+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957527+0000","last_change":"2023-05-16T07:03:09.597577+0000","last_active":"2023-05-16T07:03:09.957527+0000","last_peered":"2023-05-16T07:03:09.957527+0000","last_clean":"2023-05-16T07:03:09.957527+0000","last_became_active":"2023-05-16T07:03:09.597292+0000","last_became_peered":"2023-05-16T07:03:09.597292+0000","last_unstale":"2023-05-16T07:03:09.957527+0000","last_undegraded":"2023-05-16T07:03:09.957527+0000","last_fullsized":"2023-05-16T07:03:09.957527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577091+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579779+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580856+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955244+0000","last_change":"2023-05-16T07:03:09.586854+0000","last_active":"2023-05-16T07:03:09.955244+0000","last_peered":"2023-05-16T07:03:09.955244+0000","last_clean":"2023-05-16T07:03:09.955244+0000","last_became_active":"2023-05-16T07:03:09.586601+0000","last_became_peered":"2023-05-16T07:03:09.586601+0000","last_unstale":"2023-05-16T07:03:09.955244+0000","last_undegraded":"2023-05-16T07:03:09.955244+0000","last_fullsized":"2023-05-16T07:03:09.955244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578847+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578359+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576899+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956453+0000","last_change":"2023-05-16T07:03:09.583122+0000","last_active":"2023-05-16T07:03:09.956453+0000","last_peered":"2023-05-16T07:03:09.956453+0000","last_clean":"2023-05-16T07:03:09.956453+0000","last_became_active":"2023-05-16T07:03:09.582863+0000","last_became_peered":"2023-05-16T07:03:09.582863+0000","last_unstale":"2023-05-16T07:03:09.956453+0000","last_undegraded":"2023-05-16T07:03:09.956453+0000","last_fullsized":"2023-05-16T07:03:09.956453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971194+0000","last_change":"2023-05-16T07:03:09.586289+0000","last_active":"2023-05-16T07:03:09.971194+0000","last_peered":"2023-05-16T07:03:09.971194+0000","last_clean":"2023-05-16T07:03:09.971194+0000","last_became_active":"2023-05-16T07:03:09.583887+0000","last_became_peered":"2023-05-16T07:03:09.583887+0000","last_unstale":"2023-05-16T07:03:09.971194+0000","last_undegraded":"2023-05-16T07:03:09.971194+0000","last_fullsized":"2023-05-16T07:03:09.971194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581559+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576406+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574843+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574233+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957660+0000","last_change":"2023-05-16T07:03:09.589985+0000","last_active":"2023-05-16T07:03:09.957660+0000","last_peered":"2023-05-16T07:03:09.957660+0000","last_clean":"2023-05-16T07:03:09.957660+0000","last_became_active":"2023-05-16T07:03:09.589723+0000","last_became_peered":"2023-05-16T07:03:09.589723+0000","last_unstale":"2023-05-16T07:03:09.957660+0000","last_undegraded":"2023-05-16T07:03:09.957660+0000","last_fullsized":"2023-05-16T07:03:09.957660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-05-16T07:03:10.950904+0000","last_change":"2023-05-16T07:03:09.592795+0000","last_active":"2023-05-16T07:03:10.950904+0000","last_peered":"2023-05-16T07:03:10.950904+0000","last_clean":"2023-05-16T07:03:10.950904+0000","last_became_active":"2023-05-16T07:03:09.592523+0000","last_became_peered":"2023-05-16T07:03:09.592523+0000","last_unstale":"2023-05-16T07:03:10.950904+0000","last_undegraded":"2023-05-16T07:03:10.950904+0000","last_fullsized":"2023-05-16T07:03:10.950904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971455+0000","last_change":"2023-05-16T07:03:09.968547+0000","last_active":"2023-05-16T07:03:09.971455+0000","last_peered":"2023-05-16T07:03:09.971455+0000","last_clean":"2023-05-16T07:03:09.971455+0000","last_became_active":"2023-05-16T07:03:09.968229+0000","last_became_peered":"2023-05-16T07:03:09.968229+0000","last_unstale":"2023-05-16T07:03:09.971455+0000","last_undegraded":"2023-05-16T07:03:09.971455+0000","last_fullsized":"2023-05-16T07:03:09.971455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961282+0000","last_change":"2023-05-16T07:03:09.597726+0000","last_active":"2023-05-16T07:03:09.961282+0000","last_peered":"2023-05-16T07:03:09.961282+0000","last_clean":"2023-05-16T07:03:09.961282+0000","last_became_active":"2023-05-16T07:03:09.591726+0000","last_became_peered":"2023-05-16T07:03:09.591726+0000","last_unstale":"2023-05-16T07:03:09.961282+0000","last_undegraded":"2023-05-16T07:03:09.961282+0000","last_fullsized":"2023-05-16T07:03:09.961282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580181+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970646+0000","last_change":"2023-05-16T07:03:09.964012+0000","last_active":"2023-05-16T07:03:09.970646+0000","last_peered":"2023-05-16T07:03:09.970646+0000","last_clean":"2023-05-16T07:03:09.970646+0000","last_became_active":"2023-05-16T07:03:09.961689+0000","last_became_peered":"2023-05-16T07:03:09.961689+0000","last_unstale":"2023-05-16T07:03:09.970646+0000","last_undegraded":"2023-05-16T07:03:09.970646+0000","last_fullsized":"2023-05-16T07:03:09.970646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575998+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577724+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575953+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579100+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574777+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971493+0000","last_change":"2023-05-16T07:03:09.969463+0000","last_active":"2023-05-16T07:03:09.971493+0000","last_peered":"2023-05-16T07:03:09.971493+0000","last_clean":"2023-05-16T07:03:09.971493+0000","last_became_active":"2023-05-16T07:03:09.968757+0000","last_became_peered":"2023-05-16T07:03:09.968757+0000","last_unstale":"2023-05-16T07:03:09.971493+0000","last_undegraded":"2023-05-16T07:03:09.971493+0000","last_fullsized":"2023-05-16T07:03:09.971493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574279+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970882+0000","last_change":"2023-05-16T07:03:09.591720+0000","last_active":"2023-05-16T07:03:09.970882+0000","last_peered":"2023-05-16T07:03:09.970882+0000","last_clean":"2023-05-16T07:03:09.970882+0000","last_became_active":"2023-05-16T07:03:09.591344+0000","last_became_peered":"2023-05-16T07:03:09.591344+0000","last_unstale":"2023-05-16T07:03:09.970882+0000","last_undegraded":"2023-05-16T07:03:09.970882+0000","last_fullsized":"2023-05-16T07:03:09.970882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970694+0000","last_change":"2023-05-16T07:03:09.968386+0000","last_active":"2023-05-16T07:03:09.970694+0000","last_peered":"2023-05-16T07:03:09.970694+0000","last_clean":"2023-05-16T07:03:09.970694+0000","last_became_active":"2023-05-16T07:03:09.968165+0000","last_became_peered":"2023-05-16T07:03:09.968165+0000","last_unstale":"2023-05-16T07:03:09.970694+0000","last_undegraded":"2023-05-16T07:03:09.970694+0000","last_fullsized":"2023-05-16T07:03:09.970694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576111+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579336+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577122+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955915+0000","last_change":"2023-05-16T07:03:09.599965+0000","last_active":"2023-05-16T07:03:09.955915+0000","last_peered":"2023-05-16T07:03:09.955915+0000","last_clean":"2023-05-16T07:03:09.955915+0000","last_became_active":"2023-05-16T07:03:09.599711+0000","last_became_peered":"2023-05-16T07:03:09.599711+0000","last_unstale":"2023-05-16T07:03:09.955915+0000","last_undegraded":"2023-05-16T07:03:09.955915+0000","last_fullsized":"2023-05-16T07:03:09.955915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.972055+0000","last_change":"2023-05-16T07:03:09.970741+0000","last_active":"2023-05-16T07:03:09.972055+0000","last_peered":"2023-05-16T07:03:09.972055+0000","last_clean":"2023-05-16T07:03:09.972055+0000","last_became_active":"2023-05-16T07:03:09.970463+0000","last_became_peered":"2023-05-16T07:03:09.970463+0000","last_unstale":"2023-05-16T07:03:09.972055+0000","last_undegraded":"2023-05-16T07:03:09.972055+0000","last_fullsized":"2023-05-16T07:03:09.972055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575525+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955985+0000","last_change":"2023-05-16T07:03:09.589805+0000","last_active":"2023-05-16T07:03:09.955985+0000","last_peered":"2023-05-16T07:03:09.955985+0000","last_clean":"2023-05-16T07:03:09.955985+0000","last_became_active":"2023-05-16T07:03:09.589562+0000","last_became_peered":"2023-05-16T07:03:09.589562+0000","last_unstale":"2023-05-16T07:03:09.955985+0000","last_undegraded":"2023-05-16T07:03:09.955985+0000","last_fullsized":"2023-05-16T07:03:09.955985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579486+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578609+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975811+0000","last_change":"2023-05-16T07:03:09.592715+0000","last_active":"2023-05-16T07:03:09.975811+0000","last_peered":"2023-05-16T07:03:09.975811+0000","last_clean":"2023-05-16T07:03:09.975811+0000","last_became_active":"2023-05-16T07:03:09.592445+0000","last_became_peered":"2023-05-16T07:03:09.592445+0000","last_unstale":"2023-05-16T07:03:09.975811+0000","last_undegraded":"2023-05-16T07:03:09.975811+0000","last_fullsized":"2023-05-16T07:03:09.975811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963495+0000","last_change":"2023-05-16T07:03:09.581278+0000","last_active":"2023-05-16T07:03:09.963495+0000","last_peered":"2023-05-16T07:03:09.963495+0000","last_clean":"2023-05-16T07:03:09.963495+0000","last_became_active":"2023-05-16T07:03:09.580950+0000","last_became_peered":"2023-05-16T07:03:09.580950+0000","last_unstale":"2023-05-16T07:03:09.963495+0000","last_undegraded":"2023-05-16T07:03:09.963495+0000","last_fullsized":"2023-05-16T07:03:09.963495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963155+0000","last_change":"2023-05-16T07:03:09.960677+0000","last_active":"2023-05-16T07:03:09.963155+0000","last_peered":"2023-05-16T07:03:09.963155+0000","last_clean":"2023-05-16T07:03:09.963155+0000","last_became_active":"2023-05-16T07:03:09.960434+0000","last_became_peered":"2023-05-16T07:03:09.960434+0000","last_unstale":"2023-05-16T07:03:09.963155+0000","last_undegraded":"2023-05-16T07:03:09.963155+0000","last_fullsized":"2023-05-16T07:03:09.963155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955745+0000","last_change":"2023-05-16T07:03:09.602043+0000","last_active":"2023-05-16T07:03:09.955745+0000","last_peered":"2023-05-16T07:03:09.955745+0000","last_clean":"2023-05-16T07:03:09.955745+0000","last_became_active":"2023-05-16T07:03:09.601724+0000","last_became_peered":"2023-05-16T07:03:09.601724+0000","last_unstale":"2023-05-16T07:03:09.955745+0000","last_undegraded":"2023-05-16T07:03:09.955745+0000","last_fullsized":"2023-05-16T07:03:09.955745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.575511+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968676+0000","last_change":"2023-05-16T07:03:09.962406+0000","last_active":"2023-05-16T07:03:09.968676+0000","last_peered":"2023-05-16T07:03:09.968676+0000","last_clean":"2023-05-16T07:03:09.968676+0000","last_became_active":"2023-05-16T07:03:09.962107+0000","last_became_peered":"2023-05-16T07:03:09.962107+0000","last_unstale":"2023-05-16T07:03:09.968676+0000","last_undegraded":"2023-05-16T07:03:09.968676+0000","last_fullsized":"2023-05-16T07:03:09.968676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578745+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967878+0000","last_change":"2023-05-16T07:03:09.961168+0000","last_active":"2023-05-16T07:03:09.967878+0000","last_peered":"2023-05-16T07:03:09.967878+0000","last_clean":"2023-05-16T07:03:09.967878+0000","last_became_active":"2023-05-16T07:03:09.960908+0000","last_became_peered":"2023-05-16T07:03:09.960908+0000","last_unstale":"2023-05-16T07:03:09.967878+0000","last_undegraded":"2023-05-16T07:03:09.967878+0000","last_fullsized":"2023-05-16T07:03:09.967878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581002+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574852+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581103+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956116+0000","last_change":"2023-05-16T07:03:09.598598+0000","last_active":"2023-05-16T07:03:09.956116+0000","last_peered":"2023-05-16T07:03:09.956116+0000","last_clean":"2023-05-16T07:03:09.956116+0000","last_became_active":"2023-05-16T07:03:09.598382+0000","last_became_peered":"2023-05-16T07:03:09.598382+0000","last_unstale":"2023-05-16T07:03:09.956116+0000","last_undegraded":"2023-05-16T07:03:09.956116+0000","last_fullsized":"2023-05-16T07:03:09.956116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.580469+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.582010+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.577226+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.962297+0000","last_change":"2023-05-16T07:03:09.958211+0000","last_active":"2023-05-16T07:03:09.962297+0000","last_peered":"2023-05-16T07:03:09.962297+0000","last_clean":"2023-05-16T07:03:09.962297+0000","last_became_active":"2023-05-16T07:03:09.957915+0000","last_became_peered":"2023-05-16T07:03:09.957915+0000","last_unstale":"2023-05-16T07:03:09.962297+0000","last_undegraded":"2023-05-16T07:03:09.962297+0000","last_fullsized":"2023-05-16T07:03:09.962297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.579973+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574558+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956188+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:03:09.956188+0000","last_peered":"2023-05-16T07:03:09.956188+0000","last_clean":"2023-05-16T07:03:09.956188+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:03:09.956188+0000","last_undegraded":"2023-05-16T07:03:09.956188+0000","last_fullsized":"2023-05-16T07:03:09.956188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968905+0000","last_change":"2023-05-16T07:03:09.964901+0000","last_active":"2023-05-16T07:03:09.968905+0000","last_peered":"2023-05-16T07:03:09.968905+0000","last_clean":"2023-05-16T07:03:09.968905+0000","last_became_active":"2023-05-16T07:03:09.964643+0000","last_became_peered":"2023-05-16T07:03:09.964643+0000","last_unstale":"2023-05-16T07:03:09.968905+0000","last_undegraded":"2023-05-16T07:03:09.968905+0000","last_fullsized":"2023-05-16T07:03:09.968905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574270+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195849+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:03:04.195849+0000","last_peered":"2023-05-16T07:03:04.195849+0000","last_clean":"2023-05-16T07:03:04.195849+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:03:04.195849+0000","last_undegraded":"2023-05-16T07:03:04.195849+0000","last_fullsized":"2023-05-16T07:03:04.195849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.581564+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195420+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:03:04.195420+0000","last_peered":"2023-05-16T07:03:04.195420+0000","last_clean":"2023-05-16T07:03:04.195420+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:03:04.195420+0000","last_undegraded":"2023-05-16T07:03:04.195420+0000","last_fullsized":"2023-05-16T07:03:04.195420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576948+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195855+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:03:04.195855+0000","last_peered":"2023-05-16T07:03:04.195855+0000","last_clean":"2023-05-16T07:03:04.195855+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:03:04.195855+0000","last_undegraded":"2023-05-16T07:03:04.195855+0000","last_fullsized":"2023-05-16T07:03:04.195855+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195962+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:03:04.195962+0000","last_peered":"2023-05-16T07:03:04.195962+0000","last_clean":"2023-05-16T07:03:04.195962+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:03:04.195962+0000","last_undegraded":"2023-05-16T07:03:04.195962+0000","last_fullsized":"2023-05-16T07:03:04.195962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971163+0000","last_change":"2023-05-16T07:03:09.968710+0000","last_active":"2023-05-16T07:03:09.971163+0000","last_peered":"2023-05-16T07:03:09.971163+0000","last_clean":"2023-05-16T07:03:09.971163+0000","last_became_active":"2023-05-16T07:03:09.968451+0000","last_became_peered":"2023-05-16T07:03:09.968451+0000","last_unstale":"2023-05-16T07:03:09.971163+0000","last_undegraded":"2023-05-16T07:03:09.971163+0000","last_fullsized":"2023-05-16T07:03:09.971163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.578664+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195958+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:03:04.195958+0000","last_peered":"2023-05-16T07:03:04.195958+0000","last_clean":"2023-05-16T07:03:04.195958+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:03:04.195958+0000","last_undegraded":"2023-05-16T07:03:04.195958+0000","last_fullsized":"2023-05-16T07:03:04.195958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970552+0000","last_change":"2023-05-16T07:03:09.964329+0000","last_active":"2023-05-16T07:03:09.970552+0000","last_peered":"2023-05-16T07:03:09.970552+0000","last_clean":"2023-05-16T07:03:09.970552+0000","last_became_active":"2023-05-16T07:03:09.960866+0000","last_became_peered":"2023-05-16T07:03:09.960866+0000","last_unstale":"2023-05-16T07:03:09.970552+0000","last_undegraded":"2023-05-16T07:03:09.970552+0000","last_fullsized":"2023-05-16T07:03:09.970552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.574770+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:04.195984+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:03:04.195984+0000","last_peered":"2023-05-16T07:03:04.195984+0000","last_clean":"2023-05-16T07:03:04.195984+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:03:04.195984+0000","last_undegraded":"2023-05-16T07:03:04.195984+0000","last_fullsized":"2023-05-16T07:03:04.195984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-05-16T07:03:08.567973+0000","last_change":"2023-05-16T07:03:08.576596+0000","last_active":"2023-05-16T07:03:08.567973+0000","last_peered":"2023-05-16T07:03:08.567973+0000","last_clean":"2023-05-16T07:03:08.567973+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:08.567973+0000","last_undegraded":"2023-05-16T07:03:08.567973+0000","last_fullsized":"2023-05-16T07:03:08.567973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957557+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:03:09.957557+0000","last_peered":"2023-05-16T07:03:09.957557+0000","last_clean":"2023-05-16T07:03:09.957557+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:03:09.957557+0000","last_undegraded":"2023-05-16T07:03:09.957557+0000","last_fullsized":"2023-05-16T07:03:09.957557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-16T07:03:09.571960+0000","last_change":"2023-05-16T07:03:09.571960+0000","last_active":"2023-05-16T07:03:09.571960+0000","last_peered":"2023-05-16T07:03:09.571960+0000","last_clean":"2023-05-16T07:03:09.571960+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-16T07:03:09.571960+0000","last_undegraded":"2023-05-16T07:03:09.571960+0000","last_fullsized":"2023-05-16T07:03:09.571960+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.954982+0000","last_change":"2023-05-16T07:03:09.590083+0000","last_active":"2023-05-16T07:03:09.954982+0000","last_peered":"2023-05-16T07:03:09.954982+0000","last_clean":"2023-05-16T07:03:09.954982+0000","last_became_active":"2023-05-16T07:03:09.589801+0000","last_became_peered":"2023-05-16T07:03:09.589801+0000","last_unstale":"2023-05-16T07:03:09.954982+0000","last_undegraded":"2023-05-16T07:03:09.954982+0000","last_fullsized":"2023-05-16T07:03:09.954982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-05-16T07:03:03.735035+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:03:03.735035+0000","last_peered":"2023-05-16T07:03:03.735035+0000","last_clean":"2023-05-16T07:03:03.735035+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:03:03.735035+0000","last_undegraded":"2023-05-16T07:03:03.735035+0000","last_fullsized":"2023-05-16T07:03:03.735035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":2},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":748,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5640,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366200,"statfs":{"total":96636764160,"available":96630988800,"internally_reserved":0,"allocated":393216,"data_stored":158860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5456,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366384,"statfs":{"total":96636764160,"available":96631177216,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5328,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366512,"statfs":{"total":96636764160,"available":96631308288,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366060,"statfs":{"total":96636764160,"available":96630845440,"internally_reserved":0,"allocated":405504,"data_stored":160340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5520,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366320,"statfs":{"total":96636764160,"available":96631111680,"internally_reserved":0,"allocated":335872,"data_stored":131917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5404,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366436,"statfs":{"total":96636764160,"available":96631230464,"internally_reserved":0,"allocated":348160,"data_stored":138138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:03:17.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:03:17.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.492+0000 7fbb3e4e9700 1 Processor -- start 2023-05-16T07:03:17.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.492+0000 7fbb3e4e9700 1 -- start start 2023-05-16T07:03:17.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb3e4e9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbb38154550 0x7fbb380696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb3e4e9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 0x7fbb380c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb3e4e9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380c0660 con 0x7fbb38154170 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb3e4e9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3806a000 con 0x7fbb38154550 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb3e4e9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380bfd30 con 0x7fbb38069b90 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb37fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbb38154170 0x7fbb380c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52984/0 (socket says 172.21.15.92:52984) 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.493+0000 7fbb37fff700 1 -- 172.21.15.92:0/4206540840 learned_addr learned my addr 172.21.15.92:0/4206540840 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:17.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397527242 0 0) 0x7fbb380c0660 con 0x7fbb38154170 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb3ca86700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 0x7fbb380c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb240033f0 con 0x7fbb38154170 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb2ffff700 1 --2- 172.21.15.92:0/4206540840 >> v2:172.21.15.158:3300/0 conn(0x7fbb38154550 0x7fbb380696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64757957 0 0) 0x7fbb240033f0 con 0x7fbb38154170 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 msgr2=0x7fbb380c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 0x7fbb380c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 >> v2:172.21.15.158:3300/0 conn(0x7fbb38154550 msgr2=0x7fbb380696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 --2- 172.21.15.92:0/4206540840 >> v2:172.21.15.158:3300/0 conn(0x7fbb38154550 0x7fbb380696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380c0010 con 0x7fbb38154170 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fbb28001470 con 0x7fbb38154170 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 >> v1:172.21.15.92:6789/0 conn(0x7fbb38154170 legacy=0x7fbb380c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb240038e0 0x7fbb24003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 0x7fbb24004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb24004b80 0x7fbb24005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3806a000 con 0x7fbb240038e0 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380bfd30 con 0x7fbb24004b80 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2400b8e0 con 0x7fbb38069b90 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb37fff700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb240038e0 0x7fbb24003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb2ffff700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 0x7fbb24004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb3ca86700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb24004b80 0x7fbb24005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb37fff700 1 -- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 msgr2=0x7fbb24004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:17.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb37fff700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb38069b90 0x7fbb24004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb37fff700 1 -- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb24004b80 msgr2=0x7fbb24005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb37fff700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb24004b80 0x7fbb24005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb37fff700 1 -- 172.21.15.92:0/4206540840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240033f0 con 0x7fbb240038e0 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.494+0000 7fbb3ca86700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb24004b80 0x7fbb24005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb37fff700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb240038e0 0x7fbb24003d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fbb2800c210 tx=0x7fbb28001e90).ready entity=mon.0 client_cookie=c8a660f4a3b8827b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28007a60 con 0x7fbb240038e0 2023-05-16T07:03:17.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2800cae0 con 0x7fbb240038e0 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb377fe700 1 -- 172.21.15.92:0/4206540840 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800ce60 con 0x7fbb240038e0 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/4206540840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb240038e0 msgr2=0x7fbb24003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/4206540840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb240038e0 0x7fbb24003d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fbb2800c210 tx=0x7fbb28001e90).stop 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3ca86700 1 -- 172.21.15.92:0/4206540840 reap_dead start 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/4206540840 shutdown_connections 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/4206540840 >> 172.21.15.92:0/4206540840 conn(0x7fbb380cb700 msgr2=0x7fbb380c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:17.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/4206540840 shutdown_connections 2023-05-16T07:03:17.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.495+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/4206540840 wait complete. 2023-05-16T07:03:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.496+0000 7fbb3e4e9700 1 Processor -- start 2023-05-16T07:03:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.496+0000 7fbb3e4e9700 1 -- start start 2023-05-16T07:03:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3e4e9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb240038e0 0x7fbb38168780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3e4e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 0x7fbb3814cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3e4e9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3814d050 0x7fbb3815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3e4e9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380b0220 con 0x7fbb380c22c0 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3e4e9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb38168c70 con 0x7fbb3814d050 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3e4e9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380c27a0 con 0x7fbb240038e0 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 0x7fbb3814cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 0x7fbb3814cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42250/0 (socket says 172.21.15.92:42250) 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 -- 172.21.15.92:0/3152398084 learned_addr learned my addr 172.21.15.92:0/3152398084 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb3ca86700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3814d050 0x7fbb3815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb37fff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb240038e0 0x7fbb38168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 -- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb240038e0 msgr2=0x7fbb38168780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb240038e0 0x7fbb38168780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 -- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3814d050 msgr2=0x7fbb3815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.497+0000 7fbb2ffff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3814d050 0x7fbb3815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb2ffff700 1 -- 172.21.15.92:0/3152398084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380c2a80 con 0x7fbb380c22c0 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb37fff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb240038e0 0x7fbb38168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:17.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb2ffff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 0x7fbb3814cb60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fbb200119f0 tx=0x7fbb20009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000f8c0 con 0x7fbb380c22c0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000fa20 con 0x7fbb380c22c0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb380c2d60 con 0x7fbb380c22c0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000fda0 con 0x7fbb380c22c0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.498+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb380c3320 con 0x7fbb380c22c0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.499+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbb20024070 con 0x7fbb380c22c0 2023-05-16T07:03:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.499+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb380c8d60 con 0x7fbb380c22c0 2023-05-16T07:03:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.499+0000 7fbb34ff9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1805ed90 0x7fbb18061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.499+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbb2007a5f0 con 0x7fbb380c22c0 2023-05-16T07:03:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.499+0000 7fbb37fff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1805ed90 0x7fbb18061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.500+0000 7fbb37fff700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1805ed90 0x7fbb18061250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fbb28001ed0 tx=0x7fbb28015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.503+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbb2004ba80 con 0x7fbb380c22c0 2023-05-16T07:03:17.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.666+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb380666c0 con 0x7fbb1805ed90 2023-05-16T07:03:17.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.677+0000 7fbb34ff9700 1 -- 172.21.15.92:0/3152398084 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336444 (secure 0 0 0) 0x7fbb380666c0 con 0x7fbb1805ed90 2023-05-16T07:03:17.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1805ed90 msgr2=0x7fbb18061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1805ed90 0x7fbb18061250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fbb28001ed0 tx=0x7fbb28015000).stop 2023-05-16T07:03:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 msgr2=0x7fbb3814cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 0x7fbb3814cb60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fbb200119f0 tx=0x7fbb20009060).stop 2023-05-16T07:03:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 shutdown_connections 2023-05-16T07:03:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1805ed90 0x7fbb18061250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb240038e0 0x7fbb38168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3814d050 0x7fbb3815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 --2- 172.21.15.92:0/3152398084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb380c22c0 0x7fbb3814cb60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 >> 172.21.15.92:0/3152398084 conn(0x7fbb380cb700 msgr2=0x7fbb380ac0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 shutdown_connections 2023-05-16T07:03:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.680+0000 7fbb3e4e9700 1 -- 172.21.15.92:0/3152398084 wait complete. 2023-05-16T07:03:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:03:17.696 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2023-05-16T07:03:17.465108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":263,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":45376,"kb_used_data":3072,"kb_used_omap":0,"kb_used_meta":42240,"kb_avail":754929344,"statfs":{"total":773094113280,"available":773047648256,"internally_reserved":0,"allocated":3145728,"data_stored":1274212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43253760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2759,"num_objects":-7,"num_object_clones":0,"num_object_copies":-14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-16,"num_write_kb":-10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.085063"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970323+0000","last_change":"2023-05-16T07:03:09.969708+0000","last_active":"2023-05-16T07:03:09.970323+0000","last_peered":"2023-05-16T07:03:09.970323+0000","last_clean":"2023-05-16T07:03:09.970323+0000","last_became_active":"2023-05-16T07:03:09.969440+0000","last_became_peered":"2023-05-16T07:03:09.969440+0000","last_unstale":"2023-05-16T07:03:09.970323+0000","last_undegraded":"2023-05-16T07:03:09.970323+0000","last_fullsized":"2023-05-16T07:03:09.970323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969644+0000","last_change":"2023-05-16T07:03:09.603326+0000","last_active":"2023-05-16T07:03:09.969644+0000","last_peered":"2023-05-16T07:03:09.969644+0000","last_clean":"2023-05-16T07:03:09.969644+0000","last_became_active":"2023-05-16T07:03:09.602980+0000","last_became_peered":"2023-05-16T07:03:09.602980+0000","last_unstale":"2023-05-16T07:03:09.969644+0000","last_undegraded":"2023-05-16T07:03:09.969644+0000","last_fullsized":"2023-05-16T07:03:09.969644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970682+0000","last_change":"2023-05-16T07:03:09.968357+0000","last_active":"2023-05-16T07:03:09.970682+0000","last_peered":"2023-05-16T07:03:09.970682+0000","last_clean":"2023-05-16T07:03:09.970682+0000","last_became_active":"2023-05-16T07:03:09.968085+0000","last_became_peered":"2023-05-16T07:03:09.968085+0000","last_unstale":"2023-05-16T07:03:09.970682+0000","last_undegraded":"2023-05-16T07:03:09.970682+0000","last_fullsized":"2023-05-16T07:03:09.970682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956551+0000","last_change":"2023-05-16T07:03:09.592121+0000","last_active":"2023-05-16T07:03:09.956551+0000","last_peered":"2023-05-16T07:03:09.956551+0000","last_clean":"2023-05-16T07:03:09.956551+0000","last_became_active":"2023-05-16T07:03:09.590810+0000","last_became_peered":"2023-05-16T07:03:09.590810+0000","last_unstale":"2023-05-16T07:03:09.956551+0000","last_undegraded":"2023-05-16T07:03:09.956551+0000","last_fullsized":"2023-05-16T07:03:09.956551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967924+0000","last_change":"2023-05-16T07:03:09.961688+0000","last_active":"2023-05-16T07:03:09.967924+0000","last_peered":"2023-05-16T07:03:09.967924+0000","last_clean":"2023-05-16T07:03:09.967924+0000","last_became_active":"2023-05-16T07:03:09.961428+0000","last_became_peered":"2023-05-16T07:03:09.961428+0000","last_unstale":"2023-05-16T07:03:09.967924+0000","last_undegraded":"2023-05-16T07:03:09.967924+0000","last_fullsized":"2023-05-16T07:03:09.967924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.958895+0000","last_change":"2023-05-16T07:03:09.602060+0000","last_active":"2023-05-16T07:03:09.958895+0000","last_peered":"2023-05-16T07:03:09.958895+0000","last_clean":"2023-05-16T07:03:09.958895+0000","last_became_active":"2023-05-16T07:03:09.601899+0000","last_became_peered":"2023-05-16T07:03:09.601899+0000","last_unstale":"2023-05-16T07:03:09.958895+0000","last_undegraded":"2023-05-16T07:03:09.958895+0000","last_fullsized":"2023-05-16T07:03:09.958895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968927+0000","last_change":"2023-05-16T07:03:09.963887+0000","last_active":"2023-05-16T07:03:09.968927+0000","last_peered":"2023-05-16T07:03:09.968927+0000","last_clean":"2023-05-16T07:03:09.968927+0000","last_became_active":"2023-05-16T07:03:09.963611+0000","last_became_peered":"2023-05-16T07:03:09.963611+0000","last_unstale":"2023-05-16T07:03:09.968927+0000","last_undegraded":"2023-05-16T07:03:09.968927+0000","last_fullsized":"2023-05-16T07:03:09.968927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968997+0000","last_change":"2023-05-16T07:03:09.598293+0000","last_active":"2023-05-16T07:03:09.968997+0000","last_peered":"2023-05-16T07:03:09.968997+0000","last_clean":"2023-05-16T07:03:09.968997+0000","last_became_active":"2023-05-16T07:03:09.598000+0000","last_became_peered":"2023-05-16T07:03:09.598000+0000","last_unstale":"2023-05-16T07:03:09.968997+0000","last_undegraded":"2023-05-16T07:03:09.968997+0000","last_fullsized":"2023-05-16T07:03:09.968997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961523+0000","last_change":"2023-05-16T07:03:09.958946+0000","last_active":"2023-05-16T07:03:09.961523+0000","last_peered":"2023-05-16T07:03:09.961523+0000","last_clean":"2023-05-16T07:03:09.961523+0000","last_became_active":"2023-05-16T07:03:09.958685+0000","last_became_peered":"2023-05-16T07:03:09.958685+0000","last_unstale":"2023-05-16T07:03:09.961523+0000","last_undegraded":"2023-05-16T07:03:09.961523+0000","last_fullsized":"2023-05-16T07:03:09.961523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.962531+0000","last_change":"2023-05-16T07:03:09.595702+0000","last_active":"2023-05-16T07:03:09.962531+0000","last_peered":"2023-05-16T07:03:09.962531+0000","last_clean":"2023-05-16T07:03:09.962531+0000","last_became_active":"2023-05-16T07:03:09.595214+0000","last_became_peered":"2023-05-16T07:03:09.595214+0000","last_unstale":"2023-05-16T07:03:09.962531+0000","last_undegraded":"2023-05-16T07:03:09.962531+0000","last_fullsized":"2023-05-16T07:03:09.962531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967830+0000","last_change":"2023-05-16T07:03:09.961061+0000","last_active":"2023-05-16T07:03:09.967830+0000","last_peered":"2023-05-16T07:03:09.967830+0000","last_clean":"2023-05-16T07:03:09.967830+0000","last_became_active":"2023-05-16T07:03:09.960798+0000","last_became_peered":"2023-05-16T07:03:09.960798+0000","last_unstale":"2023-05-16T07:03:09.967830+0000","last_undegraded":"2023-05-16T07:03:09.967830+0000","last_fullsized":"2023-05-16T07:03:09.967830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967380+0000","last_change":"2023-05-16T07:03:09.601574+0000","last_active":"2023-05-16T07:03:09.967380+0000","last_peered":"2023-05-16T07:03:09.967380+0000","last_clean":"2023-05-16T07:03:09.967380+0000","last_became_active":"2023-05-16T07:03:09.601284+0000","last_became_peered":"2023-05-16T07:03:09.601284+0000","last_unstale":"2023-05-16T07:03:09.967380+0000","last_undegraded":"2023-05-16T07:03:09.967380+0000","last_fullsized":"2023-05-16T07:03:09.967380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968186+0000","last_change":"2023-05-16T07:03:09.961726+0000","last_active":"2023-05-16T07:03:09.968186+0000","last_peered":"2023-05-16T07:03:09.968186+0000","last_clean":"2023-05-16T07:03:09.968186+0000","last_became_active":"2023-05-16T07:03:09.961457+0000","last_became_peered":"2023-05-16T07:03:09.961457+0000","last_unstale":"2023-05-16T07:03:09.968186+0000","last_undegraded":"2023-05-16T07:03:09.968186+0000","last_fullsized":"2023-05-16T07:03:09.968186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952671+0000","last_change":"2023-05-16T07:03:09.593962+0000","last_active":"2023-05-16T07:03:09.952671+0000","last_peered":"2023-05-16T07:03:09.952671+0000","last_clean":"2023-05-16T07:03:09.952671+0000","last_became_active":"2023-05-16T07:03:09.593492+0000","last_became_peered":"2023-05-16T07:03:09.593492+0000","last_unstale":"2023-05-16T07:03:09.952671+0000","last_undegraded":"2023-05-16T07:03:09.952671+0000","last_fullsized":"2023-05-16T07:03:09.952671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966293+0000","last_change":"2023-05-16T07:03:09.964493+0000","last_active":"2023-05-16T07:03:09.966293+0000","last_peered":"2023-05-16T07:03:09.966293+0000","last_clean":"2023-05-16T07:03:09.966293+0000","last_became_active":"2023-05-16T07:03:09.964229+0000","last_became_peered":"2023-05-16T07:03:09.964229+0000","last_unstale":"2023-05-16T07:03:09.966293+0000","last_undegraded":"2023-05-16T07:03:09.966293+0000","last_fullsized":"2023-05-16T07:03:09.966293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.954467+0000","last_change":"2023-05-16T07:03:09.594827+0000","last_active":"2023-05-16T07:03:09.954467+0000","last_peered":"2023-05-16T07:03:09.954467+0000","last_clean":"2023-05-16T07:03:09.954467+0000","last_became_active":"2023-05-16T07:03:09.594520+0000","last_became_peered":"2023-05-16T07:03:09.594520+0000","last_unstale":"2023-05-16T07:03:09.954467+0000","last_undegraded":"2023-05-16T07:03:09.954467+0000","last_fullsized":"2023-05-16T07:03:09.954467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963230+0000","last_change":"2023-05-16T07:03:09.959683+0000","last_active":"2023-05-16T07:03:09.963230+0000","last_peered":"2023-05-16T07:03:09.963230+0000","last_clean":"2023-05-16T07:03:09.963230+0000","last_became_active":"2023-05-16T07:03:09.959416+0000","last_became_peered":"2023-05-16T07:03:09.959416+0000","last_unstale":"2023-05-16T07:03:09.963230+0000","last_undegraded":"2023-05-16T07:03:09.963230+0000","last_fullsized":"2023-05-16T07:03:09.963230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952139+0000","last_change":"2023-05-16T07:03:09.585865+0000","last_active":"2023-05-16T07:03:09.952139+0000","last_peered":"2023-05-16T07:03:09.952139+0000","last_clean":"2023-05-16T07:03:09.952139+0000","last_became_active":"2023-05-16T07:03:09.585584+0000","last_became_peered":"2023-05-16T07:03:09.585584+0000","last_unstale":"2023-05-16T07:03:09.952139+0000","last_undegraded":"2023-05-16T07:03:09.952139+0000","last_fullsized":"2023-05-16T07:03:09.952139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.972149+0000","last_change":"2023-05-16T07:03:09.968978+0000","last_active":"2023-05-16T07:03:09.972149+0000","last_peered":"2023-05-16T07:03:09.972149+0000","last_clean":"2023-05-16T07:03:09.972149+0000","last_became_active":"2023-05-16T07:03:09.968551+0000","last_became_peered":"2023-05-16T07:03:09.968551+0000","last_unstale":"2023-05-16T07:03:09.972149+0000","last_undegraded":"2023-05-16T07:03:09.972149+0000","last_fullsized":"2023-05-16T07:03:09.972149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957527+0000","last_change":"2023-05-16T07:03:09.597577+0000","last_active":"2023-05-16T07:03:09.957527+0000","last_peered":"2023-05-16T07:03:09.957527+0000","last_clean":"2023-05-16T07:03:09.957527+0000","last_became_active":"2023-05-16T07:03:09.597292+0000","last_became_peered":"2023-05-16T07:03:09.597292+0000","last_unstale":"2023-05-16T07:03:09.957527+0000","last_undegraded":"2023-05-16T07:03:09.957527+0000","last_fullsized":"2023-05-16T07:03:09.957527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.976603+0000","last_change":"2023-05-16T07:03:09.976197+0000","last_active":"2023-05-16T07:03:09.976603+0000","last_peered":"2023-05-16T07:03:09.976603+0000","last_clean":"2023-05-16T07:03:09.976603+0000","last_became_active":"2023-05-16T07:03:09.975874+0000","last_became_peered":"2023-05-16T07:03:09.975874+0000","last_unstale":"2023-05-16T07:03:09.976603+0000","last_undegraded":"2023-05-16T07:03:09.976603+0000","last_fullsized":"2023-05-16T07:03:09.976603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952323+0000","last_change":"2023-05-16T07:03:09.600103+0000","last_active":"2023-05-16T07:03:09.952323+0000","last_peered":"2023-05-16T07:03:09.952323+0000","last_clean":"2023-05-16T07:03:09.952323+0000","last_became_active":"2023-05-16T07:03:09.599761+0000","last_became_peered":"2023-05-16T07:03:09.599761+0000","last_unstale":"2023-05-16T07:03:09.952323+0000","last_undegraded":"2023-05-16T07:03:09.952323+0000","last_fullsized":"2023-05-16T07:03:09.952323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963236+0000","last_change":"2023-05-16T07:03:09.959845+0000","last_active":"2023-05-16T07:03:09.963236+0000","last_peered":"2023-05-16T07:03:09.963236+0000","last_clean":"2023-05-16T07:03:09.963236+0000","last_became_active":"2023-05-16T07:03:09.959565+0000","last_became_peered":"2023-05-16T07:03:09.959565+0000","last_unstale":"2023-05-16T07:03:09.963236+0000","last_undegraded":"2023-05-16T07:03:09.963236+0000","last_fullsized":"2023-05-16T07:03:09.963236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970056+0000","last_change":"2023-05-16T07:03:09.601705+0000","last_active":"2023-05-16T07:03:09.970056+0000","last_peered":"2023-05-16T07:03:09.970056+0000","last_clean":"2023-05-16T07:03:09.970056+0000","last_became_active":"2023-05-16T07:03:09.601110+0000","last_became_peered":"2023-05-16T07:03:09.601110+0000","last_unstale":"2023-05-16T07:03:09.970056+0000","last_undegraded":"2023-05-16T07:03:09.970056+0000","last_fullsized":"2023-05-16T07:03:09.970056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.976201+0000","last_change":"2023-05-16T07:03:09.971457+0000","last_active":"2023-05-16T07:03:09.976201+0000","last_peered":"2023-05-16T07:03:09.976201+0000","last_clean":"2023-05-16T07:03:09.976201+0000","last_became_active":"2023-05-16T07:03:09.971215+0000","last_became_peered":"2023-05-16T07:03:09.971215+0000","last_unstale":"2023-05-16T07:03:09.976201+0000","last_undegraded":"2023-05-16T07:03:09.976201+0000","last_fullsized":"2023-05-16T07:03:09.976201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968865+0000","last_change":"2023-05-16T07:03:09.593988+0000","last_active":"2023-05-16T07:03:09.968865+0000","last_peered":"2023-05-16T07:03:09.968865+0000","last_clean":"2023-05-16T07:03:09.968865+0000","last_became_active":"2023-05-16T07:03:09.593731+0000","last_became_peered":"2023-05-16T07:03:09.593731+0000","last_unstale":"2023-05-16T07:03:09.968865+0000","last_undegraded":"2023-05-16T07:03:09.968865+0000","last_fullsized":"2023-05-16T07:03:09.968865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969441+0000","last_change":"2023-05-16T07:03:09.964586+0000","last_active":"2023-05-16T07:03:09.969441+0000","last_peered":"2023-05-16T07:03:09.969441+0000","last_clean":"2023-05-16T07:03:09.969441+0000","last_became_active":"2023-05-16T07:03:09.964294+0000","last_became_peered":"2023-05-16T07:03:09.964294+0000","last_unstale":"2023-05-16T07:03:09.969441+0000","last_undegraded":"2023-05-16T07:03:09.969441+0000","last_fullsized":"2023-05-16T07:03:09.969441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955244+0000","last_change":"2023-05-16T07:03:09.586854+0000","last_active":"2023-05-16T07:03:09.955244+0000","last_peered":"2023-05-16T07:03:09.955244+0000","last_clean":"2023-05-16T07:03:09.955244+0000","last_became_active":"2023-05-16T07:03:09.586601+0000","last_became_peered":"2023-05-16T07:03:09.586601+0000","last_unstale":"2023-05-16T07:03:09.955244+0000","last_undegraded":"2023-05-16T07:03:09.955244+0000","last_fullsized":"2023-05-16T07:03:09.955244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.964330+0000","last_change":"2023-05-16T07:03:09.959864+0000","last_active":"2023-05-16T07:03:09.964330+0000","last_peered":"2023-05-16T07:03:09.964330+0000","last_clean":"2023-05-16T07:03:09.964330+0000","last_became_active":"2023-05-16T07:03:09.958562+0000","last_became_peered":"2023-05-16T07:03:09.958562+0000","last_unstale":"2023-05-16T07:03:09.964330+0000","last_undegraded":"2023-05-16T07:03:09.964330+0000","last_fullsized":"2023-05-16T07:03:09.964330+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.954998+0000","last_change":"2023-05-16T07:03:09.599836+0000","last_active":"2023-05-16T07:03:09.954998+0000","last_peered":"2023-05-16T07:03:09.954998+0000","last_clean":"2023-05-16T07:03:09.954998+0000","last_became_active":"2023-05-16T07:03:09.599571+0000","last_became_peered":"2023-05-16T07:03:09.599571+0000","last_unstale":"2023-05-16T07:03:09.954998+0000","last_undegraded":"2023-05-16T07:03:09.954998+0000","last_fullsized":"2023-05-16T07:03:09.954998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.976062+0000","last_change":"2023-05-16T07:03:09.975296+0000","last_active":"2023-05-16T07:03:09.976062+0000","last_peered":"2023-05-16T07:03:09.976062+0000","last_clean":"2023-05-16T07:03:09.976062+0000","last_became_active":"2023-05-16T07:03:09.974998+0000","last_became_peered":"2023-05-16T07:03:09.974998+0000","last_unstale":"2023-05-16T07:03:09.976062+0000","last_undegraded":"2023-05-16T07:03:09.976062+0000","last_fullsized":"2023-05-16T07:03:09.976062+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.980827+0000","last_change":"2023-05-16T07:03:09.596120+0000","last_active":"2023-05-16T07:03:09.980827+0000","last_peered":"2023-05-16T07:03:09.980827+0000","last_clean":"2023-05-16T07:03:09.980827+0000","last_became_active":"2023-05-16T07:03:09.595851+0000","last_became_peered":"2023-05-16T07:03:09.595851+0000","last_unstale":"2023-05-16T07:03:09.980827+0000","last_undegraded":"2023-05-16T07:03:09.980827+0000","last_fullsized":"2023-05-16T07:03:09.980827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966353+0000","last_change":"2023-05-16T07:03:09.960348+0000","last_active":"2023-05-16T07:03:09.966353+0000","last_peered":"2023-05-16T07:03:09.966353+0000","last_clean":"2023-05-16T07:03:09.966353+0000","last_became_active":"2023-05-16T07:03:09.960008+0000","last_became_peered":"2023-05-16T07:03:09.960008+0000","last_unstale":"2023-05-16T07:03:09.966353+0000","last_undegraded":"2023-05-16T07:03:09.966353+0000","last_fullsized":"2023-05-16T07:03:09.966353+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952971+0000","last_change":"2023-05-16T07:03:09.593975+0000","last_active":"2023-05-16T07:03:09.952971+0000","last_peered":"2023-05-16T07:03:09.952971+0000","last_clean":"2023-05-16T07:03:09.952971+0000","last_became_active":"2023-05-16T07:03:09.593483+0000","last_became_peered":"2023-05-16T07:03:09.593483+0000","last_unstale":"2023-05-16T07:03:09.952971+0000","last_undegraded":"2023-05-16T07:03:09.952971+0000","last_fullsized":"2023-05-16T07:03:09.952971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969906+0000","last_change":"2023-05-16T07:03:09.969326+0000","last_active":"2023-05-16T07:03:09.969906+0000","last_peered":"2023-05-16T07:03:09.969906+0000","last_clean":"2023-05-16T07:03:09.969906+0000","last_became_active":"2023-05-16T07:03:09.969073+0000","last_became_peered":"2023-05-16T07:03:09.969073+0000","last_unstale":"2023-05-16T07:03:09.969906+0000","last_undegraded":"2023-05-16T07:03:09.969906+0000","last_fullsized":"2023-05-16T07:03:09.969906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956453+0000","last_change":"2023-05-16T07:03:09.583122+0000","last_active":"2023-05-16T07:03:09.956453+0000","last_peered":"2023-05-16T07:03:09.956453+0000","last_clean":"2023-05-16T07:03:09.956453+0000","last_became_active":"2023-05-16T07:03:09.582863+0000","last_became_peered":"2023-05-16T07:03:09.582863+0000","last_unstale":"2023-05-16T07:03:09.956453+0000","last_undegraded":"2023-05-16T07:03:09.956453+0000","last_fullsized":"2023-05-16T07:03:09.956453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966452+0000","last_change":"2023-05-16T07:03:09.960249+0000","last_active":"2023-05-16T07:03:09.966452+0000","last_peered":"2023-05-16T07:03:09.966452+0000","last_clean":"2023-05-16T07:03:09.966452+0000","last_became_active":"2023-05-16T07:03:09.959715+0000","last_became_peered":"2023-05-16T07:03:09.959715+0000","last_unstale":"2023-05-16T07:03:09.966452+0000","last_undegraded":"2023-05-16T07:03:09.966452+0000","last_fullsized":"2023-05-16T07:03:09.966452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971194+0000","last_change":"2023-05-16T07:03:09.586289+0000","last_active":"2023-05-16T07:03:09.971194+0000","last_peered":"2023-05-16T07:03:09.971194+0000","last_clean":"2023-05-16T07:03:09.971194+0000","last_became_active":"2023-05-16T07:03:09.583887+0000","last_became_peered":"2023-05-16T07:03:09.583887+0000","last_unstale":"2023-05-16T07:03:09.971194+0000","last_undegraded":"2023-05-16T07:03:09.971194+0000","last_fullsized":"2023-05-16T07:03:09.971194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975403+0000","last_change":"2023-05-16T07:03:09.966293+0000","last_active":"2023-05-16T07:03:09.975403+0000","last_peered":"2023-05-16T07:03:09.975403+0000","last_clean":"2023-05-16T07:03:09.975403+0000","last_became_active":"2023-05-16T07:03:09.962111+0000","last_became_peered":"2023-05-16T07:03:09.962111+0000","last_unstale":"2023-05-16T07:03:09.975403+0000","last_undegraded":"2023-05-16T07:03:09.975403+0000","last_fullsized":"2023-05-16T07:03:09.975403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.954033+0000","last_change":"2023-05-16T07:03:09.602879+0000","last_active":"2023-05-16T07:03:09.954033+0000","last_peered":"2023-05-16T07:03:09.954033+0000","last_clean":"2023-05-16T07:03:09.954033+0000","last_became_active":"2023-05-16T07:03:09.602572+0000","last_became_peered":"2023-05-16T07:03:09.602572+0000","last_unstale":"2023-05-16T07:03:09.954033+0000","last_undegraded":"2023-05-16T07:03:09.954033+0000","last_fullsized":"2023-05-16T07:03:09.954033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.980666+0000","last_change":"2023-05-16T07:03:09.975127+0000","last_active":"2023-05-16T07:03:09.980666+0000","last_peered":"2023-05-16T07:03:09.980666+0000","last_clean":"2023-05-16T07:03:09.980666+0000","last_became_active":"2023-05-16T07:03:09.974832+0000","last_became_peered":"2023-05-16T07:03:09.974832+0000","last_unstale":"2023-05-16T07:03:09.980666+0000","last_undegraded":"2023-05-16T07:03:09.980666+0000","last_fullsized":"2023-05-16T07:03:09.980666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963873+0000","last_change":"2023-05-16T07:03:09.586463+0000","last_active":"2023-05-16T07:03:09.963873+0000","last_peered":"2023-05-16T07:03:09.963873+0000","last_clean":"2023-05-16T07:03:09.963873+0000","last_became_active":"2023-05-16T07:03:09.586162+0000","last_became_peered":"2023-05-16T07:03:09.586162+0000","last_unstale":"2023-05-16T07:03:09.963873+0000","last_undegraded":"2023-05-16T07:03:09.963873+0000","last_fullsized":"2023-05-16T07:03:09.963873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975654+0000","last_change":"2023-05-16T07:03:09.971820+0000","last_active":"2023-05-16T07:03:09.975654+0000","last_peered":"2023-05-16T07:03:09.975654+0000","last_clean":"2023-05-16T07:03:09.975654+0000","last_became_active":"2023-05-16T07:03:09.969927+0000","last_became_peered":"2023-05-16T07:03:09.969927+0000","last_unstale":"2023-05-16T07:03:09.975654+0000","last_undegraded":"2023-05-16T07:03:09.975654+0000","last_fullsized":"2023-05-16T07:03:09.975654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951153+0000","last_change":"2023-05-16T07:03:09.591790+0000","last_active":"2023-05-16T07:03:09.951153+0000","last_peered":"2023-05-16T07:03:09.951153+0000","last_clean":"2023-05-16T07:03:09.951153+0000","last_became_active":"2023-05-16T07:03:09.591514+0000","last_became_peered":"2023-05-16T07:03:09.591514+0000","last_unstale":"2023-05-16T07:03:09.951153+0000","last_undegraded":"2023-05-16T07:03:09.951153+0000","last_fullsized":"2023-05-16T07:03:09.951153+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.964289+0000","last_change":"2023-05-16T07:03:09.958059+0000","last_active":"2023-05-16T07:03:09.964289+0000","last_peered":"2023-05-16T07:03:09.964289+0000","last_clean":"2023-05-16T07:03:09.964289+0000","last_became_active":"2023-05-16T07:03:09.957679+0000","last_became_peered":"2023-05-16T07:03:09.957679+0000","last_unstale":"2023-05-16T07:03:09.964289+0000","last_undegraded":"2023-05-16T07:03:09.964289+0000","last_fullsized":"2023-05-16T07:03:09.964289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.948332+0000","last_change":"2023-05-16T07:03:09.590810+0000","last_active":"2023-05-16T07:03:09.948332+0000","last_peered":"2023-05-16T07:03:09.948332+0000","last_clean":"2023-05-16T07:03:09.948332+0000","last_became_active":"2023-05-16T07:03:09.590528+0000","last_became_peered":"2023-05-16T07:03:09.590528+0000","last_unstale":"2023-05-16T07:03:09.948332+0000","last_undegraded":"2023-05-16T07:03:09.948332+0000","last_fullsized":"2023-05-16T07:03:09.948332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971714+0000","last_change":"2023-05-16T07:03:09.969085+0000","last_active":"2023-05-16T07:03:09.971714+0000","last_peered":"2023-05-16T07:03:09.971714+0000","last_clean":"2023-05-16T07:03:09.971714+0000","last_became_active":"2023-05-16T07:03:09.968798+0000","last_became_peered":"2023-05-16T07:03:09.968798+0000","last_unstale":"2023-05-16T07:03:09.971714+0000","last_undegraded":"2023-05-16T07:03:09.971714+0000","last_fullsized":"2023-05-16T07:03:09.971714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955456+0000","last_change":"2023-05-16T07:03:09.599302+0000","last_active":"2023-05-16T07:03:09.955456+0000","last_peered":"2023-05-16T07:03:09.955456+0000","last_clean":"2023-05-16T07:03:09.955456+0000","last_became_active":"2023-05-16T07:03:09.599133+0000","last_became_peered":"2023-05-16T07:03:09.599133+0000","last_unstale":"2023-05-16T07:03:09.955456+0000","last_undegraded":"2023-05-16T07:03:09.955456+0000","last_fullsized":"2023-05-16T07:03:09.955456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969474+0000","last_change":"2023-05-16T07:03:09.965779+0000","last_active":"2023-05-16T07:03:09.969474+0000","last_peered":"2023-05-16T07:03:09.969474+0000","last_clean":"2023-05-16T07:03:09.969474+0000","last_became_active":"2023-05-16T07:03:09.965499+0000","last_became_peered":"2023-05-16T07:03:09.965499+0000","last_unstale":"2023-05-16T07:03:09.969474+0000","last_undegraded":"2023-05-16T07:03:09.969474+0000","last_fullsized":"2023-05-16T07:03:09.969474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957660+0000","last_change":"2023-05-16T07:03:09.589985+0000","last_active":"2023-05-16T07:03:09.957660+0000","last_peered":"2023-05-16T07:03:09.957660+0000","last_clean":"2023-05-16T07:03:09.957660+0000","last_became_active":"2023-05-16T07:03:09.589723+0000","last_became_peered":"2023-05-16T07:03:09.589723+0000","last_unstale":"2023-05-16T07:03:09.957660+0000","last_undegraded":"2023-05-16T07:03:09.957660+0000","last_fullsized":"2023-05-16T07:03:09.957660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966409+0000","last_change":"2023-05-16T07:03:09.959116+0000","last_active":"2023-05-16T07:03:09.966409+0000","last_peered":"2023-05-16T07:03:09.966409+0000","last_clean":"2023-05-16T07:03:09.966409+0000","last_became_active":"2023-05-16T07:03:09.958827+0000","last_became_peered":"2023-05-16T07:03:09.958827+0000","last_unstale":"2023-05-16T07:03:09.966409+0000","last_undegraded":"2023-05-16T07:03:09.966409+0000","last_fullsized":"2023-05-16T07:03:09.966409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-05-16T07:03:10.950904+0000","last_change":"2023-05-16T07:03:09.592795+0000","last_active":"2023-05-16T07:03:10.950904+0000","last_peered":"2023-05-16T07:03:10.950904+0000","last_clean":"2023-05-16T07:03:10.950904+0000","last_became_active":"2023-05-16T07:03:09.592523+0000","last_became_peered":"2023-05-16T07:03:09.592523+0000","last_unstale":"2023-05-16T07:03:10.950904+0000","last_undegraded":"2023-05-16T07:03:10.950904+0000","last_fullsized":"2023-05-16T07:03:10.950904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971455+0000","last_change":"2023-05-16T07:03:09.968547+0000","last_active":"2023-05-16T07:03:09.971455+0000","last_peered":"2023-05-16T07:03:09.971455+0000","last_clean":"2023-05-16T07:03:09.971455+0000","last_became_active":"2023-05-16T07:03:09.968229+0000","last_became_peered":"2023-05-16T07:03:09.968229+0000","last_unstale":"2023-05-16T07:03:09.971455+0000","last_undegraded":"2023-05-16T07:03:09.971455+0000","last_fullsized":"2023-05-16T07:03:09.971455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961282+0000","last_change":"2023-05-16T07:03:09.597726+0000","last_active":"2023-05-16T07:03:09.961282+0000","last_peered":"2023-05-16T07:03:09.961282+0000","last_clean":"2023-05-16T07:03:09.961282+0000","last_became_active":"2023-05-16T07:03:09.591726+0000","last_became_peered":"2023-05-16T07:03:09.591726+0000","last_unstale":"2023-05-16T07:03:09.961282+0000","last_undegraded":"2023-05-16T07:03:09.961282+0000","last_fullsized":"2023-05-16T07:03:09.961282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969369+0000","last_change":"2023-05-16T07:03:09.964984+0000","last_active":"2023-05-16T07:03:09.969369+0000","last_peered":"2023-05-16T07:03:09.969369+0000","last_clean":"2023-05-16T07:03:09.969369+0000","last_became_active":"2023-05-16T07:03:09.964670+0000","last_became_peered":"2023-05-16T07:03:09.964670+0000","last_unstale":"2023-05-16T07:03:09.969369+0000","last_undegraded":"2023-05-16T07:03:09.969369+0000","last_fullsized":"2023-05-16T07:03:09.969369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956256+0000","last_change":"2023-05-16T07:03:09.601508+0000","last_active":"2023-05-16T07:03:09.956256+0000","last_peered":"2023-05-16T07:03:09.956256+0000","last_clean":"2023-05-16T07:03:09.956256+0000","last_became_active":"2023-05-16T07:03:09.601173+0000","last_became_peered":"2023-05-16T07:03:09.601173+0000","last_unstale":"2023-05-16T07:03:09.956256+0000","last_undegraded":"2023-05-16T07:03:09.956256+0000","last_fullsized":"2023-05-16T07:03:09.956256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970646+0000","last_change":"2023-05-16T07:03:09.964012+0000","last_active":"2023-05-16T07:03:09.970646+0000","last_peered":"2023-05-16T07:03:09.970646+0000","last_clean":"2023-05-16T07:03:09.970646+0000","last_became_active":"2023-05-16T07:03:09.961689+0000","last_became_peered":"2023-05-16T07:03:09.961689+0000","last_unstale":"2023-05-16T07:03:09.970646+0000","last_undegraded":"2023-05-16T07:03:09.970646+0000","last_fullsized":"2023-05-16T07:03:09.970646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951351+0000","last_change":"2023-05-16T07:03:09.582462+0000","last_active":"2023-05-16T07:03:09.951351+0000","last_peered":"2023-05-16T07:03:09.951351+0000","last_clean":"2023-05-16T07:03:09.951351+0000","last_became_active":"2023-05-16T07:03:09.582181+0000","last_became_peered":"2023-05-16T07:03:09.582181+0000","last_unstale":"2023-05-16T07:03:09.951351+0000","last_undegraded":"2023-05-16T07:03:09.951351+0000","last_fullsized":"2023-05-16T07:03:09.951351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975633+0000","last_change":"2023-05-16T07:03:09.970595+0000","last_active":"2023-05-16T07:03:09.975633+0000","last_peered":"2023-05-16T07:03:09.975633+0000","last_clean":"2023-05-16T07:03:09.975633+0000","last_became_active":"2023-05-16T07:03:09.970312+0000","last_became_peered":"2023-05-16T07:03:09.970312+0000","last_unstale":"2023-05-16T07:03:09.975633+0000","last_undegraded":"2023-05-16T07:03:09.975633+0000","last_fullsized":"2023-05-16T07:03:09.975633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951825+0000","last_change":"2023-05-16T07:03:09.595242+0000","last_active":"2023-05-16T07:03:09.951825+0000","last_peered":"2023-05-16T07:03:09.951825+0000","last_clean":"2023-05-16T07:03:09.951825+0000","last_became_active":"2023-05-16T07:03:09.594627+0000","last_became_peered":"2023-05-16T07:03:09.594627+0000","last_unstale":"2023-05-16T07:03:09.951825+0000","last_undegraded":"2023-05-16T07:03:09.951825+0000","last_fullsized":"2023-05-16T07:03:09.951825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966387+0000","last_change":"2023-05-16T07:03:09.960915+0000","last_active":"2023-05-16T07:03:09.966387+0000","last_peered":"2023-05-16T07:03:09.966387+0000","last_clean":"2023-05-16T07:03:09.966387+0000","last_became_active":"2023-05-16T07:03:09.960652+0000","last_became_peered":"2023-05-16T07:03:09.960652+0000","last_unstale":"2023-05-16T07:03:09.966387+0000","last_undegraded":"2023-05-16T07:03:09.966387+0000","last_fullsized":"2023-05-16T07:03:09.966387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.948848+0000","last_change":"2023-05-16T07:03:09.584558+0000","last_active":"2023-05-16T07:03:09.948848+0000","last_peered":"2023-05-16T07:03:09.948848+0000","last_clean":"2023-05-16T07:03:09.948848+0000","last_became_active":"2023-05-16T07:03:09.584289+0000","last_became_peered":"2023-05-16T07:03:09.584289+0000","last_unstale":"2023-05-16T07:03:09.948848+0000","last_undegraded":"2023-05-16T07:03:09.948848+0000","last_fullsized":"2023-05-16T07:03:09.948848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966741+0000","last_change":"2023-05-16T07:03:09.961688+0000","last_active":"2023-05-16T07:03:09.966741+0000","last_peered":"2023-05-16T07:03:09.966741+0000","last_clean":"2023-05-16T07:03:09.966741+0000","last_became_active":"2023-05-16T07:03:09.961410+0000","last_became_peered":"2023-05-16T07:03:09.961410+0000","last_unstale":"2023-05-16T07:03:09.966741+0000","last_undegraded":"2023-05-16T07:03:09.966741+0000","last_fullsized":"2023-05-16T07:03:09.966741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961056+0000","last_change":"2023-05-16T07:03:09.597538+0000","last_active":"2023-05-16T07:03:09.961056+0000","last_peered":"2023-05-16T07:03:09.961056+0000","last_clean":"2023-05-16T07:03:09.961056+0000","last_became_active":"2023-05-16T07:03:09.597231+0000","last_became_peered":"2023-05-16T07:03:09.597231+0000","last_unstale":"2023-05-16T07:03:09.961056+0000","last_undegraded":"2023-05-16T07:03:09.961056+0000","last_fullsized":"2023-05-16T07:03:09.961056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975419+0000","last_change":"2023-05-16T07:03:09.962486+0000","last_active":"2023-05-16T07:03:09.975419+0000","last_peered":"2023-05-16T07:03:09.975419+0000","last_clean":"2023-05-16T07:03:09.975419+0000","last_became_active":"2023-05-16T07:03:09.961951+0000","last_became_peered":"2023-05-16T07:03:09.961951+0000","last_unstale":"2023-05-16T07:03:09.975419+0000","last_undegraded":"2023-05-16T07:03:09.975419+0000","last_fullsized":"2023-05-16T07:03:09.975419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969725+0000","last_change":"2023-05-16T07:03:09.591750+0000","last_active":"2023-05-16T07:03:09.969725+0000","last_peered":"2023-05-16T07:03:09.969725+0000","last_clean":"2023-05-16T07:03:09.969725+0000","last_became_active":"2023-05-16T07:03:09.591464+0000","last_became_peered":"2023-05-16T07:03:09.591464+0000","last_unstale":"2023-05-16T07:03:09.969725+0000","last_undegraded":"2023-05-16T07:03:09.969725+0000","last_fullsized":"2023-05-16T07:03:09.969725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971493+0000","last_change":"2023-05-16T07:03:09.969463+0000","last_active":"2023-05-16T07:03:09.971493+0000","last_peered":"2023-05-16T07:03:09.971493+0000","last_clean":"2023-05-16T07:03:09.971493+0000","last_became_active":"2023-05-16T07:03:09.968757+0000","last_became_peered":"2023-05-16T07:03:09.968757+0000","last_unstale":"2023-05-16T07:03:09.971493+0000","last_undegraded":"2023-05-16T07:03:09.971493+0000","last_fullsized":"2023-05-16T07:03:09.971493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.972719+0000","last_change":"2023-05-16T07:03:09.591404+0000","last_active":"2023-05-16T07:03:09.972719+0000","last_peered":"2023-05-16T07:03:09.972719+0000","last_clean":"2023-05-16T07:03:09.972719+0000","last_became_active":"2023-05-16T07:03:09.591133+0000","last_became_peered":"2023-05-16T07:03:09.591133+0000","last_unstale":"2023-05-16T07:03:09.972719+0000","last_undegraded":"2023-05-16T07:03:09.972719+0000","last_fullsized":"2023-05-16T07:03:09.972719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.964367+0000","last_change":"2023-05-16T07:03:09.959508+0000","last_active":"2023-05-16T07:03:09.964367+0000","last_peered":"2023-05-16T07:03:09.964367+0000","last_clean":"2023-05-16T07:03:09.964367+0000","last_became_active":"2023-05-16T07:03:09.959203+0000","last_became_peered":"2023-05-16T07:03:09.959203+0000","last_unstale":"2023-05-16T07:03:09.964367+0000","last_undegraded":"2023-05-16T07:03:09.964367+0000","last_fullsized":"2023-05-16T07:03:09.964367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970882+0000","last_change":"2023-05-16T07:03:09.591720+0000","last_active":"2023-05-16T07:03:09.970882+0000","last_peered":"2023-05-16T07:03:09.970882+0000","last_clean":"2023-05-16T07:03:09.970882+0000","last_became_active":"2023-05-16T07:03:09.591344+0000","last_became_peered":"2023-05-16T07:03:09.591344+0000","last_unstale":"2023-05-16T07:03:09.970882+0000","last_undegraded":"2023-05-16T07:03:09.970882+0000","last_fullsized":"2023-05-16T07:03:09.970882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970694+0000","last_change":"2023-05-16T07:03:09.968386+0000","last_active":"2023-05-16T07:03:09.970694+0000","last_peered":"2023-05-16T07:03:09.970694+0000","last_clean":"2023-05-16T07:03:09.970694+0000","last_became_active":"2023-05-16T07:03:09.968165+0000","last_became_peered":"2023-05-16T07:03:09.968165+0000","last_unstale":"2023-05-16T07:03:09.970694+0000","last_undegraded":"2023-05-16T07:03:09.970694+0000","last_fullsized":"2023-05-16T07:03:09.970694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951625+0000","last_change":"2023-05-16T07:03:09.592619+0000","last_active":"2023-05-16T07:03:09.951625+0000","last_peered":"2023-05-16T07:03:09.951625+0000","last_clean":"2023-05-16T07:03:09.951625+0000","last_became_active":"2023-05-16T07:03:09.592287+0000","last_became_peered":"2023-05-16T07:03:09.592287+0000","last_unstale":"2023-05-16T07:03:09.951625+0000","last_undegraded":"2023-05-16T07:03:09.951625+0000","last_fullsized":"2023-05-16T07:03:09.951625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.965934+0000","last_change":"2023-05-16T07:03:09.961110+0000","last_active":"2023-05-16T07:03:09.965934+0000","last_peered":"2023-05-16T07:03:09.965934+0000","last_clean":"2023-05-16T07:03:09.965934+0000","last_became_active":"2023-05-16T07:03:09.960864+0000","last_became_peered":"2023-05-16T07:03:09.960864+0000","last_unstale":"2023-05-16T07:03:09.965934+0000","last_undegraded":"2023-05-16T07:03:09.965934+0000","last_fullsized":"2023-05-16T07:03:09.965934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955277+0000","last_change":"2023-05-16T07:03:09.599340+0000","last_active":"2023-05-16T07:03:09.955277+0000","last_peered":"2023-05-16T07:03:09.955277+0000","last_clean":"2023-05-16T07:03:09.955277+0000","last_became_active":"2023-05-16T07:03:09.599154+0000","last_became_peered":"2023-05-16T07:03:09.599154+0000","last_unstale":"2023-05-16T07:03:09.955277+0000","last_undegraded":"2023-05-16T07:03:09.955277+0000","last_fullsized":"2023-05-16T07:03:09.955277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971833+0000","last_change":"2023-05-16T07:03:09.970793+0000","last_active":"2023-05-16T07:03:09.971833+0000","last_peered":"2023-05-16T07:03:09.971833+0000","last_clean":"2023-05-16T07:03:09.971833+0000","last_became_active":"2023-05-16T07:03:09.970478+0000","last_became_peered":"2023-05-16T07:03:09.970478+0000","last_unstale":"2023-05-16T07:03:09.971833+0000","last_undegraded":"2023-05-16T07:03:09.971833+0000","last_fullsized":"2023-05-16T07:03:09.971833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951762+0000","last_change":"2023-05-16T07:03:09.595411+0000","last_active":"2023-05-16T07:03:09.951762+0000","last_peered":"2023-05-16T07:03:09.951762+0000","last_clean":"2023-05-16T07:03:09.951762+0000","last_became_active":"2023-05-16T07:03:09.594984+0000","last_became_peered":"2023-05-16T07:03:09.594984+0000","last_unstale":"2023-05-16T07:03:09.951762+0000","last_undegraded":"2023-05-16T07:03:09.951762+0000","last_fullsized":"2023-05-16T07:03:09.951762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969360+0000","last_change":"2023-05-16T07:03:09.964525+0000","last_active":"2023-05-16T07:03:09.969360+0000","last_peered":"2023-05-16T07:03:09.969360+0000","last_clean":"2023-05-16T07:03:09.969360+0000","last_became_active":"2023-05-16T07:03:09.964292+0000","last_became_peered":"2023-05-16T07:03:09.964292+0000","last_unstale":"2023-05-16T07:03:09.969360+0000","last_undegraded":"2023-05-16T07:03:09.969360+0000","last_fullsized":"2023-05-16T07:03:09.969360+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955915+0000","last_change":"2023-05-16T07:03:09.599965+0000","last_active":"2023-05-16T07:03:09.955915+0000","last_peered":"2023-05-16T07:03:09.955915+0000","last_clean":"2023-05-16T07:03:09.955915+0000","last_became_active":"2023-05-16T07:03:09.599711+0000","last_became_peered":"2023-05-16T07:03:09.599711+0000","last_unstale":"2023-05-16T07:03:09.955915+0000","last_undegraded":"2023-05-16T07:03:09.955915+0000","last_fullsized":"2023-05-16T07:03:09.955915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.972055+0000","last_change":"2023-05-16T07:03:09.970741+0000","last_active":"2023-05-16T07:03:09.972055+0000","last_peered":"2023-05-16T07:03:09.972055+0000","last_clean":"2023-05-16T07:03:09.972055+0000","last_became_active":"2023-05-16T07:03:09.970463+0000","last_became_peered":"2023-05-16T07:03:09.970463+0000","last_unstale":"2023-05-16T07:03:09.972055+0000","last_undegraded":"2023-05-16T07:03:09.972055+0000","last_fullsized":"2023-05-16T07:03:09.972055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951248+0000","last_change":"2023-05-16T07:03:09.595643+0000","last_active":"2023-05-16T07:03:09.951248+0000","last_peered":"2023-05-16T07:03:09.951248+0000","last_clean":"2023-05-16T07:03:09.951248+0000","last_became_active":"2023-05-16T07:03:09.594157+0000","last_became_peered":"2023-05-16T07:03:09.594157+0000","last_unstale":"2023-05-16T07:03:09.951248+0000","last_undegraded":"2023-05-16T07:03:09.951248+0000","last_fullsized":"2023-05-16T07:03:09.951248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.966045+0000","last_change":"2023-05-16T07:03:09.964800+0000","last_active":"2023-05-16T07:03:09.966045+0000","last_peered":"2023-05-16T07:03:09.966045+0000","last_clean":"2023-05-16T07:03:09.966045+0000","last_became_active":"2023-05-16T07:03:09.964537+0000","last_became_peered":"2023-05-16T07:03:09.964537+0000","last_unstale":"2023-05-16T07:03:09.966045+0000","last_undegraded":"2023-05-16T07:03:09.966045+0000","last_fullsized":"2023-05-16T07:03:09.966045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955985+0000","last_change":"2023-05-16T07:03:09.589805+0000","last_active":"2023-05-16T07:03:09.955985+0000","last_peered":"2023-05-16T07:03:09.955985+0000","last_clean":"2023-05-16T07:03:09.955985+0000","last_became_active":"2023-05-16T07:03:09.589562+0000","last_became_peered":"2023-05-16T07:03:09.589562+0000","last_unstale":"2023-05-16T07:03:09.955985+0000","last_undegraded":"2023-05-16T07:03:09.955985+0000","last_fullsized":"2023-05-16T07:03:09.955985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975682+0000","last_change":"2023-05-16T07:03:09.966723+0000","last_active":"2023-05-16T07:03:09.975682+0000","last_peered":"2023-05-16T07:03:09.975682+0000","last_clean":"2023-05-16T07:03:09.975682+0000","last_became_active":"2023-05-16T07:03:09.962597+0000","last_became_peered":"2023-05-16T07:03:09.962597+0000","last_unstale":"2023-05-16T07:03:09.975682+0000","last_undegraded":"2023-05-16T07:03:09.975682+0000","last_fullsized":"2023-05-16T07:03:09.975682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955844+0000","last_change":"2023-05-16T07:03:09.600606+0000","last_active":"2023-05-16T07:03:09.955844+0000","last_peered":"2023-05-16T07:03:09.955844+0000","last_clean":"2023-05-16T07:03:09.955844+0000","last_became_active":"2023-05-16T07:03:09.600320+0000","last_became_peered":"2023-05-16T07:03:09.600320+0000","last_unstale":"2023-05-16T07:03:09.955844+0000","last_undegraded":"2023-05-16T07:03:09.955844+0000","last_fullsized":"2023-05-16T07:03:09.955844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.977920+0000","last_change":"2023-05-16T07:03:09.972578+0000","last_active":"2023-05-16T07:03:09.977920+0000","last_peered":"2023-05-16T07:03:09.977920+0000","last_clean":"2023-05-16T07:03:09.977920+0000","last_became_active":"2023-05-16T07:03:09.972277+0000","last_became_peered":"2023-05-16T07:03:09.972277+0000","last_unstale":"2023-05-16T07:03:09.977920+0000","last_undegraded":"2023-05-16T07:03:09.977920+0000","last_fullsized":"2023-05-16T07:03:09.977920+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.972697+0000","last_change":"2023-05-16T07:03:09.600517+0000","last_active":"2023-05-16T07:03:09.972697+0000","last_peered":"2023-05-16T07:03:09.972697+0000","last_clean":"2023-05-16T07:03:09.972697+0000","last_became_active":"2023-05-16T07:03:09.600188+0000","last_became_peered":"2023-05-16T07:03:09.600188+0000","last_unstale":"2023-05-16T07:03:09.972697+0000","last_undegraded":"2023-05-16T07:03:09.972697+0000","last_fullsized":"2023-05-16T07:03:09.972697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968071+0000","last_change":"2023-05-16T07:03:09.960426+0000","last_active":"2023-05-16T07:03:09.968071+0000","last_peered":"2023-05-16T07:03:09.968071+0000","last_clean":"2023-05-16T07:03:09.968071+0000","last_became_active":"2023-05-16T07:03:09.960086+0000","last_became_peered":"2023-05-16T07:03:09.960086+0000","last_unstale":"2023-05-16T07:03:09.968071+0000","last_undegraded":"2023-05-16T07:03:09.968071+0000","last_fullsized":"2023-05-16T07:03:09.968071+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.975811+0000","last_change":"2023-05-16T07:03:09.592715+0000","last_active":"2023-05-16T07:03:09.975811+0000","last_peered":"2023-05-16T07:03:09.975811+0000","last_clean":"2023-05-16T07:03:09.975811+0000","last_became_active":"2023-05-16T07:03:09.592445+0000","last_became_peered":"2023-05-16T07:03:09.592445+0000","last_unstale":"2023-05-16T07:03:09.975811+0000","last_undegraded":"2023-05-16T07:03:09.975811+0000","last_fullsized":"2023-05-16T07:03:09.975811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963199+0000","last_change":"2023-05-16T07:03:09.958074+0000","last_active":"2023-05-16T07:03:09.963199+0000","last_peered":"2023-05-16T07:03:09.963199+0000","last_clean":"2023-05-16T07:03:09.963199+0000","last_became_active":"2023-05-16T07:03:09.957885+0000","last_became_peered":"2023-05-16T07:03:09.957885+0000","last_unstale":"2023-05-16T07:03:09.963199+0000","last_undegraded":"2023-05-16T07:03:09.963199+0000","last_fullsized":"2023-05-16T07:03:09.963199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963495+0000","last_change":"2023-05-16T07:03:09.581278+0000","last_active":"2023-05-16T07:03:09.963495+0000","last_peered":"2023-05-16T07:03:09.963495+0000","last_clean":"2023-05-16T07:03:09.963495+0000","last_became_active":"2023-05-16T07:03:09.580950+0000","last_became_peered":"2023-05-16T07:03:09.580950+0000","last_unstale":"2023-05-16T07:03:09.963495+0000","last_undegraded":"2023-05-16T07:03:09.963495+0000","last_fullsized":"2023-05-16T07:03:09.963495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963155+0000","last_change":"2023-05-16T07:03:09.960677+0000","last_active":"2023-05-16T07:03:09.963155+0000","last_peered":"2023-05-16T07:03:09.963155+0000","last_clean":"2023-05-16T07:03:09.963155+0000","last_became_active":"2023-05-16T07:03:09.960434+0000","last_became_peered":"2023-05-16T07:03:09.960434+0000","last_unstale":"2023-05-16T07:03:09.963155+0000","last_undegraded":"2023-05-16T07:03:09.963155+0000","last_fullsized":"2023-05-16T07:03:09.963155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955745+0000","last_change":"2023-05-16T07:03:09.602043+0000","last_active":"2023-05-16T07:03:09.955745+0000","last_peered":"2023-05-16T07:03:09.955745+0000","last_clean":"2023-05-16T07:03:09.955745+0000","last_became_active":"2023-05-16T07:03:09.601724+0000","last_became_peered":"2023-05-16T07:03:09.601724+0000","last_unstale":"2023-05-16T07:03:09.955745+0000","last_undegraded":"2023-05-16T07:03:09.955745+0000","last_fullsized":"2023-05-16T07:03:09.955745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.974632+0000","last_change":"2023-05-16T07:03:09.965995+0000","last_active":"2023-05-16T07:03:09.974632+0000","last_peered":"2023-05-16T07:03:09.974632+0000","last_clean":"2023-05-16T07:03:09.974632+0000","last_became_active":"2023-05-16T07:03:09.962388+0000","last_became_peered":"2023-05-16T07:03:09.962388+0000","last_unstale":"2023-05-16T07:03:09.974632+0000","last_undegraded":"2023-05-16T07:03:09.974632+0000","last_fullsized":"2023-05-16T07:03:09.974632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.948748+0000","last_change":"2023-05-16T07:03:09.591622+0000","last_active":"2023-05-16T07:03:09.948748+0000","last_peered":"2023-05-16T07:03:09.948748+0000","last_clean":"2023-05-16T07:03:09.948748+0000","last_became_active":"2023-05-16T07:03:09.591352+0000","last_became_peered":"2023-05-16T07:03:09.591352+0000","last_unstale":"2023-05-16T07:03:09.948748+0000","last_undegraded":"2023-05-16T07:03:09.948748+0000","last_fullsized":"2023-05-16T07:03:09.948748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968676+0000","last_change":"2023-05-16T07:03:09.962406+0000","last_active":"2023-05-16T07:03:09.968676+0000","last_peered":"2023-05-16T07:03:09.968676+0000","last_clean":"2023-05-16T07:03:09.968676+0000","last_became_active":"2023-05-16T07:03:09.962107+0000","last_became_peered":"2023-05-16T07:03:09.962107+0000","last_unstale":"2023-05-16T07:03:09.968676+0000","last_undegraded":"2023-05-16T07:03:09.968676+0000","last_fullsized":"2023-05-16T07:03:09.968676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.976081+0000","last_change":"2023-05-16T07:03:09.598390+0000","last_active":"2023-05-16T07:03:09.976081+0000","last_peered":"2023-05-16T07:03:09.976081+0000","last_clean":"2023-05-16T07:03:09.976081+0000","last_became_active":"2023-05-16T07:03:09.598096+0000","last_became_peered":"2023-05-16T07:03:09.598096+0000","last_unstale":"2023-05-16T07:03:09.976081+0000","last_undegraded":"2023-05-16T07:03:09.976081+0000","last_fullsized":"2023-05-16T07:03:09.976081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967878+0000","last_change":"2023-05-16T07:03:09.961168+0000","last_active":"2023-05-16T07:03:09.967878+0000","last_peered":"2023-05-16T07:03:09.967878+0000","last_clean":"2023-05-16T07:03:09.967878+0000","last_became_active":"2023-05-16T07:03:09.960908+0000","last_became_peered":"2023-05-16T07:03:09.960908+0000","last_unstale":"2023-05-16T07:03:09.967878+0000","last_undegraded":"2023-05-16T07:03:09.967878+0000","last_fullsized":"2023-05-16T07:03:09.967878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968287+0000","last_change":"2023-05-16T07:03:09.603171+0000","last_active":"2023-05-16T07:03:09.968287+0000","last_peered":"2023-05-16T07:03:09.968287+0000","last_clean":"2023-05-16T07:03:09.968287+0000","last_became_active":"2023-05-16T07:03:09.602364+0000","last_became_peered":"2023-05-16T07:03:09.602364+0000","last_unstale":"2023-05-16T07:03:09.968287+0000","last_undegraded":"2023-05-16T07:03:09.968287+0000","last_fullsized":"2023-05-16T07:03:09.968287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.973038+0000","last_change":"2023-05-16T07:03:09.972272+0000","last_active":"2023-05-16T07:03:09.973038+0000","last_peered":"2023-05-16T07:03:09.973038+0000","last_clean":"2023-05-16T07:03:09.973038+0000","last_became_active":"2023-05-16T07:03:09.971962+0000","last_became_peered":"2023-05-16T07:03:09.971962+0000","last_unstale":"2023-05-16T07:03:09.973038+0000","last_undegraded":"2023-05-16T07:03:09.973038+0000","last_fullsized":"2023-05-16T07:03:09.973038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968325+0000","last_change":"2023-05-16T07:03:09.601906+0000","last_active":"2023-05-16T07:03:09.968325+0000","last_peered":"2023-05-16T07:03:09.968325+0000","last_clean":"2023-05-16T07:03:09.968325+0000","last_became_active":"2023-05-16T07:03:09.601599+0000","last_became_peered":"2023-05-16T07:03:09.601599+0000","last_unstale":"2023-05-16T07:03:09.968325+0000","last_undegraded":"2023-05-16T07:03:09.968325+0000","last_fullsized":"2023-05-16T07:03:09.968325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.969515+0000","last_change":"2023-05-16T07:03:09.965431+0000","last_active":"2023-05-16T07:03:09.969515+0000","last_peered":"2023-05-16T07:03:09.969515+0000","last_clean":"2023-05-16T07:03:09.969515+0000","last_became_active":"2023-05-16T07:03:09.965149+0000","last_became_peered":"2023-05-16T07:03:09.965149+0000","last_unstale":"2023-05-16T07:03:09.969515+0000","last_undegraded":"2023-05-16T07:03:09.969515+0000","last_fullsized":"2023-05-16T07:03:09.969515+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951870+0000","last_change":"2023-05-16T07:03:09.590732+0000","last_active":"2023-05-16T07:03:09.951870+0000","last_peered":"2023-05-16T07:03:09.951870+0000","last_clean":"2023-05-16T07:03:09.951870+0000","last_became_active":"2023-05-16T07:03:09.590418+0000","last_became_peered":"2023-05-16T07:03:09.590418+0000","last_unstale":"2023-05-16T07:03:09.951870+0000","last_undegraded":"2023-05-16T07:03:09.951870+0000","last_fullsized":"2023-05-16T07:03:09.951870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.964963+0000","last_change":"2023-05-16T07:03:09.960581+0000","last_active":"2023-05-16T07:03:09.964963+0000","last_peered":"2023-05-16T07:03:09.964963+0000","last_clean":"2023-05-16T07:03:09.964963+0000","last_became_active":"2023-05-16T07:03:09.960185+0000","last_became_peered":"2023-05-16T07:03:09.960185+0000","last_unstale":"2023-05-16T07:03:09.964963+0000","last_undegraded":"2023-05-16T07:03:09.964963+0000","last_fullsized":"2023-05-16T07:03:09.964963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956116+0000","last_change":"2023-05-16T07:03:09.598598+0000","last_active":"2023-05-16T07:03:09.956116+0000","last_peered":"2023-05-16T07:03:09.956116+0000","last_clean":"2023-05-16T07:03:09.956116+0000","last_became_active":"2023-05-16T07:03:09.598382+0000","last_became_peered":"2023-05-16T07:03:09.598382+0000","last_unstale":"2023-05-16T07:03:09.956116+0000","last_undegraded":"2023-05-16T07:03:09.956116+0000","last_fullsized":"2023-05-16T07:03:09.956116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.976230+0000","last_change":"2023-05-16T07:03:09.975110+0000","last_active":"2023-05-16T07:03:09.976230+0000","last_peered":"2023-05-16T07:03:09.976230+0000","last_clean":"2023-05-16T07:03:09.976230+0000","last_became_active":"2023-05-16T07:03:09.974821+0000","last_became_peered":"2023-05-16T07:03:09.974821+0000","last_unstale":"2023-05-16T07:03:09.976230+0000","last_undegraded":"2023-05-16T07:03:09.976230+0000","last_fullsized":"2023-05-16T07:03:09.976230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961030+0000","last_change":"2023-05-16T07:03:09.597147+0000","last_active":"2023-05-16T07:03:09.961030+0000","last_peered":"2023-05-16T07:03:09.961030+0000","last_clean":"2023-05-16T07:03:09.961030+0000","last_became_active":"2023-05-16T07:03:09.596904+0000","last_became_peered":"2023-05-16T07:03:09.596904+0000","last_unstale":"2023-05-16T07:03:09.961030+0000","last_undegraded":"2023-05-16T07:03:09.961030+0000","last_fullsized":"2023-05-16T07:03:09.961030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.980602+0000","last_change":"2023-05-16T07:03:09.976290+0000","last_active":"2023-05-16T07:03:09.980602+0000","last_peered":"2023-05-16T07:03:09.980602+0000","last_clean":"2023-05-16T07:03:09.980602+0000","last_became_active":"2023-05-16T07:03:09.975963+0000","last_became_peered":"2023-05-16T07:03:09.975963+0000","last_unstale":"2023-05-16T07:03:09.980602+0000","last_undegraded":"2023-05-16T07:03:09.980602+0000","last_fullsized":"2023-05-16T07:03:09.980602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.950992+0000","last_change":"2023-05-16T07:03:09.591772+0000","last_active":"2023-05-16T07:03:09.950992+0000","last_peered":"2023-05-16T07:03:09.950992+0000","last_clean":"2023-05-16T07:03:09.950992+0000","last_became_active":"2023-05-16T07:03:09.591477+0000","last_became_peered":"2023-05-16T07:03:09.591477+0000","last_unstale":"2023-05-16T07:03:09.950992+0000","last_undegraded":"2023-05-16T07:03:09.950992+0000","last_fullsized":"2023-05-16T07:03:09.950992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967870+0000","last_change":"2023-05-16T07:03:09.963643+0000","last_active":"2023-05-16T07:03:09.967870+0000","last_peered":"2023-05-16T07:03:09.967870+0000","last_clean":"2023-05-16T07:03:09.967870+0000","last_became_active":"2023-05-16T07:03:09.963292+0000","last_became_peered":"2023-05-16T07:03:09.963292+0000","last_unstale":"2023-05-16T07:03:09.967870+0000","last_undegraded":"2023-05-16T07:03:09.967870+0000","last_fullsized":"2023-05-16T07:03:09.967870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967337+0000","last_change":"2023-05-16T07:03:09.594244+0000","last_active":"2023-05-16T07:03:09.967337+0000","last_peered":"2023-05-16T07:03:09.967337+0000","last_clean":"2023-05-16T07:03:09.967337+0000","last_became_active":"2023-05-16T07:03:09.593800+0000","last_became_peered":"2023-05-16T07:03:09.593800+0000","last_unstale":"2023-05-16T07:03:09.967337+0000","last_undegraded":"2023-05-16T07:03:09.967337+0000","last_fullsized":"2023-05-16T07:03:09.967337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.962297+0000","last_change":"2023-05-16T07:03:09.958211+0000","last_active":"2023-05-16T07:03:09.962297+0000","last_peered":"2023-05-16T07:03:09.962297+0000","last_clean":"2023-05-16T07:03:09.962297+0000","last_became_active":"2023-05-16T07:03:09.957915+0000","last_became_peered":"2023-05-16T07:03:09.957915+0000","last_unstale":"2023-05-16T07:03:09.962297+0000","last_undegraded":"2023-05-16T07:03:09.962297+0000","last_fullsized":"2023-05-16T07:03:09.962297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952547+0000","last_change":"2023-05-16T07:03:09.594868+0000","last_active":"2023-05-16T07:03:09.952547+0000","last_peered":"2023-05-16T07:03:09.952547+0000","last_clean":"2023-05-16T07:03:09.952547+0000","last_became_active":"2023-05-16T07:03:09.594600+0000","last_became_peered":"2023-05-16T07:03:09.594600+0000","last_unstale":"2023-05-16T07:03:09.952547+0000","last_undegraded":"2023-05-16T07:03:09.952547+0000","last_fullsized":"2023-05-16T07:03:09.952547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.963215+0000","last_change":"2023-05-16T07:03:09.959075+0000","last_active":"2023-05-16T07:03:09.963215+0000","last_peered":"2023-05-16T07:03:09.963215+0000","last_clean":"2023-05-16T07:03:09.963215+0000","last_became_active":"2023-05-16T07:03:09.958819+0000","last_became_peered":"2023-05-16T07:03:09.958819+0000","last_unstale":"2023-05-16T07:03:09.963215+0000","last_undegraded":"2023-05-16T07:03:09.963215+0000","last_fullsized":"2023-05-16T07:03:09.963215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.947467+0000","last_change":"2023-05-16T07:03:09.594731+0000","last_active":"2023-05-16T07:03:09.947467+0000","last_peered":"2023-05-16T07:03:09.947467+0000","last_clean":"2023-05-16T07:03:09.947467+0000","last_became_active":"2023-05-16T07:03:09.592826+0000","last_became_peered":"2023-05-16T07:03:09.592826+0000","last_unstale":"2023-05-16T07:03:09.947467+0000","last_undegraded":"2023-05-16T07:03:09.947467+0000","last_fullsized":"2023-05-16T07:03:09.947467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.956188+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:03:09.956188+0000","last_peered":"2023-05-16T07:03:09.956188+0000","last_clean":"2023-05-16T07:03:09.956188+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:03:09.956188+0000","last_undegraded":"2023-05-16T07:03:09.956188+0000","last_fullsized":"2023-05-16T07:03:09.956188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968905+0000","last_change":"2023-05-16T07:03:09.964901+0000","last_active":"2023-05-16T07:03:09.968905+0000","last_peered":"2023-05-16T07:03:09.968905+0000","last_clean":"2023-05-16T07:03:09.968905+0000","last_became_active":"2023-05-16T07:03:09.964643+0000","last_became_peered":"2023-05-16T07:03:09.964643+0000","last_unstale":"2023-05-16T07:03:09.968905+0000","last_undegraded":"2023-05-16T07:03:09.968905+0000","last_fullsized":"2023-05-16T07:03:09.968905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.947593+0000","last_change":"2023-05-16T07:03:09.588443+0000","last_active":"2023-05-16T07:03:09.947593+0000","last_peered":"2023-05-16T07:03:09.947593+0000","last_clean":"2023-05-16T07:03:09.947593+0000","last_became_active":"2023-05-16T07:03:09.588195+0000","last_became_peered":"2023-05-16T07:03:09.588195+0000","last_unstale":"2023-05-16T07:03:09.947593+0000","last_undegraded":"2023-05-16T07:03:09.947593+0000","last_fullsized":"2023-05-16T07:03:09.947593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955150+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:03:09.955150+0000","last_peered":"2023-05-16T07:03:09.955150+0000","last_clean":"2023-05-16T07:03:09.955150+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:03:09.955150+0000","last_undegraded":"2023-05-16T07:03:09.955150+0000","last_fullsized":"2023-05-16T07:03:09.955150+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.974665+0000","last_change":"2023-05-16T07:03:09.966062+0000","last_active":"2023-05-16T07:03:09.974665+0000","last_peered":"2023-05-16T07:03:09.974665+0000","last_clean":"2023-05-16T07:03:09.974665+0000","last_became_active":"2023-05-16T07:03:09.965722+0000","last_became_peered":"2023-05-16T07:03:09.965722+0000","last_unstale":"2023-05-16T07:03:09.974665+0000","last_undegraded":"2023-05-16T07:03:09.974665+0000","last_fullsized":"2023-05-16T07:03:09.974665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.950819+0000","last_change":"2023-05-16T07:03:09.602909+0000","last_active":"2023-05-16T07:03:09.950819+0000","last_peered":"2023-05-16T07:03:09.950819+0000","last_clean":"2023-05-16T07:03:09.950819+0000","last_became_active":"2023-05-16T07:03:09.602594+0000","last_became_peered":"2023-05-16T07:03:09.602594+0000","last_unstale":"2023-05-16T07:03:09.950819+0000","last_undegraded":"2023-05-16T07:03:09.950819+0000","last_fullsized":"2023-05-16T07:03:09.950819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.950880+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:03:09.950880+0000","last_peered":"2023-05-16T07:03:09.950880+0000","last_clean":"2023-05-16T07:03:09.950880+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:03:09.950880+0000","last_undegraded":"2023-05-16T07:03:09.950880+0000","last_fullsized":"2023-05-16T07:03:09.950880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.967944+0000","last_change":"2023-05-16T07:03:09.963700+0000","last_active":"2023-05-16T07:03:09.967944+0000","last_peered":"2023-05-16T07:03:09.967944+0000","last_clean":"2023-05-16T07:03:09.967944+0000","last_became_active":"2023-05-16T07:03:09.963336+0000","last_became_peered":"2023-05-16T07:03:09.963336+0000","last_unstale":"2023-05-16T07:03:09.967944+0000","last_undegraded":"2023-05-16T07:03:09.967944+0000","last_fullsized":"2023-05-16T07:03:09.967944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952948+0000","last_change":"2023-05-16T07:03:09.593440+0000","last_active":"2023-05-16T07:03:09.952948+0000","last_peered":"2023-05-16T07:03:09.952948+0000","last_clean":"2023-05-16T07:03:09.952948+0000","last_became_active":"2023-05-16T07:03:09.593062+0000","last_became_peered":"2023-05-16T07:03:09.593062+0000","last_unstale":"2023-05-16T07:03:09.952948+0000","last_undegraded":"2023-05-16T07:03:09.952948+0000","last_fullsized":"2023-05-16T07:03:09.952948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.955173+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:03:09.955173+0000","last_peered":"2023-05-16T07:03:09.955173+0000","last_clean":"2023-05-16T07:03:09.955173+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:03:09.955173+0000","last_undegraded":"2023-05-16T07:03:09.955173+0000","last_fullsized":"2023-05-16T07:03:09.955173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961139+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:03:09.961139+0000","last_peered":"2023-05-16T07:03:09.961139+0000","last_clean":"2023-05-16T07:03:09.961139+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:03:09.961139+0000","last_undegraded":"2023-05-16T07:03:09.961139+0000","last_fullsized":"2023-05-16T07:03:09.961139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971163+0000","last_change":"2023-05-16T07:03:09.968710+0000","last_active":"2023-05-16T07:03:09.971163+0000","last_peered":"2023-05-16T07:03:09.971163+0000","last_clean":"2023-05-16T07:03:09.971163+0000","last_became_active":"2023-05-16T07:03:09.968451+0000","last_became_peered":"2023-05-16T07:03:09.968451+0000","last_unstale":"2023-05-16T07:03:09.971163+0000","last_undegraded":"2023-05-16T07:03:09.971163+0000","last_fullsized":"2023-05-16T07:03:09.971163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.971038+0000","last_change":"2023-05-16T07:03:09.601345+0000","last_active":"2023-05-16T07:03:09.971038+0000","last_peered":"2023-05-16T07:03:09.971038+0000","last_clean":"2023-05-16T07:03:09.971038+0000","last_became_active":"2023-05-16T07:03:09.601010+0000","last_became_peered":"2023-05-16T07:03:09.601010+0000","last_unstale":"2023-05-16T07:03:09.971038+0000","last_undegraded":"2023-05-16T07:03:09.971038+0000","last_fullsized":"2023-05-16T07:03:09.971038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.961313+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:03:09.961313+0000","last_peered":"2023-05-16T07:03:09.961313+0000","last_clean":"2023-05-16T07:03:09.961313+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:03:09.961313+0000","last_undegraded":"2023-05-16T07:03:09.961313+0000","last_fullsized":"2023-05-16T07:03:09.961313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.970552+0000","last_change":"2023-05-16T07:03:09.964329+0000","last_active":"2023-05-16T07:03:09.970552+0000","last_peered":"2023-05-16T07:03:09.970552+0000","last_clean":"2023-05-16T07:03:09.970552+0000","last_became_active":"2023-05-16T07:03:09.960866+0000","last_became_peered":"2023-05-16T07:03:09.960866+0000","last_unstale":"2023-05-16T07:03:09.970552+0000","last_undegraded":"2023-05-16T07:03:09.970552+0000","last_fullsized":"2023-05-16T07:03:09.970552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.949300+0000","last_change":"2023-05-16T07:03:09.591525+0000","last_active":"2023-05-16T07:03:09.949300+0000","last_peered":"2023-05-16T07:03:09.949300+0000","last_clean":"2023-05-16T07:03:09.949300+0000","last_became_active":"2023-05-16T07:03:09.591221+0000","last_became_peered":"2023-05-16T07:03:09.591221+0000","last_unstale":"2023-05-16T07:03:09.949300+0000","last_undegraded":"2023-05-16T07:03:09.949300+0000","last_fullsized":"2023-05-16T07:03:09.949300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.952429+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:03:09.952429+0000","last_peered":"2023-05-16T07:03:09.952429+0000","last_clean":"2023-05-16T07:03:09.952429+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:03:09.952429+0000","last_undegraded":"2023-05-16T07:03:09.952429+0000","last_fullsized":"2023-05-16T07:03:09.952429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.968048+0000","last_change":"2023-05-16T07:03:09.960309+0000","last_active":"2023-05-16T07:03:09.968048+0000","last_peered":"2023-05-16T07:03:09.968048+0000","last_clean":"2023-05-16T07:03:09.968048+0000","last_became_active":"2023-05-16T07:03:09.960103+0000","last_became_peered":"2023-05-16T07:03:09.960103+0000","last_unstale":"2023-05-16T07:03:09.968048+0000","last_undegraded":"2023-05-16T07:03:09.968048+0000","last_fullsized":"2023-05-16T07:03:09.968048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.951037+0000","last_change":"2023-05-16T07:03:09.591671+0000","last_active":"2023-05-16T07:03:09.951037+0000","last_peered":"2023-05-16T07:03:09.951037+0000","last_clean":"2023-05-16T07:03:09.951037+0000","last_became_active":"2023-05-16T07:03:09.591378+0000","last_became_peered":"2023-05-16T07:03:09.591378+0000","last_unstale":"2023-05-16T07:03:09.951037+0000","last_undegraded":"2023-05-16T07:03:09.951037+0000","last_fullsized":"2023-05-16T07:03:09.951037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.957557+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:03:09.957557+0000","last_peered":"2023-05-16T07:03:09.957557+0000","last_clean":"2023-05-16T07:03:09.957557+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:03:09.957557+0000","last_undegraded":"2023-05-16T07:03:09.957557+0000","last_fullsized":"2023-05-16T07:03:09.957557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.974675+0000","last_change":"2023-05-16T07:03:09.965992+0000","last_active":"2023-05-16T07:03:09.974675+0000","last_peered":"2023-05-16T07:03:09.974675+0000","last_clean":"2023-05-16T07:03:09.974675+0000","last_became_active":"2023-05-16T07:03:09.965687+0000","last_became_peered":"2023-05-16T07:03:09.965687+0000","last_unstale":"2023-05-16T07:03:09.974675+0000","last_undegraded":"2023-05-16T07:03:09.974675+0000","last_fullsized":"2023-05-16T07:03:09.974675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:09.571960+0000","last_clean_scrub_stamp":"2023-05-16T07:03:09.571960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.954982+0000","last_change":"2023-05-16T07:03:09.590083+0000","last_active":"2023-05-16T07:03:09.954982+0000","last_peered":"2023-05-16T07:03:09.954982+0000","last_clean":"2023-05-16T07:03:09.954982+0000","last_became_active":"2023-05-16T07:03:09.589801+0000","last_became_peered":"2023-05-16T07:03:09.589801+0000","last_unstale":"2023-05-16T07:03:09.954982+0000","last_undegraded":"2023-05-16T07:03:09.954982+0000","last_fullsized":"2023-05-16T07:03:09.954982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:03:08.567973+0000","last_clean_scrub_stamp":"2023-05-16T07:03:08.567973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-05-16T07:03:09.949619+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:03:09.949619+0000","last_peered":"2023-05-16T07:03:09.949619+0000","last_clean":"2023-05-16T07:03:09.949619+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:03:09.949619+0000","last_undegraded":"2023-05-16T07:03:09.949619+0000","last_fullsized":"2023-05-16T07:03:09.949619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5704,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366136,"statfs":{"total":96636764160,"available":96630923264,"internally_reserved":0,"allocated":393216,"data_stored":159344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5636,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366204,"statfs":{"total":96636764160,"available":96630992896,"internally_reserved":0,"allocated":389120,"data_stored":159254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5640,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366200,"statfs":{"total":96636764160,"available":96630988800,"internally_reserved":0,"allocated":393216,"data_stored":158860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5636,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366204,"statfs":{"total":96636764160,"available":96630992896,"internally_reserved":0,"allocated":389120,"data_stored":158734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5504,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94366336,"statfs":{"total":96636764160,"available":96631128064,"internally_reserved":0,"allocated":385024,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366060,"statfs":{"total":96636764160,"available":96630845440,"internally_reserved":0,"allocated":405504,"data_stored":160340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5764,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366076,"statfs":{"total":96636764160,"available":96630861824,"internally_reserved":0,"allocated":389120,"data_stored":158722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5712,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366128,"statfs":{"total":96636764160,"available":96630915072,"internally_reserved":0,"allocated":401408,"data_stored":160258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:03:17.704 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:03:17.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:03:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.849+0000 7fc41a693700 1 Processor -- start 2023-05-16T07:03:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.849+0000 7fc41a693700 1 -- start start 2023-05-16T07:03:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc41a693700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc41414f3b0 0x7fc41414f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc41a693700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc414064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc41a693700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc414152bc0 0x7fc4140a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc41a693700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41405ec20 con 0x7fc414152bc0 2023-05-16T07:03:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc41a693700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140c3170 con 0x7fc41414f3b0 2023-05-16T07:03:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc41a693700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41405e940 con 0x7fc41405e460 2023-05-16T07:03:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc414152bc0 0x7fc4140a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc414152bc0 0x7fc4140a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42260/0 (socket says 172.21.15.92:42260) 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 -- 172.21.15.92:0/116192501 learned_addr learned my addr 172.21.15.92:0/116192501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419691700 1 --2- 172.21.15.92:0/116192501 >> v2:172.21.15.158:3300/0 conn(0x7fc41414f3b0 0x7fc41414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc418e90700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc414064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 -- 172.21.15.92:0/116192501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 msgr2=0x7fc414064150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc414064150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 -- 172.21.15.92:0/116192501 >> v2:172.21.15.158:3300/0 conn(0x7fc41414f3b0 msgr2=0x7fc41414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 --2- 172.21.15.92:0/116192501 >> v2:172.21.15.158:3300/0 conn(0x7fc41414f3b0 0x7fc41414f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 -- 172.21.15.92:0/116192501 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414063670 con 0x7fc414152bc0 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc418e90700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc414064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419691700 1 --2- 172.21.15.92:0/116192501 >> v2:172.21.15.158:3300/0 conn(0x7fc41414f3b0 0x7fc41414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.850+0000 7fc419e92700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414152bc0 0x7fc4140a76f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fc414067c60 tx=0x7fc4100090f0).ready entity=mon.0 client_cookie=50ece3cc1d0c9693 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc40bfff700 1 -- 172.21.15.92:0/116192501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41000b600 con 0x7fc414152bc0 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc40bfff700 1 -- 172.21.15.92:0/116192501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41000b760 con 0x7fc414152bc0 2023-05-16T07:03:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc40bfff700 1 -- 172.21.15.92:0/116192501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41000bae0 con 0x7fc414152bc0 2023-05-16T07:03:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 -- 172.21.15.92:0/116192501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414152bc0 msgr2=0x7fc4140a76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414152bc0 0x7fc4140a76f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fc414067c60 tx=0x7fc4100090f0).stop 2023-05-16T07:03:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 -- 172.21.15.92:0/116192501 shutdown_connections 2023-05-16T07:03:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414152bc0 0x7fc4140a76f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 --2- 172.21.15.92:0/116192501 >> v2:172.21.15.158:3300/0 conn(0x7fc41414f3b0 0x7fc41414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 --2- 172.21.15.92:0/116192501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc414064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 -- 172.21.15.92:0/116192501 >> 172.21.15.92:0/116192501 conn(0x7fc414062d80 msgr2=0x7fc41405dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 -- 172.21.15.92:0/116192501 shutdown_connections 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.851+0000 7fc41a693700 1 -- 172.21.15.92:0/116192501 wait complete. 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.852+0000 7fc41a693700 1 Processor -- start 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.852+0000 7fc41a693700 1 -- start start 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc41a693700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc41414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc41a693700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 0x7fc414155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc41a693700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc414152bc0 0x7fc414156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc41a693700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140ac610 con 0x7fc41414f3b0 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc41a693700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140aba00 con 0x7fc414152bc0 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc41a693700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140abce0 con 0x7fc41405e460 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc418e90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 0x7fc414155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc418e90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 0x7fc414155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42262/0 (socket says 172.21.15.92:42262) 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc418e90700 1 -- 172.21.15.92:0/3004644018 learned_addr learned my addr 172.21.15.92:0/3004644018 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc419e92700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc414152bc0 0x7fc414156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.853+0000 7fc419691700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc41414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc418e90700 1 -- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 msgr2=0x7fc41414d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc418e90700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc41414d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc418e90700 1 -- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc414152bc0 msgr2=0x7fc414156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc418e90700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc414152bc0 0x7fc414156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc418e90700 1 -- 172.21.15.92:0/3004644018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410009370 con 0x7fc41414f3b0 2023-05-16T07:03:17.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc418e90700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 0x7fc414155230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fc40400d860 tx=0x7fc404008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400f520 con 0x7fc41414f3b0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40400f680 con 0x7fc41414f3b0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140abfc0 con 0x7fc41414f3b0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400fa00 con 0x7fc41414f3b0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.854+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc414165360 con 0x7fc41414f3b0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.855+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140ac780 con 0x7fc41414f3b0 2023-05-16T07:03:17.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.855+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc40401a030 con 0x7fc41414f3b0 2023-05-16T07:03:17.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.855+0000 7fc4097fa700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc3f805ed90 0x7fc3f8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:17.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.856+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc404079600 con 0x7fc41414f3b0 2023-05-16T07:03:17.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.856+0000 7fc419691700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc3f805ed90 0x7fc3f8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:17.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.856+0000 7fc419691700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc3f805ed90 0x7fc3f8061250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fc40c009fd0 tx=0x7fc40c009960).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:17.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:17.859+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc40404aba0 con 0x7fc41414f3b0 2023-05-16T07:03:18.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.088+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fc414047150 con 0x7fc41414f3b0 2023-05-16T07:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.094+0000 7fc4097fa700 1 -- 172.21.15.92:0/3004644018 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc40404e1c0 con 0x7fc41414f3b0 2023-05-16T07:03:18.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.095+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc3f805ed90 msgr2=0x7fc3f8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.095+0000 7fc41a693700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc3f805ed90 0x7fc3f8061250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fc40c009fd0 tx=0x7fc40c009960).stop 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.095+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 msgr2=0x7fc414155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.095+0000 7fc41a693700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 0x7fc414155230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fc40400d860 tx=0x7fc404008040).stop 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 shutdown_connections 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc3f805ed90 0x7fc3f8061250 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc41405e460 0x7fc41414d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc414152bc0 0x7fc414156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 --2- 172.21.15.92:0/3004644018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc41414f3b0 0x7fc414155230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 >> 172.21.15.92:0/3004644018 conn(0x7fc414062d80 msgr2=0x7fc4140c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:18.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 shutdown_connections 2023-05-16T07:03:18.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.096+0000 7fc41a693700 1 -- 172.21.15.92:0/3004644018 wait complete. 2023-05-16T07:03:18.101 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:03:18.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:03:18.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.217+0000 7fede33c4700 1 Processor -- start 2023-05-16T07:03:18.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.217+0000 7fede33c4700 1 -- start start 2023-05-16T07:03:18.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede33c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feddc154170 0x7feddc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede33c4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7feddc069db0 0x7feddc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede33c4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc0c0660 con 0x7feddc154550 2023-05-16T07:03:18.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede33c4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0c1b00 con 0x7feddc154170 2023-05-16T07:03:18.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede33c4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0bfd30 con 0x7feddc069db0 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feddc154170 0x7feddc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feddc154170 0x7feddc0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42272/0 (socket says 172.21.15.92:42272) 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 -- 172.21.15.92:0/772139530 learned_addr learned my addr 172.21.15.92:0/772139530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 -- 172.21.15.92:0/772139530 >> v2:172.21.15.158:3300/0 conn(0x7feddc069db0 msgr2=0x7feddc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1961700 1 --2- 172.21.15.92:0/772139530 >> v2:172.21.15.158:3300/0 conn(0x7feddc069db0 0x7feddc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 --2- 172.21.15.92:0/772139530 >> v2:172.21.15.158:3300/0 conn(0x7feddc069db0 0x7feddc0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 -- 172.21.15.92:0/772139530 >> v1:172.21.15.158:6789/0 conn(0x7feddc154550 legacy=0x7feddc0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 -- 172.21.15.92:0/772139530 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0c0010 con 0x7feddc154170 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.218+0000 7fede1160700 1 --2- 172.21.15.92:0/772139530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc154170 0x7feddc0c5c70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7feddc0c5210 tx=0x7fedd4009840).ready entity=mon.0 client_cookie=840a84060c9b8eb8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fedd3fff700 1 -- 172.21.15.92:0/772139530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd400e070 con 0x7feddc154170 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fedd3fff700 1 -- 172.21.15.92:0/772139530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd400ba10 con 0x7feddc154170 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fedd3fff700 1 -- 172.21.15.92:0/772139530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd400bd90 con 0x7feddc154170 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 -- 172.21.15.92:0/772139530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc154170 msgr2=0x7feddc0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 --2- 172.21.15.92:0/772139530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc154170 0x7feddc0c5c70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7feddc0c5210 tx=0x7fedd4009840).stop 2023-05-16T07:03:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 -- 172.21.15.92:0/772139530 shutdown_connections 2023-05-16T07:03:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 --2- 172.21.15.92:0/772139530 >> v2:172.21.15.158:3300/0 conn(0x7feddc069db0 0x7feddc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 --2- 172.21.15.92:0/772139530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc154170 0x7feddc0c5c70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 -- 172.21.15.92:0/772139530 >> 172.21.15.92:0/772139530 conn(0x7feddc0cb700 msgr2=0x7feddc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 -- 172.21.15.92:0/772139530 shutdown_connections 2023-05-16T07:03:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.219+0000 7fede33c4700 1 -- 172.21.15.92:0/772139530 wait complete. 2023-05-16T07:03:18.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.220+0000 7fede33c4700 1 Processor -- start 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.220+0000 7fede33c4700 1 -- start start 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede33c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 0x7feddc168a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede33c4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feddc154170 0x7feddc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede33c4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feddc154550 0x7feddc147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede33c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0c2dc0 con 0x7feddc069db0 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede33c4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc168f50 con 0x7feddc154170 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede33c4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc145940 con 0x7feddc154550 2023-05-16T07:03:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede1160700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 0x7feddc168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede1160700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 0x7feddc168a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42276/0 (socket says 172.21.15.92:42276) 2023-05-16T07:03:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede1160700 1 -- 172.21.15.92:0/2076956825 learned_addr learned my addr 172.21.15.92:0/2076956825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede095f700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feddc154170 0x7feddc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.221+0000 7fede1961700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feddc154550 0x7feddc147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1160700 1 -- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feddc154550 msgr2=0x7feddc147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1160700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feddc154550 0x7feddc147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1160700 1 -- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feddc154170 msgr2=0x7feddc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1160700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feddc154170 0x7feddc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1160700 1 -- 172.21.15.92:0/2076956825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4009ae0 con 0x7feddc069db0 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede095f700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feddc154170 0x7feddc0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1961700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feddc154550 0x7feddc147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede1160700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 0x7feddc168a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fedd4009540 tx=0x7fedd4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40070e0 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4013d10 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc0c3010 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd401e6c0 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.222+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc145fe0 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.223+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc146350 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.223+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fedd4025030 con 0x7feddc069db0 2023-05-16T07:03:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.223+0000 7fedd17fa700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fedc005ed90 0x7fedc0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.223+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fedd407b150 con 0x7feddc069db0 2023-05-16T07:03:18.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.224+0000 7fede095f700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fedc005ed90 0x7fedc0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.224+0000 7fede095f700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fedc005ed90 0x7fedc0061250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fedcc006fd0 tx=0x7fedcc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.228+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fedd404c6f0 con 0x7feddc069db0 2023-05-16T07:03:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.429+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feddc0666c0 con 0x7feddc069db0 2023-05-16T07:03:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.429+0000 7fedd17fa700 1 -- 172.21.15.92:0/2076956825 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4489 (secure 0 0 0) 0x7fedd4023070 con 0x7feddc069db0 2023-05-16T07:03:18.434 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:03:18.434 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:03:09.943261+0000","modified":"2023-05-16T07:03:11.948935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2108552286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2108552286},{"type":"v1","addr":"172.21.15.92:6835","nonce":2108552286}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-05-16T07:03:18.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fedc005ed90 msgr2=0x7fedc0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fedc005ed90 0x7fedc0061250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fedcc006fd0 tx=0x7fedcc008040).stop 2023-05-16T07:03:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 msgr2=0x7feddc168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 0x7feddc168a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fedd4009540 tx=0x7fedd4009840).stop 2023-05-16T07:03:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 shutdown_connections 2023-05-16T07:03:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fedc005ed90 0x7fedc0061250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feddc154550 0x7feddc147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feddc069db0 0x7feddc168a60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 --2- 172.21.15.92:0/2076956825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feddc154170 0x7feddc0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 >> 172.21.15.92:0/2076956825 conn(0x7feddc0cb700 msgr2=0x7feddc0c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 shutdown_connections 2023-05-16T07:03:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.431+0000 7fede33c4700 1 -- 172.21.15.92:0/2076956825 wait complete. 2023-05-16T07:03:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2023-05-16T07:03:18.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:03:09.943261+0000', 'modified': '2023-05-16T07:03:11.948935+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4935}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4935': {'gid': 4935, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/2108552286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6833', 'nonce': 2108552286}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2108552286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:03:18.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:03:18.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-16T07:03:18.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.963+0000 7efc1b53b700 1 Processor -- start 2023-05-16T07:03:18.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.963+0000 7efc1b53b700 1 -- start start 2023-05-16T07:03:18.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1b53b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc14069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1b53b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc14153b00 0x7efc14153f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1b53b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1406a920 con 0x7efc14069750 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1b53b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc140c65f0 con 0x7efc14153b00 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1b53b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140bfa30 con 0x7efc140aab40 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc14153b00 0x7efc14153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc14153b00 0x7efc14153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42288/0 (socket says 172.21.15.92:42288) 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 -- 172.21.15.92:0/949126253 learned_addr learned my addr 172.21.15.92:0/949126253 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 -- 172.21.15.92:0/949126253 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 msgr2=0x7efc14069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc19d38700 1 --2- 172.21.15.92:0/949126253 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc14069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/949126253 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc14069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 -- 172.21.15.92:0/949126253 >> v1:172.21.15.158:6789/0 conn(0x7efc14069750 legacy=0x7efc140c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 -- 172.21.15.92:0/949126253 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140bfc80 con 0x7efc14153b00 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/949126253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14153b00 0x7efc14153f70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7efc0c006fd0 tx=0x7efc0c0090f0).ready entity=mon.0 client_cookie=603cec087e65fd45 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc19537700 1 -- 172.21.15.92:0/949126253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00b600 con 0x7efc14153b00 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc19537700 1 -- 172.21.15.92:0/949126253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c00b760 con 0x7efc14153b00 2023-05-16T07:03:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.964+0000 7efc19537700 1 -- 172.21.15.92:0/949126253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00bae0 con 0x7efc14153b00 2023-05-16T07:03:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 -- 172.21.15.92:0/949126253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14153b00 msgr2=0x7efc14153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 --2- 172.21.15.92:0/949126253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14153b00 0x7efc14153f70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7efc0c006fd0 tx=0x7efc0c0090f0).stop 2023-05-16T07:03:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 -- 172.21.15.92:0/949126253 shutdown_connections 2023-05-16T07:03:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 --2- 172.21.15.92:0/949126253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14153b00 0x7efc14153f70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 --2- 172.21.15.92:0/949126253 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc14069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 -- 172.21.15.92:0/949126253 >> 172.21.15.92:0/949126253 conn(0x7efc140cb550 msgr2=0x7efc140c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 -- 172.21.15.92:0/949126253 shutdown_connections 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.965+0000 7efc1b53b700 1 -- 172.21.15.92:0/949126253 wait complete. 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.966+0000 7efc1b53b700 1 Processor -- start 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.966+0000 7efc1b53b700 1 -- start start 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 0x7efc140c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc1413e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc14153b00 0x7efc1414de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140c49e0 con 0x7efc14069750 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140c2d00 con 0x7efc14153b00 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1413ec90 con 0x7efc140aab40 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 0x7efc140c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 0x7efc140c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42290/0 (socket says 172.21.15.92:42290) 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 -- 172.21.15.92:0/1338644922 learned_addr learned my addr 172.21.15.92:0/1338644922 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc19d38700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc1413e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc14153b00 0x7efc1414de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 -- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 msgr2=0x7efc1413e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc1413e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 -- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc14153b00 msgr2=0x7efc1414de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc14153b00 0x7efc1414de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 -- 172.21.15.92:0/1338644922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c009370 con 0x7efc14069750 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc14153b00 0x7efc1414de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc19d38700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc1413e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1a539700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 0x7efc140c3190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7efc1000c840 tx=0x7efc100025e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc0a7fc700 1 -- 172.21.15.92:0/1338644922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000a580 con 0x7efc14069750 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc0a7fc700 1 -- 172.21.15.92:0/1338644922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1000a700 con 0x7efc14069750 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc140c4700 con 0x7efc14069750 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc0a7fc700 1 -- 172.21.15.92:0/1338644922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000aac0 con 0x7efc14069750 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.967+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc140c2990 con 0x7efc14069750 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.969+0000 7efc0a7fc700 1 -- 172.21.15.92:0/1338644922 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7efc10010070 con 0x7efc14069750 2023-05-16T07:03:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.969+0000 7efc0a7fc700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efbf805ed90 0x7efbf8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.969+0000 7efc0a7fc700 1 -- 172.21.15.92:0/1338644922 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efc100799a0 con 0x7efc14069750 2023-05-16T07:03:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.969+0000 7efc19d38700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efbf805ed90 0x7efbf8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.970+0000 7efc19d38700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efbf805ed90 0x7efbf8061250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7efc140acda0 tx=0x7efc04008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.971+0000 7efc2e12fb80 1 Processor -- start 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.971+0000 7efc2e12fb80 1 -- start start 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd61da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd5bb5e0 con 0x561fcd63e350 2023-05-16T07:03:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd5328a0 con 0x561fcd5f0660 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fcd547900 con 0x561fcd5f56b0 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42298/0 (socket says 172.21.15.92:42298) 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 -- 172.21.15.92:0/3508953487 learned_addr learned my addr 172.21.15.92:0/3508953487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 -- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 msgr2=0x561fcd61da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc1a539700 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd61da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd61da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 -- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 msgr2=0x561fcd5f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 -- 172.21.15.92:0/3508953487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fcd3ea310 con 0x561fcd63e350 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efc19d38700 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f5a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7efc0400ea70 tx=0x7efc04013000).ready entity=mon.0 client_cookie=c377588592cf4014 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efbf77fe700 1 -- 172.21.15.92:0/3508953487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04012750 con 0x561fcd63e350 2023-05-16T07:03:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.972+0000 7efbf77fe700 1 -- 172.21.15.92:0/3508953487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0400c030 con 0x561fcd63e350 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efbf77fe700 1 -- 172.21.15.92:0/3508953487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04012ac0 con 0x561fcd63e350 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 -- 172.21.15.92:0/3508953487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 msgr2=0x561fcd5f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f5a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7efc0400ea70 tx=0x7efc04013000).stop 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 -- 172.21.15.92:0/3508953487 shutdown_connections 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd61da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/3508953487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f5a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 -- 172.21.15.92:0/3508953487 >> 172.21.15.92:0/3508953487 conn(0x561fcd3b2e40 msgr2=0x561fcd63e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 -- 172.21.15.92:0/3508953487 shutdown_connections 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.973+0000 7efc2e12fb80 1 -- 172.21.15.92:0/3508953487 wait complete. 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.974+0000 7efc2e12fb80 1 -- start start 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.974+0000 7efc2e12fb80 1 Processor -- start 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd643460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd620ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd64d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd50eed0 con 0x561fcd5f0660 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd61f180 con 0x561fcd63e350 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fcd61f5e0 con 0x561fcd5f56b0 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 10 client.0 ll_register_callbacks cb 0x561fcd5b53b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd643460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd643460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42304/0 (socket says 172.21.15.92:42304) 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 -- 172.21.15.92:0/96113798 learned_addr learned my addr 172.21.15.92:0/96113798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc19d38700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd620ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd64d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 -- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 msgr2=0x561fcd620ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd620ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 -- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 msgr2=0x561fcd64d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd64d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 -- 172.21.15.92:0/96113798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0400e700 con 0x561fcd5f0660 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc19d38700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd620ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc1a539700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd643460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7efc10002700 tx=0x7efc10050040).ready entity=mon.0 client_cookie=15cdc1169748ac05 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efbf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efbf4ff9700 1 -- 172.21.15.92:0/96113798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10054300 con 0x561fcd5f0660 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efbf4ff9700 1 -- 172.21.15.92:0/96113798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efc1004e0f0 con 0x561fcd5f0660 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fcd5eda40 con 0x561fcd5f0660 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efbf4ff9700 1 -- 172.21.15.92:0/96113798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc1004a9f0 con 0x561fcd5f0660 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 10 client.5136 fetch_fsmap learned FSMap version 12 2023-05-16T07:03:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x561fcd540880 con 0x561fcd5f0660 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.975+0000 7efbf4ff9700 1 -- 172.21.15.92:0/96113798 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561fcd5eda40 con 0x561fcd5f0660 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efbf4ff9700 1 -- 172.21.15.92:0/96113798 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 12) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efc1004ab50 con 0x561fcd5f0660 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc2e12fb80 10 client.5136 fetch_fsmap finished waiting for FSMap version 12 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc2e12fb80 10 client.5136 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc2e12fb80 20 client.5136 populate_metadata read hostname 'smithi092' 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd623db0 0x561fcd624190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc2e12fb80 4 client.5136 mds_command: new command op to 4935 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:03:18.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 --> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fcd3aeca0 con 0x561fcd623db0 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc19d38700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd623db0 0x561fcd624190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efc19d38700 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd623db0 0x561fcd624190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6e5a9a60c331bad5 server_cookie=807e1b4a2c6d10bc in_seq=0 out_seq=0 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efbf4ff9700 10 client.5136 ms_handle_connect on v2:172.21.15.92:6833/2108552286 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efbf4ff9700 1 -- 172.21.15.92:0/96113798 <== mds.0 v2:172.21.15.92:6833/2108552286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x561fcd3aeca0 con 0x561fcd623db0 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.976+0000 7efbf4ff9700 10 client.5136 handle_command_reply: tid=0 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 client.5136 shutdown 2023-05-16T07:03:18.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 msgr2=0x561fcd643460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd643460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7efc10002700 tx=0x7efc10050040).stop 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 shutdown_connections 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd623db0 0x561fcd624190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd64d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efbf4ff9700 0 client.5136 ms_handle_reset on v2:172.21.15.92:6833/2108552286 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd643460 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/96113798 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd620ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:18.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 >> 172.21.15.92:0/96113798 conn(0x561fcd3b2e40 msgr2=0x561fcd5f38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:18.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 shutdown_connections 2023-05-16T07:03:18.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 1 -- 172.21.15.92:0/96113798 wait complete. 2023-05-16T07:03:18.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.977+0000 7efc2e12fb80 20 client.5136 trim_cache size 0 max 16384 2023-05-16T07:03:18.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.993+0000 7efc2e12fb80 1 Processor -- start 2023-05-16T07:03:18.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.993+0000 7efc2e12fb80 1 -- start start 2023-05-16T07:03:18.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:18.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd643a60 con 0x561fcd63e350 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd63ac40 con 0x561fcd5f0660 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fcd5f3db0 con 0x561fcd5f56b0 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42314/0 (socket says 172.21.15.92:42314) 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 -- 172.21.15.92:0/1608733955 learned_addr learned my addr 172.21.15.92:0/1608733955 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 -- 172.21.15.92:0/1608733955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 msgr2=0x561fcd63eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:19.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 -- 172.21.15.92:0/1608733955 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 msgr2=0x561fcd5f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.994+0000 7efc1ad3a700 1 -- 172.21.15.92:0/1608733955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fcd53c070 con 0x561fcd63e350 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7efc14083d30 tx=0x7efc0c017000).ready entity=mon.0 client_cookie=53ead42cec75b8af server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efbf7fff700 1 -- 172.21.15.92:0/1608733955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00b9a0 con 0x561fcd63e350 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efbf7fff700 1 -- 172.21.15.92:0/1608733955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c01d430 con 0x561fcd63e350 2023-05-16T07:03:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efbf7fff700 1 -- 172.21.15.92:0/1608733955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c01d7b0 con 0x561fcd63e350 2023-05-16T07:03:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1608733955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 msgr2=0x561fcd5f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7efc14083d30 tx=0x7efc0c017000).stop 2023-05-16T07:03:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1608733955 shutdown_connections 2023-05-16T07:03:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd5f0660 0x561fcd5f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1608733955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:19.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1608733955 >> 172.21.15.92:0/1608733955 conn(0x561fcd3b2e40 msgr2=0x561fcd654000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:19.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1608733955 shutdown_connections 2023-05-16T07:03:19.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.995+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1608733955 wait complete. 2023-05-16T07:03:19.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.996+0000 7efc2e12fb80 1 -- start start 2023-05-16T07:03:19.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.996+0000 7efc2e12fb80 1 Processor -- start 2023-05-16T07:03:19.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd69f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd639da0 con 0x561fcd5f0660 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fcd6232a0 con 0x561fcd63e350 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fcd5eff70 con 0x561fcd5f56b0 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 10 client.0 ll_register_callbacks cb 0x7efc1004e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd69f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd69f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42318/0 (socket says 172.21.15.92:42318) 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 -- 172.21.15.92:0/1751037931 learned_addr learned my addr 172.21.15.92:0/1751037931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:19.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1ad3a700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 -- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 msgr2=0x561fcd63eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc19d38700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 -- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 msgr2=0x561fcd5f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 -- 172.21.15.92:0/1751037931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc140bfc80 con 0x561fcd5f0660 2023-05-16T07:03:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc1a539700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd69f200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7efc1004b540 tx=0x7efc1005f000).ready entity=mon.0 client_cookie=e7c9d1df46c152f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efbf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efbf4ff9700 1 -- 172.21.15.92:0/1751037931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1004e0f0 con 0x561fcd5f0660 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efbf4ff9700 1 -- 172.21.15.92:0/1751037931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efc1004db60 con 0x561fcd5f0660 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.997+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc0400c030 con 0x561fcd5f0660 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 1 -- 172.21.15.92:0/1751037931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc1004df30 con 0x561fcd5f0660 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 10 client.5142 fetch_fsmap learned FSMap version 12 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x561fcd53bdc0 con 0x561fcd5f0660 2023-05-16T07:03:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 1 -- 172.21.15.92:0/1751037931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0400c030 con 0x561fcd5f0660 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 1 -- 172.21.15.92:0/1751037931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 12) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efc100791b0 con 0x561fcd5f0660 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 10 client.5142 fetch_fsmap finished waiting for FSMap version 12 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 10 client.5142 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 20 client.5142 populate_metadata read hostname 'smithi092' 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd636db0 0x561fcd637190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 4 client.5142 mds_command: new command op to 4935 tid=0[{"prefix": "status"}] 2023-05-16T07:03:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 --> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561fcd3e5ca0 con 0x561fcd636db0 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc19d38700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd636db0 0x561fcd637190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc19d38700 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd636db0 0x561fcd637190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a21d2a2ac904a343 server_cookie=1abaacf9c07a4b56 in_seq=0 out_seq=0 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 10 client.5142 ms_handle_connect on v2:172.21.15.92:6833/2108552286 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 1 -- 172.21.15.92:0/1751037931 <== mds.0 v2:172.21.15.92:6833/2108552286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561fcd3e5ca0 con 0x561fcd636db0 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 10 client.5142 handle_command_reply: tid=0 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 client.5142 shutdown 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 msgr2=0x561fcd69f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd69f200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7efc1004b540 tx=0x7efc1005f000).stop 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 shutdown_connections 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:6833/2108552286,v1:172.21.15.92:6835/2108552286] conn(0x561fcd636db0 0x561fcd637190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x561fcd63e350 0x561fcd5f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efbf4ff9700 0 client.5142 ms_handle_reset on v2:172.21.15.92:6833/2108552286 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fcd5f0660 0x561fcd69f200 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 --2- 172.21.15.92:0/1751037931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x561fcd5f56b0 0x561fcd63eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.998+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 >> 172.21.15.92:0/1751037931 conn(0x561fcd3b2e40 msgr2=0x561fcd64d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:19.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 shutdown_connections 2023-05-16T07:03:19.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc2e12fb80 1 -- 172.21.15.92:0/1751037931 wait complete. 2023-05-16T07:03:19.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc2e12fb80 20 client.5142 trim_cache size 0 max 16384 2023-05-16T07:03:19.013 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:03:19.013 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:03:19.013 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4935, 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 9.0544436220000009, 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 12, 2023-05-16T07:03:19.014 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 34, 2023-05-16T07:03:19.015 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 34, 2023-05-16T07:03:19.015 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.307077136 2023-05-16T07:03:19.015 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:03:19.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efbf805ed90 msgr2=0x7efbf8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efbf805ed90 0x7efbf8061250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7efc140acda0 tx=0x7efc04008040).stop 2023-05-16T07:03:19.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 msgr2=0x7efc140c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 0x7efc140c3190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7efc1000c840 tx=0x7efc100025e0).stop 2023-05-16T07:03:19.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 shutdown_connections 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efbf805ed90 0x7efbf8061250 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efc14153b00 0x7efc1414de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc14069750 0x7efc140c3190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 --2- 172.21.15.92:0/1338644922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efc140aab40 0x7efc1413e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 >> 172.21.15.92:0/1338644922 conn(0x7efc140cb550 msgr2=0x7efc140c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 shutdown_connections 2023-05-16T07:03:19.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:18.999+0000 7efc1b53b700 1 -- 172.21.15.92:0/1338644922 wait complete. 2023-05-16T07:03:19.018 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:03:19.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.112+0000 7f8bf9b74700 1 Processor -- start 2023-05-16T07:03:19.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.112+0000 7f8bf9b74700 1 -- start start 2023-05-16T07:03:19.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf9b74700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf413b650 0x7f8bf4068ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf9b74700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8bf40a8df0 0x7f8bf40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf9b74700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf4069ab0 con 0x7f8bf413b270 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf9b74700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf4069d90 con 0x7f8bf40a8df0 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf9b74700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf406a070 con 0x7f8bf413b650 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8bf413b270 0x7f8bf4156110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53074/0 (socket says 172.21.15.92:53074) 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf37fe700 1 -- 172.21.15.92:0/445252484 learned_addr learned my addr 172.21.15.92:0/445252484 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/445252484 >> v2:172.21.15.158:3300/0 conn(0x7f8bf40a8df0 0x7f8bf40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.113+0000 7f8bf2ffd700 1 --2- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf413b650 0x7f8bf4068ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf27fc700 1 -- 172.21.15.92:0/445252484 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092772478 0 0) 0x7f8bf4069ab0 con 0x7f8bf413b270 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf27fc700 1 -- 172.21.15.92:0/445252484 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bdc0033f0 con 0x7f8bf413b270 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf3fff700 1 -- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf413b650 msgr2=0x7f8bf4068ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf413b650 0x7f8bf4068ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf3fff700 1 -- 172.21.15.92:0/445252484 >> v1:172.21.15.92:6789/0 conn(0x7f8bf413b270 legacy=0x7f8bf4156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf3fff700 1 -- 172.21.15.92:0/445252484 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40c43a0 con 0x7f8bf40a8df0 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf27fc700 1 -- 172.21.15.92:0/445252484 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557837754 0 0) 0x7f8bdc0033f0 con 0x7f8bf413b270 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf40a8df0 0x7f8bf40a9280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f8bec009760 tx=0x7f8bec009060).ready entity=mon.1 client_cookie=1030994f8c8051bd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf27fc700 1 -- 172.21.15.92:0/445252484 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be40017f0 con 0x7f8bf40a8df0 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf27fc700 1 -- 172.21.15.92:0/445252484 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec011400 con 0x7f8bf40a8df0 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.114+0000 7f8bf27fc700 1 -- 172.21.15.92:0/445252484 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec011780 con 0x7f8bf40a8df0 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 -- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf40a8df0 msgr2=0x7f8bf40a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf40a8df0 0x7f8bf40a9280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f8bec009760 tx=0x7f8bec009060).stop 2023-05-16T07:03:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 -- 172.21.15.92:0/445252484 shutdown_connections 2023-05-16T07:03:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf40a8df0 0x7f8bf40a9280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/445252484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf413b650 0x7f8bf4068ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 -- 172.21.15.92:0/445252484 >> 172.21.15.92:0/445252484 conn(0x7f8bf4063ec0 msgr2=0x7f8bf40669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 -- 172.21.15.92:0/445252484 shutdown_connections 2023-05-16T07:03:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.115+0000 7f8bf9b74700 1 -- 172.21.15.92:0/445252484 wait complete. 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.116+0000 7f8bf9b74700 1 Processor -- start 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.116+0000 7f8bf9b74700 1 -- start start 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf40a8df0 0x7f8bf41659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf413b270 0x7f8bf40bc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 0x7f8bf4138240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40bd9d0 con 0x7f8bf413b650 2023-05-16T07:03:19.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40c6a90 con 0x7f8bf413b270 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40bce60 con 0x7f8bf40a8df0 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 0x7f8bf4138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 0x7f8bf4138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42336/0 (socket says 172.21.15.92:42336) 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 -- 172.21.15.92:0/209008026 learned_addr learned my addr 172.21.15.92:0/209008026 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf2ffd700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf413b270 0x7f8bf40bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf37fe700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf40a8df0 0x7f8bf41659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 -- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf40a8df0 msgr2=0x7f8bf41659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf40a8df0 0x7f8bf41659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 -- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf413b270 msgr2=0x7f8bf40bc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf413b270 0x7f8bf40bc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 -- 172.21.15.92:0/209008026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0092a0 con 0x7f8bf413b650 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf37fe700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf40a8df0 0x7f8bf41659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 0x7f8bf4138240 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f8bec009d50 tx=0x7f8bec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec010bb0 con 0x7f8bf413b650 2023-05-16T07:03:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec010d10 con 0x7f8bf413b650 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf40bd130 con 0x7f8bf413b650 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec01d660 con 0x7f8bf413b650 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.117+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf40bd6f0 con 0x7f8bf413b650 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.118+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf40c6860 con 0x7f8bf413b650 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.119+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8bec024030 con 0x7f8bf413b650 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.119+0000 7f8bdb7fe700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8be005ed90 0x7f8be0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.119+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8bec07a8f0 con 0x7f8bf413b650 2023-05-16T07:03:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.119+0000 7f8bf37fe700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8be005ed90 0x7f8be0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.120+0000 7f8bf37fe700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8be005ed90 0x7f8be0061250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f8be4001560 tx=0x7f8be4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.123+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8bf40c6860 con 0x7f8bf413b650 2023-05-16T07:03:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.324+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8bf40c6c20 con 0x7f8bf413b650 2023-05-16T07:03:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.324+0000 7f8bdb7fe700 1 -- 172.21.15.92:0/209008026 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8bec022070 con 0x7f8bf413b650 2023-05-16T07:03:19.329 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:03:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8be005ed90 msgr2=0x7f8be0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8be005ed90 0x7f8be0061250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f8be4001560 tx=0x7f8be4009040).stop 2023-05-16T07:03:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 msgr2=0x7f8bf4138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 0x7f8bf4138240 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f8bec009d50 tx=0x7f8bec016000).stop 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 shutdown_connections 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8be005ed90 0x7f8be0061250 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bf40a8df0 0x7f8bf41659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf413b650 0x7f8bf4138240 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 --2- 172.21.15.92:0/209008026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bf413b270 0x7f8bf40bc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 >> 172.21.15.92:0/209008026 conn(0x7f8bf4063ec0 msgr2=0x7f8bf4154900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:19.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 shutdown_connections 2023-05-16T07:03:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:19.326+0000 7f8bf9b74700 1 -- 172.21.15.92:0/209008026 wait complete. 2023-05-16T07:03:19.340 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:03:19.341 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:03:19.341 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:03:19.341 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:03:19.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:03:19.342 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:03:19.342 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:03:19.342 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:03:19.342 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:03:19.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.343 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:03:19.358 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:03:19.358 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:03:19.359 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:03:19.360 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:03:19.360 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:03:19.360 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:03:19.360 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:03:19.360 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:03:19.360 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:03:19.361 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41956sec preferred_lft 41956sec 2023-05-16T07:03:19.361 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:03:19.361 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.361 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:03:19.361 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:03:19.361 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:03:19.362 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.362 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:03:19.362 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.362 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:03:19.362 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:03:19.363 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:03:19.363 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.363 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:03:19.363 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:03:19.363 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:03:19.363 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:03:19.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.364 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:03:19.420 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:03:19.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.422 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:03:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:03:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:03:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:03:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:03:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:03:19.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.478 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:03:19.478 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:03:19.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:03:19.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:03:19.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:03:19.480 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:03:19.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.575 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:03:19.589 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:03:19.590 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:03:19.590 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:03:19.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:03:19.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.658 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:03:19.712 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:03:19.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:03:19.789 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:03:19.790 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:03:19.832 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:03:19.832 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:03:19.834 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:03:19.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:03:19.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:03:19.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:03:20.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.048+0000 7fe5f78f4700 1 Processor -- start 2023-05-16T07:03:20.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.048+0000 7fe5f78f4700 1 -- start start 2023-05-16T07:03:20.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f78f4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f78f4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5f0063e30 0x7fe5f00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f78f4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f00bdc30 con 0x7fe5f013b930 2023-05-16T07:03:20.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f78f4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00c5400 con 0x7fe5f0063e30 2023-05-16T07:03:20.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f78f4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00c4e40 con 0x7fe5f013b550 2023-05-16T07:03:20.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f5e91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5f0063e30 0x7fe5f00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f4e8f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe5f013b930 0x7fe5f0063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53094/0 (socket says 172.21.15.92:53094) 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f4e8f700 1 -- 172.21.15.92:0/4129877119 learned_addr learned my addr 172.21.15.92:0/4129877119 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.049+0000 7fe5f5e91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5f0063e30 0x7fe5f00abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42346/0 (socket says 172.21.15.92:42346) 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5e91700 1 -- 172.21.15.92:0/4129877119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 msgr2=0x7fe5f00c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5690700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5e91700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5e91700 1 -- 172.21.15.92:0/4129877119 >> v1:172.21.15.92:6789/0 conn(0x7fe5f013b930 legacy=0x7fe5f0063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5effff700 1 -- 172.21.15.92:0/4129877119 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193921665 0 0) 0x7fe5f00bdc30 con 0x7fe5f013b930 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5e91700 1 -- 172.21.15.92:0/4129877119 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00c5120 con 0x7fe5f0063e30 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5690700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f5e91700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f00abb20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fe5e8009760 tx=0x7fe5e8009060).ready entity=mon.0 client_cookie=92a1724b0029cca1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5effff700 1 -- 172.21.15.92:0/4129877119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800b580 con 0x7fe5f0063e30 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5effff700 1 -- 172.21.15.92:0/4129877119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e800b6e0 con 0x7fe5f0063e30 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5effff700 1 -- 172.21.15.92:0/4129877119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800ba60 con 0x7fe5f0063e30 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f78f4700 1 -- 172.21.15.92:0/4129877119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 msgr2=0x7fe5f00abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f00abb20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fe5e8009760 tx=0x7fe5e8009060).stop 2023-05-16T07:03:20.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f78f4700 1 -- 172.21.15.92:0/4129877119 shutdown_connections 2023-05-16T07:03:20.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/4129877119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f00abb20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.050+0000 7fe5f78f4700 1 -- 172.21.15.92:0/4129877119 >> 172.21.15.92:0/4129877119 conn(0x7fe5f00c8aa0 msgr2=0x7fe5f00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:20.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.051+0000 7fe5f78f4700 1 -- 172.21.15.92:0/4129877119 shutdown_connections 2023-05-16T07:03:20.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.051+0000 7fe5f78f4700 1 -- 172.21.15.92:0/4129877119 wait complete. 2023-05-16T07:03:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.052+0000 7fe5f78f4700 1 Processor -- start 2023-05-16T07:03:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.052+0000 7fe5f78f4700 1 -- start start 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.052+0000 7fe5f78f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f0135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.052+0000 7fe5f78f4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f78f4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe5f013b930 0x7fe5f01371d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f78f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0155280 con 0x7fe5f0063e30 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f78f4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0154df0 con 0x7fe5f013b930 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f78f4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01541e0 con 0x7fe5f013b550 2023-05-16T07:03:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f0135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f0135440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42352/0 (socket says 172.21.15.92:42352) 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 -- 172.21.15.92:0/1752844952 learned_addr learned my addr 172.21.15.92:0/1752844952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 -- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 msgr2=0x7fe5f00c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f4e8f700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5e91700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe5f013b930 0x7fe5f01371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 -- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe5f013b930 msgr2=0x7fe5f01371d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe5f013b930 0x7fe5f01371d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 -- 172.21.15.92:0/1752844952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80092a0 con 0x7fe5f0063e30 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f5690700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f0135440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fe5e000a400 tx=0x7fe5e0007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000ae70 con 0x7fe5f0063e30 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e000d650 con 0x7fe5f0063e30 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f01544c0 con 0x7fe5f0063e30 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000d9d0 con 0x7fe5f0063e30 2023-05-16T07:03:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.053+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f0154a80 con 0x7fe5f0063e30 2023-05-16T07:03:20.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.054+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f006c350 con 0x7fe5f0063e30 2023-05-16T07:03:20.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.055+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe5e001d060 con 0x7fe5f0063e30 2023-05-16T07:03:20.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.055+0000 7fe5ed7fa700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe5dc05ed60 0x7fe5dc061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.055+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe5e007a850 con 0x7fe5f0063e30 2023-05-16T07:03:20.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.055+0000 7fe5f4e8f700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe5dc05ed60 0x7fe5dc061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.055+0000 7fe5f4e8f700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe5dc05ed60 0x7fe5dc061220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fe5f0139060 tx=0x7fe5e4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.059+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe5e004bdf0 con 0x7fe5f0063e30 2023-05-16T07:03:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.246+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5f00c4b50 con 0x7fe5dc05ed60 2023-05-16T07:03:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.248+0000 7fe5ed7fa700 1 -- 172.21.15.92:0/1752844952 <== mgr.4100 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe5f00c4b50 con 0x7fe5dc05ed60 2023-05-16T07:03:20.251 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:03:20.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:03:20.252 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:03:20.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:03:20.252 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:03:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe5dc05ed60 msgr2=0x7fe5dc061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe5dc05ed60 0x7fe5dc061220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fe5f0139060 tx=0x7fe5e4009040).stop 2023-05-16T07:03:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 msgr2=0x7fe5f0135440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f0135440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fe5e000a400 tx=0x7fe5e0007910).stop 2023-05-16T07:03:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 shutdown_connections 2023-05-16T07:03:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe5dc05ed60 0x7fe5dc061220 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe5f013b930 0x7fe5f01371d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f0063e30 0x7fe5f0135440 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 --2- 172.21.15.92:0/1752844952 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5f013b550 0x7fe5f00c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 >> 172.21.15.92:0/1752844952 conn(0x7fe5f00c8aa0 msgr2=0x7fe5f00c5540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 shutdown_connections 2023-05-16T07:03:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.250+0000 7fe5f78f4700 1 -- 172.21.15.92:0/1752844952 wait complete. 2023-05-16T07:03:20.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:03:20.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.374+0000 7f6fd54ff700 1 Processor -- start 2023-05-16T07:03:20.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.374+0000 7f6fd54ff700 1 -- start start 2023-05-16T07:03:20.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fd54ff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 0x7f6fd006a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fd54ff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6fd0150650 0x7f6fd0063d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fd54ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00c3130 con 0x7f6fd01546f0 2023-05-16T07:03:20.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fd54ff700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd0154b50 con 0x7f6fd0150650 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fd54ff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0150aa0 con 0x7f6fd0154310 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fce7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6fd01546f0 0x7f6fd014ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53108/0 (socket says 172.21.15.92:53108) 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fce7fc700 1 -- 172.21.15.92:0/3425568970 learned_addr learned my addr 172.21.15.92:0/3425568970 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fcf7fe700 1 --2- 172.21.15.92:0/3425568970 >> v2:172.21.15.158:3300/0 conn(0x7f6fd0150650 0x7f6fd0063d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fceffd700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 0x7f6fd006a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.375+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005320184 0 0) 0x7f6fd00c3130 con 0x7f6fd01546f0 2023-05-16T07:03:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0033f0 con 0x7f6fd01546f0 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585058674 0 0) 0x7f6fbc0033f0 con 0x7f6fd01546f0 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 msgr2=0x7f6fd006a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 0x7f6fd006a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 >> v2:172.21.15.158:3300/0 conn(0x7f6fd0150650 msgr2=0x7f6fd0063d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 --2- 172.21.15.92:0/3425568970 >> v2:172.21.15.158:3300/0 conn(0x7f6fd0150650 0x7f6fd0063d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0068540 con 0x7f6fd01546f0 2023-05-16T07:03:20.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6fb8001470 con 0x7f6fd01546f0 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 >> v1:172.21.15.92:6789/0 conn(0x7f6fd01546f0 legacy=0x7f6fd014ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fbc0038e0 0x7f6fbc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 0x7f6fbc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fbc004b80 0x7f6fbc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0154b50 con 0x7f6fbc004b80 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0044c0 con 0x7f6fbc0038e0 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc00ca40 con 0x7f6fd0154310 2023-05-16T07:03:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fbc004b80 0x7f6fbc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fceffd700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fbc0038e0 0x7f6fbc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fce7fc700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 0x7f6fbc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 -- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 msgr2=0x7f6fbc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fd0154310 0x7f6fbc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 -- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fbc0038e0 msgr2=0x7f6fbc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fbc0038e0 0x7f6fbc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 -- 172.21.15.92:0/3425568970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0033f0 con 0x7f6fbc004b80 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcf7fe700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fbc004b80 0x7f6fbc005000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f6fc8009ab0 tx=0x7f6fc8009060).ready entity=mon.0 client_cookie=f4fafd48402d1de2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.376+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8009430 con 0x7f6fbc004b80 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800e640 con 0x7f6fbc004b80 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fcdffb700 1 -- 172.21.15.92:0/3425568970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800e9c0 con 0x7f6fbc004b80 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fd54ff700 1 -- 172.21.15.92:0/3425568970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fbc004b80 msgr2=0x7f6fbc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/3425568970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fbc004b80 0x7f6fbc005000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f6fc8009ab0 tx=0x7f6fc8009060).stop 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fcf7fe700 1 -- 172.21.15.92:0/3425568970 reap_dead start 2023-05-16T07:03:20.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fd54ff700 1 -- 172.21.15.92:0/3425568970 shutdown_connections 2023-05-16T07:03:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fd54ff700 1 -- 172.21.15.92:0/3425568970 >> 172.21.15.92:0/3425568970 conn(0x7f6fd00c3d90 msgr2=0x7f6fd00c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fd54ff700 1 -- 172.21.15.92:0/3425568970 shutdown_connections 2023-05-16T07:03:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.377+0000 7f6fd54ff700 1 -- 172.21.15.92:0/3425568970 wait complete. 2023-05-16T07:03:20.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.378+0000 7f6fd54ff700 1 Processor -- start 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.378+0000 7f6fd54ff700 1 -- start start 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fd54ff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fbc004b80 0x7f6fd0153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fd54ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 0x7f6fd00a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fd54ff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fd014c290 0x7f6fd00a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fd54ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00ac650 con 0x7f6fd014be10 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fd54ff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0153f70 con 0x7f6fd014c290 2023-05-16T07:03:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fd54ff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00a7620 con 0x7f6fbc004b80 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 0x7f6fd00a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 0x7f6fd00a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42372/0 (socket says 172.21.15.92:42372) 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 -- 172.21.15.92:0/84731930 learned_addr learned my addr 172.21.15.92:0/84731930 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fcf7fe700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fd014c290 0x7f6fd00a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fceffd700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fbc004b80 0x7f6fd0153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 -- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fbc004b80 msgr2=0x7f6fd0153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fbc004b80 0x7f6fd0153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 -- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fd014c290 msgr2=0x7f6fd00a8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fd014c290 0x7f6fd00a8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 -- 172.21.15.92:0/84731930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00a7900 con 0x7f6fd014be10 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fceffd700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fbc004b80 0x7f6fd0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.379+0000 7f6fce7fc700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 0x7f6fd00a86c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f6fb80049c0 tx=0x7f6fd00686d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.380+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800f6c0 con 0x7f6fd014be10 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.380+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb800f820 con 0x7f6fd014be10 2023-05-16T07:03:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.380+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd00a7be0 con 0x7f6fd014be10 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.380+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800fba0 con 0x7f6fd014be10 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.380+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00a81a0 con 0x7f6fd014be10 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.380+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd00a8390 con 0x7f6fd014be10 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.381+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6fb801b030 con 0x7f6fd014be10 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.381+0000 7f6fc6ffd700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fb005ed90 0x7f6fb0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.381+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6fb80793c0 con 0x7f6fd014be10 2023-05-16T07:03:20.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.381+0000 7f6fceffd700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fb005ed90 0x7f6fb0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.382+0000 7f6fceffd700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fb005ed90 0x7f6fb0061250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f6fc0009fd0 tx=0x7f6fc000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.385+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6fb804a960 con 0x7f6fd014be10 2023-05-16T07:03:20.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.591+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6fd006c350 con 0x7f6fd014be10 2023-05-16T07:03:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.620+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 5) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f6fb80772c0 con 0x7f6fd014be10 2023-05-16T07:03:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.622+0000 7f6fc6ffd700 1 -- 172.21.15.92:0/84731930 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v5) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6fb8072040 con 0x7f6fd014be10 2023-05-16T07:03:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fb005ed90 msgr2=0x7f6fb0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fb005ed90 0x7f6fb0061250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f6fc0009fd0 tx=0x7f6fc000b040).stop 2023-05-16T07:03:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 msgr2=0x7f6fd00a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 0x7f6fd00a86c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f6fb80049c0 tx=0x7f6fd00686d0).stop 2023-05-16T07:03:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 shutdown_connections 2023-05-16T07:03:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fb005ed90 0x7f6fb0061250 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fbc004b80 0x7f6fd0153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fd014c290 0x7f6fd00a8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 --2- 172.21.15.92:0/84731930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fd014be10 0x7f6fd00a86c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 >> 172.21.15.92:0/84731930 conn(0x7f6fd00c3d90 msgr2=0x7f6fd0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 shutdown_connections 2023-05-16T07:03:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.624+0000 7f6fd54ff700 1 -- 172.21.15.92:0/84731930 wait complete. 2023-05-16T07:03:20.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:03:20.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.753+0000 7f9fa34a5700 1 Processor -- start 2023-05-16T07:03:20.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.754+0000 7f9fa34a5700 1 -- start start 2023-05-16T07:03:20.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.754+0000 7f9fa34a5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f9c0a4a40 0x7f9f9c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa34a5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9f9c068a70 0x7f9f9c068f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa34a5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c14e940 con 0x7f9f9c1549e0 2023-05-16T07:03:20.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa34a5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c137430 con 0x7f9f9c0a4a40 2023-05-16T07:03:20.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa34a5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c06a390 con 0x7f9f9c068a70 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f9c0a4a40 0x7f9f9c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f9c0a4a40 0x7f9f9c0a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42412/0 (socket says 172.21.15.92:42412) 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 -- 172.21.15.92:0/3092981806 learned_addr learned my addr 172.21.15.92:0/3092981806 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1a42700 1 --2- 172.21.15.92:0/3092981806 >> v2:172.21.15.158:3300/0 conn(0x7f9f9c068a70 0x7f9f9c068f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 -- 172.21.15.92:0/3092981806 >> v2:172.21.15.158:3300/0 conn(0x7f9f9c068a70 msgr2=0x7f9f9c068f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 --2- 172.21.15.92:0/3092981806 >> v2:172.21.15.158:3300/0 conn(0x7f9f9c068a70 0x7f9f9c068f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 -- 172.21.15.92:0/3092981806 >> v1:172.21.15.158:6789/0 conn(0x7f9f9c1549e0 legacy=0x7f9f9c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 -- 172.21.15.92:0/3092981806 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c069780 con 0x7f9f9c0a4a40 2023-05-16T07:03:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9fa1241700 1 --2- 172.21.15.92:0/3092981806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c0a4e20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9f9c0c7080 tx=0x7f9f8c0090f0).ready entity=mon.0 client_cookie=dc862f53e9051cf9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9f93fff700 1 -- 172.21.15.92:0/3092981806 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00b520 con 0x7f9f9c0a4a40 2023-05-16T07:03:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9f93fff700 1 -- 172.21.15.92:0/3092981806 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c00b680 con 0x7f9f9c0a4a40 2023-05-16T07:03:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.755+0000 7f9f93fff700 1 -- 172.21.15.92:0/3092981806 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00ba00 con 0x7f9f9c0a4a40 2023-05-16T07:03:20.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 -- 172.21.15.92:0/3092981806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c0a4a40 msgr2=0x7f9f9c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/3092981806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c0a4e20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9f9c0c7080 tx=0x7f9f8c0090f0).stop 2023-05-16T07:03:20.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 -- 172.21.15.92:0/3092981806 shutdown_connections 2023-05-16T07:03:20.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/3092981806 >> v2:172.21.15.158:3300/0 conn(0x7f9f9c068a70 0x7f9f9c068f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/3092981806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c0a4e20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 -- 172.21.15.92:0/3092981806 >> 172.21.15.92:0/3092981806 conn(0x7f9f9c0c4620 msgr2=0x7f9f9c066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 -- 172.21.15.92:0/3092981806 shutdown_connections 2023-05-16T07:03:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.756+0000 7f9fa34a5700 1 -- 172.21.15.92:0/3092981806 wait complete. 2023-05-16T07:03:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.757+0000 7f9fa34a5700 1 Processor -- start 2023-05-16T07:03:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.757+0000 7f9fa34a5700 1 -- start start 2023-05-16T07:03:20.765 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:20.757+0000 7f336e8de680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:03:20.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa34a5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f9c068a70 0x7f9f9c1655b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa34a5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c167110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa34a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 0x7f9f9c0aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa34a5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0a72a0 con 0x7f9f9c1549e0 2023-05-16T07:03:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa34a5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c165280 con 0x7f9f9c0a4a40 2023-05-16T07:03:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa34a5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0bf120 con 0x7f9f9c068a70 2023-05-16T07:03:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 0x7f9f9c0aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 0x7f9f9c0aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42422/0 (socket says 172.21.15.92:42422) 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 -- 172.21.15.92:0/1571798964 learned_addr learned my addr 172.21.15.92:0/1571798964 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa0a40700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1241700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f9c068a70 0x7f9f9c1655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 -- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f9c068a70 msgr2=0x7f9f9c1655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f9c068a70 0x7f9f9c1655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 -- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f9c0a4a40 msgr2=0x7f9f9c167110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c167110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 -- 172.21.15.92:0/1571798964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c009370 con 0x7f9f9c1549e0 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1241700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f9c068a70 0x7f9f9c1655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.758+0000 7f9fa1a42700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 0x7f9f9c0aa060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9f94009e90 tx=0x7f9f9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.759+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400cb50 con 0x7f9f9c1549e0 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.759+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9400f040 con 0x7f9f9c1549e0 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.759+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c0bf400 con 0x7f9f9c1549e0 2023-05-16T07:03:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.759+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400cec0 con 0x7f9f9c1549e0 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.759+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c165aa0 con 0x7f9f9c1549e0 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.759+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c0bf820 con 0x7f9f9c1549e0 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.760+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f9f94013610 con 0x7f9f9c1549e0 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.761+0000 7f9f917fa700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9f8005ede0 0x7f9f800612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.761+0000 7f9fa1241700 1 -- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9f8005ede0 msgr2=0x7f9f800612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.761+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9f9407af00 con 0x7f9f9c1549e0 2023-05-16T07:03:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.761+0000 7f9fa1241700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9f8005ede0 0x7f9f800612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:03:20.771 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:20.760+0000 7febd90a1680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:03:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.764+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9f9404c440 con 0x7f9f9c1549e0 2023-05-16T07:03:20.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:20.822+0000 7febd90a1680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:03:20.826 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:20.822+0000 7f336e8de680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:03:20.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.937+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f9f9c06c350 con 0x7f9f9c1549e0 2023-05-16T07:03:20.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.941+0000 7f9f917fa700 1 -- 172.21.15.92:0/1571798964 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v3)=0 v3) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9f9404fa60 con 0x7f9f9c1549e0 2023-05-16T07:03:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9f8005ede0 msgr2=0x7f9f800612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:03:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9f8005ede0 0x7f9f800612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 msgr2=0x7f9f9c0aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 0x7f9f9c0aa060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9f94009e90 tx=0x7f9f9400b040).stop 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 shutdown_connections 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9f8005ede0 0x7f9f800612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f9c068a70 0x7f9f9c1655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f9c1549e0 0x7f9f9c0aa060 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 --2- 172.21.15.92:0/1571798964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f9c0a4a40 0x7f9f9c167110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 >> 172.21.15.92:0/1571798964 conn(0x7f9f9c0c4620 msgr2=0x7f9f9c0c1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 shutdown_connections 2023-05-16T07:03:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:20.943+0000 7f9fa34a5700 1 -- 172.21.15.92:0/1571798964 wait complete. 2023-05-16T07:03:20.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:20.960 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:03:20.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:20.980 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-05-16T07:03:21.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-05-16T07:03:21.095 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:21.091+0000 7febd90a1680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:03:21.108 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:21.104+0000 7f336e8de680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:03:21.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 Processor -- start 2023-05-16T07:03:21.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 -- start start 2023-05-16T07:03:21.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef48062e80 0x7fef480bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:21.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fef48067f60 0x7fef480622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:21.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef48069070 con 0x7fef48153ed0 2023-05-16T07:03:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480c3b40 con 0x7fef48062e80 2023-05-16T07:03:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.193+0000 7fef4ea38700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef48068460 con 0x7fef48067f60 2023-05-16T07:03:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef48062e80 0x7fef480bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef48062e80 0x7fef480bf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42472/0 (socket says 172.21.15.92:42472) 2023-05-16T07:03:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 -- 172.21.15.92:0/3227592407 learned_addr learned my addr 172.21.15.92:0/3227592407 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef47fff700 1 --1- 172.21.15.92:0/3227592407 >> v1:172.21.15.92:6789/0 conn(0x7fef48153ed0 0x7fef480c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53218/0 (socket says 172.21.15.92:53218) 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef4cfd5700 1 --2- 172.21.15.92:0/3227592407 >> v2:172.21.15.158:3300/0 conn(0x7fef48067f60 0x7fef480622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 -- 172.21.15.92:0/3227592407 >> v2:172.21.15.158:3300/0 conn(0x7fef48067f60 msgr2=0x7fef480622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 --2- 172.21.15.92:0/3227592407 >> v2:172.21.15.158:3300/0 conn(0x7fef48067f60 0x7fef480622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 -- 172.21.15.92:0/3227592407 >> v1:172.21.15.92:6789/0 conn(0x7fef48153ed0 legacy=0x7fef480c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 -- 172.21.15.92:0/3227592407 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48068740 con 0x7fef48062e80 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef46ffd700 1 -- 172.21.15.92:0/3227592407 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644977706 0 0) 0x7fef48069070 con 0x7fef48153ed0 2023-05-16T07:03:21.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef477fe700 1 --2- 172.21.15.92:0/3227592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef480bf1d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fef48064cc0 tx=0x7fef380090f0).ready entity=mon.0 client_cookie=4c79b8b3e4b4d513 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:21.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef46ffd700 1 -- 172.21.15.92:0/3227592407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800b600 con 0x7fef48062e80 2023-05-16T07:03:21.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef46ffd700 1 -- 172.21.15.92:0/3227592407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3800b760 con 0x7fef48062e80 2023-05-16T07:03:21.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.194+0000 7fef46ffd700 1 -- 172.21.15.92:0/3227592407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800bae0 con 0x7fef48062e80 2023-05-16T07:03:21.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 -- 172.21.15.92:0/3227592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 msgr2=0x7fef480bf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:21.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 --2- 172.21.15.92:0/3227592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef480bf1d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fef48064cc0 tx=0x7fef380090f0).stop 2023-05-16T07:03:21.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 -- 172.21.15.92:0/3227592407 shutdown_connections 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 --2- 172.21.15.92:0/3227592407 >> v2:172.21.15.158:3300/0 conn(0x7fef48067f60 0x7fef480622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 --2- 172.21.15.92:0/3227592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef480bf1d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 -- 172.21.15.92:0/3227592407 >> 172.21.15.92:0/3227592407 conn(0x7fef4805dd50 msgr2=0x7fef4805e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 -- 172.21.15.92:0/3227592407 shutdown_connections 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.195+0000 7fef4ea38700 1 -- 172.21.15.92:0/3227592407 wait complete. 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.196+0000 7fef4ea38700 1 Processor -- start 2023-05-16T07:03:21.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.196+0000 7fef4ea38700 1 -- start start 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4ea38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef4814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4ea38700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef48067f60 0x7fef480a8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4ea38700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef48153ed0 0x7fef48156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4ea38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef480a8a60 con 0x7fef48062e80 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4ea38700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef48157160 con 0x7fef48153ed0 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4ea38700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef480a9880 con 0x7fef48067f60 2023-05-16T07:03:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef4814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef4814ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42476/0 (socket says 172.21.15.92:42476) 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef47fff700 1 -- 172.21.15.92:0/727270792 learned_addr learned my addr 172.21.15.92:0/727270792 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef48067f60 0x7fef480a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.197+0000 7fef4cfd5700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef48153ed0 0x7fef48156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef47fff700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef48067f60 msgr2=0x7fef480a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef47fff700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef48067f60 0x7fef480a8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:21.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef47fff700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef48153ed0 msgr2=0x7fef48156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef47fff700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef48153ed0 0x7fef48156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef47fff700 1 -- 172.21.15.92:0/727270792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009370 con 0x7fef48062e80 2023-05-16T07:03:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef47fff700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef4814ca00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fef3c00c440 tx=0x7fef3c007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0070d0 con 0x7fef48062e80 2023-05-16T07:03:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c007230 con 0x7fef48062e80 2023-05-16T07:03:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef480a8c70 con 0x7fef48062e80 2023-05-16T07:03:21.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0056e0 con 0x7fef48062e80 2023-05-16T07:03:21.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.198+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef480a9230 con 0x7fef48062e80 2023-05-16T07:03:21.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.199+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4806c350 con 0x7fef48062e80 2023-05-16T07:03:21.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.199+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fef3c010070 con 0x7fef48062e80 2023-05-16T07:03:21.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.200+0000 7fef2ffff700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:21.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.200+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fef3c079010 con 0x7fef48062e80 2023-05-16T07:03:21.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.200+0000 7fef477fe700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef340612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:03:21.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.200+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:03:21.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.204+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fef3c04a5b0 con 0x7fef48062e80 2023-05-16T07:03:21.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.399+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fef48068a20 con 0x7fef3405ede0 2023-05-16T07:03:21.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.401+0000 7fef477fe700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef340612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:03:21.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.401+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:03:21.694 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:21.691+0000 7febd90a1680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:03:21.720 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:21.716+0000 7f336e8de680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:03:21.765 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:21.762+0000 7febd90a1680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:03:21.790 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:21.787+0000 7f336e8de680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:03:21.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.802+0000 7fef477fe700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef340612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:03:21.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:21.802+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:03:21.948 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:21.944+0000 7febd90a1680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:03:21.970 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:21.967+0000 7f336e8de680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:03:22.358 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:22.355+0000 7febd90a1680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:03:22.377 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:22.374+0000 7f336e8de680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:03:22.459 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:22.456+0000 7febd90a1680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:03:22.477 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:22.474+0000 7f336e8de680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:03:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:22.603+0000 7fef477fe700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef340612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:03:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:22.603+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:03:22.610 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:22.606+0000 7febd90a1680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:03:22.626 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:22.622+0000 7f336e8de680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:03:22.682 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:22.679+0000 7febd90a1680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:03:22.699 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:22.695+0000 7f336e8de680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:03:22.749 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:22.745+0000 7febd90a1680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:03:22.765 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:22.761+0000 7f336e8de680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:03:23.000 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:22.996+0000 7febd90a1680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:03:23.016 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:23.012+0000 7f336e8de680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:03:23.181 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:23.177+0000 7febd90a1680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:03:23.192 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:23.188+0000 7f336e8de680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:03:23.816 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:23.813+0000 7febd90a1680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:03:23.825 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:23.821+0000 7f336e8de680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:03:23.882 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:23.878+0000 7febd90a1680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:03:23.891 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:23.887+0000 7f336e8de680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:03:23.954 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:23.951+0000 7febd90a1680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:03:23.962 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:23.959+0000 7f336e8de680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:03:24.096 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.092+0000 7febd90a1680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:03:24.103 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:24.100+0000 7f336e8de680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:03:24.163 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.160+0000 7febd90a1680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:03:24.170 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:24.167+0000 7f336e8de680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:03:24.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:24.205+0000 7fef477fe700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef340612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:03:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:24.205+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:03:24.276 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.272+0000 7febd90a1680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:03:24.282 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:24.278+0000 7f336e8de680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:03:24.380 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.376+0000 7febd90a1680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:03:24.386 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:24.382+0000 7f336e8de680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:03:24.666 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.662+0000 7febd90a1680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:03:24.667 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:24.663+0000 7f336e8de680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:03:24.734 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.730+0000 7febd90a1680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:03:24.735 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:03:24.731+0000 7f336e8de680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:03:24.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:24.800+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 6) v1 ==== 75109+0+0 (secure 0 0 0) 0x7fef3c076f10 con 0x7fef48062e80 2023-05-16T07:03:24.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:24.800+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef340612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:03:24.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:24.800+0000 7fef2ffff700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef340612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:24.860 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb57883700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.861 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb57883700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.861 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb57883700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.861 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb57883700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.861 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb57883700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.861 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb5487d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.862 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb5487d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.862 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb5487d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.862 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb5487d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:24.862 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:03:24.857+0000 7feb5487d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:03:25.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.809+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 7) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fef3c077290 con 0x7fef48062e80 2023-05-16T07:03:25.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.809+0000 7fef2ffff700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef3402fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.809+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fef48068a20 con 0x7fef3405ede0 2023-05-16T07:03:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.810+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef3402fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.812+0000 7fef477fe700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef3402fd40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fef38009fd0 tx=0x7fef38017000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.838+0000 7fef2ffff700 1 -- 172.21.15.92:0/727270792 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fef48068a20 con 0x7fef3405ede0 2023-05-16T07:03:25.843 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir0 2023-05-16T07:03:25.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.840+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 msgr2=0x7fef3402fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:25.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.840+0000 7fef4ea38700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef3402fd40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fef38009fd0 tx=0x7fef38017000).stop 2023-05-16T07:03:25.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 msgr2=0x7fef4814ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:25.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef4814ca00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fef3c00c440 tx=0x7fef3c007e30).stop 2023-05-16T07:03:25.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 shutdown_connections 2023-05-16T07:03:25.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef3405ede0 0x7fef3402fd40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:25.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef48153ed0 0x7fef48156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:25.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef48062e80 0x7fef4814ca00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:25.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 --2- 172.21.15.92:0/727270792 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef48067f60 0x7fef480a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:25.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 >> 172.21.15.92:0/727270792 conn(0x7fef4805dd50 msgr2=0x7fef48069500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:25.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 shutdown_connections 2023-05-16T07:03:25.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:25.841+0000 7fef4ea38700 1 -- 172.21.15.92:0/727270792 wait complete. 2023-05-16T07:03:25.858 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-05-16T07:03:25.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:25.859 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-05-16T07:03:25.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-05-16T07:03:26.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.071+0000 7ff8fff84700 1 Processor -- start 2023-05-16T07:03:26.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.071+0000 7ff8fff84700 1 -- start start 2023-05-16T07:03:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fff84700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8f8068870 0x7ff8f80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fff84700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80c94d0 con 0x7ff8f8154470 2023-05-16T07:03:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fff84700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80cb7e0 con 0x7ff8f8154090 2023-05-16T07:03:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fff84700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f80c88c0 con 0x7ff8f8068870 2023-05-16T07:03:26.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fe521700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8f8068870 0x7ff8f80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fd51f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff8f8154470 0x7ff8f8068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53386/0 (socket says 172.21.15.92:53386) 2023-05-16T07:03:26.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fe521700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8f8068870 0x7ff8f80aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42636/0 (socket says 172.21.15.92:42636) 2023-05-16T07:03:26.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.072+0000 7ff8fd51f700 1 -- 172.21.15.92:0/2425817202 learned_addr learned my addr 172.21.15.92:0/2425817202 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:26.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fe521700 1 -- 172.21.15.92:0/2425817202 >> v1:172.21.15.158:6789/0 conn(0x7ff8f8154090 legacy=0x7ff8f80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fe521700 1 -- 172.21.15.92:0/2425817202 >> v1:172.21.15.92:6789/0 conn(0x7ff8f8154470 legacy=0x7ff8f8068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fe521700 1 -- 172.21.15.92:0/2425817202 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80c8ba0 con 0x7ff8f8068870 2023-05-16T07:03:26.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fcd1e700 1 -- 172.21.15.92:0/2425817202 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944982670 0 0) 0x7ff8f80c94d0 con 0x7ff8f8154470 2023-05-16T07:03:26.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fe521700 1 --2- 172.21.15.92:0/2425817202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8068870 0x7ff8f80aaca0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff8f0009760 tx=0x7ff8f0009060).ready entity=mon.0 client_cookie=46a8047e66df5724 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fcd1e700 1 -- 172.21.15.92:0/2425817202 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000b580 con 0x7ff8f8068870 2023-05-16T07:03:26.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fcd1e700 1 -- 172.21.15.92:0/2425817202 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f000b6e0 con 0x7ff8f8068870 2023-05-16T07:03:26.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fcd1e700 1 -- 172.21.15.92:0/2425817202 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000ba60 con 0x7ff8f8068870 2023-05-16T07:03:26.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fff84700 1 -- 172.21.15.92:0/2425817202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8068870 msgr2=0x7ff8f80aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fff84700 1 --2- 172.21.15.92:0/2425817202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8068870 0x7ff8f80aaca0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff8f0009760 tx=0x7ff8f0009060).stop 2023-05-16T07:03:26.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fff84700 1 -- 172.21.15.92:0/2425817202 shutdown_connections 2023-05-16T07:03:26.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fff84700 1 --2- 172.21.15.92:0/2425817202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8068870 0x7ff8f80aaca0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.073+0000 7ff8fff84700 1 -- 172.21.15.92:0/2425817202 >> 172.21.15.92:0/2425817202 conn(0x7ff8f8067920 msgr2=0x7ff8f80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:26.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.074+0000 7ff8fff84700 1 -- 172.21.15.92:0/2425817202 shutdown_connections 2023-05-16T07:03:26.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.074+0000 7ff8fff84700 1 -- 172.21.15.92:0/2425817202 wait complete. 2023-05-16T07:03:26.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 Processor -- start 2023-05-16T07:03:26.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 -- start start 2023-05-16T07:03:26.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8f8068870 0x7ff8f8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8f8154090 0x7ff8f8156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 0x7ff8f814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8156e20 con 0x7ff8f8154470 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80af2a0 con 0x7ff8f8154090 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fff84700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80ae690 con 0x7ff8f8068870 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.075+0000 7ff8fe521700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 0x7ff8f814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 0x7ff8f814e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42642/0 (socket says 172.21.15.92:42642) 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 -- 172.21.15.92:0/1986399812 learned_addr learned my addr 172.21.15.92:0/1986399812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fd51f700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8f8154090 0x7ff8f8156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fdd20700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8f8068870 0x7ff8f8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 -- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8f8068870 msgr2=0x7ff8f8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8f8068870 0x7ff8f8156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 -- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8f8154090 msgr2=0x7ff8f8156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8f8154090 0x7ff8f8156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 -- 172.21.15.92:0/1986399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00092a0 con 0x7ff8f8154470 2023-05-16T07:03:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fd51f700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8f8154090 0x7ff8f8156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fe521700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 0x7ff8f814e250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7ff8f0009eb0 tx=0x7ff8f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000ba60 con 0x7ff8f8154470 2023-05-16T07:03:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f001b410 con 0x7ff8f8154470 2023-05-16T07:03:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f80ae970 con 0x7ff8f8154470 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f001b790 con 0x7ff8f8154470 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.076+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f80aef30 con 0x7ff8f8154470 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.077+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f80a9c60 con 0x7ff8f8154470 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.077+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 75179+0+0 (secure 0 0 0) 0x7ff8f000f070 con 0x7ff8f8154470 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.078+0000 7ff8f5ffb700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8e405ed40 0x7ff8e4061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.078+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff8f007b820 con 0x7ff8f8154470 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.078+0000 7ff8fdd20700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8e405ed40 0x7ff8e4061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.078+0000 7ff8fdd20700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8e405ed40 0x7ff8e4061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff8e8007fd0 tx=0x7ff8e8009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.082+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff8f004cd50 con 0x7ff8f8154470 2023-05-16T07:03:26.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.268+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8f80af430 con 0x7ff8e405ed40 2023-05-16T07:03:26.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.273+0000 7ff8f5ffb700 1 -- 172.21.15.92:0/1986399812 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff8f80af430 con 0x7ff8e405ed40 2023-05-16T07:03:26.278 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir1 2023-05-16T07:03:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8e405ed40 msgr2=0x7ff8e4061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8e405ed40 0x7ff8e4061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff8e8007fd0 tx=0x7ff8e8009040).stop 2023-05-16T07:03:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 msgr2=0x7ff8f814e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 0x7ff8f814e250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7ff8f0009eb0 tx=0x7ff8f0017000).stop 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 shutdown_connections 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8e405ed40 0x7ff8e4061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8f8068870 0x7ff8f8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8f8154470 0x7ff8f814e250 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 --2- 172.21.15.92:0/1986399812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8f8154090 0x7ff8f8156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 >> 172.21.15.92:0/1986399812 conn(0x7ff8f8067920 msgr2=0x7ff8f80abd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 shutdown_connections 2023-05-16T07:03:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.275+0000 7ff8fff84700 1 -- 172.21.15.92:0/1986399812 wait complete. 2023-05-16T07:03:26.291 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-05-16T07:03:26.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:26.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-05-16T07:03:26.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-05-16T07:03:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.457+0000 7fd747332700 1 Processor -- start 2023-05-16T07:03:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.457+0000 7fd747332700 1 -- start start 2023-05-16T07:03:26.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd747332700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 0x7fd7401542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd747332700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd740155870 con 0x7fd740062cc0 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd747332700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd740063120 con 0x7fd7400be280 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd747332700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd740155670 con 0x7fd740153ee0 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd7448cd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd740062cc0 0x7fd7400bdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53398/0 (socket says 172.21.15.92:53398) 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd7448cd700 1 -- 172.21.15.92:0/29309212 learned_addr learned my addr 172.21.15.92:0/29309212 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129622371 0 0) 0x7fd740155870 con 0x7fd740062cc0 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd7450ce700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 0x7fd7401542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd72c0033f0 con 0x7fd740062cc0 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 462641428 0 0) 0x7fd72c0033f0 con 0x7fd740062cc0 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 msgr2=0x7fd7401542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 0x7fd7401542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 >> v1:172.21.15.158:6789/0 conn(0x7fd7400be280 legacy=0x7fd740064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7401548b0 con 0x7fd740062cc0 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd730001470 con 0x7fd740062cc0 2023-05-16T07:03:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 >> v1:172.21.15.92:6789/0 conn(0x7fd740062cc0 legacy=0x7fd7400bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 0x7fd72c003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c003e40 0x7fd72c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd72c004b80 0x7fd72c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740155670 con 0x7fd72c004b80 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c00b990 con 0x7fd72c003e40 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.458+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c00bb50 con 0x7fd740153ee0 2023-05-16T07:03:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd72c004b80 0x7fd72c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7450ce700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 0x7fd72c003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7448cd700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c003e40 0x7fd72c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 -- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 msgr2=0x7fd72c003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd740153ee0 0x7fd72c003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 -- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c003e40 msgr2=0x7fd72c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c003e40 0x7fd72c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 -- 172.21.15.92:0/29309212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0033f0 con 0x7fd72c004b80 2023-05-16T07:03:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd7458cf700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd72c004b80 0x7fd72c005000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fd738001e30 tx=0x7fd738007f30).ready entity=mon.0 client_cookie=5d8b0123abc526f5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738006a20 con 0x7fd72c004b80 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd738006b80 con 0x7fd72c004b80 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.459+0000 7fd737fff700 1 -- 172.21.15.92:0/29309212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73800c5d0 con 0x7fd72c004b80 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd747332700 1 -- 172.21.15.92:0/29309212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd72c004b80 msgr2=0x7fd72c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd747332700 1 --2- 172.21.15.92:0/29309212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd72c004b80 0x7fd72c005000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fd738001e30 tx=0x7fd738007f30).stop 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd7458cf700 1 -- 172.21.15.92:0/29309212 reap_dead start 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd747332700 1 -- 172.21.15.92:0/29309212 shutdown_connections 2023-05-16T07:03:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd747332700 1 -- 172.21.15.92:0/29309212 >> 172.21.15.92:0/29309212 conn(0x7fd7400c8420 msgr2=0x7fd7400c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd747332700 1 -- 172.21.15.92:0/29309212 shutdown_connections 2023-05-16T07:03:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.460+0000 7fd747332700 1 -- 172.21.15.92:0/29309212 wait complete. 2023-05-16T07:03:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.461+0000 7fd747332700 1 Processor -- start 2023-05-16T07:03:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.461+0000 7fd747332700 1 -- start start 2023-05-16T07:03:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c004b80 0x7fd74013ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 0x7fd74013ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7400c5760 0x7fd7400c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74014dca0 con 0x7fd74013b5c0 2023-05-16T07:03:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74013cdb0 con 0x7fd72c004b80 2023-05-16T07:03:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74013bf30 con 0x7fd7400c5760 2023-05-16T07:03:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 0x7fd74013ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 0x7fd74013ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42662/0 (socket says 172.21.15.92:42662) 2023-05-16T07:03:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 -- 172.21.15.92:0/3765984840 learned_addr learned my addr 172.21.15.92:0/3765984840 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7458cf700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7400c5760 0x7fd7400c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7450ce700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c004b80 0x7fd74013ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 -- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7400c5760 msgr2=0x7fd7400c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7400c5760 0x7fd7400c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 -- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c004b80 msgr2=0x7fd74013ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c004b80 0x7fd74013ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 -- 172.21.15.92:0/3765984840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400c6090 con 0x7fd74013b5c0 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7458cf700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7400c5760 0x7fd7400c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7450ce700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c004b80 0x7fd74013ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7448cd700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 0x7fd74013ba40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fd7300049e0 tx=0x7fd730001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730001bb0 con 0x7fd74013b5c0 2023-05-16T07:03:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730000ca0 con 0x7fd74013b5c0 2023-05-16T07:03:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000fae0 con 0x7fd74013b5c0 2023-05-16T07:03:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74013c1a0 con 0x7fd74013b5c0 2023-05-16T07:03:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.462+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74013c760 con 0x7fd74013b5c0 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.463+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74013cf60 con 0x7fd74013b5c0 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.464+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fd7300133f0 con 0x7fd74013b5c0 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.464+0000 7fd7357fa700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd72405ed40 0x7fd724061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.464+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd730079280 con 0x7fd74013b5c0 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.464+0000 7fd7450ce700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd72405ed40 0x7fd724061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.464+0000 7fd7450ce700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd72405ed40 0x7fd724061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd73c009270 tx=0x7fd73c008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.468+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd7300498f0 con 0x7fd74013b5c0 2023-05-16T07:03:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.653+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd74006c350 con 0x7fd72405ed40 2023-05-16T07:03:26.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.657+0000 7fd7357fa700 1 -- 172.21.15.92:0/3765984840 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd74006c350 con 0x7fd72405ed40 2023-05-16T07:03:26.662 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir2 2023-05-16T07:03:26.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd72405ed40 msgr2=0x7fd724061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd72405ed40 0x7fd724061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd73c009270 tx=0x7fd73c008040).stop 2023-05-16T07:03:26.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 msgr2=0x7fd74013ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 0x7fd74013ba40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fd7300049e0 tx=0x7fd730001e50).stop 2023-05-16T07:03:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 shutdown_connections 2023-05-16T07:03:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd72405ed40 0x7fd724061200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7400c5760 0x7fd7400c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd72c004b80 0x7fd74013ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 --2- 172.21.15.92:0/3765984840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd74013b5c0 0x7fd74013ba40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 >> 172.21.15.92:0/3765984840 conn(0x7fd7400c8420 msgr2=0x7fd740157d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:26.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 shutdown_connections 2023-05-16T07:03:26.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.659+0000 7fd747332700 1 -- 172.21.15.92:0/3765984840 wait complete. 2023-05-16T07:03:26.675 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-05-16T07:03:26.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:26.676 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-05-16T07:03:26.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-05-16T07:03:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.844+0000 7fdb856fc700 1 Processor -- start 2023-05-16T07:03:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.844+0000 7fdb856fc700 1 -- start start 2023-05-16T07:03:26.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb856fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb80154090 0x7fdb800aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb856fc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb800c94d0 con 0x7fdb80068160 2023-05-16T07:03:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb856fc700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb800cb7e0 con 0x7fdb80154470 2023-05-16T07:03:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb856fc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb800c88c0 con 0x7fdb80154090 2023-05-16T07:03:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb7effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb80154090 0x7fdb800aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb7f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdb80068160 0x7fdb800c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53418/0 (socket says 172.21.15.92:53418) 2023-05-16T07:03:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.845+0000 7fdb7f7fe700 1 -- 172.21.15.92:0/2933063187 learned_addr learned my addr 172.21.15.92:0/2933063187 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7effd700 1 -- 172.21.15.92:0/2933063187 >> v1:172.21.15.158:6789/0 conn(0x7fdb80154470 legacy=0x7fdb800a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7effd700 1 -- 172.21.15.92:0/2933063187 >> v1:172.21.15.92:6789/0 conn(0x7fdb80068160 legacy=0x7fdb800c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7effd700 1 -- 172.21.15.92:0/2933063187 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800c8ba0 con 0x7fdb80154090 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7dffb700 1 -- 172.21.15.92:0/2933063187 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939948423 0 0) 0x7fdb800c94d0 con 0x7fdb80068160 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7effd700 1 --2- 172.21.15.92:0/2933063187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154090 0x7fdb800aaca0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fdb70009e60 tx=0x7fdb70009780).ready entity=mon.0 client_cookie=1f8f4a1df25928b2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7dffb700 1 -- 172.21.15.92:0/2933063187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000e070 con 0x7fdb80154090 2023-05-16T07:03:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7dffb700 1 -- 172.21.15.92:0/2933063187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7000bbf0 con 0x7fdb80154090 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb7dffb700 1 -- 172.21.15.92:0/2933063187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70012410 con 0x7fdb80154090 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 -- 172.21.15.92:0/2933063187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154090 msgr2=0x7fdb800aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2933063187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154090 0x7fdb800aaca0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fdb70009e60 tx=0x7fdb70009780).stop 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 -- 172.21.15.92:0/2933063187 shutdown_connections 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2933063187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154090 0x7fdb800aaca0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 -- 172.21.15.92:0/2933063187 >> 172.21.15.92:0/2933063187 conn(0x7fdb80067920 msgr2=0x7fdb800cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 -- 172.21.15.92:0/2933063187 shutdown_connections 2023-05-16T07:03:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.846+0000 7fdb856fc700 1 -- 172.21.15.92:0/2933063187 wait complete. 2023-05-16T07:03:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 Processor -- start 2023-05-16T07:03:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 -- start start 2023-05-16T07:03:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdb80068160 0x7fdb800c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdb80154090 0x7fdb8014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 0x7fdb8013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb800c40e0 con 0x7fdb80154470 2023-05-16T07:03:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb80164db0 con 0x7fdb80068160 2023-05-16T07:03:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb856fc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb80150110 con 0x7fdb80154090 2023-05-16T07:03:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb7f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 0x7fdb8013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.848+0000 7fdb7f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 0x7fdb8013e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42678/0 (socket says 172.21.15.92:42678) 2023-05-16T07:03:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 -- 172.21.15.92:0/2704934564 learned_addr learned my addr 172.21.15.92:0/2704934564 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7e7fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdb80154090 0x7fdb8014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7effd700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdb80068160 0x7fdb800c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 -- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdb80154090 msgr2=0x7fdb8014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdb80154090 0x7fdb8014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 -- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdb80068160 msgr2=0x7fdb800c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdb80068160 0x7fdb800c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 -- 172.21.15.92:0/2704934564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70009a20 con 0x7fdb80154470 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7effd700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdb80068160 0x7fdb800c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb7f7fe700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 0x7fdb8013e810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fdb7800c490 tx=0x7fdb78007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780072e0 con 0x7fdb80154470 2023-05-16T07:03:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb78007440 con 0x7fdb80154470 2023-05-16T07:03:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb800c42f0 con 0x7fdb80154470 2023-05-16T07:03:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780057a0 con 0x7fdb80154470 2023-05-16T07:03:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.849+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb801647e0 con 0x7fdb80154470 2023-05-16T07:03:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.850+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb80164f60 con 0x7fdb80154470 2023-05-16T07:03:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.850+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fdb78010070 con 0x7fdb80154470 2023-05-16T07:03:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.851+0000 7fdb76ffd700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdb6005ee30 0x7fdb600612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.851+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fdb78079370 con 0x7fdb80154470 2023-05-16T07:03:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.851+0000 7fdb7effd700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdb6005ee30 0x7fdb600612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.851+0000 7fdb7effd700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdb6005ee30 0x7fdb600612f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fdb7000b9b0 tx=0x7fdb70009780).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:26.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:26.855+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdb80164f60 con 0x7fdb80154470 2023-05-16T07:03:27.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.043+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb80164f20 con 0x7fdb6005ee30 2023-05-16T07:03:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.048+0000 7fdb76ffd700 1 -- 172.21.15.92:0/2704934564 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fdb80164f20 con 0x7fdb6005ee30 2023-05-16T07:03:27.053 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir3 2023-05-16T07:03:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdb6005ee30 msgr2=0x7fdb600612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdb6005ee30 0x7fdb600612f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fdb7000b9b0 tx=0x7fdb70009780).stop 2023-05-16T07:03:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 msgr2=0x7fdb8013e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 0x7fdb8013e810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fdb7800c490 tx=0x7fdb78007c10).stop 2023-05-16T07:03:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 shutdown_connections 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdb6005ee30 0x7fdb600612f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80154470 0x7fdb8013e810 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdb80068160 0x7fdb800c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 --2- 172.21.15.92:0/2704934564 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdb80154090 0x7fdb8014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 >> 172.21.15.92:0/2704934564 conn(0x7fdb80067920 msgr2=0x7fdb8014dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 shutdown_connections 2023-05-16T07:03:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.050+0000 7fdb856fc700 1 -- 172.21.15.92:0/2704934564 wait complete. 2023-05-16T07:03:27.067 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-05-16T07:03:27.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:27.068 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-05-16T07:03:27.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-05-16T07:03:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.234+0000 7fd30ae13700 1 Processor -- start 2023-05-16T07:03:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.234+0000 7fd30ae13700 1 -- start start 2023-05-16T07:03:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd30ae13700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd3040be280 0x7fd3041542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd30ae13700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd304155870 con 0x7fd304062cc0 2023-05-16T07:03:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd30ae13700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3041548b0 con 0x7fd304153ee0 2023-05-16T07:03:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd30ae13700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd304063150 con 0x7fd3040be280 2023-05-16T07:03:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3093b0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd304062cc0 0x7fd304064e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53434/0 (socket says 172.21.15.92:53434) 2023-05-16T07:03:27.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3093b0700 1 -- 172.21.15.92:0/3118394947 learned_addr learned my addr 172.21.15.92:0/3118394947 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd303fff700 1 --2- 172.21.15.92:0/3118394947 >> v2:172.21.15.158:3300/0 conn(0x7fd3040be280 0x7fd3041542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481230795 0 0) 0x7fd304155870 con 0x7fd304062cc0 2023-05-16T07:03:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f00033f0 con 0x7fd304062cc0 2023-05-16T07:03:27.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 411567002 0 0) 0x7fd2f00033f0 con 0x7fd304062cc0 2023-05-16T07:03:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 >> v2:172.21.15.158:3300/0 conn(0x7fd3040be280 msgr2=0x7fd3041542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 --2- 172.21.15.92:0/3118394947 >> v2:172.21.15.158:3300/0 conn(0x7fd3040be280 0x7fd3041542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 >> v1:172.21.15.158:6789/0 conn(0x7fd304153ee0 legacy=0x7fd3040bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304154b90 con 0x7fd304062cc0 2023-05-16T07:03:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.235+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd2ec001470 con 0x7fd304062cc0 2023-05-16T07:03:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 >> v1:172.21.15.92:6789/0 conn(0x7fd304062cc0 legacy=0x7fd304064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd2f0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2f0004690 0x7fd2f0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2f0005000 0x7fd2f000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304063150 con 0x7fd2f00038e0 2023-05-16T07:03:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f000beb0 con 0x7fd2f0004690 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f000c070 con 0x7fd2f0005000 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd2f0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd303fff700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2f0004690 0x7fd2f0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3093b0700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2f0005000 0x7fd2f000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 -- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2f0005000 msgr2=0x7fd2f000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2f0005000 0x7fd2f000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 -- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2f0004690 msgr2=0x7fd2f0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2f0004690 0x7fd2f0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 -- 172.21.15.92:0/3118394947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00033f0 con 0x7fd2f00038e0 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd303fff700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2f0004690 0x7fd2f0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3093b0700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2f0005000 0x7fd2f000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3093b0700 1 -- 172.21.15.92:0/3118394947 reap_dead start 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd308baf700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd2f0003d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fd2fc001e30 tx=0x7fd2fc007f30).ready entity=mon.0 client_cookie=6a948e6d53d4a813 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc006a20 con 0x7fd2f00038e0 2023-05-16T07:03:27.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc006b80 con 0x7fd2f00038e0 2023-05-16T07:03:27.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.236+0000 7fd3037fe700 1 -- 172.21.15.92:0/3118394947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00c5d0 con 0x7fd2f00038e0 2023-05-16T07:03:27.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 -- 172.21.15.92:0/3118394947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 msgr2=0x7fd2f0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd2f0003d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fd2fc001e30 tx=0x7fd2fc007f30).stop 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 -- 172.21.15.92:0/3118394947 shutdown_connections 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 --2- 172.21.15.92:0/3118394947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd2f0003d00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 -- 172.21.15.92:0/3118394947 >> 172.21.15.92:0/3118394947 conn(0x7fd3040c8420 msgr2=0x7fd3040c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 -- 172.21.15.92:0/3118394947 shutdown_connections 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.237+0000 7fd30ae13700 1 -- 172.21.15.92:0/3118394947 wait complete. 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.238+0000 7fd30ae13700 1 Processor -- start 2023-05-16T07:03:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.238+0000 7fd30ae13700 1 -- start start 2023-05-16T07:03:27.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd3040c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3040bf210 0x7fd3041670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3041675d0 0x7fd3040c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040c3840 con 0x7fd2f00038e0 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040c2fa0 con 0x7fd3040bf210 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040c3280 con 0x7fd3041675d0 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd3040c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd3040c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42694/0 (socket says 172.21.15.92:42694) 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 -- 172.21.15.92:0/1319534186 learned_addr learned my addr 172.21.15.92:0/1319534186 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 -- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3041675d0 msgr2=0x7fd3040c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3041675d0 0x7fd3040c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 -- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3040bf210 msgr2=0x7fd3041670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd303fff700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3040bf210 0x7fd3041670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3040bf210 0x7fd3041670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 -- 172.21.15.92:0/1319534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040c3560 con 0x7fd2f00038e0 2023-05-16T07:03:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd308baf700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd3040c2900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fd2f000be80 tx=0x7fd2fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:27.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc012610 con 0x7fd2f00038e0 2023-05-16T07:03:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc012770 con 0x7fd2f00038e0 2023-05-16T07:03:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd304151530 con 0x7fd2f00038e0 2023-05-16T07:03:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.239+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc012b30 con 0x7fd2f00038e0 2023-05-16T07:03:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.240+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3040c10d0 con 0x7fd2f00038e0 2023-05-16T07:03:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.240+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3041518a0 con 0x7fd2f00038e0 2023-05-16T07:03:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.241+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fd2fc025030 con 0x7fd2f00038e0 2023-05-16T07:03:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.241+0000 7fd300ff9700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2e405ee30 0x7fd2e40612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.241+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd2fc07aa60 con 0x7fd2f00038e0 2023-05-16T07:03:27.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.241+0000 7fd303fff700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2e405ee30 0x7fd2e40612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.242+0000 7fd303fff700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2e405ee30 0x7fd2e40612f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd2f4006d80 tx=0x7fd2f4008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:27.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.245+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd2fc04bf30 con 0x7fd2f00038e0 2023-05-16T07:03:27.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.431+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3040c1680 con 0x7fd2e405ee30 2023-05-16T07:03:27.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.436+0000 7fd300ff9700 1 -- 172.21.15.92:0/1319534186 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd2f4009710 con 0x7fd2e405ee30 2023-05-16T07:03:27.441 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir4 2023-05-16T07:03:27.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2e405ee30 msgr2=0x7fd2e40612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2e405ee30 0x7fd2e40612f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd2f4006d80 tx=0x7fd2f4008040).stop 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 msgr2=0x7fd3040c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd3040c2900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fd2f000be80 tx=0x7fd2fc016000).stop 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 shutdown_connections 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2e405ee30 0x7fd2e40612f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3041675d0 0x7fd3040c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00038e0 0x7fd3040c2900 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 --2- 172.21.15.92:0/1319534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3040bf210 0x7fd3041670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 >> 172.21.15.92:0/1319534186 conn(0x7fd3040c8420 msgr2=0x7fd30414f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:27.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 shutdown_connections 2023-05-16T07:03:27.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.437+0000 7fd30ae13700 1 -- 172.21.15.92:0/1319534186 wait complete. 2023-05-16T07:03:27.454 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-05-16T07:03:27.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:27.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-05-16T07:03:27.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-05-16T07:03:27.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.618+0000 7f78dd3da700 1 Processor -- start 2023-05-16T07:03:27.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.618+0000 7f78dd3da700 1 -- start start 2023-05-16T07:03:27.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.619+0000 7f78dd3da700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f78d8154090 0x7f78d80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.619+0000 7f78dd3da700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80c94d0 con 0x7f78d8068160 2023-05-16T07:03:27.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.619+0000 7f78dd3da700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80cb7e0 con 0x7f78d8154470 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.619+0000 7f78dd3da700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d80c88c0 con 0x7f78d8154090 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.619+0000 7f78d77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f78d8068160 0x7f78d80c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53452/0 (socket says 172.21.15.92:53452) 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.619+0000 7f78d77fe700 1 -- 172.21.15.92:0/3280536129 learned_addr learned my addr 172.21.15.92:0/3280536129 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3280536129 >> v2:172.21.15.158:3300/0 conn(0x7f78d8154090 0x7f78d80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042142118 0 0) 0x7f78d80c94d0 con 0x7f78d8068160 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c40033f0 con 0x7f78d8068160 2023-05-16T07:03:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435572977 0 0) 0x7f78c40033f0 con 0x7f78d8068160 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 >> v2:172.21.15.158:3300/0 conn(0x7f78d8154090 msgr2=0x7f78d80aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 --2- 172.21.15.92:0/3280536129 >> v2:172.21.15.158:3300/0 conn(0x7f78d8154090 0x7f78d80aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 >> v1:172.21.15.158:6789/0 conn(0x7f78d8154470 legacy=0x7f78d80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80c8ba0 con 0x7f78d8068160 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f78d0001470 con 0x7f78d8068160 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3280536129 >> v2:172.21.15.158:3300/0 conn(0x7f78d8154090 0x7f78d80aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:27.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 >> v1:172.21.15.92:6789/0 conn(0x7f78d8068160 legacy=0x7f78d80c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c40038e0 0x7f78c4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78c4004690 0x7f78c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4005000 0x7f78c400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80c88c0 con 0x7f78c4005000 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c400beb0 con 0x7f78c4004690 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c400c070 con 0x7f78c40038e0 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4005000 0x7f78c400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c40038e0 0x7f78c4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d67fc700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78c4004690 0x7f78c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 -- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c40038e0 msgr2=0x7f78c4003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c40038e0 0x7f78c4003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 -- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78c4004690 msgr2=0x7f78c4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78c4004690 0x7f78c4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 -- 172.21.15.92:0/3280536129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40033f0 con 0x7f78c4005000 2023-05-16T07:03:27.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.620+0000 7f78d77fe700 1 -- 172.21.15.92:0/3280536129 reap_dead start 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78d77fe700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4005000 0x7f78c400b8e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f78d000c190 tx=0x7f78d0001e70).ready entity=mon.0 client_cookie=12f3b3e02f7df3d7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0007a40 con 0x7f78c4005000 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d000ca60 con 0x7f78c4005000 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78d5ffb700 1 -- 172.21.15.92:0/3280536129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000ce00 con 0x7f78c4005000 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 -- 172.21.15.92:0/3280536129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4005000 msgr2=0x7f78c400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4005000 0x7f78c400b8e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f78d000c190 tx=0x7f78d0001e70).stop 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 -- 172.21.15.92:0/3280536129 shutdown_connections 2023-05-16T07:03:27.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3280536129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4005000 0x7f78c400b8e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 -- 172.21.15.92:0/3280536129 >> 172.21.15.92:0/3280536129 conn(0x7f78d8067920 msgr2=0x7f78d80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:27.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 -- 172.21.15.92:0/3280536129 shutdown_connections 2023-05-16T07:03:27.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.621+0000 7f78dd3da700 1 -- 172.21.15.92:0/3280536129 wait complete. 2023-05-16T07:03:27.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.622+0000 7f78dd3da700 1 Processor -- start 2023-05-16T07:03:27.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.622+0000 7f78dd3da700 1 -- start start 2023-05-16T07:03:27.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78dd3da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c4005000 0x7f78d8164d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78dd3da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 0x7f78d8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78dd3da700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78d8155740 0x7f78d80ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78dd3da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8165320 con 0x7f78d80b05a0 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78dd3da700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81552b0 con 0x7f78d8155740 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78dd3da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d814e520 con 0x7f78c4005000 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78d67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 0x7f78d8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78d67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 0x7f78d8157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42716/0 (socket says 172.21.15.92:42716) 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78d67fc700 1 -- 172.21.15.92:0/3084333778 learned_addr learned my addr 172.21.15.92:0/3084333778 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c4005000 0x7f78d8164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.623+0000 7f78d77fe700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78d8155740 0x7f78d80ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d67fc700 1 -- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c4005000 msgr2=0x7f78d8164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d67fc700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c4005000 0x7f78d8164d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d67fc700 1 -- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78d8155740 msgr2=0x7f78d80ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d67fc700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78d8155740 0x7f78d80ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d67fc700 1 -- 172.21.15.92:0/3084333778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8154980 con 0x7f78d80b05a0 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c4005000 0x7f78d8164d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78d67fc700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 0x7f78d8157300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f78c0002a30 tx=0x7f78c0007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0003340 con 0x7f78d80b05a0 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c000f8d0 con 0x7f78d80b05a0 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000fc50 con 0x7f78d80b05a0 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d8154c60 con 0x7f78d80b05a0 2023-05-16T07:03:27.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.624+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d8153850 con 0x7f78d80b05a0 2023-05-16T07:03:27.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.625+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8066730 con 0x7f78d80b05a0 2023-05-16T07:03:27.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.625+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f78c001b030 con 0x7f78d80b05a0 2023-05-16T07:03:27.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.625+0000 7f78ceffd700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f78b805ee00 0x7f78b80612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:27.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.625+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f78c0079a60 con 0x7f78d80b05a0 2023-05-16T07:03:27.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.625+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f78b805ee00 0x7f78b80612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:27.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.626+0000 7f78d6ffd700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f78b805ee00 0x7f78b80612c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f78c8009fd0 tx=0x7f78c800b040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:27.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.629+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f78c004af30 con 0x7f78d80b05a0 2023-05-16T07:03:27.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.818+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f78d80c8e80 con 0x7f78b805ee00 2023-05-16T07:03:27.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.823+0000 7f78ceffd700 1 -- 172.21.15.92:0/3084333778 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f78d80c8e80 con 0x7f78b805ee00 2023-05-16T07:03:27.828 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir5 2023-05-16T07:03:27.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f78b805ee00 msgr2=0x7f78b80612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f78b805ee00 0x7f78b80612c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f78c8009fd0 tx=0x7f78c800b040).stop 2023-05-16T07:03:27.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 msgr2=0x7f78d8157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:27.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 0x7f78d8157300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f78c0002a30 tx=0x7f78c0007b90).stop 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 shutdown_connections 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f78b805ee00 0x7f78b80612c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78c4005000 0x7f78d8164d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78d8155740 0x7f78d80ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 --2- 172.21.15.92:0/3084333778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78d80b05a0 0x7f78d8157300 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 >> 172.21.15.92:0/3084333778 conn(0x7f78d8067920 msgr2=0x7f78d80c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 shutdown_connections 2023-05-16T07:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:27.825+0000 7f78dd3da700 1 -- 172.21.15.92:0/3084333778 wait complete. 2023-05-16T07:03:27.842 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-05-16T07:03:27.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:27.844 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-05-16T07:03:27.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-05-16T07:03:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.010+0000 7fef3b2a0700 1 Processor -- start 2023-05-16T07:03:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.010+0000 7fef3b2a0700 1 -- start start 2023-05-16T07:03:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3b2a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef340c6410 0x7fef34155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3b2a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3406a140 con 0x7fef34153f90 2023-05-16T07:03:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3b2a0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3406a420 con 0x7fef340c1160 2023-05-16T07:03:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3b2a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef34069f60 con 0x7fef340c6410 2023-05-16T07:03:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3a29e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fef34153f90 0x7fef340a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53470/0 (socket says 172.21.15.92:53470) 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3aa9f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef340c6410 0x7fef34155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.011+0000 7fef3a29e700 1 -- 172.21.15.92:0/2358061858 learned_addr learned my addr 172.21.15.92:0/2358061858 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3929c700 1 -- 172.21.15.92:0/2358061858 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316098444 0 0) 0x7fef3406a140 con 0x7fef34153f90 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3929c700 1 -- 172.21.15.92:0/2358061858 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef200033f0 con 0x7fef34153f90 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3aa9f700 1 -- 172.21.15.92:0/2358061858 >> v1:172.21.15.158:6789/0 conn(0x7fef340c1160 legacy=0x7fef340c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3aa9f700 1 -- 172.21.15.92:0/2358061858 >> v1:172.21.15.92:6789/0 conn(0x7fef34153f90 legacy=0x7fef340a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3aa9f700 1 -- 172.21.15.92:0/2358061858 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3406a700 con 0x7fef340c6410 2023-05-16T07:03:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3aa9f700 1 --2- 172.21.15.92:0/2358061858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c6410 0x7fef34155380 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fef2c009e60 tx=0x7fef2c009780).ready entity=mon.0 client_cookie=f8d3cc07cea4e3af server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3929c700 1 -- 172.21.15.92:0/2358061858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00e070 con 0x7fef340c6410 2023-05-16T07:03:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3929c700 1 -- 172.21.15.92:0/2358061858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c00bb60 con 0x7fef340c6410 2023-05-16T07:03:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3929c700 1 -- 172.21.15.92:0/2358061858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c012410 con 0x7fef340c6410 2023-05-16T07:03:28.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3b2a0700 1 -- 172.21.15.92:0/2358061858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c6410 msgr2=0x7fef34155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.012+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/2358061858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c6410 0x7fef34155380 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fef2c009e60 tx=0x7fef2c009780).stop 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.013+0000 7fef3b2a0700 1 -- 172.21.15.92:0/2358061858 shutdown_connections 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.013+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/2358061858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c6410 0x7fef34155380 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.013+0000 7fef3b2a0700 1 -- 172.21.15.92:0/2358061858 >> 172.21.15.92:0/2358061858 conn(0x7fef340cb600 msgr2=0x7fef340c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.013+0000 7fef3b2a0700 1 -- 172.21.15.92:0/2358061858 shutdown_connections 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.013+0000 7fef3b2a0700 1 -- 172.21.15.92:0/2358061858 wait complete. 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 Processor -- start 2023-05-16T07:03:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 -- start start 2023-05-16T07:03:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 0x7fef340c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef340c6410 0x7fef3414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef34153f90 0x7fef34152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c2fb0 con 0x7fef340c1160 2023-05-16T07:03:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341569f0 con 0x7fef340c6410 2023-05-16T07:03:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.014+0000 7fef3b2a0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c4740 con 0x7fef34153f90 2023-05-16T07:03:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 0x7fef340c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 0x7fef340c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42728/0 (socket says 172.21.15.92:42728) 2023-05-16T07:03:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 -- 172.21.15.92:0/3582287030 learned_addr learned my addr 172.21.15.92:0/3582287030 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef39a9d700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef340c6410 0x7fef3414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3aa9f700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef34153f90 0x7fef34152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 -- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef34153f90 msgr2=0x7fef34152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef34153f90 0x7fef34152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 -- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef340c6410 msgr2=0x7fef3414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef340c6410 0x7fef3414b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 -- 172.21.15.92:0/3582287030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c009a20 con 0x7fef340c1160 2023-05-16T07:03:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3a29e700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 0x7fef340c2080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fef3000c420 tx=0x7fef30007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300072e0 con 0x7fef340c1160 2023-05-16T07:03:28.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef30007440 con 0x7fef340c1160 2023-05-16T07:03:28.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef340c2710 con 0x7fef340c1160 2023-05-16T07:03:28.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300057a0 con 0x7fef340c1160 2023-05-16T07:03:28.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.015+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340c2cd0 con 0x7fef340c1160 2023-05-16T07:03:28.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.016+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34083120 con 0x7fef340c1160 2023-05-16T07:03:28.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.016+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fef3001a060 con 0x7fef340c1160 2023-05-16T07:03:28.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.017+0000 7fef2a7fc700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef1805ee00 0x7fef180612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.017+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fef300790e0 con 0x7fef340c1160 2023-05-16T07:03:28.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.017+0000 7fef39a9d700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef1805ee00 0x7fef180612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.017+0000 7fef39a9d700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef1805ee00 0x7fef180612c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fef340c9fd0 tx=0x7fef24009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.021+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fef3004a500 con 0x7fef340c1160 2023-05-16T07:03:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.208+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fef34047150 con 0x7fef1805ee00 2023-05-16T07:03:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.215+0000 7fef2a7fc700 1 -- 172.21.15.92:0/3582287030 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fef34047150 con 0x7fef1805ee00 2023-05-16T07:03:28.220 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir6 2023-05-16T07:03:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef1805ee00 msgr2=0x7fef180612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef1805ee00 0x7fef180612c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fef340c9fd0 tx=0x7fef24009040).stop 2023-05-16T07:03:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 msgr2=0x7fef340c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 0x7fef340c2080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fef3000c420 tx=0x7fef30007c80).stop 2023-05-16T07:03:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 shutdown_connections 2023-05-16T07:03:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef1805ee00 0x7fef180612c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fef34153f90 0x7fef34152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef340c1160 0x7fef340c2080 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 --2- 172.21.15.92:0/3582287030 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef340c6410 0x7fef3414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.217+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 >> 172.21.15.92:0/3582287030 conn(0x7fef340cb600 msgr2=0x7fef3414e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.218+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 shutdown_connections 2023-05-16T07:03:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.218+0000 7fef3b2a0700 1 -- 172.21.15.92:0/3582287030 wait complete. 2023-05-16T07:03:28.234 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-05-16T07:03:28.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:28.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-05-16T07:03:28.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-05-16T07:03:28.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.399+0000 7fbc5f59e700 1 Processor -- start 2023-05-16T07:03:28.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.399+0000 7fbc5f59e700 1 -- start start 2023-05-16T07:03:28.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc600c1160 0x7fbc600c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5f59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbc600c61f0 0x7fbc60154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6006a140 con 0x7fbc60153f90 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc60069f60 con 0x7fbc600c1160 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5f59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6006a9e0 con 0x7fbc600c61f0 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc600c1160 0x7fbc600c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbc60153f90 0x7fbc600a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53484/0 (socket says 172.21.15.92:53484) 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.400+0000 7fbc5e59c700 1 -- 172.21.15.92:0/1600619519 learned_addr learned my addr 172.21.15.92:0/1600619519 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:28.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5ed9d700 1 --2- 172.21.15.92:0/1600619519 >> v2:172.21.15.158:3300/0 conn(0x7fbc600c61f0 0x7fbc60154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5dd9b700 1 -- 172.21.15.92:0/1600619519 >> v2:172.21.15.158:3300/0 conn(0x7fbc600c61f0 msgr2=0x7fbc60154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5dd9b700 1 --2- 172.21.15.92:0/1600619519 >> v2:172.21.15.158:3300/0 conn(0x7fbc600c61f0 0x7fbc60154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5dd9b700 1 -- 172.21.15.92:0/1600619519 >> v1:172.21.15.92:6789/0 conn(0x7fbc60153f90 legacy=0x7fbc600a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5d59a700 1 -- 172.21.15.92:0/1600619519 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802530778 0 0) 0x7fbc6006a140 con 0x7fbc60153f90 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5dd9b700 1 -- 172.21.15.92:0/1600619519 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6006a420 con 0x7fbc600c1160 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5ed9d700 1 --2- 172.21.15.92:0/1600619519 >> v2:172.21.15.158:3300/0 conn(0x7fbc600c61f0 0x7fbc60154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:28.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5dd9b700 1 --2- 172.21.15.92:0/1600619519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c5d00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fbc600c1b50 tx=0x7fbc48009060).ready entity=mon.0 client_cookie=816a902ef09add1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5d59a700 1 -- 172.21.15.92:0/1600619519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800b580 con 0x7fbc600c1160 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5d59a700 1 -- 172.21.15.92:0/1600619519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4800b6e0 con 0x7fbc600c1160 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5d59a700 1 -- 172.21.15.92:0/1600619519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800ba60 con 0x7fbc600c1160 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 -- 172.21.15.92:0/1600619519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 msgr2=0x7fbc600c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/1600619519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c5d00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fbc600c1b50 tx=0x7fbc48009060).stop 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 -- 172.21.15.92:0/1600619519 shutdown_connections 2023-05-16T07:03:28.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/1600619519 >> v2:172.21.15.158:3300/0 conn(0x7fbc600c61f0 0x7fbc60154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/1600619519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c5d00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 -- 172.21.15.92:0/1600619519 >> 172.21.15.92:0/1600619519 conn(0x7fbc600cb600 msgr2=0x7fbc600c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:28.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 -- 172.21.15.92:0/1600619519 shutdown_connections 2023-05-16T07:03:28.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.401+0000 7fbc5f59e700 1 -- 172.21.15.92:0/1600619519 wait complete. 2023-05-16T07:03:28.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.402+0000 7fbc5f59e700 1 Processor -- start 2023-05-16T07:03:28.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 -- start start 2023-05-16T07:03:28.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc600c61f0 0x7fbc600b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc60153f90 0x7fbc60152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600c4690 con 0x7fbc600c1160 2023-05-16T07:03:28.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60164ea0 con 0x7fbc600c61f0 2023-05-16T07:03:28.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600c3320 con 0x7fbc60153f90 2023-05-16T07:03:28.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42742/0 (socket says 172.21.15.92:42742) 2023-05-16T07:03:28.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5e59c700 1 -- 172.21.15.92:0/115954604 learned_addr learned my addr 172.21.15.92:0/115954604 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:28.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.403+0000 7fbc5dd9b700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc600c61f0 0x7fbc600b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5ed9d700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc60153f90 0x7fbc60152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5e59c700 1 -- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc60153f90 msgr2=0x7fbc60152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc60153f90 0x7fbc60152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5e59c700 1 -- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc600c61f0 msgr2=0x7fbc600b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc600c61f0 0x7fbc600b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5e59c700 1 -- 172.21.15.92:0/115954604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480092a0 con 0x7fbc600c1160 2023-05-16T07:03:28.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5dd9b700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc600c61f0 0x7fbc600b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:28.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c37b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fbc5000c960 tx=0x7fbc50001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50005d30 con 0x7fbc600c1160 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc50005e90 con 0x7fbc600c1160 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600c3ca0 con 0x7fbc600c1160 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50007320 con 0x7fbc600c1160 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.404+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc600c2d50 con 0x7fbc600c1160 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.405+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc60083120 con 0x7fbc600c1160 2023-05-16T07:03:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.405+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fbc50010030 con 0x7fbc600c1160 2023-05-16T07:03:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.405+0000 7fbc567fc700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc4005ee60 0x7fbc40061320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.405+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fbc50079db0 con 0x7fbc600c1160 2023-05-16T07:03:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.406+0000 7fbc5dd9b700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc4005ee60 0x7fbc40061320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.406+0000 7fbc5dd9b700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc4005ee60 0x7fbc40061320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fbc600c15f0 tx=0x7fbc48017000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.409+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbc5004b1d0 con 0x7fbc600c1160 2023-05-16T07:03:28.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.599+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc60047150 con 0x7fbc4005ee60 2023-05-16T07:03:28.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.604+0000 7fbc567fc700 1 -- 172.21.15.92:0/115954604 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbc60047150 con 0x7fbc4005ee60 2023-05-16T07:03:28.609 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir7 2023-05-16T07:03:28.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc4005ee60 msgr2=0x7fbc40061320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc4005ee60 0x7fbc40061320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fbc600c15f0 tx=0x7fbc48017000).stop 2023-05-16T07:03:28.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 msgr2=0x7fbc600c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c37b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fbc5000c960 tx=0x7fbc50001630).stop 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 shutdown_connections 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc4005ee60 0x7fbc40061320 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc60153f90 0x7fbc60152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc600c1160 0x7fbc600c37b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 --2- 172.21.15.92:0/115954604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc600c61f0 0x7fbc600b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 >> 172.21.15.92:0/115954604 conn(0x7fbc600cb600 msgr2=0x7fbc60151390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 shutdown_connections 2023-05-16T07:03:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.606+0000 7fbc5f59e700 1 -- 172.21.15.92:0/115954604 wait complete. 2023-05-16T07:03:28.622 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-05-16T07:03:28.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:28.623 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-05-16T07:03:28.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-05-16T07:03:28.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.789+0000 7fa656986700 1 Processor -- start 2023-05-16T07:03:28.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.790+0000 7fa656986700 1 -- start start 2023-05-16T07:03:28.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.790+0000 7fa656986700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 0x7fa650155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.790+0000 7fa656986700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65014ead0 con 0x7fa6500c4d40 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.790+0000 7fa656986700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa650154cb0 con 0x7fa6500629c0 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.790+0000 7fa656986700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500c6020 con 0x7fa65014e6f0 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa6500c4d40 0x7fa6500622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53500/0 (socket says 172.21.15.92:53500) 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64f7fe700 1 -- 172.21.15.92:0/2409435245 learned_addr learned my addr 172.21.15.92:0/2409435245 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64ffff700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 0x7fa650155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896646525 0 0) 0x7fa65014ead0 con 0x7fa6500c4d40 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6380033f0 con 0x7fa6500c4d40 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082268249 0 0) 0x7fa6380033f0 con 0x7fa6500c4d40 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 msgr2=0x7fa650155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 0x7fa650155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 >> v1:172.21.15.158:6789/0 conn(0x7fa6500629c0 legacy=0x7fa650063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650154660 con 0x7fa6500c4d40 2023-05-16T07:03:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fa644001470 con 0x7fa6500c4d40 2023-05-16T07:03:28.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 >> v1:172.21.15.92:6789/0 conn(0x7fa6500c4d40 legacy=0x7fa6500622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa638003910 0x7fa638003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa638004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 0x7fa638005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500c6020 con 0x7fa638004690 2023-05-16T07:03:28.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63800b8e0 con 0x7fa638003910 2023-05-16T07:03:28.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63800ba50 con 0x7fa65014e6f0 2023-05-16T07:03:28.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa638004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa64ffff700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa638003910 0x7fa638003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.791+0000 7fa654f23700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 0x7fa638005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64f7fe700 1 -- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 msgr2=0x7fa638005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65014e6f0 0x7fa638005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64f7fe700 1 -- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa638003910 msgr2=0x7fa638003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa638003910 0x7fa638003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64f7fe700 1 -- 172.21.15.92:0/2409435245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6380033f0 con 0x7fa638004690 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64ffff700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa638003910 0x7fa638003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:28.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa638004b10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fa64400c160 tx=0x7fa644001e70).ready entity=mon.0 client_cookie=ad1a394c74e9745d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644007a40 con 0x7fa638004690 2023-05-16T07:03:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64400ca30 con 0x7fa638004690 2023-05-16T07:03:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa64effd700 1 -- 172.21.15.92:0/2409435245 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400cdb0 con 0x7fa638004690 2023-05-16T07:03:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa656986700 1 -- 172.21.15.92:0/2409435245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 msgr2=0x7fa638004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa656986700 1 --2- 172.21.15.92:0/2409435245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa638004b10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fa64400c160 tx=0x7fa644001e70).stop 2023-05-16T07:03:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa654f23700 1 -- 172.21.15.92:0/2409435245 reap_dead start 2023-05-16T07:03:28.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa656986700 1 -- 172.21.15.92:0/2409435245 shutdown_connections 2023-05-16T07:03:28.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa656986700 1 -- 172.21.15.92:0/2409435245 >> 172.21.15.92:0/2409435245 conn(0x7fa65005e820 msgr2=0x7fa6500c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:28.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa656986700 1 -- 172.21.15.92:0/2409435245 shutdown_connections 2023-05-16T07:03:28.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.792+0000 7fa656986700 1 -- 172.21.15.92:0/2409435245 wait complete. 2023-05-16T07:03:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.793+0000 7fa656986700 1 Processor -- start 2023-05-16T07:03:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 -- start start 2023-05-16T07:03:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa65014c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa6501649c0 0x7fa6500aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa650164e40 0x7fa6500ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500ab5a0 con 0x7fa638004690 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500ae2d0 con 0x7fa650164e40 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa656986700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500ad230 con 0x7fa6501649c0 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa65014c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa65014c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42762/0 (socket says 172.21.15.92:42762) 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.794+0000 7fa64ffff700 1 -- 172.21.15.92:0/2255374524 learned_addr learned my addr 172.21.15.92:0/2255374524 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa6501649c0 0x7fa6500aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa654f23700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa650164e40 0x7fa6500ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64ffff700 1 -- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa6501649c0 msgr2=0x7fa6500aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64ffff700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa6501649c0 0x7fa6500aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64ffff700 1 -- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa650164e40 msgr2=0x7fa6500ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64ffff700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa650164e40 0x7fa6500ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64ffff700 1 -- 172.21.15.92:0/2255374524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500ad510 con 0x7fa638004690 2023-05-16T07:03:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa64ffff700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa65014c7f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fa640011960 tx=0x7fa640008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640013620 con 0x7fa638004690 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640013780 con 0x7fa638004690 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500ad7f0 con 0x7fa638004690 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640013b00 con 0x7fa638004690 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.795+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500addb0 con 0x7fa638004690 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.796+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6500ae120 con 0x7fa638004690 2023-05-16T07:03:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.796+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fa640012040 con 0x7fa638004690 2023-05-16T07:03:28.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.797+0000 7fa637fff700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ee30 0x7fa63c0612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:28.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.797+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa64007a3e0 con 0x7fa638004690 2023-05-16T07:03:28.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.797+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ee30 0x7fa63c0612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:28.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.797+0000 7fa64f7fe700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ee30 0x7fa63c0612f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fa6500aaac0 tx=0x7fa644015000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:28.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.801+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa64004b800 con 0x7fa638004690 2023-05-16T07:03:28.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.986+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa650064830 con 0x7fa63c05ee30 2023-05-16T07:03:28.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.990+0000 7fa637fff700 1 -- 172.21.15.92:0/2255374524 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa650064830 con 0x7fa63c05ee30 2023-05-16T07:03:28.995 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir8 2023-05-16T07:03:28.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ee30 msgr2=0x7fa63c0612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ee30 0x7fa63c0612f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fa6500aaac0 tx=0x7fa644015000).stop 2023-05-16T07:03:28.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 msgr2=0x7fa65014c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:28.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa65014c7f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fa640011960 tx=0x7fa640008040).stop 2023-05-16T07:03:28.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 shutdown_connections 2023-05-16T07:03:28.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ee30 0x7fa63c0612f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa650164e40 0x7fa6500ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa638004690 0x7fa65014c7f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 --2- 172.21.15.92:0/2255374524 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa6501649c0 0x7fa6500aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:28.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 >> 172.21.15.92:0/2255374524 conn(0x7fa65005e820 msgr2=0x7fa6500c51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:28.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 shutdown_connections 2023-05-16T07:03:28.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:28.992+0000 7fa656986700 1 -- 172.21.15.92:0/2255374524 wait complete. 2023-05-16T07:03:29.009 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-05-16T07:03:29.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:03:29.011 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-05-16T07:03:29.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-05-16T07:03:29.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.176+0000 7f124880e700 1 Processor -- start 2023-05-16T07:03:29.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.176+0000 7f124880e700 1 -- start start 2023-05-16T07:03:29.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.177+0000 7f124880e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f124414f990 0x7f12440c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.177+0000 7f124880e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1244152e40 0x7f12440c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.177+0000 7f124880e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1244063690 con 0x7f12440622b0 2023-05-16T07:03:29.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.177+0000 7f124880e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124414fd70 con 0x7f1244152e40 2023-05-16T07:03:29.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.177+0000 7f124880e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c52f0 con 0x7f124414f990 2023-05-16T07:03:29.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1244152e40 0x7f12440c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f124259c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f124414f990 0x7f12440c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1242d9d700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f12440622b0 0x7f1244063b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:57120/0 (socket says 172.21.15.92:57120) 2023-05-16T07:03:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1242d9d700 1 -- 172.21.15.92:0/780816796 learned_addr learned my addr 172.21.15.92:0/780816796 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 --2- 172.21.15.92:0/780816796 >> v2:172.21.15.158:3300/0 conn(0x7f1244152e40 0x7f12440c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:46410/0 (socket says 172.21.15.92:46410) 2023-05-16T07:03:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 -- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f124414f990 msgr2=0x7f12440c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 --2- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f124414f990 0x7f12440c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 -- 172.21.15.92:0/780816796 >> v1:172.21.15.158:6789/0 conn(0x7f12440622b0 legacy=0x7f1244063b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 -- 172.21.15.92:0/780816796 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440c5ce0 con 0x7f1244152e40 2023-05-16T07:03:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f124159a700 1 -- 172.21.15.92:0/780816796 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679569504 0 0) 0x7f1244063690 con 0x7f12440622b0 2023-05-16T07:03:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f124259c700 1 --2- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f124414f990 0x7f12440c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f1241d9b700 1 --2- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12440c3430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f122c009760 tx=0x7f122c009060).ready entity=mon.1 client_cookie=413f762dd5d2bb2d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f124159a700 1 -- 172.21.15.92:0/780816796 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00f510 con 0x7f1244152e40 2023-05-16T07:03:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f124159a700 1 -- 172.21.15.92:0/780816796 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c00f670 con 0x7f1244152e40 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.178+0000 7f124159a700 1 -- 172.21.15.92:0/780816796 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00f9f0 con 0x7f1244152e40 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 -- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 msgr2=0x7f12440c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 --2- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12440c3430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f122c009760 tx=0x7f122c009060).stop 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 -- 172.21.15.92:0/780816796 shutdown_connections 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 --2- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f124414f990 0x7f12440c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 --2- 172.21.15.92:0/780816796 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12440c3430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 -- 172.21.15.92:0/780816796 >> 172.21.15.92:0/780816796 conn(0x7f124405e820 msgr2=0x7f12440c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:29.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 -- 172.21.15.92:0/780816796 shutdown_connections 2023-05-16T07:03:29.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.179+0000 7f124880e700 1 -- 172.21.15.92:0/780816796 wait complete. 2023-05-16T07:03:29.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.180+0000 7f124880e700 1 Processor -- start 2023-05-16T07:03:29.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.180+0000 7f124880e700 1 -- start start 2023-05-16T07:03:29.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12440622b0 0x7f1244165560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 0x7f12441503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12441508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1244150db0 con 0x7f124414f990 2023-05-16T07:03:29.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c28f0 con 0x7f1244152e40 2023-05-16T07:03:29.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c1a10 con 0x7f12440622b0 2023-05-16T07:03:29.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 0x7f12441503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 0x7f12441503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42778/0 (socket says 172.21.15.92:42778) 2023-05-16T07:03:29.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 -- 172.21.15.92:0/345358986 learned_addr learned my addr 172.21.15.92:0/345358986 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:29.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124259c700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12440622b0 0x7f1244165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1242d9d700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12441508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 -- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12440622b0 msgr2=0x7f1244165560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12440622b0 0x7f1244165560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 -- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 msgr2=0x7f12441508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12441508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 -- 172.21.15.92:0/345358986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1234009a20 con 0x7f124414f990 2023-05-16T07:03:29.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1242d9d700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12441508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:29.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f1241d9b700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 0x7f12441503d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1244066c40 tx=0x7f122c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00fe80 con 0x7f124414f990 2023-05-16T07:03:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c00f7e0 con 0x7f124414f990 2023-05-16T07:03:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.181+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c0092a0 con 0x7f124414f990 2023-05-16T07:03:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.182+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c022620 con 0x7f124414f990 2023-05-16T07:03:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.182+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12440c2040 con 0x7f124414f990 2023-05-16T07:03:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.183+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f122c00f070 con 0x7f124414f990 2023-05-16T07:03:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.183+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12440c2a80 con 0x7f124414f990 2023-05-16T07:03:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.183+0000 7f123a7fc700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f122405ee30 0x7f12240612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.183+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f122c07b130 con 0x7f124414f990 2023-05-16T07:03:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.183+0000 7f124259c700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f122405ee30 0x7f12240612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.184+0000 7f124259c700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f122405ee30 0x7f12240612f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1234009fd0 tx=0x7f123400b040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.187+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f122c04c550 con 0x7f124414f990 2023-05-16T07:03:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.374+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f124406c350 con 0x7f122405ee30 2023-05-16T07:03:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.379+0000 7f123a7fc700 1 -- 172.21.15.92:0/345358986 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f124406c350 con 0x7f122405ee30 2023-05-16T07:03:29.384 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/dir9 2023-05-16T07:03:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f122405ee30 msgr2=0x7f12240612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f122405ee30 0x7f12240612f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1234009fd0 tx=0x7f123400b040).stop 2023-05-16T07:03:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 msgr2=0x7f12441503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 0x7f12441503d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1244066c40 tx=0x7f122c017000).stop 2023-05-16T07:03:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 shutdown_connections 2023-05-16T07:03:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f122405ee30 0x7f12240612f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12440622b0 0x7f1244165560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1244152e40 0x7f12441508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 --2- 172.21.15.92:0/345358986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f124414f990 0x7f12441503d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.380+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 >> 172.21.15.92:0/345358986 conn(0x7f124405e820 msgr2=0x7f12440c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.381+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 shutdown_connections 2023-05-16T07:03:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.381+0000 7f124880e700 1 -- 172.21.15.92:0/345358986 wait complete. 2023-05-16T07:03:29.397 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-05-16T07:03:29.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-05-16T07:03:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.509+0000 7f259513a700 1 Processor -- start 2023-05-16T07:03:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.509+0000 7f259513a700 1 -- start start 2023-05-16T07:03:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f259513a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2590154090 0x7f25900aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f259513a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2590068160 0x7f25900685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f259513a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25900c94d0 con 0x7f2590154470 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f259513a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25900cb7e0 con 0x7f2590154090 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f259513a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2590068aa0 con 0x7f2590068160 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2590154090 0x7f25900aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2590154470 0x7f25900a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53532/0 (socket says 172.21.15.92:53532) 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2590154090 0x7f25900aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42784/0 (socket says 172.21.15.92:42784) 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258e59c700 1 -- 172.21.15.92:0/946413782 learned_addr learned my addr 172.21.15.92:0/946413782 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 -- 172.21.15.92:0/946413782 >> v2:172.21.15.158:3300/0 conn(0x7f2590068160 msgr2=0x7f25900685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258f59e700 1 --2- 172.21.15.92:0/946413782 >> v2:172.21.15.158:3300/0 conn(0x7f2590068160 0x7f25900685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258dd9b700 1 -- 172.21.15.92:0/946413782 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755474169 0 0) 0x7f25900c94d0 con 0x7f2590154470 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 --2- 172.21.15.92:0/946413782 >> v2:172.21.15.158:3300/0 conn(0x7f2590068160 0x7f25900685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 -- 172.21.15.92:0/946413782 >> v1:172.21.15.92:6789/0 conn(0x7f2590154470 legacy=0x7f25900a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 -- 172.21.15.92:0/946413782 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900c88c0 con 0x7f2590154090 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258f59e700 1 --2- 172.21.15.92:0/946413782 >> v2:172.21.15.158:3300/0 conn(0x7f2590068160 0x7f25900685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258ed9d700 1 --2- 172.21.15.92:0/946413782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25900aaca0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f259005f9e0 tx=0x7f2580009780).ready entity=mon.0 client_cookie=6942076636e109b2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258dd9b700 1 -- 172.21.15.92:0/946413782 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000e070 con 0x7f2590154090 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258dd9b700 1 -- 172.21.15.92:0/946413782 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258000bc30 con 0x7f2590154090 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.510+0000 7f258dd9b700 1 -- 172.21.15.92:0/946413782 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580012410 con 0x7f2590154090 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 -- 172.21.15.92:0/946413782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 msgr2=0x7f25900aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 --2- 172.21.15.92:0/946413782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25900aaca0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f259005f9e0 tx=0x7f2580009780).stop 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 -- 172.21.15.92:0/946413782 shutdown_connections 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 --2- 172.21.15.92:0/946413782 >> v2:172.21.15.158:3300/0 conn(0x7f2590068160 0x7f25900685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 --2- 172.21.15.92:0/946413782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25900aaca0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 -- 172.21.15.92:0/946413782 >> 172.21.15.92:0/946413782 conn(0x7f2590067920 msgr2=0x7f25900cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:29.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 -- 172.21.15.92:0/946413782 shutdown_connections 2023-05-16T07:03:29.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.511+0000 7f259513a700 1 -- 172.21.15.92:0/946413782 wait complete. 2023-05-16T07:03:29.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.512+0000 7f259513a700 1 Processor -- start 2023-05-16T07:03:29.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.512+0000 7f259513a700 1 -- start start 2023-05-16T07:03:29.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f259513a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2590068160 0x7f259014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f259513a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25901505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f259513a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2590154470 0x7f259015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f259513a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259015d4c0 con 0x7f2590154090 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f259513a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259014fee0 con 0x7f2590154470 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f259513a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259015c8b0 con 0x7f2590068160 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25901505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25901505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42792/0 (socket says 172.21.15.92:42792) 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 -- 172.21.15.92:0/3192537783 learned_addr learned my addr 172.21.15.92:0/3192537783 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258ed9d700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2590068160 0x7f259014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258f59e700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2590154470 0x7f259015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 -- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2590068160 msgr2=0x7f259014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2590068160 0x7f259014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 -- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2590154470 msgr2=0x7f259015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2590154470 0x7f259015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.513+0000 7f258e59c700 1 -- 172.21.15.92:0/3192537783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580009a20 con 0x7f2590154090 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f258e59c700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25901505a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f2578001eb0 tx=0x7f2578007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780072e0 con 0x7f2590154090 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578007440 con 0x7f2590154090 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259015cb90 con 0x7f2590154090 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780057a0 con 0x7f2590154090 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259015d150 con 0x7f2590154090 2023-05-16T07:03:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.514+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259015d670 con 0x7f2590154090 2023-05-16T07:03:29.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.515+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f257801a060 con 0x7f2590154090 2023-05-16T07:03:29.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.516+0000 7f2586ffd700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f257005ee30 0x7f25700612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.516+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f25780799a0 con 0x7f2590154090 2023-05-16T07:03:29.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.516+0000 7f258ed9d700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f257005ee30 0x7f25700612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.516+0000 7f258ed9d700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f257005ee30 0x7f25700612f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2580009f90 tx=0x7f2580009780).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.519+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f257804ae40 con 0x7f2590154090 2023-05-16T07:03:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.706+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f259006c350 con 0x7f257005ee30 2023-05-16T07:03:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.713+0000 7f2586ffd700 1 -- 172.21.15.92:0/3192537783 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f259006c350 con 0x7f257005ee30 2023-05-16T07:03:29.718 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-05-16T07:03:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.715+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f257005ee30 msgr2=0x7f25700612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.715+0000 7f259513a700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f257005ee30 0x7f25700612f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2580009f90 tx=0x7f2580009780).stop 2023-05-16T07:03:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.715+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 msgr2=0x7f25901505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.715+0000 7f259513a700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25901505a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f2578001eb0 tx=0x7f2578007e90).stop 2023-05-16T07:03:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 shutdown_connections 2023-05-16T07:03:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f257005ee30 0x7f25700612f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2590068160 0x7f259014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2590154470 0x7f259015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 --2- 172.21.15.92:0/3192537783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2590154090 0x7f25901505a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 >> 172.21.15.92:0/3192537783 conn(0x7f2590067920 msgr2=0x7f25900ac0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 shutdown_connections 2023-05-16T07:03:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.716+0000 7f259513a700 1 -- 172.21.15.92:0/3192537783 wait complete. 2023-05-16T07:03:29.732 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-05-16T07:03:29.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-05-16T07:03:29.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.842+0000 7fb27bf26700 1 Processor -- start 2023-05-16T07:03:29.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.842+0000 7fb27bf26700 1 -- start start 2023-05-16T07:03:29.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb27bf26700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 0x7fb2740aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb27bf26700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740c94d0 con 0x7fb274068160 2023-05-16T07:03:29.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb27bf26700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740cb7e0 con 0x7fb274154470 2023-05-16T07:03:29.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb27bf26700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740c88c0 con 0x7fb274154090 2023-05-16T07:03:29.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb27a4c3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb274068160 0x7fb274068580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53550/0 (socket says 172.21.15.92:53550) 2023-05-16T07:03:29.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb27a4c3700 1 -- 172.21.15.92:0/3505227653 learned_addr learned my addr 172.21.15.92:0/3505227653 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:29.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb279cc2700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 0x7fb2740aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664294056 0 0) 0x7fb2740c94d0 con 0x7fb274068160 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2600033f0 con 0x7fb274068160 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3449495601 0 0) 0x7fb2600033f0 con 0x7fb274068160 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 msgr2=0x7fb2740aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 0x7fb2740aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 >> v1:172.21.15.158:6789/0 conn(0x7fb274154470 legacy=0x7fb2740a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740c8ba0 con 0x7fb274068160 2023-05-16T07:03:29.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fb270001470 con 0x7fb274068160 2023-05-16T07:03:29.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 >> v1:172.21.15.92:6789/0 conn(0x7fb274068160 legacy=0x7fb274068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 0x7fb260003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260003e40 0x7fb260004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb260004b80 0x7fb260005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740c88c0 con 0x7fb260004b80 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26000b990 con 0x7fb260003e40 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.843+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26000bb50 con 0x7fb274154090 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb260004b80 0x7fb260005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb279cc2700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 0x7fb260003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb2794c1700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260003e40 0x7fb260004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 -- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 msgr2=0x7fb260003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb274154090 0x7fb260003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 -- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260003e40 msgr2=0x7fb260004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:29.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260003e40 0x7fb260004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 -- 172.21.15.92:0/3505227653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600033f0 con 0x7fb260004b80 2023-05-16T07:03:29.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb260004b80 0x7fb260005000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fb27000c190 tx=0x7fb270001e50).ready entity=mon.0 client_cookie=893b29f8f198b0e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270007a20 con 0x7fb260004b80 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27000cac0 con 0x7fb260004b80 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb278cc0700 1 -- 172.21.15.92:0/3505227653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000ce40 con 0x7fb260004b80 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27bf26700 1 -- 172.21.15.92:0/3505227653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb260004b80 msgr2=0x7fb260005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27bf26700 1 --2- 172.21.15.92:0/3505227653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb260004b80 0x7fb260005000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fb27000c190 tx=0x7fb270001e50).stop 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.844+0000 7fb27a4c3700 1 -- 172.21.15.92:0/3505227653 reap_dead start 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.845+0000 7fb27bf26700 1 -- 172.21.15.92:0/3505227653 shutdown_connections 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.845+0000 7fb27bf26700 1 -- 172.21.15.92:0/3505227653 >> 172.21.15.92:0/3505227653 conn(0x7fb274067920 msgr2=0x7fb2740cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:29.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.845+0000 7fb27bf26700 1 -- 172.21.15.92:0/3505227653 shutdown_connections 2023-05-16T07:03:29.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.845+0000 7fb27bf26700 1 -- 172.21.15.92:0/3505227653 wait complete. 2023-05-16T07:03:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 Processor -- start 2023-05-16T07:03:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 -- start start 2023-05-16T07:03:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260004b80 0x7fb2740af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2740afec0 0x7fb2740b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 0x7fb274164910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740ae890 con 0x7fb2741644b0 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740b0810 con 0x7fb260004b80 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.846+0000 7fb27bf26700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740adc80 con 0x7fb2740afec0 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 0x7fb274164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 0x7fb274164910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42814/0 (socket says 172.21.15.92:42814) 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 -- 172.21.15.92:0/2339863482 learned_addr learned my addr 172.21.15.92:0/2339863482 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb279cc2700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260004b80 0x7fb2740af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb2794c1700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2740afec0 0x7fb2740b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 -- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2740afec0 msgr2=0x7fb2740b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2740afec0 0x7fb2740b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 -- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260004b80 msgr2=0x7fb2740af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260004b80 0x7fb2740af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 -- 172.21.15.92:0/2339863482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740adf60 con 0x7fb2741644b0 2023-05-16T07:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb2794c1700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2740afec0 0x7fb2740b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27a4c3700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 0x7fb274164910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fb27000a000 tx=0x7fb270015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270001470 con 0x7fb2741644b0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb270000e40 con 0x7fb2741644b0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2740ae240 con 0x7fb2741644b0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270004ad0 con 0x7fb2741644b0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.847+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2740afbd0 con 0x7fb2741644b0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.848+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb274066730 con 0x7fb2741644b0 2023-05-16T07:03:29.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.849+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fb270010070 con 0x7fb2741644b0 2023-05-16T07:03:29.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.849+0000 7fb269ffb700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb25805ee30 0x7fb2580612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:29.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.849+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb27007a9f0 con 0x7fb2741644b0 2023-05-16T07:03:29.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.849+0000 7fb279cc2700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb25805ee30 0x7fb2580612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:29.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.850+0000 7fb279cc2700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb25805ee30 0x7fb2580612f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb2740af3c0 tx=0x7fb26c00b040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:29.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:29.853+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb27007ce60 con 0x7fb2741644b0 2023-05-16T07:03:30.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.041+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb27406c350 con 0x7fb25805ee30 2023-05-16T07:03:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.045+0000 7fb269ffb700 1 -- 172.21.15.92:0/2339863482 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb27406c350 con 0x7fb25805ee30 2023-05-16T07:03:30.051 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-05-16T07:03:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.047+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb25805ee30 msgr2=0x7fb2580612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.047+0000 7fb27bf26700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb25805ee30 0x7fb2580612f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb2740af3c0 tx=0x7fb26c00b040).stop 2023-05-16T07:03:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.047+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 msgr2=0x7fb274164910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.047+0000 7fb27bf26700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 0x7fb274164910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fb27000a000 tx=0x7fb270015000).stop 2023-05-16T07:03:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 shutdown_connections 2023-05-16T07:03:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb25805ee30 0x7fb2580612f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2741644b0 0x7fb274164910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb260004b80 0x7fb2740af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 --2- 172.21.15.92:0/2339863482 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2740afec0 0x7fb2740b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 >> 172.21.15.92:0/2339863482 conn(0x7fb274067920 msgr2=0x7fb2740c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 shutdown_connections 2023-05-16T07:03:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.048+0000 7fb27bf26700 1 -- 172.21.15.92:0/2339863482 wait complete. 2023-05-16T07:03:30.064 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-05-16T07:03:30.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-05-16T07:03:30.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.175+0000 7fbd0e7de700 1 Processor -- start 2023-05-16T07:03:30.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.175+0000 7fbd0e7de700 1 -- start start 2023-05-16T07:03:30.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd0e7de700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 0x7fbd080aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd0e7de700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd080c94d0 con 0x7fbd08154470 2023-05-16T07:03:30.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd0e7de700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd080cb7e0 con 0x7fbd08068160 2023-05-16T07:03:30.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd0e7de700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080c88c0 con 0x7fbd08154090 2023-05-16T07:03:30.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbd08154470 0x7fbd080a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53566/0 (socket says 172.21.15.92:53566) 2023-05-16T07:03:30.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd077fe700 1 -- 172.21.15.92:0/694196349 learned_addr learned my addr 172.21.15.92:0/694196349 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd07fff700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 0x7fbd080aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780986052 0 0) 0x7fbd080c94d0 con 0x7fbd08154470 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcf00033f0 con 0x7fbd08154470 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4200902874 0 0) 0x7fbcf00033f0 con 0x7fbd08154470 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 msgr2=0x7fbd080aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 0x7fbd080aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 >> v1:172.21.15.158:6789/0 conn(0x7fbd08068160 legacy=0x7fbd08068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd080c8ba0 con 0x7fbd08154470 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fbcfc001470 con 0x7fbd08154470 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 >> v1:172.21.15.92:6789/0 conn(0x7fbd08154470 legacy=0x7fbd080a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 0x7fbcf0003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcf0003e40 0x7fbcf0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf0004b80 0x7fbcf0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080c88c0 con 0x7fbcf0004b80 2023-05-16T07:03:30.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf000b990 con 0x7fbcf0003e40 2023-05-16T07:03:30.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf000bb50 con 0x7fbd08154090 2023-05-16T07:03:30.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.176+0000 7fbd0cd7b700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf0004b80 0x7fbcf0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd07fff700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 0x7fbcf0003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd077fe700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcf0003e40 0x7fbcf0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 -- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 msgr2=0x7fbcf0003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08154090 0x7fbcf0003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 -- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcf0003e40 msgr2=0x7fbcf0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbcf0003e40 0x7fbcf0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 -- 172.21.15.92:0/694196349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00033f0 con 0x7fbcf0004b80 2023-05-16T07:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf0004b80 0x7fbcf0005000 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fbd00001e30 tx=0x7fbd00007f30).ready entity=mon.0 client_cookie=5075f5031e7ab4ff server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00006a20 con 0x7fbcf0004b80 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00006b80 con 0x7fbcf0004b80 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd06ffd700 1 -- 172.21.15.92:0/694196349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000c5d0 con 0x7fbcf0004b80 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0e7de700 1 -- 172.21.15.92:0/694196349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf0004b80 msgr2=0x7fbcf0005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/694196349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf0004b80 0x7fbcf0005000 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fbd00001e30 tx=0x7fbd00007f30).stop 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0cd7b700 1 -- 172.21.15.92:0/694196349 reap_dead start 2023-05-16T07:03:30.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0e7de700 1 -- 172.21.15.92:0/694196349 shutdown_connections 2023-05-16T07:03:30.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.177+0000 7fbd0e7de700 1 -- 172.21.15.92:0/694196349 >> 172.21.15.92:0/694196349 conn(0x7fbd08067920 msgr2=0x7fbd080cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:30.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.178+0000 7fbd0e7de700 1 -- 172.21.15.92:0/694196349 shutdown_connections 2023-05-16T07:03:30.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.178+0000 7fbd0e7de700 1 -- 172.21.15.92:0/694196349 wait complete. 2023-05-16T07:03:30.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 Processor -- start 2023-05-16T07:03:30.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 -- start start 2023-05-16T07:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 0x7fbd080c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd080c4710 0x7fbd081569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08156ea0 0x7fbd080c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080c3320 con 0x7fbd080c1cd0 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08156520 con 0x7fbd080c4710 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.179+0000 7fbd0e7de700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080c4f80 con 0x7fbd08156ea0 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 0x7fbd080c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 0x7fbd080c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42830/0 (socket says 172.21.15.92:42830) 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 -- 172.21.15.92:0/948221607 learned_addr learned my addr 172.21.15.92:0/948221607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd0cd7b700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08156ea0 0x7fbd080c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd077fe700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd080c4710 0x7fbd081569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 -- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08156ea0 msgr2=0x7fbd080c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08156ea0 0x7fbd080c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 -- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd080c4710 msgr2=0x7fbd081569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd080c4710 0x7fbd081569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 -- 172.21.15.92:0/948221607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd08155bf0 con 0x7fbd080c1cd0 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd0cd7b700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08156ea0 0x7fbd080c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd07fff700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 0x7fbd080c2e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fbcf80119f0 tx=0x7fbcf8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800f8e0 con 0x7fbd080c1cd0 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf800fa40 con 0x7fbd080c1cd0 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800fe00 con 0x7fbd080c1cd0 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd08155ed0 con 0x7fbd080c1cd0 2023-05-16T07:03:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.180+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0814c410 con 0x7fbd080c1cd0 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.181+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd08156800 con 0x7fbd080c1cd0 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.181+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fbcf8024070 con 0x7fbd080c1cd0 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.182+0000 7fbceffff700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbcf405ee30 0x7fbcf40612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.182+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fbcf807a890 con 0x7fbd080c1cd0 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.182+0000 7fbd077fe700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbcf405ee30 0x7fbcf40612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.182+0000 7fbd077fe700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbcf405ee30 0x7fbcf40612f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fbcfc000e10 tx=0x7fbcfc009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.186+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbcf804bcb0 con 0x7fbd080c1cd0 2023-05-16T07:03:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.372+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd080ca170 con 0x7fbcf405ee30 2023-05-16T07:03:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.376+0000 7fbceffff700 1 -- 172.21.15.92:0/948221607 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbd080ca170 con 0x7fbcf405ee30 2023-05-16T07:03:30.381 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-05-16T07:03:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbcf405ee30 msgr2=0x7fbcf40612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbcf405ee30 0x7fbcf40612f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fbcfc000e10 tx=0x7fbcfc009040).stop 2023-05-16T07:03:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 msgr2=0x7fbd080c2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 0x7fbd080c2e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fbcf80119f0 tx=0x7fbcf8009060).stop 2023-05-16T07:03:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 shutdown_connections 2023-05-16T07:03:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbcf405ee30 0x7fbcf40612f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbd08156ea0 0x7fbd080c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd080c1cd0 0x7fbd080c2e30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 --2- 172.21.15.92:0/948221607 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd080c4710 0x7fbd081569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 >> 172.21.15.92:0/948221607 conn(0x7fbd08067920 msgr2=0x7fbd0814d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 shutdown_connections 2023-05-16T07:03:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.378+0000 7fbd0e7de700 1 -- 172.21.15.92:0/948221607 wait complete. 2023-05-16T07:03:30.394 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-05-16T07:03:30.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-05-16T07:03:30.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.506+0000 7f42dddc6700 1 Processor -- start 2023-05-16T07:03:30.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.506+0000 7f42dddc6700 1 -- start start 2023-05-16T07:03:30.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42dddc6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42d8150bb0 0x7f42d8067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42dddc6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f42d8150f90 0x7f42d8068450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42dddc6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d8064e90 con 0x7f42d80a6780 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42dddc6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d8068940 con 0x7f42d8150bb0 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42dddc6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d8065a10 con 0x7f42d8150f90 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42d8150bb0 0x7f42d8067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42d77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42d8150bb0 0x7f42d8067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42838/0 (socket says 172.21.15.92:42838) 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.507+0000 7f42d77fe700 1 -- 172.21.15.92:0/56878341 learned_addr learned my addr 172.21.15.92:0/56878341 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:30.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d6ffd700 1 --2- 172.21.15.92:0/56878341 >> v2:172.21.15.158:3300/0 conn(0x7f42d8150f90 0x7f42d8068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d77fe700 1 -- 172.21.15.92:0/56878341 >> v2:172.21.15.158:3300/0 conn(0x7f42d8150f90 msgr2=0x7f42d8068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d77fe700 1 --2- 172.21.15.92:0/56878341 >> v2:172.21.15.158:3300/0 conn(0x7f42d8150f90 0x7f42d8068450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d77fe700 1 -- 172.21.15.92:0/56878341 >> v1:172.21.15.158:6789/0 conn(0x7f42d80a6780 legacy=0x7f42d8154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d77fe700 1 -- 172.21.15.92:0/56878341 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80635b0 con 0x7f42d8150bb0 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d77fe700 1 --2- 172.21.15.92:0/56878341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150bb0 0x7f42d8067f60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f42d8066430 tx=0x7f42c8009840).ready entity=mon.0 client_cookie=df384b5fd4db5f35 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d67fc700 1 -- 172.21.15.92:0/56878341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c800e070 con 0x7f42d8150bb0 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d67fc700 1 -- 172.21.15.92:0/56878341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c800ba10 con 0x7f42d8150bb0 2023-05-16T07:03:30.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42d67fc700 1 -- 172.21.15.92:0/56878341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c800bd90 con 0x7f42d8150bb0 2023-05-16T07:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 -- 172.21.15.92:0/56878341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150bb0 msgr2=0x7f42d8067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 --2- 172.21.15.92:0/56878341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150bb0 0x7f42d8067f60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f42d8066430 tx=0x7f42c8009840).stop 2023-05-16T07:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 -- 172.21.15.92:0/56878341 shutdown_connections 2023-05-16T07:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 --2- 172.21.15.92:0/56878341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150bb0 0x7f42d8067f60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 --2- 172.21.15.92:0/56878341 >> v2:172.21.15.158:3300/0 conn(0x7f42d8150f90 0x7f42d8068450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 -- 172.21.15.92:0/56878341 >> 172.21.15.92:0/56878341 conn(0x7f42d80c8420 msgr2=0x7f42d80c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:30.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 -- 172.21.15.92:0/56878341 shutdown_connections 2023-05-16T07:03:30.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.508+0000 7f42dddc6700 1 -- 172.21.15.92:0/56878341 wait complete. 2023-05-16T07:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 Processor -- start 2023-05-16T07:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 -- start start 2023-05-16T07:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42d80a6780 0x7f42d80ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42d8150bb0 0x7f42d813e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 0x7f42d813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80bf0e0 con 0x7f42d8150f90 2023-05-16T07:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80bf3c0 con 0x7f42d8150bb0 2023-05-16T07:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.510+0000 7f42dddc6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80bf6a0 con 0x7f42d80a6780 2023-05-16T07:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 0x7f42d813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 0x7f42d813ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42848/0 (socket says 172.21.15.92:42848) 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 -- 172.21.15.92:0/905419159 learned_addr learned my addr 172.21.15.92:0/905419159 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 -- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42d80a6780 msgr2=0x7f42d80ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d77fe700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42d80a6780 0x7f42d80ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42d80a6780 0x7f42d80ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 -- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42d8150bb0 msgr2=0x7f42d813e810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42d8150bb0 0x7f42d813e810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 -- 172.21.15.92:0/905419159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009ae0 con 0x7f42d8150f90 2023-05-16T07:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d77fe700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42d80a6780 0x7f42d80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42d7fff700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 0x7f42d813ed00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f42d000c490 tx=0x7f42d0007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0002ea0 con 0x7f42d8150f90 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000f470 con 0x7f42d8150f90 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d80bf980 con 0x7f42d8150f90 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000f810 con 0x7f42d8150f90 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.511+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d80bec40 con 0x7f42d8150f90 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.512+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d80bfea0 con 0x7f42d8150f90 2023-05-16T07:03:30.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.512+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f42d0010070 con 0x7f42d8150f90 2023-05-16T07:03:30.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.513+0000 7f42bf7fe700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42c405ee30 0x7f42c40612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.513+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f42d00790d0 con 0x7f42d8150f90 2023-05-16T07:03:30.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.513+0000 7f42d77fe700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42c405ee30 0x7f42c40612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.514+0000 7f42d77fe700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42c405ee30 0x7f42c40612f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f42c8009540 tx=0x7f42c8009840).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.517+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f42d00478f0 con 0x7f42d8150f90 2023-05-16T07:03:30.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.704+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f42d80bf530 con 0x7f42c405ee30 2023-05-16T07:03:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.709+0000 7f42bf7fe700 1 -- 172.21.15.92:0/905419159 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f42d80bf530 con 0x7f42c405ee30 2023-05-16T07:03:30.714 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-05-16T07:03:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42c405ee30 msgr2=0x7f42c40612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42c405ee30 0x7f42c40612f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f42c8009540 tx=0x7f42c8009840).stop 2023-05-16T07:03:30.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 msgr2=0x7f42d813ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 0x7f42d813ed00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f42d000c490 tx=0x7f42d0007de0).stop 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 shutdown_connections 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42c405ee30 0x7f42c40612f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42d80a6780 0x7f42d80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d8150f90 0x7f42d813ed00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 --2- 172.21.15.92:0/905419159 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42d8150bb0 0x7f42d813e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 >> 172.21.15.92:0/905419159 conn(0x7f42d80c8420 msgr2=0x7f42d80c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 shutdown_connections 2023-05-16T07:03:30.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.711+0000 7f42dddc6700 1 -- 172.21.15.92:0/905419159 wait complete. 2023-05-16T07:03:30.727 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-05-16T07:03:30.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-05-16T07:03:30.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 Processor -- start 2023-05-16T07:03:30.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 -- start start 2023-05-16T07:03:30.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa578154470 0x7fa5780a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5780aa570 con 0x7fa5780aa150 2023-05-16T07:03:30.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5780c94d0 con 0x7fa578154470 2023-05-16T07:03:30.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.840+0000 7fa57eb0b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780cb7e0 con 0x7fa578154090 2023-05-16T07:03:30.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa577fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa578154470 0x7fa5780a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa57d0a8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa5780aa150 0x7fa578068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53604/0 (socket says 172.21.15.92:53604) 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa57d0a8700 1 -- 172.21.15.92:0/2812287703 learned_addr learned my addr 172.21.15.92:0/2812287703 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa57c8a7700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa577fff700 1 -- 172.21.15.92:0/2812287703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 msgr2=0x7fa5780aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa577fff700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa577fff700 1 -- 172.21.15.92:0/2812287703 >> v1:172.21.15.92:6789/0 conn(0x7fa5780aa150 legacy=0x7fa578068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa577fff700 1 -- 172.21.15.92:0/2812287703 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780c88c0 con 0x7fa578154470 2023-05-16T07:03:30.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa5777fe700 1 -- 172.21.15.92:0/2812287703 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618710435 0 0) 0x7fa570001110 con 0x7fa5780aa150 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa57c8a7700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa577fff700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa578154470 0x7fa5780a9c60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fa560009e60 tx=0x7fa560009780).ready entity=mon.0 client_cookie=1411d1f76b98ae48 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa5777fe700 1 -- 172.21.15.92:0/2812287703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000e070 con 0x7fa578154470 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa5777fe700 1 -- 172.21.15.92:0/2812287703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56000bc30 con 0x7fa578154470 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.841+0000 7fa5777fe700 1 -- 172.21.15.92:0/2812287703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560012410 con 0x7fa578154470 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2812287703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa578154470 msgr2=0x7fa5780a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa578154470 0x7fa5780a9c60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fa560009e60 tx=0x7fa560009780).stop 2023-05-16T07:03:30.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2812287703 shutdown_connections 2023-05-16T07:03:30.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2812287703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa578154470 0x7fa5780a9c60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:30.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2812287703 >> 172.21.15.92:0/2812287703 conn(0x7fa578067920 msgr2=0x7fa5780cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:30.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2812287703 shutdown_connections 2023-05-16T07:03:30.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.842+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2812287703 wait complete. 2023-05-16T07:03:30.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.843+0000 7fa57eb0b700 1 Processor -- start 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.843+0000 7fa57eb0b700 1 -- start start 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 0x7fa57813bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa578154470 0x7fa5780af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa578152db0 con 0x7fa5780aa150 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780c24d0 con 0x7fa578154470 2023-05-16T07:03:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780c2770 con 0x7fa578154090 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 0x7fa57813bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 0x7fa57813bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42858/0 (socket says 172.21.15.92:42858) 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 -- 172.21.15.92:0/2164347132 learned_addr learned my addr 172.21.15.92:0/2164347132 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 -- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 msgr2=0x7fa5780aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 -- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa578154470 msgr2=0x7fa5780af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:30.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa578154470 0x7fa5780af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:30.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 -- 172.21.15.92:0/2164347132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560009a20 con 0x7fa5780aa150 2023-05-16T07:03:30.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57c8a7700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 0x7fa57813bfa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa56800c880 tx=0x7fa568008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568010580 con 0x7fa5780aa150 2023-05-16T07:03:30.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680106e0 con 0x7fa5780aa150 2023-05-16T07:03:30.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5780c2a50 con 0x7fa5780aa150 2023-05-16T07:03:30.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568010a60 con 0x7fa5780aa150 2023-05-16T07:03:30.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57813c490 con 0x7fa5780aa150 2023-05-16T07:03:30.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5780c30a0 con 0x7fa5780aa150 2023-05-16T07:03:30.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.845+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fa56801b460 con 0x7fa5780aa150 2023-05-16T07:03:30.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.846+0000 7fa574ff9700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa55805ee30 0x7fa5580612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:30.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.846+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa568079ec0 con 0x7fa5780aa150 2023-05-16T07:03:30.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.846+0000 7fa577fff700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa55805ee30 0x7fa5580612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:30.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.847+0000 7fa577fff700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa55805ee30 0x7fa5580612f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa578158ec0 tx=0x7fa560009780).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:30.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:30.850+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa56804b280 con 0x7fa5780aa150 2023-05-16T07:03:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.036+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa57806c350 con 0x7fa55805ee30 2023-05-16T07:03:31.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.041+0000 7fa574ff9700 1 -- 172.21.15.92:0/2164347132 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa57806c350 con 0x7fa55805ee30 2023-05-16T07:03:31.047 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-05-16T07:03:31.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa55805ee30 msgr2=0x7fa5580612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa55805ee30 0x7fa5580612f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa578158ec0 tx=0x7fa560009780).stop 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 msgr2=0x7fa57813bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 0x7fa57813bfa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa56800c880 tx=0x7fa568008040).stop 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 shutdown_connections 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa55805ee30 0x7fa5580612f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa578154470 0x7fa5780af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5780aa150 0x7fa57813bfa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 --2- 172.21.15.92:0/2164347132 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa578154090 0x7fa5780aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:31.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 >> 172.21.15.92:0/2164347132 conn(0x7fa578067920 msgr2=0x7fa5780c5060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:31.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 shutdown_connections 2023-05-16T07:03:31.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.043+0000 7fa57eb0b700 1 -- 172.21.15.92:0/2164347132 wait complete. 2023-05-16T07:03:31.060 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-05-16T07:03:31.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-05-16T07:03:31.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.170+0000 7f09b1d61700 1 Processor -- start 2023-05-16T07:03:31.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.170+0000 7f09b1d61700 1 -- start start 2023-05-16T07:03:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09b1d61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09ac154470 0x7f09ac0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09b1d61700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac068160 0x7f09ac0685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09b1d61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac0c94d0 con 0x7f09ac154090 2023-05-16T07:03:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09b1d61700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac0cb7e0 con 0x7f09ac154470 2023-05-16T07:03:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09b1d61700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac068aa0 con 0x7f09ac068160 2023-05-16T07:03:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09ac154470 0x7f09ac0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09ab7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09ac154090 0x7f09ac0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53614/0 (socket says 172.21.15.92:53614) 2023-05-16T07:03:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09ab7fe700 1 -- 172.21.15.92:0/302680548 learned_addr learned my addr 172.21.15.92:0/302680548 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09abfff700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac068160 0x7f09ac0685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aaffd700 1 -- 172.21.15.92:0/302680548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac068160 msgr2=0x7f09ac0685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aaffd700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac068160 0x7f09ac0685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aaffd700 1 -- 172.21.15.92:0/302680548 >> v1:172.21.15.92:6789/0 conn(0x7f09ac154090 legacy=0x7f09ac0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aa7fc700 1 -- 172.21.15.92:0/302680548 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478998059 0 0) 0x7f09ac0c94d0 con 0x7f09ac154090 2023-05-16T07:03:31.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aaffd700 1 -- 172.21.15.92:0/302680548 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0c88c0 con 0x7f09ac154470 2023-05-16T07:03:31.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09abfff700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac068160 0x7f09ac0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:31.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aaffd700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0aaca0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f09a0009760 tx=0x7f09a0009060).ready entity=mon.0 client_cookie=834662daf6f1bb8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aa7fc700 1 -- 172.21.15.92:0/302680548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000b580 con 0x7f09ac154470 2023-05-16T07:03:31.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.171+0000 7f09aa7fc700 1 -- 172.21.15.92:0/302680548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a000b6e0 con 0x7f09ac154470 2023-05-16T07:03:31.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09aa7fc700 1 -- 172.21.15.92:0/302680548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000ba60 con 0x7f09ac154470 2023-05-16T07:03:31.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 -- 172.21.15.92:0/302680548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 msgr2=0x7f09ac0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0aaca0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f09a0009760 tx=0x7f09a0009060).stop 2023-05-16T07:03:31.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 -- 172.21.15.92:0/302680548 shutdown_connections 2023-05-16T07:03:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac068160 0x7f09ac0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 --2- 172.21.15.92:0/302680548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0aaca0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 -- 172.21.15.92:0/302680548 >> 172.21.15.92:0/302680548 conn(0x7f09ac067920 msgr2=0x7f09ac0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 -- 172.21.15.92:0/302680548 shutdown_connections 2023-05-16T07:03:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.172+0000 7f09b1d61700 1 -- 172.21.15.92:0/302680548 wait complete. 2023-05-16T07:03:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.173+0000 7f09b1d61700 1 Processor -- start 2023-05-16T07:03:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.173+0000 7f09b1d61700 1 -- start start 2023-05-16T07:03:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09b1d61700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09ac068160 0x7f09ac157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09b1d61700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac154090 0x7f09ac14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09b1d61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0c1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09b1d61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac157670 con 0x7f09ac154470 2023-05-16T07:03:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09b1d61700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac151b90 con 0x7f09ac068160 2023-05-16T07:03:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09b1d61700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac151e30 con 0x7f09ac154090 2023-05-16T07:03:31.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42876/0 (socket says 172.21.15.92:42876) 2023-05-16T07:03:31.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 -- 172.21.15.92:0/3039193397 learned_addr learned my addr 172.21.15.92:0/3039193397 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:31.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09aaffd700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac154090 0x7f09ac14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09ab7fe700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09ac068160 0x7f09ac157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 -- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac154090 msgr2=0x7f09ac14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac154090 0x7f09ac14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 -- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09ac068160 msgr2=0x7f09ac157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09ac068160 0x7f09ac157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.174+0000 7f09abfff700 1 -- 172.21.15.92:0/3039193397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a00092a0 con 0x7f09ac154470 2023-05-16T07:03:31.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09aaffd700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac154090 0x7f09ac14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:31.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09ab7fe700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09ac068160 0x7f09ac157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:03:31.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09abfff700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0c1000 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f09a400c880 tx=0x7f09a4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4010580 con 0x7f09ac154470 2023-05-16T07:03:31.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a40106e0 con 0x7f09ac154470 2023-05-16T07:03:31.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac152110 con 0x7f09ac154470 2023-05-16T07:03:31.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4010a60 con 0x7f09ac154470 2023-05-16T07:03:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.175+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac1526d0 con 0x7f09ac154470 2023-05-16T07:03:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.176+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac066730 con 0x7f09ac154470 2023-05-16T07:03:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.176+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f09a401b460 con 0x7f09ac154470 2023-05-16T07:03:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.176+0000 7f09937fe700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f099805ee30 0x7f09980612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.177+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f09a4079f30 con 0x7f09ac154470 2023-05-16T07:03:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.177+0000 7f09ab7fe700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f099805ee30 0x7f09980612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.177+0000 7f09ab7fe700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f099805ee30 0x7f09980612f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f099c001e40 tx=0x7f099c009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.181+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f09a404b2f0 con 0x7f09ac154470 2023-05-16T07:03:31.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.370+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f09ac06bd40 con 0x7f099805ee30 2023-05-16T07:03:31.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.374+0000 7f09937fe700 1 -- 172.21.15.92:0/3039193397 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f09ac06bd40 con 0x7f099805ee30 2023-05-16T07:03:31.380 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-05-16T07:03:31.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f099805ee30 msgr2=0x7f09980612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f099805ee30 0x7f09980612f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f099c001e40 tx=0x7f099c009040).stop 2023-05-16T07:03:31.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 msgr2=0x7f09ac0c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0c1000 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f09a400c880 tx=0x7f09a4008040).stop 2023-05-16T07:03:31.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 shutdown_connections 2023-05-16T07:03:31.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f099805ee30 0x7f09980612f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09ac154470 0x7f09ac0c1000 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09ac068160 0x7f09ac157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 --2- 172.21.15.92:0/3039193397 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09ac154090 0x7f09ac14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 >> 172.21.15.92:0/3039193397 conn(0x7f09ac067920 msgr2=0x7f09ac14f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:31.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 shutdown_connections 2023-05-16T07:03:31.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.376+0000 7f09b1d61700 1 -- 172.21.15.92:0/3039193397 wait complete. 2023-05-16T07:03:31.393 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-05-16T07:03:31.394 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-05-16T07:03:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.505+0000 7f6bc9595700 1 Processor -- start 2023-05-16T07:03:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.505+0000 7f6bc9595700 1 -- start start 2023-05-16T07:03:31.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc9595700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bc4154090 0x7f6bc40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc9595700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40c94d0 con 0x7f6bc4154470 2023-05-16T07:03:31.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc9595700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40cb7e0 con 0x7f6bc4068160 2023-05-16T07:03:31.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc9595700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc40c88c0 con 0x7f6bc4154090 2023-05-16T07:03:31.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bc4154090 0x7f6bc40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6bc4154470 0x7f6bc40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53630/0 (socket says 172.21.15.92:53630) 2023-05-16T07:03:31.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bc4154090 0x7f6bc40aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42880/0 (socket says 172.21.15.92:42880) 2023-05-16T07:03:31.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc27fc700 1 -- 172.21.15.92:0/456415134 learned_addr learned my addr 172.21.15.92:0/456415134 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:31.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc2ffd700 1 -- 172.21.15.92:0/456415134 >> v1:172.21.15.158:6789/0 conn(0x7f6bc4068160 legacy=0x7f6bc40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:31.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc2ffd700 1 -- 172.21.15.92:0/456415134 >> v1:172.21.15.92:6789/0 conn(0x7f6bc4154470 legacy=0x7f6bc40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc2ffd700 1 -- 172.21.15.92:0/456415134 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40c8ba0 con 0x7f6bc4154090 2023-05-16T07:03:31.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.506+0000 7f6bc1ffb700 1 -- 172.21.15.92:0/456415134 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579046809 0 0) 0x7f6bc40c94d0 con 0x7f6bc4154470 2023-05-16T07:03:31.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc2ffd700 1 --2- 172.21.15.92:0/456415134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40aaca0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6bb4009e60 tx=0x7f6bb4009780).ready entity=mon.0 client_cookie=d48e9212e5f95d03 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc1ffb700 1 -- 172.21.15.92:0/456415134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400e070 con 0x7f6bc4154090 2023-05-16T07:03:31.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc1ffb700 1 -- 172.21.15.92:0/456415134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb400bbf0 con 0x7f6bc4154090 2023-05-16T07:03:31.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc1ffb700 1 -- 172.21.15.92:0/456415134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4012410 con 0x7f6bc4154090 2023-05-16T07:03:31.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 -- 172.21.15.92:0/456415134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 msgr2=0x7f6bc40aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 --2- 172.21.15.92:0/456415134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40aaca0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6bb4009e60 tx=0x7f6bb4009780).stop 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 -- 172.21.15.92:0/456415134 shutdown_connections 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 --2- 172.21.15.92:0/456415134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40aaca0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 -- 172.21.15.92:0/456415134 >> 172.21.15.92:0/456415134 conn(0x7f6bc4067920 msgr2=0x7f6bc40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 -- 172.21.15.92:0/456415134 shutdown_connections 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.507+0000 7f6bc9595700 1 -- 172.21.15.92:0/456415134 wait complete. 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.508+0000 7f6bc9595700 1 Processor -- start 2023-05-16T07:03:31.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.508+0000 7f6bc9595700 1 -- start start 2023-05-16T07:03:31.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc9595700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6bc4068160 0x7f6bc4167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc9595700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc9595700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6bc4154470 0x7f6bc40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc9595700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40c2b00 con 0x7f6bc4154090 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc9595700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4168290 con 0x7f6bc4068160 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc9595700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4168570 con 0x7f6bc4154470 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42888/0 (socket says 172.21.15.92:42888) 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc27fc700 1 -- 172.21.15.92:0/1198059817 learned_addr learned my addr 172.21.15.92:0/1198059817 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.509+0000 7f6bc2ffd700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6bc4068160 0x7f6bc4167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc37fe700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6bc4154470 0x7f6bc40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc27fc700 1 -- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6bc4154470 msgr2=0x7f6bc40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc27fc700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6bc4154470 0x7f6bc40b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc27fc700 1 -- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6bc4068160 msgr2=0x7f6bc4167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc27fc700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6bc4068160 0x7f6bc4167cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc27fc700 1 -- 172.21.15.92:0/1198059817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4009a20 con 0x7f6bc4154090 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc2ffd700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6bc4068160 0x7f6bc4167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc27fc700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40ae750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f6bac001eb0 tx=0x7f6bac007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0072e0 con 0x7f6bc4154090 2023-05-16T07:03:31.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac007440 con 0x7f6bc4154090 2023-05-16T07:03:31.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc4168850 con 0x7f6bc4154090 2023-05-16T07:03:31.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0057a0 con 0x7f6bc4154090 2023-05-16T07:03:31.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.510+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc40aec40 con 0x7f6bc4154090 2023-05-16T07:03:31.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.511+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc40c2cb0 con 0x7f6bc4154090 2023-05-16T07:03:31.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.511+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f6bac01a060 con 0x7f6bc4154090 2023-05-16T07:03:31.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.511+0000 7f6bbaffd700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ba405ee60 0x7f6ba4061320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.511+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f6bac079460 con 0x7f6bc4154090 2023-05-16T07:03:31.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.512+0000 7f6bc2ffd700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ba405ee60 0x7f6ba4061320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.512+0000 7f6bc2ffd700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ba405ee60 0x7f6ba4061320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6bb400b9b0 tx=0x7f6bb4009780).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.515+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6bc40c2cb0 con 0x7f6bc4154090 2023-05-16T07:03:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.703+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bc406c350 con 0x7f6ba405ee60 2023-05-16T07:03:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.707+0000 7f6bbaffd700 1 -- 172.21.15.92:0/1198059817 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6bc406c350 con 0x7f6ba405ee60 2023-05-16T07:03:31.711 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-05-16T07:03:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ba405ee60 msgr2=0x7f6ba4061320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ba405ee60 0x7f6ba4061320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6bb400b9b0 tx=0x7f6bb4009780).stop 2023-05-16T07:03:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 msgr2=0x7f6bc40ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40ae750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f6bac001eb0 tx=0x7f6bac007e90).stop 2023-05-16T07:03:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 shutdown_connections 2023-05-16T07:03:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6ba405ee60 0x7f6ba4061320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6bc4154470 0x7f6bc40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6bc4068160 0x7f6bc4167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 --2- 172.21.15.92:0/1198059817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4154090 0x7f6bc40ae750 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 >> 172.21.15.92:0/1198059817 conn(0x7f6bc4067920 msgr2=0x7f6bc40acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 shutdown_connections 2023-05-16T07:03:31.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.709+0000 7f6bc9595700 1 -- 172.21.15.92:0/1198059817 wait complete. 2023-05-16T07:03:31.725 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-05-16T07:03:31.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-05-16T07:03:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.839+0000 7fc3352ba700 1 Processor -- start 2023-05-16T07:03:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.839+0000 7fc3352ba700 1 -- start start 2023-05-16T07:03:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.839+0000 7fc3352ba700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc330154090 0x7fc3300aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc3352ba700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc3300685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc3352ba700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300c94d0 con 0x7fc330154470 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc3352ba700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3300cb7e0 con 0x7fc330154090 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc3352ba700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc330068aa0 con 0x7fc330068160 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc330154090 0x7fc3300aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc330154090 0x7fc3300aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42894/0 (socket says 172.21.15.92:42894) 2023-05-16T07:03:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 -- 172.21.15.92:0/2801260707 learned_addr learned my addr 172.21.15.92:0/2801260707 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32f7fe700 1 --2- 172.21.15.92:0/2801260707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc3300685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 -- 172.21.15.92:0/2801260707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 msgr2=0x7fc3300685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 --2- 172.21.15.92:0/2801260707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc3300685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 -- 172.21.15.92:0/2801260707 >> v1:172.21.15.158:6789/0 conn(0x7fc330154470 legacy=0x7fc3300a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 -- 172.21.15.92:0/2801260707 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3300c88c0 con 0x7fc330154090 2023-05-16T07:03:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32effd700 1 --2- 172.21.15.92:0/2801260707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154090 0x7fc3300aaca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fc320006fd0 tx=0x7fc3200090f0).ready entity=mon.0 client_cookie=527ad8e0347eabf3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32e7fc700 1 -- 172.21.15.92:0/2801260707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000b600 con 0x7fc330154090 2023-05-16T07:03:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32e7fc700 1 -- 172.21.15.92:0/2801260707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32000b760 con 0x7fc330154090 2023-05-16T07:03:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.840+0000 7fc32e7fc700 1 -- 172.21.15.92:0/2801260707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000bae0 con 0x7fc330154090 2023-05-16T07:03:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 -- 172.21.15.92:0/2801260707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154090 msgr2=0x7fc3300aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2801260707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154090 0x7fc3300aaca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fc320006fd0 tx=0x7fc3200090f0).stop 2023-05-16T07:03:31.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 -- 172.21.15.92:0/2801260707 shutdown_connections 2023-05-16T07:03:31.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2801260707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154090 0x7fc3300aaca0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2801260707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc3300685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:31.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 -- 172.21.15.92:0/2801260707 >> 172.21.15.92:0/2801260707 conn(0x7fc330067920 msgr2=0x7fc3300cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:31.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 -- 172.21.15.92:0/2801260707 shutdown_connections 2023-05-16T07:03:31.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.841+0000 7fc3352ba700 1 -- 172.21.15.92:0/2801260707 wait complete. 2023-05-16T07:03:31.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.842+0000 7fc3352ba700 1 Processor -- start 2023-05-16T07:03:31.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.842+0000 7fc3352ba700 1 -- start start 2023-05-16T07:03:31.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc330151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc330154090 0x7fc330150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 0x7fc3300c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330152470 con 0x7fc330154470 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330153270 con 0x7fc330154090 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc330153700 con 0x7fc330068160 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 0x7fc3300c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 0x7fc3300c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42902/0 (socket says 172.21.15.92:42902) 2023-05-16T07:03:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 -- 172.21.15.92:0/2094519003 learned_addr learned my addr 172.21.15.92:0/2094519003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 -- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 msgr2=0x7fc330151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc330151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 -- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc330154090 msgr2=0x7fc330150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc330154090 0x7fc330150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 -- 172.21.15.92:0/2094519003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc320009370 con 0x7fc330154470 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc32f7fe700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 0x7fc3300c4410 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fc328009fa0 tx=0x7fc328009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800ed20 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32800ee80 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc330152660 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.843+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328014610 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.844+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3301529f0 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.844+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc330153020 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.845+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fc32801b030 con 0x7fc330154470 2023-05-16T07:03:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.845+0000 7fc3277fe700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc31005ee30 0x7fc3100612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.845+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fc32807a210 con 0x7fc330154470 2023-05-16T07:03:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.845+0000 7fc32effd700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc31005ee30 0x7fc3100612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.846+0000 7fc32effd700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc31005ee30 0x7fc3100612f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc320009130 tx=0x7fc320017000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:31.849+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc32804b630 con 0x7fc330154470 2023-05-16T07:03:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.035+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc330066730 con 0x7fc31005ee30 2023-05-16T07:03:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.040+0000 7fc3277fe700 1 -- 172.21.15.92:0/2094519003 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc330066730 con 0x7fc31005ee30 2023-05-16T07:03:32.044 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-05-16T07:03:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.041+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc31005ee30 msgr2=0x7fc3100612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.041+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc31005ee30 0x7fc3100612f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc320009130 tx=0x7fc320017000).stop 2023-05-16T07:03:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.041+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 msgr2=0x7fc3300c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.041+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 0x7fc3300c4410 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fc328009fa0 tx=0x7fc328009900).stop 2023-05-16T07:03:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 shutdown_connections 2023-05-16T07:03:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc31005ee30 0x7fc3100612f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc330068160 0x7fc330151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc330154470 0x7fc3300c4410 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 --2- 172.21.15.92:0/2094519003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc330154090 0x7fc330150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:03:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 >> 172.21.15.92:0/2094519003 conn(0x7fc330067920 msgr2=0x7fc3300ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 shutdown_connections 2023-05-16T07:03:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.042+0000 7fc3352ba700 1 -- 172.21.15.92:0/2094519003 wait complete. 2023-05-16T07:03:32.057 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-05-16T07:03:32.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-05-16T07:03:32.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.169+0000 7fb6032ef700 1 Processor -- start 2023-05-16T07:03:32.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.169+0000 7fb6032ef700 1 -- start start 2023-05-16T07:03:32.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6032ef700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb5fc153bb0 0x7fb5fc0698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6032ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc0bdf20 0x7fb5fc067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6032ef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc05e760 con 0x7fb5fc153f90 2023-05-16T07:03:32.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6032ef700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc069ef0 con 0x7fb5fc153bb0 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6032ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0bf540 con 0x7fb5fc0bdf20 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb601aec700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5fc153f90 0x7fb5fc0bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53656/0 (socket says 172.21.15.92:53656) 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb601aec700 1 -- 172.21.15.92:0/1429773907 learned_addr learned my addr 172.21.15.92:0/1429773907 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb602aee700 1 --2- 172.21.15.92:0/1429773907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc0bdf20 0x7fb5fc067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6022ed700 1 --2- 172.21.15.92:0/1429773907 >> v2:172.21.15.158:3300/0 conn(0x7fb5fc153bb0 0x7fb5fc0698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6012eb700 1 -- 172.21.15.92:0/1429773907 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274166401 0 0) 0x7fb5fc05e760 con 0x7fb5fc153f90 2023-05-16T07:03:32.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6012eb700 1 -- 172.21.15.92:0/1429773907 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e40033f0 con 0x7fb5fc153f90 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6012eb700 1 -- 172.21.15.92:0/1429773907 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401081060 0 0) 0x7fb5e40033f0 con 0x7fb5fc153f90 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb602aee700 1 -- 172.21.15.92:0/1429773907 >> v2:172.21.15.158:3300/0 conn(0x7fb5fc153bb0 msgr2=0x7fb5fc0698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb602aee700 1 --2- 172.21.15.92:0/1429773907 >> v2:172.21.15.158:3300/0 conn(0x7fb5fc153bb0 0x7fb5fc0698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb602aee700 1 -- 172.21.15.92:0/1429773907 >> v1:172.21.15.92:6789/0 conn(0x7fb5fc153f90 legacy=0x7fb5fc0bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb602aee700 1 -- 172.21.15.92:0/1429773907 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc068450 con 0x7fb5fc0bdf20 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb6022ed700 1 --2- 172.21.15.92:0/1429773907 >> v2:172.21.15.158:3300/0 conn(0x7fb5fc153bb0 0x7fb5fc0698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:03:32.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.170+0000 7fb602aee700 1 --2- 172.21.15.92:0/1429773907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc0bdf20 0x7fb5fc067f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb5f4009760 tx=0x7fb5f4009060).ready entity=mon.2 client_cookie=f624520abc7f26cc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:32.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6012eb700 1 -- 172.21.15.92:0/1429773907 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0001470 con 0x7fb5fc0bdf20 2023-05-16T07:03:32.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6012eb700 1 -- 172.21.15.92:0/1429773907 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4011400 con 0x7fb5fc0bdf20 2023-05-16T07:03:32.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6012eb700 1 -- 172.21.15.92:0/1429773907 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4011780 con 0x7fb5fc0bdf20 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 -- 172.21.15.92:0/1429773907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc0bdf20 msgr2=0x7fb5fc067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 --2- 172.21.15.92:0/1429773907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc0bdf20 0x7fb5fc067f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb5f4009760 tx=0x7fb5f4009060).stop 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 -- 172.21.15.92:0/1429773907 shutdown_connections 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 --2- 172.21.15.92:0/1429773907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc0bdf20 0x7fb5fc067f60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 --2- 172.21.15.92:0/1429773907 >> v2:172.21.15.158:3300/0 conn(0x7fb5fc153bb0 0x7fb5fc0698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 -- 172.21.15.92:0/1429773907 >> 172.21.15.92:0/1429773907 conn(0x7fb5fc062d80 msgr2=0x7fb5fc05dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 -- 172.21.15.92:0/1429773907 shutdown_connections 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.171+0000 7fb6032ef700 1 -- 172.21.15.92:0/1429773907 wait complete. 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.172+0000 7fb6032ef700 1 Processor -- start 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.172+0000 7fb6032ef700 1 -- start start 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6032ef700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5fc0bdf20 0x7fb5fc1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6032ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc153bb0 0x7fb5fc157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6032ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 0x7fb5fc1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6032ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc157ee0 con 0x7fb5fc153f90 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6032ef700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1547d0 con 0x7fb5fc0bdf20 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6032ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0c1750 con 0x7fb5fc153bb0 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb602aee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 0x7fb5fc1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb602aee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 0x7fb5fc1579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42914/0 (socket says 172.21.15.92:42914) 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb602aee700 1 -- 172.21.15.92:0/2357190663 learned_addr learned my addr 172.21.15.92:0/2357190663 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb6022ed700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5fc0bdf20 0x7fb5fc1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.173+0000 7fb601aec700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc153bb0 0x7fb5fc157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb602aee700 1 -- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc153bb0 msgr2=0x7fb5fc157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb602aee700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc153bb0 0x7fb5fc157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb602aee700 1 -- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5fc0bdf20 msgr2=0x7fb5fc1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb602aee700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5fc0bdf20 0x7fb5fc1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb602aee700 1 -- 172.21.15.92:0/2357190663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec009a20 con 0x7fb5fc153f90 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb6022ed700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5fc0bdf20 0x7fb5fc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb602aee700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 0x7fb5fc1579f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fb5fc0bf680 tx=0x7fb5f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4010b80 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4010ce0 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40092a0 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f401d640 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.174+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc0c1d10 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.175+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc154940 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.175+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fb5f4024030 con 0x7fb5fc153f90 2023-05-16T07:03:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.175+0000 7fb5fa7fc700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e805ee30 0x7fb5e80612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.175+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb5f407aba0 con 0x7fb5fc153f90 2023-05-16T07:03:32.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.175+0000 7fb6022ed700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e805ee30 0x7fb5e80612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.176+0000 7fb6022ed700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e805ee30 0x7fb5e80612f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb5ec0099f0 tx=0x7fb5ec00b040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:32.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.179+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb5f404c070 con 0x7fb5fc153f90 2023-05-16T07:03:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.367+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5fc047150 con 0x7fb5e805ee30 2023-05-16T07:03:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.374+0000 7fb5fa7fc700 1 -- 172.21.15.92:0/2357190663 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb5fc047150 con 0x7fb5e805ee30 2023-05-16T07:03:32.378 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-05-16T07:03:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e805ee30 msgr2=0x7fb5e80612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e805ee30 0x7fb5e80612f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb5ec0099f0 tx=0x7fb5ec00b040).stop 2023-05-16T07:03:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 msgr2=0x7fb5fc1579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 0x7fb5fc1579f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fb5fc0bf680 tx=0x7fb5f4016000).stop 2023-05-16T07:03:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 shutdown_connections 2023-05-16T07:03:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e805ee30 0x7fb5e80612f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc153f90 0x7fb5fc1579f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5fc0bdf20 0x7fb5fc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 --2- 172.21.15.92:0/2357190663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5fc153bb0 0x7fb5fc157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 >> 172.21.15.92:0/2357190663 conn(0x7fb5fc062d80 msgr2=0x7fb5fc0bd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 shutdown_connections 2023-05-16T07:03:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.376+0000 7fb6032ef700 1 -- 172.21.15.92:0/2357190663 wait complete. 2023-05-16T07:03:32.391 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-05-16T07:03:32.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-05-16T07:03:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.506+0000 7f7dd8d57700 1 Processor -- start 2023-05-16T07:03:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.506+0000 7f7dd8d57700 1 -- start start 2023-05-16T07:03:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.506+0000 7f7dd8d57700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7dd4155140 0x7f7dd40622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.506+0000 7f7dd8d57700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd414ead0 con 0x7f7dd40627a0 2023-05-16T07:03:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.506+0000 7f7dd8d57700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd4154cb0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.506+0000 7f7dd8d57700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd40c6020 con 0x7f7dd4155140 2023-05-16T07:03:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd2d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7dd40627a0 0x7f7dd4063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53672/0 (socket says 172.21.15.92:53672) 2023-05-16T07:03:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd2d9d700 1 -- 172.21.15.92:0/924002441 learned_addr learned my addr 172.21.15.92:0/924002441 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:03:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dcbfff700 1 --2- 172.21.15.92:0/924002441 >> v2:172.21.15.158:3300/0 conn(0x7f7dd4155140 0x7f7dd40622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255471081 0 0) 0x7f7dd414ead0 con 0x7f7dd40627a0 2023-05-16T07:03:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc00033f0 con 0x7f7dd40627a0 2023-05-16T07:03:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482651846 0 0) 0x7f7dd4154cb0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dd414ead0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dcbfff700 1 -- 172.21.15.92:0/924002441 >> v1:172.21.15.158:6789/0 conn(0x7f7dd414e6f0 legacy=0x7f7dd40c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dcbfff700 1 -- 172.21.15.92:0/924002441 >> v1:172.21.15.92:6789/0 conn(0x7f7dd40627a0 legacy=0x7f7dd4063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133417731 0 0) 0x7f7dc00033f0 con 0x7f7dd40627a0 2023-05-16T07:03:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dcbfff700 1 -- 172.21.15.92:0/924002441 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd4154660 con 0x7f7dd4155140 2023-05-16T07:03:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dcbfff700 1 --2- 172.21.15.92:0/924002441 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40622b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f7dbc009760 tx=0x7f7dbc009060).ready entity=mon.1 client_cookie=3ffcd79b616baa62 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0017f0 con 0x7f7dd4155140 2023-05-16T07:03:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc011400 con 0x7f7dd4155140 2023-05-16T07:03:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.507+0000 7f7dd1d9b700 1 -- 172.21.15.92:0/924002441 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011780 con 0x7f7dd4155140 2023-05-16T07:03:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 -- 172.21.15.92:0/924002441 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 msgr2=0x7f7dd40622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/924002441 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40622b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f7dbc009760 tx=0x7f7dbc009060).stop 2023-05-16T07:03:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 -- 172.21.15.92:0/924002441 shutdown_connections 2023-05-16T07:03:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/924002441 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40622b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 -- 172.21.15.92:0/924002441 >> 172.21.15.92:0/924002441 conn(0x7f7dd405e820 msgr2=0x7f7dd40c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 -- 172.21.15.92:0/924002441 shutdown_connections 2023-05-16T07:03:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.508+0000 7f7dd8d57700 1 -- 172.21.15.92:0/924002441 wait complete. 2023-05-16T07:03:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.509+0000 7f7dd8d57700 1 Processor -- start 2023-05-16T07:03:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.509+0000 7f7dd8d57700 1 -- start start 2023-05-16T07:03:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dd40627a0 0x7f7dd4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 0x7f7dd41651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40abb50 con 0x7f7dd414e6f0 2023-05-16T07:03:32.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40a8a40 con 0x7f7dd4155140 2023-05-16T07:03:32.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd41656e0 con 0x7f7dd40627a0 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 0x7f7dd41651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 0x7f7dd41651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42928/0 (socket says 172.21.15.92:42928) 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 -- 172.21.15.92:0/848104771 learned_addr learned my addr 172.21.15.92:0/848104771 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 -- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dd40627a0 msgr2=0x7f7dd4150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd259c700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dd40627a0 0x7f7dd4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd2d9d700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dd40627a0 0x7f7dd4150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 -- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 msgr2=0x7f7dd40ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 -- 172.21.15.92:0/848104771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0092a0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dcbfff700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 0x7f7dd41651f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f7dbc009d50 tx=0x7f7dbc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0108e0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc010a40 con 0x7f7dd414e6f0 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd40a86f0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.510+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc010dc0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.511+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4151890 con 0x7f7dd414e6f0 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.511+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd40a8bb0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.512+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f7dbc00f070 con 0x7f7dd414e6f0 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.512+0000 7f7dca7fc700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7db405ee30 0x7f7db40612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.512+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f7dbc07b250 con 0x7f7dd414e6f0 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.512+0000 7f7dd259c700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7db405ee30 0x7f7db40612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:03:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.513+0000 7f7dd259c700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7db405ee30 0x7f7db40612f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7dd40ab5e0 tx=0x7f7dc4009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:03:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.516+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7dbc04c6b0 con 0x7f7dd414e6f0 2023-05-16T07:03:32.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.704+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dd4151a80 con 0x7f7db405ee30 2023-05-16T07:03:32.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.708+0000 7f7dca7fc700 1 -- 172.21.15.92:0/848104771 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7dd4151a80 con 0x7f7db405ee30 2023-05-16T07:03:32.712 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-05-16T07:03:32.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.710+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7db405ee30 msgr2=0x7f7db40612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.710+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7db405ee30 0x7f7db40612f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7dd40ab5e0 tx=0x7f7dc4009040).stop 2023-05-16T07:03:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.710+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 msgr2=0x7f7dd41651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:03:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.710+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 0x7f7dd41651f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f7dbc009d50 tx=0x7f7dbc016000).stop 2023-05-16T07:03:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 shutdown_connections 2023-05-16T07:03:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7db405ee30 0x7f7db40612f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dd40627a0 0x7f7dd4150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dd4155140 0x7f7dd40ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 --2- 172.21.15.92:0/848104771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dd414e6f0 0x7f7dd41651f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:03:32.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 >> 172.21.15.92:0/848104771 conn(0x7f7dd405e820 msgr2=0x7f7dd40c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:03:32.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 shutdown_connections 2023-05-16T07:03:32.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:03:32.711+0000 7f7dd8d57700 1 -- 172.21.15.92:0/848104771 wait complete. 2023-05-16T07:03:32.726 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-05-16T07:03:35.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:03:35.086 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:04:00.048 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:04:00.043+0000 7feb4485d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:04:00.048 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:04:00.043+0000 7feb4485d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:04:00.048 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:04:00.043+0000 7feb4485d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:04:00.048 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:04:00.043+0000 7feb4485d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:04:00.049 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:04:00.043+0000 7feb4485d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:04:05.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:04:05.168 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:04:35.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:04:35.230 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:05:05.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:05:05.296 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:05:35.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:05:35.366 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:06:05.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:06:05.464 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:06:35.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:06:35.530 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:07:05.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:07:05.625 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:07:35.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:07:35.727 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:08:05.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:08:05.796 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:08:35.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:08:35.859 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:09:05.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:09:05.913 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:09:35.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:09:35.973 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:10:06.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:10:06.040 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:10:36.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:10:36.103 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:11:06.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:11:06.165 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:11:36.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:11:36.219 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:12:06.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:12:06.297 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:12:36.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:12:36.358 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:13:06.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:13:06.419 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:13:36.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:13:36.481 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:14:06.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:14:06.542 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:14:18.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-05-16T07:14:18.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.854+0000 7f300359e700 1 Processor -- start 2023-05-16T07:14:18.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.854+0000 7f300359e700 1 -- start start 2023-05-16T07:14:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.855+0000 7f300359e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3004153f90 0x7f30040a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.855+0000 7f300359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3004155380 0x7f30040c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:18.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.855+0000 7f300359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040c66a0 con 0x7f30040c5d00 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.855+0000 7f300359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3004069f60 con 0x7f3004155380 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.855+0000 7f300359e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300406a9e0 con 0x7f3004153f90 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3004155380 0x7f30040c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3004155380 0x7f30040c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42934/0 (socket says 172.21.15.92:42934) 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 -- 172.21.15.92:0/702218504 learned_addr learned my addr 172.21.15.92:0/702218504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f300259c700 1 --2- 172.21.15.92:0/702218504 >> v2:172.21.15.158:3300/0 conn(0x7f3004153f90 0x7f30040a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 -- 172.21.15.92:0/702218504 >> v2:172.21.15.158:3300/0 conn(0x7f3004153f90 msgr2=0x7f30040a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 --2- 172.21.15.92:0/702218504 >> v2:172.21.15.158:3300/0 conn(0x7f3004153f90 0x7f30040a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 -- 172.21.15.92:0/702218504 >> v1:172.21.15.158:6789/0 conn(0x7f30040c5d00 legacy=0x7f30040c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 -- 172.21.15.92:0/702218504 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004155780 con 0x7f3004155380 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f300259c700 1 --2- 172.21.15.92:0/702218504 >> v2:172.21.15.158:3300/0 conn(0x7f3004153f90 0x7f30040a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f3002d9d700 1 --2- 172.21.15.92:0/702218504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004155380 0x7f30040c61b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f30040caa40 tx=0x7f2ffc0090f0).ready entity=mon.0 client_cookie=d35db9f60f440c68 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f300159a700 1 -- 172.21.15.92:0/702218504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00b5a0 con 0x7f3004155380 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f300159a700 1 -- 172.21.15.92:0/702218504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc00b700 con 0x7f3004155380 2023-05-16T07:14:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.856+0000 7f300159a700 1 -- 172.21.15.92:0/702218504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00ba80 con 0x7f3004155380 2023-05-16T07:14:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 -- 172.21.15.92:0/702218504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004155380 msgr2=0x7f30040c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 --2- 172.21.15.92:0/702218504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004155380 0x7f30040c61b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f30040caa40 tx=0x7f2ffc0090f0).stop 2023-05-16T07:14:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 -- 172.21.15.92:0/702218504 shutdown_connections 2023-05-16T07:14:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 --2- 172.21.15.92:0/702218504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004155380 0x7f30040c61b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 --2- 172.21.15.92:0/702218504 >> v2:172.21.15.158:3300/0 conn(0x7f3004153f90 0x7f30040a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 -- 172.21.15.92:0/702218504 >> 172.21.15.92:0/702218504 conn(0x7f30040cb600 msgr2=0x7f30040c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 -- 172.21.15.92:0/702218504 shutdown_connections 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.857+0000 7f300359e700 1 -- 172.21.15.92:0/702218504 wait complete. 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.858+0000 7f300359e700 1 Processor -- start 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.858+0000 7f300359e700 1 -- start start 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f30040c5d00 0x7f3004164790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 0x7f300415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3004155380 0x7f300414f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30041505a0 con 0x7f3004153f90 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300415a600 con 0x7f3004155380 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300414fa30 con 0x7f30040c5d00 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 0x7f300415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 0x7f300415a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42940/0 (socket says 172.21.15.92:42940) 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 -- 172.21.15.92:0/2551656439 learned_addr learned my addr 172.21.15.92:0/2551656439 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3002d9d700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3004155380 0x7f300414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300259c700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f30040c5d00 0x7f3004164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 -- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f30040c5d00 msgr2=0x7f3004164790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:18.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f30040c5d00 0x7f3004164790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:18.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 -- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3004155380 msgr2=0x7f300414f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:18.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3004155380 0x7f300414f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:18.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 -- 172.21.15.92:0/2551656439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc009370 con 0x7f3004153f90 2023-05-16T07:14:18.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f3001d9b700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 0x7f300415a110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2fec00b490 tx=0x7f2fec006f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:18.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec002320 con 0x7f3004153f90 2023-05-16T07:14:18.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec0024c0 con 0x7f3004153f90 2023-05-16T07:14:18.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300414fd00 con 0x7f3004153f90 2023-05-16T07:14:18.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec002880 con 0x7f3004153f90 2023-05-16T07:14:18.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.859+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30041502c0 con 0x7f3004153f90 2023-05-16T07:14:18.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.860+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3004083120 con 0x7f3004153f90 2023-05-16T07:14:18.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.861+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f2fec00f030 con 0x7f3004153f90 2023-05-16T07:14:18.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.861+0000 7f2ffa7fc700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fe405ee70 0x7f2fe4061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:18.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.861+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f2fec079f80 con 0x7f3004153f90 2023-05-16T07:14:18.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.861+0000 7f300259c700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fe405ee70 0x7f2fe4061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:18.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.862+0000 7f300259c700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fe405ee70 0x7f2fe4061330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f2ff400e650 tx=0x7f2ff4007710).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:18.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:18.865+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2fec04b4b0 con 0x7f3004153f90 2023-05-16T07:14:19.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.053+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3004047150 con 0x7f2fe405ee70 2023-05-16T07:14:19.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.057+0000 7f2ffa7fc700 1 -- 172.21.15.92:0/2551656439 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3004047150 con 0x7f2fe405ee70 2023-05-16T07:14:19.062 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-05-16T07:14:19.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fe405ee70 msgr2=0x7f2fe4061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fe405ee70 0x7f2fe4061330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f2ff400e650 tx=0x7f2ff4007710).stop 2023-05-16T07:14:19.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 msgr2=0x7f300415a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 0x7f300415a110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2fec00b490 tx=0x7f2fec006f00).stop 2023-05-16T07:14:19.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 shutdown_connections 2023-05-16T07:14:19.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fe405ee70 0x7f2fe4061330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.059+0000 7f300359e700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f30040c5d00 0x7f3004164790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.060+0000 7f300359e700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3004155380 0x7f300414f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.060+0000 7f300359e700 1 --2- 172.21.15.92:0/2551656439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004153f90 0x7f300415a110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.060+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 >> 172.21.15.92:0/2551656439 conn(0x7f30040cb600 msgr2=0x7f30040ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.060+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 shutdown_connections 2023-05-16T07:14:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.060+0000 7f300359e700 1 -- 172.21.15.92:0/2551656439 wait complete. 2023-05-16T07:14:19.075 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-05-16T07:14:19.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-05-16T07:14:19.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.185+0000 7f37ccb29700 1 Processor -- start 2023-05-16T07:14:19.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.185+0000 7f37ccb29700 1 -- start start 2023-05-16T07:14:19.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37ccb29700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37c8153f90 0x7f37c80a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37ccb29700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f37c8155380 0x7f37c80c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37ccb29700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c80c66a0 con 0x7f37c80c5d00 2023-05-16T07:14:19.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37ccb29700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c8069f60 con 0x7f37c8153f90 2023-05-16T07:14:19.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37ccb29700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c806a9e0 con 0x7f37c8155380 2023-05-16T07:14:19.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37c77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37c8153f90 0x7f37c80a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37c77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37c8153f90 0x7f37c80a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42946/0 (socket says 172.21.15.92:42946) 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.186+0000 7f37c77fe700 1 -- 172.21.15.92:0/2658358845 learned_addr learned my addr 172.21.15.92:0/2658358845 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c7fff700 1 --2- 172.21.15.92:0/2658358845 >> v2:172.21.15.158:3300/0 conn(0x7f37c8155380 0x7f37c80c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c77fe700 1 -- 172.21.15.92:0/2658358845 >> v2:172.21.15.158:3300/0 conn(0x7f37c8155380 msgr2=0x7f37c80c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c77fe700 1 --2- 172.21.15.92:0/2658358845 >> v2:172.21.15.158:3300/0 conn(0x7f37c8155380 0x7f37c80c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c77fe700 1 -- 172.21.15.92:0/2658358845 >> v1:172.21.15.158:6789/0 conn(0x7f37c80c5d00 legacy=0x7f37c80c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c77fe700 1 -- 172.21.15.92:0/2658358845 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c8155780 con 0x7f37c8153f90 2023-05-16T07:14:19.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c77fe700 1 --2- 172.21.15.92:0/2658358845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c8153f90 0x7f37c80a9d20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f37c80c1b50 tx=0x7f37b8009840).ready entity=mon.0 client_cookie=dc73961d5f3defa5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c67fc700 1 -- 172.21.15.92:0/2658358845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800e070 con 0x7f37c8153f90 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c67fc700 1 -- 172.21.15.92:0/2658358845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800ba10 con 0x7f37c8153f90 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37c67fc700 1 -- 172.21.15.92:0/2658358845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800bd90 con 0x7f37c8153f90 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 -- 172.21.15.92:0/2658358845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c8153f90 msgr2=0x7f37c80a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2658358845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c8153f90 0x7f37c80a9d20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f37c80c1b50 tx=0x7f37b8009840).stop 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 -- 172.21.15.92:0/2658358845 shutdown_connections 2023-05-16T07:14:19.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2658358845 >> v2:172.21.15.158:3300/0 conn(0x7f37c8155380 0x7f37c80c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2658358845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c8153f90 0x7f37c80a9d20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 -- 172.21.15.92:0/2658358845 >> 172.21.15.92:0/2658358845 conn(0x7f37c80cb600 msgr2=0x7f37c80c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:19.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 -- 172.21.15.92:0/2658358845 shutdown_connections 2023-05-16T07:14:19.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.187+0000 7f37ccb29700 1 -- 172.21.15.92:0/2658358845 wait complete. 2023-05-16T07:14:19.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 Processor -- start 2023-05-16T07:14:19.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 -- start start 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 0x7f37c8142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37c8153f90 0x7f37c80c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37c8155380 0x7f37c80adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8167ab0 con 0x7f37c80c5d00 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8167d50 con 0x7f37c8155380 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37ccb29700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8168030 con 0x7f37c8153f90 2023-05-16T07:14:19.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 0x7f37c8142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 0x7f37c8142750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42952/0 (socket says 172.21.15.92:42952) 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.189+0000 7f37c77fe700 1 -- 172.21.15.92:0/2164162909 learned_addr learned my addr 172.21.15.92:0/2164162909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c7fff700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37c8155380 0x7f37c80adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c6ffd700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37c8153f90 0x7f37c80c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c77fe700 1 -- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37c8153f90 msgr2=0x7f37c80c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c77fe700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37c8153f90 0x7f37c80c2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c77fe700 1 -- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37c8155380 msgr2=0x7f37c80adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c77fe700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37c8155380 0x7f37c80adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c77fe700 1 -- 172.21.15.92:0/2164162909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8009ae0 con 0x7f37c80c5d00 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c7fff700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37c8155380 0x7f37c80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37c77fe700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 0x7f37c8142750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f37c80668c0 tx=0x7f37b8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8013d10 con 0x7f37c80c5d00 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b8013e70 con 0x7f37c80c5d00 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c8168310 con 0x7f37c80c5d00 2023-05-16T07:14:19.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b801d620 con 0x7f37c80c5d00 2023-05-16T07:14:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.190+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c8142c40 con 0x7f37c80c5d00 2023-05-16T07:14:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.191+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c8083120 con 0x7f37c80c5d00 2023-05-16T07:14:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.191+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f37b801b040 con 0x7f37c80c5d00 2023-05-16T07:14:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.192+0000 7f37af7fe700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f37b405ee70 0x7f37b4061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.192+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f37b807b6b0 con 0x7f37c80c5d00 2023-05-16T07:14:19.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.192+0000 7f37c6ffd700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f37b405ee70 0x7f37b4061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.192+0000 7f37c6ffd700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f37b405ee70 0x7f37b4061330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f37c8161ec0 tx=0x7f37bc008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.196+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f37b804cb60 con 0x7f37c80c5d00 2023-05-16T07:14:19.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.385+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f37c80c66a0 con 0x7f37b405ee70 2023-05-16T07:14:19.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.389+0000 7f37af7fe700 1 -- 172.21.15.92:0/2164162909 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f37c80c66a0 con 0x7f37b405ee70 2023-05-16T07:14:19.393 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-05-16T07:14:19.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f37b405ee70 msgr2=0x7f37b4061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f37b405ee70 0x7f37b4061330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f37c8161ec0 tx=0x7f37bc008040).stop 2023-05-16T07:14:19.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 msgr2=0x7f37c8142750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 0x7f37c8142750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f37c80668c0 tx=0x7f37b8009840).stop 2023-05-16T07:14:19.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 shutdown_connections 2023-05-16T07:14:19.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f37b405ee70 0x7f37b4061330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37c8155380 0x7f37c80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37c80c5d00 0x7f37c8142750 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 --2- 172.21.15.92:0/2164162909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f37c8153f90 0x7f37c80c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 >> 172.21.15.92:0/2164162909 conn(0x7f37c80cb600 msgr2=0x7f37c80c0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:19.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 shutdown_connections 2023-05-16T07:14:19.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.391+0000 7f37ccb29700 1 -- 172.21.15.92:0/2164162909 wait complete. 2023-05-16T07:14:19.406 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-05-16T07:14:19.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-05-16T07:14:19.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.519+0000 7f51c707a700 1 Processor -- start 2023-05-16T07:14:19.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.519+0000 7f51c707a700 1 -- start start 2023-05-16T07:14:19.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51c707a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f51c0154470 0x7f51c00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51c707a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c00c94d0 con 0x7f51c0154090 2023-05-16T07:14:19.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51c707a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c00cb7e0 con 0x7f51c0068160 2023-05-16T07:14:19.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51c707a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c00c88c0 con 0x7f51c0154470 2023-05-16T07:14:19.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51c4e16700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51c0154090 0x7f51c00a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53708/0 (socket says 172.21.15.92:53708) 2023-05-16T07:14:19.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51c4e16700 1 -- 172.21.15.92:0/2197613395 learned_addr learned my addr 172.21.15.92:0/2197613395 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bffff700 1 --2- 172.21.15.92:0/2197613395 >> v2:172.21.15.158:3300/0 conn(0x7f51c0154470 0x7f51c00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660951888 0 0) 0x7f51c00c94d0 con 0x7f51c0154090 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ac0033f0 con 0x7f51c0154090 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400636367 0 0) 0x7f51ac0033f0 con 0x7f51c0154090 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 >> v2:172.21.15.158:3300/0 conn(0x7f51c0154470 msgr2=0x7f51c00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 --2- 172.21.15.92:0/2197613395 >> v2:172.21.15.158:3300/0 conn(0x7f51c0154470 0x7f51c00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 >> v1:172.21.15.158:6789/0 conn(0x7f51c0068160 legacy=0x7f51c00c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00c8ba0 con 0x7f51c0154090 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f51b00017f0 con 0x7f51c0154090 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 >> v1:172.21.15.92:6789/0 conn(0x7f51c0154090 legacy=0x7f51c00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac0038e0 0x7f51ac003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51ac004690 0x7f51ac004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.520+0000 7f51bf7fe700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51ac005000 0x7f51ac00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c88c0 con 0x7f51ac004690 2023-05-16T07:14:19.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac00bf60 con 0x7f51ac005000 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac00c0f0 con 0x7f51ac0038e0 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51ac004690 0x7f51ac004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c4e16700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac0038e0 0x7f51ac003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c5617700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51ac005000 0x7f51ac00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 -- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac0038e0 msgr2=0x7f51ac003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac0038e0 0x7f51ac003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 -- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51ac005000 msgr2=0x7f51ac00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51ac005000 0x7f51ac00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 -- 172.21.15.92:0/2197613395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0033f0 con 0x7f51ac004690 2023-05-16T07:14:19.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c5617700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51ac005000 0x7f51ac00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:19.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c4e16700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac0038e0 0x7f51ac003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:19.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c5617700 1 -- 172.21.15.92:0/2197613395 reap_dead start 2023-05-16T07:14:19.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bffff700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51ac004690 0x7f51ac004b10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f51a8009a80 tx=0x7f51a8009060).ready entity=mon.0 client_cookie=8a4213c4c7399da9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8009430 con 0x7f51ac004690 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800e640 con 0x7f51ac004690 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51bf7fe700 1 -- 172.21.15.92:0/2197613395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800e9c0 con 0x7f51ac004690 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c707a700 1 -- 172.21.15.92:0/2197613395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51ac004690 msgr2=0x7f51ac004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.521+0000 7f51c707a700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51ac004690 0x7f51ac004b10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f51a8009a80 tx=0x7f51a8009060).stop 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.522+0000 7f51c707a700 1 -- 172.21.15.92:0/2197613395 shutdown_connections 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.522+0000 7f51c707a700 1 --2- 172.21.15.92:0/2197613395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51ac004690 0x7f51ac004b10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.522+0000 7f51c707a700 1 -- 172.21.15.92:0/2197613395 >> 172.21.15.92:0/2197613395 conn(0x7f51c0067920 msgr2=0x7f51c00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:19.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.522+0000 7f51c707a700 1 -- 172.21.15.92:0/2197613395 shutdown_connections 2023-05-16T07:14:19.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.522+0000 7f51c707a700 1 -- 172.21.15.92:0/2197613395 wait complete. 2023-05-16T07:14:19.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.523+0000 7f51c707a700 1 Processor -- start 2023-05-16T07:14:19.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.523+0000 7f51c707a700 1 -- start start 2023-05-16T07:14:19.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.523+0000 7f51c707a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac004690 0x7f51c00b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 0x7f51c01675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51c0167ad0 0x7f51c014ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0165b30 con 0x7f51c0164960 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00b0c50 con 0x7f51c0167ad0 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0164e60 con 0x7f51ac004690 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 0x7f51c01675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 0x7f51c01675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42974/0 (socket says 172.21.15.92:42974) 2023-05-16T07:14:19.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 -- 172.21.15.92:0/214444646 learned_addr learned my addr 172.21.15.92:0/214444646 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c5617700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51c0167ad0 0x7f51c014ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c4e16700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac004690 0x7f51c00b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 -- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac004690 msgr2=0x7f51c00b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac004690 0x7f51c00b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 -- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51c0167ad0 msgr2=0x7f51c014ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51c0167ad0 0x7f51c014ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 -- 172.21.15.92:0/214444646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0165140 con 0x7f51c0164960 2023-05-16T07:14:19.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c5617700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51c0167ad0 0x7f51c014ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:19.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bffff700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 0x7f51c01675e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f51a800e4b0 tx=0x7f51a801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8000cb0 con 0x7f51c0164960 2023-05-16T07:14:19.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800e7b0 con 0x7f51c0164960 2023-05-16T07:14:19.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c0165420 con 0x7f51c0164960 2023-05-16T07:14:19.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80243f0 con 0x7f51c0164960 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.524+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c00ade20 con 0x7f51c0164960 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.525+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c0066730 con 0x7f51c0164960 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.526+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f51a802b030 con 0x7f51c0164960 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.526+0000 7f51bcff9700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f51a005ee70 0x7f51a0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.526+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f51a807b100 con 0x7f51c0164960 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.526+0000 7f51c4e16700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f51a005ee70 0x7f51a0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.527+0000 7f51c4e16700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f51a005ee70 0x7f51a0061330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f51b0007fd0 tx=0x7f51b0009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.530+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f51a804c5b0 con 0x7f51c0164960 2023-05-16T07:14:19.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.718+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51c006c350 con 0x7f51a005ee70 2023-05-16T07:14:19.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.723+0000 7f51bcff9700 1 -- 172.21.15.92:0/214444646 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f51c006c350 con 0x7f51a005ee70 2023-05-16T07:14:19.727 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-05-16T07:14:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f51a005ee70 msgr2=0x7f51a0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f51a005ee70 0x7f51a0061330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f51b0007fd0 tx=0x7f51b0009040).stop 2023-05-16T07:14:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 msgr2=0x7f51c01675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 0x7f51c01675e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f51a800e4b0 tx=0x7f51a801a000).stop 2023-05-16T07:14:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 shutdown_connections 2023-05-16T07:14:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f51a005ee70 0x7f51a0061330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51ac004690 0x7f51c00b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51c0167ad0 0x7f51c014ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 --2- 172.21.15.92:0/214444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0164960 0x7f51c01675e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 >> 172.21.15.92:0/214444646 conn(0x7f51c0067920 msgr2=0x7f51c014daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 shutdown_connections 2023-05-16T07:14:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.725+0000 7f51c707a700 1 -- 172.21.15.92:0/214444646 wait complete. 2023-05-16T07:14:19.740 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-05-16T07:14:19.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-05-16T07:14:19.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.852+0000 7fafbc309700 1 Processor -- start 2023-05-16T07:14:19.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.852+0000 7fafbc309700 1 -- start start 2023-05-16T07:14:19.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafbc309700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fafb414f990 0x7fafb40c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafbc309700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb4063690 con 0x7fafb4152e40 2023-05-16T07:14:19.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafbc309700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb414fd70 con 0x7fafb40622b0 2023-05-16T07:14:19.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafbc309700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb40c52f0 con 0x7fafb414f990 2023-05-16T07:14:19.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafb98a4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fafb4152e40 0x7fafb40c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53730/0 (socket says 172.21.15.92:53730) 2023-05-16T07:14:19.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafb98a4700 1 -- 172.21.15.92:0/1939167861 learned_addr learned my addr 172.21.15.92:0/1939167861 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:19.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932321198 0 0) 0x7fafb4063690 con 0x7fafb4152e40 2023-05-16T07:14:19.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf9c0033f0 con 0x7fafb4152e40 2023-05-16T07:14:19.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.853+0000 7fafba0a5700 1 --2- 172.21.15.92:0/1939167861 >> v2:172.21.15.158:3300/0 conn(0x7fafb414f990 0x7fafb40c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033622354 0 0) 0x7faf9c0033f0 con 0x7fafb4152e40 2023-05-16T07:14:19.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 >> v2:172.21.15.158:3300/0 conn(0x7fafb414f990 msgr2=0x7fafb40c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 --2- 172.21.15.92:0/1939167861 >> v2:172.21.15.158:3300/0 conn(0x7fafb414f990 0x7fafb40c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 >> v1:172.21.15.158:6789/0 conn(0x7fafb40622b0 legacy=0x7fafb4063b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40c5ce0 con 0x7fafb4152e40 2023-05-16T07:14:19.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fafa8001470 con 0x7fafb4152e40 2023-05-16T07:14:19.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 >> v1:172.21.15.92:6789/0 conn(0x7fafb4152e40 legacy=0x7fafb40c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c0038e0 0x7faf9c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c004690 0x7faf9c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf9c005000 0x7faf9c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40c52f0 con 0x7faf9c004690 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c00beb0 con 0x7faf9c005000 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c00c070 con 0x7faf9c0038e0 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c004690 0x7faf9c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafba8a6700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf9c005000 0x7faf9c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafba0a5700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c0038e0 0x7faf9c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 -- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c0038e0 msgr2=0x7faf9c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c0038e0 0x7faf9c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 -- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf9c005000 msgr2=0x7faf9c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf9c005000 0x7faf9c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 -- 172.21.15.92:0/1939167861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0033f0 con 0x7faf9c004690 2023-05-16T07:14:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafba8a6700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf9c005000 0x7faf9c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafba8a6700 1 -- 172.21.15.92:0/1939167861 reap_dead start 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb98a4700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c004690 0x7faf9c004b10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fafa800c190 tx=0x7fafa8001e50).ready entity=mon.0 client_cookie=fb96051d7dff673a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8007a20 con 0x7faf9c004690 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa800cac0 con 0x7faf9c004690 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.854+0000 7fafb90a3700 1 -- 172.21.15.92:0/1939167861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa800ce40 con 0x7faf9c004690 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 -- 172.21.15.92:0/1939167861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c004690 msgr2=0x7faf9c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c004690 0x7faf9c004b10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fafa800c190 tx=0x7fafa8001e50).stop 2023-05-16T07:14:19.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 -- 172.21.15.92:0/1939167861 shutdown_connections 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 --2- 172.21.15.92:0/1939167861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9c004690 0x7faf9c004b10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 -- 172.21.15.92:0/1939167861 >> 172.21.15.92:0/1939167861 conn(0x7fafb405e820 msgr2=0x7fafb40c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 -- 172.21.15.92:0/1939167861 shutdown_connections 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.855+0000 7fafbc309700 1 -- 172.21.15.92:0/1939167861 wait complete. 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.856+0000 7fafbc309700 1 Processor -- start 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.856+0000 7fafbc309700 1 -- start start 2023-05-16T07:14:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c004690 0x7fafb40bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafb4137110 0x7fafb4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 0x7fafb4164240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4136d70 con 0x7fafb4163dc0 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41609b0 con 0x7fafb4137110 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40c09b0 con 0x7faf9c004690 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 0x7fafb4164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 0x7fafb4164240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42994/0 (socket says 172.21.15.92:42994) 2023-05-16T07:14:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 -- 172.21.15.92:0/2435981850 learned_addr learned my addr 172.21.15.92:0/2435981850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafb98a4700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafb4137110 0x7fafb4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba0a5700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c004690 0x7fafb40bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 -- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c004690 msgr2=0x7fafb40bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c004690 0x7fafb40bfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 -- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafb4137110 msgr2=0x7fafb4155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafb4137110 0x7fafb4155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:19.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 -- 172.21.15.92:0/2435981850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40c0110 con 0x7fafb4163dc0 2023-05-16T07:14:19.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafba8a6700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 0x7fafb4164240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fafb4066c70 tx=0x7fafac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac00ecc0 con 0x7fafb4163dc0 2023-05-16T07:14:19.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac00ee20 con 0x7fafb4163dc0 2023-05-16T07:14:19.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb40c03f0 con 0x7fafb4163dc0 2023-05-16T07:14:19.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac014640 con 0x7fafb4163dc0 2023-05-16T07:14:19.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.857+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4155f00 con 0x7fafb4163dc0 2023-05-16T07:14:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.858+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb406c350 con 0x7fafb4163dc0 2023-05-16T07:14:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.858+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fafac01c060 con 0x7fafb4163dc0 2023-05-16T07:14:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.859+0000 7fafb27fc700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafa005ee70 0x7fafa0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.859+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fafac079170 con 0x7fafb4163dc0 2023-05-16T07:14:19.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.859+0000 7fafba0a5700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafa005ee70 0x7fafa0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:19.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.859+0000 7fafba0a5700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafa005ee70 0x7fafa0061330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fafa40078e0 tx=0x7fafa400b040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:19.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:19.863+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fafac04a620 con 0x7fafb4163dc0 2023-05-16T07:14:20.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.049+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb40c5fc0 con 0x7fafa005ee70 2023-05-16T07:14:20.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.053+0000 7fafb27fc700 1 -- 172.21.15.92:0/2435981850 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fafb40c5fc0 con 0x7fafa005ee70 2023-05-16T07:14:20.057 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-05-16T07:14:20.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafa005ee70 msgr2=0x7fafa0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafa005ee70 0x7fafa0061330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fafa40078e0 tx=0x7fafa400b040).stop 2023-05-16T07:14:20.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 msgr2=0x7fafb4164240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 0x7fafb4164240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fafb4066c70 tx=0x7fafac009040).stop 2023-05-16T07:14:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 shutdown_connections 2023-05-16T07:14:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafa005ee70 0x7fafa0061330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf9c004690 0x7fafb40bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb4163dc0 0x7fafb4164240 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 --2- 172.21.15.92:0/2435981850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafb4137110 0x7fafb4155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 >> 172.21.15.92:0/2435981850 conn(0x7fafb405e820 msgr2=0x7fafb4152240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 shutdown_connections 2023-05-16T07:14:20.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.055+0000 7fafbc309700 1 -- 172.21.15.92:0/2435981850 wait complete. 2023-05-16T07:14:20.070 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-05-16T07:14:20.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-05-16T07:14:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.181+0000 7fc17c6db700 1 Processor -- start 2023-05-16T07:14:20.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.181+0000 7fc17c6db700 1 -- start start 2023-05-16T07:14:20.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc17c6db700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 0x7fc1741540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc17c6db700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740bf950 con 0x7fc174063fd0 2023-05-16T07:14:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc17c6db700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740a7c90 con 0x7fc174153d00 2023-05-16T07:14:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc17c6db700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc174067f60 con 0x7fc1740629c0 2023-05-16T07:14:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179c76700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc174063fd0 0x7fc1740622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53748/0 (socket says 172.21.15.92:53748) 2023-05-16T07:14:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179c76700 1 -- 172.21.15.92:0/2296502558 learned_addr learned my addr 172.21.15.92:0/2296502558 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442402903 0 0) 0x7fc1740bf950 con 0x7fc174063fd0 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1600033f0 con 0x7fc174063fd0 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc17ac78700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 0x7fc1741540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8814513 0 0) 0x7fc1600033f0 con 0x7fc174063fd0 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 msgr2=0x7fc1741540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 0x7fc1741540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 >> v1:172.21.15.158:6789/0 conn(0x7fc174153d00 legacy=0x7fc1740bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740a7f00 con 0x7fc174063fd0 2023-05-16T07:14:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc164001470 con 0x7fc174063fd0 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 >> v1:172.21.15.92:6789/0 conn(0x7fc174063fd0 legacy=0x7fc1740622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160003910 0x7fc160003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc160004690 0x7fc160004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 0x7fc160005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000b910 con 0x7fc160004690 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000ba80 con 0x7fc160003910 2023-05-16T07:14:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.182+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000bc40 con 0x7fc1740629c0 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc160004690 0x7fc160004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc17a477700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160003910 0x7fc160003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc17ac78700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 0x7fc160005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 -- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 msgr2=0x7fc160005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740629c0 0x7fc160005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 -- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160003910 msgr2=0x7fc160003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160003910 0x7fc160003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 -- 172.21.15.92:0/2296502558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1600033f0 con 0x7fc160004690 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179c76700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc160004690 0x7fc160004b10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fc16400c190 tx=0x7fc164001e50).ready entity=mon.0 client_cookie=e75e0057fb5ce0e6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164007a20 con 0x7fc160004690 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16400cac0 con 0x7fc160004690 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc179475700 1 -- 172.21.15.92:0/2296502558 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16400ce40 con 0x7fc160004690 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc17c6db700 1 -- 172.21.15.92:0/2296502558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc160004690 msgr2=0x7fc160004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc17c6db700 1 --2- 172.21.15.92:0/2296502558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc160004690 0x7fc160004b10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fc16400c190 tx=0x7fc164001e50).stop 2023-05-16T07:14:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.183+0000 7fc17ac78700 1 -- 172.21.15.92:0/2296502558 reap_dead start 2023-05-16T07:14:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.184+0000 7fc17c6db700 1 -- 172.21.15.92:0/2296502558 shutdown_connections 2023-05-16T07:14:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.184+0000 7fc17c6db700 1 -- 172.21.15.92:0/2296502558 >> 172.21.15.92:0/2296502558 conn(0x7fc17405e820 msgr2=0x7fc1740c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.184+0000 7fc17c6db700 1 -- 172.21.15.92:0/2296502558 shutdown_connections 2023-05-16T07:14:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.184+0000 7fc17c6db700 1 -- 172.21.15.92:0/2296502558 wait complete. 2023-05-16T07:14:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 Processor -- start 2023-05-16T07:14:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 -- start start 2023-05-16T07:14:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160004690 0x7fc174149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 0x7fc1740a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740a9e40 0x7fc1740a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740aaed0 con 0x7fc1740c2980 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740aa2c0 con 0x7fc160004690 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.185+0000 7fc17c6db700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740aa5a0 con 0x7fc1740a9e40 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 0x7fc1740a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 0x7fc1740a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43010/0 (socket says 172.21.15.92:43010) 2023-05-16T07:14:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 -- 172.21.15.92:0/4134395980 learned_addr learned my addr 172.21.15.92:0/4134395980 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc17a477700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160004690 0x7fc174149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc17ac78700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740a9e40 0x7fc1740a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 -- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740a9e40 msgr2=0x7fc1740a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740a9e40 0x7fc1740a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 -- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160004690 msgr2=0x7fc174149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160004690 0x7fc174149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 -- 172.21.15.92:0/4134395980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740aa880 con 0x7fc1740c2980 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc17a477700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160004690 0x7fc174149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc179c76700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 0x7fc1740a9950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc16000ba50 tx=0x7fc164015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640075c0 con 0x7fc1740c2980 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1640046b0 con 0x7fc1740c2980 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1740aaed0 con 0x7fc1740c2980 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164004a30 con 0x7fc1740c2980 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.186+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc174149a60 con 0x7fc1740c2980 2023-05-16T07:14:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.187+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1740ab060 con 0x7fc1740c2980 2023-05-16T07:14:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.187+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fc164010070 con 0x7fc1740c2980 2023-05-16T07:14:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.188+0000 7fc16a7fc700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc15805ee70 0x7fc158061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.188+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fc16407ab40 con 0x7fc1740c2980 2023-05-16T07:14:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.188+0000 7fc17a477700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc15805ee70 0x7fc158061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.188+0000 7fc17a477700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc15805ee70 0x7fc158061330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc16c000f80 tx=0x7fc16c007f30).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.192+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc16404bff0 con 0x7fc1740c2980 2023-05-16T07:14:20.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.378+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc17406c350 con 0x7fc15805ee70 2023-05-16T07:14:20.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.385+0000 7fc16a7fc700 1 -- 172.21.15.92:0/4134395980 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc17406c350 con 0x7fc15805ee70 2023-05-16T07:14:20.389 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-05-16T07:14:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc15805ee70 msgr2=0x7fc158061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc15805ee70 0x7fc158061330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc16c000f80 tx=0x7fc16c007f30).stop 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 msgr2=0x7fc1740a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 0x7fc1740a9950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc16000ba50 tx=0x7fc164015000).stop 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 shutdown_connections 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc15805ee70 0x7fc158061330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1740a9e40 0x7fc1740a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc160004690 0x7fc174149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 --2- 172.21.15.92:0/4134395980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1740c2980 0x7fc1740a9950 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.386+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 >> 172.21.15.92:0/4134395980 conn(0x7fc17405e820 msgr2=0x7fc1740682b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.387+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 shutdown_connections 2023-05-16T07:14:20.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.387+0000 7fc17c6db700 1 -- 172.21.15.92:0/4134395980 wait complete. 2023-05-16T07:14:20.402 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-05-16T07:14:20.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-05-16T07:14:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.512+0000 7f4752c49700 1 Processor -- start 2023-05-16T07:14:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.512+0000 7f4752c49700 1 -- start start 2023-05-16T07:14:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752c49700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c153f90 0x7f474c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752c49700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f474c0c5d00 0x7f474c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752c49700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c0c6670 con 0x7f474c0c0150 2023-05-16T07:14:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752c49700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c069f60 con 0x7f474c0c5d00 2023-05-16T07:14:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752c49700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c06a9e0 con 0x7f474c153f90 2023-05-16T07:14:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4751446700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f474c0c5d00 0x7f474c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752448700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f474c0c0150 0x7f474c0c05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53768/0 (socket says 172.21.15.92:53768) 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4751446700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f474c0c5d00 0x7f474c0c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43020/0 (socket says 172.21.15.92:43020) 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4752448700 1 -- 172.21.15.92:0/2660204880 learned_addr learned my addr 172.21.15.92:0/2660204880 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.513+0000 7f4751c47700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c153f90 0x7f474c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4751446700 1 -- 172.21.15.92:0/2660204880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c153f90 msgr2=0x7f474c0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4751446700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c153f90 0x7f474c0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4751446700 1 -- 172.21.15.92:0/2660204880 >> v1:172.21.15.92:6789/0 conn(0x7f474c0c0150 legacy=0x7f474c0c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4751446700 1 -- 172.21.15.92:0/2660204880 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c06a140 con 0x7f474c0c5d00 2023-05-16T07:14:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4750c45700 1 -- 172.21.15.92:0/2660204880 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916638439 0 0) 0x7f474c0c6670 con 0x7f474c0c0150 2023-05-16T07:14:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4751c47700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c153f90 0x7f474c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4751446700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c0c5d00 0x7f474c0c6180 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f4740009760 tx=0x7f4740009060).ready entity=mon.0 client_cookie=8577fba2fa757a91 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4750c45700 1 -- 172.21.15.92:0/2660204880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000b580 con 0x7f474c0c5d00 2023-05-16T07:14:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4750c45700 1 -- 172.21.15.92:0/2660204880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474000b6e0 con 0x7f474c0c5d00 2023-05-16T07:14:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4750c45700 1 -- 172.21.15.92:0/2660204880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000ba60 con 0x7f474c0c5d00 2023-05-16T07:14:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 -- 172.21.15.92:0/2660204880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c0c5d00 msgr2=0x7f474c0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c0c5d00 0x7f474c0c6180 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f4740009760 tx=0x7f4740009060).stop 2023-05-16T07:14:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 -- 172.21.15.92:0/2660204880 shutdown_connections 2023-05-16T07:14:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c153f90 0x7f474c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 --2- 172.21.15.92:0/2660204880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c0c5d00 0x7f474c0c6180 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 -- 172.21.15.92:0/2660204880 >> 172.21.15.92:0/2660204880 conn(0x7f474c0cb600 msgr2=0x7f474c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 -- 172.21.15.92:0/2660204880 shutdown_connections 2023-05-16T07:14:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.514+0000 7f4752c49700 1 -- 172.21.15.92:0/2660204880 wait complete. 2023-05-16T07:14:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.515+0000 7f4752c49700 1 Processor -- start 2023-05-16T07:14:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.515+0000 7f4752c49700 1 -- start start 2023-05-16T07:14:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752c49700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c0c0150 0x7f474c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752c49700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f474c0c5d00 0x7f474c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752c49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 0x7f474c167860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752c49700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c0aeee0 con 0x7f474c153f90 2023-05-16T07:14:20.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752c49700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c0ae2d0 con 0x7f474c0c5d00 2023-05-16T07:14:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752c49700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c0ae5b0 con 0x7f474c0c0150 2023-05-16T07:14:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752448700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 0x7f474c167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752448700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 0x7f474c167860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43026/0 (socket says 172.21.15.92:43026) 2023-05-16T07:14:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4752448700 1 -- 172.21.15.92:0/2031496039 learned_addr learned my addr 172.21.15.92:0/2031496039 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4751c47700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c0c0150 0x7f474c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.516+0000 7f4751446700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f474c0c5d00 0x7f474c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752448700 1 -- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c0c0150 msgr2=0x7f474c0c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752448700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c0c0150 0x7f474c0c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752448700 1 -- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f474c0c5d00 msgr2=0x7f474c0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752448700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f474c0c5d00 0x7f474c0adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752448700 1 -- 172.21.15.92:0/2031496039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400092a0 con 0x7f474c153f90 2023-05-16T07:14:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752448700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 0x7f474c167860 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f474400c490 tx=0x7f4744007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47440072e0 con 0x7f474c153f90 2023-05-16T07:14:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744007440 con 0x7f474c153f90 2023-05-16T07:14:20.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c0ae890 con 0x7f474c153f90 2023-05-16T07:14:20.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47440057a0 con 0x7f474c153f90 2023-05-16T07:14:20.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.517+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c0adfd0 con 0x7f474c153f90 2023-05-16T07:14:20.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.518+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f4744010070 con 0x7f474c153f90 2023-05-16T07:14:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.518+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474c083120 con 0x7f474c153f90 2023-05-16T07:14:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.518+0000 7f4749ffb700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f473805ee70 0x7f4738061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.518+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f4744079380 con 0x7f474c153f90 2023-05-16T07:14:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.519+0000 7f4751c47700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f473805ee70 0x7f4738061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.519+0000 7f4751c47700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f473805ee70 0x7f4738061330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f473c007600 tx=0x7f473c009850).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.522+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f474404a830 con 0x7f474c153f90 2023-05-16T07:14:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.712+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f474c047150 con 0x7f473805ee70 2023-05-16T07:14:20.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.716+0000 7f4749ffb700 1 -- 172.21.15.92:0/2031496039 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f474c047150 con 0x7f473805ee70 2023-05-16T07:14:20.720 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-05-16T07:14:20.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f473805ee70 msgr2=0x7f4738061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f473805ee70 0x7f4738061330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f473c007600 tx=0x7f473c009850).stop 2023-05-16T07:14:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 msgr2=0x7f474c167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 0x7f474c167860 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f474400c490 tx=0x7f4744007c10).stop 2023-05-16T07:14:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 shutdown_connections 2023-05-16T07:14:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f473805ee70 0x7f4738061330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f474c0c0150 0x7f474c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f474c153f90 0x7f474c167860 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 --2- 172.21.15.92:0/2031496039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f474c0c5d00 0x7f474c0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 >> 172.21.15.92:0/2031496039 conn(0x7f474c0cb600 msgr2=0x7f474c1510c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 shutdown_connections 2023-05-16T07:14:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.718+0000 7f4752c49700 1 -- 172.21.15.92:0/2031496039 wait complete. 2023-05-16T07:14:20.733 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-05-16T07:14:20.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-05-16T07:14:20.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 Processor -- start 2023-05-16T07:14:20.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 -- start start 2023-05-16T07:14:20.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8158064b90 0x7f81580bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8158064f70 0x7f81580653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81580669c0 con 0x7f8158064b90 2023-05-16T07:14:20.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8158150890 con 0x7f8158064f70 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.846+0000 7f815d830700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81580bcf30 con 0x7f815806a410 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8158064b90 0x7f81580bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8158064b90 0x7f81580bc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43030/0 (socket says 172.21.15.92:43030) 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 -- 172.21.15.92:0/705911846 learned_addr learned my addr 172.21.15.92:0/705911846 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f81577fe700 1 --2- 172.21.15.92:0/705911846 >> v2:172.21.15.158:3300/0 conn(0x7f8158064f70 0x7f81580653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 -- 172.21.15.92:0/705911846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 msgr2=0x7f81580664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f81567fc700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 -- 172.21.15.92:0/705911846 >> v2:172.21.15.158:3300/0 conn(0x7f8158064f70 msgr2=0x7f81580653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 --2- 172.21.15.92:0/705911846 >> v2:172.21.15.158:3300/0 conn(0x7f8158064f70 0x7f81580653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 -- 172.21.15.92:0/705911846 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8158069930 con 0x7f8158064b90 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f81577fe700 1 --2- 172.21.15.92:0/705911846 >> v2:172.21.15.158:3300/0 conn(0x7f8158064f70 0x7f81580653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f81567fc700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8156ffd700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f81580bc8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f81580c39a0 tx=0x7f8148009840).ready entity=mon.0 client_cookie=6f563a72ced778b1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8155ffb700 1 -- 172.21.15.92:0/705911846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814800e070 con 0x7f8158064b90 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8155ffb700 1 -- 172.21.15.92:0/705911846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814800ba50 con 0x7f8158064b90 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f8155ffb700 1 -- 172.21.15.92:0/705911846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814800bdd0 con 0x7f8158064b90 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.847+0000 7f815d830700 1 -- 172.21.15.92:0/705911846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 msgr2=0x7f81580bc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f81580bc8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f81580c39a0 tx=0x7f8148009840).stop 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 -- 172.21.15.92:0/705911846 shutdown_connections 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 --2- 172.21.15.92:0/705911846 >> v2:172.21.15.158:3300/0 conn(0x7f8158064f70 0x7f81580653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f81580bc8a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 --2- 172.21.15.92:0/705911846 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 -- 172.21.15.92:0/705911846 >> 172.21.15.92:0/705911846 conn(0x7f81580c4450 msgr2=0x7f8158066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:20.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 -- 172.21.15.92:0/705911846 shutdown_connections 2023-05-16T07:14:20.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.848+0000 7f815d830700 1 -- 172.21.15.92:0/705911846 wait complete. 2023-05-16T07:14:20.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.849+0000 7f815d830700 1 Processor -- start 2023-05-16T07:14:20.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.849+0000 7f815d830700 1 -- start start 2023-05-16T07:14:20.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f8158145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8158064f70 0x7f81580a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81580a6570 con 0x7f8158064b90 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8158145f70 con 0x7f8158064f70 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81580a4de0 con 0x7f815806a410 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f8158145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f8158145a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43036/0 (socket says 172.21.15.92:43036) 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 -- 172.21.15.92:0/702033965 learned_addr learned my addr 172.21.15.92:0/702033965 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 -- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 msgr2=0x7f81580a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f81577fe700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f81567fc700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8158064f70 0x7f81580a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 -- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8158064f70 msgr2=0x7f81580a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8158064f70 0x7f81580a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:20.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 -- 172.21.15.92:0/702033965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148009ae0 con 0x7f8158064b90 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f81577fe700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f81567fc700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8158064f70 0x7f81580a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f8156ffd700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f8158145a80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f8148003350 tx=0x7f8148009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814800bbc0 con 0x7f8158064b90 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814800bd20 con 0x7f8158064b90 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81580beaa0 con 0x7f8158064b90 2023-05-16T07:14:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814801e680 con 0x7f8158064b90 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.850+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81580be450 con 0x7f8158064b90 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.851+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8158047150 con 0x7f8158064b90 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.852+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f8148025030 con 0x7f8158064b90 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.852+0000 7f814effd700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f813805ee70 0x7f8138061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.852+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f814807b3f0 con 0x7f8158064b90 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.852+0000 7f81567fc700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f813805ee70 0x7f8138061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.853+0000 7f81567fc700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f813805ee70 0x7f8138061330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f81580a66d0 tx=0x7f8140008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:20.856+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f814804c950 con 0x7f8158064b90 2023-05-16T07:14:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.047+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8158068810 con 0x7f813805ee70 2023-05-16T07:14:21.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.051+0000 7f814effd700 1 -- 172.21.15.92:0/702033965 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8158068810 con 0x7f813805ee70 2023-05-16T07:14:21.055 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-05-16T07:14:21.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f813805ee70 msgr2=0x7f8138061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f813805ee70 0x7f8138061330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f81580a66d0 tx=0x7f8140008040).stop 2023-05-16T07:14:21.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 msgr2=0x7f8158145a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f8158145a80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f8148003350 tx=0x7f8148009840).stop 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 shutdown_connections 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f813805ee70 0x7f8138061330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f815806a410 0x7f81580a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158064b90 0x7f8158145a80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 --2- 172.21.15.92:0/702033965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8158064f70 0x7f81580a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 >> 172.21.15.92:0/702033965 conn(0x7f81580c4450 msgr2=0x7f81580653d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:21.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 shutdown_connections 2023-05-16T07:14:21.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.053+0000 7f815d830700 1 -- 172.21.15.92:0/702033965 wait complete. 2023-05-16T07:14:21.068 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-05-16T07:14:21.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-05-16T07:14:21.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.178+0000 7f53808ad700 1 Processor -- start 2023-05-16T07:14:21.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.179+0000 7f53808ad700 1 -- start start 2023-05-16T07:14:21.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.179+0000 7f53808ad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f537c154090 0x7f537c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.179+0000 7f53808ad700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0c94d0 con 0x7f537c154470 2023-05-16T07:14:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.179+0000 7f53808ad700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0cb7e0 con 0x7f537c068160 2023-05-16T07:14:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.179+0000 7f53808ad700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0c88c0 con 0x7f537c154090 2023-05-16T07:14:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f537c154090 0x7f537c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f537c154090 0x7f537c0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43044/0 (socket says 172.21.15.92:43044) 2023-05-16T07:14:21.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f5379d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f537c154470 0x7f537c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53796/0 (socket says 172.21.15.92:53796) 2023-05-16T07:14:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 -- 172.21.15.92:0/2446657495 learned_addr learned my addr 172.21.15.92:0/2446657495 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 -- 172.21.15.92:0/2446657495 >> v1:172.21.15.158:6789/0 conn(0x7f537c068160 legacy=0x7f537c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 -- 172.21.15.92:0/2446657495 >> v1:172.21.15.92:6789/0 conn(0x7f537c154470 legacy=0x7f537c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 -- 172.21.15.92:0/2446657495 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0c8ba0 con 0x7f537c154090 2023-05-16T07:14:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537959a700 1 -- 172.21.15.92:0/2446657495 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450905184 0 0) 0x7f537c0c94d0 con 0x7f537c154470 2023-05-16T07:14:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537a59c700 1 --2- 172.21.15.92:0/2446657495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c0aaca0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f5374006fd0 tx=0x7f53740090f0).ready entity=mon.0 client_cookie=a992627eecbcc16 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537959a700 1 -- 172.21.15.92:0/2446657495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400b600 con 0x7f537c154090 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537959a700 1 -- 172.21.15.92:0/2446657495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537400b760 con 0x7f537c154090 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.180+0000 7f537959a700 1 -- 172.21.15.92:0/2446657495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400bae0 con 0x7f537c154090 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 -- 172.21.15.92:0/2446657495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 msgr2=0x7f537c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 --2- 172.21.15.92:0/2446657495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c0aaca0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f5374006fd0 tx=0x7f53740090f0).stop 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 -- 172.21.15.92:0/2446657495 shutdown_connections 2023-05-16T07:14:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 --2- 172.21.15.92:0/2446657495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c0aaca0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 -- 172.21.15.92:0/2446657495 >> 172.21.15.92:0/2446657495 conn(0x7f537c067920 msgr2=0x7f537c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 -- 172.21.15.92:0/2446657495 shutdown_connections 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.181+0000 7f53808ad700 1 -- 172.21.15.92:0/2446657495 wait complete. 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.182+0000 7f53808ad700 1 Processor -- start 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.182+0000 7f53808ad700 1 -- start start 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f537c068160 0x7f537c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f537c154470 0x7f537c154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0af930 con 0x7f537c154090 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0adbe0 con 0x7f537c154470 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0adec0 con 0x7f537c068160 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c14d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43052/0 (socket says 172.21.15.92:43052) 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 -- 172.21.15.92:0/497987270 learned_addr learned my addr 172.21.15.92:0/497987270 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:21.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 -- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f537c068160 msgr2=0x7f537c0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f537c068160 0x7f537c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 -- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f537c154470 msgr2=0x7f537c154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f537c154470 0x7f537c154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 -- 172.21.15.92:0/497987270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374009370 con 0x7f537c154090 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f5379d9b700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c14d870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f537000c440 tx=0x7f5370007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700072e0 con 0x7f537c154090 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5370007440 con 0x7f537c154090 2023-05-16T07:14:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c155130 con 0x7f537c154090 2023-05-16T07:14:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370005910 con 0x7f537c154090 2023-05-16T07:14:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.183+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1556f0 con 0x7f537c154090 2023-05-16T07:14:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.184+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c066730 con 0x7f537c154090 2023-05-16T07:14:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.184+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f5370010070 con 0x7f537c154090 2023-05-16T07:14:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.185+0000 7f536a7fc700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f535c05ee70 0x7f535c061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.185+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f5370079260 con 0x7f537c154090 2023-05-16T07:14:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.185+0000 7f537a59c700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f535c05ee70 0x7f535c061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.186+0000 7f537a59c700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f535c05ee70 0x7f535c061330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f537c0c4b10 tx=0x7f5374017000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.189+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f537004a710 con 0x7f537c154090 2023-05-16T07:14:21.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.375+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f537c0ae030 con 0x7f535c05ee70 2023-05-16T07:14:21.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.379+0000 7f536a7fc700 1 -- 172.21.15.92:0/497987270 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f537c0ae030 con 0x7f535c05ee70 2023-05-16T07:14:21.383 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-05-16T07:14:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.381+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f535c05ee70 msgr2=0x7f535c061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.381+0000 7f53808ad700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f535c05ee70 0x7f535c061330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f537c0c4b10 tx=0x7f5374017000).stop 2023-05-16T07:14:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.381+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 msgr2=0x7f537c14d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.381+0000 7f53808ad700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c14d870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f537000c440 tx=0x7f5370007e30).stop 2023-05-16T07:14:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 shutdown_connections 2023-05-16T07:14:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f535c05ee70 0x7f535c061330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f537c068160 0x7f537c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:21.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f537c154470 0x7f537c154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:21.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 --2- 172.21.15.92:0/497987270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f537c154090 0x7f537c14d870 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 >> 172.21.15.92:0/497987270 conn(0x7f537c067920 msgr2=0x7f537c0ac430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:21.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 shutdown_connections 2023-05-16T07:14:21.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.382+0000 7f53808ad700 1 -- 172.21.15.92:0/497987270 wait complete. 2023-05-16T07:14:21.397 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-05-16T07:14:21.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-05-16T07:14:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.508+0000 7f8257e92700 1 Processor -- start 2023-05-16T07:14:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.508+0000 7f8257e92700 1 -- start start 2023-05-16T07:14:21.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f8257e92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82500629c0 0x7f8250063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f8257e92700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825014ead0 con 0x7f8250155140 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f8257e92700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8250154cb0 con 0x7f825014e6f0 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f8257e92700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500c6020 con 0x7f82500629c0 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f825542d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8250155140 0x7f82500622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53808/0 (socket says 172.21.15.92:53808) 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f825642f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82500629c0 0x7f8250063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f825542d700 1 -- 172.21.15.92:0/490612058 learned_addr learned my addr 172.21.15.92:0/490612058 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f8254c2c700 1 -- 172.21.15.92:0/490612058 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749393973 0 0) 0x7f825014ead0 con 0x7f8250155140 2023-05-16T07:14:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f8254c2c700 1 -- 172.21.15.92:0/490612058 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82380033f0 con 0x7f8250155140 2023-05-16T07:14:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f825642f700 1 -- 172.21.15.92:0/490612058 >> v1:172.21.15.158:6789/0 conn(0x7f825014e6f0 legacy=0x7f82500c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f825642f700 1 -- 172.21.15.92:0/490612058 >> v1:172.21.15.92:6789/0 conn(0x7f8250155140 legacy=0x7f82500622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.509+0000 7f825642f700 1 -- 172.21.15.92:0/490612058 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250154660 con 0x7f82500629c0 2023-05-16T07:14:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f825642f700 1 --2- 172.21.15.92:0/490612058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82500629c0 0x7f8250063600 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f8248009760 tx=0x7f8248009060).ready entity=mon.0 client_cookie=9d402784c8c6e84a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8254c2c700 1 -- 172.21.15.92:0/490612058 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800f510 con 0x7f82500629c0 2023-05-16T07:14:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8254c2c700 1 -- 172.21.15.92:0/490612058 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824800f670 con 0x7f82500629c0 2023-05-16T07:14:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8254c2c700 1 -- 172.21.15.92:0/490612058 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800f9f0 con 0x7f82500629c0 2023-05-16T07:14:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 -- 172.21.15.92:0/490612058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82500629c0 msgr2=0x7f8250063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 --2- 172.21.15.92:0/490612058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82500629c0 0x7f8250063600 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f8248009760 tx=0x7f8248009060).stop 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 -- 172.21.15.92:0/490612058 shutdown_connections 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 --2- 172.21.15.92:0/490612058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82500629c0 0x7f8250063600 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 -- 172.21.15.92:0/490612058 >> 172.21.15.92:0/490612058 conn(0x7f825005e820 msgr2=0x7f82500c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 -- 172.21.15.92:0/490612058 shutdown_connections 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.510+0000 7f8257e92700 1 -- 172.21.15.92:0/490612058 wait complete. 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.511+0000 7f8257e92700 1 Processor -- start 2023-05-16T07:14:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.511+0000 7f8257e92700 1 -- start start 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8257e92700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82500629c0 0x7f82500ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8257e92700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f825014e6f0 0x7f82500ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8257e92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 0x7f82500aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8257e92700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82500abe30 con 0x7f8250155140 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8257e92700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825014df20 con 0x7f825014e6f0 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8257e92700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825014d960 con 0x7f82500629c0 2023-05-16T07:14:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 0x7f82500aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 0x7f82500aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43068/0 (socket says 172.21.15.92:43068) 2023-05-16T07:14:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 -- 172.21.15.92:0/863831767 learned_addr learned my addr 172.21.15.92:0/863831767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825542d700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f825014e6f0 0x7f82500ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f8255c2e700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82500629c0 0x7f82500ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 -- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82500629c0 msgr2=0x7f82500ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82500629c0 0x7f82500ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 -- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f825014e6f0 msgr2=0x7f82500ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f825014e6f0 0x7f82500ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.512+0000 7f825642f700 1 -- 172.21.15.92:0/863831767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82480092a0 con 0x7f8250155140 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.513+0000 7f825642f700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 0x7f82500aa4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f8248009e00 tx=0x7f8248017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.513+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800f9f0 con 0x7f8250155140 2023-05-16T07:14:21.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.513+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824801b440 con 0x7f8250155140 2023-05-16T07:14:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.513+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825014dc40 con 0x7f8250155140 2023-05-16T07:14:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.513+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824801b800 con 0x7f8250155140 2023-05-16T07:14:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.513+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82500ad980 con 0x7f8250155140 2023-05-16T07:14:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.514+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825006c350 con 0x7f8250155140 2023-05-16T07:14:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.514+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f824802a070 con 0x7f8250155140 2023-05-16T07:14:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.514+0000 7f824dffb700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f823c05ee70 0x7f823c061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.514+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f824807b490 con 0x7f8250155140 2023-05-16T07:14:21.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.515+0000 7f8255c2e700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f823c05ee70 0x7f823c061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.515+0000 7f8255c2e700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f823c05ee70 0x7f823c061330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8240003dd0 tx=0x7f8240009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.518+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f824804c940 con 0x7f8250155140 2023-05-16T07:14:21.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.712+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f82500622b0 con 0x7f823c05ee70 2023-05-16T07:14:21.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.717+0000 7f824dffb700 1 -- 172.21.15.92:0/863831767 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f82500622b0 con 0x7f823c05ee70 2023-05-16T07:14:21.721 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-05-16T07:14:21.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f823c05ee70 msgr2=0x7f823c061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f823c05ee70 0x7f823c061330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8240003dd0 tx=0x7f8240009040).stop 2023-05-16T07:14:21.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 msgr2=0x7f82500aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 0x7f82500aa4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f8248009e00 tx=0x7f8248017000).stop 2023-05-16T07:14:21.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 shutdown_connections 2023-05-16T07:14:21.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f823c05ee70 0x7f823c061330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82500629c0 0x7f82500ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8250155140 0x7f82500aa4a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 --2- 172.21.15.92:0/863831767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f825014e6f0 0x7f82500ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 >> 172.21.15.92:0/863831767 conn(0x7f825005e820 msgr2=0x7f8250067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:21.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 shutdown_connections 2023-05-16T07:14:21.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.719+0000 7f8257e92700 1 -- 172.21.15.92:0/863831767 wait complete. 2023-05-16T07:14:21.734 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-05-16T07:14:21.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-05-16T07:14:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.846+0000 7f4c02f5c700 1 Processor -- start 2023-05-16T07:14:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 -- start start 2023-05-16T07:14:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4bfc153ee0 0x7f4bfc1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4bfc062cc0 0x7f4bfc066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc0bdb70 0x7f4bfc0be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc067340 con 0x7f4bfc062cc0 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc155670 con 0x7f4bfc153ee0 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4c02f5c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc155870 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.847+0000 7f4bfbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4bfc062cc0 0x7f4bfc066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4bfc062cc0 0x7f4bfc066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43076/0 (socket says 172.21.15.92:43076) 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2084408579 learned_addr learned my addr 172.21.15.92:0/2084408579 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4c00cf8700 1 --2- 172.21.15.92:0/2084408579 >> v2:172.21.15.158:3300/0 conn(0x7f4bfc153ee0 0x7f4bfc1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4c014f9700 1 --2- 172.21.15.92:0/2084408579 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc0bdb70 0x7f4bfc0be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2084408579 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc0bdb70 msgr2=0x7f4bfc0be000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2084408579 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc0bdb70 0x7f4bfc0be000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2084408579 >> v2:172.21.15.158:3300/0 conn(0x7f4bfc153ee0 msgr2=0x7f4bfc1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2084408579 >> v2:172.21.15.158:3300/0 conn(0x7f4bfc153ee0 0x7f4bfc1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2084408579 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc063120 con 0x7f4bfc062cc0 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2084408579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc066e50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f4bfc0c7860 tx=0x7f4bf00090f0).ready entity=mon.0 client_cookie=44cc44516a027f94 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfb7fe700 1 -- 172.21.15.92:0/2084408579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000b600 con 0x7f4bfc062cc0 2023-05-16T07:14:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfb7fe700 1 -- 172.21.15.92:0/2084408579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000b760 con 0x7f4bfc062cc0 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4bfb7fe700 1 -- 172.21.15.92:0/2084408579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000bae0 con 0x7f4bfc062cc0 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4c02f5c700 1 -- 172.21.15.92:0/2084408579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc062cc0 msgr2=0x7f4bfc066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/2084408579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc066e50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f4bfc0c7860 tx=0x7f4bf00090f0).stop 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.848+0000 7f4c02f5c700 1 -- 172.21.15.92:0/2084408579 shutdown_connections 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.849+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/2084408579 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc0bdb70 0x7f4bfc0be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.849+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/2084408579 >> v2:172.21.15.158:3300/0 conn(0x7f4bfc153ee0 0x7f4bfc1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.849+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/2084408579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc066e50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.849+0000 7f4c02f5c700 1 -- 172.21.15.92:0/2084408579 >> 172.21.15.92:0/2084408579 conn(0x7f4bfc0c8420 msgr2=0x7f4bfc0c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:21.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.849+0000 7f4c02f5c700 1 -- 172.21.15.92:0/2084408579 shutdown_connections 2023-05-16T07:14:21.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.849+0000 7f4c02f5c700 1 -- 172.21.15.92:0/2084408579 wait complete. 2023-05-16T07:14:21.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.850+0000 7f4c02f5c700 1 Processor -- start 2023-05-16T07:14:21.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.850+0000 7f4c02f5c700 1 -- start start 2023-05-16T07:14:21.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.850+0000 7f4c02f5c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc1649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 0x7f4bfc0a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc153ee0 0x7f4bfc166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc164e90 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0a9330 con 0x7f4bfc062cc0 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0a9610 con 0x7f4bfc153ee0 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 0x7f4bfc0a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 0x7f4bfc0a8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43080/0 (socket says 172.21.15.92:43080) 2023-05-16T07:14:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 -- 172.21.15.92:0/1405679240 learned_addr learned my addr 172.21.15.92:0/1405679240 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c00cf8700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc1649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c014f9700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc153ee0 0x7f4bfc166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 -- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc153ee0 msgr2=0x7f4bfc166ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc153ee0 0x7f4bfc166ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 -- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bfc062cc0 msgr2=0x7f4bfc1649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc1649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 -- 172.21.15.92:0/1405679240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0009370 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c014f9700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc153ee0 0x7f4bfc166ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 0x7f4bfc0a8ac0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4bfc06a4b0 tx=0x7f4bf0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000b8d0 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000ba30 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc0a98f0 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf001c670 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.851+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc0a9eb0 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.852+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc06c350 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.853+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f4bf002c070 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.853+0000 7f4bf8ff9700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805ee70 0x7f4be8061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.853+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f4bf007b7c0 con 0x7f4bfc0bdb70 2023-05-16T07:14:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.853+0000 7f4c00cf8700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805ee70 0x7f4be8061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.854+0000 7f4c00cf8700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805ee70 0x7f4be8061330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4bec006fd0 tx=0x7f4bec008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:21.857+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4bf004cc70 con 0x7f4bfc0bdb70 2023-05-16T07:14:22.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.047+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bfc063120 con 0x7f4be805ee70 2023-05-16T07:14:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.051+0000 7f4bf8ff9700 1 -- 172.21.15.92:0/1405679240 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4bfc063120 con 0x7f4be805ee70 2023-05-16T07:14:22.055 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-05-16T07:14:22.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805ee70 msgr2=0x7f4be8061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805ee70 0x7f4be8061330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4bec006fd0 tx=0x7f4bec008040).stop 2023-05-16T07:14:22.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 msgr2=0x7f4bfc0a8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 0x7f4bfc0a8ac0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4bfc06a4b0 tx=0x7f4bf0017000).stop 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 shutdown_connections 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805ee70 0x7f4be8061330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc153ee0 0x7f4bfc166ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bfc062cc0 0x7f4bfc1649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 --2- 172.21.15.92:0/1405679240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc0bdb70 0x7f4bfc0a8ac0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 >> 172.21.15.92:0/1405679240 conn(0x7f4bfc0c8420 msgr2=0x7f4bfc0bd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:22.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 shutdown_connections 2023-05-16T07:14:22.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.053+0000 7f4c02f5c700 1 -- 172.21.15.92:0/1405679240 wait complete. 2023-05-16T07:14:22.068 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-05-16T07:14:22.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:22.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-05-16T07:14:22.085 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:22.085 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:22.085 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:22.085 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:22.085 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:22.085 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:22.086 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:22.086 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:22.086 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:22.086 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:22.086 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:22.086 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:22.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-05-16T07:14:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.235+0000 7fda53fff700 1 Processor -- start 2023-05-16T07:14:22.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.235+0000 7fda53fff700 1 -- start start 2023-05-16T07:14:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda53fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda54153f90 0x7fda540a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda53fff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fda540c5d00 0x7fda540c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda53fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda540c0150 0x7fda540c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda53fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda54069f60 con 0x7fda540c0150 2023-05-16T07:14:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda53fff700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5406a9e0 con 0x7fda540c5d00 2023-05-16T07:14:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda53fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540c6670 con 0x7fda54153f90 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda540c0150 0x7fda540c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda540c0150 0x7fda540c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43088/0 (socket says 172.21.15.92:43088) 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 -- 172.21.15.92:0/3752510682 learned_addr learned my addr 172.21.15.92:0/3752510682 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda52ffd700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda54153f90 0x7fda540a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda527fc700 1 --2- 172.21.15.92:0/3752510682 >> v2:172.21.15.158:3300/0 conn(0x7fda540c5d00 0x7fda540c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 -- 172.21.15.92:0/3752510682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda54153f90 msgr2=0x7fda540a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda54153f90 0x7fda540a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 -- 172.21.15.92:0/3752510682 >> v2:172.21.15.158:3300/0 conn(0x7fda540c5d00 msgr2=0x7fda540c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 --2- 172.21.15.92:0/3752510682 >> v2:172.21.15.158:3300/0 conn(0x7fda540c5d00 0x7fda540c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 -- 172.21.15.92:0/3752510682 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5406a140 con 0x7fda540c0150 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda527fc700 1 --2- 172.21.15.92:0/3752510682 >> v2:172.21.15.158:3300/0 conn(0x7fda540c5d00 0x7fda540c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda52ffd700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda54153f90 0x7fda540a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda537fe700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c0150 0x7fda540c05c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fda54083000 tx=0x7fda4c0090f0).ready entity=mon.0 client_cookie=58b64ba5ed693194 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda51ffb700 1 -- 172.21.15.92:0/3752510682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00b600 con 0x7fda540c0150 2023-05-16T07:14:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda51ffb700 1 -- 172.21.15.92:0/3752510682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00b760 con 0x7fda540c0150 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.236+0000 7fda51ffb700 1 -- 172.21.15.92:0/3752510682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00bae0 con 0x7fda540c0150 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 -- 172.21.15.92:0/3752510682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c0150 msgr2=0x7fda540c05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c0150 0x7fda540c05c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fda54083000 tx=0x7fda4c0090f0).stop 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 -- 172.21.15.92:0/3752510682 shutdown_connections 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c0150 0x7fda540c05c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 --2- 172.21.15.92:0/3752510682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda54153f90 0x7fda540a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 --2- 172.21.15.92:0/3752510682 >> v2:172.21.15.158:3300/0 conn(0x7fda540c5d00 0x7fda540c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 -- 172.21.15.92:0/3752510682 >> 172.21.15.92:0/3752510682 conn(0x7fda540cb600 msgr2=0x7fda540c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 -- 172.21.15.92:0/3752510682 shutdown_connections 2023-05-16T07:14:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.237+0000 7fda53fff700 1 -- 172.21.15.92:0/3752510682 wait complete. 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.238+0000 7fda53fff700 1 Processor -- start 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.238+0000 7fda53fff700 1 -- start start 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda53fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda540c0150 0x7fda5414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 0x7fda54150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda53fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda54153f90 0x7fda540addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda53fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540c2ba0 con 0x7fda540c5d00 2023-05-16T07:14:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda53fff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540c1f90 con 0x7fda54153f90 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda53fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540c2270 con 0x7fda540c0150 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 0x7fda54150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 0x7fda54150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43090/0 (socket says 172.21.15.92:43090) 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 -- 172.21.15.92:0/3390520289 learned_addr learned my addr 172.21.15.92:0/3390520289 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 -- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda540c0150 msgr2=0x7fda5414dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda540c0150 0x7fda5414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 -- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda54153f90 msgr2=0x7fda540addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda54153f90 0x7fda540addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 -- 172.21.15.92:0/3390520289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c009370 con 0x7fda540c5d00 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.239+0000 7fda527fc700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 0x7fda54150760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fda3c00d860 tx=0x7fda3c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.240+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011590 con 0x7fda540c5d00 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.240+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c0116f0 con 0x7fda540c5d00 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.240+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda540c2550 con 0x7fda540c5d00 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.240+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011a70 con 0x7fda540c5d00 2023-05-16T07:14:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.240+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5414e0c0 con 0x7fda540c5d00 2023-05-16T07:14:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.240+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda540c1830 con 0x7fda540c5d00 2023-05-16T07:14:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.241+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fda3c012420 con 0x7fda540c5d00 2023-05-16T07:14:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.241+0000 7fda4affd700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda3405ee70 0x7fda34061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.241+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fda3c079950 con 0x7fda540c5d00 2023-05-16T07:14:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.241+0000 7fda52ffd700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda3405ee70 0x7fda34061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.242+0000 7fda52ffd700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda3405ee70 0x7fda34061330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fda440094d0 tx=0x7fda44009590).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.245+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fda3c04ada0 con 0x7fda540c5d00 2023-05-16T07:14:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.433+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda540c2ce0 con 0x7fda3405ee70 2023-05-16T07:14:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.435+0000 7fda4affd700 1 -- 172.21.15.92:0/3390520289 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fda540c2ce0 con 0x7fda3405ee70 2023-05-16T07:14:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:22.439 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:25", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda3405ee70 msgr2=0x7fda34061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda3405ee70 0x7fda34061330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fda440094d0 tx=0x7fda44009590).stop 2023-05-16T07:14:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 msgr2=0x7fda54150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 0x7fda54150760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fda3c00d860 tx=0x7fda3c008040).stop 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 shutdown_connections 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda3405ee70 0x7fda34061330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda540c0150 0x7fda5414dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda54153f90 0x7fda540addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 --2- 172.21.15.92:0/3390520289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540c5d00 0x7fda54150760 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 >> 172.21.15.92:0/3390520289 conn(0x7fda540cb600 msgr2=0x7fda540c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 shutdown_connections 2023-05-16T07:14:22.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.437+0000 7fda53fff700 1 -- 172.21.15.92:0/3390520289 wait complete. 2023-05-16T07:14:22.452 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:25", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:22.453 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:25', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:22.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:22.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-05-16T07:14:22.468 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:22.469 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:22.469 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:22.469 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:22.469 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:22.469 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:22.469 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:22.470 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:22.470 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:22.470 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:22.470 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:22.470 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:22.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-05-16T07:14:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.618+0000 7fa075423700 1 Processor -- start 2023-05-16T07:14:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.618+0000 7fa075423700 1 -- start start 2023-05-16T07:14:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa075423700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 0x7fa0700aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa075423700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700c94d0 con 0x7fa070154470 2023-05-16T07:14:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa075423700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700cb7e0 con 0x7fa070068160 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa075423700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c88c0 con 0x7fa070154090 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa06e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa070154470 0x7fa0700a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53850/0 (socket says 172.21.15.92:53850) 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa06e7fc700 1 -- 172.21.15.92:0/2619652011 learned_addr learned my addr 172.21.15.92:0/2619652011 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.619+0000 7fa06effd700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 0x7fa0700aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874421466 0 0) 0x7fa0700c94d0 con 0x7fa070154470 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c0033f0 con 0x7fa070154470 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1343667799 0 0) 0x7fa05c0033f0 con 0x7fa070154470 2023-05-16T07:14:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 msgr2=0x7fa0700aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 0x7fa0700aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 >> v1:172.21.15.158:6789/0 conn(0x7fa070068160 legacy=0x7fa070068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700c8ba0 con 0x7fa070154470 2023-05-16T07:14:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fa058001470 con 0x7fa070154470 2023-05-16T07:14:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 >> v1:172.21.15.92:6789/0 conn(0x7fa070154470 legacy=0x7fa0700a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa05c003910 0x7fa05c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa05c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 0x7fa05c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c88c0 con 0x7fa05c004690 2023-05-16T07:14:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c00b910 con 0x7fa05c003910 2023-05-16T07:14:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c00bad0 con 0x7fa070154090 2023-05-16T07:14:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa05c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06effd700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa05c003910 0x7fa05c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06f7fe700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 0x7fa05c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 -- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 msgr2=0x7fa05c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa070154090 0x7fa05c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 -- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa05c003910 msgr2=0x7fa05c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa05c003910 0x7fa05c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 -- 172.21.15.92:0/2619652011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0033f0 con 0x7fa05c004690 2023-05-16T07:14:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa05c004b10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa070067e30 tx=0x7fa058001e70).ready entity=mon.0 client_cookie=95d5857f130af8f2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.620+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058007a40 con 0x7fa05c004690 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05800ca30 con 0x7fa05c004690 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa06dffb700 1 -- 172.21.15.92:0/2619652011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800cdb0 con 0x7fa05c004690 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa075423700 1 -- 172.21.15.92:0/2619652011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 msgr2=0x7fa05c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa075423700 1 --2- 172.21.15.92:0/2619652011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa05c004b10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa070067e30 tx=0x7fa058001e70).stop 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa06f7fe700 1 -- 172.21.15.92:0/2619652011 reap_dead start 2023-05-16T07:14:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa075423700 1 -- 172.21.15.92:0/2619652011 shutdown_connections 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa075423700 1 -- 172.21.15.92:0/2619652011 >> 172.21.15.92:0/2619652011 conn(0x7fa070067920 msgr2=0x7fa0700cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa075423700 1 -- 172.21.15.92:0/2619652011 shutdown_connections 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.621+0000 7fa075423700 1 -- 172.21.15.92:0/2619652011 wait complete. 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.622+0000 7fa075423700 1 Processor -- start 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 -- start start 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa0700b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa07013c160 0x7fa070167b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0700add30 0x7fa0700ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa070165680 con 0x7fa05c004690 2023-05-16T07:14:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700ac040 con 0x7fa07013c160 2023-05-16T07:14:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa075423700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700ae6a0 con 0x7fa0700add30 2023-05-16T07:14:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa06effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa0700b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa06effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa0700b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43110/0 (socket says 172.21.15.92:43110) 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa06effd700 1 -- 172.21.15.92:0/594616320 learned_addr learned my addr 172.21.15.92:0/594616320 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa07013c160 0x7fa070167b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.623+0000 7fa06f7fe700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0700add30 0x7fa0700ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06effd700 1 -- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0700add30 msgr2=0x7fa0700ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06effd700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0700add30 0x7fa0700ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06effd700 1 -- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa07013c160 msgr2=0x7fa070167b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06effd700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa07013c160 0x7fa070167b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06effd700 1 -- 172.21.15.92:0/594616320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07013c5e0 con 0x7fa05c004690 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06f7fe700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0700add30 0x7fa0700ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa06effd700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa0700b07d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fa06000f1b0 tx=0x7fa06000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000e040 con 0x7fa05c004690 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06000f040 con 0x7fa05c004690 2023-05-16T07:14:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa070168350 con 0x7fa05c004690 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060015450 con 0x7fa05c004690 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.624+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa070168910 con 0x7fa05c004690 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.625+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0700ad650 con 0x7fa05c004690 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.625+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fa060015840 con 0x7fa05c004690 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.625+0000 7fa066ffd700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa05005ee70 0x7fa050061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.626+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fa06007af90 con 0x7fa05c004690 2023-05-16T07:14:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.626+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa05005ee70 0x7fa050061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.626+0000 7fa06e7fc700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa05005ee70 0x7fa050061330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa070067e30 tx=0x7fa058015000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.630+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa0700ad650 con 0x7fa05c004690 2023-05-16T07:14:22.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.819+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa070066730 con 0x7fa05005ee70 2023-05-16T07:14:22.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.821+0000 7fa066ffd700 1 -- 172.21.15.92:0/594616320 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa070066730 con 0x7fa05005ee70 2023-05-16T07:14:22.825 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:22.825 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:26", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.822+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa05005ee70 msgr2=0x7fa050061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa05005ee70 0x7fa050061330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa070067e30 tx=0x7fa058015000).stop 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 msgr2=0x7fa0700b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa0700b07d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fa06000f1b0 tx=0x7fa06000b040).stop 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 shutdown_connections 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa05005ee70 0x7fa050061330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0700add30 0x7fa0700ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa05c004690 0x7fa0700b07d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 --2- 172.21.15.92:0/594616320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa07013c160 0x7fa070167b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 >> 172.21.15.92:0/594616320 conn(0x7fa070067920 msgr2=0x7fa0700ab280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 shutdown_connections 2023-05-16T07:14:22.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:22.823+0000 7fa075423700 1 -- 172.21.15.92:0/594616320 wait complete. 2023-05-16T07:14:22.839 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:26", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:22.839 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:26', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:22.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:22.840 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-05-16T07:14:22.855 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:22.857 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:22.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-05-16T07:14:23.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.005+0000 7fdbe359e700 1 Processor -- start 2023-05-16T07:14:23.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.005+0000 7fdbe359e700 1 -- start start 2023-05-16T07:14:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe40c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdbe40c60a0 0x7fdbe415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe40bfcc0 con 0x7fdbe4157710 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe406a9a0 con 0x7fdbe40c60a0 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40aa410 con 0x7fdbe40c1160 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdbe40c60a0 0x7fdbe415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdbe40c60a0 0x7fdbe415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43120/0 (socket says 172.21.15.92:43120) 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.006+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2788515939 learned_addr learned my addr 172.21.15.92:0/2788515939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe1d9b700 1 --2- 172.21.15.92:0/2788515939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe40c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2788515939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 msgr2=0x7fdbe40c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe2d9d700 1 --2- 172.21.15.92:0/2788515939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe40c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2788515939 >> v1:172.21.15.158:6789/0 conn(0x7fdbe4157710 legacy=0x7fdbe40a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2788515939 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe406a140 con 0x7fdbe40c60a0 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe2d9d700 1 --2- 172.21.15.92:0/2788515939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe40c60a0 0x7fdbe415de60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fdbdc006fd0 tx=0x7fdbdc0090f0).ready entity=mon.0 client_cookie=1e4a35b2baee7f77 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe159a700 1 -- 172.21.15.92:0/2788515939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00b600 con 0x7fdbe40c60a0 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe159a700 1 -- 172.21.15.92:0/2788515939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc00b760 con 0x7fdbe40c60a0 2023-05-16T07:14:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe159a700 1 -- 172.21.15.92:0/2788515939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00bae0 con 0x7fdbe40c60a0 2023-05-16T07:14:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe359e700 1 -- 172.21.15.92:0/2788515939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe40c60a0 msgr2=0x7fdbe415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.007+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2788515939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe40c60a0 0x7fdbe415de60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fdbdc006fd0 tx=0x7fdbdc0090f0).stop 2023-05-16T07:14:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.008+0000 7fdbe359e700 1 -- 172.21.15.92:0/2788515939 shutdown_connections 2023-05-16T07:14:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.008+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2788515939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe40c60a0 0x7fdbe415de60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.008+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2788515939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe40c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.008+0000 7fdbe359e700 1 -- 172.21.15.92:0/2788515939 >> 172.21.15.92:0/2788515939 conn(0x7fdbe40cb600 msgr2=0x7fdbe40c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.008+0000 7fdbe359e700 1 -- 172.21.15.92:0/2788515939 shutdown_connections 2023-05-16T07:14:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.008+0000 7fdbe359e700 1 -- 172.21.15.92:0/2788515939 wait complete. 2023-05-16T07:14:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 Processor -- start 2023-05-16T07:14:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 -- start start 2023-05-16T07:14:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe414b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbe40c60a0 0x7fdbe413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 0x7fdbe40c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40c1bd0 con 0x7fdbe4157710 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe414c2d0 con 0x7fdbe40c60a0 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe414c5b0 con 0x7fdbe40c1160 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.009+0000 7fdbe2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 0x7fdbe40c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 0x7fdbe40c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43128/0 (socket says 172.21.15.92:43128) 2023-05-16T07:14:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2576982079 learned_addr learned my addr 172.21.15.92:0/2576982079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe259c700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe414b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe1d9b700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbe40c60a0 0x7fdbe413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 msgr2=0x7fdbe414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe414b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbe40c60a0 msgr2=0x7fdbe413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbe40c60a0 0x7fdbe413d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 -- 172.21.15.92:0/2576982079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc009370 con 0x7fdbe4157710 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe1d9b700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbe40c60a0 0x7fdbe413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe2d9d700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 0x7fdbe40c1ff0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fdbdc009f50 tx=0x7fdbdc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00bae0 con 0x7fdbe4157710 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc00bea0 con 0x7fdbe4157710 2023-05-16T07:14:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe414c890 con 0x7fdbe4157710 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc024680 con 0x7fdbe4157710 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.010+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe414be80 con 0x7fdbe4157710 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.011+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe414c0a0 con 0x7fdbe4157710 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.011+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fdbdc00f070 con 0x7fdbe4157710 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.012+0000 7fdbda7fc700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbc405ee70 0x7fdbc4061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.012+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fdbdc07bbb0 con 0x7fdbe4157710 2023-05-16T07:14:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.012+0000 7fdbe259c700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbc405ee70 0x7fdbc4061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.012+0000 7fdbe259c700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbc405ee70 0x7fdbc4061330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fdbd4006fd0 tx=0x7fdbd4008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.016+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdbdc082080 con 0x7fdbe4157710 2023-05-16T07:14:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.199+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbe40c1d10 con 0x7fdbc405ee70 2023-05-16T07:14:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.201+0000 7fdbda7fc700 1 -- 172.21.15.92:0/2576982079 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fdbe40c1d10 con 0x7fdbc405ee70 2023-05-16T07:14:23.206 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:23.206 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:26", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:23.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbc405ee70 msgr2=0x7fdbc4061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbc405ee70 0x7fdbc4061330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fdbd4006fd0 tx=0x7fdbd4008040).stop 2023-05-16T07:14:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 msgr2=0x7fdbe40c1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 0x7fdbe40c1ff0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fdbdc009f50 tx=0x7fdbdc017000).stop 2023-05-16T07:14:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 shutdown_connections 2023-05-16T07:14:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbc405ee70 0x7fdbc4061330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbe40c1160 0x7fdbe414b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe4157710 0x7fdbe40c1ff0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 --2- 172.21.15.92:0/2576982079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbe40c60a0 0x7fdbe413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 >> 172.21.15.92:0/2576982079 conn(0x7fdbe40cb600 msgr2=0x7fdbe4159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 shutdown_connections 2023-05-16T07:14:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.203+0000 7fdbe359e700 1 -- 172.21.15.92:0/2576982079 wait complete. 2023-05-16T07:14:23.218 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:26", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:23.218 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:26', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:23.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:23.219 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-05-16T07:14:23.236 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:23.236 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:23.236 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:23.237 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:23.237 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:23.237 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:23.239 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:23.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-05-16T07:14:23.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.386+0000 7efeda71e700 1 Processor -- start 2023-05-16T07:14:23.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.386+0000 7efeda71e700 1 -- start start 2023-05-16T07:14:23.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efeda71e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 0x7efed40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efeda71e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7efed4068160 0x7efed40685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efeda71e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed40c94d0 con 0x7efed4154090 2023-05-16T07:14:23.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efeda71e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed40cb7e0 con 0x7efed4068160 2023-05-16T07:14:23.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efeda71e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed4068ac0 con 0x7efed4154470 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efed3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efed4154090 0x7efed40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53880/0 (socket says 172.21.15.92:53880) 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efed3fff700 1 -- 172.21.15.92:0/806924649 learned_addr learned my addr 172.21.15.92:0/806924649 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efed37fe700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 0x7efed40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efed8cbb700 1 --2- 172.21.15.92:0/806924649 >> v2:172.21.15.158:3300/0 conn(0x7efed4068160 0x7efed40685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234788130 0 0) 0x7efed40c94d0 con 0x7efed4154090 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.387+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efebc0033f0 con 0x7efed4154090 2023-05-16T07:14:23.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401733564 0 0) 0x7efebc0033f0 con 0x7efed4154090 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 msgr2=0x7efed40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 0x7efed40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 >> v2:172.21.15.158:3300/0 conn(0x7efed4068160 msgr2=0x7efed40685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 --2- 172.21.15.92:0/806924649 >> v2:172.21.15.158:3300/0 conn(0x7efed4068160 0x7efed40685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40c88c0 con 0x7efed4154090 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7efec4001470 con 0x7efed4154090 2023-05-16T07:14:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 >> v1:172.21.15.92:6789/0 conn(0x7efed4154090 legacy=0x7efed40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efebc0038e0 0x7efebc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc004690 0x7efebc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 0x7efebc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40cb7e0 con 0x7efebc004690 2023-05-16T07:14:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed4068ac0 con 0x7efebc0038e0 2023-05-16T07:14:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efebc00b8e0 con 0x7efed4154470 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed8cbb700 1 --2- 172.21.15.92:0/806924649 >> v2:172.21.15.158:3300/0 conn(0x7efed4068160 0x7efed40685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc004690 0x7efebc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed3fff700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efebc0038e0 0x7efebc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed8cbb700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 0x7efebc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 -- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 msgr2=0x7efebc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efed4154470 0x7efebc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 -- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efebc0038e0 msgr2=0x7efebc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efebc0038e0 0x7efebc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 -- 172.21.15.92:0/806924649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc0033f0 con 0x7efebc004690 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed37fe700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc004690 0x7efebc004b10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7efec8009a80 tx=0x7efec8009060).ready entity=mon.0 client_cookie=667fb9cefd025438 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.388+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8009430 con 0x7efebc004690 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec800e640 con 0x7efebc004690 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efed2ffd700 1 -- 172.21.15.92:0/806924649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec800e9c0 con 0x7efebc004690 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efeda71e700 1 -- 172.21.15.92:0/806924649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc004690 msgr2=0x7efebc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efeda71e700 1 --2- 172.21.15.92:0/806924649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc004690 0x7efebc004b10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7efec8009a80 tx=0x7efec8009060).stop 2023-05-16T07:14:23.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efed8cbb700 1 -- 172.21.15.92:0/806924649 reap_dead start 2023-05-16T07:14:23.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efeda71e700 1 -- 172.21.15.92:0/806924649 shutdown_connections 2023-05-16T07:14:23.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efeda71e700 1 -- 172.21.15.92:0/806924649 >> 172.21.15.92:0/806924649 conn(0x7efed4067920 msgr2=0x7efed40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:23.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efeda71e700 1 -- 172.21.15.92:0/806924649 shutdown_connections 2023-05-16T07:14:23.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.389+0000 7efeda71e700 1 -- 172.21.15.92:0/806924649 wait complete. 2023-05-16T07:14:23.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.390+0000 7efeda71e700 1 Processor -- start 2023-05-16T07:14:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.390+0000 7efeda71e700 1 -- start start 2023-05-16T07:14:23.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efeda71e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efebc004690 0x7efed41448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efeda71e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efed4144dc0 0x7efed40c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efeda71e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 0x7efed40c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efeda71e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40b07d0 con 0x7efed40c21c0 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efeda71e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40c2640 con 0x7efed4144dc0 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efeda71e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed40c28b0 con 0x7efebc004690 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 0x7efed40c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 0x7efed40c0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43146/0 (socket says 172.21.15.92:43146) 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 -- 172.21.15.92:0/1372572281 learned_addr learned my addr 172.21.15.92:0/1372572281 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed3fff700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efebc004690 0x7efed41448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed37fe700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efed4144dc0 0x7efed40c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 -- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efebc004690 msgr2=0x7efed41448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efebc004690 0x7efed41448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 -- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efed4144dc0 msgr2=0x7efed40c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efed4144dc0 0x7efed40c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed8cbb700 1 -- 172.21.15.92:0/1372572281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40c2b90 con 0x7efed40c21c0 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.391+0000 7efed37fe700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efed4144dc0 0x7efed40c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.392+0000 7efed8cbb700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 0x7efed40c0e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7efed405efb0 tx=0x7efecc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.392+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00e070 con 0x7efed40c21c0 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.392+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc011d00 con 0x7efed40c21c0 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.392+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed40c2e70 con 0x7efed40c21c0 2023-05-16T07:14:23.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.392+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc016660 con 0x7efed40c21c0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.392+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed40c3430 con 0x7efed40c21c0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.393+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed4066730 con 0x7efed40c21c0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.393+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7efecc01d030 con 0x7efed40c21c0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.393+0000 7efebbfff700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efec005ee70 0x7efec0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.393+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7efecc07a270 con 0x7efed40c21c0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.393+0000 7efed3fff700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efec005ee70 0x7efec0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.394+0000 7efed3fff700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efec005ee70 0x7efec0061330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7efec4001620 tx=0x7efec4001e90).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.397+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7efecc04b770 con 0x7efed40c21c0 2023-05-16T07:14:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.582+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efed4151370 con 0x7efec005ee70 2023-05-16T07:14:23.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.584+0000 7efebbfff700 1 -- 172.21.15.92:0/1372572281 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7efed4151370 con 0x7efec005ee70 2023-05-16T07:14:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:23.588 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:27", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efec005ee70 msgr2=0x7efec0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efec005ee70 0x7efec0061330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7efec4001620 tx=0x7efec4001e90).stop 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 msgr2=0x7efed40c0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 0x7efed40c0e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7efed405efb0 tx=0x7efecc00b040).stop 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 shutdown_connections 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efec005ee70 0x7efec0061330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efebc004690 0x7efed41448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed40c21c0 0x7efed40c0e20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 --2- 172.21.15.92:0/1372572281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efed4144dc0 0x7efed40c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.585+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 >> 172.21.15.92:0/1372572281 conn(0x7efed4067920 msgr2=0x7efed40c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.586+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 shutdown_connections 2023-05-16T07:14:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.586+0000 7efeda71e700 1 -- 172.21.15.92:0/1372572281 wait complete. 2023-05-16T07:14:23.601 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:27", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:23.601 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:27', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:23.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:23.602 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-05-16T07:14:23.616 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:23.616 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:23.617 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:23.618 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:23.618 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:23.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-05-16T07:14:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.768+0000 7f218a121700 1 Processor -- start 2023-05-16T07:14:23.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.768+0000 7f218a121700 1 -- start start 2023-05-16T07:14:23.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218a121700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2184153f90 0x7f21840a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218a121700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21840c5d00 0x7f21840c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218a121700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21840c66a0 con 0x7f21840c0150 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218a121700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2184069f60 con 0x7f21840c5d00 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218a121700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218406a9e0 con 0x7f2184153f90 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218891e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21840c5d00 0x7f21840c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218891e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21840c5d00 0x7f21840c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43156/0 (socket says 172.21.15.92:43156) 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.769+0000 7f218891e700 1 -- 172.21.15.92:0/3037418318 learned_addr learned my addr 172.21.15.92:0/3037418318 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:23.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218891e700 1 -- 172.21.15.92:0/3037418318 >> v2:172.21.15.158:3300/0 conn(0x7f2184153f90 msgr2=0x7f21840a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218911f700 1 --2- 172.21.15.92:0/3037418318 >> v2:172.21.15.158:3300/0 conn(0x7f2184153f90 0x7f21840a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218891e700 1 --2- 172.21.15.92:0/3037418318 >> v2:172.21.15.158:3300/0 conn(0x7f2184153f90 0x7f21840a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218891e700 1 -- 172.21.15.92:0/3037418318 >> v1:172.21.15.158:6789/0 conn(0x7f21840c0150 legacy=0x7f21840c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218891e700 1 -- 172.21.15.92:0/3037418318 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218406a140 con 0x7f21840c5d00 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218911f700 1 --2- 172.21.15.92:0/3037418318 >> v2:172.21.15.158:3300/0 conn(0x7f2184153f90 0x7f21840a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218891e700 1 --2- 172.21.15.92:0/3037418318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c5d00 0x7f21840c61b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f21840caa40 tx=0x7f21780090f0).ready entity=mon.0 client_cookie=6151979e622d59bd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f2183fff700 1 -- 172.21.15.92:0/3037418318 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800b5a0 con 0x7f21840c5d00 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f2183fff700 1 -- 172.21.15.92:0/3037418318 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217800b700 con 0x7f21840c5d00 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f2183fff700 1 -- 172.21.15.92:0/3037418318 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800ba80 con 0x7f21840c5d00 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218a121700 1 -- 172.21.15.92:0/3037418318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c5d00 msgr2=0x7f21840c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218a121700 1 --2- 172.21.15.92:0/3037418318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c5d00 0x7f21840c61b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f21840caa40 tx=0x7f21780090f0).stop 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218a121700 1 -- 172.21.15.92:0/3037418318 shutdown_connections 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218a121700 1 --2- 172.21.15.92:0/3037418318 >> v2:172.21.15.158:3300/0 conn(0x7f2184153f90 0x7f21840a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.770+0000 7f218a121700 1 --2- 172.21.15.92:0/3037418318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c5d00 0x7f21840c61b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.771+0000 7f218a121700 1 -- 172.21.15.92:0/3037418318 >> 172.21.15.92:0/3037418318 conn(0x7f21840cb600 msgr2=0x7f21840c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.771+0000 7f218a121700 1 -- 172.21.15.92:0/3037418318 shutdown_connections 2023-05-16T07:14:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.771+0000 7f218a121700 1 -- 172.21.15.92:0/3037418318 wait complete. 2023-05-16T07:14:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.772+0000 7f218a121700 1 Processor -- start 2023-05-16T07:14:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.772+0000 7f218a121700 1 -- start start 2023-05-16T07:14:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 0x7f21840addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f21840c5d00 0x7f218414e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2184153f90 0x7f21840b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2184164320 con 0x7f21840c0150 2023-05-16T07:14:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2184164600 con 0x7f2184153f90 2023-05-16T07:14:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21841648e0 con 0x7f21840c5d00 2023-05-16T07:14:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 0x7f21840addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 0x7f21840addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43158/0 (socket says 172.21.15.92:43158) 2023-05-16T07:14:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 -- 172.21.15.92:0/613188391 learned_addr learned my addr 172.21.15.92:0/613188391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218891e700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f21840c5d00 0x7f218414e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f2189920700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2184153f90 0x7f21840b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 -- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f21840c5d00 msgr2=0x7f218414e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f21840c5d00 0x7f218414e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 -- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2184153f90 msgr2=0x7f21840b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2184153f90 0x7f21840b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 -- 172.21.15.92:0/613188391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178009370 con 0x7f21840c0150 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f2189920700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2184153f90 0x7f21840b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218911f700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 0x7f21840addd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f217400cfb0 tx=0x7f2174007870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400ae30 con 0x7f21840c0150 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174016400 con 0x7f21840c0150 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174016780 con 0x7f21840c0150 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2184164bc0 con 0x7f21840c0150 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.773+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21841686c0 con 0x7f21840c0150 2023-05-16T07:14:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.774+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2184164f30 con 0x7f21840c0150 2023-05-16T07:14:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.775+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f21740203f0 con 0x7f21840c0150 2023-05-16T07:14:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.775+0000 7f21817fa700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f217005ee70 0x7f2170061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.775+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f217407ac10 con 0x7f21840c0150 2023-05-16T07:14:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.775+0000 7f218891e700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f217005ee70 0x7f2170061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.776+0000 7f218891e700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f217005ee70 0x7f2170061330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2178009130 tx=0x7f2178015000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.779+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f217404c0e0 con 0x7f21840c0150 2023-05-16T07:14:23.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.964+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2184083120 con 0x7f217005ee70 2023-05-16T07:14:23.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.966+0000 7f21817fa700 1 -- 172.21.15.92:0/613188391 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f2184083120 con 0x7f217005ee70 2023-05-16T07:14:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:23.970 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:27", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f217005ee70 msgr2=0x7f2170061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f217005ee70 0x7f2170061330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2178009130 tx=0x7f2178015000).stop 2023-05-16T07:14:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 msgr2=0x7f21840addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 0x7f21840addd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f217400cfb0 tx=0x7f2174007870).stop 2023-05-16T07:14:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 shutdown_connections 2023-05-16T07:14:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f217005ee70 0x7f2170061330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2184153f90 0x7f21840b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21840c0150 0x7f21840addd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 --2- 172.21.15.92:0/613188391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f21840c5d00 0x7f218414e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 >> 172.21.15.92:0/613188391 conn(0x7f21840cb600 msgr2=0x7f21840c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 shutdown_connections 2023-05-16T07:14:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:23.968+0000 7f218a121700 1 -- 172.21.15.92:0/613188391 wait complete. 2023-05-16T07:14:23.983 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:27", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:23.983 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:27', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:23.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:23.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-05-16T07:14:23.999 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:23.999 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:23.999 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:23.999 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:23.999 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:24.000 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:24.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-05-16T07:14:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.147+0000 7fc2f2632700 1 Processor -- start 2023-05-16T07:14:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.147+0000 7fc2f2632700 1 -- start start 2023-05-16T07:14:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f2632700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f2632700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2ec068160 0x7fc2ec068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f2632700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec0c94d0 con 0x7fc2ec154470 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f2632700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cb7e0 con 0x7fc2ec068160 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f2632700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec068a70 con 0x7fc2ec154090 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2ec068160 0x7fc2ec068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2ec068160 0x7fc2ec068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43168/0 (socket says 172.21.15.92:43168) 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1520600338 learned_addr learned my addr 172.21.15.92:0/1520600338 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2ebfff700 1 --2- 172.21.15.92:0/1520600338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1520600338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 msgr2=0x7fc2ec0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 --2- 172.21.15.92:0/1520600338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1520600338 >> v1:172.21.15.158:6789/0 conn(0x7fc2ec154470 legacy=0x7fc2ec0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1520600338 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0c88c0 con 0x7fc2ec068160 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2f0bcf700 1 --2- 172.21.15.92:0/1520600338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec068160 0x7fc2ec068580 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fc2e4009f20 tx=0x7fc2e4009840).ready entity=mon.0 client_cookie=a550aec592c44c5a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2eaffd700 1 -- 172.21.15.92:0/1520600338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400e070 con 0x7fc2ec068160 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2eaffd700 1 -- 172.21.15.92:0/1520600338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e400ba80 con 0x7fc2ec068160 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.148+0000 7fc2eaffd700 1 -- 172.21.15.92:0/1520600338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400be00 con 0x7fc2ec068160 2023-05-16T07:14:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 -- 172.21.15.92:0/1520600338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec068160 msgr2=0x7fc2ec068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1520600338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec068160 0x7fc2ec068580 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fc2e4009f20 tx=0x7fc2e4009840).stop 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 -- 172.21.15.92:0/1520600338 shutdown_connections 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1520600338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec068160 0x7fc2ec068580 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1520600338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 -- 172.21.15.92:0/1520600338 >> 172.21.15.92:0/1520600338 conn(0x7fc2ec067920 msgr2=0x7fc2ec0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 -- 172.21.15.92:0/1520600338 shutdown_connections 2023-05-16T07:14:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.149+0000 7fc2f2632700 1 -- 172.21.15.92:0/1520600338 wait complete. 2023-05-16T07:14:24.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.150+0000 7fc2f2632700 1 Processor -- start 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.150+0000 7fc2f2632700 1 -- start start 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f2632700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ec068160 0x7fc2ec155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f2632700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f2632700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 0x7fc2ec167d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f2632700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec156da0 con 0x7fc2ec154470 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f2632700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec156190 con 0x7fc2ec068160 2023-05-16T07:14:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f2632700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec156470 con 0x7fc2ec154090 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 0x7fc2ec167d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 0x7fc2ec167d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43170/0 (socket says 172.21.15.92:43170) 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1601790703 learned_addr learned my addr 172.21.15.92:0/1601790703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2ebfff700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ec068160 0x7fc2ec155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2eb7fe700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 msgr2=0x7fc2ec0b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ec068160 msgr2=0x7fc2ec155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ec068160 0x7fc2ec155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 -- 172.21.15.92:0/1601790703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4009ae0 con 0x7fc2ec154470 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2eb7fe700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.151+0000 7fc2f0bcf700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 0x7fc2ec167d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fc2ec06bb30 tx=0x7fc2e4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.152+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40092e0 con 0x7fc2ec154470 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.152+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e401e470 con 0x7fc2ec154470 2023-05-16T07:14:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.152+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec156750 con 0x7fc2ec154470 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.152+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e401e830 con 0x7fc2ec154470 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.152+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec0c2340 con 0x7fc2ec154470 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.152+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec0aac60 con 0x7fc2ec154470 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.153+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fc2e4025030 con 0x7fc2ec154470 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.153+0000 7fc2d3fff700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc2d805ee70 0x7fc2d8061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.153+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fc2e407b3f0 con 0x7fc2ec154470 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.153+0000 7fc2ebfff700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc2d805ee70 0x7fc2d8061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.154+0000 7fc2ebfff700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc2d805ee70 0x7fc2d8061330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fc2dc00bbc0 tx=0x7fc2dc008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.157+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc2e404c950 con 0x7fc2ec154470 2023-05-16T07:14:24.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.342+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2ec06c350 con 0x7fc2d805ee70 2023-05-16T07:14:24.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.344+0000 7fc2d3fff700 1 -- 172.21.15.92:0/1601790703 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc2ec06c350 con 0x7fc2d805ee70 2023-05-16T07:14:24.348 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:24.348 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:27", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:24.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc2d805ee70 msgr2=0x7fc2d8061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc2d805ee70 0x7fc2d8061330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fc2dc00bbc0 tx=0x7fc2dc008040).stop 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 msgr2=0x7fc2ec167d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 0x7fc2ec167d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fc2ec06bb30 tx=0x7fc2e4009840).stop 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 shutdown_connections 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc2d805ee70 0x7fc2d8061330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ec154470 0x7fc2ec167d60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ec068160 0x7fc2ec155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 --2- 172.21.15.92:0/1601790703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2ec154090 0x7fc2ec0b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 >> 172.21.15.92:0/1601790703 conn(0x7fc2ec067920 msgr2=0x7fc2ec152f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:24.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 shutdown_connections 2023-05-16T07:14:24.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.345+0000 7fc2f2632700 1 -- 172.21.15.92:0/1601790703 wait complete. 2023-05-16T07:14:24.361 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:27", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:24.361 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:27', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:24.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:24.362 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-05-16T07:14:24.377 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:24.377 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:24.377 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:24.378 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:24.378 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:24.378 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:24.378 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:24.378 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:24.379 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:24.379 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:24.380 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:24.380 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:24.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-05-16T07:14:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.527+0000 7f3a8e825700 1 Processor -- start 2023-05-16T07:14:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.527+0000 7f3a8e825700 1 -- start start 2023-05-16T07:14:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8e825700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a880629c0 0x7f3a88063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8e825700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8814ead0 con 0x7f3a88155140 2023-05-16T07:14:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8e825700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a88154cb0 con 0x7f3a8814e6f0 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8e825700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880c6020 con 0x7f3a880629c0 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8cdc2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a880629c0 0x7f3a88063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a877fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3a88155140 0x7f3a880622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53928/0 (socket says 172.21.15.92:53928) 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a877fe700 1 -- 172.21.15.92:0/1773670097 learned_addr learned my addr 172.21.15.92:0/1773670097 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8cdc2700 1 -- 172.21.15.92:0/1773670097 >> v1:172.21.15.158:6789/0 conn(0x7f3a8814e6f0 legacy=0x7f3a880c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8cdc2700 1 -- 172.21.15.92:0/1773670097 >> v1:172.21.15.92:6789/0 conn(0x7f3a88155140 legacy=0x7f3a880622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a86ffd700 1 -- 172.21.15.92:0/1773670097 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444844428 0 0) 0x7f3a8814ead0 con 0x7f3a88155140 2023-05-16T07:14:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8cdc2700 1 -- 172.21.15.92:0/1773670097 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88154660 con 0x7f3a880629c0 2023-05-16T07:14:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a8cdc2700 1 --2- 172.21.15.92:0/1773670097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a880629c0 0x7f3a88063600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f3a80009e60 tx=0x7f3a80009780).ready entity=mon.0 client_cookie=2d2afc6a0d9a76d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a86ffd700 1 -- 172.21.15.92:0/1773670097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000e070 con 0x7f3a880629c0 2023-05-16T07:14:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a86ffd700 1 -- 172.21.15.92:0/1773670097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000bc30 con 0x7f3a880629c0 2023-05-16T07:14:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.528+0000 7f3a86ffd700 1 -- 172.21.15.92:0/1773670097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80012410 con 0x7f3a880629c0 2023-05-16T07:14:24.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 -- 172.21.15.92:0/1773670097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a880629c0 msgr2=0x7f3a88063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1773670097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a880629c0 0x7f3a88063600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f3a80009e60 tx=0x7f3a80009780).stop 2023-05-16T07:14:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 -- 172.21.15.92:0/1773670097 shutdown_connections 2023-05-16T07:14:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1773670097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a880629c0 0x7f3a88063600 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 -- 172.21.15.92:0/1773670097 >> 172.21.15.92:0/1773670097 conn(0x7f3a8805e820 msgr2=0x7f3a880c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 -- 172.21.15.92:0/1773670097 shutdown_connections 2023-05-16T07:14:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.529+0000 7f3a8e825700 1 -- 172.21.15.92:0/1773670097 wait complete. 2023-05-16T07:14:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.530+0000 7f3a8e825700 1 Processor -- start 2023-05-16T07:14:24.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.530+0000 7f3a8e825700 1 -- start start 2023-05-16T07:14:24.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8e825700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3a880629c0 0x7f3a8814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8e825700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 0x7f3a8814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8e825700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3a88155140 0x7f3a8814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8e825700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880a9540 con 0x7f3a8814e6f0 2023-05-16T07:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8e825700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881515f0 con 0x7f3a88155140 2023-05-16T07:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8e825700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881518d0 con 0x7f3a880629c0 2023-05-16T07:14:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 0x7f3a8814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 0x7f3a8814f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43186/0 (socket says 172.21.15.92:43186) 2023-05-16T07:14:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a877fe700 1 -- 172.21.15.92:0/1669241198 learned_addr learned my addr 172.21.15.92:0/1669241198 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a87fff700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3a880629c0 0x7f3a8814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a8cdc2700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3a88155140 0x7f3a8814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a877fe700 1 -- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3a880629c0 msgr2=0x7f3a8814d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.531+0000 7f3a877fe700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3a880629c0 0x7f3a8814d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a877fe700 1 -- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3a88155140 msgr2=0x7f3a8814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a877fe700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3a88155140 0x7f3a8814f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a877fe700 1 -- 172.21.15.92:0/1669241198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009a20 con 0x7f3a8814e6f0 2023-05-16T07:14:24.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a877fe700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 0x7f3a8814f4e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f3a7800c4c0 tx=0x7f3a78007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800ce30 con 0x7f3a8814e6f0 2023-05-16T07:14:24.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a78015b70 con 0x7f3a8814e6f0 2023-05-16T07:14:24.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a880a9790 con 0x7f3a8814e6f0 2023-05-16T07:14:24.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780070d0 con 0x7f3a8814e6f0 2023-05-16T07:14:24.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.532+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a880a9b20 con 0x7f3a8814e6f0 2023-05-16T07:14:24.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.533+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a880a9cb0 con 0x7f3a8814e6f0 2023-05-16T07:14:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.533+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f3a78025070 con 0x7f3a8814e6f0 2023-05-16T07:14:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.533+0000 7f3a6ffff700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3a7405ee70 0x7f3a74061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.534+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f3a78081b40 con 0x7f3a8814e6f0 2023-05-16T07:14:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.534+0000 7f3a87fff700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3a7405ee70 0x7f3a74061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.534+0000 7f3a87fff700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3a7405ee70 0x7f3a74061330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3a7c003840 tx=0x7f3a7c007d40).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.537+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3a880a9cb0 con 0x7f3a8814e6f0 2023-05-16T07:14:24.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.724+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a880a9cb0 con 0x7f3a7405ee70 2023-05-16T07:14:24.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.725+0000 7f3a6ffff700 1 -- 172.21.15.92:0/1669241198 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3a880a9cb0 con 0x7f3a7405ee70 2023-05-16T07:14:24.730 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:24.730 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:28", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3a7405ee70 msgr2=0x7f3a74061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3a7405ee70 0x7f3a74061330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3a7c003840 tx=0x7f3a7c007d40).stop 2023-05-16T07:14:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 msgr2=0x7f3a8814f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 0x7f3a8814f4e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f3a7800c4c0 tx=0x7f3a78007c10).stop 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 shutdown_connections 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3a7405ee70 0x7f3a74061330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3a880629c0 0x7f3a8814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3a88155140 0x7f3a8814f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 --2- 172.21.15.92:0/1669241198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8814e6f0 0x7f3a8814f4e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 >> 172.21.15.92:0/1669241198 conn(0x7f3a8805e820 msgr2=0x7f3a880c7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:24.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 shutdown_connections 2023-05-16T07:14:24.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.727+0000 7f3a8e825700 1 -- 172.21.15.92:0/1669241198 wait complete. 2023-05-16T07:14:24.742 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:28", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:24.743 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:28', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:24.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:24.743 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-05-16T07:14:24.758 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:24.759 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:24.759 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:24.759 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:24.759 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:24.759 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:24.759 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:24.760 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:24.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-05-16T07:14:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.909+0000 7f4c038ce700 1 Processor -- start 2023-05-16T07:14:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.909+0000 7f4c038ce700 1 -- start start 2023-05-16T07:14:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4c038ce700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 0x7f4bfc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4c038ce700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc0c94d0 con 0x7f4bfc154090 2023-05-16T07:14:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4c038ce700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc0cb7e0 con 0x7f4bfc068160 2023-05-16T07:14:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4c038ce700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0c88c0 con 0x7f4bfc154470 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4c0166a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4bfc154090 0x7f4bfc0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53940/0 (socket says 172.21.15.92:53940) 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4c0166a700 1 -- 172.21.15.92:0/2243636205 learned_addr learned my addr 172.21.15.92:0/2243636205 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204061790 0 0) 0x7f4bfc0c94d0 con 0x7f4bfc154090 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.910+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be40033f0 con 0x7f4bfc154090 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 0x7f4bfc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866996582 0 0) 0x7f4be40033f0 con 0x7f4bfc154090 2023-05-16T07:14:24.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 msgr2=0x7f4bfc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 0x7f4bfc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 >> v1:172.21.15.158:6789/0 conn(0x7f4bfc068160 legacy=0x7f4bfc0c9960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0c8ba0 con 0x7f4bfc154090 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4bec0017f0 con 0x7f4bfc154090 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 0x7f4bfc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 >> v1:172.21.15.92:6789/0 conn(0x7f4bfc154090 legacy=0x7f4bfc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4003910 0x7f4be4003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be4004690 0x7f4be4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 0x7f4be4005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0c88c0 con 0x7f4be4004690 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400ba70 con 0x7f4be4003910 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400bc30 con 0x7f4bfc154470 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be4004690 0x7f4be4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c0166a700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4003910 0x7f4be4003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 -- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 msgr2=0x7f4be4005140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc154470 0x7f4be4005140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 -- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4003910 msgr2=0x7f4be4003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4003910 0x7f4be4003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 -- 172.21.15.92:0/2243636205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40033f0 con 0x7f4be4004690 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c0166a700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4003910 0x7f4be4003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be4004690 0x7f4be4004b10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4bf000aa60 tx=0x7f4bf0008040).ready entity=mon.0 client_cookie=a594ab9af0cca2d7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000aea0 con 0x7f4be4004690 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000c600 con 0x7f4be4004690 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.911+0000 7f4bfbfff700 1 -- 172.21.15.92:0/2243636205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000c980 con 0x7f4be4004690 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c038ce700 1 -- 172.21.15.92:0/2243636205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be4004690 msgr2=0x7f4be4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2243636205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be4004690 0x7f4be4004b10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4bf000aa60 tx=0x7f4bf0008040).stop 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c01e6b700 1 -- 172.21.15.92:0/2243636205 reap_dead start 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c038ce700 1 -- 172.21.15.92:0/2243636205 shutdown_connections 2023-05-16T07:14:24.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c038ce700 1 -- 172.21.15.92:0/2243636205 >> 172.21.15.92:0/2243636205 conn(0x7f4bfc067920 msgr2=0x7f4bfc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:24.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c038ce700 1 -- 172.21.15.92:0/2243636205 shutdown_connections 2023-05-16T07:14:24.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.912+0000 7f4c038ce700 1 -- 172.21.15.92:0/2243636205 wait complete. 2023-05-16T07:14:24.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.913+0000 7f4c038ce700 1 Processor -- start 2023-05-16T07:14:24.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.913+0000 7f4c038ce700 1 -- start start 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c038ce700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4004690 0x7f4bfc0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c038ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 0x7f4bfc164c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c038ce700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc165180 0x7f4bfc0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c038ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc14cb80 con 0x7f4bfc14cd30 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c038ce700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc165600 con 0x7f4be4004690 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c038ce700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc164800 con 0x7f4bfc165180 2023-05-16T07:14:24.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 0x7f4bfc164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 0x7f4bfc164c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43204/0 (socket says 172.21.15.92:43204) 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 -- 172.21.15.92:0/2744976671 learned_addr learned my addr 172.21.15.92:0/2744976671 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c01e6b700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc165180 0x7f4bfc0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c0166a700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4004690 0x7f4bfc0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 -- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc165180 msgr2=0x7f4bfc0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc165180 0x7f4bfc0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 -- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4004690 msgr2=0x7f4bfc0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4004690 0x7f4bfc0b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 -- 172.21.15.92:0/2744976671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0a9f80 con 0x7f4bfc14cd30 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c01e6b700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc165180 0x7f4bfc0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c0166a700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4004690 0x7f4bfc0b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.914+0000 7f4c00e69700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 0x7f4bfc164c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f4bf000c430 tx=0x7f4bf0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.915+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000aea0 con 0x7f4bfc14cd30 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.915+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000ccf0 con 0x7f4bfc14cd30 2023-05-16T07:14:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.915+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000ce60 con 0x7f4bfc14cd30 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.915+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc0aa260 con 0x7f4bfc14cd30 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.915+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc0adce0 con 0x7f4bfc14cd30 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.915+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc066730 con 0x7f4bfc14cd30 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.916+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f4bf002c070 con 0x7f4bfc14cd30 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.916+0000 7f4bf97fa700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805eea0 0x7f4be8061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.916+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f4bf007b2c0 con 0x7f4bfc14cd30 2023-05-16T07:14:24.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.916+0000 7f4c0166a700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805eea0 0x7f4be8061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:24.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.917+0000 7f4c0166a700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805eea0 0x7f4be8061360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4bec001920 tx=0x7f4bec007f90).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:24.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:24.920+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4bf004c770 con 0x7f4bfc14cd30 2023-05-16T07:14:25.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.104+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bfc164970 con 0x7f4be805eea0 2023-05-16T07:14:25.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.107+0000 7f4bf97fa700 1 -- 172.21.15.92:0/2744976671 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4bfc164970 con 0x7f4be805eea0 2023-05-16T07:14:25.110 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:25.111 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:28", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:25.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805eea0 msgr2=0x7f4be8061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805eea0 0x7f4be8061360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4bec001920 tx=0x7f4bec007f90).stop 2023-05-16T07:14:25.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 msgr2=0x7f4bfc164c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 0x7f4bfc164c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f4bf000c430 tx=0x7f4bf0008040).stop 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 shutdown_connections 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be805eea0 0x7f4be8061360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bfc165180 0x7f4bfc0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4be4004690 0x7f4bfc0b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 --2- 172.21.15.92:0/2744976671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bfc14cd30 0x7f4bfc164c90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 >> 172.21.15.92:0/2744976671 conn(0x7f4bfc067920 msgr2=0x7f4bfc14db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 shutdown_connections 2023-05-16T07:14:25.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.108+0000 7f4c038ce700 1 -- 172.21.15.92:0/2744976671 wait complete. 2023-05-16T07:14:25.124 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:28", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:25.124 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:28', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:25.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:25.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-05-16T07:14:25.140 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:25.140 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:25.140 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:25.141 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:25.142 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:25.142 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:25.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-05-16T07:14:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.290+0000 7f39f484c700 1 Processor -- start 2023-05-16T07:14:25.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.290+0000 7f39f484c700 1 -- start start 2023-05-16T07:14:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39f484c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39f0154470 0x7f39f00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39f484c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0068160 0x7f39f00685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39f484c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f00c94d0 con 0x7f39f0154090 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39f484c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f00cb7e0 con 0x7f39f0154470 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39f484c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0068aa0 con 0x7f39f0068160 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39f0154470 0x7f39f00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39f0154470 0x7f39f00aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43214/0 (socket says 172.21.15.92:43214) 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 -- 172.21.15.92:0/876832687 learned_addr learned my addr 172.21.15.92:0/876832687 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39eed9d700 1 --2- 172.21.15.92:0/876832687 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0068160 0x7f39f00685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 -- 172.21.15.92:0/876832687 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0068160 msgr2=0x7f39f00685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:25.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 --2- 172.21.15.92:0/876832687 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0068160 0x7f39f00685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 -- 172.21.15.92:0/876832687 >> v1:172.21.15.158:6789/0 conn(0x7f39f0154090 legacy=0x7f39f00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:25.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 -- 172.21.15.92:0/876832687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00c88c0 con 0x7f39f0154470 2023-05-16T07:14:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.291+0000 7f39edd9b700 1 --2- 172.21.15.92:0/876832687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154470 0x7f39f00aaca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f39e4006fd0 tx=0x7f39e40090f0).ready entity=mon.0 client_cookie=156c03f6f740e206 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39ed59a700 1 -- 172.21.15.92:0/876832687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400b600 con 0x7f39f0154470 2023-05-16T07:14:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39ed59a700 1 -- 172.21.15.92:0/876832687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e400b760 con 0x7f39f0154470 2023-05-16T07:14:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39ed59a700 1 -- 172.21.15.92:0/876832687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400bae0 con 0x7f39f0154470 2023-05-16T07:14:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 -- 172.21.15.92:0/876832687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154470 msgr2=0x7f39f00aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 --2- 172.21.15.92:0/876832687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154470 0x7f39f00aaca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f39e4006fd0 tx=0x7f39e40090f0).stop 2023-05-16T07:14:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 -- 172.21.15.92:0/876832687 shutdown_connections 2023-05-16T07:14:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 --2- 172.21.15.92:0/876832687 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0068160 0x7f39f00685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 --2- 172.21.15.92:0/876832687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154470 0x7f39f00aaca0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 -- 172.21.15.92:0/876832687 >> 172.21.15.92:0/876832687 conn(0x7f39f0067920 msgr2=0x7f39f00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 -- 172.21.15.92:0/876832687 shutdown_connections 2023-05-16T07:14:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.292+0000 7f39f484c700 1 -- 172.21.15.92:0/876832687 wait complete. 2023-05-16T07:14:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.293+0000 7f39f484c700 1 Processor -- start 2023-05-16T07:14:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.293+0000 7f39f484c700 1 -- start start 2023-05-16T07:14:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39f484c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39f0068160 0x7f39f0152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39f484c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 0x7f39f01503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39f484c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0154470 0x7f39f00c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39f484c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f01508e0 con 0x7f39f0154090 2023-05-16T07:14:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39f484c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0155a30 con 0x7f39f0068160 2023-05-16T07:14:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39f484c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0152f80 con 0x7f39f0154470 2023-05-16T07:14:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 0x7f39f01503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 0x7f39f01503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43220/0 (socket says 172.21.15.92:43220) 2023-05-16T07:14:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 -- 172.21.15.92:0/892748688 learned_addr learned my addr 172.21.15.92:0/892748688 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39eed9d700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0154470 0x7f39f00c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39ee59c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39f0068160 0x7f39f0152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 -- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0154470 msgr2=0x7f39f00c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0154470 0x7f39f00c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 -- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39f0068160 msgr2=0x7f39f0152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39f0068160 0x7f39f0152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 -- 172.21.15.92:0/892748688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4009370 con 0x7f39f0154090 2023-05-16T07:14:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39edd9b700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 0x7f39f01503f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f39f006bb30 tx=0x7f39e4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400bae0 con 0x7f39f0154090 2023-05-16T07:14:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.294+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4000c90 con 0x7f39f0154090 2023-05-16T07:14:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.295+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e401b660 con 0x7f39f0154090 2023-05-16T07:14:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.295+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f0153260 con 0x7f39f0154090 2023-05-16T07:14:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.295+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f00c4a60 con 0x7f39f0154090 2023-05-16T07:14:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.295+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f0066730 con 0x7f39f0154090 2023-05-16T07:14:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.296+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f39e402b070 con 0x7f39f0154090 2023-05-16T07:14:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.296+0000 7f39de7fc700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f39d005ee70 0x7f39d0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.296+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f39e407b8c0 con 0x7f39f0154090 2023-05-16T07:14:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.296+0000 7f39ee59c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f39d005ee70 0x7f39d0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.297+0000 7f39ee59c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f39d005ee70 0x7f39d0061330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f39e00036e0 tx=0x7f39e0009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.300+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f39e404cea0 con 0x7f39f0154090 2023-05-16T07:14:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.484+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39f0155bc0 con 0x7f39d005ee70 2023-05-16T07:14:25.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.486+0000 7f39de7fc700 1 -- 172.21.15.92:0/892748688 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f39f0155bc0 con 0x7f39d005ee70 2023-05-16T07:14:25.490 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:25.490 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:28", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f39d005ee70 msgr2=0x7f39d0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f39d005ee70 0x7f39d0061330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f39e00036e0 tx=0x7f39e0009040).stop 2023-05-16T07:14:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 msgr2=0x7f39f01503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 0x7f39f01503f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f39f006bb30 tx=0x7f39e4017000).stop 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 shutdown_connections 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f39d005ee70 0x7f39d0061330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39f0154470 0x7f39f00c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39f0068160 0x7f39f0152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 --2- 172.21.15.92:0/892748688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f0154090 0x7f39f01503f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 >> 172.21.15.92:0/892748688 conn(0x7f39f0067920 msgr2=0x7f39f00c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:25.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 shutdown_connections 2023-05-16T07:14:25.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.488+0000 7f39f484c700 1 -- 172.21.15.92:0/892748688 wait complete. 2023-05-16T07:14:25.503 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:28", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:25.503 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:28', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:25.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:25.504 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-05-16T07:14:25.519 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:25.519 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:25.519 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:25.519 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:25.521 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:25.521 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_04_00_UTC', 'scheduled-2023-05-16-07_05_00_UTC', 'scheduled-2023-05-16-07_06_00_UTC', 'scheduled-2023-05-16-07_07_00_UTC', 'scheduled-2023-05-16-07_08_00_UTC', 'scheduled-2023-05-16-07_09_00_UTC', 'scheduled-2023-05-16-07_10_00_UTC', 'scheduled-2023-05-16-07_11_00_UTC', 'scheduled-2023-05-16-07_12_00_UTC', 'scheduled-2023-05-16-07_13_00_UTC', 'scheduled-2023-05-16-07_14_00_UTC'] 2023-05-16T07:14:25.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-05-16T07:14:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.669+0000 7fc55c5b9700 1 Processor -- start 2023-05-16T07:14:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.670+0000 7fc55c5b9700 1 -- start start 2023-05-16T07:14:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.670+0000 7fc55c5b9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc554154090 0x7fc5540aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.670+0000 7fc55c5b9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc554154470 0x7fc5540a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.670+0000 7fc55c5b9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5540c94d0 con 0x7fc5540aa150 2023-05-16T07:14:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.670+0000 7fc55c5b9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540aa5a0 con 0x7fc554154470 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.670+0000 7fc55c5b9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540cb7e0 con 0x7fc554154090 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559b54700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc554154470 0x7fc5540a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc55ab56700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5540aa150 0x7fc554068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53978/0 (socket says 172.21.15.92:53978) 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc55ab56700 1 -- 172.21.15.92:0/2350209919 learned_addr learned my addr 172.21.15.92:0/2350209919 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc55a355700 1 --2- 172.21.15.92:0/2350209919 >> v2:172.21.15.158:3300/0 conn(0x7fc554154090 0x7fc5540aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559b54700 1 -- 172.21.15.92:0/2350209919 >> v2:172.21.15.158:3300/0 conn(0x7fc554154090 msgr2=0x7fc5540aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559b54700 1 --2- 172.21.15.92:0/2350209919 >> v2:172.21.15.158:3300/0 conn(0x7fc554154090 0x7fc5540aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559b54700 1 -- 172.21.15.92:0/2350209919 >> v1:172.21.15.92:6789/0 conn(0x7fc5540aa150 legacy=0x7fc554068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559353700 1 -- 172.21.15.92:0/2350209919 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905527253 0 0) 0x7fc5540c94d0 con 0x7fc5540aa150 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559b54700 1 -- 172.21.15.92:0/2350209919 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540c88c0 con 0x7fc554154470 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559b54700 1 --2- 172.21.15.92:0/2350209919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154470 0x7fc5540a9c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fc55406bd10 tx=0x7fc548009060).ready entity=mon.0 client_cookie=879c1dc3965dfba8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559353700 1 -- 172.21.15.92:0/2350209919 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800b580 con 0x7fc554154470 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559353700 1 -- 172.21.15.92:0/2350209919 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54800b6e0 con 0x7fc554154470 2023-05-16T07:14:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.671+0000 7fc559353700 1 -- 172.21.15.92:0/2350209919 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800ba60 con 0x7fc554154470 2023-05-16T07:14:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 -- 172.21.15.92:0/2350209919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154470 msgr2=0x7fc5540a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/2350209919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154470 0x7fc5540a9c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fc55406bd10 tx=0x7fc548009060).stop 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 -- 172.21.15.92:0/2350209919 shutdown_connections 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/2350209919 >> v2:172.21.15.158:3300/0 conn(0x7fc554154090 0x7fc5540aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/2350209919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154470 0x7fc5540a9c60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 -- 172.21.15.92:0/2350209919 >> 172.21.15.92:0/2350209919 conn(0x7fc554067920 msgr2=0x7fc5540cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 -- 172.21.15.92:0/2350209919 shutdown_connections 2023-05-16T07:14:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.672+0000 7fc55c5b9700 1 -- 172.21.15.92:0/2350209919 wait complete. 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.673+0000 7fc55c5b9700 1 Processor -- start 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.673+0000 7fc55c5b9700 1 -- start start 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5540aa150 0x7fc55413f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 0x7fc554156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc554154470 0x7fc5540ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc554155860 con 0x7fc554154090 2023-05-16T07:14:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc554156ae0 con 0x7fc554154470 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc554155580 con 0x7fc5540aa150 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 0x7fc554156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 0x7fc554156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43234/0 (socket says 172.21.15.92:43234) 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 -- 172.21.15.92:0/1059747608 learned_addr learned my addr 172.21.15.92:0/1059747608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 -- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5540aa150 msgr2=0x7fc55413f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5540aa150 0x7fc55413f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55ab56700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc554154470 0x7fc5540ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 -- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc554154470 msgr2=0x7fc5540ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc554154470 0x7fc5540ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 -- 172.21.15.92:0/1059747608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480092a0 con 0x7fc554154090 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc559b54700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 0x7fc554156f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fc55406bb30 tx=0x7fc548017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800b850 con 0x7fc554154090 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54800b9b0 con 0x7fc554154090 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54801b690 con 0x7fc554154090 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc554156490 con 0x7fc554154090 2023-05-16T07:14:25.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.674+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55413fd70 con 0x7fc554154090 2023-05-16T07:14:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.675+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc554066730 con 0x7fc554154090 2023-05-16T07:14:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.676+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fc54800f070 con 0x7fc554154090 2023-05-16T07:14:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.676+0000 7fc5527fc700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc54005ee70 0x7fc540061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:25.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.676+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fc54807b430 con 0x7fc554154090 2023-05-16T07:14:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.676+0000 7fc55a355700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc54005ee70 0x7fc540061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.677+0000 7fc55a355700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc54005ee70 0x7fc540061330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fc544007460 tx=0x7fc544009a10).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.680+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc54804c990 con 0x7fc554154090 2023-05-16T07:14:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.864+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5540c8ba0 con 0x7fc54005ee70 2023-05-16T07:14:25.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.866+0000 7fc5527fc700 1 -- 172.21.15.92:0/1059747608 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc5540c8ba0 con 0x7fc54005ee70 2023-05-16T07:14:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:25.870 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:29", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc54005ee70 msgr2=0x7fc540061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc54005ee70 0x7fc540061330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fc544007460 tx=0x7fc544009a10).stop 2023-05-16T07:14:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 msgr2=0x7fc554156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 0x7fc554156f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fc55406bb30 tx=0x7fc548017000).stop 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 shutdown_connections 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc54005ee70 0x7fc540061330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5540aa150 0x7fc55413f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc554154470 0x7fc5540ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 --2- 172.21.15.92:0/1059747608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554154090 0x7fc554156f70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 >> 172.21.15.92:0/1059747608 conn(0x7fc554067920 msgr2=0x7fc55414edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 shutdown_connections 2023-05-16T07:14:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.868+0000 7fc55c5b9700 1 -- 172.21.15.92:0/1059747608 wait complete. 2023-05-16T07:14:25.883 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:03:29", "first": "2023-05-16T07:04:00", "last": "2023-05-16T07:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-05-16T07:14:25.883 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:03:29', 'first': '2023-05-16T07:04:00', 'last': '2023-05-16T07:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-05-16T07:14:25.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-05-16T07:14:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.995+0000 7f1b8dc52700 1 Processor -- start 2023-05-16T07:14:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.995+0000 7f1b8dc52700 1 -- start start 2023-05-16T07:14:26.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.996+0000 7f1b8dc52700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1b88150920 0x7f1b880a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.996+0000 7f1b8dc52700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b88151520 con 0x7f1b88150d00 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.996+0000 7f1b8dc52700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b880a7aa0 con 0x7f1b88062980 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.996+0000 7f1b8dc52700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b881520c0 con 0x7f1b88150920 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.996+0000 7f1b87fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b88150d00 0x7f1b881519b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53992/0 (socket says 172.21.15.92:53992) 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b87fff700 1 -- 172.21.15.92:0/373655220 learned_addr learned my addr 172.21.15.92:0/373655220 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b8cc50700 1 --2- 172.21.15.92:0/373655220 >> v2:172.21.15.158:3300/0 conn(0x7f1b88150920 0x7f1b880a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094870494 0 0) 0x7f1b88151520 con 0x7f1b88150d00 2023-05-16T07:14:26.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b740033f0 con 0x7f1b88150d00 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3284256718 0 0) 0x7f1b740033f0 con 0x7f1b88150d00 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 >> v2:172.21.15.158:3300/0 conn(0x7f1b88150920 msgr2=0x7f1b880a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 --2- 172.21.15.92:0/373655220 >> v2:172.21.15.158:3300/0 conn(0x7f1b88150920 0x7f1b880a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 >> v1:172.21.15.158:6789/0 conn(0x7f1b88062980 legacy=0x7f1b88154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880a8480 con 0x7f1b88150d00 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f1b70001470 con 0x7f1b88150d00 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 >> v1:172.21.15.92:6789/0 conn(0x7f1b88150d00 legacy=0x7f1b881519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b740038e0 0x7f1b74003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b74004690 0x7f1b74004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b881520c0 con 0x7f1b74005000 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7400bf60 con 0x7f1b74004690 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7400c0f0 con 0x7f1b740038e0 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b8d451700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b7400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b87fff700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b74004690 0x7f1b74004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.997+0000 7f1b8cc50700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b740038e0 0x7f1b74003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 -- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b740038e0 msgr2=0x7f1b74003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b740038e0 0x7f1b74003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 -- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b74004690 msgr2=0x7f1b74004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b74004690 0x7f1b74004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 -- 172.21.15.92:0/373655220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740033f0 con 0x7f1b74005000 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 -- 172.21.15.92:0/373655220 reap_dead start 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8d451700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b7400b8e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f1b80001730 tx=0x7f1b80007bb0).ready entity=mon.0 client_cookie=b6d268bc046d4310 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007020 con 0x7f1b74005000 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80007180 con 0x7f1b74005000 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b877fe700 1 -- 172.21.15.92:0/373655220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007520 con 0x7f1b74005000 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 -- 172.21.15.92:0/373655220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 msgr2=0x7f1b7400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b7400b8e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f1b80001730 tx=0x7f1b80007bb0).stop 2023-05-16T07:14:26.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 -- 172.21.15.92:0/373655220 shutdown_connections 2023-05-16T07:14:26.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/373655220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b7400b8e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 -- 172.21.15.92:0/373655220 >> 172.21.15.92:0/373655220 conn(0x7f1b88068a30 msgr2=0x7f1b880c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:26.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 -- 172.21.15.92:0/373655220 shutdown_connections 2023-05-16T07:14:26.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.998+0000 7f1b8dc52700 1 -- 172.21.15.92:0/373655220 wait complete. 2023-05-16T07:14:26.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:25.999+0000 7f1b8dc52700 1 Processor -- start 2023-05-16T07:14:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 -- start start 2023-05-16T07:14:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b8814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b880abd10 0x7f1b880ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b880a5690 0x7f1b880a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880aaa20 con 0x7f1b74005000 2023-05-16T07:14:26.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880ab880 con 0x7f1b880a5690 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8dc52700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880a6000 con 0x7f1b880abd10 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8cc50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b8814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8cc50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b8814b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43252/0 (socket says 172.21.15.92:43252) 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8cc50700 1 -- 172.21.15.92:0/2096725615 learned_addr learned my addr 172.21.15.92:0/2096725615 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b87fff700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b880abd10 0x7f1b880ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.000+0000 7f1b8d451700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b880a5690 0x7f1b880a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8cc50700 1 -- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b880abd10 msgr2=0x7f1b880ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8cc50700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b880abd10 0x7f1b880ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8cc50700 1 -- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b880a5690 msgr2=0x7f1b880a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8cc50700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b880a5690 0x7f1b880a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8cc50700 1 -- 172.21.15.92:0/2096725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880aac70 con 0x7f1b74005000 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8cc50700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b8814b3c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f1b78011e80 tx=0x7f1b7800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800e040 con 0x7f1b74005000 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7800f040 con 0x7f1b74005000 2023-05-16T07:14:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b880aaf50 con 0x7f1b74005000 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800fe50 con 0x7f1b74005000 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.001+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b880ab510 con 0x7f1b74005000 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.002+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b880abb60 con 0x7f1b74005000 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.002+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f1b78015660 con 0x7f1b74005000 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.002+0000 7f1b84ff9700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b6805ee70 0x7f1b68061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.003+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f1b7807ad10 con 0x7f1b74005000 2023-05-16T07:14:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.003+0000 7f1b87fff700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b6805ee70 0x7f1b68061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.003+0000 7f1b87fff700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b6805ee70 0x7f1b68061330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1b700015a0 tx=0x7f1b70001e50).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.007+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1b7804c1c0 con 0x7f1b74005000 2023-05-16T07:14:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.190+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b881534b0 con 0x7f1b6805ee70 2023-05-16T07:14:26.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.195+0000 7f1b84ff9700 1 -- 172.21.15.92:0/2096725615 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f1b881534b0 con 0x7f1b6805ee70 2023-05-16T07:14:26.199 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-05-16T07:14:26.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b6805ee70 msgr2=0x7f1b68061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b6805ee70 0x7f1b68061330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1b700015a0 tx=0x7f1b70001e50).stop 2023-05-16T07:14:26.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 msgr2=0x7f1b8814b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:26.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b8814b3c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f1b78011e80 tx=0x7f1b7800b040).stop 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 shutdown_connections 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b6805ee70 0x7f1b68061330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b880a5690 0x7f1b880a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b74005000 0x7f1b8814b3c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 --2- 172.21.15.92:0/2096725615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b880abd10 0x7f1b880ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 >> 172.21.15.92:0/2096725615 conn(0x7f1b88068a30 msgr2=0x7f1b88062d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:26.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 shutdown_connections 2023-05-16T07:14:26.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:26.197+0000 7f1b8dc52700 1 -- 172.21.15.92:0/2096725615 wait complete. 2023-05-16T07:14:26.212 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-05-16T07:14:26.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-05-16T07:14:26.228 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:26.228 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:26.228 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:26.228 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:26.229 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:26.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:26.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.230 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:26.289 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:26.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.290 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:26.350 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:26.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.351 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:26.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:26.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.412 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:26.473 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:26.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.474 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:26.535 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:26.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.536 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:26.596 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:26.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.597 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:26.657 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:26.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:26.717 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:26.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:26.776 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:26.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.776 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:26.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:26.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.835 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:26.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:26.894 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-05-16T07:14:26.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-05-16T07:14:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.102+0000 7fda31ef4700 1 Processor -- start 2023-05-16T07:14:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.102+0000 7fda31ef4700 1 -- start start 2023-05-16T07:14:27.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda31ef4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda2c14e6f0 0x7fda2c155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:27.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda31ef4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c14ead0 con 0x7fda2c0c4d40 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda31ef4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c154cb0 con 0x7fda2c0629c0 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda31ef4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c0c6020 con 0x7fda2c14e6f0 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda2c14e6f0 0x7fda2c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda2c14e6f0 0x7fda2c155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43260/0 (socket says 172.21.15.92:43260) 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fda2c0c4d40 0x7fda2c0622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54010/0 (socket says 172.21.15.92:54010) 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 -- 172.21.15.92:0/700113502 learned_addr learned my addr 172.21.15.92:0/700113502 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 -- 172.21.15.92:0/700113502 >> v1:172.21.15.158:6789/0 conn(0x7fda2c0629c0 legacy=0x7fda2c063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 -- 172.21.15.92:0/700113502 >> v1:172.21.15.92:6789/0 conn(0x7fda2c0c4d40 legacy=0x7fda2c0622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 -- 172.21.15.92:0/700113502 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c154660 con 0x7fda2c14e6f0 2023-05-16T07:14:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2b7fe700 1 --2- 172.21.15.92:0/700113502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c14e6f0 0x7fda2c155140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fda1c009f20 tx=0x7fda1c009840).ready entity=mon.0 client_cookie=d4f46b8ea1b46d9e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.103+0000 7fda2a7fc700 1 -- 172.21.15.92:0/700113502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00e070 con 0x7fda2c14e6f0 2023-05-16T07:14:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda2a7fc700 1 -- 172.21.15.92:0/700113502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c00ba40 con 0x7fda2c14e6f0 2023-05-16T07:14:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda2a7fc700 1 -- 172.21.15.92:0/700113502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00bdc0 con 0x7fda2c14e6f0 2023-05-16T07:14:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 -- 172.21.15.92:0/700113502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c14e6f0 msgr2=0x7fda2c155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 --2- 172.21.15.92:0/700113502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c14e6f0 0x7fda2c155140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fda1c009f20 tx=0x7fda1c009840).stop 2023-05-16T07:14:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 -- 172.21.15.92:0/700113502 shutdown_connections 2023-05-16T07:14:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 --2- 172.21.15.92:0/700113502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c14e6f0 0x7fda2c155140 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 -- 172.21.15.92:0/700113502 >> 172.21.15.92:0/700113502 conn(0x7fda2c05e820 msgr2=0x7fda2c0c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 -- 172.21.15.92:0/700113502 shutdown_connections 2023-05-16T07:14:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.104+0000 7fda31ef4700 1 -- 172.21.15.92:0/700113502 wait complete. 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.105+0000 7fda31ef4700 1 Processor -- start 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.105+0000 7fda31ef4700 1 -- start start 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda31ef4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda2c0629c0 0x7fda2c0c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda31ef4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 0x7fda2c0a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda31ef4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda2c14e6f0 0x7fda2c0aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda31ef4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0a8920 con 0x7fda2c0c4d40 2023-05-16T07:14:27.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda31ef4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0ac500 con 0x7fda2c14e6f0 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda31ef4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0ac7e0 con 0x7fda2c0629c0 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 0x7fda2c0a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 0x7fda2c0a9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43268/0 (socket says 172.21.15.92:43268) 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 -- 172.21.15.92:0/1590410065 learned_addr learned my addr 172.21.15.92:0/1590410065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2b7fe700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda2c0629c0 0x7fda2c0c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2bfff700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda2c14e6f0 0x7fda2c0aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:27.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 -- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda2c0629c0 msgr2=0x7fda2c0c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda2c0629c0 0x7fda2c0c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 -- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda2c14e6f0 msgr2=0x7fda2c0aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda2c14e6f0 0x7fda2c0aba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2affd700 1 -- 172.21.15.92:0/1590410065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009ae0 con 0x7fda2c0c4d40 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.106+0000 7fda2bfff700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda2c14e6f0 0x7fda2c0aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.107+0000 7fda2affd700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 0x7fda2c0a9370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fda20007fd0 tx=0x7fda20007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.107+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200072e0 con 0x7fda2c0c4d40 2023-05-16T07:14:27.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.107+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20007440 con 0x7fda2c0c4d40 2023-05-16T07:14:27.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.107+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c0a8b70 con 0x7fda2c0c4d40 2023-05-16T07:14:27.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.107+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200058f0 con 0x7fda2c0c4d40 2023-05-16T07:14:27.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.107+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c0ab210 con 0x7fda2c0c4d40 2023-05-16T07:14:27.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.108+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c0a9090 con 0x7fda2c0c4d40 2023-05-16T07:14:27.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.108+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fda2001a060 con 0x7fda2c0c4d40 2023-05-16T07:14:27.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.108+0000 7fda137fe700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda1805ee70 0x7fda18061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:27.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.108+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fda20079390 con 0x7fda2c0c4d40 2023-05-16T07:14:27.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.109+0000 7fda2b7fe700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda1805ee70 0x7fda18061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:27.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.109+0000 7fda2b7fe700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda1805ee70 0x7fda18061330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fda1c00c010 tx=0x7fda1c009840).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:27.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.112+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fda2c0a9090 con 0x7fda2c0c4d40 2023-05-16T07:14:27.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.298+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fda2c0a9090 con 0x7fda1805ee70 2023-05-16T07:14:27.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.302+0000 7fda137fe700 1 -- 172.21.15.92:0/1590410065 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fda2c0a9090 con 0x7fda1805ee70 2023-05-16T07:14:27.306 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-05-16T07:14:27.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda1805ee70 msgr2=0x7fda18061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:27.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda1805ee70 0x7fda18061330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fda1c00c010 tx=0x7fda1c009840).stop 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 msgr2=0x7fda2c0a9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 0x7fda2c0a9370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fda20007fd0 tx=0x7fda20007df0).stop 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 shutdown_connections 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fda1805ee70 0x7fda18061330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fda2c0629c0 0x7fda2c0c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda2c14e6f0 0x7fda2c0aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 --2- 172.21.15.92:0/1590410065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda2c0c4d40 0x7fda2c0a9370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:27.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 >> 172.21.15.92:0/1590410065 conn(0x7fda2c05e820 msgr2=0x7fda2c0c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:27.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 shutdown_connections 2023-05-16T07:14:27.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:27.304+0000 7fda31ef4700 1 -- 172.21.15.92:0/1590410065 wait complete. 2023-05-16T07:14:27.320 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-05-16T07:14:27.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.320 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-05-16T07:14:27.335 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:27.335 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:27.335 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:27.335 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:27.337 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:27.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.337 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:27.395 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:27.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.396 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:27.455 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:27.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:27.514 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:27.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.515 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:27.573 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:27.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:27.634 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:27.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.635 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:27.694 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:27.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.695 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:27.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:27.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.755 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:27.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:27.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:27.874 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:27.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.875 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:27.935 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:27.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:27.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:28.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.000 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-05-16T07:14:28.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-05-16T07:14:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.208+0000 7f62bbaf4700 1 Processor -- start 2023-05-16T07:14:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.208+0000 7f62bbaf4700 1 -- start start 2023-05-16T07:14:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.209+0000 7f62bbaf4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f62b4154470 0x7f62b40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.209+0000 7f62bbaf4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b40c94d0 con 0x7f62b4154090 2023-05-16T07:14:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.209+0000 7f62bbaf4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b40cb7e0 con 0x7f62b4068160 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.209+0000 7f62bbaf4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b40c88c0 con 0x7f62b4154470 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.209+0000 7f62b9890700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f62b4154090 0x7f62b40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54022/0 (socket says 172.21.15.92:54022) 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.209+0000 7f62b9890700 1 -- 172.21.15.92:0/2507036025 learned_addr learned my addr 172.21.15.92:0/2507036025 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b908f700 1 --2- 172.21.15.92:0/2507036025 >> v2:172.21.15.158:3300/0 conn(0x7f62b4154470 0x7f62b40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715603344 0 0) 0x7f62b40c94d0 con 0x7f62b4154090 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c0033f0 con 0x7f62b4154090 2023-05-16T07:14:28.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025400525 0 0) 0x7f629c0033f0 con 0x7f62b4154090 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 >> v2:172.21.15.158:3300/0 conn(0x7f62b4154470 msgr2=0x7f62b40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 --2- 172.21.15.92:0/2507036025 >> v2:172.21.15.158:3300/0 conn(0x7f62b4154470 0x7f62b40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 >> v1:172.21.15.158:6789/0 conn(0x7f62b4068160 legacy=0x7f62b40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40c8ba0 con 0x7f62b4154090 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f62a40017f0 con 0x7f62b4154090 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 >> v1:172.21.15.92:6789/0 conn(0x7f62b4154090 legacy=0x7f62b40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f629c0038e0 0x7f629c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c004690 0x7f629c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f629c005000 0x7f629c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40c88c0 con 0x7f629c0038e0 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c00bf60 con 0x7f629c005000 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c00c0f0 con 0x7f629c004690 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b9890700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f629c0038e0 0x7f629c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b908f700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c004690 0x7f629c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62ba091700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f629c005000 0x7f629c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b9890700 1 -- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c004690 msgr2=0x7f629c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b9890700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c004690 0x7f629c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b9890700 1 -- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f629c005000 msgr2=0x7f629c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:28.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b9890700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f629c005000 0x7f629c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62b9890700 1 -- 172.21.15.92:0/2507036025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0033f0 con 0x7f629c0038e0 2023-05-16T07:14:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.210+0000 7f62ba091700 1 -- 172.21.15.92:0/2507036025 reap_dead start 2023-05-16T07:14:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62b9890700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f629c0038e0 0x7f629c003d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f62a400c480 tx=0x7f62a400c790).ready entity=mon.0 client_cookie=84c88a8000d9dcbf server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4002030 con 0x7f629c0038e0 2023-05-16T07:14:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a400ce20 con 0x7f629c0038e0 2023-05-16T07:14:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62b888e700 1 -- 172.21.15.92:0/2507036025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4005d70 con 0x7f629c0038e0 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 -- 172.21.15.92:0/2507036025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f629c0038e0 msgr2=0x7f629c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f629c0038e0 0x7f629c003d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f62a400c480 tx=0x7f62a400c790).stop 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 -- 172.21.15.92:0/2507036025 shutdown_connections 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/2507036025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f629c0038e0 0x7f629c003d00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 -- 172.21.15.92:0/2507036025 >> 172.21.15.92:0/2507036025 conn(0x7f62b4067920 msgr2=0x7f62b40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 -- 172.21.15.92:0/2507036025 shutdown_connections 2023-05-16T07:14:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.211+0000 7f62bbaf4700 1 -- 172.21.15.92:0/2507036025 wait complete. 2023-05-16T07:14:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.212+0000 7f62bbaf4700 1 Processor -- start 2023-05-16T07:14:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.212+0000 7f62bbaf4700 1 -- start start 2023-05-16T07:14:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62bbaf4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c0038e0 0x7f62b414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62bbaf4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62b40c3610 0x7f62b41532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62bbaf4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 0x7f62b40c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62bbaf4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40c3e60 con 0x7f62b4151330 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62bbaf4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40ae8b0 con 0x7f62b40c3610 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62bbaf4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40aeb90 con 0x7f629c0038e0 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 0x7f62b40c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 0x7f62b40c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43290/0 (socket says 172.21.15.92:43290) 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 -- 172.21.15.92:0/242712550 learned_addr learned my addr 172.21.15.92:0/242712550 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62b908f700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62b40c3610 0x7f62b41532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62b9890700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c0038e0 0x7f62b414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 -- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c0038e0 msgr2=0x7f62b414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c0038e0 0x7f62b414fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 -- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62b40c3610 msgr2=0x7f62b41532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62b40c3610 0x7f62b41532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62ba091700 1 -- 172.21.15.92:0/242712550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40aee70 con 0x7f62b4151330 2023-05-16T07:14:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.213+0000 7f62b9890700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c0038e0 0x7f62b414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.214+0000 7f62ba091700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 0x7f62b40c46f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f62b405efb0 tx=0x7f62ac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.214+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00ed90 con 0x7f62b4151330 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.214+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac007030 con 0x7f62b4151330 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.214+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b40af150 con 0x7f62b4151330 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.214+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0073b0 con 0x7f62b4151330 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.214+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b40c3a90 con 0x7f62b4151330 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.215+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b4066730 con 0x7f62b4151330 2023-05-16T07:14:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.215+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f62ac0068b0 con 0x7f62b4151330 2023-05-16T07:14:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.215+0000 7f62b1ffb700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62a005ee70 0x7f62a0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.215+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f62ac079920 con 0x7f62b4151330 2023-05-16T07:14:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.216+0000 7f62b9890700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62a005ee70 0x7f62a0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.216+0000 7f62b9890700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62a005ee70 0x7f62a0061330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f62a400c7d0 tx=0x7f62a4015000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.219+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f62ac04add0 con 0x7f62b4151330 2023-05-16T07:14:28.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.407+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b406c350 con 0x7f62a005ee70 2023-05-16T07:14:28.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.411+0000 7f62b1ffb700 1 -- 172.21.15.92:0/242712550 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f62b406c350 con 0x7f62a005ee70 2023-05-16T07:14:28.415 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-05-16T07:14:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62a005ee70 msgr2=0x7f62a0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62a005ee70 0x7f62a0061330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f62a400c7d0 tx=0x7f62a4015000).stop 2023-05-16T07:14:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 msgr2=0x7f62b40c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 0x7f62b40c46f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f62b405efb0 tx=0x7f62ac009040).stop 2023-05-16T07:14:28.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 shutdown_connections 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62a005ee70 0x7f62a0061330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f629c0038e0 0x7f62b414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4151330 0x7f62b40c46f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 --2- 172.21.15.92:0/242712550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62b40c3610 0x7f62b41532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 >> 172.21.15.92:0/242712550 conn(0x7f62b4067920 msgr2=0x7f62b40ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 shutdown_connections 2023-05-16T07:14:28.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:28.413+0000 7f62bbaf4700 1 -- 172.21.15.92:0/242712550 wait complete. 2023-05-16T07:14:28.428 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-05-16T07:14:28.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.428 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-05-16T07:14:28.443 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:28.445 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:28.445 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:28.445 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:28.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:28.447 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:28.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.447 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:28.504 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:28.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.505 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:28.565 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:28.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.566 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:28.629 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:28.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.630 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:28.704 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:28.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.705 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:28.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:28.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.781 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:28.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:28.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.841 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:28.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:28.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.906 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:28.973 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:28.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:28.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:29.033 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:29.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.034 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:29.092 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:29.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:29.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.152 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-05-16T07:14:29.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-05-16T07:14:29.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.358+0000 7f426359e700 1 Processor -- start 2023-05-16T07:14:29.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.358+0000 7f426359e700 1 -- start start 2023-05-16T07:14:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f426359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42640c1160 0x7f42640c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f426359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426406a140 con 0x7f4264153f90 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f426359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426406a420 con 0x7f42640c61f0 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f426359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4264069f60 con 0x7f42640c1160 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f4261d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42640c1160 0x7f42640c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f4261d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42640c1160 0x7f42640c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43298/0 (socket says 172.21.15.92:43298) 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f4261d9b700 1 -- 172.21.15.92:0/2290690827 learned_addr learned my addr 172.21.15.92:0/2290690827 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:29.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.359+0000 7f426259c700 1 --1- 172.21.15.92:0/2290690827 >> v1:172.21.15.92:6789/0 conn(0x7f4264153f90 0x7f42640a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54042/0 (socket says 172.21.15.92:54042) 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f4261d9b700 1 -- 172.21.15.92:0/2290690827 >> v1:172.21.15.158:6789/0 conn(0x7f42640c61f0 legacy=0x7f4264154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f4261d9b700 1 -- 172.21.15.92:0/2290690827 >> v1:172.21.15.92:6789/0 conn(0x7f4264153f90 legacy=0x7f42640a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426159a700 1 -- 172.21.15.92:0/2290690827 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973201166 0 0) 0x7f426406a140 con 0x7f4264153f90 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f4261d9b700 1 -- 172.21.15.92:0/2290690827 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426406a700 con 0x7f42640c1160 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f4261d9b700 1 --2- 172.21.15.92:0/2290690827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c1160 0x7f42640c5d00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f424c006fd0 tx=0x7f424c0090f0).ready entity=mon.0 client_cookie=95103c08656c1867 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426159a700 1 -- 172.21.15.92:0/2290690827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00b600 con 0x7f42640c1160 2023-05-16T07:14:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426159a700 1 -- 172.21.15.92:0/2290690827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00b760 con 0x7f42640c1160 2023-05-16T07:14:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426159a700 1 -- 172.21.15.92:0/2290690827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00bae0 con 0x7f42640c1160 2023-05-16T07:14:29.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 -- 172.21.15.92:0/2290690827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c1160 msgr2=0x7f42640c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 --2- 172.21.15.92:0/2290690827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c1160 0x7f42640c5d00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f424c006fd0 tx=0x7f424c0090f0).stop 2023-05-16T07:14:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 -- 172.21.15.92:0/2290690827 shutdown_connections 2023-05-16T07:14:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 --2- 172.21.15.92:0/2290690827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c1160 0x7f42640c5d00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 -- 172.21.15.92:0/2290690827 >> 172.21.15.92:0/2290690827 conn(0x7f42640cb600 msgr2=0x7f42640c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 -- 172.21.15.92:0/2290690827 shutdown_connections 2023-05-16T07:14:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.360+0000 7f426359e700 1 -- 172.21.15.92:0/2290690827 wait complete. 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.361+0000 7f426359e700 1 Processor -- start 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 -- start start 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42640c1160 0x7f42640aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 0x7f426414e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4264153f90 0x7f4264164080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c2ba0 con 0x7f42640c61f0 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c36b0 con 0x7f42640c1160 2023-05-16T07:14:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c2d80 con 0x7f4264153f90 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f4261d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 0x7f426414e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f4261d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 0x7f426414e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43302/0 (socket says 172.21.15.92:43302) 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f4261d9b700 1 -- 172.21.15.92:0/2255850594 learned_addr learned my addr 172.21.15.92:0/2255850594 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f426259c700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42640c1160 0x7f42640aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.362+0000 7f4262d9d700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4264153f90 0x7f4264164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f4261d9b700 1 -- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4264153f90 msgr2=0x7f4264164080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f4261d9b700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4264153f90 0x7f4264164080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f4261d9b700 1 -- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42640c1160 msgr2=0x7f42640aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f4261d9b700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42640c1160 0x7f42640aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f4261d9b700 1 -- 172.21.15.92:0/2255850594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c009370 con 0x7f42640c61f0 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f4261d9b700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 0x7f426414e7a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f424c009130 tx=0x7f424c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00b8d0 con 0x7f42640c61f0 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00ba30 con 0x7f42640c61f0 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42640c3060 con 0x7f42640c61f0 2023-05-16T07:14:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c01a430 con 0x7f42640c61f0 2023-05-16T07:14:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.363+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42640af010 con 0x7f42640c61f0 2023-05-16T07:14:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.364+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42640af200 con 0x7f42640c61f0 2023-05-16T07:14:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.364+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f424c02b070 con 0x7f42640c61f0 2023-05-16T07:14:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.365+0000 7f425a7fc700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f424405ee70 0x7f4244061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.365+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6858+0+0 (secure 0 0 0) 0x7f424c07b6d0 con 0x7f42640c61f0 2023-05-16T07:14:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.365+0000 7f426259c700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f424405ee70 0x7f4244061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.365+0000 7f426259c700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f424405ee70 0x7f4244061330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4254007fd0 tx=0x7f4254001500).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.369+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f424c04c860 con 0x7f42640c61f0 2023-05-16T07:14:29.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.554+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f42640c3500 con 0x7f424405ee70 2023-05-16T07:14:29.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.558+0000 7f425a7fc700 1 -- 172.21.15.92:0/2255850594 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f42640c3500 con 0x7f424405ee70 2023-05-16T07:14:29.562 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-05-16T07:14:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f424405ee70 msgr2=0x7f4244061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f424405ee70 0x7f4244061330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4254007fd0 tx=0x7f4254001500).stop 2023-05-16T07:14:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 msgr2=0x7f426414e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 0x7f426414e7a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f424c009130 tx=0x7f424c017000).stop 2023-05-16T07:14:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 shutdown_connections 2023-05-16T07:14:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f424405ee70 0x7f4244061330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4264153f90 0x7f4264164080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42640c1160 0x7f42640aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 --2- 172.21.15.92:0/2255850594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42640c61f0 0x7f426414e7a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:29.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 >> 172.21.15.92:0/2255850594 conn(0x7f42640cb600 msgr2=0x7f42640c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:29.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 shutdown_connections 2023-05-16T07:14:29.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:29.560+0000 7f426359e700 1 -- 172.21.15.92:0/2255850594 wait complete. 2023-05-16T07:14:29.575 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-05-16T07:14:29.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.576 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-05-16T07:14:29.590 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:29.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:29.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:29.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:29.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:29.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:29.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:29.592 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:29.592 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:29.592 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:29.592 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:29.593 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:29.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.593 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:29.654 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:29.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.655 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:29.725 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:29.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.726 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:29.786 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:29.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.787 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:29.846 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:29.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:29.906 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:29.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.906 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:29.965 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:29.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:29.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:30.025 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:30.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.026 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:30.086 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:30.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.087 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:30.146 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:30.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.146 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:30.211 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:30.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.212 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:30.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.272 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-05-16T07:14:30.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-05-16T07:14:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.481+0000 7f8912d61700 1 Processor -- start 2023-05-16T07:14:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.481+0000 7f8912d61700 1 -- start start 2023-05-16T07:14:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f8912d61700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f890c154470 0x7f890c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f8912d61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0c94d0 con 0x7f890c154090 2023-05-16T07:14:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f8912d61700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0cb7e0 con 0x7f890c068160 2023-05-16T07:14:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f8912d61700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c0c88c0 con 0x7f890c154470 2023-05-16T07:14:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f8910afd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f890c154090 0x7f890c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54056/0 (socket says 172.21.15.92:54056) 2023-05-16T07:14:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f8910afd700 1 -- 172.21.15.92:0/2641982898 learned_addr learned my addr 172.21.15.92:0/2641982898 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756437173 0 0) 0x7f890c0c94d0 con 0x7f890c154090 2023-05-16T07:14:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890bfff700 1 --2- 172.21.15.92:0/2641982898 >> v2:172.21.15.158:3300/0 conn(0x7f890c154470 0x7f890c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f40033f0 con 0x7f890c154090 2023-05-16T07:14:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2657474320 0 0) 0x7f88f40033f0 con 0x7f890c154090 2023-05-16T07:14:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 >> v2:172.21.15.158:3300/0 conn(0x7f890c154470 msgr2=0x7f890c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 --2- 172.21.15.92:0/2641982898 >> v2:172.21.15.158:3300/0 conn(0x7f890c154470 0x7f890c0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 >> v1:172.21.15.158:6789/0 conn(0x7f890c068160 legacy=0x7f890c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0c8ba0 con 0x7f890c154090 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f88fc001470 con 0x7f890c154090 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 >> v1:172.21.15.92:6789/0 conn(0x7f890c154090 legacy=0x7f890c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.482+0000 7f890b7fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f88f40038e0 0x7f88f4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4004690 0x7f88f4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f4005000 0x7f88f400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0c88c0 con 0x7f88f4005000 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f400bf60 con 0x7f88f4004690 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f400c0f0 con 0x7f88f40038e0 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f4005000 0x7f88f400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890bfff700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4004690 0x7f88f4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f8910afd700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f88f40038e0 0x7f88f4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 -- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f88f40038e0 msgr2=0x7f88f4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f88f40038e0 0x7f88f4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 -- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4004690 msgr2=0x7f88f4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4004690 0x7f88f4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 -- 172.21.15.92:0/2641982898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40033f0 con 0x7f88f4005000 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 -- 172.21.15.92:0/2641982898 reap_dead start 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f89112fe700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f4005000 0x7f88f400b8e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f8904001e30 tx=0x7f8904007f30).ready entity=mon.0 client_cookie=58117dc1e86a0103 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904006a20 con 0x7f88f4005000 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904006b80 con 0x7f88f4005000 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.483+0000 7f890b7fe700 1 -- 172.21.15.92:0/2641982898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400c5d0 con 0x7f88f4005000 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 -- 172.21.15.92:0/2641982898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f4005000 msgr2=0x7f88f400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f4005000 0x7f88f400b8e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f8904001e30 tx=0x7f8904007f30).stop 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 -- 172.21.15.92:0/2641982898 shutdown_connections 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 --2- 172.21.15.92:0/2641982898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f4005000 0x7f88f400b8e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 -- 172.21.15.92:0/2641982898 >> 172.21.15.92:0/2641982898 conn(0x7f890c067920 msgr2=0x7f890c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 -- 172.21.15.92:0/2641982898 shutdown_connections 2023-05-16T07:14:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.484+0000 7f8912d61700 1 -- 172.21.15.92:0/2641982898 wait complete. 2023-05-16T07:14:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.485+0000 7f8912d61700 1 Processor -- start 2023-05-16T07:14:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.485+0000 7f8912d61700 1 -- start start 2023-05-16T07:14:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4005000 0x7f890c14d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f890c0a9e20 0x7f890c0aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 0x7f890c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c1569f0 con 0x7f890c156e80 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c155b00 con 0x7f88f4005000 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c155de0 con 0x7f890c0a9e20 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 0x7f890c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 0x7f890c157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43324/0 (socket says 172.21.15.92:43324) 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 -- 172.21.15.92:0/556960948 learned_addr learned my addr 172.21.15.92:0/556960948 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 -- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f890c0a9e20 msgr2=0x7f890c0aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f890c0a9e20 0x7f890c0aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 -- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4005000 msgr2=0x7f890c14d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4005000 0x7f890c14d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 -- 172.21.15.92:0/556960948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c1560c0 con 0x7f890c156e80 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f89112fe700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 0x7f890c157300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f8904001600 tx=0x7f8904016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400c5d0 con 0x7f890c156e80 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904001250 con 0x7f890c156e80 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c1563a0 con 0x7f890c156e80 2023-05-16T07:14:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904006a20 con 0x7f890c156e80 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.486+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c14de30 con 0x7f890c156e80 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.487+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c156cd0 con 0x7f890c156e80 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.487+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f8904010070 con 0x7f890c156e80 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.488+0000 7f8908ff9700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f88f805ee70 0x7f88f8061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.488+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6874+0+0 (secure 0 0 0) 0x7f890407b6d0 con 0x7f890c156e80 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.488+0000 7f8910afd700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f88f805ee70 0x7f88f8061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.489+0000 7f8910afd700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f88f805ee70 0x7f88f8061330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f88fc005930 tx=0x7f88fc0014c0).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.492+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f890404c960 con 0x7f890c156e80 2023-05-16T07:14:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.678+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f890c156b30 con 0x7f88f805ee70 2023-05-16T07:14:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.683+0000 7f8908ff9700 1 -- 172.21.15.92:0/556960948 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f890c156b30 con 0x7f88f805ee70 2023-05-16T07:14:30.687 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f88f805ee70 msgr2=0x7f88f8061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f88f805ee70 0x7f88f8061330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f88fc005930 tx=0x7f88fc0014c0).stop 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 msgr2=0x7f890c157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 0x7f890c157300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f8904001600 tx=0x7f8904016000).stop 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 shutdown_connections 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f88f805ee70 0x7f88f8061330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:30.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f890c156e80 0x7f890c157300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:30.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f88f4005000 0x7f890c14d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:30.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 --2- 172.21.15.92:0/556960948 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f890c0a9e20 0x7f890c0aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:30.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 >> 172.21.15.92:0/556960948 conn(0x7f890c067920 msgr2=0x7f890c14e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:30.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 shutdown_connections 2023-05-16T07:14:30.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:30.685+0000 7f8912d61700 1 -- 172.21.15.92:0/556960948 wait complete. 2023-05-16T07:14:30.705 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-05-16T07:14:30.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.706 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-05-16T07:14:30.723 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:30.724 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:30.725 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:30.725 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:30.725 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:30.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:30.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.726 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:30.798 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:30.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.799 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:30.858 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:30.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.859 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:30.924 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:30.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.925 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:30.984 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:30.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:30.985 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:31.045 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:31.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:31.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:31.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.105 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:31.164 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:31.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.165 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:31.225 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:31.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.226 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:31.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:31.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.286 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:31.345 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:31.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.346 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:31.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.406 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-05-16T07:14:31.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-05-16T07:14:31.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.612+0000 7fa0551d3700 1 Processor -- start 2023-05-16T07:14:31.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.612+0000 7fa0551d3700 1 -- start start 2023-05-16T07:14:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa0551d3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa050153f90 0x7fa0500a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa0551d3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa0500c5d00 0x7fa0500c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa0551d3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa0551d3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa050069f60 con 0x7fa050153f90 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa0551d3700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05006a9e0 con 0x7fa0500c5d00 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa0551d3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500c66a0 con 0x7fa0500c0150 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa04ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa050153f90 0x7fa0500a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa04ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa050153f90 0x7fa0500a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43328/0 (socket says 172.21.15.92:43328) 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.613+0000 7fa04ffff700 1 -- 172.21.15.92:0/2959767574 learned_addr learned my addr 172.21.15.92:0/2959767574 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04ffff700 1 -- 172.21.15.92:0/2959767574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 msgr2=0x7fa0500c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04ffff700 1 --2- 172.21.15.92:0/2959767574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04ffff700 1 -- 172.21.15.92:0/2959767574 >> v2:172.21.15.158:3300/0 conn(0x7fa0500c5d00 msgr2=0x7fa0500c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04ffff700 1 --2- 172.21.15.92:0/2959767574 >> v2:172.21.15.158:3300/0 conn(0x7fa0500c5d00 0x7fa0500c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04ffff700 1 -- 172.21.15.92:0/2959767574 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05006a140 con 0x7fa050153f90 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04ffff700 1 --2- 172.21.15.92:0/2959767574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050153f90 0x7fa0500a9d20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fa0500c8240 tx=0x7fa0400090f0).ready entity=mon.0 client_cookie=cf8ed6b1972a4933 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04effd700 1 -- 172.21.15.92:0/2959767574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000b600 con 0x7fa050153f90 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04effd700 1 -- 172.21.15.92:0/2959767574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04000b760 con 0x7fa050153f90 2023-05-16T07:14:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa04effd700 1 -- 172.21.15.92:0/2959767574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000bae0 con 0x7fa050153f90 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 -- 172.21.15.92:0/2959767574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050153f90 msgr2=0x7fa0500a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 --2- 172.21.15.92:0/2959767574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050153f90 0x7fa0500a9d20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fa0500c8240 tx=0x7fa0400090f0).stop 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 -- 172.21.15.92:0/2959767574 shutdown_connections 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 --2- 172.21.15.92:0/2959767574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 --2- 172.21.15.92:0/2959767574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050153f90 0x7fa0500a9d20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 --2- 172.21.15.92:0/2959767574 >> v2:172.21.15.158:3300/0 conn(0x7fa0500c5d00 0x7fa0500c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 -- 172.21.15.92:0/2959767574 >> 172.21.15.92:0/2959767574 conn(0x7fa0500cb600 msgr2=0x7fa0500c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.614+0000 7fa0551d3700 1 -- 172.21.15.92:0/2959767574 shutdown_connections 2023-05-16T07:14:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.615+0000 7fa0551d3700 1 -- 172.21.15.92:0/2959767574 wait complete. 2023-05-16T07:14:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 Processor -- start 2023-05-16T07:14:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 -- start start 2023-05-16T07:14:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 0x7fa0501648b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa050153f90 0x7fa0500b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500adc80 con 0x7fa0500c5d00 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500adf60 con 0x7fa050153f90 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.616+0000 7fa0551d3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500ae240 con 0x7fa0500c0150 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 0x7fa0501648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 0x7fa0501648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43338/0 (socket says 172.21.15.92:43338) 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 -- 172.21.15.92:0/612025750 learned_addr learned my addr 172.21.15.92:0/612025750 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04ffff700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa0549d2700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa050153f90 0x7fa0500b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 -- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 msgr2=0x7fa0500aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 -- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa050153f90 msgr2=0x7fa0500b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa050153f90 0x7fa0500b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 -- 172.21.15.92:0/612025750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040009370 con 0x7fa0500c5d00 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04ffff700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa04f7fe700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 0x7fa0501648b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fa04400b5e0 tx=0x7fa044009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044002320 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa044002480 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044002840 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0500ae520 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.617+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0500af210 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.618+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050083120 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.618+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fa04400f030 con 0x7fa0500c5d00 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.619+0000 7fa037fff700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa03c05ee70 0x7fa03c061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.619+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7034+0+0 (secure 0 0 0) 0x7fa044079cd0 con 0x7fa0500c5d00 2023-05-16T07:14:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.619+0000 7fa04ffff700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa03c05ee70 0x7fa03c061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.619+0000 7fa04ffff700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa03c05ee70 0x7fa03c061330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa040009130 tx=0x7fa040017000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.623+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa04404ae60 con 0x7fa0500c5d00 2023-05-16T07:14:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.841+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa050047150 con 0x7fa03c05ee70 2023-05-16T07:14:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-05-16T07:14:31.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.849+0000 7fa037fff700 1 -- 172.21.15.92:0/612025750 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa050047150 con 0x7fa03c05ee70 2023-05-16T07:14:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa03c05ee70 msgr2=0x7fa03c061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa03c05ee70 0x7fa03c061330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa040009130 tx=0x7fa040017000).stop 2023-05-16T07:14:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 msgr2=0x7fa0501648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 0x7fa0501648b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fa04400b5e0 tx=0x7fa044009060).stop 2023-05-16T07:14:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 shutdown_connections 2023-05-16T07:14:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa03c05ee70 0x7fa03c061330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa0500c0150 0x7fa0500aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa050153f90 0x7fa0500b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 --2- 172.21.15.92:0/612025750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0500c5d00 0x7fa0501648b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.853+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 >> 172.21.15.92:0/612025750 conn(0x7fa0500cb600 msgr2=0x7fa0500aaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.854+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 shutdown_connections 2023-05-16T07:14:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:31.854+0000 7fa0551d3700 1 -- 172.21.15.92:0/612025750 wait complete. 2023-05-16T07:14:31.876 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-05-16T07:14:31.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.877 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-05-16T07:14:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:31.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:31.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.894 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:31.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:31.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:31.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:32.017 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:32.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.018 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:32.080 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:32.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:32.144 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:32.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:32.204 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:32.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:32.265 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:32.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:32.325 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:32.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:32.386 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:32.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:32.445 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:32.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.446 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:32.506 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:32.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.507 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:32.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:32.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-05-16T07:14:32.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-05-16T07:14:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.809+0000 7f1eceba2700 1 Processor -- start 2023-05-16T07:14:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.809+0000 7f1eceba2700 1 -- start start 2023-05-16T07:14:32.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eceba2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec80c6410 0x7f1ec8155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eceba2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec806a140 con 0x7f1ec80c1160 2023-05-16T07:14:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eceba2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec806a420 con 0x7f1ec8153f90 2023-05-16T07:14:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eceba2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec8069f60 con 0x7f1ec80c6410 2023-05-16T07:14:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ecd39f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1ec80c1160 0x7f1ec80c5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54092/0 (socket says 172.21.15.92:54092) 2023-05-16T07:14:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ecd39f700 1 -- 172.21.15.92:0/2174258862 learned_addr learned my addr 172.21.15.92:0/2174258862 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ece3a1700 1 --2- 172.21.15.92:0/2174258862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec80c6410 0x7f1ec8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946032422 0 0) 0x7f1ec806a140 con 0x7f1ec80c1160 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb00033f0 con 0x7f1ec80c1160 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647012602 0 0) 0x7f1ec806a420 con 0x7f1ec8153f90 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec806a140 con 0x7f1ec8153f90 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ece3a1700 1 -- 172.21.15.92:0/2174258862 >> v1:172.21.15.158:6789/0 conn(0x7f1ec8153f90 legacy=0x7f1ec80a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ece3a1700 1 -- 172.21.15.92:0/2174258862 >> v1:172.21.15.92:6789/0 conn(0x7f1ec80c1160 legacy=0x7f1ec80c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ece3a1700 1 -- 172.21.15.92:0/2174258862 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec806a700 con 0x7f1ec80c6410 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1ece3a1700 1 --2- 172.21.15.92:0/2174258862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec80c6410 0x7f1ec8155380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f1ec0009790 tx=0x7f1ec0009060).ready entity=mon.2 client_cookie=4db6e9a79071973b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.810+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000f510 con 0x7f1ec80c6410 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec000f670 con 0x7f1ec80c6410 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eccb9e700 1 -- 172.21.15.92:0/2174258862 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000f9f0 con 0x7f1ec80c6410 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 -- 172.21.15.92:0/2174258862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec80c6410 msgr2=0x7f1ec8155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2174258862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec80c6410 0x7f1ec8155380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f1ec0009790 tx=0x7f1ec0009060).stop 2023-05-16T07:14:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 -- 172.21.15.92:0/2174258862 shutdown_connections 2023-05-16T07:14:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2174258862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec80c6410 0x7f1ec8155380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 -- 172.21.15.92:0/2174258862 >> 172.21.15.92:0/2174258862 conn(0x7f1ec80cb600 msgr2=0x7f1ec80c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 -- 172.21.15.92:0/2174258862 shutdown_connections 2023-05-16T07:14:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.811+0000 7f1eceba2700 1 -- 172.21.15.92:0/2174258862 wait complete. 2023-05-16T07:14:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.812+0000 7f1eceba2700 1 Processor -- start 2023-05-16T07:14:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.812+0000 7f1eceba2700 1 -- start start 2023-05-16T07:14:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1eceba2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ec80c1160 0x7f1ec80adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1eceba2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 0x7f1ec80ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1eceba2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec8153f90 0x7f1ec80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1eceba2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80ae950 con 0x7f1ec80c6410 2023-05-16T07:14:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1eceba2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c2610 con 0x7f1ec80c1160 2023-05-16T07:14:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1eceba2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c1ce0 con 0x7f1ec8153f90 2023-05-16T07:14:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 0x7f1ec80ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 0x7f1ec80ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43348/0 (socket says 172.21.15.92:43348) 2023-05-16T07:14:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 -- 172.21.15.92:0/2145324360 learned_addr learned my addr 172.21.15.92:0/2145324360 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ece3a1700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec8153f90 0x7f1ec80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 -- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec8153f90 msgr2=0x7f1ec80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec8153f90 0x7f1ec80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 -- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ec80c1160 msgr2=0x7f1ec80adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecdba0700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ec80c1160 0x7f1ec80adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ec80c1160 0x7f1ec80adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 -- 172.21.15.92:0/2145324360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00092a0 con 0x7f1ec80c6410 2023-05-16T07:14:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ece3a1700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec8153f90 0x7f1ec80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecdba0700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ec80c1160 0x7f1ec80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ecd39f700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 0x7f1ec80ae460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1ebc00c560 tx=0x7f1ebc007cc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.813+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0072e0 con 0x7f1ec80c6410 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.814+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc007440 con 0x7f1ec80c6410 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.814+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec80c1fc0 con 0x7f1ec80c6410 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.814+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0057a0 con 0x7f1ec80c6410 2023-05-16T07:14:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.814+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80aeb60 con 0x7f1ec80c6410 2023-05-16T07:14:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.815+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec80c2460 con 0x7f1ec80c6410 2023-05-16T07:14:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.815+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f1ebc01a060 con 0x7f1ec80c6410 2023-05-16T07:14:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.815+0000 7f1ec5ffb700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1eb405ee70 0x7f1eb4061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.815+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f1ebc0797d0 con 0x7f1ec80c6410 2023-05-16T07:14:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.815+0000 7f1ecdba0700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1eb405ee70 0x7f1eb4061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.816+0000 7f1ecdba0700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1eb405ee70 0x7f1eb4061330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1eb8001d70 tx=0x7f1eb8009040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:32.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:32.819+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1ebc04a9a0 con 0x7f1ec80c6410 2023-05-16T07:14:33.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.017+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ec8083120 con 0x7f1eb405ee70 2023-05-16T07:14:33.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.021+0000 7f1ec5ffb700 1 -- 172.21.15.92:0/2145324360 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f1ec8083120 con 0x7f1eb405ee70 2023-05-16T07:14:33.025 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-05-16T07:14:33.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.022+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1eb405ee70 msgr2=0x7f1eb4061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.022+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1eb405ee70 0x7f1eb4061330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1eb8001d70 tx=0x7f1eb8009040).stop 2023-05-16T07:14:33.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.022+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 msgr2=0x7f1ec80ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.022+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 0x7f1ec80ae460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1ebc00c560 tx=0x7f1ebc007cc0).stop 2023-05-16T07:14:33.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 shutdown_connections 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1eb405ee70 0x7f1eb4061330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1ec8153f90 0x7f1ec80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1ec80c1160 0x7f1ec80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 --2- 172.21.15.92:0/2145324360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ec80c6410 0x7f1ec80ae460 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 >> 172.21.15.92:0/2145324360 conn(0x7f1ec80cb600 msgr2=0x7f1ec8154830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 shutdown_connections 2023-05-16T07:14:33.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.023+0000 7f1eceba2700 1 -- 172.21.15.92:0/2145324360 wait complete. 2023-05-16T07:14:33.037 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-05-16T07:14:33.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.038 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-05-16T07:14:33.054 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:33.054 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:33.054 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:33.056 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:33.056 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:33.056 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:33.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.057 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:33.115 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:33.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.117 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:33.177 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:33.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.178 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:33.237 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:33.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.237 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:33.298 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:33.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.300 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:33.359 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:33.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.360 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:33.419 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:33.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.419 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:33.479 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:33.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.480 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:33.538 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:33.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.539 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:33.598 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:33.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.599 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:33.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:33.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.685 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:33.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:33.745 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-05-16T07:14:33.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-05-16T07:14:33.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.953+0000 7f34ac212700 1 Processor -- start 2023-05-16T07:14:33.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.953+0000 7f34ac212700 1 -- start start 2023-05-16T07:14:33.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34ac212700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 0x7f34a40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34ac212700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a40c94d0 con 0x7f34a4154470 2023-05-16T07:14:33.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34ac212700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a40cb7e0 con 0x7f34a4068160 2023-05-16T07:14:33.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34ac212700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40c88c0 con 0x7f34a4154090 2023-05-16T07:14:33.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34a97ad700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f34a4154470 0x7f34a40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54108/0 (socket says 172.21.15.92:54108) 2023-05-16T07:14:33.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34a97ad700 1 -- 172.21.15.92:0/58203247 learned_addr learned my addr 172.21.15.92:0/58203247 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:33.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34a9fae700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 0x7f34a40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541410834 0 0) 0x7f34a40c94d0 con 0x7f34a4154470 2023-05-16T07:14:33.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34900033f0 con 0x7f34a4154470 2023-05-16T07:14:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.954+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944775693 0 0) 0x7f34a40cb7e0 con 0x7f34a4068160 2023-05-16T07:14:33.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a40c94d0 con 0x7f34a4068160 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2972728976 0 0) 0x7f34900033f0 con 0x7f34a4154470 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 msgr2=0x7f34a40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 0x7f34a40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 >> v1:172.21.15.158:6789/0 conn(0x7f34a4068160 legacy=0x7f34a4068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40c8ba0 con 0x7f34a4154470 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f3494001470 con 0x7f34a4154470 2023-05-16T07:14:33.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 >> v1:172.21.15.92:6789/0 conn(0x7f34a4154470 legacy=0x7f34a40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3490003aa0 0x7f3490003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490004690 0x7f3490004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 0x7f3490005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40c88c0 con 0x7f3490003aa0 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349000b910 con 0x7f3490004690 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349000bad0 con 0x7f34a4154090 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3490003aa0 0x7f3490003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a97ad700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490004690 0x7f3490004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34aa7af700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 0x7f3490005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 -- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 msgr2=0x7f3490005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:33.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a4154090 0x7f3490005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 -- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490004690 msgr2=0x7f3490004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490004690 0x7f3490004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 -- 172.21.15.92:0/58203247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900033f0 con 0x7f3490003aa0 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a9fae700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3490003aa0 0x7f3490003ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f34a0009a80 tx=0x7f34a0009060).ready entity=mon.0 client_cookie=dc3283807fe83700 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0009430 con 0x7f3490003aa0 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a000e640 con 0x7f3490003aa0 2023-05-16T07:14:33.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.955+0000 7f34a8fac700 1 -- 172.21.15.92:0/58203247 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000e9c0 con 0x7f3490003aa0 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34ac212700 1 -- 172.21.15.92:0/58203247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3490003aa0 msgr2=0x7f3490003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34ac212700 1 --2- 172.21.15.92:0/58203247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3490003aa0 0x7f3490003ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f34a0009a80 tx=0x7f34a0009060).stop 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34aa7af700 1 -- 172.21.15.92:0/58203247 reap_dead start 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34ac212700 1 -- 172.21.15.92:0/58203247 shutdown_connections 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34ac212700 1 -- 172.21.15.92:0/58203247 >> 172.21.15.92:0/58203247 conn(0x7f34a4067920 msgr2=0x7f34a40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34ac212700 1 -- 172.21.15.92:0/58203247 shutdown_connections 2023-05-16T07:14:33.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.956+0000 7f34ac212700 1 -- 172.21.15.92:0/58203247 wait complete. 2023-05-16T07:14:33.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.957+0000 7f34ac212700 1 Processor -- start 2023-05-16T07:14:33.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.957+0000 7f34ac212700 1 -- start start 2023-05-16T07:14:33.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490003aa0 0x7f34a413b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 0x7f34a40c2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a40c3fa0 0x7f34a40c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40c2640 con 0x7f34a40c1cd0 2023-05-16T07:14:33.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a413b780 con 0x7f3490003aa0 2023-05-16T07:14:33.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40c4d10 con 0x7f34a40c3fa0 2023-05-16T07:14:33.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 0x7f34a40c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 0x7f34a40c2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43370/0 (socket says 172.21.15.92:43370) 2023-05-16T07:14:33.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 -- 172.21.15.92:0/1967105119 learned_addr learned my addr 172.21.15.92:0/1967105119 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:33.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34aa7af700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a40c3fa0 0x7f34a40c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a9fae700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490003aa0 0x7f34a413b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 -- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a40c3fa0 msgr2=0x7f34a40c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a40c3fa0 0x7f34a40c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 -- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490003aa0 msgr2=0x7f34a413b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490003aa0 0x7f34a413b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 -- 172.21.15.92:0/1967105119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40c4ff0 con 0x7f34a40c1cd0 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a9fae700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490003aa0 0x7f34a413b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:33.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34a97ad700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 0x7f34a40c2150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f3494000c00 tx=0x7f3494001e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400f660 con 0x7f34a40c1cd0 2023-05-16T07:14:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349400f7c0 con 0x7f34a40c1cd0 2023-05-16T07:14:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a40c52d0 con 0x7f34a40c1cd0 2023-05-16T07:14:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.958+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400fb40 con 0x7f34a40c1cd0 2023-05-16T07:14:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.959+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a40c4910 con 0x7f34a40c1cd0 2023-05-16T07:14:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.960+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a40aeb90 con 0x7f34a40c1cd0 2023-05-16T07:14:33.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.960+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f349401b030 con 0x7f34a40c1cd0 2023-05-16T07:14:33.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.960+0000 7f3499ffb700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f348805ee70 0x7f3488061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:33.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.960+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 7478+0+0 (secure 0 0 0) 0x7f3494079970 con 0x7f34a40c1cd0 2023-05-16T07:14:33.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.960+0000 7f34a9fae700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f348805ee70 0x7f3488061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:33.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.961+0000 7f34a9fae700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f348805ee70 0x7f3488061330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f34a4150280 tx=0x7f34a001a000).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:33.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:33.965+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f34a40aeb90 con 0x7f34a40c1cd0 2023-05-16T07:14:34.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.149+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f34a40c4b00 con 0x7f348805ee70 2023-05-16T07:14:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.153+0000 7f3499ffb700 1 -- 172.21.15.92:0/1967105119 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f34a40c4b00 con 0x7f348805ee70 2023-05-16T07:14:34.157 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-05-16T07:14:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f348805ee70 msgr2=0x7f3488061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f348805ee70 0x7f3488061330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f34a4150280 tx=0x7f34a001a000).stop 2023-05-16T07:14:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 msgr2=0x7f34a40c2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 0x7f34a40c2150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f3494000c00 tx=0x7f3494001e30).stop 2023-05-16T07:14:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 shutdown_connections 2023-05-16T07:14:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f348805ee70 0x7f3488061330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:34.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f34a40c3fa0 0x7f34a40c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:34.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3490003aa0 0x7f34a413b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:34.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 --2- 172.21.15.92:0/1967105119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a40c1cd0 0x7f34a40c2150 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:34.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 >> 172.21.15.92:0/1967105119 conn(0x7f34a4067920 msgr2=0x7f34a40aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:34.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 shutdown_connections 2023-05-16T07:14:34.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:34.155+0000 7f34ac212700 1 -- 172.21.15.92:0/1967105119 wait complete. 2023-05-16T07:14:34.170 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-05-16T07:14:34.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.170 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-05-16T07:14:34.186 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:34.186 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:34.187 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:34.188 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:34.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:34.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.189 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:34.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:34.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.249 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:34.309 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:34.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.311 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:34.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:34.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.371 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:34.431 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:34.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.432 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:34.492 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:34.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:34.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:34.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.554 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:34.647 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:34.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.647 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:34.669 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:34.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.670 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:34.748 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:34.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.749 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:34.809 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:34.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.810 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:34.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:34.869 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-05-16T07:14:34.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-05-16T07:14:35.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.077+0000 7f9ffa575700 1 Processor -- start 2023-05-16T07:14:35.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.077+0000 7f9ffa575700 1 -- start start 2023-05-16T07:14:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ffa575700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9ff4154470 0x7f9ff40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ffa575700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40c94d0 con 0x7f9ff4154090 2023-05-16T07:14:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ffa575700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40cb7e0 con 0x7f9ff4068160 2023-05-16T07:14:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ffa575700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40c88c0 con 0x7f9ff4154470 2023-05-16T07:14:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ff4154090 0x7f9ff40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54124/0 (socket says 172.21.15.92:54124) 2023-05-16T07:14:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff3fff700 1 -- 172.21.15.92:0/3528037064 learned_addr learned my addr 172.21.15.92:0/3528037064 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/3528037064 >> v2:172.21.15.158:3300/0 conn(0x7f9ff4154470 0x7f9ff40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628706488 0 0) 0x7f9ff40c94d0 con 0x7f9ff4154090 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fdc0033f0 con 0x7f9ff4154090 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908867359 0 0) 0x7f9fdc0033f0 con 0x7f9ff4154090 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 >> v2:172.21.15.158:3300/0 conn(0x7f9ff4154470 msgr2=0x7f9ff40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 --2- 172.21.15.92:0/3528037064 >> v2:172.21.15.158:3300/0 conn(0x7f9ff4154470 0x7f9ff40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 >> v1:172.21.15.158:6789/0 conn(0x7f9ff4068160 legacy=0x7f9ff40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40c8ba0 con 0x7f9ff4154090 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f9fe4001470 con 0x7f9ff4154090 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/3528037064 >> v2:172.21.15.158:3300/0 conn(0x7f9ff4154470 0x7f9ff40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 >> v1:172.21.15.92:6789/0 conn(0x7f9ff4154090 legacy=0x7f9ff40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc0038e0 0x7f9fdc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc004690 0x7f9fdc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fdc005000 0x7f9fdc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40c88c0 con 0x7f9fdc004690 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc00bf60 con 0x7f9fdc0038e0 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc00c0f0 con 0x7f9fdc005000 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc004690 0x7f9fdc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.078+0000 7f9ff3fff700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc0038e0 0x7f9fdc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff8b12700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fdc005000 0x7f9fdc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff37fe700 1 -- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fdc005000 msgr2=0x7f9fdc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fdc005000 0x7f9fdc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff37fe700 1 -- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc0038e0 msgr2=0x7f9fdc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc0038e0 0x7f9fdc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff37fe700 1 -- 172.21.15.92:0/3528037064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0033f0 con 0x7f9fdc004690 2023-05-16T07:14:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff8b12700 1 -- 172.21.15.92:0/3528037064 reap_dead start 2023-05-16T07:14:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc004690 0x7f9fdc004b10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f9fe8009c70 tx=0x7f9fe8009060).ready entity=mon.0 client_cookie=2e13a191dd5b1cc4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80095c0 con 0x7f9fdc004690 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe800e6f0 con 0x7f9fdc004690 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ff2ffd700 1 -- 172.21.15.92:0/3528037064 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800ea70 con 0x7f9fdc004690 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 -- 172.21.15.92:0/3528037064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc004690 msgr2=0x7f9fdc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc004690 0x7f9fdc004b10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f9fe8009c70 tx=0x7f9fe8009060).stop 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 -- 172.21.15.92:0/3528037064 shutdown_connections 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 --2- 172.21.15.92:0/3528037064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc004690 0x7f9fdc004b10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 -- 172.21.15.92:0/3528037064 >> 172.21.15.92:0/3528037064 conn(0x7f9ff4067920 msgr2=0x7f9ff40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 -- 172.21.15.92:0/3528037064 shutdown_connections 2023-05-16T07:14:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.079+0000 7f9ffa575700 1 -- 172.21.15.92:0/3528037064 wait complete. 2023-05-16T07:14:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.080+0000 7f9ffa575700 1 Processor -- start 2023-05-16T07:14:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 -- start start 2023-05-16T07:14:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc004690 0x7f9ff414cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4164dd0 0x7f9ff4165250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 0x7f9ff40c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4165fa0 con 0x7f9ff40c9960 2023-05-16T07:14:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4165de0 con 0x7f9fdc004690 2023-05-16T07:14:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ffa575700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40af610 con 0x7f9ff4164dd0 2023-05-16T07:14:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ff8b12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 0x7f9ff40c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ff8b12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 0x7f9ff40c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43392/0 (socket says 172.21.15.92:43392) 2023-05-16T07:14:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.081+0000 7f9ff8b12700 1 -- 172.21.15.92:0/2513633504 learned_addr learned my addr 172.21.15.92:0/2513633504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff37fe700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4164dd0 0x7f9ff4165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff3fff700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc004690 0x7f9ff414cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff8b12700 1 -- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4164dd0 msgr2=0x7f9ff4165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff8b12700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4164dd0 0x7f9ff4165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff8b12700 1 -- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc004690 msgr2=0x7f9ff414cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff8b12700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc004690 0x7f9ff414cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff8b12700 1 -- 172.21.15.92:0/2513633504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40ca2d0 con 0x7f9ff40c9960 2023-05-16T07:14:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ff8b12700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 0x7f9ff40c9de0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f9ff4069010 tx=0x7f9fec009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec003030 con 0x7f9ff40c9960 2023-05-16T07:14:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec003190 con 0x7f9ff40c9960 2023-05-16T07:14:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff4165820 con 0x7f9ff40c9960 2023-05-16T07:14:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec003510 con 0x7f9ff40c9960 2023-05-16T07:14:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.082+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff40af2a0 con 0x7f9ff40c9960 2023-05-16T07:14:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.083+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff40af100 con 0x7f9ff40c9960 2023-05-16T07:14:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.083+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f9fec01c060 con 0x7f9ff40c9960 2023-05-16T07:14:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.084+0000 7f9fdbfff700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ee70 0x7f9fe0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.084+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 7082+0+0 (secure 0 0 0) 0x7f9fec079610 con 0x7f9ff40c9960 2023-05-16T07:14:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.084+0000 7f9ff3fff700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ee70 0x7f9fe0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.084+0000 7f9ff3fff700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ee70 0x7f9fe0061330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9fe4001ce0 tx=0x7f9fe4001e50).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:35.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.088+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9ff40af100 con 0x7f9ff40c9960 2023-05-16T07:14:35.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.275+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ff40af100 con 0x7f9fe005ee70 2023-05-16T07:14:35.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.279+0000 7f9fdbfff700 1 -- 172.21.15.92:0/2513633504 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9ff40af100 con 0x7f9fe005ee70 2023-05-16T07:14:35.283 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-05-16T07:14:35.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ee70 msgr2=0x7f9fe0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ee70 0x7f9fe0061330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9fe4001ce0 tx=0x7f9fe4001e50).stop 2023-05-16T07:14:35.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 msgr2=0x7f9ff40c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:35.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 0x7f9ff40c9de0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f9ff4069010 tx=0x7f9fec009040).stop 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 shutdown_connections 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fe005ee70 0x7f9fe0061330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ff40c9960 0x7f9ff40c9de0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fdc004690 0x7f9ff414cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 --2- 172.21.15.92:0/2513633504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ff4164dd0 0x7f9ff4165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 >> 172.21.15.92:0/2513633504 conn(0x7f9ff4067920 msgr2=0x7f9ff40c0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 shutdown_connections 2023-05-16T07:14:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:35.281+0000 7f9ffa575700 1 -- 172.21.15.92:0/2513633504 wait complete. 2023-05-16T07:14:35.296 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-05-16T07:14:35.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-05-16T07:14:35.312 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:35.314 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:35.314 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:35.314 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:35.314 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:35.314 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:35.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.315 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:35.374 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:35.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.375 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:35.435 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:35.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.436 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:35.497 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:35.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.498 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:35.558 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:35.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.559 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:35.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:35.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:35.694 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:35.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.695 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:35.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:35.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.755 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:35.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:35.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:35.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:35.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.881 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:35.940 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:35.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:35.942 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:36.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.001 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-05-16T07:14:36.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-05-16T07:14:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.208+0000 7ff81caad700 1 Processor -- start 2023-05-16T07:14:36.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.208+0000 7ff81caad700 1 -- start start 2023-05-16T07:14:36.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff81caad700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154090 0x7ff8180aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:36.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff81caad700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff818068160 0x7ff818068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:36.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff81caad700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8180c94d0 con 0x7ff818154470 2023-05-16T07:14:36.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff81caad700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8180cb7e0 con 0x7ff818068160 2023-05-16T07:14:36.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff81caad700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818068a70 con 0x7ff818154090 2023-05-16T07:14:36.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff816d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff818068160 0x7ff818068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff81659c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154090 0x7ff8180aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff816d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff818068160 0x7ff818068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:47034/0 (socket says 172.21.15.92:47034) 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.209+0000 7ff816d9d700 1 -- 172.21.15.92:0/2943566752 learned_addr learned my addr 172.21.15.92:0/2943566752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff816d9d700 1 -- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154090 msgr2=0x7ff8180aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff816d9d700 1 --2- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154090 0x7ff8180aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff816d9d700 1 -- 172.21.15.92:0/2943566752 >> v1:172.21.15.158:6789/0 conn(0x7ff818154470 legacy=0x7ff8180a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff816d9d700 1 -- 172.21.15.92:0/2943566752 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8180c88c0 con 0x7ff818068160 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff81659c700 1 --2- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154090 0x7ff8180aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff816d9d700 1 --2- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818068160 0x7ff818068580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff800006fd0 tx=0x7ff8000090f0).ready entity=mon.1 client_cookie=52d3539194daf81d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff81559a700 1 -- 172.21.15.92:0/2943566752 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000b580 con 0x7ff818068160 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff81559a700 1 -- 172.21.15.92:0/2943566752 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000b6e0 con 0x7ff818068160 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.210+0000 7ff81559a700 1 -- 172.21.15.92:0/2943566752 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000ba60 con 0x7ff818068160 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 -- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818068160 msgr2=0x7ff818068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 --2- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818068160 0x7ff818068580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff800006fd0 tx=0x7ff8000090f0).stop 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 -- 172.21.15.92:0/2943566752 shutdown_connections 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 --2- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818068160 0x7ff818068580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 --2- 172.21.15.92:0/2943566752 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154090 0x7ff8180aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 -- 172.21.15.92:0/2943566752 >> 172.21.15.92:0/2943566752 conn(0x7ff818067920 msgr2=0x7ff8180cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 -- 172.21.15.92:0/2943566752 shutdown_connections 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.211+0000 7ff81caad700 1 -- 172.21.15.92:0/2943566752 wait complete. 2023-05-16T07:14:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.213+0000 7ff81caad700 1 Processor -- start 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.213+0000 7ff81caad700 1 -- start start 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81caad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff818068160 0x7ff81814f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81caad700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818154090 0x7ff818150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81caad700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154470 0x7ff8181575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81caad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818151900 con 0x7ff818068160 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81caad700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818154960 con 0x7ff818154090 2023-05-16T07:14:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81caad700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180af3a0 con 0x7ff818154470 2023-05-16T07:14:36.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff818068160 0x7ff81814f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:36.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818154090 0x7ff818150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:36.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff818068160 0x7ff81814f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43402/0 (socket says 172.21.15.92:43402) 2023-05-16T07:14:36.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff81659c700 1 -- 172.21.15.92:0/2056993654 learned_addr learned my addr 172.21.15.92:0/2056993654 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:36.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff816d9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154470 0x7ff8181575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:36.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 -- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154470 msgr2=0x7ff8181575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:36.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154470 0x7ff8181575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 -- 172.21.15.92:0/2056993654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff818068160 msgr2=0x7ff81814f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff818068160 0x7ff81814f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 -- 172.21.15.92:0/2056993654 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810009a20 con 0x7ff818154090 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.214+0000 7ff815d9b700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818154090 0x7ff818150f90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7ff818158ec0 tx=0x7ff808007d40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.215+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808007020 con 0x7ff818154090 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.215+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff808007180 con 0x7ff818154090 2023-05-16T07:14:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.215+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff800009370 con 0x7ff818154090 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.215+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808007500 con 0x7ff818154090 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.215+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff818151620 con 0x7ff818154090 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.216+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff818066730 con 0x7ff818154090 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.217+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7ff808010070 con 0x7ff818154090 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.217+0000 7ff80e7fc700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff7f805ee40 0x7ff7f8061300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.217+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 7066+0+0 (secure 0 0 0) 0x7ff808079690 con 0x7ff818154090 2023-05-16T07:14:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.217+0000 7ff81659c700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff7f805ee40 0x7ff7f8061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:36.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.218+0000 7ff81659c700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff7f805ee40 0x7ff7f8061300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7ff81000f800 tx=0x7ff81000b040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:36.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.224+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff80804a800 con 0x7ff818154090 2023-05-16T07:14:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.427+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff81806c350 con 0x7ff7f805ee40 2023-05-16T07:14:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.431+0000 7ff80e7fc700 1 -- 172.21.15.92:0/2056993654 <== mgr.4990 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ff81806c350 con 0x7ff7f805ee40 2023-05-16T07:14:36.436 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-05-16T07:14:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff7f805ee40 msgr2=0x7ff7f8061300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:36.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff7f805ee40 0x7ff7f8061300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7ff81000f800 tx=0x7ff81000b040).stop 2023-05-16T07:14:36.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818154090 msgr2=0x7ff818150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:36.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818154090 0x7ff818150f90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7ff818158ec0 tx=0x7ff808007d40).stop 2023-05-16T07:14:36.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 shutdown_connections 2023-05-16T07:14:36.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff7f805ee40 0x7ff7f8061300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff818154470 0x7ff8181575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff818068160 0x7ff81814f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 --2- 172.21.15.92:0/2056993654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff818154090 0x7ff818150f90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:36.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 >> 172.21.15.92:0/2056993654 conn(0x7ff818067920 msgr2=0x7ff8180a9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:36.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 shutdown_connections 2023-05-16T07:14:36.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:36.433+0000 7ff81caad700 1 -- 172.21.15.92:0/2056993654 wait complete. 2023-05-16T07:14:36.449 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-05-16T07:14:36.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.450 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-05-16T07:14:36.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:36.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:36.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:36.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:36.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:36.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:36.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:36.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:36.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:36.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:36.468 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:36.468 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_04_00_UTC 2023-05-16T07:14:36.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.469 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_04_00_UTC) 2023-05-16T07:14:36.528 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_05_00_UTC 2023-05-16T07:14:36.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.529 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_05_00_UTC) 2023-05-16T07:14:36.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_06_00_UTC 2023-05-16T07:14:36.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.590 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_06_00_UTC) 2023-05-16T07:14:36.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:14:36.632 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:14:36.658 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_07_00_UTC 2023-05-16T07:14:36.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.660 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_07_00_UTC) 2023-05-16T07:14:36.742 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_08_00_UTC 2023-05-16T07:14:36.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.743 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_08_00_UTC) 2023-05-16T07:14:36.802 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_09_00_UTC 2023-05-16T07:14:36.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.803 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_09_00_UTC) 2023-05-16T07:14:36.864 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_10_00_UTC 2023-05-16T07:14:36.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.864 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_10_00_UTC) 2023-05-16T07:14:36.928 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_11_00_UTC 2023-05-16T07:14:36.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_11_00_UTC) 2023-05-16T07:14:36.988 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_12_00_UTC 2023-05-16T07:14:36.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:36.989 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_12_00_UTC) 2023-05-16T07:14:37.047 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_13_00_UTC 2023-05-16T07:14:37.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:37.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_13_00_UTC) 2023-05-16T07:14:37.108 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_14_00_UTC 2023-05-16T07:14:37.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:37.109 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-05-16-07_14_00_UTC) 2023-05-16T07:14:37.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:37.170 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-05-16T07:14:37.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:14:37.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 Processor -- start 2023-05-16T07:14:37.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 -- start start 2023-05-16T07:14:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f62d014e210 0x7f62d014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62d0157be0 0x7f62d0158090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d0158580 con 0x7f62d0068ba0 2023-05-16T07:14:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d006a470 con 0x7f62d0157be0 2023-05-16T07:14:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.392+0000 7f62d6179700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d0137670 con 0x7f62d014e210 2023-05-16T07:14:37.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62d0157be0 0x7f62d0158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62d0157be0 0x7f62d0158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43410/0 (socket says 172.21.15.92:43410) 2023-05-16T07:14:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 -- 172.21.15.92:0/3300133830 learned_addr learned my addr 172.21.15.92:0/3300133830 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62cffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f62d0068ba0 0x7f62d0068fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54158/0 (socket says 172.21.15.92:54158) 2023-05-16T07:14:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62cf7fe700 1 --2- 172.21.15.92:0/3300133830 >> v2:172.21.15.158:3300/0 conn(0x7f62d014e210 0x7f62d014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 -- 172.21.15.92:0/3300133830 >> v2:172.21.15.158:3300/0 conn(0x7f62d014e210 msgr2=0x7f62d014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 --2- 172.21.15.92:0/3300133830 >> v2:172.21.15.158:3300/0 conn(0x7f62d014e210 0x7f62d014e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 -- 172.21.15.92:0/3300133830 >> v1:172.21.15.92:6789/0 conn(0x7f62d0068ba0 legacy=0x7f62d0068fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ce7fc700 1 -- 172.21.15.92:0/3300133830 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317778214 0 0) 0x7f62c8001490 con 0x7f62d0068ba0 2023-05-16T07:14:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 -- 172.21.15.92:0/3300133830 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00697f0 con 0x7f62d0157be0 2023-05-16T07:14:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ceffd700 1 --2- 172.21.15.92:0/3300133830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d0157be0 0x7f62d0158090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f62d00c4df0 tx=0x7f62c40090f0).ready entity=mon.0 client_cookie=d281fff65f92892c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ce7fc700 1 -- 172.21.15.92:0/3300133830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400b600 con 0x7f62d0157be0 2023-05-16T07:14:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ce7fc700 1 -- 172.21.15.92:0/3300133830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400b760 con 0x7f62d0157be0 2023-05-16T07:14:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.393+0000 7f62ce7fc700 1 -- 172.21.15.92:0/3300133830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400bb00 con 0x7f62d0157be0 2023-05-16T07:14:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 -- 172.21.15.92:0/3300133830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d0157be0 msgr2=0x7f62d0158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 --2- 172.21.15.92:0/3300133830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d0157be0 0x7f62d0158090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f62d00c4df0 tx=0x7f62c40090f0).stop 2023-05-16T07:14:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 -- 172.21.15.92:0/3300133830 shutdown_connections 2023-05-16T07:14:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 --2- 172.21.15.92:0/3300133830 >> v2:172.21.15.158:3300/0 conn(0x7f62d014e210 0x7f62d014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:37.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 --2- 172.21.15.92:0/3300133830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d0157be0 0x7f62d0158090 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:37.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 -- 172.21.15.92:0/3300133830 >> 172.21.15.92:0/3300133830 conn(0x7f62d00c4600 msgr2=0x7f62d0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:37.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 -- 172.21.15.92:0/3300133830 shutdown_connections 2023-05-16T07:14:37.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.394+0000 7f62d6179700 1 -- 172.21.15.92:0/3300133830 wait complete. 2023-05-16T07:14:37.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.395+0000 7f62d6179700 1 Processor -- start 2023-05-16T07:14:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.395+0000 7f62d6179700 1 -- start start 2023-05-16T07:14:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62d0068ba0 0x7f62d013a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 0x7f62d00bed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62d0157be0 0x7f62d00bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00bf6e0 con 0x7f62d014e210 2023-05-16T07:14:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0136260 con 0x7f62d0157be0 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0135650 con 0x7f62d0068ba0 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 0x7f62d00bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 0x7f62d00bed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43416/0 (socket says 172.21.15.92:43416) 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 -- 172.21.15.92:0/698800848 learned_addr learned my addr 172.21.15.92:0/698800848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62cffff700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62d0157be0 0x7f62d00bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62cf7fe700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62d0068ba0 0x7f62d013a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 -- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62d0068ba0 msgr2=0x7f62d013a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62d0068ba0 0x7f62d013a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 -- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62d0157be0 msgr2=0x7f62d00bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62d0157be0 0x7f62d00bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 -- 172.21.15.92:0/698800848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c4009370 con 0x7f62d014e210 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62cf7fe700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62d0068ba0 0x7f62d013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62ceffd700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 0x7f62d00bed00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f62c4006fd0 tx=0x7f62c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:37.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400b8d0 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400ba30 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d0135930 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c401c670 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.396+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d0135ef0 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.397+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d01363d0 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.398+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f62c402c070 con 0x7f62d014e210 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.398+0000 7f62b77fe700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62bc05ee70 0x7f62bc061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:37.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.398+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7082+0+0 (secure 0 0 0) 0x7f62c407bb90 con 0x7f62d014e210 2023-05-16T07:14:37.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.398+0000 7f62cf7fe700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62bc05ee70 0x7f62bc061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:37.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.399+0000 7f62cf7fe700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62bc05ee70 0x7f62bc061330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f62c000b870 tx=0x7f62c0008040).ready entity=mgr.4990 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:37.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.402+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f62c404cd50 con 0x7f62d014e210 2023-05-16T07:14:37.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:37.608+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f62d006c350 con 0x7f62d014e210 2023-05-16T07:14:38.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.619+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f62c4022030 con 0x7f62d014e210 2023-05-16T07:14:38.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.621+0000 7f62b77fe700 1 -- 172.21.15.92:0/698800848 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v11) v1 ==== 93+0+0 (secure 0 0 0) 0x7f62c4042a60 con 0x7f62d014e210 2023-05-16T07:14:38.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62bc05ee70 msgr2=0x7f62bc061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:38.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62bc05ee70 0x7f62bc061330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f62c000b870 tx=0x7f62c0008040).stop 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 msgr2=0x7f62d00bed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 0x7f62d00bed00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f62c4006fd0 tx=0x7f62c4017000).stop 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 shutdown_connections 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f62bc05ee70 0x7f62bc061330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62d0068ba0 0x7f62d013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f62d0157be0 0x7f62d00bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 --2- 172.21.15.92:0/698800848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d014e210 0x7f62d00bed00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 >> 172.21.15.92:0/698800848 conn(0x7f62d00c4600 msgr2=0x7f62d0068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:38.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 shutdown_connections 2023-05-16T07:14:38.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.624+0000 7f62d6179700 1 -- 172.21.15.92:0/698800848 wait complete. 2023-05-16T07:14:38.644 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:14:38.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:14:38.673 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:14:38.700 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:14:38.701 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:14:38.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:14:38.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:14:38.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:14:38.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:38.747 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:14:38.768 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:38.763+0000 7fcf5515d680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:14:38.775 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:38.772+0000 7fb5359cf680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:14:38.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:38.852 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:38.843+0000 7fcf5515d680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:14:38.868 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:38.856+0000 7fb5359cf680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:14:38.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.962+0000 7fa1dba7d700 1 Processor -- start 2023-05-16T07:14:38.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.962+0000 7fa1dba7d700 1 -- start start 2023-05-16T07:14:38.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.963+0000 7fa1dba7d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa1d4063e30 0x7fa1d4069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:38.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.963+0000 7fa1dba7d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:38.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.963+0000 7fa1dba7d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d4062960 con 0x7fa1d4154040 2023-05-16T07:14:38.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.963+0000 7fa1dba7d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d4154420 con 0x7fa1d4063e30 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.963+0000 7fa1dba7d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4069fb0 con 0x7fa1d40bf4f0 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.963+0000 7fa1d9018700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa1d4063e30 0x7fa1d4069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa1d4063e30 0x7fa1d4069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43470/0 (socket says 172.21.15.92:43470) 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 -- 172.21.15.92:0/150445255 learned_addr learned my addr 172.21.15.92:0/150445255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1da01a700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 -- 172.21.15.92:0/150445255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 msgr2=0x7fa1d4062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 -- 172.21.15.92:0/150445255 >> v1:172.21.15.158:6789/0 conn(0x7fa1d4154040 legacy=0x7fa1d4063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 -- 172.21.15.92:0/150445255 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4062350 con 0x7fa1d4063e30 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1da01a700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d9018700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d4069ac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fa1c8006fd0 tx=0x7fa1c80090f0).ready entity=mon.0 client_cookie=d4e9798d04b3b4a4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d8817700 1 -- 172.21.15.92:0/150445255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800b600 con 0x7fa1d4063e30 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d8817700 1 -- 172.21.15.92:0/150445255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c800b760 con 0x7fa1d4063e30 2023-05-16T07:14:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.964+0000 7fa1d8817700 1 -- 172.21.15.92:0/150445255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800bae0 con 0x7fa1d4063e30 2023-05-16T07:14:38.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 -- 172.21.15.92:0/150445255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 msgr2=0x7fa1d4069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:38.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d4069ac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fa1c8006fd0 tx=0x7fa1c80090f0).stop 2023-05-16T07:14:38.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 -- 172.21.15.92:0/150445255 shutdown_connections 2023-05-16T07:14:38.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/150445255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d4069ac0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 -- 172.21.15.92:0/150445255 >> 172.21.15.92:0/150445255 conn(0x7fa1d40c8aa0 msgr2=0x7fa1d40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 -- 172.21.15.92:0/150445255 shutdown_connections 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.965+0000 7fa1dba7d700 1 -- 172.21.15.92:0/150445255 wait complete. 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.967+0000 7fa1dba7d700 1 Processor -- start 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.967+0000 7fa1dba7d700 1 -- start start 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1dba7d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d40c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1dba7d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1dba7d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1d4154040 0x7fa1d40a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:38.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1dba7d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40aca20 con 0x7fa1d4063e30 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1dba7d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40acd00 con 0x7fa1d4154040 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1dba7d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40acfe0 con 0x7fa1d40bf4f0 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d40c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d40c0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43474/0 (socket says 172.21.15.92:43474) 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 -- 172.21.15.92:0/3481894806 learned_addr learned my addr 172.21.15.92:0/3481894806 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1da01a700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1d4154040 0x7fa1d40a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:38.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9018700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 -- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 msgr2=0x7fa1d4167a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4167a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 -- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1d4154040 msgr2=0x7fa1d40a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1d4154040 0x7fa1d40a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.968+0000 7fa1d9819700 1 -- 172.21.15.92:0/3481894806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c8009370 con 0x7fa1d4063e30 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.969+0000 7fa1d9819700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d40c0a50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fa1d000c4c0 tx=0x7fa1d0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:38.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.969+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000ce70 con 0x7fa1d4063e30 2023-05-16T07:14:38.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.969+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0001e90 con 0x7fa1d4063e30 2023-05-16T07:14:38.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.969+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0007250 con 0x7fa1d4063e30 2023-05-16T07:14:38.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.969+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d40ad2c0 con 0x7fa1d4063e30 2023-05-16T07:14:38.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.969+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d40ad880 con 0x7fa1d4063e30 2023-05-16T07:14:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.970+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d406c350 con 0x7fa1d4063e30 2023-05-16T07:14:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.970+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fa1d0010070 con 0x7fa1d4063e30 2023-05-16T07:14:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.971+0000 7fa1c5ffb700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 0x7fa1b80612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.971+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7638+0+0 (secure 0 0 0) 0x7fa1d00798b0 con 0x7fa1d4063e30 2023-05-16T07:14:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.971+0000 7fa1d9018700 1 -- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 msgr2=0x7fa1b80612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.971+0000 7fa1d9018700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 0x7fa1b80612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:38.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:38.975+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa1d004a840 con 0x7fa1d4063e30 2023-05-16T07:14:39.138 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:39.135+0000 7fb5359cf680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:14:39.147 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:39.142+0000 7fcf5515d680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:14:39.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.172+0000 7fa1d9018700 1 -- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 msgr2=0x7fa1b80612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:39.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.172+0000 7fa1d9018700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 0x7fa1b80612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:14:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.186+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1d40c6b20 con 0x7fa1d4063e30 2023-05-16T07:14:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.186+0000 7fa1c5ffb700 1 -- 172.21.15.92:0/3481894806 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4489 (secure 0 0 0) 0x7fa1d004de60 con 0x7fa1d4063e30 2023-05-16T07:14:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:39.191 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:03:09.943261+0000","modified":"2023-05-16T07:03:11.948935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2108552286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2108552286},{"type":"v1","addr":"172.21.15.92:6835","nonce":2108552286}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-05-16T07:14:39.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 msgr2=0x7fa1b80612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:39.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 0x7fa1b80612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:39.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 msgr2=0x7fa1d40c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d40c0a50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fa1d000c4c0 tx=0x7fa1d0007e00).stop 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 shutdown_connections 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa1b805ee20 0x7fa1b80612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1d4154040 0x7fa1d40a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d4063e30 0x7fa1d40c0a50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 --2- 172.21.15.92:0/3481894806 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1d40bf4f0 0x7fa1d4167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 >> 172.21.15.92:0/3481894806 conn(0x7fa1d40c8aa0 msgr2=0x7fa1d4156580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 shutdown_connections 2023-05-16T07:14:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.188+0000 7fa1dba7d700 1 -- 172.21.15.92:0/3481894806 wait complete. 2023-05-16T07:14:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2023-05-16T07:14:39.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:39.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.354+0000 7f65defc0700 1 Processor -- start 2023-05-16T07:14:39.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.354+0000 7f65defc0700 1 -- start start 2023-05-16T07:14:39.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65defc0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f65d81386a0 0x7f65d80c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65defc0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 0x7f65d80622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65defc0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d80c37f0 con 0x7f65d80627a0 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65defc0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d8138a80 con 0x7f65d81386a0 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65defc0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8062c30 con 0x7f65d80c30a0 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 0x7f65d80622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65dcd5c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f65d81386a0 0x7f65d80c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 0x7f65d80622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:36090/0 (socket says 172.21.15.92:36090) 2023-05-16T07:14:39.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 -- 172.21.15.92:0/441626209 learned_addr learned my addr 172.21.15.92:0/441626209 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:39.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 -- 172.21.15.92:0/441626209 >> v2:172.21.15.158:3300/0 conn(0x7f65d81386a0 msgr2=0x7f65d80c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:39.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 --2- 172.21.15.92:0/441626209 >> v2:172.21.15.158:3300/0 conn(0x7f65d81386a0 0x7f65d80c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 -- 172.21.15.92:0/441626209 >> v1:172.21.15.158:6789/0 conn(0x7f65d80627a0 legacy=0x7f65d8063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.356+0000 7f65d7fff700 1 -- 172.21.15.92:0/441626209 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80bef90 con 0x7f65d80c30a0 2023-05-16T07:14:39.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.357+0000 7f65d7fff700 1 --2- 172.21.15.92:0/441626209 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 0x7f65d80622b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f65c4006fd0 tx=0x7f65c40090f0).ready entity=mon.2 client_cookie=4139552f312af616 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:39.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.357+0000 7f65d77fe700 1 -- 172.21.15.92:0/441626209 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400b580 con 0x7f65d80c30a0 2023-05-16T07:14:39.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.357+0000 7f65d77fe700 1 -- 172.21.15.92:0/441626209 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c400b6e0 con 0x7f65d80c30a0 2023-05-16T07:14:39.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.357+0000 7f65d77fe700 1 -- 172.21.15.92:0/441626209 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400ba60 con 0x7f65d80c30a0 2023-05-16T07:14:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.357+0000 7f65defc0700 1 -- 172.21.15.92:0/441626209 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 msgr2=0x7f65d80622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.357+0000 7f65defc0700 1 --2- 172.21.15.92:0/441626209 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 0x7f65d80622b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f65c4006fd0 tx=0x7f65c40090f0).stop 2023-05-16T07:14:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.358+0000 7f65defc0700 1 -- 172.21.15.92:0/441626209 shutdown_connections 2023-05-16T07:14:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.358+0000 7f65defc0700 1 --2- 172.21.15.92:0/441626209 >> v2:172.21.15.158:3300/0 conn(0x7f65d81386a0 0x7f65d80c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.358+0000 7f65defc0700 1 --2- 172.21.15.92:0/441626209 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d80c30a0 0x7f65d80622b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.358+0000 7f65defc0700 1 -- 172.21.15.92:0/441626209 >> 172.21.15.92:0/441626209 conn(0x7f65d80c8aa0 msgr2=0x7f65d80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.358+0000 7f65defc0700 1 -- 172.21.15.92:0/441626209 shutdown_connections 2023-05-16T07:14:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.358+0000 7f65defc0700 1 -- 172.21.15.92:0/441626209 wait complete. 2023-05-16T07:14:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.359+0000 7f65defc0700 1 Processor -- start 2023-05-16T07:14:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.360+0000 7f65defc0700 1 -- start start 2023-05-16T07:14:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65defc0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d80627a0 0x7f65d80a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65defc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 0x7f65d8150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65defc0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d81386a0 0x7f65d80ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65defc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8150600 con 0x7f65d80c30a0 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65defc0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8165080 con 0x7f65d80627a0 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65defc0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8165360 con 0x7f65d81386a0 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 0x7f65d8150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 0x7f65d8150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43522/0 (socket says 172.21.15.92:43522) 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 -- 172.21.15.92:0/350446728 learned_addr learned my addr 172.21.15.92:0/350446728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65dcd5c700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d80627a0 0x7f65d80a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65dd55d700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d81386a0 0x7f65d80ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 -- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d81386a0 msgr2=0x7f65d80ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d81386a0 0x7f65d80ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 -- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d80627a0 msgr2=0x7f65d80a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d80627a0 0x7f65d80a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.361+0000 7f65d7fff700 1 -- 172.21.15.92:0/350446728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4009370 con 0x7f65d80c30a0 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.362+0000 7f65d7fff700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 0x7f65d8150110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f65c4009130 tx=0x7f65c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.362+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4000bc0 con 0x7f65d80c30a0 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.362+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c400b850 con 0x7f65d80c30a0 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.362+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d8165640 con 0x7f65d80c30a0 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.362+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4000d20 con 0x7f65d80c30a0 2023-05-16T07:14:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.362+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d8165c00 con 0x7f65d80c30a0 2023-05-16T07:14:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.363+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d806c350 con 0x7f65d80c30a0 2023-05-16T07:14:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.363+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f65c402c070 con 0x7f65d80c30a0 2023-05-16T07:14:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.363+0000 7f65d4ff9700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f65bc05ee20 0x7f65bc0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.364+0000 7f65dcd5c700 1 -- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f65bc05ee20 msgr2=0x7f65bc0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.364+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7638+0+0 (secure 0 0 0) 0x7f65c407bd10 con 0x7f65d80c30a0 2023-05-16T07:14:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.364+0000 7f65dcd5c700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f65bc05ee20 0x7f65bc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:39.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.368+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f65c404cb90 con 0x7f65d80c30a0 2023-05-16T07:14:39.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.534+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65d81521c0 con 0x7f65d80c30a0 2023-05-16T07:14:39.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.535+0000 7f65d4ff9700 1 -- 172.21.15.92:0/350446728 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+19282 (secure 0 0 0) 0x7f65c402a020 con 0x7f65d80c30a0 2023-05-16T07:14:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:39.540 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:38.601105+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:03:07.846932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-05-16T07:03:08.910782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/1428741227":"2023-05-17T07:03:24.745231+0000","172.21.15.158:0/2276917568":"2023-05-17T07:03:24.745231+0000","172.21.15.158:6833/86301":"2023-05-17T07:03:24.745231+0000","172.21.15.158:6832/86301":"2023-05-17T07:03:24.745231+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":1},{"begin":186,"length":35}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":19,"length":1},{"begin":29,"length":1},{"begin":39,"length":1},{"begin":49,"length":1},{"begin":59,"length":1},{"begin":69,"length":1},{"begin":79,"length":1},{"begin":89,"length":1},{"begin":99,"length":1},{"begin":212,"length":9}]}],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":3,"length":1},{"begin":6,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":13,"length":1},{"begin":16,"length":1},{"begin":18,"length":1},{"begin":20,"length":1},{"begin":23,"length":1},{"begin":26,"length":3},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":36,"length":3},{"begin":40,"length":1},{"begin":43,"length":1},{"begin":46,"length":3},{"begin":50,"length":1},{"begin":53,"length":1},{"begin":56,"length":3},{"begin":60,"length":1},{"begin":63,"length":1},{"begin":66,"length":3},{"begin":70,"length":1},{"begin":73,"length":1},{"begin":76,"length":3},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":86,"length":2},{"begin":90,"length":1},{"begin":93,"length":1},{"begin":96,"length":2},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":106,"length":2},{"begin":110,"length":1},{"begin":136,"length":50}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:39.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f65bc05ee20 msgr2=0x7f65bc0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:39.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f65bc05ee20 0x7f65bc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:39.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 msgr2=0x7f65d8150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 0x7f65d8150110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f65c4009130 tx=0x7f65c4017000).stop 2023-05-16T07:14:39.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 shutdown_connections 2023-05-16T07:14:39.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f65bc05ee20 0x7f65bc0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:39.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d81386a0 0x7f65d80ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d80627a0 0x7f65d80a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 --2- 172.21.15.92:0/350446728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d80c30a0 0x7f65d8150110 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 >> 172.21.15.92:0/350446728 conn(0x7f65d80c8aa0 msgr2=0x7f65d8136470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:39.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 shutdown_connections 2023-05-16T07:14:39.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.537+0000 7f65defc0700 1 -- 172.21.15.92:0/350446728 wait complete. 2023-05-16T07:14:39.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:03:07.846932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:03:08.910782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:14:39.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:14:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.671+0000 7f22a4e86700 1 Processor -- start 2023-05-16T07:14:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.671+0000 7f22a4e86700 1 -- start start 2023-05-16T07:14:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f22a4e86700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f22a0154310 0x7f22a006a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f22a4e86700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a01546f0 0x7f22a0063d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f22a4e86700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a00c3130 con 0x7f22a014ff40 2023-05-16T07:14:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f22a4e86700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a0154b50 con 0x7f22a0154310 2023-05-16T07:14:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f22a4e86700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a0150a80 con 0x7f22a01546f0 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f229ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f22a014ff40 0x7f22a0150370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54280/0 (socket says 172.21.15.92:54280) 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f229ed9d700 1 -- 172.21.15.92:0/2238713693 learned_addr learned my addr 172.21.15.92:0/2238713693 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f229dd9b700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a01546f0 0x7f22a0063d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.672+0000 7f229e59c700 1 --2- 172.21.15.92:0/2238713693 >> v2:172.21.15.158:3300/0 conn(0x7f22a0154310 0x7f22a006a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229d59a700 1 -- 172.21.15.92:0/2238713693 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806929236 0 0) 0x7f22a00c3130 con 0x7f22a014ff40 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229d59a700 1 -- 172.21.15.92:0/2238713693 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f228c0033f0 con 0x7f22a014ff40 2023-05-16T07:14:39.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229e59c700 1 -- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a01546f0 msgr2=0x7f22a0063d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229e59c700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a01546f0 0x7f22a0063d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229e59c700 1 -- 172.21.15.92:0/2238713693 >> v1:172.21.15.92:6789/0 conn(0x7f22a014ff40 legacy=0x7f22a0150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229d59a700 1 -- 172.21.15.92:0/2238713693 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 953877246 0 0) 0x7f228c0033f0 con 0x7f22a014ff40 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229e59c700 1 -- 172.21.15.92:0/2238713693 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0068540 con 0x7f22a0154310 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229dd9b700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a01546f0 0x7f22a0063d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229e59c700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a006a0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f2288009760 tx=0x7f2288009060).ready entity=mon.1 client_cookie=3e40dbbd2f33057 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229d59a700 1 -- 172.21.15.92:0/2238713693 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980017f0 con 0x7f22a0154310 2023-05-16T07:14:39.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229d59a700 1 -- 172.21.15.92:0/2238713693 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288011400 con 0x7f22a0154310 2023-05-16T07:14:39.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.673+0000 7f229d59a700 1 -- 172.21.15.92:0/2238713693 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288011780 con 0x7f22a0154310 2023-05-16T07:14:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 -- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 msgr2=0x7f22a006a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a006a0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f2288009760 tx=0x7f2288009060).stop 2023-05-16T07:14:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 -- 172.21.15.92:0/2238713693 shutdown_connections 2023-05-16T07:14:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a006a0f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 --2- 172.21.15.92:0/2238713693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a01546f0 0x7f22a0063d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 -- 172.21.15.92:0/2238713693 >> 172.21.15.92:0/2238713693 conn(0x7f22a00c3d90 msgr2=0x7f22a00c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 -- 172.21.15.92:0/2238713693 shutdown_connections 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.674+0000 7f22a4e86700 1 -- 172.21.15.92:0/2238713693 wait complete. 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.675+0000 7f22a4e86700 1 Processor -- start 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.675+0000 7f22a4e86700 1 -- start start 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a014ff40 0x7f22a00c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a00aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 0x7f22a014a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00c62d0 con 0x7f22a01546f0 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00c6b70 con 0x7f22a0154310 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00ab620 con 0x7f22a014ff40 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 0x7f22a014a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 0x7f22a014a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43538/0 (socket says 172.21.15.92:43538) 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 -- 172.21.15.92:0/26758608 learned_addr learned my addr 172.21.15.92:0/26758608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229e59c700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a014ff40 0x7f22a00c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229dd9b700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a00aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 -- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a014ff40 msgr2=0x7f22a00c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a014ff40 0x7f22a00c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 -- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 msgr2=0x7f22a00aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a00aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 -- 172.21.15.92:0/26758608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900092a0 con 0x7f22a01546f0 2023-05-16T07:14:39.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f229ed9d700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 0x7f22a014a990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f229800cb40 tx=0x7f2298001e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298004c40 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2298004da0 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980073a0 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.676+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22880092a0 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.677+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a00c6800 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.677+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a00c6ce0 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.678+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f2298010070 con 0x7f22a01546f0 2023-05-16T07:14:39.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.678+0000 7f22967fc700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 0x7f22800612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:39.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.678+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f2298079b30 con 0x7f22a01546f0 2023-05-16T07:14:39.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.678+0000 7f229e59c700 1 -- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 msgr2=0x7f22800612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:39.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.678+0000 7f229e59c700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 0x7f22800612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:39.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.682+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f229804ad90 con 0x7f22a01546f0 2023-05-16T07:14:39.743 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:39.739+0000 7fb5359cf680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:14:39.783 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:39.774+0000 7fcf5515d680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:14:39.817 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:39.813+0000 7fb5359cf680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:14:39.851 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:39.846+0000 7fcf5515d680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:14:39.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.879+0000 7f229e59c700 1 -- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 msgr2=0x7f22800612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:39.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.879+0000 7f229e59c700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 0x7f22800612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:14:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.886+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f22a006c350 con 0x7f22a01546f0 2023-05-16T07:14:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.887+0000 7f22967fc700 1 -- 172.21.15.92:0/26758608 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v49) v1 ==== 110+0+10 (secure 0 0 0) 0x7f229804e3b0 con 0x7f22a01546f0 2023-05-16T07:14:39.893 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:14:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.888+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 msgr2=0x7f22800612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.888+0000 7f22a4e86700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 0x7f22800612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.888+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 msgr2=0x7f22a014a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.888+0000 7f22a4e86700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 0x7f22a014a990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f229800cb40 tx=0x7f2298001e20).stop 2023-05-16T07:14:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 shutdown_connections 2023-05-16T07:14:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f228005ee20 0x7f22800612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f22a014ff40 0x7f22a00c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a01546f0 0x7f22a014a990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 --2- 172.21.15.92:0/26758608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f22a0154310 0x7f22a00aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 >> 172.21.15.92:0/26758608 conn(0x7f22a00c3d90 msgr2=0x7f22a00bdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 shutdown_connections 2023-05-16T07:14:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:39.889+0000 7f22a4e86700 1 -- 172.21.15.92:0/26758608 wait complete. 2023-05-16T07:14:39.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:14:40.002 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:39.998+0000 7fb5359cf680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:14:40.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.020+0000 7f4d15b73700 1 Processor -- start 2023-05-16T07:14:40.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.020+0000 7f4d15b73700 1 -- start start 2023-05-16T07:14:40.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d15b73700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d100c5d30 0x7f4d100aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d15b73700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d15b73700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d10154620 con 0x7f4d10153aa0 2023-05-16T07:14:40.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d15b73700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100ca600 con 0x7f4d100c5d30 2023-05-16T07:14:40.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d15b73700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100c8830 con 0x7f4d100c6110 2023-05-16T07:14:40.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d100c5d30 0x7f4d100aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d100c5d30 0x7f4d100aa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43544/0 (socket says 172.21.15.92:43544) 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/3310986041 learned_addr learned my addr 172.21.15.92:0/3310986041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0effd700 1 --2- 172.21.15.92:0/3310986041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/3310986041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 msgr2=0x7f4d10069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 --2- 172.21.15.92:0/3310986041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/3310986041 >> v1:172.21.15.158:6789/0 conn(0x7f4d10153aa0 legacy=0x7f4d10153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/3310986041 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100c8b10 con 0x7f4d100c5d30 2023-05-16T07:14:40.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.021+0000 7f4d0f7fe700 1 --2- 172.21.15.92:0/3310986041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d100aa210 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f4d00009f20 tx=0x7f4d00009840).ready entity=mon.0 client_cookie=ac3d80f542c763af server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d0e7fc700 1 -- 172.21.15.92:0/3310986041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000e070 con 0x7f4d100c5d30 2023-05-16T07:14:40.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d0e7fc700 1 -- 172.21.15.92:0/3310986041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000b9d0 con 0x7f4d100c5d30 2023-05-16T07:14:40.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d0e7fc700 1 -- 172.21.15.92:0/3310986041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000bd50 con 0x7f4d100c5d30 2023-05-16T07:14:40.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 -- 172.21.15.92:0/3310986041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 msgr2=0x7f4d100aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 --2- 172.21.15.92:0/3310986041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d100aa210 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f4d00009f20 tx=0x7f4d00009840).stop 2023-05-16T07:14:40.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 -- 172.21.15.92:0/3310986041 shutdown_connections 2023-05-16T07:14:40.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 --2- 172.21.15.92:0/3310986041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d100aa210 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 --2- 172.21.15.92:0/3310986041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 -- 172.21.15.92:0/3310986041 >> 172.21.15.92:0/3310986041 conn(0x7f4d100cb700 msgr2=0x7f4d100c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 -- 172.21.15.92:0/3310986041 shutdown_connections 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.022+0000 7f4d15b73700 1 -- 172.21.15.92:0/3310986041 wait complete. 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 Processor -- start 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 -- start start 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d10144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d10153aa0 0x7f4d10145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10144640 con 0x7f4d100c5d30 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d101461e0 con 0x7f4d10153aa0 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d15b73700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10167640 con 0x7f4d100c6110 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d10144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d10144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43550/0 (socket says 172.21.15.92:43550) 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.024+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/4246051555 learned_addr learned my addr 172.21.15.92:0/4246051555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0effd700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0ffff700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d10153aa0 0x7f4d10145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 msgr2=0x7f4d10145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0f7fe700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d10153aa0 msgr2=0x7f4d10145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0f7fe700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d10153aa0 0x7f4d10145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0f7fe700 1 -- 172.21.15.92:0/4246051555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009ae0 con 0x7f4d100c5d30 2023-05-16T07:14:40.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d0f7fe700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d10144ad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f4d0000b610 tx=0x7f4d100c7060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000092e0 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000bb40 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0001d460 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10143a30 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.025+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10143ff0 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.026+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d10144920 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.026+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f4d00024030 con 0x7f4d100c5d30 2023-05-16T07:14:40.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.027+0000 7f4d077fe700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4cf005ee20 0x7f4cf00612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.027+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f4d0007b060 con 0x7f4d100c5d30 2023-05-16T07:14:40.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.027+0000 7f4d0effd700 1 -- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4cf005ee20 msgr2=0x7f4cf00612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:40.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.027+0000 7f4d0effd700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4cf005ee20 0x7f4cf00612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:40.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.031+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4d0004c370 con 0x7f4d100c5d30 2023-05-16T07:14:40.044 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:40.039+0000 7fcf5515d680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:14:40.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.198+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d1006c350 con 0x7f4d100c5d30 2023-05-16T07:14:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.198+0000 7f4d077fe700 1 -- 172.21.15.92:0/4246051555 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d00022030 con 0x7f4d100c5d30 2023-05-16T07:14:40.204 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:14:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4cf005ee20 msgr2=0x7f4cf00612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4cf005ee20 0x7f4cf00612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 msgr2=0x7f4d10144ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d10144ad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f4d0000b610 tx=0x7f4d100c7060).stop 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 shutdown_connections 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4cf005ee20 0x7f4cf00612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d10153aa0 0x7f4d10145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d100c5d30 0x7f4d10144ad0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 --2- 172.21.15.92:0/4246051555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d100c6110 0x7f4d10145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 >> 172.21.15.92:0/4246051555 conn(0x7f4d100cb700 msgr2=0x7f4d100afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 shutdown_connections 2023-05-16T07:14:40.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.200+0000 7f4d15b73700 1 -- 172.21.15.92:0/4246051555 wait complete. 2023-05-16T07:14:40.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:14:40.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.333+0000 7f3dc76c6700 1 Processor -- start 2023-05-16T07:14:40.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.333+0000 7f3dc76c6700 1 -- start start 2023-05-16T07:14:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc76c6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3dc0154090 0x7f3dc00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc76c6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3dc0154470 0x7f3dc00a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc76c6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc00aa150 0x7f3dc0068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc76c6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc00aa5a0 con 0x7f3dc0154470 2023-05-16T07:14:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc76c6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc00c94d0 con 0x7f3dc0154090 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc76c6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00cb7e0 con 0x7f3dc00aa150 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc4c61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3dc0154470 0x7f3dc00a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc4c61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3dc0154470 0x7f3dc00a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43602/0 (socket says 172.21.15.92:43602) 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc4c61700 1 -- 172.21.15.92:0/3837949494 learned_addr learned my addr 172.21.15.92:0/3837949494 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc5c63700 1 --2- 172.21.15.92:0/3837949494 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc00aa150 0x7f3dc0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.334+0000 7f3dc5462700 1 --2- 172.21.15.92:0/3837949494 >> v2:172.21.15.158:3300/0 conn(0x7f3dc0154090 0x7f3dc00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc4c61700 1 -- 172.21.15.92:0/3837949494 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc00aa150 msgr2=0x7f3dc0068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc4c61700 1 --2- 172.21.15.92:0/3837949494 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc00aa150 0x7f3dc0068160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc4c61700 1 -- 172.21.15.92:0/3837949494 >> v2:172.21.15.158:3300/0 conn(0x7f3dc0154090 msgr2=0x7f3dc00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc4c61700 1 --2- 172.21.15.92:0/3837949494 >> v2:172.21.15.158:3300/0 conn(0x7f3dc0154090 0x7f3dc00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc4c61700 1 -- 172.21.15.92:0/3837949494 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc00c88c0 con 0x7f3dc0154470 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc4c61700 1 --2- 172.21.15.92:0/3837949494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154470 0x7f3dc00a9c60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f3dc006bd10 tx=0x7f3dbc0090f0).ready entity=mon.0 client_cookie=7da75f96784d6e23 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3db7fff700 1 -- 172.21.15.92:0/3837949494 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00f510 con 0x7f3dc0154470 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3db7fff700 1 -- 172.21.15.92:0/3837949494 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc00f670 con 0x7f3dc0154470 2023-05-16T07:14:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3db7fff700 1 -- 172.21.15.92:0/3837949494 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00fa10 con 0x7f3dc0154470 2023-05-16T07:14:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 -- 172.21.15.92:0/3837949494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154470 msgr2=0x7f3dc00a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/3837949494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154470 0x7f3dc00a9c60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f3dc006bd10 tx=0x7f3dbc0090f0).stop 2023-05-16T07:14:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 -- 172.21.15.92:0/3837949494 shutdown_connections 2023-05-16T07:14:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/3837949494 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc00aa150 0x7f3dc0068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/3837949494 >> v2:172.21.15.158:3300/0 conn(0x7f3dc0154090 0x7f3dc00aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/3837949494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154470 0x7f3dc00a9c60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.335+0000 7f3dc76c6700 1 -- 172.21.15.92:0/3837949494 >> 172.21.15.92:0/3837949494 conn(0x7f3dc0067920 msgr2=0x7f3dc00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.336+0000 7f3dc76c6700 1 -- 172.21.15.92:0/3837949494 shutdown_connections 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.336+0000 7f3dc76c6700 1 -- 172.21.15.92:0/3837949494 wait complete. 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.337+0000 7f3dc76c6700 1 Processor -- start 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.337+0000 7f3dc76c6700 1 -- start start 2023-05-16T07:14:40.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.337+0000 7f3dc76c6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3dc00aa150 0x7f3dc0164790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.337+0000 7f3dc76c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 0x7f3dc01651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc76c6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc0154470 0x7f3dc00afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc76c6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0164fe0 con 0x7f3dc0154090 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc76c6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc01552b0 con 0x7f3dc00aa150 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc76c6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0154980 con 0x7f3dc0154470 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 0x7f3dc01651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 0x7f3dc01651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43608/0 (socket says 172.21.15.92:43608) 2023-05-16T07:14:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 -- 172.21.15.92:0/105676183 learned_addr learned my addr 172.21.15.92:0/105676183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc5c63700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc0154470 0x7f3dc00afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc5462700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3dc00aa150 0x7f3dc0164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 -- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc0154470 msgr2=0x7f3dc00afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc0154470 0x7f3dc00afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 -- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3dc00aa150 msgr2=0x7f3dc0164790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3dc00aa150 0x7f3dc0164790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 -- 172.21.15.92:0/105676183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009370 con 0x7f3dc0154090 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc4c61700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 0x7f3dc01651b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f3dbc006fd0 tx=0x7f3dbc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc006950 con 0x7f3dc0154090 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc000bc0 con 0x7f3dc0154090 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc0154c60 con 0x7f3dc0154090 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc024470 con 0x7f3dc0154090 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.338+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc0164c80 con 0x7f3dc0154090 2023-05-16T07:14:40.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.339+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc0066730 con 0x7f3dc0154090 2023-05-16T07:14:40.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.339+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f3dbc00f070 con 0x7f3dc0154090 2023-05-16T07:14:40.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.340+0000 7f3db57fa700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3da405ee20 0x7f3da40612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.340+0000 7f3dc5462700 1 -- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3da405ee20 msgr2=0x7f3da40612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:40.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.340+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f3dbc07b7c0 con 0x7f3dc0154090 2023-05-16T07:14:40.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.340+0000 7f3dc5462700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3da405ee20 0x7f3da40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:40.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.344+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3dbc04ca20 con 0x7f3dc0154090 2023-05-16T07:14:40.416 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:40.412+0000 7fb5359cf680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:14:40.474 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:40.470+0000 7fcf5515d680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:14:40.516 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:40.513+0000 7fb5359cf680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:14:40.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.513+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3dc006c350 con 0x7f3dc0154090 2023-05-16T07:14:40.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.514+0000 7f3db57fa700 1 -- 172.21.15.92:0/105676183 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3dbc050040 con 0x7f3dc0154090 2023-05-16T07:14:40.519 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:14:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3da405ee20 msgr2=0x7f3da40612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3da405ee20 0x7f3da40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 msgr2=0x7f3dc01651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 0x7f3dc01651b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f3dbc006fd0 tx=0x7f3dbc017000).stop 2023-05-16T07:14:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 shutdown_connections 2023-05-16T07:14:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3da405ee20 0x7f3da40612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3dc0154470 0x7f3dc00afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3dc00aa150 0x7f3dc0164790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 --2- 172.21.15.92:0/105676183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dc0154090 0x7f3dc01651b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 >> 172.21.15.92:0/105676183 conn(0x7f3dc0067920 msgr2=0x7f3dc014e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 shutdown_connections 2023-05-16T07:14:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.516+0000 7f3dc76c6700 1 -- 172.21.15.92:0/105676183 wait complete. 2023-05-16T07:14:40.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:14:40.581 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:40.576+0000 7fcf5515d680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:14:40.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.644+0000 7f57f0e3c700 1 Processor -- start 2023-05-16T07:14:40.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.644+0000 7f57f0e3c700 1 -- start start 2023-05-16T07:14:40.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.645+0000 7f57f0e3c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f57ec0cb420 0x7f57ec154500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.645+0000 7f57f0e3c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec0c6320 con 0x7f57ec155580 2023-05-16T07:14:40.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.645+0000 7f57f0e3c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec1550f0 con 0x7f57ec154120 2023-05-16T07:14:40.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.645+0000 7f57f0e3c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0c6840 con 0x7f57ec0cb420 2023-05-16T07:14:40.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57ead9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f57ec155580 0x7f57ec068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54366/0 (socket says 172.21.15.92:54366) 2023-05-16T07:14:40.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57ead9d700 1 -- 172.21.15.92:0/199871975 learned_addr learned my addr 172.21.15.92:0/199871975 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:40.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e9d9b700 1 --2- 172.21.15.92:0/199871975 >> v2:172.21.15.158:3300/0 conn(0x7f57ec0cb420 0x7f57ec154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e959a700 1 -- 172.21.15.92:0/199871975 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041369174 0 0) 0x7f57e4001490 con 0x7f57ec155580 2023-05-16T07:14:40.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e959a700 1 -- 172.21.15.92:0/199871975 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d40033f0 con 0x7f57ec155580 2023-05-16T07:14:40.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e9d9b700 1 -- 172.21.15.92:0/199871975 >> v1:172.21.15.158:6789/0 conn(0x7f57ec154120 legacy=0x7f57ec0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e9d9b700 1 -- 172.21.15.92:0/199871975 >> v1:172.21.15.92:6789/0 conn(0x7f57ec155580 legacy=0x7f57ec068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e9d9b700 1 -- 172.21.15.92:0/199871975 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec0c6560 con 0x7f57ec0cb420 2023-05-16T07:14:40.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e9d9b700 1 --2- 172.21.15.92:0/199871975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec0cb420 0x7f57ec154500 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f57dc009760 tx=0x7f57dc009060).ready entity=mon.1 client_cookie=e833c3819c058840 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e959a700 1 -- 172.21.15.92:0/199871975 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00f510 con 0x7f57ec0cb420 2023-05-16T07:14:40.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e959a700 1 -- 172.21.15.92:0/199871975 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00f670 con 0x7f57ec0cb420 2023-05-16T07:14:40.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.646+0000 7f57e959a700 1 -- 172.21.15.92:0/199871975 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00f9f0 con 0x7f57ec0cb420 2023-05-16T07:14:40.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 -- 172.21.15.92:0/199871975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec0cb420 msgr2=0x7f57ec154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/199871975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec0cb420 0x7f57ec154500 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f57dc009760 tx=0x7f57dc009060).stop 2023-05-16T07:14:40.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 -- 172.21.15.92:0/199871975 shutdown_connections 2023-05-16T07:14:40.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/199871975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec0cb420 0x7f57ec154500 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 -- 172.21.15.92:0/199871975 >> 172.21.15.92:0/199871975 conn(0x7f57ec0678e0 msgr2=0x7f57ec0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 -- 172.21.15.92:0/199871975 shutdown_connections 2023-05-16T07:14:40.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.647+0000 7f57f0e3c700 1 -- 172.21.15.92:0/199871975 wait complete. 2023-05-16T07:14:40.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.648+0000 7f57f0e3c700 1 Processor -- start 2023-05-16T07:14:40.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.648+0000 7f57f0e3c700 1 -- start start 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57f0e3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 0x7f57ec149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57f0e3c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec154120 0x7f57ec0ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57f0e3c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57ec155580 0x7f57ec0abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57f0e3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0af2a0 con 0x7f57ec0cb420 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57f0e3c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0c3070 con 0x7f57ec154120 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57f0e3c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0aea00 con 0x7f57ec155580 2023-05-16T07:14:40.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57ea59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 0x7f57ec149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57ea59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 0x7f57ec149990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43620/0 (socket says 172.21.15.92:43620) 2023-05-16T07:14:40.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57ea59c700 1 -- 172.21.15.92:0/1098608416 learned_addr learned my addr 172.21.15.92:0/1098608416 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:40.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57e9d9b700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec154120 0x7f57ec0ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.649+0000 7f57ead9d700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57ec155580 0x7f57ec0abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57ea59c700 1 -- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57ec155580 msgr2=0x7f57ec0abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57ea59c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57ec155580 0x7f57ec0abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57ea59c700 1 -- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec154120 msgr2=0x7f57ec0ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57ea59c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec154120 0x7f57ec0ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57ea59c700 1 -- 172.21.15.92:0/1098608416 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0092a0 con 0x7f57ec0cb420 2023-05-16T07:14:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57ea59c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 0x7f57ec149990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f57e000c4c0 tx=0x7f57e0007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000ce70 con 0x7f57ec0cb420 2023-05-16T07:14:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000f470 con 0x7f57ec0cb420 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0aece0 con 0x7f57ec0cb420 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000f810 con 0x7f57ec0cb420 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.650+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec0c1e20 con 0x7f57ec0cb420 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.651+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec066730 con 0x7f57ec0cb420 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.651+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f57e0010070 con 0x7f57ec0cb420 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.652+0000 7f57da7fc700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57cc05ee20 0x7f57cc0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.652+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f57e0079230 con 0x7f57ec0cb420 2023-05-16T07:14:40.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.652+0000 7f57e9d9b700 1 -- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57cc05ee20 msgr2=0x7f57cc0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:40.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.652+0000 7f57e9d9b700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57cc05ee20 0x7f57cc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:40.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.656+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f57e00478f0 con 0x7f57ec0cb420 2023-05-16T07:14:40.667 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:40.663+0000 7fb5359cf680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:14:40.736 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:40.731+0000 7fcf5515d680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:14:40.741 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:40.737+0000 7fb5359cf680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:14:40.807 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:40.804+0000 7fb5359cf680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:14:40.811 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:40.806+0000 7fcf5515d680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:14:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.822+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57ec06c350 con 0x7f57ec0cb420 2023-05-16T07:14:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.823+0000 7f57da7fc700 1 -- 172.21.15.92:0/1098608416 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57e004a490 con 0x7f57ec0cb420 2023-05-16T07:14:40.828 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:14:40.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.824+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57cc05ee20 msgr2=0x7f57cc0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:40.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.824+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57cc05ee20 0x7f57cc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:40.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.824+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 msgr2=0x7f57ec149990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 0x7f57ec149990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f57e000c4c0 tx=0x7f57e0007df0).stop 2023-05-16T07:14:40.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 shutdown_connections 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57cc05ee20 0x7f57cc0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57ec155580 0x7f57ec0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cb420 0x7f57ec149990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 --2- 172.21.15.92:0/1098608416 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57ec154120 0x7f57ec0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 >> 172.21.15.92:0/1098608416 conn(0x7f57ec0678e0 msgr2=0x7f57ec068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 shutdown_connections 2023-05-16T07:14:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.825+0000 7f57f0e3c700 1 -- 172.21.15.92:0/1098608416 wait complete. 2023-05-16T07:14:40.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:40.882 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:40.877+0000 7fcf5515d680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:14:40.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.950+0000 7fbac835f700 1 Processor -- start 2023-05-16T07:14:40.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.951+0000 7fbac835f700 1 -- start start 2023-05-16T07:14:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac835f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac00a5890 0x7fbac015f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac835f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac00bd240 con 0x7fbac0068a10 2023-05-16T07:14:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac835f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac00bc360 con 0x7fbac0157ee0 2023-05-16T07:14:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac835f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00bc630 con 0x7fbac00a5890 2023-05-16T07:14:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac60fb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbac0068a10 0x7fbac0068df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54376/0 (socket says 172.21.15.92:54376) 2023-05-16T07:14:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac60fb700 1 -- 172.21.15.92:0/1684410272 learned_addr learned my addr 172.21.15.92:0/1684410272 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac68fc700 1 --2- 172.21.15.92:0/1684410272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac00a5890 0x7fbac015f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913712881 0 0) 0x7fbac00bd240 con 0x7fbac0068a10 2023-05-16T07:14:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaac0033f0 con 0x7fbac0068a10 2023-05-16T07:14:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034166869 0 0) 0x7fbac00bc360 con 0x7fbac0157ee0 2023-05-16T07:14:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac00bd240 con 0x7fbac0157ee0 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac68fc700 1 -- 172.21.15.92:0/1684410272 >> v1:172.21.15.158:6789/0 conn(0x7fbac0157ee0 legacy=0x7fbac00bbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac68fc700 1 -- 172.21.15.92:0/1684410272 >> v1:172.21.15.92:6789/0 conn(0x7fbac0068a10 legacy=0x7fbac0068df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843363881 0 0) 0x7fbaac0033f0 con 0x7fbac0068a10 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac68fc700 1 -- 172.21.15.92:0/1684410272 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00bc910 con 0x7fbac00a5890 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac68fc700 1 --2- 172.21.15.92:0/1684410272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac00a5890 0x7fbac015f180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fbab8009e90 tx=0x7fbab8009780).ready entity=mon.2 client_cookie=8e11594ef1b99b1b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab800e070 con 0x7fbac00a5890 2023-05-16T07:14:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.952+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab800b7a0 con 0x7fbac00a5890 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac50f9700 1 -- 172.21.15.92:0/1684410272 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0001470 con 0x7fbac00a5890 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 -- 172.21.15.92:0/1684410272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac00a5890 msgr2=0x7fbac015f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 --2- 172.21.15.92:0/1684410272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac00a5890 0x7fbac015f180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fbab8009e90 tx=0x7fbab8009780).stop 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 -- 172.21.15.92:0/1684410272 shutdown_connections 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 --2- 172.21.15.92:0/1684410272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac00a5890 0x7fbac015f180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 -- 172.21.15.92:0/1684410272 >> 172.21.15.92:0/1684410272 conn(0x7fbac00c6cf0 msgr2=0x7fbac00c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 -- 172.21.15.92:0/1684410272 shutdown_connections 2023-05-16T07:14:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.953+0000 7fbac835f700 1 -- 172.21.15.92:0/1684410272 wait complete. 2023-05-16T07:14:40.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.954+0000 7fbac835f700 1 Processor -- start 2023-05-16T07:14:40.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.954+0000 7fbac835f700 1 -- start start 2023-05-16T07:14:40.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac835f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac0068a10 0x7fbac013ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac835f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbac00a5890 0x7fbac00ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac835f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 0x7fbac0065720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac835f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00acba0 con 0x7fbac0157ee0 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac835f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00abf90 con 0x7fbac00a5890 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac835f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00ac270 con 0x7fbac0068a10 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 0x7fbac0065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 0x7fbac0065720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43638/0 (socket says 172.21.15.92:43638) 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 -- 172.21.15.92:0/593752188 learned_addr learned my addr 172.21.15.92:0/593752188 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac60fb700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac0068a10 0x7fbac013ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac58fa700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbac00a5890 0x7fbac00ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:40.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 -- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac0068a10 msgr2=0x7fbac013ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac0068a10 0x7fbac013ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 -- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbac00a5890 msgr2=0x7fbac00ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbac00a5890 0x7fbac00ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.955+0000 7fbac68fc700 1 -- 172.21.15.92:0/593752188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab8009a20 con 0x7fbac0157ee0 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.956+0000 7fbac68fc700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 0x7fbac0065720 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fbac0064000 tx=0x7fbab8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.956+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab801d440 con 0x7fbac0157ee0 2023-05-16T07:14:40.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.956+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab801d5a0 con 0x7fbac0157ee0 2023-05-16T07:14:40.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.956+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac00ac550 con 0x7fbac0157ee0 2023-05-16T07:14:40.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.956+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab801d920 con 0x7fbac0157ee0 2023-05-16T07:14:40.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.956+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac013b120 con 0x7fbac0157ee0 2023-05-16T07:14:40.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.957+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac005e7b0 con 0x7fbac0157ee0 2023-05-16T07:14:40.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.957+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fbab8024030 con 0x7fbac0157ee0 2023-05-16T07:14:40.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.958+0000 7fbab67fc700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 0x7fbaa40612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:40.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.958+0000 7fbac60fb700 1 -- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 msgr2=0x7fbaa40612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:40.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.958+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7fbab807b710 con 0x7fbac0157ee0 2023-05-16T07:14:40.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.958+0000 7fbac60fb700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 0x7fbaa40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:40.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:40.962+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbab804caa0 con 0x7fbac0157ee0 2023-05-16T07:14:41.062 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:41.058+0000 7fb5359cf680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:14:41.142 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:41.137+0000 7fcf5515d680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:14:41.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.159+0000 7fbac60fb700 1 -- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 msgr2=0x7fbaa40612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:41.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.159+0000 7fbac60fb700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 0x7fbaa40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:14:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.172+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbac006c350 con 0x7fbac0157ee0 2023-05-16T07:14:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.173+0000 7fbab67fc700 1 -- 172.21.15.92:0/593752188 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4489 (secure 0 0 0) 0x7fbab8022020 con 0x7fbac0157ee0 2023-05-16T07:14:41.178 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:41.178 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:03:09.943261+0000","modified":"2023-05-16T07:03:11.948935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2108552286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2108552286},{"type":"v1","addr":"172.21.15.92:6835","nonce":2108552286}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-05-16T07:14:41.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 msgr2=0x7fbaa40612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 0x7fbaa40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 msgr2=0x7fbac0065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 0x7fbac0065720 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fbac0064000 tx=0x7fbab8009780).stop 2023-05-16T07:14:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 shutdown_connections 2023-05-16T07:14:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbaa405ee20 0x7fbaa40612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbac0068a10 0x7fbac013ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbac0157ee0 0x7fbac0065720 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 --2- 172.21.15.92:0/593752188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbac00a5890 0x7fbac00ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 >> 172.21.15.92:0/593752188 conn(0x7fbac00c6cf0 msgr2=0x7fbac013c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 shutdown_connections 2023-05-16T07:14:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.175+0000 7fbac835f700 1 -- 172.21.15.92:0/593752188 wait complete. 2023-05-16T07:14:41.181 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2023-05-16T07:14:41.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:41.241 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:41.238+0000 7fb5359cf680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:14:41.327 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:41.322+0000 7fcf5515d680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:14:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.342+0000 7fbbf06a5700 1 Processor -- start 2023-05-16T07:14:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.342+0000 7fbbf06a5700 1 -- start start 2023-05-16T07:14:41.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbf06a5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe8068f10 0x7fbbe8150f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbf06a5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbbe80bc8f0 0x7fbbe80bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbf06a5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80a61d0 con 0x7fbbe8150b50 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbf06a5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe8066330 con 0x7fbbe8068f10 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbf06a5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe80bd240 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe8068f10 0x7fbbe8150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe8068f10 0x7fbbe8150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43644/0 (socket says 172.21.15.92:43644) 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 -- 172.21.15.92:0/1710700045 learned_addr learned my addr 172.21.15.92:0/1710700045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbee441700 1 --1- 172.21.15.92:0/1710700045 >> v1:172.21.15.92:6789/0 conn(0x7fbbe8150b50 0x7fbbe80a5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54392/0 (socket says 172.21.15.92:54392) 2023-05-16T07:14:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbeec42700 1 --2- 172.21.15.92:0/1710700045 >> v2:172.21.15.158:3300/0 conn(0x7fbbe80bc8f0 0x7fbbe80bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 -- 172.21.15.92:0/1710700045 >> v2:172.21.15.158:3300/0 conn(0x7fbbe80bc8f0 msgr2=0x7fbbe80bcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 --2- 172.21.15.92:0/1710700045 >> v2:172.21.15.158:3300/0 conn(0x7fbbe80bc8f0 0x7fbbe80bcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 -- 172.21.15.92:0/1710700045 >> v1:172.21.15.92:6789/0 conn(0x7fbbe8150b50 legacy=0x7fbbe80a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbed43f700 1 -- 172.21.15.92:0/1710700045 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310151169 0 0) 0x7fbbe80a61d0 con 0x7fbbe8150b50 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.343+0000 7fbbedc40700 1 -- 172.21.15.92:0/1710700045 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8069410 con 0x7fbbe8068f10 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbedc40700 1 --2- 172.21.15.92:0/1710700045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe8068f10 0x7fbbe8150f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fbbe80c3220 tx=0x7fbbe40090f0).ready entity=mon.0 client_cookie=2e9a27b8dae3d994 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbed43f700 1 -- 172.21.15.92:0/1710700045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400b520 con 0x7fbbe8068f10 2023-05-16T07:14:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbed43f700 1 -- 172.21.15.92:0/1710700045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe400b680 con 0x7fbbe8068f10 2023-05-16T07:14:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbed43f700 1 -- 172.21.15.92:0/1710700045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400ba00 con 0x7fbbe8068f10 2023-05-16T07:14:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 -- 172.21.15.92:0/1710700045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe8068f10 msgr2=0x7fbbe8150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/1710700045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe8068f10 0x7fbbe8150f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fbbe80c3220 tx=0x7fbbe40090f0).stop 2023-05-16T07:14:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 -- 172.21.15.92:0/1710700045 shutdown_connections 2023-05-16T07:14:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/1710700045 >> v2:172.21.15.158:3300/0 conn(0x7fbbe80bc8f0 0x7fbbe80bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/1710700045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe8068f10 0x7fbbe8150f30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 -- 172.21.15.92:0/1710700045 >> 172.21.15.92:0/1710700045 conn(0x7fbbe80c7350 msgr2=0x7fbbe80c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 -- 172.21.15.92:0/1710700045 shutdown_connections 2023-05-16T07:14:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.344+0000 7fbbf06a5700 1 -- 172.21.15.92:0/1710700045 wait complete. 2023-05-16T07:14:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.345+0000 7fbbf06a5700 1 Processor -- start 2023-05-16T07:14:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.345+0000 7fbbf06a5700 1 -- start start 2023-05-16T07:14:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbf06a5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbe8068f10 0x7fbbe814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbf06a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 0x7fbbe80aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbf06a5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbe8150b50 0x7fbbe80aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbf06a5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8165c10 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbf06a5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8165040 con 0x7fbbe8150b50 2023-05-16T07:14:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbf06a5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80aabc0 con 0x7fbbe8068f10 2023-05-16T07:14:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbedc40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 0x7fbbe80aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbedc40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 0x7fbbe80aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43650/0 (socket says 172.21.15.92:43650) 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbedc40700 1 -- 172.21.15.92:0/220779221 learned_addr learned my addr 172.21.15.92:0/220779221 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbeec42700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbe8150b50 0x7fbbe80aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.346+0000 7fbbee441700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbe8068f10 0x7fbbe814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbedc40700 1 -- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbe8068f10 msgr2=0x7fbbe814b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbedc40700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbe8068f10 0x7fbbe814b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbedc40700 1 -- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbe8150b50 msgr2=0x7fbbe80aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbedc40700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbe8150b50 0x7fbbe80aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbedc40700 1 -- 172.21.15.92:0/220779221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe4009370 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbee441700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbe8068f10 0x7fbbe814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbedc40700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 0x7fbbe80aa1e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fbbe4009130 tx=0x7fbbe4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400be90 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe400b7f0 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401c670 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe81652e0 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.347+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81658a0 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.348+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8165d80 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.348+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fbbe402c070 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.348+0000 7fbbde7fc700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbcc05ee20 0x7fbbcc0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.348+0000 7fbbee441700 1 -- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbcc05ee20 msgr2=0x7fbbcc0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.348+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7fbbe407ba80 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.348+0000 7fbbee441700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbcc05ee20 0x7fbbcc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.352+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbbe404cce0 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.521+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbe806c350 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.522+0000 7fbbde7fc700 1 -- 172.21.15.92:0/220779221 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18204 (secure 0 0 0) 0x7fbbe402a020 con 0x7fbbe80bc8f0 2023-05-16T07:14:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:41.527 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:39.602984+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:03:07.846932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-05-16T07:03:08.910782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/1428741227":"2023-05-17T07:03:24.745231+0000","172.21.15.158:0/2276917568":"2023-05-17T07:03:24.745231+0000","172.21.15.158:6833/86301":"2023-05-17T07:03:24.745231+0000","172.21.15.158:6832/86301":"2023-05-17T07:03:24.745231+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":109,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:41.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbcc05ee20 msgr2=0x7fbbcc0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:41.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbcc05ee20 0x7fbbcc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 msgr2=0x7fbbe80aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 0x7fbbe80aa1e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fbbe4009130 tx=0x7fbbe4017000).stop 2023-05-16T07:14:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 shutdown_connections 2023-05-16T07:14:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbcc05ee20 0x7fbbcc0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbe8068f10 0x7fbbe814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbe8150b50 0x7fbbe80aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 --2- 172.21.15.92:0/220779221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe80bc8f0 0x7fbbe80aa1e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 >> 172.21.15.92:0/220779221 conn(0x7fbbe80c7350 msgr2=0x7fbbe80bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:41.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 shutdown_connections 2023-05-16T07:14:41.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.524+0000 7fbbf06a5700 1 -- 172.21.15.92:0/220779221 wait complete. 2023-05-16T07:14:41.541 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:14:41.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.650+0000 7f0b25676700 1 Processor -- start 2023-05-16T07:14:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.650+0000 7f0b25676700 1 -- start start 2023-05-16T07:14:41.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b25676700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b200c7db0 0x7f0b200c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b25676700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0b200a7b70 0x7f0b200c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b25676700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b20155260 con 0x7f0b20150840 2023-05-16T07:14:41.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b25676700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b20154690 con 0x7f0b200c7db0 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b25676700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200c5020 con 0x7f0b200a7b70 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b200c7db0 0x7f0b200c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b200c7db0 0x7f0b200c27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43684/0 (socket says 172.21.15.92:43684) 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/1381987506 learned_addr learned my addr 172.21.15.92:0/1381987506 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1f7fe700 1 --2- 172.21.15.92:0/1381987506 >> v2:172.21.15.158:3300/0 conn(0x7f0b200a7b70 0x7f0b200c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/1381987506 >> v2:172.21.15.158:3300/0 conn(0x7f0b200a7b70 msgr2=0x7f0b200c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 --2- 172.21.15.92:0/1381987506 >> v2:172.21.15.158:3300/0 conn(0x7f0b200a7b70 0x7f0b200c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/1381987506 >> v1:172.21.15.158:6789/0 conn(0x7f0b20150840 legacy=0x7f0b200c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/1381987506 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20154930 con 0x7f0b200c7db0 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.651+0000 7f0b1e7fc700 1 --2- 172.21.15.92:0/1381987506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b200c27d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f0b200648d0 tx=0x7f0b1000b3b0).ready entity=mon.0 client_cookie=82f7c2fd24682604 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b1dffb700 1 -- 172.21.15.92:0/1381987506 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000da40 con 0x7f0b200c7db0 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b1dffb700 1 -- 172.21.15.92:0/1381987506 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1000dba0 con 0x7f0b200c7db0 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b1dffb700 1 -- 172.21.15.92:0/1381987506 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10010470 con 0x7f0b200c7db0 2023-05-16T07:14:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 -- 172.21.15.92:0/1381987506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 msgr2=0x7f0b200c27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 --2- 172.21.15.92:0/1381987506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b200c27d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f0b200648d0 tx=0x7f0b1000b3b0).stop 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 -- 172.21.15.92:0/1381987506 shutdown_connections 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 --2- 172.21.15.92:0/1381987506 >> v2:172.21.15.158:3300/0 conn(0x7f0b200a7b70 0x7f0b200c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 --2- 172.21.15.92:0/1381987506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b200c27d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 -- 172.21.15.92:0/1381987506 >> 172.21.15.92:0/1381987506 conn(0x7f0b200689e0 msgr2=0x7f0b200695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 -- 172.21.15.92:0/1381987506 shutdown_connections 2023-05-16T07:14:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.652+0000 7f0b25676700 1 -- 172.21.15.92:0/1381987506 wait complete. 2023-05-16T07:14:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.653+0000 7f0b25676700 1 Processor -- start 2023-05-16T07:14:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.653+0000 7f0b25676700 1 -- start start 2023-05-16T07:14:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b25676700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b200a7b70 0x7f0b20167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b25676700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b201535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b25676700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b20150840 0x7f0b20153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b25676700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20141060 con 0x7f0b200c7db0 2023-05-16T07:14:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b25676700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b201401d0 con 0x7f0b20150840 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b25676700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b201414f0 con 0x7f0b200a7b70 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b1e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b201535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b1e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b201535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43694/0 (socket says 172.21.15.92:43694) 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/2640180338 learned_addr learned my addr 172.21.15.92:0/2640180338 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b1effd700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b200a7b70 0x7f0b20167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.654+0000 7f0b1f7fe700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b20150840 0x7f0b20153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b200a7b70 msgr2=0x7f0b20167cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b1e7fc700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b200a7b70 0x7f0b20167cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b20150840 msgr2=0x7f0b20153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b1e7fc700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b20150840 0x7f0b20153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b1e7fc700 1 -- 172.21.15.92:0/2640180338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1000b610 con 0x7f0b200c7db0 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b1e7fc700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b201535e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f0b10013000 tx=0x7f0b1000b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10019450 con 0x7f0b200c7db0 2023-05-16T07:14:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100195b0 con 0x7f0b200c7db0 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b20140450 con 0x7f0b200c7db0 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10019930 con 0x7f0b200c7db0 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.655+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b20140a10 con 0x7f0b200c7db0 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.656+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b20047150 con 0x7f0b200c7db0 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.656+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f0b1002a070 con 0x7f0b200c7db0 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.657+0000 7f0b06ffd700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 0x7f0b0c0612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.657+0000 7f0b1effd700 1 -- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 msgr2=0x7f0b0c0612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.657+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f0b1007b4e0 con 0x7f0b200c7db0 2023-05-16T07:14:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.657+0000 7f0b1effd700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 0x7f0b0c0612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.662+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0b1004c6e0 con 0x7f0b200c7db0 2023-05-16T07:14:41.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.858+0000 7f0b1effd700 1 -- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 msgr2=0x7f0b0c0612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:41.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.858+0000 7f0b1effd700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 0x7f0b0c0612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:14:41.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.870+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b20154c10 con 0x7f0b200c7db0 2023-05-16T07:14:41.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.871+0000 7f0b06ffd700 1 -- 172.21.15.92:0/2640180338 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f0b10028020 con 0x7f0b200c7db0 2023-05-16T07:14:41.875 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:41.875 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:03:09.943261+0000","modified":"2023-05-16T07:03:11.948935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2108552286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2108552286},{"type":"v1","addr":"172.21.15.92:6835","nonce":2108552286}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-05-16T07:14:41.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 msgr2=0x7f0b0c0612b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 0x7f0b0c0612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 msgr2=0x7f0b201535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b201535e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f0b10013000 tx=0x7f0b1000b3b0).stop 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 shutdown_connections 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b0c05edf0 0x7f0b0c0612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b200a7b70 0x7f0b20167cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b20150840 0x7f0b20153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 --2- 172.21.15.92:0/2640180338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b200c7db0 0x7f0b201535e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:41.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.872+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 >> 172.21.15.92:0/2640180338 conn(0x7f0b200689e0 msgr2=0x7f0b200a7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:41.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.873+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 shutdown_connections 2023-05-16T07:14:41.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:41.873+0000 7f0b25676700 1 -- 172.21.15.92:0/2640180338 wait complete. 2023-05-16T07:14:41.878 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2023-05-16T07:14:41.880 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:41.876+0000 7fb5359cf680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:14:41.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:41.947 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:41.943+0000 7fb5359cf680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:14:41.989 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:41.985+0000 7fcf5515d680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:14:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.000+0000 7f3baf332700 1 Processor -- start 2023-05-16T07:14:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.000+0000 7f3baf332700 1 -- start start 2023-05-16T07:14:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3baf332700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ba80c43a0 0x7f3ba80c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3baf332700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3ba80a7a80 0x7f3ba80a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3baf332700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80657a0 0x7f3ba8065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3baf332700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80660e0 con 0x7f3ba80c43a0 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3baf332700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba8064d00 con 0x7f3ba80a7a80 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3baf332700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80c2e60 con 0x7f3ba80657a0 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.001+0000 7f3bae330700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ba80c43a0 0x7f3ba80c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ba80c43a0 0x7f3ba80c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43700/0 (socket says 172.21.15.92:43700) 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 -- 172.21.15.92:0/3038073112 learned_addr learned my addr 172.21.15.92:0/3038073112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3badb2f700 1 --2- 172.21.15.92:0/3038073112 >> v2:172.21.15.158:3300/0 conn(0x7f3ba80a7a80 0x7f3ba80a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3baeb31700 1 --2- 172.21.15.92:0/3038073112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80657a0 0x7f3ba8065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 -- 172.21.15.92:0/3038073112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80657a0 msgr2=0x7f3ba8065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 --2- 172.21.15.92:0/3038073112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80657a0 0x7f3ba8065bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 -- 172.21.15.92:0/3038073112 >> v2:172.21.15.158:3300/0 conn(0x7f3ba80a7a80 msgr2=0x7f3ba80a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 --2- 172.21.15.92:0/3038073112 >> v2:172.21.15.158:3300/0 conn(0x7f3ba80a7a80 0x7f3ba80a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 -- 172.21.15.92:0/3038073112 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80c3140 con 0x7f3ba80c43a0 2023-05-16T07:14:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3badb2f700 1 --2- 172.21.15.92:0/3038073112 >> v2:172.21.15.158:3300/0 conn(0x7f3ba80a7a80 0x7f3ba80a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bae330700 1 --2- 172.21.15.92:0/3038073112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80c43a0 0x7f3ba80c4780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f3ba80c5300 tx=0x7f3ba40090f0).ready entity=mon.0 client_cookie=c89aad93a38c0565 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bad32e700 1 -- 172.21.15.92:0/3038073112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400b520 con 0x7f3ba80c43a0 2023-05-16T07:14:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bad32e700 1 -- 172.21.15.92:0/3038073112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba400b680 con 0x7f3ba80c43a0 2023-05-16T07:14:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.002+0000 7f3bad32e700 1 -- 172.21.15.92:0/3038073112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400ba00 con 0x7f3ba80c43a0 2023-05-16T07:14:42.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 -- 172.21.15.92:0/3038073112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80c43a0 msgr2=0x7f3ba80c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 --2- 172.21.15.92:0/3038073112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80c43a0 0x7f3ba80c4780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f3ba80c5300 tx=0x7f3ba40090f0).stop 2023-05-16T07:14:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 -- 172.21.15.92:0/3038073112 shutdown_connections 2023-05-16T07:14:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 --2- 172.21.15.92:0/3038073112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80657a0 0x7f3ba8065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 --2- 172.21.15.92:0/3038073112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80c43a0 0x7f3ba80c4780 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 --2- 172.21.15.92:0/3038073112 >> v2:172.21.15.158:3300/0 conn(0x7f3ba80a7a80 0x7f3ba80a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 -- 172.21.15.92:0/3038073112 >> 172.21.15.92:0/3038073112 conn(0x7f3ba8067f60 msgr2=0x7f3ba8068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 -- 172.21.15.92:0/3038073112 shutdown_connections 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.003+0000 7f3baf332700 1 -- 172.21.15.92:0/3038073112 wait complete. 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.004+0000 7f3baf332700 1 Processor -- start 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.004+0000 7f3baf332700 1 -- start start 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 0x7f3ba80be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80a7a80 0x7f3ba8164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ba80c43a0 0x7f3ba8152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80bfac0 con 0x7f3ba80657a0 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80bff50 con 0x7f3ba80c43a0 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80beeb0 con 0x7f3ba80a7a80 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 0x7f3ba80be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 0x7f3ba80be910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43706/0 (socket says 172.21.15.92:43706) 2023-05-16T07:14:42.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 -- 172.21.15.92:0/763623746 learned_addr learned my addr 172.21.15.92:0/763623746 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3badb2f700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80a7a80 0x7f3ba8164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baeb31700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ba80c43a0 0x7f3ba8152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 -- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80a7a80 msgr2=0x7f3ba8164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80a7a80 0x7f3ba8164ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 -- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ba80c43a0 msgr2=0x7f3ba8152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ba80c43a0 0x7f3ba8152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 -- 172.21.15.92:0/763623746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009370 con 0x7f3ba80657a0 2023-05-16T07:14:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baeb31700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ba80c43a0 0x7f3ba8152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3bae330700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 0x7f3ba80be910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f3ba80638d0 tx=0x7f3ba4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4000bc0 con 0x7f3ba80657a0 2023-05-16T07:14:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba400b7f0 con 0x7f3ba80657a0 2023-05-16T07:14:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4000d20 con 0x7f3ba80657a0 2023-05-16T07:14:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80bf190 con 0x7f3ba80657a0 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.005+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba80bf750 con 0x7f3ba80657a0 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.006+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba8047150 con 0x7f3ba80657a0 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.007+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f3ba4021030 con 0x7f3ba80657a0 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.007+0000 7f3b9e7fc700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 0x7f3b8c0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.007+0000 7f3badb2f700 1 -- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 msgr2=0x7f3b8c0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.007+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f3ba407b9d0 con 0x7f3ba80657a0 2023-05-16T07:14:42.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.007+0000 7f3badb2f700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 0x7f3b8c0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:42.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.011+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3ba404cc30 con 0x7f3ba80657a0 2023-05-16T07:14:42.020 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.016+0000 7fb5359cf680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:14:42.061 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.057+0000 7fcf5515d680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:14:42.136 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.132+0000 7fcf5515d680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:14:42.161 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.158+0000 7fb5359cf680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:14:42.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.208+0000 7f3badb2f700 1 -- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 msgr2=0x7f3b8c0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:42.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.208+0000 7f3badb2f700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 0x7f3b8c0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:14:42.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.219+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba8064fa0 con 0x7f3ba80657a0 2023-05-16T07:14:42.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.220+0000 7f3b9e7fc700 1 -- 172.21.15.92:0/763623746 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f3ba401f020 con 0x7f3ba80657a0 2023-05-16T07:14:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:42.226 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:03:09.943261+0000","modified":"2023-05-16T07:03:11.948935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2108552286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2108552286},{"type":"v1","addr":"172.21.15.92:6835","nonce":2108552286}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-05-16T07:14:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 msgr2=0x7f3b8c0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 0x7f3b8c0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 msgr2=0x7f3ba80be910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 0x7f3ba80be910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f3ba80638d0 tx=0x7f3ba4017000).stop 2023-05-16T07:14:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 shutdown_connections 2023-05-16T07:14:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3b8c05ee20 0x7f3b8c0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ba80c43a0 0x7f3ba8152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ba80657a0 0x7f3ba80be910 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 --2- 172.21.15.92:0/763623746 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3ba80a7a80 0x7f3ba8164ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 >> 172.21.15.92:0/763623746 conn(0x7f3ba8067f60 msgr2=0x7f3ba80a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:42.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 shutdown_connections 2023-05-16T07:14:42.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.222+0000 7f3baf332700 1 -- 172.21.15.92:0/763623746 wait complete. 2023-05-16T07:14:42.229 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2023-05-16T07:14:42.230 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.226+0000 7fb5359cf680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:14:42.237 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:42.286 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.282+0000 7fcf5515d680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:14:42.341 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.337+0000 7fb5359cf680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:14:42.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.347+0000 7fcc3aa81700 1 Processor -- start 2023-05-16T07:14:42.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.347+0000 7fcc3aa81700 1 -- start start 2023-05-16T07:14:42.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3aa81700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc34154e50 0x7fcc340acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3aa81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc3414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3aa81700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc34158970 con 0x7fcc340c7380 2023-05-16T07:14:42.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3aa81700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc34155300 con 0x7fcc34154e50 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3aa81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34157d60 con 0x7fcc340ad040 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc34154e50 0x7fcc340acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc34154e50 0x7fcc340acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43718/0 (socket says 172.21.15.92:43718) 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 -- 172.21.15.92:0/1210489848 learned_addr learned my addr 172.21.15.92:0/1210489848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3a280700 1 --2- 172.21.15.92:0/1210489848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc3414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 -- 172.21.15.92:0/1210489848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 msgr2=0x7fcc3414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 --2- 172.21.15.92:0/1210489848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc3414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 -- 172.21.15.92:0/1210489848 >> v1:172.21.15.158:6789/0 conn(0x7fcc340c7380 legacy=0x7fcc340c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 -- 172.21.15.92:0/1210489848 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34158040 con 0x7fcc34154e50 2023-05-16T07:14:42.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc3927e700 1 --2- 172.21.15.92:0/1210489848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340acb50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fcc24006fd0 tx=0x7fcc240090f0).ready entity=mon.0 client_cookie=1f6e31d5f95b4bc0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:42.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc38a7d700 1 -- 172.21.15.92:0/1210489848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400b580 con 0x7fcc34154e50 2023-05-16T07:14:42.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc38a7d700 1 -- 172.21.15.92:0/1210489848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400b6e0 con 0x7fcc34154e50 2023-05-16T07:14:42.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.348+0000 7fcc38a7d700 1 -- 172.21.15.92:0/1210489848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400ba60 con 0x7fcc34154e50 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 -- 172.21.15.92:0/1210489848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 msgr2=0x7fcc340acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/1210489848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340acb50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fcc24006fd0 tx=0x7fcc240090f0).stop 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 -- 172.21.15.92:0/1210489848 shutdown_connections 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/1210489848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc3414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/1210489848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340acb50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 -- 172.21.15.92:0/1210489848 >> 172.21.15.92:0/1210489848 conn(0x7fcc34068330 msgr2=0x7fcc34068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 -- 172.21.15.92:0/1210489848 shutdown_connections 2023-05-16T07:14:42.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.349+0000 7fcc3aa81700 1 -- 172.21.15.92:0/1210489848 wait complete. 2023-05-16T07:14:42.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.350+0000 7fcc3aa81700 1 Processor -- start 2023-05-16T07:14:42.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.350+0000 7fcc3aa81700 1 -- start start 2023-05-16T07:14:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3aa81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc34165020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3aa81700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc340c7380 0x7fcc34167a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3aa81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3aa81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340af0a0 con 0x7fcc34154e50 2023-05-16T07:14:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3aa81700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34168270 con 0x7fcc340c7380 2023-05-16T07:14:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3aa81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34168550 con 0x7fcc340ad040 2023-05-16T07:14:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43726/0 (socket says 172.21.15.92:43726) 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 -- 172.21.15.92:0/740969555 learned_addr learned my addr 172.21.15.92:0/740969555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc39a7f700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc34165020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3927e700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc340c7380 0x7fcc34167a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 -- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 msgr2=0x7fcc34165020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc34165020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 -- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc340c7380 msgr2=0x7fcc34167a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc340c7380 0x7fcc34167a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 -- 172.21.15.92:0/740969555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc24009370 con 0x7fcc34154e50 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc3a280700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340c4340 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fcc2c009e90 tx=0x7fcc2c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00caa0 con 0x7fcc34154e50 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.351+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c00f040 con 0x7fcc34154e50 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.352+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc34168830 con 0x7fcc34154e50 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.352+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00ce10 con 0x7fcc34154e50 2023-05-16T07:14:42.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.352+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc340aec90 con 0x7fcc34154e50 2023-05-16T07:14:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.353+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc340aee80 con 0x7fcc34154e50 2023-05-16T07:14:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.353+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fcc2c013610 con 0x7fcc34154e50 2023-05-16T07:14:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.353+0000 7fcc29ffb700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcc1805ee20 0x7fcc180612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.353+0000 7fcc39a7f700 1 -- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcc1805ee20 msgr2=0x7fcc180612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.353+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7fcc2c07a840 con 0x7fcc34154e50 2023-05-16T07:14:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.353+0000 7fcc39a7f700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcc1805ee20 0x7fcc180612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:42.365 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.354+0000 7fcf5515d680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:14:42.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.357+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcc2c04baa0 con 0x7fcc34154e50 2023-05-16T07:14:42.448 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.444+0000 7fb5359cf680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:14:42.474 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.471+0000 7fcf5515d680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:14:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.523+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc340af1e0 con 0x7fcc34154e50 2023-05-16T07:14:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.524+0000 7fcc29ffb700 1 -- 172.21.15.92:0/740969555 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18204 (secure 0 0 0) 0x7fcc2c04f0c0 con 0x7fcc34154e50 2023-05-16T07:14:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:42.529 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:39.602984+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:03:07.846932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-05-16T07:03:08.910782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/1428741227":"2023-05-17T07:03:24.745231+0000","172.21.15.158:0/2276917568":"2023-05-17T07:03:24.745231+0000","172.21.15.158:6833/86301":"2023-05-17T07:03:24.745231+0000","172.21.15.158:6832/86301":"2023-05-17T07:03:24.745231+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":109,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.526+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcc1805ee20 msgr2=0x7fcc180612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.526+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcc1805ee20 0x7fcc180612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.526+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 msgr2=0x7fcc340c4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.526+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340c4340 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fcc2c009e90 tx=0x7fcc2c00b040).stop 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 shutdown_connections 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcc1805ee20 0x7fcc180612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc340ad040 0x7fcc34165020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34154e50 0x7fcc340c4340 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 --2- 172.21.15.92:0/740969555 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc340c7380 0x7fcc34167a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 >> 172.21.15.92:0/740969555 conn(0x7fcc34068330 msgr2=0x7fcc34153900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 shutdown_connections 2023-05-16T07:14:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.527+0000 7fcc3aa81700 1 -- 172.21.15.92:0/740969555 wait complete. 2023-05-16T07:14:42.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:14:42.585 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.581+0000 7fcf5515d680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:14:42.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.654+0000 7f4c105d5700 1 Processor -- start 2023-05-16T07:14:42.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.654+0000 7f4c105d5700 1 -- start start 2023-05-16T07:14:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.655+0000 7f4c105d5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c08151a00 0x7f4c080663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.655+0000 7f4c105d5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c080bc760 0x7f4c08155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.655+0000 7f4c105d5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080a6ce0 con 0x7f4c08151de0 2023-05-16T07:14:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.655+0000 7f4c105d5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080c4b20 con 0x7f4c08151a00 2023-05-16T07:14:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.655+0000 7f4c105d5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08066a10 con 0x7f4c080bc760 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c08151a00 0x7f4c080663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c08151a00 0x7f4c080663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43730/0 (socket says 172.21.15.92:43730) 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 -- 172.21.15.92:0/3397867110 learned_addr learned my addr 172.21.15.92:0/3397867110 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0eb72700 1 --2- 172.21.15.92:0/3397867110 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c080bc760 0x7f4c08155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 -- 172.21.15.92:0/3397867110 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c080bc760 msgr2=0x7f4c08155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 --2- 172.21.15.92:0/3397867110 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c080bc760 0x7f4c08155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 -- 172.21.15.92:0/3397867110 >> v1:172.21.15.158:6789/0 conn(0x7f4c08151de0 legacy=0x7f4c080bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 -- 172.21.15.92:0/3397867110 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080c4560 con 0x7f4c08151a00 2023-05-16T07:14:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0e371700 1 --2- 172.21.15.92:0/3397867110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08151a00 0x7f4c080663a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f4bf8006fd0 tx=0x7f4bf80090f0).ready entity=mon.0 client_cookie=74102ae552a19623 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0d36f700 1 -- 172.21.15.92:0/3397867110 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800b580 con 0x7f4c08151a00 2023-05-16T07:14:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0d36f700 1 -- 172.21.15.92:0/3397867110 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf800b6e0 con 0x7f4c08151a00 2023-05-16T07:14:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.656+0000 7f4c0d36f700 1 -- 172.21.15.92:0/3397867110 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800ba60 con 0x7f4c08151a00 2023-05-16T07:14:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 -- 172.21.15.92:0/3397867110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08151a00 msgr2=0x7f4c080663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 --2- 172.21.15.92:0/3397867110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08151a00 0x7f4c080663a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f4bf8006fd0 tx=0x7f4bf80090f0).stop 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 -- 172.21.15.92:0/3397867110 shutdown_connections 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 --2- 172.21.15.92:0/3397867110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08151a00 0x7f4c080663a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 --2- 172.21.15.92:0/3397867110 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c080bc760 0x7f4c08155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 -- 172.21.15.92:0/3397867110 >> 172.21.15.92:0/3397867110 conn(0x7f4c080c7bc0 msgr2=0x7f4c080c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 -- 172.21.15.92:0/3397867110 shutdown_connections 2023-05-16T07:14:42.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.657+0000 7f4c105d5700 1 -- 172.21.15.92:0/3397867110 wait complete. 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.658+0000 7f4c105d5700 1 Processor -- start 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.658+0000 7f4c105d5700 1 -- start start 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c105d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 0x7f4c08163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c105d5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c08151a00 0x7f4c0812d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c105d5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c08151de0 0x7f4c08162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c105d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0812e040 con 0x7f4c080bc760 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c105d5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08163820 con 0x7f4c08151a00 2023-05-16T07:14:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c105d5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08162c10 con 0x7f4c08151de0 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 0x7f4c08163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 0x7f4c08163cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43732/0 (socket says 172.21.15.92:43732) 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 -- 172.21.15.92:0/2396157672 learned_addr learned my addr 172.21.15.92:0/2396157672 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0eb72700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c08151de0 0x7f4c08162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0db70700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c08151a00 0x7f4c0812d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c08151de0 msgr2=0x7f4c08162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c08151de0 0x7f4c08162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c08151a00 msgr2=0x7f4c0812d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c08151a00 0x7f4c0812d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 -- 172.21.15.92:0/2396157672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009370 con 0x7f4c080bc760 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.659+0000 7f4c0e371700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 0x7f4c08163cb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4bf8009130 tx=0x7f4bf8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.660+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8000bc0 con 0x7f4c080bc760 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.660+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf800b850 con 0x7f4c080bc760 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.660+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c08162ef0 con 0x7f4c080bc760 2023-05-16T07:14:42.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.660+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8000d20 con 0x7f4c080bc760 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.660+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c081634b0 con 0x7f4c080bc760 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.661+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c08163b00 con 0x7f4c080bc760 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.661+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f4bf802c070 con 0x7f4c080bc760 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.661+0000 7f4bfe7fc700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 0x7f4bec0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.661+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6922+0+0 (secure 0 0 0) 0x7f4bf807bd00 con 0x7f4c080bc760 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.661+0000 7f4c0db70700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 msgr2=0x7f4bec0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:42.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.662+0000 7f4c0db70700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 0x7f4bec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:14:42.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.666+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4bf804ce50 con 0x7f4c080bc760 2023-05-16T07:14:42.736 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.732+0000 7fb5359cf680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:14:42.805 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:42.802+0000 7fb5359cf680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:14:42.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.863+0000 7f4c0db70700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 msgr2=0x7f4bec0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:42.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.863+0000 7f4c0db70700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 0x7f4bec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:14:42.878 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.874+0000 7fcf5515d680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:14:42.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:42.877+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c0806c350 con 0x7f4c080bc760 2023-05-16T07:14:42.948 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:14:42.944+0000 7fcf5515d680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:14:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.264+0000 7f4c0db70700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 msgr2=0x7f4bec0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:14:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.264+0000 7f4c0db70700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 0x7f4bec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:14:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.748+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 12) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f4bf80aa8f0 con 0x7f4c080bc760 2023-05-16T07:14:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.748+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 msgr2=0x7f4bec0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:14:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.748+0000 7f4bfe7fc700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 0x7f4bec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:43.805 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b59b4700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.805 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b59b4700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.806 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b59b4700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.806 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b59b4700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.806 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b59b4700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.806 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b79b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.807 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b79b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.807 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b79b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.807 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b79b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.808 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:14:43.801+0000 7fb4b79b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:14:43.808 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:14:43.804+0000 7fc3137d7700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:14:43.809 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:14:43.804+0000 7fc3137d7700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:14:43.809 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:14:43.804+0000 7fc3137d7700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:14:43.820 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:14:43.816+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:14:43.823 INFO:tasks.ceph.mds.a.smithi092.stderr: -5> 2023-05-16T07:14:43.804+0000 7fc3137d7700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:14:43.824 INFO:tasks.ceph.mds.a.smithi092.stderr: -4> 2023-05-16T07:14:43.804+0000 7fc3137d7700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:14:43.824 INFO:tasks.ceph.mds.a.smithi092.stderr: -2> 2023-05-16T07:14:43.804+0000 7fc3137d7700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:14:43.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.820+0000 7f4bfe7fc700 1 -- 172.21.15.92:0/2396157672 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v13) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4bf8041af0 con 0x7f4c080bc760 2023-05-16T07:14:43.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 msgr2=0x7f4c08163cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:43.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 0x7f4c08163cb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4bf8009130 tx=0x7f4bf8017000).stop 2023-05-16T07:14:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 shutdown_connections 2023-05-16T07:14:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bec05ee20 0x7f4bec0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c08151de0 0x7f4c08162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c080bc760 0x7f4c08163cb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 --2- 172.21.15.92:0/2396157672 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c08151a00 0x7f4c0812d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 >> 172.21.15.92:0/2396157672 conn(0x7f4c080c7bc0 msgr2=0x7f4c080a7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:43.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 shutdown_connections 2023-05-16T07:14:43.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.822+0000 7f4c105d5700 1 -- 172.21.15.92:0/2396157672 wait complete. 2023-05-16T07:14:43.829 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:14:43.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:14:43.844 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:14:43.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.952+0000 7f3f8d5d9700 1 Processor -- start 2023-05-16T07:14:43.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.952+0000 7f3f8d5d9700 1 -- start start 2023-05-16T07:14:43.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f8d5d9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 0x7f3f880aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f8d5d9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 0x7f3f880685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f8d5d9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f880c94d0 con 0x7f3f88154090 2023-05-16T07:14:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f8d5d9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cb7e0 con 0x7f3f88068160 2023-05-16T07:14:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f8d5d9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88068ac0 con 0x7f3f88154470 2023-05-16T07:14:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f867fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 0x7f3f880aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f877fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 0x7f3f880685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:43.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f877fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 0x7f3f880685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:47398/0 (socket says 172.21.15.92:47398) 2023-05-16T07:14:43.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f867fc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 0x7f3f880aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:36334/0 (socket says 172.21.15.92:36334) 2023-05-16T07:14:43.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.953+0000 7f3f877fe700 1 -- 172.21.15.92:0/3873402906 learned_addr learned my addr 172.21.15.92:0/3873402906 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:43.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f867fc700 1 -- 172.21.15.92:0/3873402906 >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 msgr2=0x7f3f880685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:43.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f867fc700 1 --2- 172.21.15.92:0/3873402906 >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 0x7f3f880685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f867fc700 1 -- 172.21.15.92:0/3873402906 >> v1:172.21.15.158:6789/0 conn(0x7f3f88154090 legacy=0x7f3f880a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f867fc700 1 -- 172.21.15.92:0/3873402906 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880c88c0 con 0x7f3f88154470 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f85ffb700 1 -- 172.21.15.92:0/3873402906 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140017101 0 0) 0x7f3f880c94d0 con 0x7f3f88154090 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f877fe700 1 --2- 172.21.15.92:0/3873402906 >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 0x7f3f880685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f867fc700 1 --2- 172.21.15.92:0/3873402906 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 0x7f3f880aaca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f3f7c009760 tx=0x7f3f7c009060).ready entity=mon.2 client_cookie=731f5018cd0bb08f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f85ffb700 1 -- 172.21.15.92:0/3873402906 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00f510 con 0x7f3f88154470 2023-05-16T07:14:43.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f85ffb700 1 -- 172.21.15.92:0/3873402906 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00f670 con 0x7f3f88154470 2023-05-16T07:14:43.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f85ffb700 1 -- 172.21.15.92:0/3873402906 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00f9f0 con 0x7f3f88154470 2023-05-16T07:14:43.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/3873402906 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 msgr2=0x7f3f880aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:43.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/3873402906 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 0x7f3f880aaca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f3f7c009760 tx=0x7f3f7c009060).stop 2023-05-16T07:14:43.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/3873402906 shutdown_connections 2023-05-16T07:14:43.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/3873402906 >> v2:172.21.15.158:3300/0 conn(0x7f3f88068160 0x7f3f880685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/3873402906 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88154470 0x7f3f880aaca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/3873402906 >> 172.21.15.92:0/3873402906 conn(0x7f3f88067920 msgr2=0x7f3f880cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:43.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/3873402906 shutdown_connections 2023-05-16T07:14:43.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.954+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/3873402906 wait complete. 2023-05-16T07:14:43.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 Processor -- start 2023-05-16T07:14:43.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 -- start start 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88068160 0x7f3f880b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f88154090 0x7f3f881439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f88154470 0x7f3f880ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881563e0 con 0x7f3f88154090 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88155760 con 0x7f3f88154470 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.956+0000 7f3f8d5d9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88155a40 con 0x7f3f88068160 2023-05-16T07:14:43.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f88154090 0x7f3f881439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:43.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f86ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88068160 0x7f3f880b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:43.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f86ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88068160 0x7f3f880b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:36338/0 (socket says 172.21.15.92:36338) 2023-05-16T07:14:43.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f86ffd700 1 -- 172.21.15.92:0/2723218347 learned_addr learned my addr 172.21.15.92:0/2723218347 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:43.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f877fe700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f88154470 0x7f3f880ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:43.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 -- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88068160 msgr2=0x7f3f880b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:43.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88068160 0x7f3f880b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 -- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f88154470 msgr2=0x7f3f880ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:43.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f88154470 0x7f3f880ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:43.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 -- 172.21.15.92:0/2723218347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009370 con 0x7f3f88154090 2023-05-16T07:14:43.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f867fc700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f88154090 0x7f3f881439e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f3f7c014000 tx=0x7f3f7c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:43.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00f7e0 con 0x7f3f88154090 2023-05-16T07:14:43.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00f940 con 0x7f3f88154090 2023-05-16T07:14:43.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c01b660 con 0x7f3f88154090 2023-05-16T07:14:43.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c0092a0 con 0x7f3f88154090 2023-05-16T07:14:43.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.957+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88156000 con 0x7f3f88154090 2023-05-16T07:14:43.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.958+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f88066730 con 0x7f3f88154090 2023-05-16T07:14:43.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.958+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f3f7c02b070 con 0x7f3f88154090 2023-05-16T07:14:43.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.959+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7136+0+0 (secure 0 0 0) 0x7f3f7c07b510 con 0x7f3f88154090 2023-05-16T07:14:43.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:43.963+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3f7c04c760 con 0x7f3f88154090 2023-05-16T07:14:44.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.165+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3f8806c350 con 0x7f3f88154090 2023-05-16T07:14:44.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.751+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 13) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f3f7c021030 con 0x7f3f88154090 2023-05-16T07:14:44.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.752+0000 7f3f6effd700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3f74030350 0x7f3f74067c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.753+0000 7f3f86ffd700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3f74030350 0x7f3f74067c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.755+0000 7f3f86ffd700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3f74030350 0x7f3f74067c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3f78001560 tx=0x7f3f78001630).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:44.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.820+0000 7f3f6effd700 1 -- 172.21.15.92:0/2723218347 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3f7c055200 con 0x7f3f88154090 2023-05-16T07:14:44.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.822+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3f74030350 msgr2=0x7f3f74067c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.822+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3f74030350 0x7f3f74067c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3f78001560 tx=0x7f3f78001630).stop 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.822+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f88154090 msgr2=0x7f3f881439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.822+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f88154090 0x7f3f881439e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f3f7c014000 tx=0x7f3f7c017000).stop 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.822+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 shutdown_connections 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.822+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3f74030350 0x7f3f74067c30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.823+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f88068160 0x7f3f880b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.823+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f88154470 0x7f3f880ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.823+0000 7f3f8d5d9700 1 --2- 172.21.15.92:0/2723218347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f88154090 0x7f3f881439e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.823+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 >> 172.21.15.92:0/2723218347 conn(0x7f3f88067920 msgr2=0x7f3f8814fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:44.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.823+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 shutdown_connections 2023-05-16T07:14:44.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.823+0000 7f3f8d5d9700 1 -- 172.21.15.92:0/2723218347 wait complete. 2023-05-16T07:14:44.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:14:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.958+0000 7efd5e4fd700 1 Processor -- start 2023-05-16T07:14:44.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.959+0000 7efd5e4fd700 1 -- start start 2023-05-16T07:14:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.959+0000 7efd5e4fd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7efd580a4930 0x7efd58154030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.959+0000 7efd5e4fd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd58069a90 con 0x7efd580a75b0 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.959+0000 7efd5e4fd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd58069d70 con 0x7efd58153c50 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.959+0000 7efd5e4fd700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5806a050 con 0x7efd580a4930 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.959+0000 7efd5dcfc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd580a75b0 0x7efd580658a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54540/0 (socket says 172.21.15.92:54540) 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5dcfc700 1 -- 172.21.15.92:0/1487479421 learned_addr learned my addr 172.21.15.92:0/1487479421 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5ccfa700 1 --2- 172.21.15.92:0/1487479421 >> v2:172.21.15.158:3300/0 conn(0x7efd580a4930 0x7efd58154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196000499 0 0) 0x7efd58069a90 con 0x7efd580a75b0 2023-05-16T07:14:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd400033f0 con 0x7efd580a75b0 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2665486031 0 0) 0x7efd400033f0 con 0x7efd580a75b0 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 >> v2:172.21.15.158:3300/0 conn(0x7efd580a4930 msgr2=0x7efd58154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 --2- 172.21.15.92:0/1487479421 >> v2:172.21.15.158:3300/0 conn(0x7efd580a4930 0x7efd58154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 >> v1:172.21.15.158:6789/0 conn(0x7efd58153c50 legacy=0x7efd580a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd580650a0 con 0x7efd580a75b0 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7efd50001470 con 0x7efd580a75b0 2023-05-16T07:14:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 >> v1:172.21.15.92:6789/0 conn(0x7efd580a75b0 legacy=0x7efd580658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd40003930 0x7efd40003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd40004690 0x7efd40004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40005000 0x7efd4000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5806a050 con 0x7efd40003930 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4000bf30 con 0x7efd40005000 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4000c0f0 con 0x7efd40004690 2023-05-16T07:14:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd40003930 0x7efd40003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5dcfc700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40005000 0x7efd4000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5ccfa700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd40004690 0x7efd40004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5d4fb700 1 -- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd40004690 msgr2=0x7efd40004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:44.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd40004690 0x7efd40004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5d4fb700 1 -- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40005000 msgr2=0x7efd4000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:44.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40005000 0x7efd4000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.960+0000 7efd5d4fb700 1 -- 172.21.15.92:0/1487479421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd400033f0 con 0x7efd40003930 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5dcfc700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40005000 0x7efd4000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5dcfc700 1 -- 172.21.15.92:0/1487479421 reap_dead start 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd40003930 0x7efd40003d50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7efd48001e30 tx=0x7efd48007f30).ready entity=mon.0 client_cookie=5ee6ac77f8f0aaa1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48006a20 con 0x7efd40003930 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd48006b80 con 0x7efd40003930 2023-05-16T07:14:44.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd57fff700 1 -- 172.21.15.92:0/1487479421 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800c5d0 con 0x7efd40003930 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1487479421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd40003930 msgr2=0x7efd40003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd40003930 0x7efd40003d50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7efd48001e30 tx=0x7efd48007f30).stop 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1487479421 shutdown_connections 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1487479421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd40003930 0x7efd40003d50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1487479421 >> 172.21.15.92:0/1487479421 conn(0x7efd58068a30 msgr2=0x7efd580c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1487479421 shutdown_connections 2023-05-16T07:14:44.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.961+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1487479421 wait complete. 2023-05-16T07:14:44.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.962+0000 7efd5e4fd700 1 Processor -- start 2023-05-16T07:14:44.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.962+0000 7efd5e4fd700 1 -- start start 2023-05-16T07:14:44.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5e4fd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40003930 0x7efd58149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5e4fd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd580acec0 0x7efd58155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5e4fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 0x7efd58155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5e4fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd58156d70 con 0x7efd58155980 2023-05-16T07:14:44.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5e4fd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580ad730 con 0x7efd40003930 2023-05-16T07:14:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5e4fd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd581566b0 con 0x7efd580acec0 2023-05-16T07:14:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5dcfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 0x7efd58155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5dcfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 0x7efd58155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43804/0 (socket says 172.21.15.92:43804) 2023-05-16T07:14:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.963+0000 7efd5dcfc700 1 -- 172.21.15.92:0/1255659381 learned_addr learned my addr 172.21.15.92:0/1255659381 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40003930 0x7efd58149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5ccfa700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd580acec0 0x7efd58155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5dcfc700 1 -- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd580acec0 msgr2=0x7efd58155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5dcfc700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd580acec0 0x7efd58155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5dcfc700 1 -- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40003930 msgr2=0x7efd58149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5dcfc700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40003930 0x7efd58149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5dcfc700 1 -- 172.21.15.92:0/1255659381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd58156990 con 0x7efd58155980 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5ccfa700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd580acec0 0x7efd58155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5dcfc700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 0x7efd58155e00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7efd50000fd0 tx=0x7efd50001470).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50010520 con 0x7efd58155980 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50010680 con 0x7efd58155980 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd580ab990 con 0x7efd58155980 2023-05-16T07:14:44.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50010a40 con 0x7efd58155980 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.964+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd580ad340 con 0x7efd58155980 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.965+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd581564b0 con 0x7efd58155980 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.965+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 75162+0+0 (secure 0 0 0) 0x7efd50014430 con 0x7efd58155980 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.965+0000 7efd557fa700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4405ecf0 0x7efd440611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.966+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7136+0+0 (secure 0 0 0) 0x7efd50079c70 con 0x7efd58155980 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.966+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4405ecf0 0x7efd440611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.966+0000 7efd5d4fb700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4405ecf0 0x7efd440611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7efd48001390 tx=0x7efd48016000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:44.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:44.970+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7efd5004ae60 con 0x7efd58155980 2023-05-16T07:14:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.131+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd58047150 con 0x7efd58155980 2023-05-16T07:14:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.821+0000 7efd557fa700 1 -- 172.21.15.92:0/1255659381 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7efd5004e480 con 0x7efd58155980 2023-05-16T07:14:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.824+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4405ecf0 msgr2=0x7efd440611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.824+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4405ecf0 0x7efd440611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7efd48001390 tx=0x7efd48016000).stop 2023-05-16T07:14:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.824+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 msgr2=0x7efd58155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.824+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 0x7efd58155e00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7efd50000fd0 tx=0x7efd50001470).stop 2023-05-16T07:14:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 shutdown_connections 2023-05-16T07:14:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4405ecf0 0x7efd440611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd58155980 0x7efd58155e00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd40003930 0x7efd58149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 --2- 172.21.15.92:0/1255659381 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd580acec0 0x7efd58155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 >> 172.21.15.92:0/1255659381 conn(0x7efd58068a30 msgr2=0x7efd580c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 shutdown_connections 2023-05-16T07:14:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.825+0000 7efd5e4fd700 1 -- 172.21.15.92:0/1255659381 wait complete. 2023-05-16T07:14:45.831 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:14:45.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:14:45.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.955+0000 7f99f8161700 1 Processor -- start 2023-05-16T07:14:45.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.955+0000 7f99f8161700 1 -- start start 2023-05-16T07:14:45.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f8161700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 0x7f99f00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:45.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f8161700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f99f0154470 0x7f99f00a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:45.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f8161700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00aa570 con 0x7f99f00aa150 2023-05-16T07:14:45.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f8161700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00c94d0 con 0x7f99f0154470 2023-05-16T07:14:45.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f8161700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00cb7e0 con 0x7f99f0154090 2023-05-16T07:14:45.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f5efd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 0x7f99f00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:45.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f5efd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 0x7f99f00aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:36382/0 (socket says 172.21.15.92:36382) 2023-05-16T07:14:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f5efd700 1 -- 172.21.15.92:0/82817797 learned_addr learned my addr 172.21.15.92:0/82817797 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.956+0000 7f99f56fc700 1 --2- 172.21.15.92:0/82817797 >> v2:172.21.15.158:3300/0 conn(0x7f99f0154470 0x7f99f00a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f5efd700 1 -- 172.21.15.92:0/82817797 >> v2:172.21.15.158:3300/0 conn(0x7f99f0154470 msgr2=0x7f99f00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f5efd700 1 --2- 172.21.15.92:0/82817797 >> v2:172.21.15.158:3300/0 conn(0x7f99f0154470 0x7f99f00a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f5efd700 1 -- 172.21.15.92:0/82817797 >> v1:172.21.15.158:6789/0 conn(0x7f99f00aa150 legacy=0x7f99f0068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f5efd700 1 -- 172.21.15.92:0/82817797 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00c88c0 con 0x7f99f0154090 2023-05-16T07:14:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f56fc700 1 --2- 172.21.15.92:0/82817797 >> v2:172.21.15.158:3300/0 conn(0x7f99f0154470 0x7f99f00a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:45.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f5efd700 1 --2- 172.21.15.92:0/82817797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 0x7f99f00aaca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f99e0009f20 tx=0x7f99e0009840).ready entity=mon.2 client_cookie=5505b410fecc1db9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f4efb700 1 -- 172.21.15.92:0/82817797 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e000e070 con 0x7f99f0154090 2023-05-16T07:14:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f4efb700 1 -- 172.21.15.92:0/82817797 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e000b9d0 con 0x7f99f0154090 2023-05-16T07:14:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.957+0000 7f99f4efb700 1 -- 172.21.15.92:0/82817797 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e000bd50 con 0x7f99f0154090 2023-05-16T07:14:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 -- 172.21.15.92:0/82817797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 msgr2=0x7f99f00aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 --2- 172.21.15.92:0/82817797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 0x7f99f00aaca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f99e0009f20 tx=0x7f99e0009840).stop 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 -- 172.21.15.92:0/82817797 shutdown_connections 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 --2- 172.21.15.92:0/82817797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f0154090 0x7f99f00aaca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 --2- 172.21.15.92:0/82817797 >> v2:172.21.15.158:3300/0 conn(0x7f99f0154470 0x7f99f00a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 -- 172.21.15.92:0/82817797 >> 172.21.15.92:0/82817797 conn(0x7f99f0067920 msgr2=0x7f99f00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 -- 172.21.15.92:0/82817797 shutdown_connections 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.958+0000 7f99f8161700 1 -- 172.21.15.92:0/82817797 wait complete. 2023-05-16T07:14:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.959+0000 7f99f8161700 1 Processor -- start 2023-05-16T07:14:45.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.959+0000 7f99f8161700 1 -- start start 2023-05-16T07:14:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f00aa150 0x7f99f00c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99f0154090 0x7f99f013e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 0x7f99f014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0168a30 con 0x7f99f0154470 2023-05-16T07:14:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c4b10 con 0x7f99f0154090 2023-05-16T07:14:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c41e0 con 0x7f99f00aa150 2023-05-16T07:14:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 0x7f99f014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 0x7f99f014e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43814/0 (socket says 172.21.15.92:43814) 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 -- 172.21.15.92:0/2926587861 learned_addr learned my addr 172.21.15.92:0/2926587861 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f5efd700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f00aa150 0x7f99f00c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f56fc700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99f0154090 0x7f99f013e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 -- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f00aa150 msgr2=0x7f99f00c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f00aa150 0x7f99f00c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 -- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99f0154090 msgr2=0x7f99f013e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99f0154090 0x7f99f013e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 -- 172.21.15.92:0/2926587861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40092a0 con 0x7f99f0154470 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f66fe700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 0x7f99f014e360 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f99e800c440 tx=0x7f99e8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80123f0 con 0x7f99f0154470 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8001e90 con 0x7f99f0154470 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8012760 con 0x7f99f0154470 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.960+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e0009ae0 con 0x7f99f0154470 2023-05-16T07:14:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.961+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f00c47a0 con 0x7f99f0154470 2023-05-16T07:14:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.962+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0066730 con 0x7f99f0154470 2023-05-16T07:14:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.962+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f99e8010070 con 0x7f99f0154470 2023-05-16T07:14:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.962+0000 7f99edffb700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f99dc05ed90 0x7f99dc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.962+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f99e8078d50 con 0x7f99f0154470 2023-05-16T07:14:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.962+0000 7f99f5efd700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f99dc05ed90 0x7f99dc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.963+0000 7f99f5efd700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f99dc05ed90 0x7f99dc061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f99e000b9a0 tx=0x7f99e0009840).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:45.966+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f99e80478f0 con 0x7f99f0154470 2023-05-16T07:14:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.134+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99f00aa570 con 0x7f99f0154470 2023-05-16T07:14:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.829+0000 7f99edffb700 1 -- 172.21.15.92:0/2926587861 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f99f00aa570 con 0x7f99f0154470 2023-05-16T07:14:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.831+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f99dc05ed90 msgr2=0x7f99dc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.831+0000 7f99f8161700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f99dc05ed90 0x7f99dc061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f99e000b9a0 tx=0x7f99e0009840).stop 2023-05-16T07:14:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 msgr2=0x7f99f014e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 0x7f99f014e360 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f99e800c440 tx=0x7f99e8007d70).stop 2023-05-16T07:14:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 shutdown_connections 2023-05-16T07:14:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f99dc05ed90 0x7f99dc061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f99f00aa150 0x7f99f00c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f0154470 0x7f99f014e360 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 --2- 172.21.15.92:0/2926587861 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f99f0154090 0x7f99f013e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 >> 172.21.15.92:0/2926587861 conn(0x7f99f0067920 msgr2=0x7f99f00c1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 shutdown_connections 2023-05-16T07:14:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.832+0000 7f99f8161700 1 -- 172.21.15.92:0/2926587861 wait complete. 2023-05-16T07:14:46.838 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:14:46.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:14:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.962+0000 7f2010b3f700 1 Processor -- start 2023-05-16T07:14:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.962+0000 7f2010b3f700 1 -- start start 2023-05-16T07:14:46.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f2010b3f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f200c153d00 0x7f200c063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:46.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f2010b3f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f200c1540e0 0x7f200c0bf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:46.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f2010b3f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c0a7c90 con 0x7f200c0622b0 2023-05-16T07:14:46.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f2010b3f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c067f60 con 0x7f200c153d00 2023-05-16T07:14:46.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f2010b3f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c0a7f00 con 0x7f200c1540e0 2023-05-16T07:14:46.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f200a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f200c153d00 0x7f200c063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:46.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f200a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f200c153d00 0x7f200c063fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43820/0 (socket says 172.21.15.92:43820) 2023-05-16T07:14:46.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.963+0000 7f200a59c700 1 -- 172.21.15.92:0/113814356 learned_addr learned my addr 172.21.15.92:0/113814356 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:46.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200a59c700 1 -- 172.21.15.92:0/113814356 >> v2:172.21.15.158:3300/0 conn(0x7f200c1540e0 msgr2=0x7f200c0bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:46.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200a59c700 1 --2- 172.21.15.92:0/113814356 >> v2:172.21.15.158:3300/0 conn(0x7f200c1540e0 0x7f200c0bf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:46.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200a59c700 1 -- 172.21.15.92:0/113814356 >> v1:172.21.15.158:6789/0 conn(0x7f200c0622b0 legacy=0x7f200c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:46.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200a59c700 1 -- 172.21.15.92:0/113814356 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c0a81e0 con 0x7f200c153d00 2023-05-16T07:14:46.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200a59c700 1 --2- 172.21.15.92:0/113814356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c153d00 0x7f200c063fd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f200c158ca0 tx=0x7f1ffc009840).ready entity=mon.0 client_cookie=c2438b4ab7d0baca server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:46.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200959a700 1 -- 172.21.15.92:0/113814356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00e070 con 0x7f200c153d00 2023-05-16T07:14:46.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200959a700 1 -- 172.21.15.92:0/113814356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc00ba10 con 0x7f200c153d00 2023-05-16T07:14:46.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f200959a700 1 -- 172.21.15.92:0/113814356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00bd90 con 0x7f200c153d00 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 -- 172.21.15.92:0/113814356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c153d00 msgr2=0x7f200c063fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 --2- 172.21.15.92:0/113814356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c153d00 0x7f200c063fd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f200c158ca0 tx=0x7f1ffc009840).stop 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 -- 172.21.15.92:0/113814356 shutdown_connections 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 --2- 172.21.15.92:0/113814356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c153d00 0x7f200c063fd0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 --2- 172.21.15.92:0/113814356 >> v2:172.21.15.158:3300/0 conn(0x7f200c1540e0 0x7f200c0bf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 -- 172.21.15.92:0/113814356 >> 172.21.15.92:0/113814356 conn(0x7f200c05e820 msgr2=0x7f200c0c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 -- 172.21.15.92:0/113814356 shutdown_connections 2023-05-16T07:14:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.964+0000 7f2010b3f700 1 -- 172.21.15.92:0/113814356 wait complete. 2023-05-16T07:14:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.965+0000 7f2010b3f700 1 Processor -- start 2023-05-16T07:14:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 -- start start 2023-05-16T07:14:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 0x7f200c0aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f200c153d00 0x7f200c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f200c1540e0 0x7f200c14c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0aa920 con 0x7f200c0622b0 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0aabf0 con 0x7f200c153d00 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2010b3f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0aba40 con 0x7f200c1540e0 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f200a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 0x7f200c0aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f200a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 0x7f200c0aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43826/0 (socket says 172.21.15.92:43826) 2023-05-16T07:14:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f200a59c700 1 -- 172.21.15.92:0/768312974 learned_addr learned my addr 172.21.15.92:0/768312974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.966+0000 7f2009d9b700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f200c153d00 0x7f200c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200ad9d700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f200c1540e0 0x7f200c14c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200a59c700 1 -- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f200c1540e0 msgr2=0x7f200c14c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200a59c700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f200c1540e0 0x7f200c14c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200a59c700 1 -- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f200c153d00 msgr2=0x7f200c0a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200a59c700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f200c153d00 0x7f200c0a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200a59c700 1 -- 172.21.15.92:0/768312974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc009ae0 con 0x7f200c0622b0 2023-05-16T07:14:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f2009d9b700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f200c153d00 0x7f200c0a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f200a59c700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 0x7f200c0aa430 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f200c065e20 tx=0x7f1ffc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc013c60 con 0x7f200c0622b0 2023-05-16T07:14:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc013dc0 con 0x7f200c0622b0 2023-05-16T07:14:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c0abd20 con 0x7f200c0622b0 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc01d640 con 0x7f200c0622b0 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.967+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c0ac2e0 con 0x7f200c0622b0 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.968+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200c06c350 con 0x7f200c0622b0 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.968+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1ffc01b040 con 0x7f200c0622b0 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.968+0000 7f20027fc700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fec05ed60 0x7f1fec061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.968+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f1ffc07b640 con 0x7f200c0622b0 2023-05-16T07:14:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.969+0000 7f2009d9b700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fec05ed60 0x7f1fec061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.969+0000 7f2009d9b700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fec05ed60 0x7f1fec061220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1ff400b910 tx=0x7f1ff4008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:46.972+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1ffc04cb60 con 0x7f200c0622b0 2023-05-16T07:14:47.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.136+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f200c06c350 con 0x7f200c0622b0 2023-05-16T07:14:47.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.139+0000 7f20027fc700 1 -- 172.21.15.92:0/768312974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v4) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ffc01c370 con 0x7f200c0622b0 2023-05-16T07:14:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fec05ed60 msgr2=0x7f1fec061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fec05ed60 0x7f1fec061220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1ff400b910 tx=0x7f1ff4008040).stop 2023-05-16T07:14:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 msgr2=0x7f200c0aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 0x7f200c0aa430 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f200c065e20 tx=0x7f1ffc009840).stop 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 shutdown_connections 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fec05ed60 0x7f1fec061220 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f200c1540e0 0x7f200c14c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c0622b0 0x7f200c0aa430 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 --2- 172.21.15.92:0/768312974 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f200c153d00 0x7f200c0a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 >> 172.21.15.92:0/768312974 conn(0x7f200c05e820 msgr2=0x7f200c0639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 shutdown_connections 2023-05-16T07:14:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.141+0000 7f2010b3f700 1 -- 172.21.15.92:0/768312974 wait complete. 2023-05-16T07:14:47.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:14:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.267+0000 7f0dd6e99700 1 Processor -- start 2023-05-16T07:14:47.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.267+0000 7f0dd6e99700 1 -- start start 2023-05-16T07:14:47.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd6e99700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0dd0154090 0x7f0dd00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd6e99700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd0154470 0x7f0dd00a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd6e99700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00c94d0 con 0x7f0dd00aa150 2023-05-16T07:14:47.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd6e99700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd00aa5c0 con 0x7f0dd0154090 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd6e99700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00cb7e0 con 0x7f0dd0154470 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0dd0154090 0x7f0dd00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0dd0154090 0x7f0dd00aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43834/0 (socket says 172.21.15.92:43834) 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 -- 172.21.15.92:0/1524887668 learned_addr learned my addr 172.21.15.92:0/1524887668 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 -- 172.21.15.92:0/1524887668 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd0154470 msgr2=0x7f0dd00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dcffff700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd0154470 0x7f0dd00a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd0154470 0x7f0dd00a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 -- 172.21.15.92:0/1524887668 >> v1:172.21.15.158:6789/0 conn(0x7f0dd00aa150 legacy=0x7f0dd0068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 -- 172.21.15.92:0/1524887668 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd00c88c0 con 0x7f0dd0154090 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dcffff700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd0154470 0x7f0dd00a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dd4c35700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154090 0x7f0dd00aaca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f0dc8006fd0 tx=0x7f0dc80090f0).ready entity=mon.0 client_cookie=731dd7040ba8f17b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dcf7fe700 1 -- 172.21.15.92:0/1524887668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800b600 con 0x7f0dd0154090 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.268+0000 7f0dcf7fe700 1 -- 172.21.15.92:0/1524887668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc800b760 con 0x7f0dd0154090 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dcf7fe700 1 -- 172.21.15.92:0/1524887668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800bae0 con 0x7f0dd0154090 2023-05-16T07:14:47.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 -- 172.21.15.92:0/1524887668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154090 msgr2=0x7f0dd00aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154090 0x7f0dd00aaca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f0dc8006fd0 tx=0x7f0dc80090f0).stop 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 -- 172.21.15.92:0/1524887668 shutdown_connections 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154090 0x7f0dd00aaca0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/1524887668 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd0154470 0x7f0dd00a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 -- 172.21.15.92:0/1524887668 >> 172.21.15.92:0/1524887668 conn(0x7f0dd0067920 msgr2=0x7f0dd00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 -- 172.21.15.92:0/1524887668 shutdown_connections 2023-05-16T07:14:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.269+0000 7f0dd6e99700 1 -- 172.21.15.92:0/1524887668 wait complete. 2023-05-16T07:14:47.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.270+0000 7f0dd6e99700 1 Processor -- start 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.270+0000 7f0dd6e99700 1 -- start start 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd6e99700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd00aa150 0x7f0dd013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd6e99700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0dd0154090 0x7f0dd014fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd6e99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 0x7f0dd014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd6e99700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0150440 con 0x7f0dd0154470 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd6e99700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd013c400 con 0x7f0dd0154090 2023-05-16T07:14:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd6e99700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0156820 con 0x7f0dd00aa150 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 0x7f0dd014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 0x7f0dd014ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43844/0 (socket says 172.21.15.92:43844) 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 -- 172.21.15.92:0/336598577 learned_addr learned my addr 172.21.15.92:0/336598577 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dcffff700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0dd0154090 0x7f0dd014fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd4c35700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd00aa150 0x7f0dd013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 -- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd00aa150 msgr2=0x7f0dd013bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd00aa150 0x7f0dd013bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 -- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0dd0154090 msgr2=0x7f0dd014fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0dd0154090 0x7f0dd014fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.271+0000 7f0dd5436700 1 -- 172.21.15.92:0/336598577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc8009370 con 0x7f0dd0154470 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.272+0000 7f0dd5436700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 0x7f0dd014ff50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f0db8009470 tx=0x7f0db8006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.272+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8013460 con 0x7f0dd0154470 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.272+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db8013600 con 0x7f0dd0154470 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.272+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0156b00 con 0x7f0dd0154470 2023-05-16T07:14:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.272+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db80139c0 con 0x7f0dd0154470 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.272+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd01570c0 con 0x7f0dd0154470 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.273+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd0066730 con 0x7f0dd0154470 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.273+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0db801a060 con 0x7f0dd0154470 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.273+0000 7f0dccff9700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0db005ed90 0x7f0db0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.273+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f0db80791e0 con 0x7f0dd0154470 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.273+0000 7f0dd4c35700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0db005ed90 0x7f0db0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.274+0000 7f0dd4c35700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0db005ed90 0x7f0db0061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0dc8009130 tx=0x7f0dc8017000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.277+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0db804a7b0 con 0x7f0dd0154470 2023-05-16T07:14:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.443+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f0dd006c350 con 0x7f0dd0154470 2023-05-16T07:14:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.446+0000 7f0dccff9700 1 -- 172.21.15.92:0/336598577 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v5) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0db8018070 con 0x7f0dd0154470 2023-05-16T07:14:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0db005ed90 msgr2=0x7f0db0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0db005ed90 0x7f0db0061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0dc8009130 tx=0x7f0dc8017000).stop 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 msgr2=0x7f0dd014ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 0x7f0dd014ff50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f0db8009470 tx=0x7f0db8006ea0).stop 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 shutdown_connections 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0db005ed90 0x7f0db0061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0dd00aa150 0x7f0dd013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd0154470 0x7f0dd014ff50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 --2- 172.21.15.92:0/336598577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0dd0154090 0x7f0dd014fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 >> 172.21.15.92:0/336598577 conn(0x7f0dd0067920 msgr2=0x7f0dd0153050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 shutdown_connections 2023-05-16T07:14:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.448+0000 7f0dd6e99700 1 -- 172.21.15.92:0/336598577 wait complete. 2023-05-16T07:14:47.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-05-16T07:14:47.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.571+0000 7f8f4ee69700 1 Processor -- start 2023-05-16T07:14:47.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.571+0000 7f8f4ee69700 1 -- start start 2023-05-16T07:14:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f4ee69700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8f48154170 0x7f8f480c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f4ee69700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8f48154550 0x7f8f480c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f4ee69700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480c0660 con 0x7f8f480696a0 2023-05-16T07:14:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f4ee69700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480c1b00 con 0x7f8f48154550 2023-05-16T07:14:47.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f4ee69700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480bfd30 con 0x7f8f48154170 2023-05-16T07:14:47.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8f48154550 0x7f8f480c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8f48154550 0x7f8f480c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43848/0 (socket says 172.21.15.92:43848) 2023-05-16T07:14:47.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 -- 172.21.15.92:0/1633622474 learned_addr learned my addr 172.21.15.92:0/1633622474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:47.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 -- 172.21.15.92:0/1633622474 >> v2:172.21.15.158:3300/0 conn(0x7f8f48154170 msgr2=0x7f8f480c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f4cc05700 1 --2- 172.21.15.92:0/1633622474 >> v2:172.21.15.158:3300/0 conn(0x7f8f48154170 0x7f8f480c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 --2- 172.21.15.92:0/1633622474 >> v2:172.21.15.158:3300/0 conn(0x7f8f48154170 0x7f8f480c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 -- 172.21.15.92:0/1633622474 >> v1:172.21.15.158:6789/0 conn(0x7f8f480696a0 legacy=0x7f8f48157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 -- 172.21.15.92:0/1633622474 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480c0010 con 0x7f8f48154550 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f47fff700 1 --2- 172.21.15.92:0/1633622474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480c6160 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f8f48083030 tx=0x7f8f340090f0).ready entity=mon.0 client_cookie=b72821ce7ae93a41 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f477fe700 1 -- 172.21.15.92:0/1633622474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400b5a0 con 0x7f8f48154550 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f477fe700 1 -- 172.21.15.92:0/1633622474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400b700 con 0x7f8f48154550 2023-05-16T07:14:47.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.572+0000 7f8f477fe700 1 -- 172.21.15.92:0/1633622474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400baa0 con 0x7f8f48154550 2023-05-16T07:14:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 -- 172.21.15.92:0/1633622474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 msgr2=0x7f8f480c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/1633622474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480c6160 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f8f48083030 tx=0x7f8f340090f0).stop 2023-05-16T07:14:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 -- 172.21.15.92:0/1633622474 shutdown_connections 2023-05-16T07:14:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/1633622474 >> v2:172.21.15.158:3300/0 conn(0x7f8f48154170 0x7f8f480c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/1633622474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480c6160 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 -- 172.21.15.92:0/1633622474 >> 172.21.15.92:0/1633622474 conn(0x7f8f480cb700 msgr2=0x7f8f480c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 -- 172.21.15.92:0/1633622474 shutdown_connections 2023-05-16T07:14:47.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.573+0000 7f8f4ee69700 1 -- 172.21.15.92:0/1633622474 wait complete. 2023-05-16T07:14:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.574+0000 7f8f4ee69700 1 Processor -- start 2023-05-16T07:14:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.574+0000 7f8f4ee69700 1 -- start start 2023-05-16T07:14:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4ee69700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f480696a0 0x7f8f4814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4ee69700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f48154170 0x7f8f480af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4ee69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4ee69700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480afaf0 con 0x7f8f48154550 2023-05-16T07:14:47.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4ee69700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48168080 con 0x7f8f480696a0 2023-05-16T07:14:47.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4ee69700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48168360 con 0x7f8f48154170 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43856/0 (socket says 172.21.15.92:43856) 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 -- 172.21.15.92:0/2744882169 learned_addr learned my addr 172.21.15.92:0/2744882169 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4cc05700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f480696a0 0x7f8f4814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f47fff700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f48154170 0x7f8f480af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 -- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f48154170 msgr2=0x7f8f480af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f48154170 0x7f8f480af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 -- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f480696a0 msgr2=0x7f8f4814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f480696a0 0x7f8f4814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 -- 172.21.15.92:0/2744882169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009370 con 0x7f8f48154550 2023-05-16T07:14:47.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4cc05700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f480696a0 0x7f8f4814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:47.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.575+0000 7f8f4d406700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480af600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f8f4000b7e0 tx=0x7f8f40009310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.576+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000d620 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.576+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000d7c0 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.576+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f48168640 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.576+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000db80 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.576+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f48168c00 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.577+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4814fc70 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.577+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8f4000f480 con 0x7f8f48154550 2023-05-16T07:14:47.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.577+0000 7f8f44ff9700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f2c05ed60 0x7f8f2c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.577+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f8f40079f30 con 0x7f8f48154550 2023-05-16T07:14:47.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.577+0000 7f8f4cc05700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f2c05ed60 0x7f8f2c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.578+0000 7f8f4cc05700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f2c05ed60 0x7f8f2c061220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8f3c00aa10 tx=0x7f8f3c008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.581+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8f4814fc70 con 0x7f8f48154550 2023-05-16T07:14:47.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.809+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f8f4814fc70 con 0x7f8f48154550 2023-05-16T07:14:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.831+0000 7f8f44ff9700 1 -- 172.21.15.92:0/2744882169 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v435) v1 ==== 158+0+0 (secure 0 0 0) 0x7f8f4814fc70 con 0x7f8f48154550 2023-05-16T07:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.832+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f2c05ed60 msgr2=0x7f8f2c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.832+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f2c05ed60 0x7f8f2c061220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8f3c00aa10 tx=0x7f8f3c008040).stop 2023-05-16T07:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.832+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 msgr2=0x7f8f480af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.832+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480af600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f8f4000b7e0 tx=0x7f8f40009310).stop 2023-05-16T07:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 shutdown_connections 2023-05-16T07:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f2c05ed60 0x7f8f2c061220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f48154550 0x7f8f480af600 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f480696a0 0x7f8f4814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 --2- 172.21.15.92:0/2744882169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f48154170 0x7f8f480af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 >> 172.21.15.92:0/2744882169 conn(0x7f8f480cb700 msgr2=0x7f8f48153460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 shutdown_connections 2023-05-16T07:14:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.833+0000 7f8f4ee69700 1 -- 172.21.15.92:0/2744882169 wait complete. 2023-05-16T07:14:47.848 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-05-16T07:14:47.849 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-05-16T07:14:47.849 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:14:47.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-05-16T07:14:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.956+0000 7fb234050700 1 Processor -- start 2023-05-16T07:14:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.956+0000 7fb234050700 1 -- start start 2023-05-16T07:14:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb234050700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb22c069db0 0x7fb22c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb234050700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c0c0660 con 0x7fb22c154550 2023-05-16T07:14:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb234050700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c0c1b00 con 0x7fb22c154170 2023-05-16T07:14:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb234050700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22c0bfd30 con 0x7fb22c069db0 2023-05-16T07:14:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb2315eb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb22c154550 0x7fb22c0696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54612/0 (socket says 172.21.15.92:54612) 2023-05-16T07:14:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb2315eb700 1 -- 172.21.15.92:0/3924388171 learned_addr learned my addr 172.21.15.92:0/3924388171 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb2325ed700 1 --2- 172.21.15.92:0/3924388171 >> v2:172.21.15.158:3300/0 conn(0x7fb22c069db0 0x7fb22c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528771078 0 0) 0x7fb22c0c0660 con 0x7fb22c154550 2023-05-16T07:14:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2180033f0 con 0x7fb22c154550 2023-05-16T07:14:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098307206 0 0) 0x7fb2180033f0 con 0x7fb22c154550 2023-05-16T07:14:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 >> v2:172.21.15.158:3300/0 conn(0x7fb22c069db0 msgr2=0x7fb22c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 --2- 172.21.15.92:0/3924388171 >> v2:172.21.15.158:3300/0 conn(0x7fb22c069db0 0x7fb22c15f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 >> v1:172.21.15.158:6789/0 conn(0x7fb22c154170 legacy=0x7fb22c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0c0010 con 0x7fb22c154550 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fb21c001470 con 0x7fb22c154550 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 >> v1:172.21.15.92:6789/0 conn(0x7fb22c154550 legacy=0x7fb22c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2180038e0 0x7fb218003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.957+0000 7fb230dea700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218004690 0x7fb218004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218005050 0x7fb21800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0bfd30 con 0x7fb218004690 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21800bfe0 con 0x7fb218005050 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21800c1a0 con 0x7fb2180038e0 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218004690 0x7fb218004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb231dec700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2180038e0 0x7fb218003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2325ed700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218005050 0x7fb21800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 -- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2180038e0 msgr2=0x7fb218003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb2180038e0 0x7fb218003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 -- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218005050 msgr2=0x7fb21800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218005050 0x7fb21800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2325ed700 1 -- 172.21.15.92:0/3924388171 reap_dead start 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 -- 172.21.15.92:0/3924388171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180033f0 con 0x7fb218004690 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb2315eb700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218004690 0x7fb218004b10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fb22c154930 tx=0x7fb21c001e50).ready entity=mon.0 client_cookie=c5fd1d04c1070c60 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c007a20 con 0x7fb218004690 2023-05-16T07:14:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c00ca20 con 0x7fb218004690 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.958+0000 7fb230dea700 1 -- 172.21.15.92:0/3924388171 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00cdc0 con 0x7fb218004690 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 -- 172.21.15.92:0/3924388171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218004690 msgr2=0x7fb218004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218004690 0x7fb218004b10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fb22c154930 tx=0x7fb21c001e50).stop 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 -- 172.21.15.92:0/3924388171 shutdown_connections 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 --2- 172.21.15.92:0/3924388171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218004690 0x7fb218004b10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 -- 172.21.15.92:0/3924388171 >> 172.21.15.92:0/3924388171 conn(0x7fb22c0cb700 msgr2=0x7fb22c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 -- 172.21.15.92:0/3924388171 shutdown_connections 2023-05-16T07:14:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.959+0000 7fb234050700 1 -- 172.21.15.92:0/3924388171 wait complete. 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.960+0000 7fb234050700 1 Processor -- start 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.960+0000 7fb234050700 1 -- start start 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218004690 0x7fb22c1429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb22c155850 0x7fb22c156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 0x7fb22c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c156bc0 con 0x7fb22c156740 2023-05-16T07:14:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c156dc0 con 0x7fb218004690 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1553c0 con 0x7fb22c155850 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 0x7fb22c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 0x7fb22c0c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43878/0 (socket says 172.21.15.92:43878) 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 -- 172.21.15.92:0/2838951004 learned_addr learned my addr 172.21.15.92:0/2838951004 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb231dec700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218004690 0x7fb22c1429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2315eb700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb22c155850 0x7fb22c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 -- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb22c155850 msgr2=0x7fb22c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb22c155850 0x7fb22c156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 -- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218004690 msgr2=0x7fb22c1429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218004690 0x7fb22c1429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 -- 172.21.15.92:0/2838951004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c142eb0 con 0x7fb22c156740 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb2325ed700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 0x7fb22c0c5a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fb22c083aa0 tx=0x7fb228008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228015440 con 0x7fb22c156740 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2280155a0 con 0x7fb22c156740 2023-05-16T07:14:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c158670 con 0x7fb22c156740 2023-05-16T07:14:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228015940 con 0x7fb22c156740 2023-05-16T07:14:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.961+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c154d70 con 0x7fb22c156740 2023-05-16T07:14:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.962+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c0666c0 con 0x7fb22c156740 2023-05-16T07:14:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.963+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb22800f070 con 0x7fb22c156740 2023-05-16T07:14:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.963+0000 7fb221ffb700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb21005ed90 0x7fb210061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.963+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fb228079c80 con 0x7fb22c156740 2023-05-16T07:14:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.963+0000 7fb231dec700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb21005ed90 0x7fb210061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.964+0000 7fb231dec700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb21005ed90 0x7fb210061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb224004290 tx=0x7fb224009040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:47.967+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb22804b1a0 con 0x7fb22c156740 2023-05-16T07:14:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.194+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fb22c155530 con 0x7fb22c156740 2023-05-16T07:14:48.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.832+0000 7fb221ffb700 1 -- 172.21.15.92:0/2838951004 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v436) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb22804e7c0 con 0x7fb22c156740 2023-05-16T07:14:48.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb21005ed90 msgr2=0x7fb210061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:48.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb21005ed90 0x7fb210061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb224004290 tx=0x7fb224009040).stop 2023-05-16T07:14:48.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 msgr2=0x7fb22c0c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 0x7fb22c0c5a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fb22c083aa0 tx=0x7fb228008040).stop 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 shutdown_connections 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb21005ed90 0x7fb210061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c156740 0x7fb22c0c5a10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb218004690 0x7fb22c1429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 --2- 172.21.15.92:0/2838951004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb22c155850 0x7fb22c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 >> 172.21.15.92:0/2838951004 conn(0x7fb22c0cb700 msgr2=0x7fb22c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 shutdown_connections 2023-05-16T07:14:48.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.834+0000 7fb234050700 1 -- 172.21.15.92:0/2838951004 wait complete. 2023-05-16T07:14:48.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:14:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.968+0000 7fa9c8a8f700 1 Processor -- start 2023-05-16T07:14:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.968+0000 7fa9c8a8f700 1 -- start start 2023-05-16T07:14:48.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.969+0000 7fa9c8a8f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9c414e6f0 0x7fa9c4155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:48.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.969+0000 7fa9c8a8f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa9c40c4d40 0x7fa9c40622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:48.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.969+0000 7fa9c8a8f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c414ead0 con 0x7fa9c40627a0 2023-05-16T07:14:48.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.969+0000 7fa9c8a8f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c4154cb0 con 0x7fa9c414e6f0 2023-05-16T07:14:48.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.969+0000 7fa9c8a8f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c40c6020 con 0x7fa9c40c4d40 2023-05-16T07:14:48.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9c414e6f0 0x7fa9c4155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:48.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c2d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa9c40627a0 0x7fa9c4063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54632/0 (socket says 172.21.15.92:54632) 2023-05-16T07:14:48.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c2d9d700 1 -- 172.21.15.92:0/8640794 learned_addr learned my addr 172.21.15.92:0/8640794 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:48.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c1d9b700 1 --2- 172.21.15.92:0/8640794 >> v2:172.21.15.158:3300/0 conn(0x7fa9c40c4d40 0x7fa9c40622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:48.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c259c700 1 -- 172.21.15.92:0/8640794 >> v2:172.21.15.158:3300/0 conn(0x7fa9c40c4d40 msgr2=0x7fa9c40622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:48.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c259c700 1 --2- 172.21.15.92:0/8640794 >> v2:172.21.15.158:3300/0 conn(0x7fa9c40c4d40 0x7fa9c40622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c259c700 1 -- 172.21.15.92:0/8640794 >> v1:172.21.15.92:6789/0 conn(0x7fa9c40627a0 legacy=0x7fa9c4063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:48.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c259c700 1 -- 172.21.15.92:0/8640794 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c4154660 con 0x7fa9c414e6f0 2023-05-16T07:14:48.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c159a700 1 -- 172.21.15.92:0/8640794 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777779262 0 0) 0x7fa9c414ead0 con 0x7fa9c40627a0 2023-05-16T07:14:48.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c259c700 1 --2- 172.21.15.92:0/8640794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4155140 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fa9c40660c0 tx=0x7fa9b4009780).ready entity=mon.0 client_cookie=9a8fd84ec9475dec server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:48.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c159a700 1 -- 172.21.15.92:0/8640794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400e070 con 0x7fa9c414e6f0 2023-05-16T07:14:48.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c159a700 1 -- 172.21.15.92:0/8640794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400bc30 con 0x7fa9c414e6f0 2023-05-16T07:14:48.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c159a700 1 -- 172.21.15.92:0/8640794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4012410 con 0x7fa9c414e6f0 2023-05-16T07:14:48.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/8640794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 msgr2=0x7fa9c4155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:48.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.970+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/8640794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4155140 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fa9c40660c0 tx=0x7fa9b4009780).stop 2023-05-16T07:14:48.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.971+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/8640794 shutdown_connections 2023-05-16T07:14:48.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.971+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/8640794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4155140 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.971+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/8640794 >> v2:172.21.15.158:3300/0 conn(0x7fa9c40c4d40 0x7fa9c40622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.971+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/8640794 >> 172.21.15.92:0/8640794 conn(0x7fa9c405e820 msgr2=0x7fa9c40c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:48.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.971+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/8640794 shutdown_connections 2023-05-16T07:14:48.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.971+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/8640794 wait complete. 2023-05-16T07:14:48.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 Processor -- start 2023-05-16T07:14:48.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 -- start start 2023-05-16T07:14:48.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9c40627a0 0x7fa9c40abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:48.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9c40c4d40 0x7fa9c41504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:48.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:48.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4150dc0 con 0x7fa9c414e6f0 2023-05-16T07:14:48.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41510a0 con 0x7fa9c40c4d40 2023-05-16T07:14:48.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c8a8f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4151380 con 0x7fa9c40627a0 2023-05-16T07:14:48.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.972+0000 7fa9c2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:48.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4167a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43890/0 (socket says 172.21.15.92:43890) 2023-05-16T07:14:48.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 -- 172.21.15.92:0/3008107007 learned_addr learned my addr 172.21.15.92:0/3008107007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:48.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c1d9b700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9c40c4d40 0x7fa9c41504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:48.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c259c700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9c40627a0 0x7fa9c40abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:48.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 -- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9c40627a0 msgr2=0x7fa9c40abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:48.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9c40627a0 0x7fa9c40abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 -- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9c40c4d40 msgr2=0x7fa9c41504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:48.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9c40c4d40 0x7fa9c41504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:48.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 -- 172.21.15.92:0/3008107007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4009a20 con 0x7fa9c414e6f0 2023-05-16T07:14:48.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c259c700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9c40627a0 0x7fa9c40abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:48.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c2d9d700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4167a80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fa9bc00c490 tx=0x7fa9bc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:48.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0072e0 con 0x7fa9c414e6f0 2023-05-16T07:14:48.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc007440 con 0x7fa9c414e6f0 2023-05-16T07:14:48.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c40a8ed0 con 0x7fa9c414e6f0 2023-05-16T07:14:48.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0057a0 con 0x7fa9c414e6f0 2023-05-16T07:14:48.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.973+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c413ee00 con 0x7fa9c414e6f0 2023-05-16T07:14:48.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.974+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c40a93f0 con 0x7fa9c414e6f0 2023-05-16T07:14:48.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.974+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa9bc010070 con 0x7fa9c414e6f0 2023-05-16T07:14:48.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.975+0000 7fa9ba7fc700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9a405ed90 0x7fa9a4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:48.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.975+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fa9bc079190 con 0x7fa9c414e6f0 2023-05-16T07:14:48.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.975+0000 7fa9c259c700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9a405ed90 0x7fa9a4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:48.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.975+0000 7fa9c259c700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9a405ed90 0x7fa9a4061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa9b4009f90 tx=0x7fa9b4009780).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:48.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:48.979+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa9c40a93f0 con 0x7fa9c414e6f0 2023-05-16T07:14:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.141+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa9c40a93f0 con 0x7fa9c414e6f0 2023-05-16T07:14:49.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.147+0000 7fa9ba7fc700 1 -- 172.21.15.92:0/3008107007 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9c40a93f0 con 0x7fa9c414e6f0 2023-05-16T07:14:49.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9a405ed90 msgr2=0x7fa9a4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9a405ed90 0x7fa9a4061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa9b4009f90 tx=0x7fa9b4009780).stop 2023-05-16T07:14:49.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 msgr2=0x7fa9c4167a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4167a80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fa9bc00c490 tx=0x7fa9bc007c10).stop 2023-05-16T07:14:49.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 shutdown_connections 2023-05-16T07:14:49.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9a405ed90 0x7fa9a4061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9c40627a0 0x7fa9c40abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c414e6f0 0x7fa9c4167a80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.148+0000 7fa9c8a8f700 1 --2- 172.21.15.92:0/3008107007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9c40c4d40 0x7fa9c41504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.149+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 >> 172.21.15.92:0/3008107007 conn(0x7fa9c405e820 msgr2=0x7fa9c40c80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:49.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.149+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 shutdown_connections 2023-05-16T07:14:49.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.149+0000 7fa9c8a8f700 1 -- 172.21.15.92:0/3008107007 wait complete. 2023-05-16T07:14:49.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:14:49.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:14:49.191 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:14:49.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:14:49.223 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:14:49.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.337+0000 7f2069a34700 1 Processor -- start 2023-05-16T07:14:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.337+0000 7f2069a34700 1 -- start start 2023-05-16T07:14:49.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069a34700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2064150a40 0x7f2064150e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069a34700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f20640a7a10 0x7f20640a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069a34700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20640657e0 0x7f2064065c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069a34700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2064066180 con 0x7f20640657e0 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069a34700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2064065350 con 0x7f20640a7a10 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069a34700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20640a8320 con 0x7f2064150a40 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20640657e0 0x7f2064065c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20640657e0 0x7f2064065c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43896/0 (socket says 172.21.15.92:43896) 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 -- 172.21.15.92:0/4282911617 learned_addr learned my addr 172.21.15.92:0/4282911617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2063fff700 1 --2- 172.21.15.92:0/4282911617 >> v2:172.21.15.158:3300/0 conn(0x7f20640a7a10 0x7f20640a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2068a32700 1 --2- 172.21.15.92:0/4282911617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2064150a40 0x7f2064150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 -- 172.21.15.92:0/4282911617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2064150a40 msgr2=0x7f2064150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 --2- 172.21.15.92:0/4282911617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2064150a40 0x7f2064150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 -- 172.21.15.92:0/4282911617 >> v2:172.21.15.158:3300/0 conn(0x7f20640a7a10 msgr2=0x7f20640a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:49.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 --2- 172.21.15.92:0/4282911617 >> v2:172.21.15.158:3300/0 conn(0x7f20640a7a10 0x7f20640a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 -- 172.21.15.92:0/4282911617 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20640c2e00 con 0x7f20640657e0 2023-05-16T07:14:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2063fff700 1 --2- 172.21.15.92:0/4282911617 >> v2:172.21.15.158:3300/0 conn(0x7f20640a7a10 0x7f20640a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f2069233700 1 --2- 172.21.15.92:0/4282911617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20640657e0 0x7f2064065c90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f2064069450 tx=0x7f204c00b3b0).ready entity=mon.0 client_cookie=cf0b9b4ec9d9c3f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.338+0000 7f20637fe700 1 -- 172.21.15.92:0/4282911617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c00db10 con 0x7f20640657e0 2023-05-16T07:14:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f20637fe700 1 -- 172.21.15.92:0/4282911617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c00dc70 con 0x7f20640657e0 2023-05-16T07:14:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f20637fe700 1 -- 172.21.15.92:0/4282911617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c010470 con 0x7f20640657e0 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 -- 172.21.15.92:0/4282911617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20640657e0 msgr2=0x7f2064065c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 --2- 172.21.15.92:0/4282911617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20640657e0 0x7f2064065c90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f2064069450 tx=0x7f204c00b3b0).stop 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 -- 172.21.15.92:0/4282911617 shutdown_connections 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 --2- 172.21.15.92:0/4282911617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20640657e0 0x7f2064065c90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 --2- 172.21.15.92:0/4282911617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2064150a40 0x7f2064150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 --2- 172.21.15.92:0/4282911617 >> v2:172.21.15.158:3300/0 conn(0x7f20640a7a10 0x7f20640a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 -- 172.21.15.92:0/4282911617 >> 172.21.15.92:0/4282911617 conn(0x7f2064067f60 msgr2=0x7f2064068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:49.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 -- 172.21.15.92:0/4282911617 shutdown_connections 2023-05-16T07:14:49.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.339+0000 7f2069a34700 1 -- 172.21.15.92:0/4282911617 wait complete. 2023-05-16T07:14:49.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.340+0000 7f2069a34700 1 Processor -- start 2023-05-16T07:14:49.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.340+0000 7f2069a34700 1 -- start start 2023-05-16T07:14:49.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069a34700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20640657e0 0x7f20640aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069a34700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20640a7a10 0x7f206414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069a34700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 0x7f206414ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069a34700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206415fc20 con 0x7f2064150a40 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069a34700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064160a40 con 0x7f20640657e0 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069a34700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206415fe30 con 0x7f20640a7a10 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 0x7f206414ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 0x7f206414ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43902/0 (socket says 172.21.15.92:43902) 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 -- 172.21.15.92:0/2687131632 learned_addr learned my addr 172.21.15.92:0/2687131632 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2068a32700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20640657e0 0x7f20640aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2063fff700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20640a7a10 0x7f206414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 -- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20640a7a10 msgr2=0x7f206414e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20640a7a10 0x7f206414e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 -- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20640657e0 msgr2=0x7f20640aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20640657e0 0x7f20640aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.341+0000 7f2069233700 1 -- 172.21.15.92:0/2687131632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c00b610 con 0x7f2064150a40 2023-05-16T07:14:49.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.342+0000 7f2069233700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 0x7f206414ebd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f20640694b0 tx=0x7f204c00b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.342+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c020070 con 0x7f2064150a40 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.342+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c01b770 con 0x7f2064150a40 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.342+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2064160110 con 0x7f2064150a40 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.342+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c01bb30 con 0x7f2064150a40 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.342+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641606d0 con 0x7f2064150a40 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.343+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2064047150 con 0x7f2064150a40 2023-05-16T07:14:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.343+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f204c02c070 con 0x7f2064150a40 2023-05-16T07:14:49.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.343+0000 7f2060ff9700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f204405ed90 0x7f2044061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.343+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f204c07d770 con 0x7f2064150a40 2023-05-16T07:14:49.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.343+0000 7f2068a32700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f204405ed90 0x7f2044061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.344+0000 7f2068a32700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f204405ed90 0x7f2044061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f205c006fd0 tx=0x7f205c008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:49.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.347+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f204c04ec90 con 0x7f2064150a40 2023-05-16T07:14:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.548+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20640c30c0 con 0x7f2064150a40 2023-05-16T07:14:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.549+0000 7f2060ff9700 1 -- 172.21.15.92:0/2687131632 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f204c02a020 con 0x7f2064150a40 2023-05-16T07:14:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:49.553 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1188119761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1188119761},{"type":"v1","addr":"172.21.15.158:6835","nonce":1188119761}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/132699068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":132699068},{"type":"v1","addr":"172.21.15.158:6837","nonce":132699068}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2039113441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2039113441},{"type":"v1","addr":"172.21.15.92:6834","nonce":2039113441}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3951396247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3951396247},{"type":"v1","addr":"172.21.15.92:6835","nonce":3951396247}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[]} 2023-05-16T07:14:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f204405ed90 msgr2=0x7f2044061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f204405ed90 0x7f2044061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f205c006fd0 tx=0x7f205c008040).stop 2023-05-16T07:14:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 msgr2=0x7f206414ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 0x7f206414ebd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f20640694b0 tx=0x7f204c00b3b0).stop 2023-05-16T07:14:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 shutdown_connections 2023-05-16T07:14:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f204405ed90 0x7f2044061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064150a40 0x7f206414ebd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20640657e0 0x7f20640aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 --2- 172.21.15.92:0/2687131632 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20640a7a10 0x7f206414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 >> 172.21.15.92:0/2687131632 conn(0x7f2064067f60 msgr2=0x7f20640c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 shutdown_connections 2023-05-16T07:14:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.551+0000 7f2069a34700 1 -- 172.21.15.92:0/2687131632 wait complete. 2023-05-16T07:14:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 14 2023-05-16T07:14:49.567 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:14:49.567 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:14:49.568 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:14:49.568 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:14:49.568 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:14:49.568 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:14:49.569 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:14:49.569 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:14:49.569 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:14:49.569 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:14:49.569 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:14:49.570 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:14:49.570 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:14:49.570 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:14:49.570 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:14:49.571 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:14:49.576 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:14:49.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:14:49.577 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:14:49.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:14:49.583 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:14:49.583 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:14:49.584 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:14:49.585 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:14:49.620 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:14:49.620 DEBUG:teuthology.parallel:result is None 2023-05-16T07:14:49.621 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:14:49.621 DEBUG:teuthology.parallel:result is None 2023-05-16T07:14:49.625 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:14:49.626 DEBUG:teuthology.parallel:result is None 2023-05-16T07:14:49.626 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:14:49.627 DEBUG:teuthology.parallel:result is None 2023-05-16T07:14:49.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:14:49.691 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:14:49.691 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:14:49.708 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:14:49.709 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:14:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.749+0000 7f868b2a7700 1 Processor -- start 2023-05-16T07:14:49.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.749+0000 7f868b2a7700 1 -- start start 2023-05-16T07:14:49.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f868b2a7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8684154170 0x7f86840c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f868b2a7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8684154550 0x7f86840c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f868b2a7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840c0660 con 0x7f86840696a0 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f868b2a7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840c1b00 con 0x7f8684154170 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f868b2a7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840bfd30 con 0x7f8684154550 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8684154170 0x7f86840c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8684154170 0x7f86840c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43944/0 (socket says 172.21.15.92:43944) 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 -- 172.21.15.92:0/2994644025 learned_addr learned my addr 172.21.15.92:0/2994644025 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8688842700 1 --2- 172.21.15.92:0/2994644025 >> v2:172.21.15.158:3300/0 conn(0x7f8684154550 0x7f86840c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 -- 172.21.15.92:0/2994644025 >> v2:172.21.15.158:3300/0 conn(0x7f8684154550 msgr2=0x7f86840c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 --2- 172.21.15.92:0/2994644025 >> v2:172.21.15.158:3300/0 conn(0x7f8684154550 0x7f86840c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 -- 172.21.15.92:0/2994644025 >> v1:172.21.15.158:6789/0 conn(0x7f86840696a0 legacy=0x7f8684157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.750+0000 7f8689043700 1 -- 172.21.15.92:0/2994644025 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840c0010 con 0x7f8684154170 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f8689043700 1 --2- 172.21.15.92:0/2994644025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154170 0x7f86840c5c70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f86840844e0 tx=0x7f867c0090f0).ready entity=mon.0 client_cookie=d42d797c1a52fbf9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f867bfff700 1 -- 172.21.15.92:0/2994644025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00b5a0 con 0x7f8684154170 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f867bfff700 1 -- 172.21.15.92:0/2994644025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b700 con 0x7f8684154170 2023-05-16T07:14:49.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f867bfff700 1 -- 172.21.15.92:0/2994644025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00baa0 con 0x7f8684154170 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 -- 172.21.15.92:0/2994644025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154170 msgr2=0x7f86840c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 --2- 172.21.15.92:0/2994644025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154170 0x7f86840c5c70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f86840844e0 tx=0x7f867c0090f0).stop 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 -- 172.21.15.92:0/2994644025 shutdown_connections 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 --2- 172.21.15.92:0/2994644025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154170 0x7f86840c5c70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 --2- 172.21.15.92:0/2994644025 >> v2:172.21.15.158:3300/0 conn(0x7f8684154550 0x7f86840c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 -- 172.21.15.92:0/2994644025 >> 172.21.15.92:0/2994644025 conn(0x7f86840cb700 msgr2=0x7f86840c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 -- 172.21.15.92:0/2994644025 shutdown_connections 2023-05-16T07:14:49.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.751+0000 7f868b2a7700 1 -- 172.21.15.92:0/2994644025 wait complete. 2023-05-16T07:14:49.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.752+0000 7f868b2a7700 1 Processor -- start 2023-05-16T07:14:49.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.752+0000 7f868b2a7700 1 -- start start 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f868b2a7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86840696a0 0x7f8684151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f868b2a7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8684154170 0x7f86840c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f868b2a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 0x7f8684167660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f868b2a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c1de0 con 0x7f8684154550 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f868b2a7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684151f30 con 0x7f8684154170 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f868b2a7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c2080 con 0x7f86840696a0 2023-05-16T07:14:49.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f8689844700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 0x7f8684167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f8689844700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 0x7f8684167660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43954/0 (socket says 172.21.15.92:43954) 2023-05-16T07:14:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f8689844700 1 -- 172.21.15.92:0/3990766570 learned_addr learned my addr 172.21.15.92:0/3990766570 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f8688842700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8684154170 0x7f86840c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.753+0000 7f8689043700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86840696a0 0x7f8684151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f8689844700 1 -- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86840696a0 msgr2=0x7f8684151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f8689844700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86840696a0 0x7f8684151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f8689844700 1 -- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8684154170 msgr2=0x7f86840c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f8689844700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8684154170 0x7f86840c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f8689844700 1 -- 172.21.15.92:0/3990766570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009370 con 0x7f8684154550 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f8689844700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 0x7f8684167660 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f868000b7e0 tx=0x7f8680009310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680001500 con 0x7f8684154550 2023-05-16T07:14:49.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86800016a0 con 0x7f8684154550 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680001a60 con 0x7f8684154550 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840c2360 con 0x7f8684154550 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.754+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8684167b50 con 0x7f8684154550 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.755+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840c6d40 con 0x7f8684154550 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.755+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8680018030 con 0x7f8684154550 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.756+0000 7f86797fa700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f866805ed90 0x7f8668061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:49.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.756+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f8680079340 con 0x7f8684154550 2023-05-16T07:14:49.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.756+0000 7f8689043700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f866805ed90 0x7f8668061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:49.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.756+0000 7f8689043700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f866805ed90 0x7f8668061250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f867c009130 tx=0x7f867c015000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:49.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.760+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f86800478f0 con 0x7f8684154550 2023-05-16T07:14:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:49.938+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8684158dd0 con 0x7f8684154550 2023-05-16T07:14:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.839+0000 7f86797fa700 1 -- 172.21.15.92:0/3990766570 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f868004a910 con 0x7f8684154550 2023-05-16T07:14:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f866805ed90 msgr2=0x7f8668061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f866805ed90 0x7f8668061250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f867c009130 tx=0x7f867c015000).stop 2023-05-16T07:14:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 msgr2=0x7f8684167660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 0x7f8684167660 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f868000b7e0 tx=0x7f8680009310).stop 2023-05-16T07:14:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 shutdown_connections 2023-05-16T07:14:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f866805ed90 0x7f8668061250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86840696a0 0x7f8684151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8684154550 0x7f8684167660 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 --2- 172.21.15.92:0/3990766570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8684154170 0x7f86840c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 >> 172.21.15.92:0/3990766570 conn(0x7f86840cb700 msgr2=0x7f86840a9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 shutdown_connections 2023-05-16T07:14:50.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.842+0000 7f868b2a7700 1 -- 172.21.15.92:0/3990766570 wait complete. 2023-05-16T07:14:50.848 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:14:50.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:14:50.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.973+0000 7f64f2b70700 1 Processor -- start 2023-05-16T07:14:50.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.973+0000 7f64f2b70700 1 -- start start 2023-05-16T07:14:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f2b70700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f64ec154e50 0x7f64ec0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f2b70700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64ec0ad040 0x7f64ec14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f2b70700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec158970 con 0x7f64ec0c7380 2023-05-16T07:14:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f2b70700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec155300 con 0x7f64ec0ad040 2023-05-16T07:14:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f2b70700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec157d60 con 0x7f64ec154e50 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64ec0ad040 0x7f64ec14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64ec0ad040 0x7f64ec14dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43966/0 (socket says 172.21.15.92:43966) 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 -- 172.21.15.92:0/350743963 learned_addr learned my addr 172.21.15.92:0/350743963 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 -- 172.21.15.92:0/350743963 >> v2:172.21.15.158:3300/0 conn(0x7f64ec154e50 msgr2=0x7f64ec0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 --2- 172.21.15.92:0/350743963 >> v2:172.21.15.158:3300/0 conn(0x7f64ec154e50 0x7f64ec0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 -- 172.21.15.92:0/350743963 >> v1:172.21.15.158:6789/0 conn(0x7f64ec0c7380 legacy=0x7f64ec0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:50.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 -- 172.21.15.92:0/350743963 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec158040 con 0x7f64ec0ad040 2023-05-16T07:14:50.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.974+0000 7f64f236f700 1 --2- 172.21.15.92:0/350743963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec14dd70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f64ec084350 tx=0x7f64e4009840).ready entity=mon.0 client_cookie=98bac768ba0ac78f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:50.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f0b6c700 1 -- 172.21.15.92:0/350743963 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400e070 con 0x7f64ec0ad040 2023-05-16T07:14:50.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f0b6c700 1 -- 172.21.15.92:0/350743963 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400ba10 con 0x7f64ec0ad040 2023-05-16T07:14:50.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f0b6c700 1 -- 172.21.15.92:0/350743963 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400bd90 con 0x7f64ec0ad040 2023-05-16T07:14:50.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 -- 172.21.15.92:0/350743963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 msgr2=0x7f64ec14dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:50.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 --2- 172.21.15.92:0/350743963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec14dd70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f64ec084350 tx=0x7f64e4009840).stop 2023-05-16T07:14:50.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 -- 172.21.15.92:0/350743963 shutdown_connections 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 --2- 172.21.15.92:0/350743963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec14dd70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 --2- 172.21.15.92:0/350743963 >> v2:172.21.15.158:3300/0 conn(0x7f64ec154e50 0x7f64ec0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 -- 172.21.15.92:0/350743963 >> 172.21.15.92:0/350743963 conn(0x7f64ec068330 msgr2=0x7f64ec068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 -- 172.21.15.92:0/350743963 shutdown_connections 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.975+0000 7f64f2b70700 1 -- 172.21.15.92:0/350743963 wait complete. 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.976+0000 7f64f2b70700 1 Processor -- start 2023-05-16T07:14:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.976+0000 7f64f2b70700 1 -- start start 2023-05-16T07:14:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f2b70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec0c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f2b70700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64ec0c7380 0x7f64ec0c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f2b70700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ec154e50 0x7f64ec0c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f2b70700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0c25d0 con 0x7f64ec0ad040 2023-05-16T07:14:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f2b70700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec166190 con 0x7f64ec154e50 2023-05-16T07:14:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f2b70700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0c4a50 con 0x7f64ec0c7380 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec0c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec0c1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43968/0 (socket says 172.21.15.92:43968) 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 -- 172.21.15.92:0/842997292 learned_addr learned my addr 172.21.15.92:0/842997292 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 -- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64ec0c7380 msgr2=0x7f64ec0c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64ec0c7380 0x7f64ec0c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 -- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ec154e50 msgr2=0x7f64ec0c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ec154e50 0x7f64ec0c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 -- 172.21.15.92:0/842997292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4009ae0 con 0x7f64ec0ad040 2023-05-16T07:14:50.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.977+0000 7f64f1b6e700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec0c1bf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f64e800b490 tx=0x7f64e8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:50.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.978+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80090d0 con 0x7f64ec0ad040 2023-05-16T07:14:50.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.978+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e8012410 con 0x7f64ec0ad040 2023-05-16T07:14:50.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.978+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80127d0 con 0x7f64ec0ad040 2023-05-16T07:14:50.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.978+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec165860 con 0x7f64ec0ad040 2023-05-16T07:14:50.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.978+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec165e20 con 0x7f64ec0ad040 2023-05-16T07:14:50.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.979+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec047150 con 0x7f64ec0ad040 2023-05-16T07:14:50.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.979+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f64e800f030 con 0x7f64ec0ad040 2023-05-16T07:14:50.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.979+0000 7f64e1ffb700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64d005ed90 0x7f64d0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.979+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f64e8079540 con 0x7f64ec0ad040 2023-05-16T07:14:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.979+0000 7f64f136d700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64d005ed90 0x7f64d0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.980+0000 7f64f136d700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64d005ed90 0x7f64d0061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f64ec1501e0 tx=0x7f64dc006b90).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:50.983+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f64e804ac00 con 0x7f64ec0ad040 2023-05-16T07:14:51.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.209+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f64ec0bfb90 con 0x7f64ec0ad040 2023-05-16T07:14:51.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.210+0000 7f64e1ffb700 1 -- 172.21.15.92:0/842997292 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f64e804e220 con 0x7f64ec0ad040 2023-05-16T07:14:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:51.214 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:14:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:14:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:14:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:14:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:14:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:14:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:14:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:14:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:14:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:14:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:14:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:14:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:14:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:14:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:14:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:14:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:14:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:14:51.239 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:14:51.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64d005ed90 msgr2=0x7f64d0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64d005ed90 0x7f64d0061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f64ec1501e0 tx=0x7f64dc006b90).stop 2023-05-16T07:14:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 msgr2=0x7f64ec0c1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec0c1bf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f64e800b490 tx=0x7f64e8009060).stop 2023-05-16T07:14:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 shutdown_connections 2023-05-16T07:14:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f64d005ed90 0x7f64d0061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ec154e50 0x7f64ec0c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64ec0ad040 0x7f64ec0c1bf0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 --2- 172.21.15.92:0/842997292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64ec0c7380 0x7f64ec0c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 >> 172.21.15.92:0/842997292 conn(0x7f64ec068330 msgr2=0x7f64ec150920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 shutdown_connections 2023-05-16T07:14:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.212+0000 7f64f2b70700 1 -- 172.21.15.92:0/842997292 wait complete. 2023-05-16T07:14:51.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.349+0000 7f0bf9043700 1 Processor -- start 2023-05-16T07:14:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.349+0000 7f0bf9043700 1 -- start start 2023-05-16T07:14:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf9043700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bf4063e30 0x7f0bf40c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf9043700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40bdc30 con 0x7f0bf413b550 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf9043700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40c4e40 con 0x7f0bf413b930 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf9043700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf40c5400 con 0x7f0bf4063e30 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bf4063e30 0x7f0bf40c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf2d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0bf413b550 0x7f0bf4063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54724/0 (socket says 172.21.15.92:54724) 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.350+0000 7f0bf2d9d700 1 -- 172.21.15.92:0/2116717580 learned_addr learned my addr 172.21.15.92:0/2116717580 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf259c700 1 -- 172.21.15.92:0/2116717580 >> v1:172.21.15.158:6789/0 conn(0x7f0bf413b930 legacy=0x7f0bf40be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf259c700 1 -- 172.21.15.92:0/2116717580 >> v1:172.21.15.92:6789/0 conn(0x7f0bf413b550 legacy=0x7f0bf4063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf259c700 1 -- 172.21.15.92:0/2116717580 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40c5120 con 0x7f0bf4063e30 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf1d9b700 1 -- 172.21.15.92:0/2116717580 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696084286 0 0) 0x7f0bf40bdc30 con 0x7f0bf413b550 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf259c700 1 --2- 172.21.15.92:0/2116717580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf4063e30 0x7f0bf40c31f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f0bdc009760 tx=0x7f0bdc009060).ready entity=mon.0 client_cookie=312c5c0769eb2668 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf1d9b700 1 -- 172.21.15.92:0/2116717580 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00b580 con 0x7f0bf4063e30 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf1d9b700 1 -- 172.21.15.92:0/2116717580 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc00b6e0 con 0x7f0bf4063e30 2023-05-16T07:14:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf1d9b700 1 -- 172.21.15.92:0/2116717580 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00ba60 con 0x7f0bf4063e30 2023-05-16T07:14:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 -- 172.21.15.92:0/2116717580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf4063e30 msgr2=0x7f0bf40c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 --2- 172.21.15.92:0/2116717580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf4063e30 0x7f0bf40c31f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f0bdc009760 tx=0x7f0bdc009060).stop 2023-05-16T07:14:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 -- 172.21.15.92:0/2116717580 shutdown_connections 2023-05-16T07:14:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 --2- 172.21.15.92:0/2116717580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf4063e30 0x7f0bf40c31f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 -- 172.21.15.92:0/2116717580 >> 172.21.15.92:0/2116717580 conn(0x7f0bf40c8aa0 msgr2=0x7f0bf40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 -- 172.21.15.92:0/2116717580 shutdown_connections 2023-05-16T07:14:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.351+0000 7f0bf9043700 1 -- 172.21.15.92:0/2116717580 wait complete. 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 Processor -- start 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 -- start start 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bf4063e30 0x7f0bf4164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 0x7f0bf4153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bf413b930 0x7f0bf4153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c2ba0 con 0x7f0bf413b550 2023-05-16T07:14:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf41542f0 con 0x7f0bf413b930 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf9043700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4141440 con 0x7f0bf4063e30 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 0x7f0bf4153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 0x7f0bf4153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43986/0 (socket says 172.21.15.92:43986) 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf259c700 1 -- 172.21.15.92:0/3693956379 learned_addr learned my addr 172.21.15.92:0/3693956379 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf359e700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bf413b930 0x7f0bf4153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.353+0000 7f0bf2d9d700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bf4063e30 0x7f0bf4164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf259c700 1 -- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bf4063e30 msgr2=0x7f0bf4164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf259c700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bf4063e30 0x7f0bf4164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf259c700 1 -- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bf413b930 msgr2=0x7f0bf4153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf259c700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bf413b930 0x7f0bf4153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf259c700 1 -- 172.21.15.92:0/3693956379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc0092a0 con 0x7f0bf413b550 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf359e700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bf413b930 0x7f0bf4153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf2d9d700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bf4063e30 0x7f0bf4164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf259c700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 0x7f0bf4153910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f0bf406a9d0 tx=0x7f0bdc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00ba60 con 0x7f0bf413b550 2023-05-16T07:14:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc000c90 con 0x7f0bf413b550 2023-05-16T07:14:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf4141720 con 0x7f0bf413b550 2023-05-16T07:14:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01b640 con 0x7f0bf413b550 2023-05-16T07:14:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.354+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf4141d60 con 0x7f0bf413b550 2023-05-16T07:14:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.355+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf406c350 con 0x7f0bf413b550 2023-05-16T07:14:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.355+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0bdc00f070 con 0x7f0bf413b550 2023-05-16T07:14:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.356+0000 7f0beaffd700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bd405ed60 0x7f0bd4061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.356+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0bdc07b570 con 0x7f0bf413b550 2023-05-16T07:14:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.356+0000 7f0bf2d9d700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bd405ed60 0x7f0bd4061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.356+0000 7f0bf2d9d700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bd405ed60 0x7f0bd4061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0be4001560 tx=0x7f0be4009040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.360+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0bdc04cc30 con 0x7f0bf413b550 2023-05-16T07:14:51.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.522+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bf40c4b50 con 0x7f0bf413b550 2023-05-16T07:14:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.523+0000 7f0beaffd700 1 -- 172.21.15.92:0/3693956379 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+14598 (secure 0 0 0) 0x7f0bdc050250 con 0x7f0bf413b550 2023-05-16T07:14:51.527 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:51.528 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":58,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:50.834909+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bd405ed60 msgr2=0x7f0bd4061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bd405ed60 0x7f0bd4061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0be4001560 tx=0x7f0be4009040).stop 2023-05-16T07:14:51.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 msgr2=0x7f0bf4153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 0x7f0bf4153910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f0bf406a9d0 tx=0x7f0bdc017000).stop 2023-05-16T07:14:51.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 shutdown_connections 2023-05-16T07:14:51.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bd405ed60 0x7f0bd4061220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bf4063e30 0x7f0bf4164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bf413b930 0x7f0bf4153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 --2- 172.21.15.92:0/3693956379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bf413b550 0x7f0bf4153910 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 >> 172.21.15.92:0/3693956379 conn(0x7f0bf40c8aa0 msgr2=0x7f0bf40ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:51.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 shutdown_connections 2023-05-16T07:14:51.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.525+0000 7f0bf9043700 1 -- 172.21.15.92:0/3693956379 wait complete. 2023-05-16T07:14:51.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:14:51.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:14:51.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.651+0000 7f4c0b0f4700 1 Processor -- start 2023-05-16T07:14:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.651+0000 7f4c0b0f4700 1 -- start start 2023-05-16T07:14:51.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.651+0000 7f4c0b0f4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4c040c4c80 0x7f4c04152e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c0b0f4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c04063690 con 0x7f4c040622b0 2023-05-16T07:14:51.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c0b0f4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c04153330 con 0x7f4c0414f990 2023-05-16T07:14:51.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c0b0f4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040c5100 con 0x7f4c040c4c80 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c09691700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4c040622b0 0x7f4c04063b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54742/0 (socket says 172.21.15.92:54742) 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c09691700 1 -- 172.21.15.92:0/863717100 learned_addr learned my addr 172.21.15.92:0/863717100 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c03fff700 1 --2- 172.21.15.92:0/863717100 >> v2:172.21.15.158:3300/0 conn(0x7f4c040c4c80 0x7f4c04152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445482099 0 0) 0x7f4c04063690 con 0x7f4c040622b0 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf00033f0 con 0x7f4c040622b0 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515667815 0 0) 0x7f4bf00033f0 con 0x7f4c040622b0 2023-05-16T07:14:51.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 >> v2:172.21.15.158:3300/0 conn(0x7f4c040c4c80 msgr2=0x7f4c04152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 --2- 172.21.15.92:0/863717100 >> v2:172.21.15.158:3300/0 conn(0x7f4c040c4c80 0x7f4c04152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 >> v1:172.21.15.158:6789/0 conn(0x7f4c0414f990 legacy=0x7f4c040c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0414fd70 con 0x7f4c040622b0 2023-05-16T07:14:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4bfc0017f0 con 0x7f4c040622b0 2023-05-16T07:14:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 >> v1:172.21.15.92:6789/0 conn(0x7f4c040622b0 legacy=0x7f4c04063b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf00038e0 0x7f4bf0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf0004690 0x7f4bf0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0005000 0x7f4bf000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf000be80 con 0x7f4bf0004690 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf000bff0 con 0x7f4bf00038e0 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf000c1b0 con 0x7f4bf0005000 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c03fff700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf0004690 0x7f4bf0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c08e90700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf00038e0 0x7f4bf0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.652+0000 7f4c09691700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0005000 0x7f4bf000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c03fff700 1 -- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0005000 msgr2=0x7f4bf000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c03fff700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0005000 0x7f4bf000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c03fff700 1 -- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf00038e0 msgr2=0x7f4bf0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c03fff700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bf00038e0 0x7f4bf0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c03fff700 1 -- 172.21.15.92:0/863717100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00033f0 con 0x7f4bf0004690 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c09691700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0005000 0x7f4bf000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c09691700 1 -- 172.21.15.92:0/863717100 reap_dead start 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c03fff700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf0004690 0x7f4bf0004b10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f4bec009a80 tx=0x7f4bec009060).ready entity=mon.0 client_cookie=f98f64ea041abbc3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec009430 con 0x7f4bf0004690 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec00e640 con 0x7f4bf0004690 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c037fe700 1 -- 172.21.15.92:0/863717100 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00e9c0 con 0x7f4bf0004690 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/863717100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf0004690 msgr2=0x7f4bf0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf0004690 0x7f4bf0004b10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f4bec009a80 tx=0x7f4bec009060).stop 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/863717100 shutdown_connections 2023-05-16T07:14:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/863717100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bf0004690 0x7f4bf0004b10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/863717100 >> 172.21.15.92:0/863717100 conn(0x7f4c0405e820 msgr2=0x7f4c040c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:51.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/863717100 shutdown_connections 2023-05-16T07:14:51.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.653+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/863717100 wait complete. 2023-05-16T07:14:51.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.654+0000 7f4c0b0f4700 1 Processor -- start 2023-05-16T07:14:51.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 -- start start 2023-05-16T07:14:51.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0004690 0x7f4c041534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 0x7f4c04068470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c040aafd0 0x7f4c04154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c041539d0 con 0x7f4c041655c0 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040ad590 con 0x7f4c040aafd0 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c0b0f4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04153be0 con 0x7f4bf0004690 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 0x7f4c04068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.655+0000 7f4c03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 0x7f4c04068470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44004/0 (socket says 172.21.15.92:44004) 2023-05-16T07:14:51.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 -- 172.21.15.92:0/112496410 learned_addr learned my addr 172.21.15.92:0/112496410 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c08e90700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0004690 0x7f4c041534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c09691700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c040aafd0 0x7f4c04154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 -- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0004690 msgr2=0x7f4c041534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0004690 0x7f4c041534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 -- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c040aafd0 msgr2=0x7f4c04154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c040aafd0 0x7f4c04154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 -- 172.21.15.92:0/112496410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04165a40 con 0x7f4c041655c0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c09691700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c040aafd0 0x7f4c04154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c03fff700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 0x7f4c04068470 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f4bec00e4b0 tx=0x7f4bec01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec000cb0 con 0x7f4c041655c0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec00e7b0 con 0x7f4c041655c0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c040acc60 con 0x7f4c041655c0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0243f0 con 0x7f4c041655c0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.656+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c040ad220 con 0x7f4c041655c0 2023-05-16T07:14:51.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.657+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0406c350 con 0x7f4c041655c0 2023-05-16T07:14:51.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.657+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4bec02b030 con 0x7f4c041655c0 2023-05-16T07:14:51.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.657+0000 7f4c00ff9700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be405ed90 0x7f4be4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.658+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4bec07ae40 con 0x7f4c041655c0 2023-05-16T07:14:51.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.658+0000 7f4c08e90700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be405ed90 0x7f4be4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.658+0000 7f4c08e90700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be405ed90 0x7f4be4061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4c04156970 tx=0x7f4bf4007bb0).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.662+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4bec04c500 con 0x7f4c041655c0 2023-05-16T07:14:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.822+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4c040bdca0 con 0x7f4c041655c0 2023-05-16T07:14:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.822+0000 7f4c00ff9700 1 -- 172.21.15.92:0/112496410 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v58) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4bec029020 con 0x7f4c041655c0 2023-05-16T07:14:51.827 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:14:51.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be405ed90 msgr2=0x7f4be4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be405ed90 0x7f4be4061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4c04156970 tx=0x7f4bf4007bb0).stop 2023-05-16T07:14:51.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 msgr2=0x7f4c04068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 0x7f4c04068470 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f4bec00e4b0 tx=0x7f4bec01a000).stop 2023-05-16T07:14:51.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 shutdown_connections 2023-05-16T07:14:51.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4be405ed90 0x7f4be4061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bf0004690 0x7f4c041534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c040aafd0 0x7f4c04154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 --2- 172.21.15.92:0/112496410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c041655c0 0x7f4c04068470 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 >> 172.21.15.92:0/112496410 conn(0x7f4c0405e820 msgr2=0x7f4c040c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:51.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 shutdown_connections 2023-05-16T07:14:51.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.824+0000 7f4c0b0f4700 1 -- 172.21.15.92:0/112496410 wait complete. 2023-05-16T07:14:51.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:14:51.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.949+0000 7f3e0d5e5700 1 Processor -- start 2023-05-16T07:14:51.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.949+0000 7f3e0d5e5700 1 -- start start 2023-05-16T07:14:51.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e0d5e5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e08062b40 0x7f3e080aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e0d5e5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e08069440 con 0x7f3e08062f20 2023-05-16T07:14:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e0d5e5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e08069110 con 0x7f3e080698d0 2023-05-16T07:14:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e0d5e5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e08065bb0 con 0x7f3e08062b40 2023-05-16T07:14:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e08062b40 0x7f3e080aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e077fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3e08062f20 0x7f3e080a75b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54762/0 (socket says 172.21.15.92:54762) 2023-05-16T07:14:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.950+0000 7f3e077fe700 1 -- 172.21.15.92:0/2593196670 learned_addr learned my addr 172.21.15.92:0/2593196670 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e07fff700 1 -- 172.21.15.92:0/2593196670 >> v1:172.21.15.158:6789/0 conn(0x7f3e080698d0 legacy=0x7f3e080aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e07fff700 1 -- 172.21.15.92:0/2593196670 >> v1:172.21.15.92:6789/0 conn(0x7f3e08062f20 legacy=0x7f3e080a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e06ffd700 1 -- 172.21.15.92:0/2593196670 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257977021 0 0) 0x7f3e08069440 con 0x7f3e08062f20 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e07fff700 1 -- 172.21.15.92:0/2593196670 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08065e90 con 0x7f3e08062b40 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e07fff700 1 --2- 172.21.15.92:0/2593196670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062b40 0x7f3e080aa100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f3df8009760 tx=0x7f3df8009060).ready entity=mon.0 client_cookie=9b8d3e2e3dab43e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e06ffd700 1 -- 172.21.15.92:0/2593196670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df800b520 con 0x7f3e08062b40 2023-05-16T07:14:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e06ffd700 1 -- 172.21.15.92:0/2593196670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df800b680 con 0x7f3e08062b40 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e06ffd700 1 -- 172.21.15.92:0/2593196670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df800ba00 con 0x7f3e08062b40 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/2593196670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062b40 msgr2=0x7f3e080aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/2593196670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062b40 0x7f3e080aa100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f3df8009760 tx=0x7f3df8009060).stop 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.951+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/2593196670 shutdown_connections 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.952+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/2593196670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062b40 0x7f3e080aa100 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.952+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/2593196670 >> 172.21.15.92:0/2593196670 conn(0x7f3e08068a30 msgr2=0x7f3e080c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.952+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/2593196670 shutdown_connections 2023-05-16T07:14:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.952+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/2593196670 wait complete. 2023-05-16T07:14:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.953+0000 7f3e0d5e5700 1 Processor -- start 2023-05-16T07:14:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.953+0000 7f3e0d5e5700 1 -- start start 2023-05-16T07:14:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3e08062b40 0x7f3e0814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 0x7f3e0814a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3e080698d0 0x7f3e080a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0814ae50 con 0x7f3e08062f20 2023-05-16T07:14:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080a22a0 con 0x7f3e08062b40 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080a1690 con 0x7f3e080698d0 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 0x7f3e0814a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 0x7f3e0814a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44018/0 (socket says 172.21.15.92:44018) 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 -- 172.21.15.92:0/3417382327 learned_addr learned my addr 172.21.15.92:0/3417382327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e07fff700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3e08062b40 0x7f3e0814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0cde4700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3e080698d0 0x7f3e080a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 -- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3e080698d0 msgr2=0x7f3e080a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3e080698d0 0x7f3e080a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 -- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3e08062b40 msgr2=0x7f3e0814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3e08062b40 0x7f3e0814f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 -- 172.21.15.92:0/3417382327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80092a0 con 0x7f3e08062f20 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e077fe700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 0x7f3e0814a960 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3dfc00c4e0 tx=0x7f3dfc007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0072e0 con 0x7f3e08062f20 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc007440 con 0x7f3e08062f20 2023-05-16T07:14:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e080a1970 con 0x7f3e08062f20 2023-05-16T07:14:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0057a0 con 0x7f3e08062f20 2023-05-16T07:14:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.954+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e080a1f30 con 0x7f3e08062f20 2023-05-16T07:14:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.955+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e080a2450 con 0x7f3e08062f20 2023-05-16T07:14:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.955+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f3dfc01a060 con 0x7f3e08062f20 2023-05-16T07:14:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.956+0000 7f3deffff700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3df405ed90 0x7f3df4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.956+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3dfc0792e0 con 0x7f3e08062f20 2023-05-16T07:14:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.956+0000 7f3e07fff700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3df405ed90 0x7f3df4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.957+0000 7f3e07fff700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3df405ed90 0x7f3df4061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3df800b2e0 tx=0x7f3df8017000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:51.960+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3e080a2450 con 0x7f3e08062f20 2023-05-16T07:14:52.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.121+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3e080a2410 con 0x7f3e08062f20 2023-05-16T07:14:52.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.121+0000 7f3deffff700 1 -- 172.21.15.92:0/3417382327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3dfc018070 con 0x7f3e08062f20 2023-05-16T07:14:52.126 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:14:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3df405ed90 msgr2=0x7f3df4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3df405ed90 0x7f3df4061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3df800b2e0 tx=0x7f3df8017000).stop 2023-05-16T07:14:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 msgr2=0x7f3e0814a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 0x7f3e0814a960 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3dfc00c4e0 tx=0x7f3dfc007c70).stop 2023-05-16T07:14:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 shutdown_connections 2023-05-16T07:14:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3df405ed90 0x7f3df4061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3e080698d0 0x7f3e080a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3e08062b40 0x7f3e0814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 --2- 172.21.15.92:0/3417382327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e08062f20 0x7f3e0814a960 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 >> 172.21.15.92:0/3417382327 conn(0x7f3e08068a30 msgr2=0x7f3e080ac3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:52.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 shutdown_connections 2023-05-16T07:14:52.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.123+0000 7f3e0d5e5700 1 -- 172.21.15.92:0/3417382327 wait complete. 2023-05-16T07:14:52.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:14:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.246+0000 7ff39dc16700 1 Processor -- start 2023-05-16T07:14:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.246+0000 7ff39dc16700 1 -- start start 2023-05-16T07:14:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff39dc16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff39814e200 0x7ff39814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff39dc16700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff398066e50 0x7ff398157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff39dc16700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3980683d0 con 0x7ff3980672b0 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff39dc16700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff398137470 con 0x7ff39814e200 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff39dc16700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980c3520 con 0x7ff398066e50 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff3977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff39814e200 0x7ff39814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff3977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff39814e200 0x7ff39814e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44024/0 (socket says 172.21.15.92:44024) 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff3977fe700 1 -- 172.21.15.92:0/356765650 learned_addr learned my addr 172.21.15.92:0/356765650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.247+0000 7ff396ffd700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff398066e50 0x7ff398157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3977fe700 1 -- 172.21.15.92:0/356765650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff398066e50 msgr2=0x7ff398157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3977fe700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff398066e50 0x7ff398157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3977fe700 1 -- 172.21.15.92:0/356765650 >> v1:172.21.15.158:6789/0 conn(0x7ff3980672b0 legacy=0x7ff398154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3977fe700 1 -- 172.21.15.92:0/356765650 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980be060 con 0x7ff39814e200 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff396ffd700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff398066e50 0x7ff398157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3977fe700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff39814e5e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7ff388009f20 tx=0x7ff388009840).ready entity=mon.0 client_cookie=ecb92ab52e64bc46 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3967fc700 1 -- 172.21.15.92:0/356765650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800e070 con 0x7ff39814e200 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3967fc700 1 -- 172.21.15.92:0/356765650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38800ba80 con 0x7ff39814e200 2023-05-16T07:14:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff3967fc700 1 -- 172.21.15.92:0/356765650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800be00 con 0x7ff39814e200 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 -- 172.21.15.92:0/356765650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 msgr2=0x7ff39814e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff39814e5e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7ff388009f20 tx=0x7ff388009840).stop 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 -- 172.21.15.92:0/356765650 shutdown_connections 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff39814e5e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 --2- 172.21.15.92:0/356765650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff398066e50 0x7ff398157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 -- 172.21.15.92:0/356765650 >> 172.21.15.92:0/356765650 conn(0x7ff3980c7bc0 msgr2=0x7ff3980c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 -- 172.21.15.92:0/356765650 shutdown_connections 2023-05-16T07:14:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.248+0000 7ff39dc16700 1 -- 172.21.15.92:0/356765650 wait complete. 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 Processor -- start 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 -- start start 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff398066e50 0x7ff398139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3980672b0 0x7ff3981652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff398138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3981657c0 con 0x7ff39814e200 2023-05-16T07:14:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398139140 con 0x7ff398066e50 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff39dc16700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980ab6f0 con 0x7ff3980672b0 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.250+0000 7ff397fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff398138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff398138c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44034/0 (socket says 172.21.15.92:44034) 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 -- 172.21.15.92:0/3017557732 learned_addr learned my addr 172.21.15.92:0/3017557732 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff3977fe700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff398066e50 0x7ff398139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff396ffd700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3980672b0 0x7ff3981652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 -- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3980672b0 msgr2=0x7ff3981652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3980672b0 0x7ff3981652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 -- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff398066e50 msgr2=0x7ff398139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff398066e50 0x7ff398139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 -- 172.21.15.92:0/3017557732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff388009ae0 con 0x7ff39814e200 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff397fff700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff398138c50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff39000c440 tx=0x7ff390007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900123f0 con 0x7ff39814e200 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390012550 con 0x7ff39814e200 2023-05-16T07:14:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3980ab9d0 con 0x7ff39814e200 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900128d0 con 0x7ff39814e200 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.251+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3980abf90 con 0x7ff39814e200 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.252+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3980c6e40 con 0x7ff39814e200 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.253+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff390010070 con 0x7ff39814e200 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.253+0000 7ff37f7fe700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff38405ed90 0x7ff384061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.253+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff390078da0 con 0x7ff39814e200 2023-05-16T07:14:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.253+0000 7ff3977fe700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff38405ed90 0x7ff384061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.253+0000 7ff3977fe700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff38405ed90 0x7ff384061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff38800ba50 tx=0x7ff388009840).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.257+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff3900478f0 con 0x7ff39814e200 2023-05-16T07:14:52.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.421+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff3980c3940 con 0x7ff39814e200 2023-05-16T07:14:52.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.421+0000 7ff37f7fe700 1 -- 172.21.15.92:0/3017557732 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff39004a350 con 0x7ff39814e200 2023-05-16T07:14:52.425 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:14:52.426 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:14:52.427 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:14:52.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:14:52.427 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:14:52.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff38405ed90 msgr2=0x7ff384061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff38405ed90 0x7ff384061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff38800ba50 tx=0x7ff388009840).stop 2023-05-16T07:14:52.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 msgr2=0x7ff398138c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff398138c50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff39000c440 tx=0x7ff390007d40).stop 2023-05-16T07:14:52.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 shutdown_connections 2023-05-16T07:14:52.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff38405ed90 0x7ff384061250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39814e200 0x7ff398138c50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff398066e50 0x7ff398139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 --2- 172.21.15.92:0/3017557732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3980672b0 0x7ff3981652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 >> 172.21.15.92:0/3017557732 conn(0x7ff3980c7bc0 msgr2=0x7ff398157260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:52.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 shutdown_connections 2023-05-16T07:14:52.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.423+0000 7ff39dc16700 1 -- 172.21.15.92:0/3017557732 wait complete. 2023-05-16T07:14:52.439 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:14:52.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:14:52.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.552+0000 7fa81f1e8700 1 Processor -- start 2023-05-16T07:14:52.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.552+0000 7fa81f1e8700 1 -- start start 2023-05-16T07:14:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.553+0000 7fa81f1e8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 0x7fa8180698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.553+0000 7fa81f1e8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa818153f90 0x7fa818067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.553+0000 7fa81f1e8700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81805e760 con 0x7fa8180bd810 2023-05-16T07:14:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.553+0000 7fa81f1e8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa818069ef0 con 0x7fa818153f90 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.553+0000 7fa81f1e8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180bf540 con 0x7fa818153bb0 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 0x7fa8180698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81d9e5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa818153f90 0x7fa818067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e9e7700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fa8180bd810 0x7fa8180634d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:58390/0 (socket says 172.21.15.92:58390) 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 0x7fa8180698c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:36610/0 (socket says 172.21.15.92:36610) 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e9e7700 1 -- 172.21.15.92:0/3501983 learned_addr learned my addr 172.21.15.92:0/3501983 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 -- 172.21.15.92:0/3501983 >> v2:172.21.15.158:3300/0 conn(0x7fa818153f90 msgr2=0x7fa818067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 --2- 172.21.15.92:0/3501983 >> v2:172.21.15.158:3300/0 conn(0x7fa818153f90 0x7fa818067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 -- 172.21.15.92:0/3501983 >> v1:172.21.15.158:6789/0 conn(0x7fa8180bd810 legacy=0x7fa8180634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 -- 172.21.15.92:0/3501983 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa818068450 con 0x7fa818153bb0 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81e1e6700 1 --2- 172.21.15.92:0/3501983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 0x7fa8180698c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa808009e60 tx=0x7fa808009780).ready entity=mon.2 client_cookie=6fe2f780fc44494d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81d1e4700 1 -- 172.21.15.92:0/3501983 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800e070 con 0x7fa818153bb0 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81d1e4700 1 -- 172.21.15.92:0/3501983 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80800bb60 con 0x7fa818153bb0 2023-05-16T07:14:52.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.554+0000 7fa81d1e4700 1 -- 172.21.15.92:0/3501983 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808012410 con 0x7fa818153bb0 2023-05-16T07:14:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 -- 172.21.15.92:0/3501983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 msgr2=0x7fa8180698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/3501983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 0x7fa8180698c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa808009e60 tx=0x7fa808009780).stop 2023-05-16T07:14:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 -- 172.21.15.92:0/3501983 shutdown_connections 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/3501983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153bb0 0x7fa8180698c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/3501983 >> v2:172.21.15.158:3300/0 conn(0x7fa818153f90 0x7fa818067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 -- 172.21.15.92:0/3501983 >> 172.21.15.92:0/3501983 conn(0x7fa818062d80 msgr2=0x7fa81805dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 -- 172.21.15.92:0/3501983 shutdown_connections 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.555+0000 7fa81f1e8700 1 -- 172.21.15.92:0/3501983 wait complete. 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.556+0000 7fa81f1e8700 1 Processor -- start 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.556+0000 7fa81f1e8700 1 -- start start 2023-05-16T07:14:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 0x7fa8180c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa818153bb0 0x7fa8181527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153f90 0x7fa8180634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c2830 con 0x7fa8180bd810 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180a4be0 con 0x7fa818153bb0 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180ab420 con 0x7fa818153f90 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 0x7fa8180c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 0x7fa8180c2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44044/0 (socket says 172.21.15.92:44044) 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 -- 172.21.15.92:0/2512915244 learned_addr learned my addr 172.21.15.92:0/2512915244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e9e7700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153f90 0x7fa8180634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81d9e5700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa818153bb0 0x7fa8181527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 -- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153f90 msgr2=0x7fa8180634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153f90 0x7fa8180634d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 -- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa818153bb0 msgr2=0x7fa8181527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa818153bb0 0x7fa8181527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 -- 172.21.15.92:0/2512915244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808009a20 con 0x7fa8180bd810 2023-05-16T07:14:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81e1e6700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 0x7fa8180c2340 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fa808009f70 tx=0x7fa808009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80801d630 con 0x7fa8180bd810 2023-05-16T07:14:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80801d790 con 0x7fa8180bd810 2023-05-16T07:14:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180c03b0 con 0x7fa8180bd810 2023-05-16T07:14:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80801db50 con 0x7fa8180bd810 2023-05-16T07:14:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.557+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8180bfd60 con 0x7fa8180bd810 2023-05-16T07:14:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.558+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8180c0690 con 0x7fa8180bd810 2023-05-16T07:14:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.559+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa808026410 con 0x7fa8180bd810 2023-05-16T07:14:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.559+0000 7fa8167fc700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa80405ed90 0x7fa804061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.559+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa80807bb50 con 0x7fa8180bd810 2023-05-16T07:14:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.559+0000 7fa81d9e5700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa80405ed90 0x7fa804061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.560+0000 7fa81d9e5700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa80405ed90 0x7fa804061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8180ae4f0 tx=0x7fa80c008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.563+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa80804d1b0 con 0x7fa8180bd810 2023-05-16T07:14:52.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.727+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fa818047150 con 0x7fa8180bd810 2023-05-16T07:14:52.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.848+0000 7fa8167fc700 1 -- 172.21.15.92:0/2512915244 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 152+0+0 (secure 0 0 0) 0x7fa8080507d0 con 0x7fa8180bd810 2023-05-16T07:14:52.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.852+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa80405ed90 msgr2=0x7fa804061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.852+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa80405ed90 0x7fa804061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8180ae4f0 tx=0x7fa80c008040).stop 2023-05-16T07:14:52.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.852+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 msgr2=0x7fa8180c2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.852+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 0x7fa8180c2340 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fa808009f70 tx=0x7fa808009780).stop 2023-05-16T07:14:52.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.853+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 shutdown_connections 2023-05-16T07:14:52.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.853+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa80405ed90 0x7fa804061250 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8180ae4f0 tx=0x7fa80c008040).stop 2023-05-16T07:14:52.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.853+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa818153f90 0x7fa8180634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.853+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8180bd810 0x7fa8180c2340 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.853+0000 7fa81f1e8700 1 --2- 172.21.15.92:0/2512915244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa818153bb0 0x7fa8181527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.853+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 >> 172.21.15.92:0/2512915244 conn(0x7fa818062d80 msgr2=0x7fa818157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:52.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.856+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 shutdown_connections 2023-05-16T07:14:52.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.856+0000 7fa81f1e8700 1 -- 172.21.15.92:0/2512915244 wait complete. 2023-05-16T07:14:52.861 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:14:52.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:14:52.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.984+0000 7fa88714f700 1 Processor -- start 2023-05-16T07:14:52.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.984+0000 7fa88714f700 1 -- start start 2023-05-16T07:14:52.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.985+0000 7fa88714f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa880153ee0 0x7fa8801542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.985+0000 7fa88714f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa880155870 con 0x7fa8800be280 2023-05-16T07:14:52.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.985+0000 7fa88714f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa880063120 con 0x7fa880062cc0 2023-05-16T07:14:52.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.985+0000 7fa88714f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa880155670 con 0x7fa880153ee0 2023-05-16T07:14:52.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.985+0000 7fa8856ec700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8800be280 0x7fa880064e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54802/0 (socket says 172.21.15.92:54802) 2023-05-16T07:14:52.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.985+0000 7fa8856ec700 1 -- 172.21.15.92:0/3672606189 learned_addr learned my addr 172.21.15.92:0/3672606189 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:52.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 --2- 172.21.15.92:0/3672606189 >> v2:172.21.15.158:3300/0 conn(0x7fa880153ee0 0x7fa8801542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280494483 0 0) 0x7fa880155870 con 0x7fa8800be280 2023-05-16T07:14:52.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8680033f0 con 0x7fa8800be280 2023-05-16T07:14:52.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226995521 0 0) 0x7fa8680033f0 con 0x7fa8800be280 2023-05-16T07:14:52.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 >> v2:172.21.15.158:3300/0 conn(0x7fa880153ee0 msgr2=0x7fa8801542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:52.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 --2- 172.21.15.92:0/3672606189 >> v2:172.21.15.158:3300/0 conn(0x7fa880153ee0 0x7fa8801542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 >> v1:172.21.15.158:6789/0 conn(0x7fa880062cc0 legacy=0x7fa8800bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8801548b0 con 0x7fa8800be280 2023-05-16T07:14:52.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fa8780017f0 con 0x7fa8800be280 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 >> v1:172.21.15.92:6789/0 conn(0x7fa8800be280 legacy=0x7fa880064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8680038e0 0x7fa868003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa868004690 0x7fa868004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa868005000 0x7fa86800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880155670 con 0x7fa8680038e0 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86800beb0 con 0x7fa868005000 2023-05-16T07:14:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86800c070 con 0x7fa868004690 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8680038e0 0x7fa868003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa8856ec700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa868005000 0x7fa86800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa87ffff700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa868004690 0x7fa868004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 -- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa868004690 msgr2=0x7fa868004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa868004690 0x7fa868004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 -- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa868005000 msgr2=0x7fa86800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:52.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa868005000 0x7fa86800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa884eeb700 1 -- 172.21.15.92:0/3672606189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680033f0 con 0x7fa8680038e0 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa8856ec700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa868005000 0x7fa86800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.986+0000 7fa8856ec700 1 -- 172.21.15.92:0/3672606189 reap_dead start 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa884eeb700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8680038e0 0x7fa868003d00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fa870009a80 tx=0x7fa870009060).ready entity=mon.0 client_cookie=c9de69162a1401ea server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870009430 con 0x7fa8680038e0 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87000e640 con 0x7fa8680038e0 2023-05-16T07:14:52.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa87f7fe700 1 -- 172.21.15.92:0/3672606189 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e9c0 con 0x7fa8680038e0 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 -- 172.21.15.92:0/3672606189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8680038e0 msgr2=0x7fa868003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8680038e0 0x7fa868003d00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fa870009a80 tx=0x7fa870009060).stop 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 -- 172.21.15.92:0/3672606189 shutdown_connections 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 --2- 172.21.15.92:0/3672606189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8680038e0 0x7fa868003d00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 -- 172.21.15.92:0/3672606189 >> 172.21.15.92:0/3672606189 conn(0x7fa8800c8420 msgr2=0x7fa8800c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 -- 172.21.15.92:0/3672606189 shutdown_connections 2023-05-16T07:14:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.987+0000 7fa88714f700 1 -- 172.21.15.92:0/3672606189 wait complete. 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.988+0000 7fa88714f700 1 Processor -- start 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.988+0000 7fa88714f700 1 -- start start 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa88714f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8680038e0 0x7fa880150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa88714f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 0x7fa8800be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa88714f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa8800beee0 0x7fa8801681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa88714f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880164c40 con 0x7fa8800c05b0 2023-05-16T07:14:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa88714f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880164f20 con 0x7fa8680038e0 2023-05-16T07:14:53.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa88714f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880165200 con 0x7fa8800beee0 2023-05-16T07:14:53.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa87ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 0x7fa8800be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:53.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa87ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 0x7fa8800be9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44066/0 (socket says 172.21.15.92:44066) 2023-05-16T07:14:53.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa87ffff700 1 -- 172.21.15.92:0/1385412442 learned_addr learned my addr 172.21.15.92:0/1385412442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa884eeb700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8680038e0 0x7fa880150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.989+0000 7fa8856ec700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa8800beee0 0x7fa8801681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87ffff700 1 -- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa8800beee0 msgr2=0x7fa8801681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87ffff700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa8800beee0 0x7fa8801681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87ffff700 1 -- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8680038e0 msgr2=0x7fa880150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87ffff700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8680038e0 0x7fa880150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87ffff700 1 -- 172.21.15.92:0/1385412442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8801654e0 con 0x7fa8800c05b0 2023-05-16T07:14:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87ffff700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 0x7fa8800be9f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fa87400e510 tx=0x7fa874009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874014430 con 0x7fa8800c05b0 2023-05-16T07:14:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa874014590 con 0x7fa8800c05b0 2023-05-16T07:14:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874014910 con 0x7fa8800c05b0 2023-05-16T07:14:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa880167c80 con 0x7fa8800c05b0 2023-05-16T07:14:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.990+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8800c0a30 con 0x7fa8800c05b0 2023-05-16T07:14:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.991+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88006c350 con 0x7fa8800c05b0 2023-05-16T07:14:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.991+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa8740133e0 con 0x7fa8800c05b0 2023-05-16T07:14:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.992+0000 7fa87cff9700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa86c05ed90 0x7fa86c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.992+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fa8740796e0 con 0x7fa8800c05b0 2023-05-16T07:14:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.992+0000 7fa884eeb700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa86c05ed90 0x7fa86c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.992+0000 7fa884eeb700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa86c05ed90 0x7fa86c061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa870006d50 tx=0x7fa87001a000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:53.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:52.996+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa8740498f0 con 0x7fa8800c05b0 2023-05-16T07:14:53.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.159+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa880063120 con 0x7fa8800c05b0 2023-05-16T07:14:53.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.854+0000 7fa87cff9700 1 -- 172.21.15.92:0/1385412442 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa880063120 con 0x7fa8800c05b0 2023-05-16T07:14:53.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.866+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa86c05ed90 msgr2=0x7fa86c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:53.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.866+0000 7fa88714f700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa86c05ed90 0x7fa86c061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa870006d50 tx=0x7fa87001a000).stop 2023-05-16T07:14:53.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.866+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 msgr2=0x7fa8800be9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:53.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.866+0000 7fa88714f700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 0x7fa8800be9f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fa87400e510 tx=0x7fa874009040).stop 2023-05-16T07:14:53.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.871+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 shutdown_connections 2023-05-16T07:14:53.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.871+0000 7fa88714f700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa86c05ed90 0x7fa86c061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:53.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.871+0000 7fa88714f700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa8800beee0 0x7fa8801681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:53.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.871+0000 7fa88714f700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8680038e0 0x7fa880150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:53.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.871+0000 7fa88714f700 1 --2- 172.21.15.92:0/1385412442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8800c05b0 0x7fa8800be9f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:53.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.871+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 >> 172.21.15.92:0/1385412442 conn(0x7fa8800c8420 msgr2=0x7fa8800c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:53.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.874+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 shutdown_connections 2023-05-16T07:14:53.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:53.874+0000 7fa88714f700 1 -- 172.21.15.92:0/1385412442 wait complete. 2023-05-16T07:14:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:14:53.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:14:54.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.004+0000 7f1f16718700 1 Processor -- start 2023-05-16T07:14:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.004+0000 7f1f16718700 1 -- start start 2023-05-16T07:14:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.005+0000 7f1f16718700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1f10154090 0x7f1f100aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.005+0000 7f1f16718700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f100685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.005+0000 7f1f16718700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100c94d0 con 0x7f1f10154470 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.005+0000 7f1f16718700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100cb7e0 con 0x7f1f10154090 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.005+0000 7f1f16718700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10068aa0 con 0x7f1f10068160 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1f10154090 0x7f1f100aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1f10154090 0x7f1f100aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:47708/0 (socket says 172.21.15.92:47708) 2023-05-16T07:14:54.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 -- 172.21.15.92:0/301163786 learned_addr learned my addr 172.21.15.92:0/301163786 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:54.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0f7fe700 1 --1- 172.21.15.92:0/301163786 >> v1:172.21.15.158:6789/0 conn(0x7f1f10154470 0x7f1f100a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:58420/0 (socket says 172.21.15.92:58420) 2023-05-16T07:14:54.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f14cb5700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f100685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 -- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 msgr2=0x7f1f100685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f100685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 -- 172.21.15.92:0/301163786 >> v1:172.21.15.158:6789/0 conn(0x7f1f10154470 legacy=0x7f1f100a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 -- 172.21.15.92:0/301163786 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100c88c0 con 0x7f1f10154090 2023-05-16T07:14:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f14cb5700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f100685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:14:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0ffff700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f100aaca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1f00006fd0 tx=0x7f1f000090f0).ready entity=mon.1 client_cookie=f42183f6b83b56f4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.006+0000 7f1f0effd700 1 -- 172.21.15.92:0/301163786 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000b580 con 0x7f1f10154090 2023-05-16T07:14:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f0effd700 1 -- 172.21.15.92:0/301163786 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0000b6e0 con 0x7f1f10154090 2023-05-16T07:14:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f0effd700 1 -- 172.21.15.92:0/301163786 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000ba60 con 0x7f1f10154090 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 -- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 msgr2=0x7f1f100aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f100aaca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1f00006fd0 tx=0x7f1f000090f0).stop 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 -- 172.21.15.92:0/301163786 shutdown_connections 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f100aaca0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 --2- 172.21.15.92:0/301163786 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f100685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 -- 172.21.15.92:0/301163786 >> 172.21.15.92:0/301163786 conn(0x7f1f10067920 msgr2=0x7f1f100cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 -- 172.21.15.92:0/301163786 shutdown_connections 2023-05-16T07:14:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.007+0000 7f1f16718700 1 -- 172.21.15.92:0/301163786 wait complete. 2023-05-16T07:14:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.008+0000 7f1f16718700 1 Processor -- start 2023-05-16T07:14:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.008+0000 7f1f16718700 1 -- start start 2023-05-16T07:14:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f16718700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f1014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f16718700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f10155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f16718700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 0x7f1f10155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f16718700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10156860 con 0x7f1f10154470 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f16718700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100c39d0 con 0x7f1f10154090 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f16718700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101562a0 con 0x7f1f10068160 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 0x7f1f10155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 0x7f1f10155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44082/0 (socket says 172.21.15.92:44082) 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 -- 172.21.15.92:0/91902415 learned_addr learned my addr 172.21.15.92:0/91902415 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f0ffff700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f1014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f0f7fe700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f10155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 -- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 msgr2=0x7f1f1014eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f1014eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 -- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 msgr2=0x7f1f10155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f10155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.009+0000 7f1f14cb5700 1 -- 172.21.15.92:0/91902415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08009a20 con 0x7f1f10154470 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.010+0000 7f1f14cb5700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 0x7f1f10155d10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f1f08009fc0 tx=0x7f1f0800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.010+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800cbd0 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.010+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0800f040 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.010+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f00009370 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.010+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08013400 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.010+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f100c3660 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.011+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f10066730 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.012+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1f080137b0 con 0x7f1f10154470 2023-05-16T07:14:54.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.012+0000 7f1ef7fff700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1efc05ed90 0x7f1efc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.012+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f1f0807b140 con 0x7f1f10154470 2023-05-16T07:14:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.012+0000 7f1f0ffff700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1efc05ed90 0x7f1efc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.013+0000 7f1f0ffff700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1efc05ed90 0x7f1efc061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f1f00009130 tx=0x7f1f00017000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.015+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1f0804c590 con 0x7f1f10154470 2023-05-16T07:14:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.220+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1f1006c350 con 0x7f1f10154470 2023-05-16T07:14:54.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.227+0000 7f1ef7fff700 1 -- 172.21.15.92:0/91902415 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v16) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1f0804fbb0 con 0x7f1f10154470 2023-05-16T07:14:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.230+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1efc05ed90 msgr2=0x7f1efc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.230+0000 7f1f16718700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1efc05ed90 0x7f1efc061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f1f00009130 tx=0x7f1f00017000).stop 2023-05-16T07:14:54.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.231+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 msgr2=0x7f1f10155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.231+0000 7f1f16718700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 0x7f1f10155d10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f1f08009fc0 tx=0x7f1f0800b040).stop 2023-05-16T07:14:54.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 shutdown_connections 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1efc05ed90 0x7f1efc061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f10068160 0x7f1f1014eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f10154470 0x7f1f10155d10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 --2- 172.21.15.92:0/91902415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f10154090 0x7f1f10155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 >> 172.21.15.92:0/91902415 conn(0x7f1f10067920 msgr2=0x7f1f100c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 shutdown_connections 2023-05-16T07:14:54.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.232+0000 7f1f16718700 1 -- 172.21.15.92:0/91902415 wait complete. 2023-05-16T07:14:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 7 and data pool 8 2023-05-16T07:14:54.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:54.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.366+0000 7f0ef5c7e700 1 Processor -- start 2023-05-16T07:14:54.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.366+0000 7f0ef5c7e700 1 -- start start 2023-05-16T07:14:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0ef5c7e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ef0154550 0x7f0ef00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0ef5c7e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0ef5c7e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef00c0660 con 0x7f0ef0154170 2023-05-16T07:14:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0ef5c7e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef006a000 con 0x7f0ef0154550 2023-05-16T07:14:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0ef5c7e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00bfd30 con 0x7f0ef0069b90 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ef0154550 0x7f0ef00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eef7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0ef0154170 0x7f0ef00c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54834/0 (socket says 172.21.15.92:54834) 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eef7fe700 1 -- 172.21.15.92:0/3681937422 learned_addr learned my addr 172.21.15.92:0/3681937422 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeffff700 1 --2- 172.21.15.92:0/3681937422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeeffd700 1 -- 172.21.15.92:0/3681937422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 msgr2=0x7f0ef00c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeeffd700 1 --2- 172.21.15.92:0/3681937422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeeffd700 1 -- 172.21.15.92:0/3681937422 >> v1:172.21.15.92:6789/0 conn(0x7f0ef0154170 legacy=0x7f0ef00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eee7fc700 1 -- 172.21.15.92:0/3681937422 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594419103 0 0) 0x7f0ef00c0660 con 0x7f0ef0154170 2023-05-16T07:14:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeeffd700 1 -- 172.21.15.92:0/3681937422 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00c0010 con 0x7f0ef0154550 2023-05-16T07:14:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.367+0000 7f0eeeffd700 1 --2- 172.21.15.92:0/3681937422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154550 0x7f0ef00696a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f0ee4009760 tx=0x7f0ee4009060).ready entity=mon.0 client_cookie=50855bfd84a0a8f8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0eee7fc700 1 -- 172.21.15.92:0/3681937422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40090d0 con 0x7f0ef0154550 2023-05-16T07:14:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0eee7fc700 1 -- 172.21.15.92:0/3681937422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400b580 con 0x7f0ef0154550 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0eee7fc700 1 -- 172.21.15.92:0/3681937422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400b900 con 0x7f0ef0154550 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/3681937422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154550 msgr2=0x7f0ef00696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/3681937422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154550 0x7f0ef00696a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f0ee4009760 tx=0x7f0ee4009060).stop 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/3681937422 shutdown_connections 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/3681937422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/3681937422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154550 0x7f0ef00696a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/3681937422 >> 172.21.15.92:0/3681937422 conn(0x7f0ef00cb700 msgr2=0x7f0ef00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/3681937422 shutdown_connections 2023-05-16T07:14:54.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.368+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/3681937422 wait complete. 2023-05-16T07:14:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.369+0000 7f0ef5c7e700 1 Processor -- start 2023-05-16T07:14:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.369+0000 7f0ef5c7e700 1 -- start start 2023-05-16T07:14:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0ef5c7e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0ef5c7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 0x7f0ef01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0ef5c7e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ef0154550 0x7f0ef01567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0ef5c7e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00ae600 con 0x7f0ef0154170 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0ef5c7e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00af5a0 con 0x7f0ef0154550 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0ef5c7e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00af110 con 0x7f0ef0069b90 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 0x7f0ef01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 0x7f0ef01562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44094/0 (socket says 172.21.15.92:44094) 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 -- 172.21.15.92:0/1542280221 learned_addr learned my addr 172.21.15.92:0/1542280221 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eef7fe700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeffff700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ef0154550 0x7f0ef01567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 -- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 msgr2=0x7f0ef00c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 -- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ef0154550 msgr2=0x7f0ef01567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.370+0000 7f0eeeffd700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ef0154550 0x7f0ef01567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0eeeffd700 1 -- 172.21.15.92:0/1542280221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee40092a0 con 0x7f0ef0154170 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0eeeffd700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 0x7f0ef01562c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f0ef0083060 tx=0x7f0ee4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400b900 con 0x7f0ef0154170 2023-05-16T07:14:54.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400bcc0 con 0x7f0ef0154170 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef00ae7e0 con 0x7f0ef0154170 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee401b410 con 0x7f0ef0154170 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.371+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef00aeda0 con 0x7f0ef0154170 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.372+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef00666c0 con 0x7f0ef0154170 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.372+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0ee400f070 con 0x7f0ef0154170 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.372+0000 7f0ed77fe700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0edc05ed90 0x7f0edc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.373+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f0ee407b7e0 con 0x7f0ef0154170 2023-05-16T07:14:54.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.373+0000 7f0eef7fe700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0edc05ed90 0x7f0edc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.373+0000 7f0eef7fe700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0edc05ed90 0x7f0edc061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0ee0001560 tx=0x7f0ee0009040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.377+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0ee404cb50 con 0x7f0ef0154170 2023-05-16T07:14:54.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.540+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ef006c350 con 0x7f0ef0154170 2023-05-16T07:14:54.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.541+0000 7f0ed77fe700 1 -- 172.21.15.92:0/1542280221 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17827 (secure 0 0 0) 0x7f0ee4050170 con 0x7f0ef0154170 2023-05-16T07:14:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:54.545 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:54.221288+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:14:52.750879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-05-16T07:14:53.182846+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0edc05ed90 msgr2=0x7f0edc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0edc05ed90 0x7f0edc061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0ee0001560 tx=0x7f0ee0009040).stop 2023-05-16T07:14:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 msgr2=0x7f0ef01562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 0x7f0ef01562c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f0ef0083060 tx=0x7f0ee4017000).stop 2023-05-16T07:14:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 shutdown_connections 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0edc05ed90 0x7f0edc061250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ef0069b90 0x7f0ef00c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ef0154550 0x7f0ef01567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 --2- 172.21.15.92:0/1542280221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0154170 0x7f0ef01562c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 >> 172.21.15.92:0/1542280221 conn(0x7f0ef00cb700 msgr2=0x7f0ef00ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 shutdown_connections 2023-05-16T07:14:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.543+0000 7f0ef5c7e700 1 -- 172.21.15.92:0/1542280221 wait complete. 2023-05-16T07:14:54.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:54.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 Processor -- start 2023-05-16T07:14:54.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 -- start start 2023-05-16T07:14:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3710154040 0x7f3710069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3710063e30 0x7f3710062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3710062960 con 0x7f37100bf4f0 2023-05-16T07:14:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3710154420 con 0x7f3710063e30 2023-05-16T07:14:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.666+0000 7f37169a3700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3710069fb0 con 0x7f3710154040 2023-05-16T07:14:54.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3710063e30 0x7f3710062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3710063e30 0x7f3710062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44102/0 (socket says 172.21.15.92:44102) 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 -- 172.21.15.92:0/1059135651 learned_addr learned my addr 172.21.15.92:0/1059135651 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f370ffff700 1 --2- 172.21.15.92:0/1059135651 >> v2:172.21.15.158:3300/0 conn(0x7f3710154040 0x7f3710069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 -- 172.21.15.92:0/1059135651 >> v2:172.21.15.158:3300/0 conn(0x7f3710154040 msgr2=0x7f3710069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 --2- 172.21.15.92:0/1059135651 >> v2:172.21.15.158:3300/0 conn(0x7f3710154040 0x7f3710069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 -- 172.21.15.92:0/1059135651 >> v1:172.21.15.158:6789/0 conn(0x7f37100bf4f0 legacy=0x7f3710063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 -- 172.21.15.92:0/1059135651 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3710062350 con 0x7f3710063e30 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f3714f40700 1 --2- 172.21.15.92:0/1059135651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710063e30 0x7f3710062df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f37100c89b0 tx=0x7f3708009840).ready entity=mon.0 client_cookie=593c70f5b947eb3e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f370effd700 1 -- 172.21.15.92:0/1059135651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800e070 con 0x7f3710063e30 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f370effd700 1 -- 172.21.15.92:0/1059135651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370800ba10 con 0x7f3710063e30 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.667+0000 7f370effd700 1 -- 172.21.15.92:0/1059135651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800bd90 con 0x7f3710063e30 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 -- 172.21.15.92:0/1059135651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710063e30 msgr2=0x7f3710062df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 --2- 172.21.15.92:0/1059135651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710063e30 0x7f3710062df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f37100c89b0 tx=0x7f3708009840).stop 2023-05-16T07:14:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 -- 172.21.15.92:0/1059135651 shutdown_connections 2023-05-16T07:14:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 --2- 172.21.15.92:0/1059135651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710063e30 0x7f3710062df0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 --2- 172.21.15.92:0/1059135651 >> v2:172.21.15.158:3300/0 conn(0x7f3710154040 0x7f3710069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 -- 172.21.15.92:0/1059135651 >> 172.21.15.92:0/1059135651 conn(0x7f37100c8aa0 msgr2=0x7f37100c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 -- 172.21.15.92:0/1059135651 shutdown_connections 2023-05-16T07:14:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.668+0000 7f37169a3700 1 -- 172.21.15.92:0/1059135651 wait complete. 2023-05-16T07:14:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.669+0000 7f37169a3700 1 Processor -- start 2023-05-16T07:14:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.669+0000 7f37169a3700 1 -- start start 2023-05-16T07:14:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3710063e30 0x7f371014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37100bf4f0 0x7f37100aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 0x7f37100aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37100ab3c0 con 0x7f3710154040 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37100ad3f0 con 0x7f37100bf4f0 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37100ad880 con 0x7f3710063e30 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 0x7f37100aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 0x7f37100aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44110/0 (socket says 172.21.15.92:44110) 2023-05-16T07:14:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 -- 172.21.15.92:0/3503417666 learned_addr learned my addr 172.21.15.92:0/3503417666 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 -- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3710063e30 msgr2=0x7f371014ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3710063e30 0x7f371014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 -- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37100bf4f0 msgr2=0x7f37100aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37100bf4f0 0x7f37100aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 -- 172.21.15.92:0/3503417666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708009ae0 con 0x7f3710154040 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f3714f40700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 0x7f37100aaed0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f3708009540 tx=0x7f3708009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708013e90 con 0x7f3710154040 2023-05-16T07:14:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370801c400 con 0x7f3710154040 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371014f220 con 0x7f3710154040 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370801c780 con 0x7f3710154040 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.670+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37100acac0 con 0x7f3710154040 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.671+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3710164e60 con 0x7f3710154040 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.672+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f370801cb70 con 0x7f3710154040 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.672+0000 7f36f7fff700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f36fc05ed90 0x7f36fc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.672+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f370807ba30 con 0x7f3710154040 2023-05-16T07:14:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.672+0000 7f370ffff700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f36fc05ed90 0x7f36fc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.673+0000 7f370ffff700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f36fc05ed90 0x7f36fc061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f37100a84e0 tx=0x7f3704008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.676+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f370804cda0 con 0x7f3710154040 2023-05-16T07:14:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.840+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3710164e60 con 0x7f3710154040 2023-05-16T07:14:54.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.841+0000 7f36f7fff700 1 -- 172.21.15.92:0/3503417666 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17827 (secure 0 0 0) 0x7f37080503c0 con 0x7f3710154040 2023-05-16T07:14:54.845 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:54.845 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:54.221288+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:14:52.750879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-05-16T07:14:53.182846+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:54.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.842+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f36fc05ed90 msgr2=0x7f36fc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.842+0000 7f37169a3700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f36fc05ed90 0x7f36fc061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f37100a84e0 tx=0x7f3704008040).stop 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.842+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 msgr2=0x7f37100aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.842+0000 7f37169a3700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 0x7f37100aaed0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f3708009540 tx=0x7f3708009840).stop 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 shutdown_connections 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f36fc05ed90 0x7f36fc061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3710063e30 0x7f371014ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3710154040 0x7f37100aaed0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 --2- 172.21.15.92:0/3503417666 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f37100bf4f0 0x7f37100aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:54.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 >> 172.21.15.92:0/3503417666 conn(0x7f37100c8aa0 msgr2=0x7f37101562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 shutdown_connections 2023-05-16T07:14:54.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.843+0000 7f37169a3700 1 -- 172.21.15.92:0/3503417666 wait complete. 2023-05-16T07:14:54.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:14:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.968+0000 7f4be2771700 1 Processor -- start 2023-05-16T07:14:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.968+0000 7f4be2771700 1 -- start start 2023-05-16T07:14:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.968+0000 7f4be2771700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 0x7f4bdc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.968+0000 7f4be2771700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4bdc154470 0x7f4bdc0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4be2771700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc0aa570 con 0x7f4bdc0aa150 2023-05-16T07:14:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4be2771700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc0c94d0 con 0x7f4bdc154470 2023-05-16T07:14:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4be2771700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0cb7e0 con 0x7f4bdc154090 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4be0d0e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4bdc0aa150 0x7f4bdc068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54866/0 (socket says 172.21.15.92:54866) 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4be0d0e700 1 -- 172.21.15.92:0/2592630816 learned_addr learned my addr 172.21.15.92:0/2592630816 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdbfff700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 0x7f4bdc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/2592630816 >> v2:172.21.15.158:3300/0 conn(0x7f4bdc154470 0x7f4bdc0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800329785 0 0) 0x7f4bd4001490 con 0x7f4bdc0aa150 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc80033f0 con 0x7f4bdc0aa150 2023-05-16T07:14:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099303179 0 0) 0x7f4bc80033f0 con 0x7f4bdc0aa150 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 msgr2=0x7f4bdc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 0x7f4bdc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 >> v2:172.21.15.158:3300/0 conn(0x7f4bdc154470 msgr2=0x7f4bdc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 --2- 172.21.15.92:0/2592630816 >> v2:172.21.15.158:3300/0 conn(0x7f4bdc154470 0x7f4bdc0a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0c88c0 con 0x7f4bdc0aa150 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4bd4001980 con 0x7f4bdc0aa150 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/2592630816 >> v2:172.21.15.158:3300/0 conn(0x7f4bdc154470 0x7f4bdc0a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:54.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 >> v1:172.21.15.92:6789/0 conn(0x7f4bdc0aa150 legacy=0x7f4bdc068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 0x7f4bc80038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc8003e10 0x7f4bc8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8004b80 0x7f4bc8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0c94d0 con 0x7f4bc8003e10 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0cb7e0 con 0x7f4bc8004b80 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc800b910 con 0x7f4bdc154090 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc8003e10 0x7f4bc8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4bdbfff700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 0x7f4bc80038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.969+0000 7f4be0d0e700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8004b80 0x7f4bc8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bd3fff700 1 -- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 msgr2=0x7f4bc80038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc154090 0x7f4bc80038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bd3fff700 1 -- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8004b80 msgr2=0x7f4bc8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8004b80 0x7f4bc8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bd3fff700 1 -- 172.21.15.92:0/2592630816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80033f0 con 0x7f4bc8003e10 2023-05-16T07:14:54.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4be0d0e700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8004b80 0x7f4bc8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc8003e10 0x7f4bc8004690 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f4bdc067e30 tx=0x7f4bc4008040).ready entity=mon.0 client_cookie=ec773af1dda4a4a5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400b410 con 0x7f4bc8003e10 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc400d540 con 0x7f4bc8003e10 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4bdb7fe700 1 -- 172.21.15.92:0/2592630816 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400d8c0 con 0x7f4bc8003e10 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4be2771700 1 -- 172.21.15.92:0/2592630816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc8003e10 msgr2=0x7f4bc8004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4be2771700 1 --2- 172.21.15.92:0/2592630816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc8003e10 0x7f4bc8004690 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f4bdc067e30 tx=0x7f4bc4008040).stop 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4be0d0e700 1 -- 172.21.15.92:0/2592630816 reap_dead start 2023-05-16T07:14:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4be2771700 1 -- 172.21.15.92:0/2592630816 shutdown_connections 2023-05-16T07:14:54.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.970+0000 7f4be2771700 1 -- 172.21.15.92:0/2592630816 >> 172.21.15.92:0/2592630816 conn(0x7f4bdc067920 msgr2=0x7f4bdc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:54.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.971+0000 7f4be2771700 1 -- 172.21.15.92:0/2592630816 shutdown_connections 2023-05-16T07:14:54.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.971+0000 7f4be2771700 1 -- 172.21.15.92:0/2592630816 wait complete. 2023-05-16T07:14:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 Processor -- start 2023-05-16T07:14:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 -- start start 2023-05-16T07:14:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8003e10 0x7f4bdc14c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc1599b0 0x7f4bdc159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 0x7f4bdc157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0af5d0 con 0x7f4bdc156cb0 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc15a320 con 0x7f4bc8003e10 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be2771700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0aea00 con 0x7f4bdc1599b0 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be0d0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 0x7f4bdc157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be0d0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 0x7f4bdc157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44130/0 (socket says 172.21.15.92:44130) 2023-05-16T07:14:54.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.972+0000 7f4be0d0e700 1 -- 172.21.15.92:0/1599431578 learned_addr learned my addr 172.21.15.92:0/1599431578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4bdbfff700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8003e10 0x7f4bdc14c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc1599b0 0x7f4bdc159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be0d0e700 1 -- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc1599b0 msgr2=0x7f4bdc159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be0d0e700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc1599b0 0x7f4bdc159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be0d0e700 1 -- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8003e10 msgr2=0x7f4bdc14c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be0d0e700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8003e10 0x7f4bdc14c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be0d0e700 1 -- 172.21.15.92:0/1599431578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0aeca0 con 0x7f4bdc156cb0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4bd3fff700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc1599b0 0x7f4bdc159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be0d0e700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 0x7f4bdc157130 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f4bdc05efb0 tx=0x7f4bd4001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4010520 con 0x7f4bdc156cb0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd40106a0 con 0x7f4bdc156cb0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc0aef80 con 0x7f4bdc156cb0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4010ae0 con 0x7f4bdc156cb0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.973+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc157620 con 0x7f4bdc156cb0 2023-05-16T07:14:54.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.974+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc066730 con 0x7f4bdc156cb0 2023-05-16T07:14:54.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.974+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4bd4014430 con 0x7f4bdc156cb0 2023-05-16T07:14:54.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.975+0000 7f4bd8ff9700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bbc05ed90 0x7f4bbc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:54.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.975+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f4bd4079a00 con 0x7f4bdc156cb0 2023-05-16T07:14:54.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.975+0000 7f4bdbfff700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bbc05ed90 0x7f4bbc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:54.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.975+0000 7f4bdbfff700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bbc05ed90 0x7f4bbc061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4bcc006d80 tx=0x7f4bcc009060).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:54.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:54.979+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4bd40498f0 con 0x7f4bdc156cb0 2023-05-16T07:14:55.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.182+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4bdc0aa570 con 0x7f4bdc156cb0 2023-05-16T07:14:55.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.225+0000 7f4bd8ff9700 1 -- 172.21.15.92:0/1599431578 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4bdc0aa570 con 0x7f4bdc156cb0 2023-05-16T07:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bbc05ed90 msgr2=0x7f4bbc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bbc05ed90 0x7f4bbc061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4bcc006d80 tx=0x7f4bcc009060).stop 2023-05-16T07:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 msgr2=0x7f4bdc157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 0x7f4bdc157130 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f4bdc05efb0 tx=0x7f4bd4001400).stop 2023-05-16T07:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 shutdown_connections 2023-05-16T07:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4bbc05ed90 0x7f4bbc061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc156cb0 0x7f4bdc157130 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4bc8003e10 0x7f4bdc14c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 --2- 172.21.15.92:0/1599431578 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4bdc1599b0 0x7f4bdc159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 >> 172.21.15.92:0/1599431578 conn(0x7f4bdc067920 msgr2=0x7f4bdc14e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 shutdown_connections 2023-05-16T07:14:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.227+0000 7f4be2771700 1 -- 172.21.15.92:0/1599431578 wait complete. 2023-05-16T07:14:55.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:14:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.354+0000 7fa77bf51700 1 Processor -- start 2023-05-16T07:14:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.354+0000 7fa77bf51700 1 -- start start 2023-05-16T07:14:55.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa77bf51700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa774154090 0x7fa7740aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:55.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa77bf51700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774068160 0x7fa7740685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:55.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa77bf51700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7740c94d0 con 0x7fa774154470 2023-05-16T07:14:55.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa77bf51700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740cb7e0 con 0x7fa774154090 2023-05-16T07:14:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa77bf51700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa774068aa0 con 0x7fa774068160 2023-05-16T07:14:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa779ced700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa774154090 0x7fa7740aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa779ced700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa774154090 0x7fa7740aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44134/0 (socket says 172.21.15.92:44134) 2023-05-16T07:14:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa779ced700 1 -- 172.21.15.92:0/1148435696 learned_addr learned my addr 172.21.15.92:0/1148435696 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:55.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.355+0000 7fa779ced700 1 -- 172.21.15.92:0/1148435696 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774068160 msgr2=0x7fa7740685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:55.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa779ced700 1 --2- 172.21.15.92:0/1148435696 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774068160 0x7fa7740685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:55.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa779ced700 1 -- 172.21.15.92:0/1148435696 >> v1:172.21.15.158:6789/0 conn(0x7fa774154470 legacy=0x7fa7740a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:55.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa779ced700 1 -- 172.21.15.92:0/1148435696 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740c88c0 con 0x7fa774154090 2023-05-16T07:14:55.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa779ced700 1 --2- 172.21.15.92:0/1148435696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa7740aaca0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fa764009f20 tx=0x7fa764009840).ready entity=mon.0 client_cookie=2e620859129ca79a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:55.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa778ceb700 1 -- 172.21.15.92:0/1148435696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400e070 con 0x7fa774154090 2023-05-16T07:14:55.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa778ceb700 1 -- 172.21.15.92:0/1148435696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76400ba40 con 0x7fa774154090 2023-05-16T07:14:55.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa778ceb700 1 -- 172.21.15.92:0/1148435696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400bdc0 con 0x7fa774154090 2023-05-16T07:14:55.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 -- 172.21.15.92:0/1148435696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 msgr2=0x7fa7740aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:55.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1148435696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa7740aaca0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fa764009f20 tx=0x7fa764009840).stop 2023-05-16T07:14:55.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 -- 172.21.15.92:0/1148435696 shutdown_connections 2023-05-16T07:14:55.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1148435696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa7740aaca0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1148435696 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774068160 0x7fa7740685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:55.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 -- 172.21.15.92:0/1148435696 >> 172.21.15.92:0/1148435696 conn(0x7fa774067920 msgr2=0x7fa7740cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:55.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 -- 172.21.15.92:0/1148435696 shutdown_connections 2023-05-16T07:14:55.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.356+0000 7fa77bf51700 1 -- 172.21.15.92:0/1148435696 wait complete. 2023-05-16T07:14:55.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.357+0000 7fa77bf51700 1 Processor -- start 2023-05-16T07:14:55.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 -- start start 2023-05-16T07:14:55.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa774068160 0x7fa7740a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:55.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa774164430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774154470 0x7fa774152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774156770 con 0x7fa774154090 2023-05-16T07:14:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774155ba0 con 0x7fa774068160 2023-05-16T07:14:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77bf51700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa774155e40 con 0x7fa774154470 2023-05-16T07:14:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa7794ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa774164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa7794ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa774164430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44142/0 (socket says 172.21.15.92:44142) 2023-05-16T07:14:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa7794ec700 1 -- 172.21.15.92:0/1636978251 learned_addr learned my addr 172.21.15.92:0/1636978251 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa779ced700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa774068160 0x7fa7740a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.358+0000 7fa77a4ee700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774154470 0x7fa774152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa7794ec700 1 -- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774154470 msgr2=0x7fa774152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa7794ec700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774154470 0x7fa774152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa7794ec700 1 -- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa774068160 msgr2=0x7fa7740a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa7794ec700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa774068160 0x7fa7740a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa7794ec700 1 -- 172.21.15.92:0/1636978251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009ae0 con 0x7fa774154090 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa7794ec700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa774164430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fa76800b460 tx=0x7fa768006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800be60 con 0x7fa774154090 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768012410 con 0x7fa774154090 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa774156120 con 0x7fa774154090 2023-05-16T07:14:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680127b0 con 0x7fa774154090 2023-05-16T07:14:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.359+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7740aa4e0 con 0x7fa774154090 2023-05-16T07:14:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.360+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774066730 con 0x7fa774154090 2023-05-16T07:14:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.360+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa76800f030 con 0x7fa774154090 2023-05-16T07:14:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.361+0000 7fa771ffb700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa76005ed60 0x7fa760061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.361+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7fa7680798f0 con 0x7fa774154090 2023-05-16T07:14:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.361+0000 7fa779ced700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa76005ed60 0x7fa760061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:55.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.361+0000 7fa779ced700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa76005ed60 0x7fa760061220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa76400c010 tx=0x7fa764009840).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:55.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.365+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa76804ac60 con 0x7fa774154090 2023-05-16T07:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:55.569+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa77406bd40 con 0x7fa774154090 2023-05-16T07:14:56.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.234+0000 7fa771ffb700 1 -- 172.21.15.92:0/1636978251 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa76804e280 con 0x7fa774154090 2023-05-16T07:14:56.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa76005ed60 msgr2=0x7fa760061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa76005ed60 0x7fa760061220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa76400c010 tx=0x7fa764009840).stop 2023-05-16T07:14:56.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 msgr2=0x7fa774164430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa774164430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fa76800b460 tx=0x7fa768006f20).stop 2023-05-16T07:14:56.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 shutdown_connections 2023-05-16T07:14:56.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa76005ed60 0x7fa760061220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa774154470 0x7fa774152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa774068160 0x7fa7740a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 --2- 172.21.15.92:0/1636978251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa774154090 0x7fa774164430 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 >> 172.21.15.92:0/1636978251 conn(0x7fa774067920 msgr2=0x7fa7740b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:56.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 shutdown_connections 2023-05-16T07:14:56.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.236+0000 7fa77bf51700 1 -- 172.21.15.92:0/1636978251 wait complete. 2023-05-16T07:14:56.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:56.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.359+0000 7f5b6e6b2700 1 Processor -- start 2023-05-16T07:14:56.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.359+0000 7f5b6e6b2700 1 -- start start 2023-05-16T07:14:56.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.359+0000 7f5b6e6b2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5b6814e5e0 0x7f5b68157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.359+0000 7f5b6e6b2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680683d0 con 0x7f5b68157f40 2023-05-16T07:14:56.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.359+0000 7f5b6e6b2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680be060 con 0x7f5b6814e200 2023-05-16T07:14:56.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.359+0000 7f5b6e6b2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b68137470 con 0x7f5b6814e5e0 2023-05-16T07:14:56.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5b6814e5e0 0x7f5b68157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5b6814e5e0 0x7f5b68157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44150/0 (socket says 172.21.15.92:44150) 2023-05-16T07:14:56.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b6cc4f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b68157f40 0x7f5b6815ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54900/0 (socket says 172.21.15.92:54900) 2023-05-16T07:14:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 -- 172.21.15.92:0/1398674464 learned_addr learned my addr 172.21.15.92:0/1398674464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 -- 172.21.15.92:0/1398674464 >> v1:172.21.15.158:6789/0 conn(0x7f5b6814e200 legacy=0x7f5b68066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 -- 172.21.15.92:0/1398674464 >> v1:172.21.15.92:6789/0 conn(0x7f5b68157f40 legacy=0x7f5b6815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b66ffd700 1 -- 172.21.15.92:0/1398674464 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556509885 0 0) 0x7f5b680683d0 con 0x7f5b68157f40 2023-05-16T07:14:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 -- 172.21.15.92:0/1398674464 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68067560 con 0x7f5b6814e5e0 2023-05-16T07:14:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b677fe700 1 --2- 172.21.15.92:0/1398674464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6814e5e0 0x7f5b68157a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f5b5c006fd0 tx=0x7f5b5c0090f0).ready entity=mon.0 client_cookie=1480bb614d19655e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b66ffd700 1 -- 172.21.15.92:0/1398674464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00b600 con 0x7f5b6814e5e0 2023-05-16T07:14:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b66ffd700 1 -- 172.21.15.92:0/1398674464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c00b760 con 0x7f5b6814e5e0 2023-05-16T07:14:56.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.360+0000 7f5b66ffd700 1 -- 172.21.15.92:0/1398674464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00bae0 con 0x7f5b6814e5e0 2023-05-16T07:14:56.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/1398674464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6814e5e0 msgr2=0x7f5b68157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/1398674464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6814e5e0 0x7f5b68157a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f5b5c006fd0 tx=0x7f5b5c0090f0).stop 2023-05-16T07:14:56.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/1398674464 shutdown_connections 2023-05-16T07:14:56.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/1398674464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6814e5e0 0x7f5b68157a50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/1398674464 >> 172.21.15.92:0/1398674464 conn(0x7f5b680c7bc0 msgr2=0x7f5b680c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:56.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/1398674464 shutdown_connections 2023-05-16T07:14:56.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.361+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/1398674464 wait complete. 2023-05-16T07:14:56.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.362+0000 7f5b6e6b2700 1 Processor -- start 2023-05-16T07:14:56.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.362+0000 7f5b6e6b2700 1 -- start start 2023-05-16T07:14:56.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b6814e200 0x7f5b680bd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b6814e5e0 0x7f5b68156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 0x7f5b68156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68156f60 con 0x7f5b68157f40 2023-05-16T07:14:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68136460 con 0x7f5b6814e200 2023-05-16T07:14:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68164f00 con 0x7f5b6814e5e0 2023-05-16T07:14:56.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 0x7f5b68156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 0x7f5b68156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44158/0 (socket says 172.21.15.92:44158) 2023-05-16T07:14:56.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 -- 172.21.15.92:0/416217152 learned_addr learned my addr 172.21.15.92:0/416217152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:56.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b677fe700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b6814e5e0 0x7f5b68156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b67fff700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b6814e200 0x7f5b680bd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 -- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b6814e5e0 msgr2=0x7f5b68156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b6814e5e0 0x7f5b68156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 -- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b6814e200 msgr2=0x7f5b680bd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b6814e200 0x7f5b680bd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 -- 172.21.15.92:0/416217152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c009370 con 0x7f5b68157f40 2023-05-16T07:14:56.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6cc4f700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 0x7f5b68156a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f5b6000c730 tx=0x7f5b60001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:56.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60005c70 con 0x7f5b68157f40 2023-05-16T07:14:56.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b60005dd0 con 0x7f5b68157f40 2023-05-16T07:14:56.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b681642a0 con 0x7f5b68157f40 2023-05-16T07:14:56.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.363+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60007340 con 0x7f5b68157f40 2023-05-16T07:14:56.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.364+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b68164860 con 0x7f5b68157f40 2023-05-16T07:14:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.364+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6806c350 con 0x7f5b68157f40 2023-05-16T07:14:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.365+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5b60010030 con 0x7f5b68157f40 2023-05-16T07:14:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.365+0000 7f5b4ffff700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b5405ed90 0x7f5b54061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.365+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f5b60079bd0 con 0x7f5b68157f40 2023-05-16T07:14:56.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.365+0000 7f5b67fff700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b5405ed90 0x7f5b54061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.366+0000 7f5b67fff700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b5405ed90 0x7f5b54061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5b68067400 tx=0x7f5b58008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.369+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5b6004af40 con 0x7f5b68157f40 2023-05-16T07:14:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.571+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b68165040 con 0x7f5b68157f40 2023-05-16T07:14:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.572+0000 7f5b4ffff700 1 -- 172.21.15.92:0/416217152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f5b6004e560 con 0x7f5b68157f40 2023-05-16T07:14:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:56.576 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:14:54.221623+0000","modified":"2023-05-16T07:14:56.230335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/2827835616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2827835616},{"type":"v1","addr":"172.21.15.158:6837","nonce":2827835616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-05-16T07:14:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b5405ed90 msgr2=0x7f5b54061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b5405ed90 0x7f5b54061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5b68067400 tx=0x7f5b58008040).stop 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 msgr2=0x7f5b68156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 0x7f5b68156a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f5b6000c730 tx=0x7f5b60001520).stop 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 shutdown_connections 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5b5405ed90 0x7f5b54061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b68157f40 0x7f5b68156a70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b6814e200 0x7f5b680bd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 --2- 172.21.15.92:0/416217152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b6814e5e0 0x7f5b68156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.573+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 >> 172.21.15.92:0/416217152 conn(0x7f5b680c7bc0 msgr2=0x7f5b680a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:56.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.574+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 shutdown_connections 2023-05-16T07:14:56.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.574+0000 7f5b6e6b2700 1 -- 172.21.15.92:0/416217152 wait complete. 2023-05-16T07:14:56.579 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-05-16T07:14:56.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:14:56.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.696+0000 7f05d18f2700 1 Processor -- start 2023-05-16T07:14:56.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.696+0000 7f05d18f2700 1 -- start start 2023-05-16T07:14:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05d18f2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc153da0 0x7f05cc069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05d18f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05cc154180 0x7f05cc063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05d18f2700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f05cc062fa0 0x7f05cc063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05d18f2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc064100 con 0x7f05cc154180 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05d18f2700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc068620 con 0x7f05cc062fa0 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05d18f2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0622b0 con 0x7f05cc153da0 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05ca7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05cc154180 0x7f05cc063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05ca7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05cc154180 0x7f05cc063720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44162/0 (socket says 172.21.15.92:44162) 2023-05-16T07:14:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.697+0000 7f05ca7fc700 1 -- 172.21.15.92:0/3763493850 learned_addr learned my addr 172.21.15.92:0/3763493850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05cb7fe700 1 --2- 172.21.15.92:0/3763493850 >> v2:172.21.15.158:3300/0 conn(0x7f05cc062fa0 0x7f05cc063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05caffd700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc153da0 0x7f05cc069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05ca7fc700 1 -- 172.21.15.92:0/3763493850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc153da0 msgr2=0x7f05cc069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05ca7fc700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc153da0 0x7f05cc069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05ca7fc700 1 -- 172.21.15.92:0/3763493850 >> v2:172.21.15.158:3300/0 conn(0x7f05cc062fa0 msgr2=0x7f05cc063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05ca7fc700 1 --2- 172.21.15.92:0/3763493850 >> v2:172.21.15.158:3300/0 conn(0x7f05cc062fa0 0x7f05cc063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05ca7fc700 1 -- 172.21.15.92:0/3763493850 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc062b10 con 0x7f05cc154180 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05cb7fe700 1 --2- 172.21.15.92:0/3763493850 >> v2:172.21.15.158:3300/0 conn(0x7f05cc062fa0 0x7f05cc063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05caffd700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc153da0 0x7f05cc069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05ca7fc700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc154180 0x7f05cc063720 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f05cc0c3030 tx=0x7f05b40090f0).ready entity=mon.0 client_cookie=254a0ba8ab5f41c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05c9ffb700 1 -- 172.21.15.92:0/3763493850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400b600 con 0x7f05cc154180 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05c9ffb700 1 -- 172.21.15.92:0/3763493850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400b760 con 0x7f05cc154180 2023-05-16T07:14:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05c9ffb700 1 -- 172.21.15.92:0/3763493850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400bae0 con 0x7f05cc154180 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 -- 172.21.15.92:0/3763493850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc154180 msgr2=0x7f05cc063720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc154180 0x7f05cc063720 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f05cc0c3030 tx=0x7f05b40090f0).stop 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 -- 172.21.15.92:0/3763493850 shutdown_connections 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 --2- 172.21.15.92:0/3763493850 >> v2:172.21.15.158:3300/0 conn(0x7f05cc062fa0 0x7f05cc063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc153da0 0x7f05cc069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 --2- 172.21.15.92:0/3763493850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc154180 0x7f05cc063720 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 -- 172.21.15.92:0/3763493850 >> 172.21.15.92:0/3763493850 conn(0x7f05cc0c8aa0 msgr2=0x7f05cc0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 -- 172.21.15.92:0/3763493850 shutdown_connections 2023-05-16T07:14:56.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.698+0000 7f05d18f2700 1 -- 172.21.15.92:0/3763493850 wait complete. 2023-05-16T07:14:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 Processor -- start 2023-05-16T07:14:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 -- start start 2023-05-16T07:14:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc062fa0 0x7f05cc0c1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 0x7f05cc0c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05cc154180 0x7f05cc0bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc1574c0 con 0x7f05cc153da0 2023-05-16T07:14:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc1568f0 con 0x7f05cc154180 2023-05-16T07:14:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.700+0000 7f05d18f2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc156b90 con 0x7f05cc062fa0 2023-05-16T07:14:56.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 0x7f05cc0c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 0x7f05cc0c1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44170/0 (socket says 172.21.15.92:44170) 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 -- 172.21.15.92:0/1710723084 learned_addr learned my addr 172.21.15.92:0/1710723084 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05cb7fe700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05cc154180 0x7f05cc0bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05caffd700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc062fa0 0x7f05cc0c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 -- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc062fa0 msgr2=0x7f05cc0c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc062fa0 0x7f05cc0c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 -- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05cc154180 msgr2=0x7f05cc0bfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05cc154180 0x7f05cc0bfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 -- 172.21.15.92:0/1710723084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4009370 con 0x7f05cc153da0 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05ca7fc700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 0x7f05cc0c1d80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f05b4009130 tx=0x7f05b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400b8d0 con 0x7f05cc153da0 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400ba50 con 0x7f05cc153da0 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc156e70 con 0x7f05cc153da0 2023-05-16T07:14:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b401c670 con 0x7f05cc153da0 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.701+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc0c2270 con 0x7f05cc153da0 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.702+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc06c350 con 0x7f05cc153da0 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.702+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f05b400f070 con 0x7f05cc153da0 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.703+0000 7f05c2ffd700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05ac05ed90 0x7f05ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.703+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f05b407b3e0 con 0x7f05cc153da0 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.703+0000 7f05caffd700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05ac05ed90 0x7f05ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.704+0000 7f05caffd700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05ac05ed90 0x7f05ac061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f05bc00d6c0 tx=0x7f05bc007850).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.707+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f05b404c750 con 0x7f05cc153da0 2023-05-16T07:14:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.870+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05cc068340 con 0x7f05cc153da0 2023-05-16T07:14:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.871+0000 7f05c2ffd700 1 -- 172.21.15.92:0/1710723084 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17814 (secure 0 0 0) 0x7f05b4020020 con 0x7f05cc153da0 2023-05-16T07:14:56.876 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":62,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:14:55.223569+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:14:52.750879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-05-16T07:14:53.182846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:14:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05ac05ed90 msgr2=0x7f05ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05ac05ed90 0x7f05ac061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f05bc00d6c0 tx=0x7f05bc007850).stop 2023-05-16T07:14:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 msgr2=0x7f05cc0c1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 0x7f05cc0c1d80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f05b4009130 tx=0x7f05b4017000).stop 2023-05-16T07:14:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 shutdown_connections 2023-05-16T07:14:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05ac05ed90 0x7f05ac061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05cc062fa0 0x7f05cc0c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05cc154180 0x7f05cc0bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 --2- 172.21.15.92:0/1710723084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05cc153da0 0x7f05cc0c1d80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 >> 172.21.15.92:0/1710723084 conn(0x7f05cc0c8aa0 msgr2=0x7f05cc0c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 shutdown_connections 2023-05-16T07:14:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.873+0000 7f05d18f2700 1 -- 172.21.15.92:0/1710723084 wait complete. 2023-05-16T07:14:56.889 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:14:56.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:14:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.996+0000 7f532bb2d700 1 Processor -- start 2023-05-16T07:14:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.996+0000 7f532bb2d700 1 -- start start 2023-05-16T07:14:57.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f532bb2d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53240696a0 0x7f53240c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f532bb2d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240c6250 con 0x7f5324158b10 2023-05-16T07:14:57.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f532bb2d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240aaa90 con 0x7f53240c1790 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f532bb2d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53240c1be0 con 0x7f53240696a0 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53298c9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53240696a0 0x7f53240c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53290c8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5324158b10 0x7f5324069a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54924/0 (socket says 172.21.15.92:54924) 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53290c8700 1 -- 172.21.15.92:0/414286536 learned_addr learned my addr 172.21.15.92:0/414286536 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53298c9700 1 --2- 172.21.15.92:0/414286536 >> v2:172.21.15.92:3300/0 conn(0x7f53240696a0 0x7f53240c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44176/0 (socket says 172.21.15.92:44176) 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53298c9700 1 -- 172.21.15.92:0/414286536 >> v1:172.21.15.158:6789/0 conn(0x7f53240c1790 legacy=0x7f53240aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53298c9700 1 -- 172.21.15.92:0/414286536 >> v1:172.21.15.92:6789/0 conn(0x7f5324158b10 legacy=0x7f5324069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53298c9700 1 -- 172.21.15.92:0/414286536 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532406a4a0 con 0x7f53240696a0 2023-05-16T07:14:57.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53288c7700 1 -- 172.21.15.92:0/414286536 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328100191 0 0) 0x7f53240c6250 con 0x7f5324158b10 2023-05-16T07:14:57.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.997+0000 7f53298c9700 1 --2- 172.21.15.92:0/414286536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53240696a0 0x7f53240c5c70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f5320009760 tx=0x7f5320009060).ready entity=mon.0 client_cookie=35f308e9f5d10c47 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f53288c7700 1 -- 172.21.15.92:0/414286536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000b520 con 0x7f53240696a0 2023-05-16T07:14:57.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f53288c7700 1 -- 172.21.15.92:0/414286536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532000b680 con 0x7f53240696a0 2023-05-16T07:14:57.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f53288c7700 1 -- 172.21.15.92:0/414286536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000ba00 con 0x7f53240696a0 2023-05-16T07:14:57.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 -- 172.21.15.92:0/414286536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53240696a0 msgr2=0x7f53240c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 --2- 172.21.15.92:0/414286536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53240696a0 0x7f53240c5c70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f5320009760 tx=0x7f5320009060).stop 2023-05-16T07:14:57.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 -- 172.21.15.92:0/414286536 shutdown_connections 2023-05-16T07:14:57.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 --2- 172.21.15.92:0/414286536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53240696a0 0x7f53240c5c70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 -- 172.21.15.92:0/414286536 >> 172.21.15.92:0/414286536 conn(0x7f53240cb700 msgr2=0x7f53240c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 -- 172.21.15.92:0/414286536 shutdown_connections 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.998+0000 7f532bb2d700 1 -- 172.21.15.92:0/414286536 wait complete. 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.999+0000 7f532bb2d700 1 Processor -- start 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:56.999+0000 7f532bb2d700 1 -- start start 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532bb2d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53240696a0 0x7f53240ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532bb2d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53240c1790 0x7f5324153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532bb2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 0x7f53240c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532bb2d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532415c2a0 con 0x7f5324158b10 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532bb2d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532415b690 con 0x7f53240c1790 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532bb2d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532415b970 con 0x7f53240696a0 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 0x7f53240c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 0x7f53240c3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44184/0 (socket says 172.21.15.92:44184) 2023-05-16T07:14:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 -- 172.21.15.92:0/1583377535 learned_addr learned my addr 172.21.15.92:0/1583377535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f53290c8700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53240c1790 0x7f5324153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f53298c9700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53240696a0 0x7f53240ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 -- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53240696a0 msgr2=0x7f53240ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53240696a0 0x7f53240ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 -- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53240c1790 msgr2=0x7f5324153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53240c1790 0x7f5324153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 -- 172.21.15.92:0/1583377535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200092a0 con 0x7f5324158b10 2023-05-16T07:14:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.000+0000 7f532a0ca700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 0x7f53240c3e90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f531c00c810 tx=0x7f531c002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.001+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c010510 con 0x7f5324158b10 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.001+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c0106b0 con 0x7f5324158b10 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.001+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532415bc50 con 0x7f5324158b10 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.001+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c010a70 con 0x7f5324158b10 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.001+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53240aeb10 con 0x7f5324158b10 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.002+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53240aa400 con 0x7f5324158b10 2023-05-16T07:14:57.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.002+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f531c019030 con 0x7f5324158b10 2023-05-16T07:14:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.002+0000 7f5319ffb700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530805ed90 0x7f5308061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.002+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f531c079530 con 0x7f5324158b10 2023-05-16T07:14:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.003+0000 7f53298c9700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530805ed90 0x7f5308061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.003+0000 7f53298c9700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530805ed90 0x7f5308061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f532000b2e0 tx=0x7f5320017000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.006+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f531c0488f0 con 0x7f5324158b10 2023-05-16T07:14:57.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.165+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53240666c0 con 0x7f530805ed90 2023-05-16T07:14:57.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.177+0000 7f5319ffb700 1 -- 172.21.15.92:0/1583377535 <== mgr.5275 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361339 (secure 0 0 0) 0x7f53240666c0 con 0x7f530805ed90 2023-05-16T07:14:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:57.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530805ed90 msgr2=0x7f5308061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530805ed90 0x7f5308061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f532000b2e0 tx=0x7f5320017000).stop 2023-05-16T07:14:57.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 msgr2=0x7f53240c3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 0x7f53240c3e90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f531c00c810 tx=0x7f531c002560).stop 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 shutdown_connections 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530805ed90 0x7f5308061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53240696a0 0x7f53240ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5324158b10 0x7f53240c3e90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 --2- 172.21.15.92:0/1583377535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53240c1790 0x7f5324153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 >> 172.21.15.92:0/1583377535 conn(0x7f53240cb700 msgr2=0x7f53240ac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:57.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 shutdown_connections 2023-05-16T07:14:57.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.180+0000 7f532bb2d700 1 -- 172.21.15.92:0/1583377535 wait complete. 2023-05-16T07:14:57.194 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:14:57.196 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:14:55.757931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":130440,"kb_used_data":5640,"kb_used_omap":0,"kb_used_meta":124736,"kb_avail":754844280,"statfs":{"total":773094113280,"available":772960542720,"internally_reserved":0,"allocated":5775360,"data_stored":2825257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":127729664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15001262,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-723,"num_write_kb":-15553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.587893"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.237311+0000","last_change":"2023-05-16T07:14:54.237311+0000","last_active":"2023-05-16T07:14:54.237311+0000","last_peered":"2023-05-16T07:14:54.237311+0000","last_clean":"2023-05-16T07:14:54.237311+0000","last_became_active":"2023-05-16T07:14:54.237033+0000","last_became_peered":"2023-05-16T07:14:54.237033+0000","last_unstale":"2023-05-16T07:14:54.237311+0000","last_undegraded":"2023-05-16T07:14:54.237311+0000","last_fullsized":"2023-05-16T07:14:54.237311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248753+0000","last_change":"2023-05-16T07:14:53.875065+0000","last_active":"2023-05-16T07:14:54.248753+0000","last_peered":"2023-05-16T07:14:54.248753+0000","last_clean":"2023-05-16T07:14:54.248753+0000","last_became_active":"2023-05-16T07:14:53.874454+0000","last_became_peered":"2023-05-16T07:14:53.874454+0000","last_unstale":"2023-05-16T07:14:54.248753+0000","last_undegraded":"2023-05-16T07:14:54.248753+0000","last_fullsized":"2023-05-16T07:14:54.248753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247339+0000","last_change":"2023-05-16T07:14:54.247339+0000","last_active":"2023-05-16T07:14:54.247339+0000","last_peered":"2023-05-16T07:14:54.247339+0000","last_clean":"2023-05-16T07:14:54.247339+0000","last_became_active":"2023-05-16T07:14:54.246922+0000","last_became_peered":"2023-05-16T07:14:54.246922+0000","last_unstale":"2023-05-16T07:14:54.247339+0000","last_undegraded":"2023-05-16T07:14:54.247339+0000","last_fullsized":"2023-05-16T07:14:54.247339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875808+0000","last_change":"2023-05-16T07:14:53.874136+0000","last_active":"2023-05-16T07:14:53.875808+0000","last_peered":"2023-05-16T07:14:53.875808+0000","last_clean":"2023-05-16T07:14:53.875808+0000","last_became_active":"2023-05-16T07:14:53.873784+0000","last_became_peered":"2023-05-16T07:14:53.873784+0000","last_unstale":"2023-05-16T07:14:53.875808+0000","last_undegraded":"2023-05-16T07:14:53.875808+0000","last_fullsized":"2023-05-16T07:14:53.875808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246357+0000","last_change":"2023-05-16T07:14:54.246357+0000","last_active":"2023-05-16T07:14:54.246357+0000","last_peered":"2023-05-16T07:14:54.246357+0000","last_clean":"2023-05-16T07:14:54.246357+0000","last_became_active":"2023-05-16T07:14:54.246078+0000","last_became_peered":"2023-05-16T07:14:54.246078+0000","last_unstale":"2023-05-16T07:14:54.246357+0000","last_undegraded":"2023-05-16T07:14:54.246357+0000","last_fullsized":"2023-05-16T07:14:54.246357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.874675+0000","last_change":"2023-05-16T07:14:53.872565+0000","last_active":"2023-05-16T07:14:53.874675+0000","last_peered":"2023-05-16T07:14:53.874675+0000","last_clean":"2023-05-16T07:14:53.874675+0000","last_became_active":"2023-05-16T07:14:53.872216+0000","last_became_peered":"2023-05-16T07:14:53.872216+0000","last_unstale":"2023-05-16T07:14:53.874675+0000","last_undegraded":"2023-05-16T07:14:53.874675+0000","last_fullsized":"2023-05-16T07:14:53.874675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.239789+0000","last_change":"2023-05-16T07:14:54.239789+0000","last_active":"2023-05-16T07:14:54.239789+0000","last_peered":"2023-05-16T07:14:54.239789+0000","last_clean":"2023-05-16T07:14:54.239789+0000","last_became_active":"2023-05-16T07:14:54.239492+0000","last_became_peered":"2023-05-16T07:14:54.239492+0000","last_unstale":"2023-05-16T07:14:54.239789+0000","last_undegraded":"2023-05-16T07:14:54.239789+0000","last_fullsized":"2023-05-16T07:14:54.239789+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250339+0000","last_change":"2023-05-16T07:14:53.876780+0000","last_active":"2023-05-16T07:14:54.250339+0000","last_peered":"2023-05-16T07:14:54.250339+0000","last_clean":"2023-05-16T07:14:54.250339+0000","last_became_active":"2023-05-16T07:14:53.876143+0000","last_became_peered":"2023-05-16T07:14:53.876143+0000","last_unstale":"2023-05-16T07:14:54.250339+0000","last_undegraded":"2023-05-16T07:14:54.250339+0000","last_fullsized":"2023-05-16T07:14:54.250339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.242290+0000","last_change":"2023-05-16T07:14:54.242290+0000","last_active":"2023-05-16T07:14:54.242290+0000","last_peered":"2023-05-16T07:14:54.242290+0000","last_clean":"2023-05-16T07:14:54.242290+0000","last_became_active":"2023-05-16T07:14:54.242046+0000","last_became_peered":"2023-05-16T07:14:54.242046+0000","last_unstale":"2023-05-16T07:14:54.242290+0000","last_undegraded":"2023-05-16T07:14:54.242290+0000","last_fullsized":"2023-05-16T07:14:54.242290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.230463+0000","last_change":"2023-05-16T07:14:53.866415+0000","last_active":"2023-05-16T07:14:54.230463+0000","last_peered":"2023-05-16T07:14:54.230463+0000","last_clean":"2023-05-16T07:14:54.230463+0000","last_became_active":"2023-05-16T07:14:53.866109+0000","last_became_peered":"2023-05-16T07:14:53.866109+0000","last_unstale":"2023-05-16T07:14:54.230463+0000","last_undegraded":"2023-05-16T07:14:54.230463+0000","last_fullsized":"2023-05-16T07:14:54.230463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253768+0000","last_change":"2023-05-16T07:14:54.253768+0000","last_active":"2023-05-16T07:14:54.253768+0000","last_peered":"2023-05-16T07:14:54.253768+0000","last_clean":"2023-05-16T07:14:54.253768+0000","last_became_active":"2023-05-16T07:14:54.253513+0000","last_became_peered":"2023-05-16T07:14:54.253513+0000","last_unstale":"2023-05-16T07:14:54.253768+0000","last_undegraded":"2023-05-16T07:14:54.253768+0000","last_fullsized":"2023-05-16T07:14:54.253768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235592+0000","last_change":"2023-05-16T07:14:53.871132+0000","last_active":"2023-05-16T07:14:54.235592+0000","last_peered":"2023-05-16T07:14:54.235592+0000","last_clean":"2023-05-16T07:14:54.235592+0000","last_became_active":"2023-05-16T07:14:53.870861+0000","last_became_peered":"2023-05-16T07:14:53.870861+0000","last_unstale":"2023-05-16T07:14:54.235592+0000","last_undegraded":"2023-05-16T07:14:54.235592+0000","last_fullsized":"2023-05-16T07:14:54.235592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.237305+0000","last_change":"2023-05-16T07:14:54.237305+0000","last_active":"2023-05-16T07:14:54.237305+0000","last_peered":"2023-05-16T07:14:54.237305+0000","last_clean":"2023-05-16T07:14:54.237305+0000","last_became_active":"2023-05-16T07:14:54.237028+0000","last_became_peered":"2023-05-16T07:14:54.237028+0000","last_unstale":"2023-05-16T07:14:54.237305+0000","last_undegraded":"2023-05-16T07:14:54.237305+0000","last_fullsized":"2023-05-16T07:14:54.237305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.879863+0000","last_change":"2023-05-16T07:14:53.879159+0000","last_active":"2023-05-16T07:14:53.879863+0000","last_peered":"2023-05-16T07:14:53.879863+0000","last_clean":"2023-05-16T07:14:53.879863+0000","last_became_active":"2023-05-16T07:14:53.878901+0000","last_became_peered":"2023-05-16T07:14:53.878901+0000","last_unstale":"2023-05-16T07:14:53.879863+0000","last_undegraded":"2023-05-16T07:14:53.879863+0000","last_fullsized":"2023-05-16T07:14:53.879863+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250642+0000","last_change":"2023-05-16T07:14:54.250642+0000","last_active":"2023-05-16T07:14:54.250642+0000","last_peered":"2023-05-16T07:14:54.250642+0000","last_clean":"2023-05-16T07:14:54.250642+0000","last_became_active":"2023-05-16T07:14:54.250351+0000","last_became_peered":"2023-05-16T07:14:54.250351+0000","last_unstale":"2023-05-16T07:14:54.250642+0000","last_undegraded":"2023-05-16T07:14:54.250642+0000","last_fullsized":"2023-05-16T07:14:54.250642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881670+0000","last_change":"2023-05-16T07:14:53.875938+0000","last_active":"2023-05-16T07:14:53.881670+0000","last_peered":"2023-05-16T07:14:53.881670+0000","last_clean":"2023-05-16T07:14:53.881670+0000","last_became_active":"2023-05-16T07:14:53.875596+0000","last_became_peered":"2023-05-16T07:14:53.875596+0000","last_unstale":"2023-05-16T07:14:53.881670+0000","last_undegraded":"2023-05-16T07:14:53.881670+0000","last_fullsized":"2023-05-16T07:14:53.881670+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.252104+0000","last_change":"2023-05-16T07:14:54.252104+0000","last_active":"2023-05-16T07:14:54.252104+0000","last_peered":"2023-05-16T07:14:54.252104+0000","last_clean":"2023-05-16T07:14:54.252104+0000","last_became_active":"2023-05-16T07:14:54.251764+0000","last_became_peered":"2023-05-16T07:14:54.251764+0000","last_unstale":"2023-05-16T07:14:54.252104+0000","last_undegraded":"2023-05-16T07:14:54.252104+0000","last_fullsized":"2023-05-16T07:14:54.252104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884535+0000","last_change":"2023-05-16T07:14:53.880001+0000","last_active":"2023-05-16T07:14:53.884535+0000","last_peered":"2023-05-16T07:14:53.884535+0000","last_clean":"2023-05-16T07:14:53.884535+0000","last_became_active":"2023-05-16T07:14:53.879721+0000","last_became_peered":"2023-05-16T07:14:53.879721+0000","last_unstale":"2023-05-16T07:14:53.884535+0000","last_undegraded":"2023-05-16T07:14:53.884535+0000","last_fullsized":"2023-05-16T07:14:53.884535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245462+0000","last_change":"2023-05-16T07:14:54.245462+0000","last_active":"2023-05-16T07:14:54.245462+0000","last_peered":"2023-05-16T07:14:54.245462+0000","last_clean":"2023-05-16T07:14:54.245462+0000","last_became_active":"2023-05-16T07:14:54.245200+0000","last_became_peered":"2023-05-16T07:14:54.245200+0000","last_unstale":"2023-05-16T07:14:54.245462+0000","last_undegraded":"2023-05-16T07:14:54.245462+0000","last_fullsized":"2023-05-16T07:14:54.245462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881565+0000","last_change":"2023-05-16T07:14:53.875819+0000","last_active":"2023-05-16T07:14:53.881565+0000","last_peered":"2023-05-16T07:14:53.881565+0000","last_clean":"2023-05-16T07:14:53.881565+0000","last_became_active":"2023-05-16T07:14:53.874542+0000","last_became_peered":"2023-05-16T07:14:53.874542+0000","last_unstale":"2023-05-16T07:14:53.881565+0000","last_undegraded":"2023-05-16T07:14:53.881565+0000","last_fullsized":"2023-05-16T07:14:53.881565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249584+0000","last_change":"2023-05-16T07:14:54.249584+0000","last_active":"2023-05-16T07:14:54.249584+0000","last_peered":"2023-05-16T07:14:54.249584+0000","last_clean":"2023-05-16T07:14:54.249584+0000","last_became_active":"2023-05-16T07:14:54.249291+0000","last_became_peered":"2023-05-16T07:14:54.249291+0000","last_unstale":"2023-05-16T07:14:54.249584+0000","last_undegraded":"2023-05-16T07:14:54.249584+0000","last_fullsized":"2023-05-16T07:14:54.249584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884480+0000","last_change":"2023-05-16T07:14:53.879060+0000","last_active":"2023-05-16T07:14:53.884480+0000","last_peered":"2023-05-16T07:14:53.884480+0000","last_clean":"2023-05-16T07:14:53.884480+0000","last_became_active":"2023-05-16T07:14:53.878780+0000","last_became_peered":"2023-05-16T07:14:53.878780+0000","last_unstale":"2023-05-16T07:14:53.884480+0000","last_undegraded":"2023-05-16T07:14:53.884480+0000","last_fullsized":"2023-05-16T07:14:53.884480+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245958+0000","last_change":"2023-05-16T07:14:54.245958+0000","last_active":"2023-05-16T07:14:54.245958+0000","last_peered":"2023-05-16T07:14:54.245958+0000","last_clean":"2023-05-16T07:14:54.245958+0000","last_became_active":"2023-05-16T07:14:54.245660+0000","last_became_peered":"2023-05-16T07:14:54.245660+0000","last_unstale":"2023-05-16T07:14:54.245958+0000","last_undegraded":"2023-05-16T07:14:54.245958+0000","last_fullsized":"2023-05-16T07:14:54.245958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.243677+0000","last_change":"2023-05-16T07:14:53.872808+0000","last_active":"2023-05-16T07:14:54.243677+0000","last_peered":"2023-05-16T07:14:54.243677+0000","last_clean":"2023-05-16T07:14:54.243677+0000","last_became_active":"2023-05-16T07:14:53.872524+0000","last_became_peered":"2023-05-16T07:14:53.872524+0000","last_unstale":"2023-05-16T07:14:54.243677+0000","last_undegraded":"2023-05-16T07:14:54.243677+0000","last_fullsized":"2023-05-16T07:14:54.243677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253537+0000","last_change":"2023-05-16T07:14:54.253537+0000","last_active":"2023-05-16T07:14:54.253537+0000","last_peered":"2023-05-16T07:14:54.253537+0000","last_clean":"2023-05-16T07:14:54.253537+0000","last_became_active":"2023-05-16T07:14:54.253276+0000","last_became_peered":"2023-05-16T07:14:54.253276+0000","last_unstale":"2023-05-16T07:14:54.253537+0000","last_undegraded":"2023-05-16T07:14:54.253537+0000","last_fullsized":"2023-05-16T07:14:54.253537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.228056+0000","last_change":"2023-05-16T07:14:53.868058+0000","last_active":"2023-05-16T07:14:54.228056+0000","last_peered":"2023-05-16T07:14:54.228056+0000","last_clean":"2023-05-16T07:14:54.228056+0000","last_became_active":"2023-05-16T07:14:53.867773+0000","last_became_peered":"2023-05-16T07:14:53.867773+0000","last_unstale":"2023-05-16T07:14:54.228056+0000","last_undegraded":"2023-05-16T07:14:54.228056+0000","last_fullsized":"2023-05-16T07:14:54.228056+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.251222+0000","last_change":"2023-05-16T07:14:54.251222+0000","last_active":"2023-05-16T07:14:54.251222+0000","last_peered":"2023-05-16T07:14:54.251222+0000","last_clean":"2023-05-16T07:14:54.251222+0000","last_became_active":"2023-05-16T07:14:54.250922+0000","last_became_peered":"2023-05-16T07:14:54.250922+0000","last_unstale":"2023-05-16T07:14:54.251222+0000","last_undegraded":"2023-05-16T07:14:54.251222+0000","last_fullsized":"2023-05-16T07:14:54.251222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884577+0000","last_change":"2023-05-16T07:14:53.880178+0000","last_active":"2023-05-16T07:14:53.884577+0000","last_peered":"2023-05-16T07:14:53.884577+0000","last_clean":"2023-05-16T07:14:53.884577+0000","last_became_active":"2023-05-16T07:14:53.879915+0000","last_became_peered":"2023-05-16T07:14:53.879915+0000","last_unstale":"2023-05-16T07:14:53.884577+0000","last_undegraded":"2023-05-16T07:14:53.884577+0000","last_fullsized":"2023-05-16T07:14:53.884577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245930+0000","last_change":"2023-05-16T07:14:54.245930+0000","last_active":"2023-05-16T07:14:54.245930+0000","last_peered":"2023-05-16T07:14:54.245930+0000","last_clean":"2023-05-16T07:14:54.245930+0000","last_became_active":"2023-05-16T07:14:54.245647+0000","last_became_peered":"2023-05-16T07:14:54.245647+0000","last_unstale":"2023-05-16T07:14:54.245930+0000","last_undegraded":"2023-05-16T07:14:54.245930+0000","last_fullsized":"2023-05-16T07:14:54.245930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872588+0000","last_change":"2023-05-16T07:14:53.869230+0000","last_active":"2023-05-16T07:14:53.872588+0000","last_peered":"2023-05-16T07:14:53.872588+0000","last_clean":"2023-05-16T07:14:53.872588+0000","last_became_active":"2023-05-16T07:14:53.868934+0000","last_became_peered":"2023-05-16T07:14:53.868934+0000","last_unstale":"2023-05-16T07:14:53.872588+0000","last_undegraded":"2023-05-16T07:14:53.872588+0000","last_fullsized":"2023-05-16T07:14:53.872588+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240256+0000","last_change":"2023-05-16T07:14:54.240256+0000","last_active":"2023-05-16T07:14:54.240256+0000","last_peered":"2023-05-16T07:14:54.240256+0000","last_clean":"2023-05-16T07:14:54.240256+0000","last_became_active":"2023-05-16T07:14:54.240019+0000","last_became_peered":"2023-05-16T07:14:54.240019+0000","last_unstale":"2023-05-16T07:14:54.240256+0000","last_undegraded":"2023-05-16T07:14:54.240256+0000","last_fullsized":"2023-05-16T07:14:54.240256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247935+0000","last_change":"2023-05-16T07:14:53.878426+0000","last_active":"2023-05-16T07:14:54.247935+0000","last_peered":"2023-05-16T07:14:54.247935+0000","last_clean":"2023-05-16T07:14:54.247935+0000","last_became_active":"2023-05-16T07:14:53.878132+0000","last_became_peered":"2023-05-16T07:14:53.878132+0000","last_unstale":"2023-05-16T07:14:54.247935+0000","last_undegraded":"2023-05-16T07:14:54.247935+0000","last_fullsized":"2023-05-16T07:14:54.247935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246812+0000","last_change":"2023-05-16T07:14:54.246812+0000","last_active":"2023-05-16T07:14:54.246812+0000","last_peered":"2023-05-16T07:14:54.246812+0000","last_clean":"2023-05-16T07:14:54.246812+0000","last_became_active":"2023-05-16T07:14:54.246424+0000","last_became_peered":"2023-05-16T07:14:54.246424+0000","last_unstale":"2023-05-16T07:14:54.246812+0000","last_undegraded":"2023-05-16T07:14:54.246812+0000","last_fullsized":"2023-05-16T07:14:54.246812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872248+0000","last_change":"2023-05-16T07:14:53.868351+0000","last_active":"2023-05-16T07:14:53.872248+0000","last_peered":"2023-05-16T07:14:53.872248+0000","last_clean":"2023-05-16T07:14:53.872248+0000","last_became_active":"2023-05-16T07:14:53.868064+0000","last_became_peered":"2023-05-16T07:14:53.868064+0000","last_unstale":"2023-05-16T07:14:53.872248+0000","last_undegraded":"2023-05-16T07:14:53.872248+0000","last_fullsized":"2023-05-16T07:14:53.872248+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246398+0000","last_change":"2023-05-16T07:14:54.246398+0000","last_active":"2023-05-16T07:14:54.246398+0000","last_peered":"2023-05-16T07:14:54.246398+0000","last_clean":"2023-05-16T07:14:54.246398+0000","last_became_active":"2023-05-16T07:14:54.246112+0000","last_became_peered":"2023-05-16T07:14:54.246112+0000","last_unstale":"2023-05-16T07:14:54.246398+0000","last_undegraded":"2023-05-16T07:14:54.246398+0000","last_fullsized":"2023-05-16T07:14:54.246398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.879661+0000","last_change":"2023-05-16T07:14:53.878824+0000","last_active":"2023-05-16T07:14:53.879661+0000","last_peered":"2023-05-16T07:14:53.879661+0000","last_clean":"2023-05-16T07:14:53.879661+0000","last_became_active":"2023-05-16T07:14:53.878478+0000","last_became_peered":"2023-05-16T07:14:53.878478+0000","last_unstale":"2023-05-16T07:14:53.879661+0000","last_undegraded":"2023-05-16T07:14:53.879661+0000","last_fullsized":"2023-05-16T07:14:53.879661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253187+0000","last_change":"2023-05-16T07:14:54.253187+0000","last_active":"2023-05-16T07:14:54.253187+0000","last_peered":"2023-05-16T07:14:54.253187+0000","last_clean":"2023-05-16T07:14:54.253187+0000","last_became_active":"2023-05-16T07:14:54.252930+0000","last_became_peered":"2023-05-16T07:14:54.252930+0000","last_unstale":"2023-05-16T07:14:54.253187+0000","last_undegraded":"2023-05-16T07:14:54.253187+0000","last_fullsized":"2023-05-16T07:14:54.253187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2d","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.229345+0000","last_change":"2023-05-16T07:14:53.869498+0000","last_active":"2023-05-16T07:14:54.229345+0000","last_peered":"2023-05-16T07:14:54.229345+0000","last_clean":"2023-05-16T07:14:54.229345+0000","last_became_active":"2023-05-16T07:14:53.869288+0000","last_became_peered":"2023-05-16T07:14:53.869288+0000","last_unstale":"2023-05-16T07:14:54.229345+0000","last_undegraded":"2023-05-16T07:14:54.229345+0000","last_fullsized":"2023-05-16T07:14:54.229345+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249790+0000","last_change":"2023-05-16T07:14:54.249790+0000","last_active":"2023-05-16T07:14:54.249790+0000","last_peered":"2023-05-16T07:14:54.249790+0000","last_clean":"2023-05-16T07:14:54.249790+0000","last_became_active":"2023-05-16T07:14:54.248494+0000","last_became_peered":"2023-05-16T07:14:54.248494+0000","last_unstale":"2023-05-16T07:14:54.249790+0000","last_undegraded":"2023-05-16T07:14:54.249790+0000","last_fullsized":"2023-05-16T07:14:54.249790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881698+0000","last_change":"2023-05-16T07:14:53.878541+0000","last_active":"2023-05-16T07:14:53.881698+0000","last_peered":"2023-05-16T07:14:53.881698+0000","last_clean":"2023-05-16T07:14:53.881698+0000","last_became_active":"2023-05-16T07:14:53.878285+0000","last_became_peered":"2023-05-16T07:14:53.878285+0000","last_unstale":"2023-05-16T07:14:53.881698+0000","last_undegraded":"2023-05-16T07:14:53.881698+0000","last_fullsized":"2023-05-16T07:14:53.881698+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.234197+0000","last_change":"2023-05-16T07:14:54.234197+0000","last_active":"2023-05-16T07:14:54.234197+0000","last_peered":"2023-05-16T07:14:54.234197+0000","last_clean":"2023-05-16T07:14:54.234197+0000","last_became_active":"2023-05-16T07:14:54.233919+0000","last_became_peered":"2023-05-16T07:14:54.233919+0000","last_unstale":"2023-05-16T07:14:54.234197+0000","last_undegraded":"2023-05-16T07:14:54.234197+0000","last_fullsized":"2023-05-16T07:14:54.234197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235314+0000","last_change":"2023-05-16T07:14:53.869640+0000","last_active":"2023-05-16T07:14:54.235314+0000","last_peered":"2023-05-16T07:14:54.235314+0000","last_clean":"2023-05-16T07:14:54.235314+0000","last_became_active":"2023-05-16T07:14:53.869362+0000","last_became_peered":"2023-05-16T07:14:53.869362+0000","last_unstale":"2023-05-16T07:14:54.235314+0000","last_undegraded":"2023-05-16T07:14:54.235314+0000","last_fullsized":"2023-05-16T07:14:54.235314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255586+0000","last_change":"2023-05-16T07:14:54.255586+0000","last_active":"2023-05-16T07:14:54.255586+0000","last_peered":"2023-05-16T07:14:54.255586+0000","last_clean":"2023-05-16T07:14:54.255586+0000","last_became_active":"2023-05-16T07:14:54.255371+0000","last_became_peered":"2023-05-16T07:14:54.255371+0000","last_unstale":"2023-05-16T07:14:54.255586+0000","last_undegraded":"2023-05-16T07:14:54.255586+0000","last_fullsized":"2023-05-16T07:14:54.255586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.876203+0000","last_change":"2023-05-16T07:14:53.868462+0000","last_active":"2023-05-16T07:14:53.876203+0000","last_peered":"2023-05-16T07:14:53.876203+0000","last_clean":"2023-05-16T07:14:53.876203+0000","last_became_active":"2023-05-16T07:14:53.868212+0000","last_became_peered":"2023-05-16T07:14:53.868212+0000","last_unstale":"2023-05-16T07:14:53.876203+0000","last_undegraded":"2023-05-16T07:14:53.876203+0000","last_fullsized":"2023-05-16T07:14:53.876203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245021+0000","last_change":"2023-05-16T07:14:54.245021+0000","last_active":"2023-05-16T07:14:54.245021+0000","last_peered":"2023-05-16T07:14:54.245021+0000","last_clean":"2023-05-16T07:14:54.245021+0000","last_became_active":"2023-05-16T07:14:54.244732+0000","last_became_peered":"2023-05-16T07:14:54.244732+0000","last_unstale":"2023-05-16T07:14:54.245021+0000","last_undegraded":"2023-05-16T07:14:54.245021+0000","last_fullsized":"2023-05-16T07:14:54.245021+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.873704+0000","last_change":"2023-05-16T07:14:53.871416+0000","last_active":"2023-05-16T07:14:53.873704+0000","last_peered":"2023-05-16T07:14:53.873704+0000","last_clean":"2023-05-16T07:14:53.873704+0000","last_became_active":"2023-05-16T07:14:53.871005+0000","last_became_peered":"2023-05-16T07:14:53.871005+0000","last_unstale":"2023-05-16T07:14:53.873704+0000","last_undegraded":"2023-05-16T07:14:53.873704+0000","last_fullsized":"2023-05-16T07:14:53.873704+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248977+0000","last_change":"2023-05-16T07:14:54.248977+0000","last_active":"2023-05-16T07:14:54.248977+0000","last_peered":"2023-05-16T07:14:54.248977+0000","last_clean":"2023-05-16T07:14:54.248977+0000","last_became_active":"2023-05-16T07:14:54.246294+0000","last_became_peered":"2023-05-16T07:14:54.246294+0000","last_unstale":"2023-05-16T07:14:54.248977+0000","last_undegraded":"2023-05-16T07:14:54.248977+0000","last_fullsized":"2023-05-16T07:14:54.248977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.873661+0000","last_change":"2023-05-16T07:14:53.871185+0000","last_active":"2023-05-16T07:14:53.873661+0000","last_peered":"2023-05-16T07:14:53.873661+0000","last_clean":"2023-05-16T07:14:53.873661+0000","last_became_active":"2023-05-16T07:14:53.870907+0000","last_became_peered":"2023-05-16T07:14:53.870907+0000","last_unstale":"2023-05-16T07:14:53.873661+0000","last_undegraded":"2023-05-16T07:14:53.873661+0000","last_fullsized":"2023-05-16T07:14:53.873661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246683+0000","last_change":"2023-05-16T07:14:54.246683+0000","last_active":"2023-05-16T07:14:54.246683+0000","last_peered":"2023-05-16T07:14:54.246683+0000","last_clean":"2023-05-16T07:14:54.246683+0000","last_became_active":"2023-05-16T07:14:54.246136+0000","last_became_peered":"2023-05-16T07:14:54.246136+0000","last_unstale":"2023-05-16T07:14:54.246683+0000","last_undegraded":"2023-05-16T07:14:54.246683+0000","last_fullsized":"2023-05-16T07:14:54.246683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872188+0000","last_change":"2023-05-16T07:14:53.868016+0000","last_active":"2023-05-16T07:14:53.872188+0000","last_peered":"2023-05-16T07:14:53.872188+0000","last_clean":"2023-05-16T07:14:53.872188+0000","last_became_active":"2023-05-16T07:14:53.867069+0000","last_became_peered":"2023-05-16T07:14:53.867069+0000","last_unstale":"2023-05-16T07:14:53.872188+0000","last_undegraded":"2023-05-16T07:14:53.872188+0000","last_fullsized":"2023-05-16T07:14:53.872188+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.244977+0000","last_change":"2023-05-16T07:14:54.244977+0000","last_active":"2023-05-16T07:14:54.244977+0000","last_peered":"2023-05-16T07:14:54.244977+0000","last_clean":"2023-05-16T07:14:54.244977+0000","last_became_active":"2023-05-16T07:14:54.244717+0000","last_became_peered":"2023-05-16T07:14:54.244717+0000","last_unstale":"2023-05-16T07:14:54.244977+0000","last_undegraded":"2023-05-16T07:14:54.244977+0000","last_fullsized":"2023-05-16T07:14:54.244977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871524+0000","last_change":"2023-05-16T07:14:53.868302+0000","last_active":"2023-05-16T07:14:53.871524+0000","last_peered":"2023-05-16T07:14:53.871524+0000","last_clean":"2023-05-16T07:14:53.871524+0000","last_became_active":"2023-05-16T07:14:53.868039+0000","last_became_peered":"2023-05-16T07:14:53.868039+0000","last_unstale":"2023-05-16T07:14:53.871524+0000","last_undegraded":"2023-05-16T07:14:53.871524+0000","last_fullsized":"2023-05-16T07:14:53.871524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250672+0000","last_change":"2023-05-16T07:14:54.250672+0000","last_active":"2023-05-16T07:14:54.250672+0000","last_peered":"2023-05-16T07:14:54.250672+0000","last_clean":"2023-05-16T07:14:54.250672+0000","last_became_active":"2023-05-16T07:14:54.250403+0000","last_became_peered":"2023-05-16T07:14:54.250403+0000","last_unstale":"2023-05-16T07:14:54.250672+0000","last_undegraded":"2023-05-16T07:14:54.250672+0000","last_fullsized":"2023-05-16T07:14:54.250672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872403+0000","last_change":"2023-05-16T07:14:53.869460+0000","last_active":"2023-05-16T07:14:53.872403+0000","last_peered":"2023-05-16T07:14:53.872403+0000","last_clean":"2023-05-16T07:14:53.872403+0000","last_became_active":"2023-05-16T07:14:53.868722+0000","last_became_peered":"2023-05-16T07:14:53.868722+0000","last_unstale":"2023-05-16T07:14:53.872403+0000","last_undegraded":"2023-05-16T07:14:53.872403+0000","last_fullsized":"2023-05-16T07:14:53.872403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.244437+0000","last_change":"2023-05-16T07:14:54.244437+0000","last_active":"2023-05-16T07:14:54.244437+0000","last_peered":"2023-05-16T07:14:54.244437+0000","last_clean":"2023-05-16T07:14:54.244437+0000","last_became_active":"2023-05-16T07:14:54.244205+0000","last_became_peered":"2023-05-16T07:14:54.244205+0000","last_unstale":"2023-05-16T07:14:54.244437+0000","last_undegraded":"2023-05-16T07:14:54.244437+0000","last_fullsized":"2023-05-16T07:14:54.244437+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875111+0000","last_change":"2023-05-16T07:14:53.872924+0000","last_active":"2023-05-16T07:14:53.875111+0000","last_peered":"2023-05-16T07:14:53.875111+0000","last_clean":"2023-05-16T07:14:53.875111+0000","last_became_active":"2023-05-16T07:14:53.872635+0000","last_became_peered":"2023-05-16T07:14:53.872635+0000","last_unstale":"2023-05-16T07:14:53.875111+0000","last_undegraded":"2023-05-16T07:14:53.875111+0000","last_fullsized":"2023-05-16T07:14:53.875111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.236122+0000","last_change":"2023-05-16T07:14:54.236122+0000","last_active":"2023-05-16T07:14:54.236122+0000","last_peered":"2023-05-16T07:14:54.236122+0000","last_clean":"2023-05-16T07:14:54.236122+0000","last_became_active":"2023-05-16T07:14:54.235515+0000","last_became_peered":"2023-05-16T07:14:54.235515+0000","last_unstale":"2023-05-16T07:14:54.236122+0000","last_undegraded":"2023-05-16T07:14:54.236122+0000","last_fullsized":"2023-05-16T07:14:54.236122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871504+0000","last_change":"2023-05-16T07:14:53.867229+0000","last_active":"2023-05-16T07:14:53.871504+0000","last_peered":"2023-05-16T07:14:53.871504+0000","last_clean":"2023-05-16T07:14:53.871504+0000","last_became_active":"2023-05-16T07:14:53.866957+0000","last_became_peered":"2023-05-16T07:14:53.866957+0000","last_unstale":"2023-05-16T07:14:53.871504+0000","last_undegraded":"2023-05-16T07:14:53.871504+0000","last_fullsized":"2023-05-16T07:14:53.871504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.251231+0000","last_change":"2023-05-16T07:14:54.251231+0000","last_active":"2023-05-16T07:14:54.251231+0000","last_peered":"2023-05-16T07:14:54.251231+0000","last_clean":"2023-05-16T07:14:54.251231+0000","last_became_active":"2023-05-16T07:14:54.251039+0000","last_became_peered":"2023-05-16T07:14:54.251039+0000","last_unstale":"2023-05-16T07:14:54.251231+0000","last_undegraded":"2023-05-16T07:14:54.251231+0000","last_fullsized":"2023-05-16T07:14:54.251231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875298+0000","last_change":"2023-05-16T07:14:53.874530+0000","last_active":"2023-05-16T07:14:53.875298+0000","last_peered":"2023-05-16T07:14:53.875298+0000","last_clean":"2023-05-16T07:14:53.875298+0000","last_became_active":"2023-05-16T07:14:53.874246+0000","last_became_peered":"2023-05-16T07:14:53.874246+0000","last_unstale":"2023-05-16T07:14:53.875298+0000","last_undegraded":"2023-05-16T07:14:53.875298+0000","last_fullsized":"2023-05-16T07:14:53.875298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.252483+0000","last_change":"2023-05-16T07:14:54.252483+0000","last_active":"2023-05-16T07:14:54.252483+0000","last_peered":"2023-05-16T07:14:54.252483+0000","last_clean":"2023-05-16T07:14:54.252483+0000","last_became_active":"2023-05-16T07:14:54.252271+0000","last_became_peered":"2023-05-16T07:14:54.252271+0000","last_unstale":"2023-05-16T07:14:54.252483+0000","last_undegraded":"2023-05-16T07:14:54.252483+0000","last_fullsized":"2023-05-16T07:14:54.252483+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872568+0000","last_change":"2023-05-16T07:14:53.869071+0000","last_active":"2023-05-16T07:14:53.872568+0000","last_peered":"2023-05-16T07:14:53.872568+0000","last_clean":"2023-05-16T07:14:53.872568+0000","last_became_active":"2023-05-16T07:14:53.868759+0000","last_became_peered":"2023-05-16T07:14:53.868759+0000","last_unstale":"2023-05-16T07:14:53.872568+0000","last_undegraded":"2023-05-16T07:14:53.872568+0000","last_fullsized":"2023-05-16T07:14:53.872568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248759+0000","last_change":"2023-05-16T07:14:54.248759+0000","last_active":"2023-05-16T07:14:54.248759+0000","last_peered":"2023-05-16T07:14:54.248759+0000","last_clean":"2023-05-16T07:14:54.248759+0000","last_became_active":"2023-05-16T07:14:54.248304+0000","last_became_peered":"2023-05-16T07:14:54.248304+0000","last_unstale":"2023-05-16T07:14:54.248759+0000","last_undegraded":"2023-05-16T07:14:54.248759+0000","last_fullsized":"2023-05-16T07:14:54.248759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871512+0000","last_change":"2023-05-16T07:14:53.866808+0000","last_active":"2023-05-16T07:14:53.871512+0000","last_peered":"2023-05-16T07:14:53.871512+0000","last_clean":"2023-05-16T07:14:53.871512+0000","last_became_active":"2023-05-16T07:14:53.866510+0000","last_became_peered":"2023-05-16T07:14:53.866510+0000","last_unstale":"2023-05-16T07:14:53.871512+0000","last_undegraded":"2023-05-16T07:14:53.871512+0000","last_fullsized":"2023-05-16T07:14:53.871512+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246140+0000","last_change":"2023-05-16T07:14:54.246140+0000","last_active":"2023-05-16T07:14:54.246140+0000","last_peered":"2023-05-16T07:14:54.246140+0000","last_clean":"2023-05-16T07:14:54.246140+0000","last_became_active":"2023-05-16T07:14:54.245830+0000","last_became_peered":"2023-05-16T07:14:54.245830+0000","last_unstale":"2023-05-16T07:14:54.246140+0000","last_undegraded":"2023-05-16T07:14:54.246140+0000","last_fullsized":"2023-05-16T07:14:54.246140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.228001+0000","last_change":"2023-05-16T07:14:53.868849+0000","last_active":"2023-05-16T07:14:54.228001+0000","last_peered":"2023-05-16T07:14:54.228001+0000","last_clean":"2023-05-16T07:14:54.228001+0000","last_became_active":"2023-05-16T07:14:53.868295+0000","last_became_peered":"2023-05-16T07:14:53.868295+0000","last_unstale":"2023-05-16T07:14:54.228001+0000","last_undegraded":"2023-05-16T07:14:54.228001+0000","last_fullsized":"2023-05-16T07:14:54.228001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246494+0000","last_change":"2023-05-16T07:14:54.246494+0000","last_active":"2023-05-16T07:14:54.246494+0000","last_peered":"2023-05-16T07:14:54.246494+0000","last_clean":"2023-05-16T07:14:54.246494+0000","last_became_active":"2023-05-16T07:14:54.246204+0000","last_became_peered":"2023-05-16T07:14:54.246204+0000","last_unstale":"2023-05-16T07:14:54.246494+0000","last_undegraded":"2023-05-16T07:14:54.246494+0000","last_fullsized":"2023-05-16T07:14:54.246494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.233227+0000","last_change":"2023-05-16T07:14:53.869975+0000","last_active":"2023-05-16T07:14:54.233227+0000","last_peered":"2023-05-16T07:14:54.233227+0000","last_clean":"2023-05-16T07:14:54.233227+0000","last_became_active":"2023-05-16T07:14:53.869688+0000","last_became_peered":"2023-05-16T07:14:53.869688+0000","last_unstale":"2023-05-16T07:14:54.233227+0000","last_undegraded":"2023-05-16T07:14:54.233227+0000","last_fullsized":"2023-05-16T07:14:54.233227+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.244309+0000","last_change":"2023-05-16T07:14:54.244309+0000","last_active":"2023-05-16T07:14:54.244309+0000","last_peered":"2023-05-16T07:14:54.244309+0000","last_clean":"2023-05-16T07:14:54.244309+0000","last_became_active":"2023-05-16T07:14:54.244008+0000","last_became_peered":"2023-05-16T07:14:54.244008+0000","last_unstale":"2023-05-16T07:14:54.244309+0000","last_undegraded":"2023-05-16T07:14:54.244309+0000","last_fullsized":"2023-05-16T07:14:54.244309+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.241536+0000","last_change":"2023-05-16T07:14:53.865961+0000","last_active":"2023-05-16T07:14:54.241536+0000","last_peered":"2023-05-16T07:14:54.241536+0000","last_clean":"2023-05-16T07:14:54.241536+0000","last_became_active":"2023-05-16T07:14:53.865677+0000","last_became_peered":"2023-05-16T07:14:53.865677+0000","last_unstale":"2023-05-16T07:14:54.241536+0000","last_undegraded":"2023-05-16T07:14:54.241536+0000","last_fullsized":"2023-05-16T07:14:54.241536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.241327+0000","last_change":"2023-05-16T07:14:54.241327+0000","last_active":"2023-05-16T07:14:54.241327+0000","last_peered":"2023-05-16T07:14:54.241327+0000","last_clean":"2023-05-16T07:14:54.241327+0000","last_became_active":"2023-05-16T07:14:54.241062+0000","last_became_peered":"2023-05-16T07:14:54.241062+0000","last_unstale":"2023-05-16T07:14:54.241327+0000","last_undegraded":"2023-05-16T07:14:54.241327+0000","last_fullsized":"2023-05-16T07:14:54.241327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871895+0000","last_change":"2023-05-16T07:14:53.871086+0000","last_active":"2023-05-16T07:14:53.871895+0000","last_peered":"2023-05-16T07:14:53.871895+0000","last_clean":"2023-05-16T07:14:53.871895+0000","last_became_active":"2023-05-16T07:14:53.870873+0000","last_became_peered":"2023-05-16T07:14:53.870873+0000","last_unstale":"2023-05-16T07:14:53.871895+0000","last_undegraded":"2023-05-16T07:14:53.871895+0000","last_fullsized":"2023-05-16T07:14:53.871895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248571+0000","last_change":"2023-05-16T07:14:54.248571+0000","last_active":"2023-05-16T07:14:54.248571+0000","last_peered":"2023-05-16T07:14:54.248571+0000","last_clean":"2023-05-16T07:14:54.248571+0000","last_became_active":"2023-05-16T07:14:54.248263+0000","last_became_peered":"2023-05-16T07:14:54.248263+0000","last_unstale":"2023-05-16T07:14:54.248571+0000","last_undegraded":"2023-05-16T07:14:54.248571+0000","last_fullsized":"2023-05-16T07:14:54.248571+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881715+0000","last_change":"2023-05-16T07:14:53.876330+0000","last_active":"2023-05-16T07:14:53.881715+0000","last_peered":"2023-05-16T07:14:53.881715+0000","last_clean":"2023-05-16T07:14:53.881715+0000","last_became_active":"2023-05-16T07:14:53.875998+0000","last_became_peered":"2023-05-16T07:14:53.875998+0000","last_unstale":"2023-05-16T07:14:53.881715+0000","last_undegraded":"2023-05-16T07:14:53.881715+0000","last_fullsized":"2023-05-16T07:14:53.881715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253365+0000","last_change":"2023-05-16T07:14:54.253365+0000","last_active":"2023-05-16T07:14:54.253365+0000","last_peered":"2023-05-16T07:14:54.253365+0000","last_clean":"2023-05-16T07:14:54.253365+0000","last_became_active":"2023-05-16T07:14:54.253095+0000","last_became_peered":"2023-05-16T07:14:54.253095+0000","last_unstale":"2023-05-16T07:14:54.253365+0000","last_undegraded":"2023-05-16T07:14:54.253365+0000","last_fullsized":"2023-05-16T07:14:54.253365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872044+0000","last_change":"2023-05-16T07:14:53.866962+0000","last_active":"2023-05-16T07:14:53.872044+0000","last_peered":"2023-05-16T07:14:53.872044+0000","last_clean":"2023-05-16T07:14:53.872044+0000","last_became_active":"2023-05-16T07:14:53.866664+0000","last_became_peered":"2023-05-16T07:14:53.866664+0000","last_unstale":"2023-05-16T07:14:53.872044+0000","last_undegraded":"2023-05-16T07:14:53.872044+0000","last_fullsized":"2023-05-16T07:14:53.872044+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240770+0000","last_change":"2023-05-16T07:14:54.240770+0000","last_active":"2023-05-16T07:14:54.240770+0000","last_peered":"2023-05-16T07:14:54.240770+0000","last_clean":"2023-05-16T07:14:54.240770+0000","last_became_active":"2023-05-16T07:14:54.240515+0000","last_became_peered":"2023-05-16T07:14:54.240515+0000","last_unstale":"2023-05-16T07:14:54.240770+0000","last_undegraded":"2023-05-16T07:14:54.240770+0000","last_fullsized":"2023-05-16T07:14:54.240770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.873687+0000","last_change":"2023-05-16T07:14:53.871224+0000","last_active":"2023-05-16T07:14:53.873687+0000","last_peered":"2023-05-16T07:14:53.873687+0000","last_clean":"2023-05-16T07:14:53.873687+0000","last_became_active":"2023-05-16T07:14:53.870914+0000","last_became_peered":"2023-05-16T07:14:53.870914+0000","last_unstale":"2023-05-16T07:14:53.873687+0000","last_undegraded":"2023-05-16T07:14:53.873687+0000","last_fullsized":"2023-05-16T07:14:53.873687+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255828+0000","last_change":"2023-05-16T07:14:54.255828+0000","last_active":"2023-05-16T07:14:54.255828+0000","last_peered":"2023-05-16T07:14:54.255828+0000","last_clean":"2023-05-16T07:14:54.255828+0000","last_became_active":"2023-05-16T07:14:54.255463+0000","last_became_peered":"2023-05-16T07:14:54.255463+0000","last_unstale":"2023-05-16T07:14:54.255828+0000","last_undegraded":"2023-05-16T07:14:54.255828+0000","last_fullsized":"2023-05-16T07:14:54.255828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875892+0000","last_change":"2023-05-16T07:14:53.872167+0000","last_active":"2023-05-16T07:14:53.875892+0000","last_peered":"2023-05-16T07:14:53.875892+0000","last_clean":"2023-05-16T07:14:53.875892+0000","last_became_active":"2023-05-16T07:14:53.871888+0000","last_became_peered":"2023-05-16T07:14:53.871888+0000","last_unstale":"2023-05-16T07:14:53.875892+0000","last_undegraded":"2023-05-16T07:14:53.875892+0000","last_fullsized":"2023-05-16T07:14:53.875892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246596+0000","last_change":"2023-05-16T07:14:54.246596+0000","last_active":"2023-05-16T07:14:54.246596+0000","last_peered":"2023-05-16T07:14:54.246596+0000","last_clean":"2023-05-16T07:14:54.246596+0000","last_became_active":"2023-05-16T07:14:54.246359+0000","last_became_peered":"2023-05-16T07:14:54.246359+0000","last_unstale":"2023-05-16T07:14:54.246596+0000","last_undegraded":"2023-05-16T07:14:54.246596+0000","last_fullsized":"2023-05-16T07:14:54.246596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875445+0000","last_change":"2023-05-16T07:14:53.872477+0000","last_active":"2023-05-16T07:14:53.875445+0000","last_peered":"2023-05-16T07:14:53.875445+0000","last_clean":"2023-05-16T07:14:53.875445+0000","last_became_active":"2023-05-16T07:14:53.872158+0000","last_became_peered":"2023-05-16T07:14:53.872158+0000","last_unstale":"2023-05-16T07:14:53.875445+0000","last_undegraded":"2023-05-16T07:14:53.875445+0000","last_fullsized":"2023-05-16T07:14:53.875445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235331+0000","last_change":"2023-05-16T07:14:54.235331+0000","last_active":"2023-05-16T07:14:54.235331+0000","last_peered":"2023-05-16T07:14:54.235331+0000","last_clean":"2023-05-16T07:14:54.235331+0000","last_became_active":"2023-05-16T07:14:54.234976+0000","last_became_peered":"2023-05-16T07:14:54.234976+0000","last_unstale":"2023-05-16T07:14:54.235331+0000","last_undegraded":"2023-05-16T07:14:54.235331+0000","last_fullsized":"2023-05-16T07:14:54.235331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.877524+0000","last_change":"2023-05-16T07:14:53.876922+0000","last_active":"2023-05-16T07:14:53.877524+0000","last_peered":"2023-05-16T07:14:53.877524+0000","last_clean":"2023-05-16T07:14:53.877524+0000","last_became_active":"2023-05-16T07:14:53.876467+0000","last_became_peered":"2023-05-16T07:14:53.876467+0000","last_unstale":"2023-05-16T07:14:53.877524+0000","last_undegraded":"2023-05-16T07:14:53.877524+0000","last_fullsized":"2023-05-16T07:14:53.877524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249968+0000","last_change":"2023-05-16T07:14:54.249968+0000","last_active":"2023-05-16T07:14:54.249968+0000","last_peered":"2023-05-16T07:14:54.249968+0000","last_clean":"2023-05-16T07:14:54.249968+0000","last_became_active":"2023-05-16T07:14:54.249645+0000","last_became_peered":"2023-05-16T07:14:54.249645+0000","last_unstale":"2023-05-16T07:14:54.249968+0000","last_undegraded":"2023-05-16T07:14:54.249968+0000","last_fullsized":"2023-05-16T07:14:54.249968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255148+0000","last_change":"2023-05-16T07:14:53.878959+0000","last_active":"2023-05-16T07:14:54.255148+0000","last_peered":"2023-05-16T07:14:54.255148+0000","last_clean":"2023-05-16T07:14:54.255148+0000","last_became_active":"2023-05-16T07:14:53.878664+0000","last_became_peered":"2023-05-16T07:14:53.878664+0000","last_unstale":"2023-05-16T07:14:54.255148+0000","last_undegraded":"2023-05-16T07:14:54.255148+0000","last_fullsized":"2023-05-16T07:14:54.255148+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.232851+0000","last_change":"2023-05-16T07:14:54.232851+0000","last_active":"2023-05-16T07:14:54.232851+0000","last_peered":"2023-05-16T07:14:54.232851+0000","last_clean":"2023-05-16T07:14:54.232851+0000","last_became_active":"2023-05-16T07:14:54.232609+0000","last_became_peered":"2023-05-16T07:14:54.232609+0000","last_unstale":"2023-05-16T07:14:54.232851+0000","last_undegraded":"2023-05-16T07:14:54.232851+0000","last_fullsized":"2023-05-16T07:14:54.232851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.256051+0000","last_change":"2023-05-16T07:14:53.870300+0000","last_active":"2023-05-16T07:14:54.256051+0000","last_peered":"2023-05-16T07:14:54.256051+0000","last_clean":"2023-05-16T07:14:54.256051+0000","last_became_active":"2023-05-16T07:14:53.868810+0000","last_became_peered":"2023-05-16T07:14:53.868810+0000","last_unstale":"2023-05-16T07:14:54.256051+0000","last_undegraded":"2023-05-16T07:14:54.256051+0000","last_fullsized":"2023-05-16T07:14:54.256051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247341+0000","last_change":"2023-05-16T07:14:54.247341+0000","last_active":"2023-05-16T07:14:54.247341+0000","last_peered":"2023-05-16T07:14:54.247341+0000","last_clean":"2023-05-16T07:14:54.247341+0000","last_became_active":"2023-05-16T07:14:54.247084+0000","last_became_peered":"2023-05-16T07:14:54.247084+0000","last_unstale":"2023-05-16T07:14:54.247341+0000","last_undegraded":"2023-05-16T07:14:54.247341+0000","last_fullsized":"2023-05-16T07:14:54.247341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.241684+0000","last_change":"2023-05-16T07:14:53.870347+0000","last_active":"2023-05-16T07:14:54.241684+0000","last_peered":"2023-05-16T07:14:54.241684+0000","last_clean":"2023-05-16T07:14:54.241684+0000","last_became_active":"2023-05-16T07:14:53.870098+0000","last_became_peered":"2023-05-16T07:14:53.870098+0000","last_unstale":"2023-05-16T07:14:54.241684+0000","last_undegraded":"2023-05-16T07:14:54.241684+0000","last_fullsized":"2023-05-16T07:14:54.241684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255366+0000","last_change":"2023-05-16T07:14:54.255366+0000","last_active":"2023-05-16T07:14:54.255366+0000","last_peered":"2023-05-16T07:14:54.255366+0000","last_clean":"2023-05-16T07:14:54.255366+0000","last_became_active":"2023-05-16T07:14:54.255089+0000","last_became_peered":"2023-05-16T07:14:54.255089+0000","last_unstale":"2023-05-16T07:14:54.255366+0000","last_undegraded":"2023-05-16T07:14:54.255366+0000","last_fullsized":"2023-05-16T07:14:54.255366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.876802+0000","last_change":"2023-05-16T07:14:53.875530+0000","last_active":"2023-05-16T07:14:53.876802+0000","last_peered":"2023-05-16T07:14:53.876802+0000","last_clean":"2023-05-16T07:14:53.876802+0000","last_became_active":"2023-05-16T07:14:53.874783+0000","last_became_peered":"2023-05-16T07:14:53.874783+0000","last_unstale":"2023-05-16T07:14:53.876802+0000","last_undegraded":"2023-05-16T07:14:53.876802+0000","last_fullsized":"2023-05-16T07:14:53.876802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249738+0000","last_change":"2023-05-16T07:14:54.249738+0000","last_active":"2023-05-16T07:14:54.249738+0000","last_peered":"2023-05-16T07:14:54.249738+0000","last_clean":"2023-05-16T07:14:54.249738+0000","last_became_active":"2023-05-16T07:14:54.247517+0000","last_became_peered":"2023-05-16T07:14:54.247517+0000","last_unstale":"2023-05-16T07:14:54.249738+0000","last_undegraded":"2023-05-16T07:14:54.249738+0000","last_fullsized":"2023-05-16T07:14:54.249738+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875342+0000","last_change":"2023-05-16T07:14:53.872604+0000","last_active":"2023-05-16T07:14:53.875342+0000","last_peered":"2023-05-16T07:14:53.875342+0000","last_clean":"2023-05-16T07:14:53.875342+0000","last_became_active":"2023-05-16T07:14:53.872275+0000","last_became_peered":"2023-05-16T07:14:53.872275+0000","last_unstale":"2023-05-16T07:14:53.875342+0000","last_undegraded":"2023-05-16T07:14:53.875342+0000","last_fullsized":"2023-05-16T07:14:53.875342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248575+0000","last_change":"2023-05-16T07:14:54.248575+0000","last_active":"2023-05-16T07:14:54.248575+0000","last_peered":"2023-05-16T07:14:54.248575+0000","last_clean":"2023-05-16T07:14:54.248575+0000","last_became_active":"2023-05-16T07:14:54.245968+0000","last_became_peered":"2023-05-16T07:14:54.245968+0000","last_unstale":"2023-05-16T07:14:54.248575+0000","last_undegraded":"2023-05-16T07:14:54.248575+0000","last_fullsized":"2023-05-16T07:14:54.248575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.256276+0000","last_change":"2023-05-16T07:14:53.872018+0000","last_active":"2023-05-16T07:14:54.256276+0000","last_peered":"2023-05-16T07:14:54.256276+0000","last_clean":"2023-05-16T07:14:54.256276+0000","last_became_active":"2023-05-16T07:14:53.871783+0000","last_became_peered":"2023-05-16T07:14:53.871783+0000","last_unstale":"2023-05-16T07:14:54.256276+0000","last_undegraded":"2023-05-16T07:14:54.256276+0000","last_fullsized":"2023-05-16T07:14:54.256276+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248067+0000","last_change":"2023-05-16T07:14:54.248067+0000","last_active":"2023-05-16T07:14:54.248067+0000","last_peered":"2023-05-16T07:14:54.248067+0000","last_clean":"2023-05-16T07:14:54.248067+0000","last_became_active":"2023-05-16T07:14:54.245746+0000","last_became_peered":"2023-05-16T07:14:54.245746+0000","last_unstale":"2023-05-16T07:14:54.248067+0000","last_undegraded":"2023-05-16T07:14:54.248067+0000","last_fullsized":"2023-05-16T07:14:54.248067+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.242799+0000","last_change":"2023-05-16T07:14:53.876268+0000","last_active":"2023-05-16T07:14:54.242799+0000","last_peered":"2023-05-16T07:14:54.242799+0000","last_clean":"2023-05-16T07:14:54.242799+0000","last_became_active":"2023-05-16T07:14:53.875967+0000","last_became_peered":"2023-05-16T07:14:53.875967+0000","last_unstale":"2023-05-16T07:14:54.242799+0000","last_undegraded":"2023-05-16T07:14:54.242799+0000","last_fullsized":"2023-05-16T07:14:54.242799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255438+0000","last_change":"2023-05-16T07:14:54.255438+0000","last_active":"2023-05-16T07:14:54.255438+0000","last_peered":"2023-05-16T07:14:54.255438+0000","last_clean":"2023-05-16T07:14:54.255438+0000","last_became_active":"2023-05-16T07:14:54.255171+0000","last_became_peered":"2023-05-16T07:14:54.255171+0000","last_unstale":"2023-05-16T07:14:54.255438+0000","last_undegraded":"2023-05-16T07:14:54.255438+0000","last_fullsized":"2023-05-16T07:14:54.255438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240000+0000","last_change":"2023-05-16T07:14:53.867925+0000","last_active":"2023-05-16T07:14:54.240000+0000","last_peered":"2023-05-16T07:14:54.240000+0000","last_clean":"2023-05-16T07:14:54.240000+0000","last_became_active":"2023-05-16T07:14:53.867480+0000","last_became_peered":"2023-05-16T07:14:53.867480+0000","last_unstale":"2023-05-16T07:14:54.240000+0000","last_undegraded":"2023-05-16T07:14:54.240000+0000","last_fullsized":"2023-05-16T07:14:54.240000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250769+0000","last_change":"2023-05-16T07:14:54.250769+0000","last_active":"2023-05-16T07:14:54.250769+0000","last_peered":"2023-05-16T07:14:54.250769+0000","last_clean":"2023-05-16T07:14:54.250769+0000","last_became_active":"2023-05-16T07:14:54.250435+0000","last_became_peered":"2023-05-16T07:14:54.250435+0000","last_unstale":"2023-05-16T07:14:54.250769+0000","last_undegraded":"2023-05-16T07:14:54.250769+0000","last_fullsized":"2023-05-16T07:14:54.250769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871396+0000","last_change":"2023-05-16T07:14:53.866890+0000","last_active":"2023-05-16T07:14:53.871396+0000","last_peered":"2023-05-16T07:14:53.871396+0000","last_clean":"2023-05-16T07:14:53.871396+0000","last_became_active":"2023-05-16T07:14:53.866430+0000","last_became_peered":"2023-05-16T07:14:53.866430+0000","last_unstale":"2023-05-16T07:14:53.871396+0000","last_undegraded":"2023-05-16T07:14:53.871396+0000","last_fullsized":"2023-05-16T07:14:53.871396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.239912+0000","last_change":"2023-05-16T07:14:54.239912+0000","last_active":"2023-05-16T07:14:54.239912+0000","last_peered":"2023-05-16T07:14:54.239912+0000","last_clean":"2023-05-16T07:14:54.239912+0000","last_became_active":"2023-05-16T07:14:54.239644+0000","last_became_peered":"2023-05-16T07:14:54.239644+0000","last_unstale":"2023-05-16T07:14:54.239912+0000","last_undegraded":"2023-05-16T07:14:54.239912+0000","last_fullsized":"2023-05-16T07:14:54.239912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871793+0000","last_change":"2023-05-16T07:14:53.870093+0000","last_active":"2023-05-16T07:14:53.871793+0000","last_peered":"2023-05-16T07:14:53.871793+0000","last_clean":"2023-05-16T07:14:53.871793+0000","last_became_active":"2023-05-16T07:14:53.869831+0000","last_became_peered":"2023-05-16T07:14:53.869831+0000","last_unstale":"2023-05-16T07:14:53.871793+0000","last_undegraded":"2023-05-16T07:14:53.871793+0000","last_fullsized":"2023-05-16T07:14:53.871793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235403+0000","last_change":"2023-05-16T07:14:54.235403+0000","last_active":"2023-05-16T07:14:54.235403+0000","last_peered":"2023-05-16T07:14:54.235403+0000","last_clean":"2023-05-16T07:14:54.235403+0000","last_became_active":"2023-05-16T07:14:54.235165+0000","last_became_peered":"2023-05-16T07:14:54.235165+0000","last_unstale":"2023-05-16T07:14:54.235403+0000","last_undegraded":"2023-05-16T07:14:54.235403+0000","last_fullsized":"2023-05-16T07:14:54.235403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.233753+0000","last_change":"2023-05-16T07:14:53.868453+0000","last_active":"2023-05-16T07:14:54.233753+0000","last_peered":"2023-05-16T07:14:54.233753+0000","last_clean":"2023-05-16T07:14:54.233753+0000","last_became_active":"2023-05-16T07:14:53.867941+0000","last_became_peered":"2023-05-16T07:14:53.867941+0000","last_unstale":"2023-05-16T07:14:54.233753+0000","last_undegraded":"2023-05-16T07:14:54.233753+0000","last_fullsized":"2023-05-16T07:14:54.233753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.232135+0000","last_change":"2023-05-16T07:14:54.232135+0000","last_active":"2023-05-16T07:14:54.232135+0000","last_peered":"2023-05-16T07:14:54.232135+0000","last_clean":"2023-05-16T07:14:54.232135+0000","last_became_active":"2023-05-16T07:14:54.231800+0000","last_became_peered":"2023-05-16T07:14:54.231800+0000","last_unstale":"2023-05-16T07:14:54.232135+0000","last_undegraded":"2023-05-16T07:14:54.232135+0000","last_fullsized":"2023-05-16T07:14:54.232135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872137+0000","last_change":"2023-05-16T07:14:53.867435+0000","last_active":"2023-05-16T07:14:53.872137+0000","last_peered":"2023-05-16T07:14:53.872137+0000","last_clean":"2023-05-16T07:14:53.872137+0000","last_became_active":"2023-05-16T07:14:53.867211+0000","last_became_peered":"2023-05-16T07:14:53.867211+0000","last_unstale":"2023-05-16T07:14:53.872137+0000","last_undegraded":"2023-05-16T07:14:53.872137+0000","last_fullsized":"2023-05-16T07:14:53.872137+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250829+0000","last_change":"2023-05-16T07:14:54.250829+0000","last_active":"2023-05-16T07:14:54.250829+0000","last_peered":"2023-05-16T07:14:54.250829+0000","last_clean":"2023-05-16T07:14:54.250829+0000","last_became_active":"2023-05-16T07:14:54.250562+0000","last_became_peered":"2023-05-16T07:14:54.250562+0000","last_unstale":"2023-05-16T07:14:54.250829+0000","last_undegraded":"2023-05-16T07:14:54.250829+0000","last_fullsized":"2023-05-16T07:14:54.250829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884559+0000","last_change":"2023-05-16T07:14:53.880118+0000","last_active":"2023-05-16T07:14:53.884559+0000","last_peered":"2023-05-16T07:14:53.884559+0000","last_clean":"2023-05-16T07:14:53.884559+0000","last_became_active":"2023-05-16T07:14:53.879846+0000","last_became_peered":"2023-05-16T07:14:53.879846+0000","last_unstale":"2023-05-16T07:14:53.884559+0000","last_undegraded":"2023-05-16T07:14:53.884559+0000","last_fullsized":"2023-05-16T07:14:53.884559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246946+0000","last_change":"2023-05-16T07:14:54.246946+0000","last_active":"2023-05-16T07:14:54.246946+0000","last_peered":"2023-05-16T07:14:54.246946+0000","last_clean":"2023-05-16T07:14:54.246946+0000","last_became_active":"2023-05-16T07:14:54.246660+0000","last_became_peered":"2023-05-16T07:14:54.246660+0000","last_unstale":"2023-05-16T07:14:54.246946+0000","last_undegraded":"2023-05-16T07:14:54.246946+0000","last_fullsized":"2023-05-16T07:14:54.246946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871493+0000","last_change":"2023-05-16T07:14:53.864168+0000","last_active":"2023-05-16T07:14:53.871493+0000","last_peered":"2023-05-16T07:14:53.871493+0000","last_clean":"2023-05-16T07:14:53.871493+0000","last_became_active":"2023-05-16T07:14:53.863907+0000","last_became_peered":"2023-05-16T07:14:53.863907+0000","last_unstale":"2023-05-16T07:14:53.871493+0000","last_undegraded":"2023-05-16T07:14:53.871493+0000","last_fullsized":"2023-05-16T07:14:53.871493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250704+0000","last_change":"2023-05-16T07:14:54.250704+0000","last_active":"2023-05-16T07:14:54.250704+0000","last_peered":"2023-05-16T07:14:54.250704+0000","last_clean":"2023-05-16T07:14:54.250704+0000","last_became_active":"2023-05-16T07:14:54.250447+0000","last_became_peered":"2023-05-16T07:14:54.250447+0000","last_unstale":"2023-05-16T07:14:54.250704+0000","last_undegraded":"2023-05-16T07:14:54.250704+0000","last_fullsized":"2023-05-16T07:14:54.250704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.874037+0000","last_change":"2023-05-16T07:14:53.870272+0000","last_active":"2023-05-16T07:14:53.874037+0000","last_peered":"2023-05-16T07:14:53.874037+0000","last_clean":"2023-05-16T07:14:53.874037+0000","last_became_active":"2023-05-16T07:14:53.870006+0000","last_became_peered":"2023-05-16T07:14:53.870006+0000","last_unstale":"2023-05-16T07:14:53.874037+0000","last_undegraded":"2023-05-16T07:14:53.874037+0000","last_fullsized":"2023-05-16T07:14:53.874037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.858958+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:14:53.858958+0000","last_peered":"2023-05-16T07:14:53.858958+0000","last_clean":"2023-05-16T07:14:53.858958+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:14:53.858958+0000","last_undegraded":"2023-05-16T07:14:53.858958+0000","last_fullsized":"2023-05-16T07:14:53.858958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246691+0000","last_change":"2023-05-16T07:14:54.246691+0000","last_active":"2023-05-16T07:14:54.246691+0000","last_peered":"2023-05-16T07:14:54.246691+0000","last_clean":"2023-05-16T07:14:54.246691+0000","last_became_active":"2023-05-16T07:14:54.245947+0000","last_became_peered":"2023-05-16T07:14:54.245947+0000","last_unstale":"2023-05-16T07:14:54.246691+0000","last_undegraded":"2023-05-16T07:14:54.246691+0000","last_fullsized":"2023-05-16T07:14:54.246691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247493+0000","last_change":"2023-05-16T07:14:53.875329+0000","last_active":"2023-05-16T07:14:54.247493+0000","last_peered":"2023-05-16T07:14:54.247493+0000","last_clean":"2023-05-16T07:14:54.247493+0000","last_became_active":"2023-05-16T07:14:53.875014+0000","last_became_peered":"2023-05-16T07:14:53.875014+0000","last_unstale":"2023-05-16T07:14:54.247493+0000","last_undegraded":"2023-05-16T07:14:54.247493+0000","last_fullsized":"2023-05-16T07:14:54.247493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.857716+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:14:53.857716+0000","last_peered":"2023-05-16T07:14:53.857716+0000","last_clean":"2023-05-16T07:14:53.857716+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:14:53.857716+0000","last_undegraded":"2023-05-16T07:14:53.857716+0000","last_fullsized":"2023-05-16T07:14:53.857716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246032+0000","last_change":"2023-05-16T07:14:54.246032+0000","last_active":"2023-05-16T07:14:54.246032+0000","last_peered":"2023-05-16T07:14:54.246032+0000","last_clean":"2023-05-16T07:14:54.246032+0000","last_became_active":"2023-05-16T07:14:54.245611+0000","last_became_peered":"2023-05-16T07:14:54.245611+0000","last_unstale":"2023-05-16T07:14:54.246032+0000","last_undegraded":"2023-05-16T07:14:54.246032+0000","last_fullsized":"2023-05-16T07:14:54.246032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881738+0000","last_change":"2023-05-16T07:14:53.875901+0000","last_active":"2023-05-16T07:14:53.881738+0000","last_peered":"2023-05-16T07:14:53.881738+0000","last_clean":"2023-05-16T07:14:53.881738+0000","last_became_active":"2023-05-16T07:14:53.874568+0000","last_became_peered":"2023-05-16T07:14:53.874568+0000","last_unstale":"2023-05-16T07:14:53.881738+0000","last_undegraded":"2023-05-16T07:14:53.881738+0000","last_fullsized":"2023-05-16T07:14:53.881738+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.855199+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:14:53.855199+0000","last_peered":"2023-05-16T07:14:53.855199+0000","last_clean":"2023-05-16T07:14:53.855199+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:14:53.855199+0000","last_undegraded":"2023-05-16T07:14:53.855199+0000","last_fullsized":"2023-05-16T07:14:53.855199+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.239967+0000","last_change":"2023-05-16T07:14:54.239967+0000","last_active":"2023-05-16T07:14:54.239967+0000","last_peered":"2023-05-16T07:14:54.239967+0000","last_clean":"2023-05-16T07:14:54.239967+0000","last_became_active":"2023-05-16T07:14:54.239601+0000","last_became_peered":"2023-05-16T07:14:54.239601+0000","last_unstale":"2023-05-16T07:14:54.239967+0000","last_undegraded":"2023-05-16T07:14:54.239967+0000","last_fullsized":"2023-05-16T07:14:54.239967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881589+0000","last_change":"2023-05-16T07:14:53.876019+0000","last_active":"2023-05-16T07:14:53.881589+0000","last_peered":"2023-05-16T07:14:53.881589+0000","last_clean":"2023-05-16T07:14:53.881589+0000","last_became_active":"2023-05-16T07:14:53.874789+0000","last_became_peered":"2023-05-16T07:14:53.874789+0000","last_unstale":"2023-05-16T07:14:53.881589+0000","last_undegraded":"2023-05-16T07:14:53.881589+0000","last_fullsized":"2023-05-16T07:14:53.881589+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.859820+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:14:53.859820+0000","last_peered":"2023-05-16T07:14:53.859820+0000","last_clean":"2023-05-16T07:14:53.859820+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:14:53.859820+0000","last_undegraded":"2023-05-16T07:14:53.859820+0000","last_fullsized":"2023-05-16T07:14:53.859820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.864056+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:14:53.864056+0000","last_peered":"2023-05-16T07:14:53.864056+0000","last_clean":"2023-05-16T07:14:53.864056+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:14:53.864056+0000","last_undegraded":"2023-05-16T07:14:53.864056+0000","last_fullsized":"2023-05-16T07:14:53.864056+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240778+0000","last_change":"2023-05-16T07:14:54.240778+0000","last_active":"2023-05-16T07:14:54.240778+0000","last_peered":"2023-05-16T07:14:54.240778+0000","last_clean":"2023-05-16T07:14:54.240778+0000","last_became_active":"2023-05-16T07:14:54.240521+0000","last_became_peered":"2023-05-16T07:14:54.240521+0000","last_unstale":"2023-05-16T07:14:54.240778+0000","last_undegraded":"2023-05-16T07:14:54.240778+0000","last_fullsized":"2023-05-16T07:14:54.240778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875370+0000","last_change":"2023-05-16T07:14:53.872657+0000","last_active":"2023-05-16T07:14:53.875370+0000","last_peered":"2023-05-16T07:14:53.875370+0000","last_clean":"2023-05-16T07:14:53.875370+0000","last_became_active":"2023-05-16T07:14:53.872338+0000","last_became_peered":"2023-05-16T07:14:53.872338+0000","last_unstale":"2023-05-16T07:14:53.875370+0000","last_undegraded":"2023-05-16T07:14:53.875370+0000","last_fullsized":"2023-05-16T07:14:53.875370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.864134+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:14:53.864134+0000","last_peered":"2023-05-16T07:14:53.864134+0000","last_clean":"2023-05-16T07:14:53.864134+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:14:53.864134+0000","last_undegraded":"2023-05-16T07:14:53.864134+0000","last_fullsized":"2023-05-16T07:14:53.864134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.243135+0000","last_change":"2023-05-16T07:14:54.243135+0000","last_active":"2023-05-16T07:14:54.243135+0000","last_peered":"2023-05-16T07:14:54.243135+0000","last_clean":"2023-05-16T07:14:54.243135+0000","last_became_active":"2023-05-16T07:14:54.242882+0000","last_became_peered":"2023-05-16T07:14:54.242882+0000","last_unstale":"2023-05-16T07:14:54.243135+0000","last_undegraded":"2023-05-16T07:14:54.243135+0000","last_fullsized":"2023-05-16T07:14:54.243135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.867815+0000","last_change":"2023-05-16T07:14:53.864638+0000","last_active":"2023-05-16T07:14:53.867815+0000","last_peered":"2023-05-16T07:14:53.867815+0000","last_clean":"2023-05-16T07:14:53.867815+0000","last_became_active":"2023-05-16T07:14:53.864340+0000","last_became_peered":"2023-05-16T07:14:53.864340+0000","last_unstale":"2023-05-16T07:14:53.867815+0000","last_undegraded":"2023-05-16T07:14:53.867815+0000","last_fullsized":"2023-05-16T07:14:53.867815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.864849+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:14:53.864849+0000","last_peered":"2023-05-16T07:14:53.864849+0000","last_clean":"2023-05-16T07:14:53.864849+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:14:53.864849+0000","last_undegraded":"2023-05-16T07:14:53.864849+0000","last_fullsized":"2023-05-16T07:14:53.864849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249122+0000","last_change":"2023-05-16T07:14:54.249122+0000","last_active":"2023-05-16T07:14:54.249122+0000","last_peered":"2023-05-16T07:14:54.249122+0000","last_clean":"2023-05-16T07:14:54.249122+0000","last_became_active":"2023-05-16T07:14:54.248025+0000","last_became_peered":"2023-05-16T07:14:54.248025+0000","last_unstale":"2023-05-16T07:14:54.249122+0000","last_undegraded":"2023-05-16T07:14:54.249122+0000","last_fullsized":"2023-05-16T07:14:54.249122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872280+0000","last_change":"2023-05-16T07:14:53.868419+0000","last_active":"2023-05-16T07:14:53.872280+0000","last_peered":"2023-05-16T07:14:53.872280+0000","last_clean":"2023-05-16T07:14:53.872280+0000","last_became_active":"2023-05-16T07:14:53.868129+0000","last_became_peered":"2023-05-16T07:14:53.868129+0000","last_unstale":"2023-05-16T07:14:53.872280+0000","last_undegraded":"2023-05-16T07:14:53.872280+0000","last_fullsized":"2023-05-16T07:14:53.872280+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.862427+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:14:53.862427+0000","last_peered":"2023-05-16T07:14:53.862427+0000","last_clean":"2023-05-16T07:14:53.862427+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:14:53.862427+0000","last_undegraded":"2023-05-16T07:14:53.862427+0000","last_fullsized":"2023-05-16T07:14:53.862427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.251529+0000","last_change":"2023-05-16T07:14:54.251529+0000","last_active":"2023-05-16T07:14:54.251529+0000","last_peered":"2023-05-16T07:14:54.251529+0000","last_clean":"2023-05-16T07:14:54.251529+0000","last_became_active":"2023-05-16T07:14:54.251247+0000","last_became_peered":"2023-05-16T07:14:54.251247+0000","last_unstale":"2023-05-16T07:14:54.251529+0000","last_undegraded":"2023-05-16T07:14:54.251529+0000","last_fullsized":"2023-05-16T07:14:54.251529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.874734+0000","last_change":"2023-05-16T07:14:53.873387+0000","last_active":"2023-05-16T07:14:53.874734+0000","last_peered":"2023-05-16T07:14:53.874734+0000","last_clean":"2023-05-16T07:14:53.874734+0000","last_became_active":"2023-05-16T07:14:53.873049+0000","last_became_peered":"2023-05-16T07:14:53.873049+0000","last_unstale":"2023-05-16T07:14:53.874734+0000","last_undegraded":"2023-05-16T07:14:53.874734+0000","last_fullsized":"2023-05-16T07:14:53.874734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.861595+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:14:53.861595+0000","last_peered":"2023-05-16T07:14:53.861595+0000","last_clean":"2023-05-16T07:14:53.861595+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:14:53.861595+0000","last_undegraded":"2023-05-16T07:14:53.861595+0000","last_fullsized":"2023-05-16T07:14:53.861595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16180,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":15488,"kb_avail":94355660,"statfs":{"total":96636764160,"available":96620195840,"internally_reserved":0,"allocated":700416,"data_stored":342532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15859712},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.84199999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.81299999999999994},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.442},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.36799999999999999},"max":{"1min":0.753,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.311},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44600000000000001,"15min":0.46700000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.81999999999999995},"last":0.37}]},{"osd":5,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.503},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.53000000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.999},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.38500000000000001,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.95099999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.77000000000000002,"15min":0.84299999999999997},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15824,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":15104,"kb_avail":94356016,"statfs":{"total":96636764160,"available":96620560384,"internally_reserved":0,"allocated":729088,"data_stored":357572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15466496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.317},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.55300000000000005},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.29699999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.80800000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.71199999999999997,"5min":0.76400000000000001,"15min":0.83699999999999997},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72399999999999998,"5min":0.75800000000000001,"15min":0.96199999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.89400000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35999999999999999,"5min":0.34699999999999998,"15min":0.32200000000000001},"max":{"1min":4.048,"5min":4.048,"15min":4.048},"last":0.48899999999999999}]},{"osd":5,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47499999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.92300000000000004},"last":0.60899999999999999}]},{"osd":7,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.622},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.36699999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15952,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94355888,"statfs":{"total":96636764160,"available":96620429312,"internally_reserved":0,"allocated":729088,"data_stored":357592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.52600000000000002,"5min":0.65800000000000003,"15min":0.69899999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.65100000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.28000000000000003,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.60899999999999999,"15min":0.65500000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.55400000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.314},{"interface":"front","average":{"1min":0.37,"5min":0.39300000000000002,"15min":0.38600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.64000000000000001},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.94199999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":0.874,"15min":0.95499999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.88900000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86799999999999999,"5min":0.90500000000000003,"15min":0.91600000000000004},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64200000000000002,"15min":0.61899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.087,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66200000000000003,"15min":0.64200000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41699999999999998},"max":{"1min":0.78200000000000003,"5min":1.204,"15min":1.204},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96899999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84399999999999997,"5min":0.876,"15min":0.91900000000000004},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10816,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":10112,"kb_avail":94361024,"statfs":{"total":96636764160,"available":96625688576,"internally_reserved":0,"allocated":712704,"data_stored":349767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10354688},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.372,"15min":0.35299999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53700000000000003,"5min":0.58899999999999997,"15min":0.60199999999999998},"last":0.254},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.38900000000000001,"15min":0.372},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.224},"max":{"1min":0.47499999999999998,"5min":0.54800000000000004,"15min":0.67100000000000004},"last":0.312}]},{"osd":2,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.49099999999999999,"5min":0.53000000000000003,"15min":0.57799999999999996},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.40799999999999997,"15min":0.39700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.66900000000000004},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.24199999999999999},"max":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42499999999999999,"15min":0.40600000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.56599999999999995,"5min":0.61199999999999999,"15min":0.64400000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.81599999999999995,"5min":0.94699999999999995,"15min":1.117},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":1.2170000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.35299999999999998},"max":{"1min":0.86499999999999999,"5min":0.93200000000000005,"15min":1.157},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.65300000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.224},"last":0.83699999999999997}]},{"osd":6,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":0.92200000000000004,"5min":0.97999999999999998,"15min":1.177},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.433,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.90000000000000002,"15min":1.0669999999999999},"last":0.82199999999999995}]},{"osd":7,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":0.97199999999999998,"15min":1.196},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16728,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":16000,"kb_avail":94355112,"statfs":{"total":96636764160,"available":96619634688,"internally_reserved":0,"allocated":737280,"data_stored":359038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16384000},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.35399999999999998},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.184},"max":{"1min":0.53100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38800000000000001,"15min":0.36799999999999999},"min":{"1min":0.249,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.59399999999999997,"5min":0.626,"15min":0.626},"last":0.23000000000000001}]},{"osd":1,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.39900000000000002},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.56699999999999995,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64200000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41199999999999998,"15min":0.38300000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.22500000000000001},"max":{"1min":0.58299999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46400000000000002,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.26400000000000001},"max":{"1min":0.61899999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.309}]},{"osd":4,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.81999999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.871,"5min":1.113,"15min":1.274},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82399999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66700000000000004,"15min":0.622},"min":{"1min":0.434,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86099999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.622},"min":{"1min":0.374,"5min":0.374,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.47,"15min":1.47},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.63200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66000000000000003,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.93899999999999995,"15min":0.95899999999999996},"last":0.76500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19800,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":19072,"kb_avail":94352040,"statfs":{"total":96636764160,"available":96616488960,"internally_reserved":0,"allocated":737280,"data_stored":358168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19529728},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.37,"15min":0.34999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.60199999999999998},"last":0.223},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38500000000000001,"15min":0.36199999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.505,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.20999999999999999}]},{"osd":1,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.38300000000000001,"15min":0.37},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.61699999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":0.61099999999999999,"15min":0.66200000000000003},"last":0.26300000000000001}]},{"osd":2,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54300000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.438,"15min":0.41899999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.375}]},{"osd":4,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64300000000000002,"15min":0.621},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.32200000000000001},"max":{"1min":0.83499999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47199999999999998,"5min":0.41399999999999998,"15min":0.39000000000000001},"max":{"1min":0.80000000000000004,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.747,"5min":0.93100000000000005,"15min":1.0489999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.51600000000000001,"5min":0.47199999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.63800000000000001},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.38700000000000001},"max":{"1min":0.79400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67800000000000005,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.98599999999999999,"15min":1.0049999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66500000000000004,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.879,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42399999999999999},"max":{"1min":0.92200000000000004,"5min":0.93799999999999994,"15min":1.02},"last":0.69999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16080,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94355760,"statfs":{"total":96636764160,"available":96620298240,"internally_reserved":0,"allocated":729088,"data_stored":358056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.32300000000000001,"15min":0.26200000000000001},"max":{"1min":0.76200000000000001,"5min":0.78400000000000003,"15min":0.83199999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.53900000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.81200000000000006},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.78300000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.47999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.66300000000000003}]},{"osd":2,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.35399999999999998,"15min":0.33900000000000002},"max":{"1min":0.73199999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.626},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.999},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.97999999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.629,"15min":0.60999999999999999},"min":{"1min":0.47899999999999998,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.751}]},{"osd":5,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":1.0640000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.39500000000000002,"5min":0.33700000000000002,"15min":0.318},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.84499999999999997,"15min":0.94599999999999995},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.438,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.2689999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.45500000000000002,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57299999999999995,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.89200000000000002,"15min":1.046},"last":0.82599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19060,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":18368,"kb_avail":94352780,"statfs":{"total":96636764160,"available":96617246720,"internally_reserved":0,"allocated":700416,"data_stored":342532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18808832},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.61299999999999999,"15min":0.55900000000000005},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.314},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.441},{"interface":"front","average":{"1min":0.75,"5min":0.63,"15min":0.58699999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.9930000000000003,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.498}]},{"osd":2,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.629,"15min":0.58799999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.2800000000000002,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.51500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.309},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.50800000000000001},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.313},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.372,"15min":0.33700000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.47399999999999998}]},{"osd":7,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.54700000000000004,"15min":0.55700000000000005},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.38100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70399999999999996,"5min":0.86599999999999999,"15min":0.878},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:14:57.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:14:57.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.342+0000 7f6a49d81700 1 Processor -- start 2023-05-16T07:14:57.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.342+0000 7f6a49d81700 1 -- start start 2023-05-16T07:14:57.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a49d81700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6a440c7380 0x7f6a44154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a49d81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 0x7f6a4414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a49d81700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a44158970 con 0x7f6a440c7760 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a49d81700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a44155340 con 0x7f6a440c7380 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a49d81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44157d60 con 0x7f6a440ad260 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a43fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6a440c7760 0x7f6a440acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54936/0 (socket says 172.21.15.92:54936) 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a43fff700 1 -- 172.21.15.92:0/3751865285 learned_addr learned my addr 172.21.15.92:0/3751865285 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887727059 0 0) 0x7f6a44158970 con 0x7f6a440c7760 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a340033f0 con 0x7f6a440c7760 2023-05-16T07:14:57.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a49580700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 0x7f6a4414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/3751865285 >> v2:172.21.15.158:3300/0 conn(0x7f6a440c7380 0x7f6a44154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951211168 0 0) 0x7f6a340033f0 con 0x7f6a440c7760 2023-05-16T07:14:57.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 msgr2=0x7f6a4414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 0x7f6a4414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 >> v2:172.21.15.158:3300/0 conn(0x7f6a440c7380 msgr2=0x7f6a44154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 --2- 172.21.15.92:0/3751865285 >> v2:172.21.15.158:3300/0 conn(0x7f6a440c7380 0x7f6a44154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440ad6c0 con 0x7f6a440c7760 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6a30001470 con 0x7f6a440c7760 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/3751865285 >> v2:172.21.15.158:3300/0 conn(0x7f6a440c7380 0x7f6a44154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 >> v1:172.21.15.92:6789/0 conn(0x7f6a440c7760 legacy=0x7f6a440acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a340038e0 0x7f6a34003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.343+0000 7f6a437fe700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a34004690 0x7f6a34004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 0x7f6a34005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44155340 con 0x7f6a340038e0 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44157d60 con 0x7f6a34004690 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3400b8e0 con 0x7f6a440ad260 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a340038e0 0x7f6a34003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a43fff700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a34004690 0x7f6a34004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a49580700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 0x7f6a34005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 -- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 msgr2=0x7f6a34005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a440ad260 0x7f6a34005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 -- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a34004690 msgr2=0x7f6a34004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a34004690 0x7f6a34004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 -- 172.21.15.92:0/3751865285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340033f0 con 0x7f6a340038e0 2023-05-16T07:14:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a43fff700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a34004690 0x7f6a34004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a340038e0 0x7f6a34003d00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f6a3800bad0 tx=0x7f6a38008040).ready entity=mon.0 client_cookie=58ae0330adf060ba server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800b410 con 0x7f6a340038e0 2023-05-16T07:14:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3800d5d0 con 0x7f6a340038e0 2023-05-16T07:14:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a437fe700 1 -- 172.21.15.92:0/3751865285 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800d950 con 0x7f6a340038e0 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a49d81700 1 -- 172.21.15.92:0/3751865285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a340038e0 msgr2=0x7f6a34003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.344+0000 7f6a49d81700 1 --2- 172.21.15.92:0/3751865285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a340038e0 0x7f6a34003d00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f6a3800bad0 tx=0x7f6a38008040).stop 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.345+0000 7f6a49580700 1 -- 172.21.15.92:0/3751865285 reap_dead start 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.345+0000 7f6a49d81700 1 -- 172.21.15.92:0/3751865285 shutdown_connections 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.345+0000 7f6a49d81700 1 -- 172.21.15.92:0/3751865285 >> 172.21.15.92:0/3751865285 conn(0x7f6a44068330 msgr2=0x7f6a44068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.345+0000 7f6a49d81700 1 -- 172.21.15.92:0/3751865285 shutdown_connections 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.345+0000 7f6a49d81700 1 -- 172.21.15.92:0/3751865285 wait complete. 2023-05-16T07:14:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.346+0000 7f6a49d81700 1 Processor -- start 2023-05-16T07:14:57.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.346+0000 7f6a49d81700 1 -- start start 2023-05-16T07:14:57.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a340038e0 0x7f6a440c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a440c9ec0 0x7f6a440cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 0x7f6a4414d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440cafb0 con 0x7f6a4414cf00 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4414e950 con 0x7f6a440c9ec0 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440ca3a0 con 0x7f6a340038e0 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 0x7f6a4414d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 0x7f6a4414d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44204/0 (socket says 172.21.15.92:44204) 2023-05-16T07:14:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 -- 172.21.15.92:0/987719185 learned_addr learned my addr 172.21.15.92:0/987719185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a340038e0 0x7f6a440c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a43fff700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a440c9ec0 0x7f6a440cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 -- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a340038e0 msgr2=0x7f6a440c2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a340038e0 0x7f6a440c2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 -- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a440c9ec0 msgr2=0x7f6a440cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a440c9ec0 0x7f6a440cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 -- 172.21.15.92:0/987719185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440ca680 con 0x7f6a4414cf00 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a340038e0 0x7f6a440c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49580700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 0x7f6a4414d340 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f6a4406aad0 tx=0x7f6a3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00e070 con 0x7f6a4414cf00 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c011c30 con 0x7f6a4414cf00 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c016450 con 0x7f6a4414cf00 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a440ca960 con 0x7f6a4414cf00 2023-05-16T07:14:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.347+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a440c1ec0 con 0x7f6a4414cf00 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.348+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a44047150 con 0x7f6a4414cf00 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.348+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6a3c01d030 con 0x7f6a4414cf00 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.349+0000 7f6a40ff9700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a2805ed90 0x7f6a28061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.349+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f6a3c07a250 con 0x7f6a4414cf00 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.349+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a2805ed90 0x7f6a28061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.350+0000 7f6a48d7f700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a2805ed90 0x7f6a28061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6a3800b210 tx=0x7f6a38008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.353+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6a3c04f0a0 con 0x7f6a4414cf00 2023-05-16T07:14:57.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.511+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a4414dc50 con 0x7f6a2805ed90 2023-05-16T07:14:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.521+0000 7f6a40ff9700 1 -- 172.21.15.92:0/987719185 <== mgr.5275 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361339 (secure 0 0 0) 0x7f6a4414dc50 con 0x7f6a2805ed90 2023-05-16T07:14:57.532 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:57.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a2805ed90 msgr2=0x7f6a28061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a2805ed90 0x7f6a28061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6a3800b210 tx=0x7f6a38008040).stop 2023-05-16T07:14:57.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 msgr2=0x7f6a4414d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 0x7f6a4414d340 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f6a4406aad0 tx=0x7f6a3c00b040).stop 2023-05-16T07:14:57.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 shutdown_connections 2023-05-16T07:14:57.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a2805ed90 0x7f6a28061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a340038e0 0x7f6a440c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a4414cf00 0x7f6a4414d340 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 --2- 172.21.15.92:0/987719185 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a440c9ec0 0x7f6a440cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 >> 172.21.15.92:0/987719185 conn(0x7f6a44068330 msgr2=0x7f6a44152020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:57.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 shutdown_connections 2023-05-16T07:14:57.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.525+0000 7f6a49d81700 1 -- 172.21.15.92:0/987719185 wait complete. 2023-05-16T07:14:57.538 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:14:57.540 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:14:55.757931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":130440,"kb_used_data":5640,"kb_used_omap":0,"kb_used_meta":124736,"kb_avail":754844280,"statfs":{"total":773094113280,"available":772960542720,"internally_reserved":0,"allocated":5775360,"data_stored":2825257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":127729664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15001262,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-723,"num_write_kb":-15553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.587893"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.237311+0000","last_change":"2023-05-16T07:14:54.237311+0000","last_active":"2023-05-16T07:14:54.237311+0000","last_peered":"2023-05-16T07:14:54.237311+0000","last_clean":"2023-05-16T07:14:54.237311+0000","last_became_active":"2023-05-16T07:14:54.237033+0000","last_became_peered":"2023-05-16T07:14:54.237033+0000","last_unstale":"2023-05-16T07:14:54.237311+0000","last_undegraded":"2023-05-16T07:14:54.237311+0000","last_fullsized":"2023-05-16T07:14:54.237311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248753+0000","last_change":"2023-05-16T07:14:53.875065+0000","last_active":"2023-05-16T07:14:54.248753+0000","last_peered":"2023-05-16T07:14:54.248753+0000","last_clean":"2023-05-16T07:14:54.248753+0000","last_became_active":"2023-05-16T07:14:53.874454+0000","last_became_peered":"2023-05-16T07:14:53.874454+0000","last_unstale":"2023-05-16T07:14:54.248753+0000","last_undegraded":"2023-05-16T07:14:54.248753+0000","last_fullsized":"2023-05-16T07:14:54.248753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247339+0000","last_change":"2023-05-16T07:14:54.247339+0000","last_active":"2023-05-16T07:14:54.247339+0000","last_peered":"2023-05-16T07:14:54.247339+0000","last_clean":"2023-05-16T07:14:54.247339+0000","last_became_active":"2023-05-16T07:14:54.246922+0000","last_became_peered":"2023-05-16T07:14:54.246922+0000","last_unstale":"2023-05-16T07:14:54.247339+0000","last_undegraded":"2023-05-16T07:14:54.247339+0000","last_fullsized":"2023-05-16T07:14:54.247339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875808+0000","last_change":"2023-05-16T07:14:53.874136+0000","last_active":"2023-05-16T07:14:53.875808+0000","last_peered":"2023-05-16T07:14:53.875808+0000","last_clean":"2023-05-16T07:14:53.875808+0000","last_became_active":"2023-05-16T07:14:53.873784+0000","last_became_peered":"2023-05-16T07:14:53.873784+0000","last_unstale":"2023-05-16T07:14:53.875808+0000","last_undegraded":"2023-05-16T07:14:53.875808+0000","last_fullsized":"2023-05-16T07:14:53.875808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246357+0000","last_change":"2023-05-16T07:14:54.246357+0000","last_active":"2023-05-16T07:14:54.246357+0000","last_peered":"2023-05-16T07:14:54.246357+0000","last_clean":"2023-05-16T07:14:54.246357+0000","last_became_active":"2023-05-16T07:14:54.246078+0000","last_became_peered":"2023-05-16T07:14:54.246078+0000","last_unstale":"2023-05-16T07:14:54.246357+0000","last_undegraded":"2023-05-16T07:14:54.246357+0000","last_fullsized":"2023-05-16T07:14:54.246357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.874675+0000","last_change":"2023-05-16T07:14:53.872565+0000","last_active":"2023-05-16T07:14:53.874675+0000","last_peered":"2023-05-16T07:14:53.874675+0000","last_clean":"2023-05-16T07:14:53.874675+0000","last_became_active":"2023-05-16T07:14:53.872216+0000","last_became_peered":"2023-05-16T07:14:53.872216+0000","last_unstale":"2023-05-16T07:14:53.874675+0000","last_undegraded":"2023-05-16T07:14:53.874675+0000","last_fullsized":"2023-05-16T07:14:53.874675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.239789+0000","last_change":"2023-05-16T07:14:54.239789+0000","last_active":"2023-05-16T07:14:54.239789+0000","last_peered":"2023-05-16T07:14:54.239789+0000","last_clean":"2023-05-16T07:14:54.239789+0000","last_became_active":"2023-05-16T07:14:54.239492+0000","last_became_peered":"2023-05-16T07:14:54.239492+0000","last_unstale":"2023-05-16T07:14:54.239789+0000","last_undegraded":"2023-05-16T07:14:54.239789+0000","last_fullsized":"2023-05-16T07:14:54.239789+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250339+0000","last_change":"2023-05-16T07:14:53.876780+0000","last_active":"2023-05-16T07:14:54.250339+0000","last_peered":"2023-05-16T07:14:54.250339+0000","last_clean":"2023-05-16T07:14:54.250339+0000","last_became_active":"2023-05-16T07:14:53.876143+0000","last_became_peered":"2023-05-16T07:14:53.876143+0000","last_unstale":"2023-05-16T07:14:54.250339+0000","last_undegraded":"2023-05-16T07:14:54.250339+0000","last_fullsized":"2023-05-16T07:14:54.250339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.242290+0000","last_change":"2023-05-16T07:14:54.242290+0000","last_active":"2023-05-16T07:14:54.242290+0000","last_peered":"2023-05-16T07:14:54.242290+0000","last_clean":"2023-05-16T07:14:54.242290+0000","last_became_active":"2023-05-16T07:14:54.242046+0000","last_became_peered":"2023-05-16T07:14:54.242046+0000","last_unstale":"2023-05-16T07:14:54.242290+0000","last_undegraded":"2023-05-16T07:14:54.242290+0000","last_fullsized":"2023-05-16T07:14:54.242290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.230463+0000","last_change":"2023-05-16T07:14:53.866415+0000","last_active":"2023-05-16T07:14:54.230463+0000","last_peered":"2023-05-16T07:14:54.230463+0000","last_clean":"2023-05-16T07:14:54.230463+0000","last_became_active":"2023-05-16T07:14:53.866109+0000","last_became_peered":"2023-05-16T07:14:53.866109+0000","last_unstale":"2023-05-16T07:14:54.230463+0000","last_undegraded":"2023-05-16T07:14:54.230463+0000","last_fullsized":"2023-05-16T07:14:54.230463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253768+0000","last_change":"2023-05-16T07:14:54.253768+0000","last_active":"2023-05-16T07:14:54.253768+0000","last_peered":"2023-05-16T07:14:54.253768+0000","last_clean":"2023-05-16T07:14:54.253768+0000","last_became_active":"2023-05-16T07:14:54.253513+0000","last_became_peered":"2023-05-16T07:14:54.253513+0000","last_unstale":"2023-05-16T07:14:54.253768+0000","last_undegraded":"2023-05-16T07:14:54.253768+0000","last_fullsized":"2023-05-16T07:14:54.253768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235592+0000","last_change":"2023-05-16T07:14:53.871132+0000","last_active":"2023-05-16T07:14:54.235592+0000","last_peered":"2023-05-16T07:14:54.235592+0000","last_clean":"2023-05-16T07:14:54.235592+0000","last_became_active":"2023-05-16T07:14:53.870861+0000","last_became_peered":"2023-05-16T07:14:53.870861+0000","last_unstale":"2023-05-16T07:14:54.235592+0000","last_undegraded":"2023-05-16T07:14:54.235592+0000","last_fullsized":"2023-05-16T07:14:54.235592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.237305+0000","last_change":"2023-05-16T07:14:54.237305+0000","last_active":"2023-05-16T07:14:54.237305+0000","last_peered":"2023-05-16T07:14:54.237305+0000","last_clean":"2023-05-16T07:14:54.237305+0000","last_became_active":"2023-05-16T07:14:54.237028+0000","last_became_peered":"2023-05-16T07:14:54.237028+0000","last_unstale":"2023-05-16T07:14:54.237305+0000","last_undegraded":"2023-05-16T07:14:54.237305+0000","last_fullsized":"2023-05-16T07:14:54.237305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.879863+0000","last_change":"2023-05-16T07:14:53.879159+0000","last_active":"2023-05-16T07:14:53.879863+0000","last_peered":"2023-05-16T07:14:53.879863+0000","last_clean":"2023-05-16T07:14:53.879863+0000","last_became_active":"2023-05-16T07:14:53.878901+0000","last_became_peered":"2023-05-16T07:14:53.878901+0000","last_unstale":"2023-05-16T07:14:53.879863+0000","last_undegraded":"2023-05-16T07:14:53.879863+0000","last_fullsized":"2023-05-16T07:14:53.879863+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250642+0000","last_change":"2023-05-16T07:14:54.250642+0000","last_active":"2023-05-16T07:14:54.250642+0000","last_peered":"2023-05-16T07:14:54.250642+0000","last_clean":"2023-05-16T07:14:54.250642+0000","last_became_active":"2023-05-16T07:14:54.250351+0000","last_became_peered":"2023-05-16T07:14:54.250351+0000","last_unstale":"2023-05-16T07:14:54.250642+0000","last_undegraded":"2023-05-16T07:14:54.250642+0000","last_fullsized":"2023-05-16T07:14:54.250642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881670+0000","last_change":"2023-05-16T07:14:53.875938+0000","last_active":"2023-05-16T07:14:53.881670+0000","last_peered":"2023-05-16T07:14:53.881670+0000","last_clean":"2023-05-16T07:14:53.881670+0000","last_became_active":"2023-05-16T07:14:53.875596+0000","last_became_peered":"2023-05-16T07:14:53.875596+0000","last_unstale":"2023-05-16T07:14:53.881670+0000","last_undegraded":"2023-05-16T07:14:53.881670+0000","last_fullsized":"2023-05-16T07:14:53.881670+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.252104+0000","last_change":"2023-05-16T07:14:54.252104+0000","last_active":"2023-05-16T07:14:54.252104+0000","last_peered":"2023-05-16T07:14:54.252104+0000","last_clean":"2023-05-16T07:14:54.252104+0000","last_became_active":"2023-05-16T07:14:54.251764+0000","last_became_peered":"2023-05-16T07:14:54.251764+0000","last_unstale":"2023-05-16T07:14:54.252104+0000","last_undegraded":"2023-05-16T07:14:54.252104+0000","last_fullsized":"2023-05-16T07:14:54.252104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884535+0000","last_change":"2023-05-16T07:14:53.880001+0000","last_active":"2023-05-16T07:14:53.884535+0000","last_peered":"2023-05-16T07:14:53.884535+0000","last_clean":"2023-05-16T07:14:53.884535+0000","last_became_active":"2023-05-16T07:14:53.879721+0000","last_became_peered":"2023-05-16T07:14:53.879721+0000","last_unstale":"2023-05-16T07:14:53.884535+0000","last_undegraded":"2023-05-16T07:14:53.884535+0000","last_fullsized":"2023-05-16T07:14:53.884535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245462+0000","last_change":"2023-05-16T07:14:54.245462+0000","last_active":"2023-05-16T07:14:54.245462+0000","last_peered":"2023-05-16T07:14:54.245462+0000","last_clean":"2023-05-16T07:14:54.245462+0000","last_became_active":"2023-05-16T07:14:54.245200+0000","last_became_peered":"2023-05-16T07:14:54.245200+0000","last_unstale":"2023-05-16T07:14:54.245462+0000","last_undegraded":"2023-05-16T07:14:54.245462+0000","last_fullsized":"2023-05-16T07:14:54.245462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881565+0000","last_change":"2023-05-16T07:14:53.875819+0000","last_active":"2023-05-16T07:14:53.881565+0000","last_peered":"2023-05-16T07:14:53.881565+0000","last_clean":"2023-05-16T07:14:53.881565+0000","last_became_active":"2023-05-16T07:14:53.874542+0000","last_became_peered":"2023-05-16T07:14:53.874542+0000","last_unstale":"2023-05-16T07:14:53.881565+0000","last_undegraded":"2023-05-16T07:14:53.881565+0000","last_fullsized":"2023-05-16T07:14:53.881565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249584+0000","last_change":"2023-05-16T07:14:54.249584+0000","last_active":"2023-05-16T07:14:54.249584+0000","last_peered":"2023-05-16T07:14:54.249584+0000","last_clean":"2023-05-16T07:14:54.249584+0000","last_became_active":"2023-05-16T07:14:54.249291+0000","last_became_peered":"2023-05-16T07:14:54.249291+0000","last_unstale":"2023-05-16T07:14:54.249584+0000","last_undegraded":"2023-05-16T07:14:54.249584+0000","last_fullsized":"2023-05-16T07:14:54.249584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884480+0000","last_change":"2023-05-16T07:14:53.879060+0000","last_active":"2023-05-16T07:14:53.884480+0000","last_peered":"2023-05-16T07:14:53.884480+0000","last_clean":"2023-05-16T07:14:53.884480+0000","last_became_active":"2023-05-16T07:14:53.878780+0000","last_became_peered":"2023-05-16T07:14:53.878780+0000","last_unstale":"2023-05-16T07:14:53.884480+0000","last_undegraded":"2023-05-16T07:14:53.884480+0000","last_fullsized":"2023-05-16T07:14:53.884480+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245958+0000","last_change":"2023-05-16T07:14:54.245958+0000","last_active":"2023-05-16T07:14:54.245958+0000","last_peered":"2023-05-16T07:14:54.245958+0000","last_clean":"2023-05-16T07:14:54.245958+0000","last_became_active":"2023-05-16T07:14:54.245660+0000","last_became_peered":"2023-05-16T07:14:54.245660+0000","last_unstale":"2023-05-16T07:14:54.245958+0000","last_undegraded":"2023-05-16T07:14:54.245958+0000","last_fullsized":"2023-05-16T07:14:54.245958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.243677+0000","last_change":"2023-05-16T07:14:53.872808+0000","last_active":"2023-05-16T07:14:54.243677+0000","last_peered":"2023-05-16T07:14:54.243677+0000","last_clean":"2023-05-16T07:14:54.243677+0000","last_became_active":"2023-05-16T07:14:53.872524+0000","last_became_peered":"2023-05-16T07:14:53.872524+0000","last_unstale":"2023-05-16T07:14:54.243677+0000","last_undegraded":"2023-05-16T07:14:54.243677+0000","last_fullsized":"2023-05-16T07:14:54.243677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253537+0000","last_change":"2023-05-16T07:14:54.253537+0000","last_active":"2023-05-16T07:14:54.253537+0000","last_peered":"2023-05-16T07:14:54.253537+0000","last_clean":"2023-05-16T07:14:54.253537+0000","last_became_active":"2023-05-16T07:14:54.253276+0000","last_became_peered":"2023-05-16T07:14:54.253276+0000","last_unstale":"2023-05-16T07:14:54.253537+0000","last_undegraded":"2023-05-16T07:14:54.253537+0000","last_fullsized":"2023-05-16T07:14:54.253537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.228056+0000","last_change":"2023-05-16T07:14:53.868058+0000","last_active":"2023-05-16T07:14:54.228056+0000","last_peered":"2023-05-16T07:14:54.228056+0000","last_clean":"2023-05-16T07:14:54.228056+0000","last_became_active":"2023-05-16T07:14:53.867773+0000","last_became_peered":"2023-05-16T07:14:53.867773+0000","last_unstale":"2023-05-16T07:14:54.228056+0000","last_undegraded":"2023-05-16T07:14:54.228056+0000","last_fullsized":"2023-05-16T07:14:54.228056+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.251222+0000","last_change":"2023-05-16T07:14:54.251222+0000","last_active":"2023-05-16T07:14:54.251222+0000","last_peered":"2023-05-16T07:14:54.251222+0000","last_clean":"2023-05-16T07:14:54.251222+0000","last_became_active":"2023-05-16T07:14:54.250922+0000","last_became_peered":"2023-05-16T07:14:54.250922+0000","last_unstale":"2023-05-16T07:14:54.251222+0000","last_undegraded":"2023-05-16T07:14:54.251222+0000","last_fullsized":"2023-05-16T07:14:54.251222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884577+0000","last_change":"2023-05-16T07:14:53.880178+0000","last_active":"2023-05-16T07:14:53.884577+0000","last_peered":"2023-05-16T07:14:53.884577+0000","last_clean":"2023-05-16T07:14:53.884577+0000","last_became_active":"2023-05-16T07:14:53.879915+0000","last_became_peered":"2023-05-16T07:14:53.879915+0000","last_unstale":"2023-05-16T07:14:53.884577+0000","last_undegraded":"2023-05-16T07:14:53.884577+0000","last_fullsized":"2023-05-16T07:14:53.884577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245930+0000","last_change":"2023-05-16T07:14:54.245930+0000","last_active":"2023-05-16T07:14:54.245930+0000","last_peered":"2023-05-16T07:14:54.245930+0000","last_clean":"2023-05-16T07:14:54.245930+0000","last_became_active":"2023-05-16T07:14:54.245647+0000","last_became_peered":"2023-05-16T07:14:54.245647+0000","last_unstale":"2023-05-16T07:14:54.245930+0000","last_undegraded":"2023-05-16T07:14:54.245930+0000","last_fullsized":"2023-05-16T07:14:54.245930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872588+0000","last_change":"2023-05-16T07:14:53.869230+0000","last_active":"2023-05-16T07:14:53.872588+0000","last_peered":"2023-05-16T07:14:53.872588+0000","last_clean":"2023-05-16T07:14:53.872588+0000","last_became_active":"2023-05-16T07:14:53.868934+0000","last_became_peered":"2023-05-16T07:14:53.868934+0000","last_unstale":"2023-05-16T07:14:53.872588+0000","last_undegraded":"2023-05-16T07:14:53.872588+0000","last_fullsized":"2023-05-16T07:14:53.872588+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240256+0000","last_change":"2023-05-16T07:14:54.240256+0000","last_active":"2023-05-16T07:14:54.240256+0000","last_peered":"2023-05-16T07:14:54.240256+0000","last_clean":"2023-05-16T07:14:54.240256+0000","last_became_active":"2023-05-16T07:14:54.240019+0000","last_became_peered":"2023-05-16T07:14:54.240019+0000","last_unstale":"2023-05-16T07:14:54.240256+0000","last_undegraded":"2023-05-16T07:14:54.240256+0000","last_fullsized":"2023-05-16T07:14:54.240256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247935+0000","last_change":"2023-05-16T07:14:53.878426+0000","last_active":"2023-05-16T07:14:54.247935+0000","last_peered":"2023-05-16T07:14:54.247935+0000","last_clean":"2023-05-16T07:14:54.247935+0000","last_became_active":"2023-05-16T07:14:53.878132+0000","last_became_peered":"2023-05-16T07:14:53.878132+0000","last_unstale":"2023-05-16T07:14:54.247935+0000","last_undegraded":"2023-05-16T07:14:54.247935+0000","last_fullsized":"2023-05-16T07:14:54.247935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246812+0000","last_change":"2023-05-16T07:14:54.246812+0000","last_active":"2023-05-16T07:14:54.246812+0000","last_peered":"2023-05-16T07:14:54.246812+0000","last_clean":"2023-05-16T07:14:54.246812+0000","last_became_active":"2023-05-16T07:14:54.246424+0000","last_became_peered":"2023-05-16T07:14:54.246424+0000","last_unstale":"2023-05-16T07:14:54.246812+0000","last_undegraded":"2023-05-16T07:14:54.246812+0000","last_fullsized":"2023-05-16T07:14:54.246812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872248+0000","last_change":"2023-05-16T07:14:53.868351+0000","last_active":"2023-05-16T07:14:53.872248+0000","last_peered":"2023-05-16T07:14:53.872248+0000","last_clean":"2023-05-16T07:14:53.872248+0000","last_became_active":"2023-05-16T07:14:53.868064+0000","last_became_peered":"2023-05-16T07:14:53.868064+0000","last_unstale":"2023-05-16T07:14:53.872248+0000","last_undegraded":"2023-05-16T07:14:53.872248+0000","last_fullsized":"2023-05-16T07:14:53.872248+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246398+0000","last_change":"2023-05-16T07:14:54.246398+0000","last_active":"2023-05-16T07:14:54.246398+0000","last_peered":"2023-05-16T07:14:54.246398+0000","last_clean":"2023-05-16T07:14:54.246398+0000","last_became_active":"2023-05-16T07:14:54.246112+0000","last_became_peered":"2023-05-16T07:14:54.246112+0000","last_unstale":"2023-05-16T07:14:54.246398+0000","last_undegraded":"2023-05-16T07:14:54.246398+0000","last_fullsized":"2023-05-16T07:14:54.246398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.879661+0000","last_change":"2023-05-16T07:14:53.878824+0000","last_active":"2023-05-16T07:14:53.879661+0000","last_peered":"2023-05-16T07:14:53.879661+0000","last_clean":"2023-05-16T07:14:53.879661+0000","last_became_active":"2023-05-16T07:14:53.878478+0000","last_became_peered":"2023-05-16T07:14:53.878478+0000","last_unstale":"2023-05-16T07:14:53.879661+0000","last_undegraded":"2023-05-16T07:14:53.879661+0000","last_fullsized":"2023-05-16T07:14:53.879661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253187+0000","last_change":"2023-05-16T07:14:54.253187+0000","last_active":"2023-05-16T07:14:54.253187+0000","last_peered":"2023-05-16T07:14:54.253187+0000","last_clean":"2023-05-16T07:14:54.253187+0000","last_became_active":"2023-05-16T07:14:54.252930+0000","last_became_peered":"2023-05-16T07:14:54.252930+0000","last_unstale":"2023-05-16T07:14:54.253187+0000","last_undegraded":"2023-05-16T07:14:54.253187+0000","last_fullsized":"2023-05-16T07:14:54.253187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2d","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.229345+0000","last_change":"2023-05-16T07:14:53.869498+0000","last_active":"2023-05-16T07:14:54.229345+0000","last_peered":"2023-05-16T07:14:54.229345+0000","last_clean":"2023-05-16T07:14:54.229345+0000","last_became_active":"2023-05-16T07:14:53.869288+0000","last_became_peered":"2023-05-16T07:14:53.869288+0000","last_unstale":"2023-05-16T07:14:54.229345+0000","last_undegraded":"2023-05-16T07:14:54.229345+0000","last_fullsized":"2023-05-16T07:14:54.229345+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249790+0000","last_change":"2023-05-16T07:14:54.249790+0000","last_active":"2023-05-16T07:14:54.249790+0000","last_peered":"2023-05-16T07:14:54.249790+0000","last_clean":"2023-05-16T07:14:54.249790+0000","last_became_active":"2023-05-16T07:14:54.248494+0000","last_became_peered":"2023-05-16T07:14:54.248494+0000","last_unstale":"2023-05-16T07:14:54.249790+0000","last_undegraded":"2023-05-16T07:14:54.249790+0000","last_fullsized":"2023-05-16T07:14:54.249790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881698+0000","last_change":"2023-05-16T07:14:53.878541+0000","last_active":"2023-05-16T07:14:53.881698+0000","last_peered":"2023-05-16T07:14:53.881698+0000","last_clean":"2023-05-16T07:14:53.881698+0000","last_became_active":"2023-05-16T07:14:53.878285+0000","last_became_peered":"2023-05-16T07:14:53.878285+0000","last_unstale":"2023-05-16T07:14:53.881698+0000","last_undegraded":"2023-05-16T07:14:53.881698+0000","last_fullsized":"2023-05-16T07:14:53.881698+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.234197+0000","last_change":"2023-05-16T07:14:54.234197+0000","last_active":"2023-05-16T07:14:54.234197+0000","last_peered":"2023-05-16T07:14:54.234197+0000","last_clean":"2023-05-16T07:14:54.234197+0000","last_became_active":"2023-05-16T07:14:54.233919+0000","last_became_peered":"2023-05-16T07:14:54.233919+0000","last_unstale":"2023-05-16T07:14:54.234197+0000","last_undegraded":"2023-05-16T07:14:54.234197+0000","last_fullsized":"2023-05-16T07:14:54.234197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235314+0000","last_change":"2023-05-16T07:14:53.869640+0000","last_active":"2023-05-16T07:14:54.235314+0000","last_peered":"2023-05-16T07:14:54.235314+0000","last_clean":"2023-05-16T07:14:54.235314+0000","last_became_active":"2023-05-16T07:14:53.869362+0000","last_became_peered":"2023-05-16T07:14:53.869362+0000","last_unstale":"2023-05-16T07:14:54.235314+0000","last_undegraded":"2023-05-16T07:14:54.235314+0000","last_fullsized":"2023-05-16T07:14:54.235314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255586+0000","last_change":"2023-05-16T07:14:54.255586+0000","last_active":"2023-05-16T07:14:54.255586+0000","last_peered":"2023-05-16T07:14:54.255586+0000","last_clean":"2023-05-16T07:14:54.255586+0000","last_became_active":"2023-05-16T07:14:54.255371+0000","last_became_peered":"2023-05-16T07:14:54.255371+0000","last_unstale":"2023-05-16T07:14:54.255586+0000","last_undegraded":"2023-05-16T07:14:54.255586+0000","last_fullsized":"2023-05-16T07:14:54.255586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.876203+0000","last_change":"2023-05-16T07:14:53.868462+0000","last_active":"2023-05-16T07:14:53.876203+0000","last_peered":"2023-05-16T07:14:53.876203+0000","last_clean":"2023-05-16T07:14:53.876203+0000","last_became_active":"2023-05-16T07:14:53.868212+0000","last_became_peered":"2023-05-16T07:14:53.868212+0000","last_unstale":"2023-05-16T07:14:53.876203+0000","last_undegraded":"2023-05-16T07:14:53.876203+0000","last_fullsized":"2023-05-16T07:14:53.876203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.245021+0000","last_change":"2023-05-16T07:14:54.245021+0000","last_active":"2023-05-16T07:14:54.245021+0000","last_peered":"2023-05-16T07:14:54.245021+0000","last_clean":"2023-05-16T07:14:54.245021+0000","last_became_active":"2023-05-16T07:14:54.244732+0000","last_became_peered":"2023-05-16T07:14:54.244732+0000","last_unstale":"2023-05-16T07:14:54.245021+0000","last_undegraded":"2023-05-16T07:14:54.245021+0000","last_fullsized":"2023-05-16T07:14:54.245021+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.873704+0000","last_change":"2023-05-16T07:14:53.871416+0000","last_active":"2023-05-16T07:14:53.873704+0000","last_peered":"2023-05-16T07:14:53.873704+0000","last_clean":"2023-05-16T07:14:53.873704+0000","last_became_active":"2023-05-16T07:14:53.871005+0000","last_became_peered":"2023-05-16T07:14:53.871005+0000","last_unstale":"2023-05-16T07:14:53.873704+0000","last_undegraded":"2023-05-16T07:14:53.873704+0000","last_fullsized":"2023-05-16T07:14:53.873704+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248977+0000","last_change":"2023-05-16T07:14:54.248977+0000","last_active":"2023-05-16T07:14:54.248977+0000","last_peered":"2023-05-16T07:14:54.248977+0000","last_clean":"2023-05-16T07:14:54.248977+0000","last_became_active":"2023-05-16T07:14:54.246294+0000","last_became_peered":"2023-05-16T07:14:54.246294+0000","last_unstale":"2023-05-16T07:14:54.248977+0000","last_undegraded":"2023-05-16T07:14:54.248977+0000","last_fullsized":"2023-05-16T07:14:54.248977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.873661+0000","last_change":"2023-05-16T07:14:53.871185+0000","last_active":"2023-05-16T07:14:53.873661+0000","last_peered":"2023-05-16T07:14:53.873661+0000","last_clean":"2023-05-16T07:14:53.873661+0000","last_became_active":"2023-05-16T07:14:53.870907+0000","last_became_peered":"2023-05-16T07:14:53.870907+0000","last_unstale":"2023-05-16T07:14:53.873661+0000","last_undegraded":"2023-05-16T07:14:53.873661+0000","last_fullsized":"2023-05-16T07:14:53.873661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246683+0000","last_change":"2023-05-16T07:14:54.246683+0000","last_active":"2023-05-16T07:14:54.246683+0000","last_peered":"2023-05-16T07:14:54.246683+0000","last_clean":"2023-05-16T07:14:54.246683+0000","last_became_active":"2023-05-16T07:14:54.246136+0000","last_became_peered":"2023-05-16T07:14:54.246136+0000","last_unstale":"2023-05-16T07:14:54.246683+0000","last_undegraded":"2023-05-16T07:14:54.246683+0000","last_fullsized":"2023-05-16T07:14:54.246683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872188+0000","last_change":"2023-05-16T07:14:53.868016+0000","last_active":"2023-05-16T07:14:53.872188+0000","last_peered":"2023-05-16T07:14:53.872188+0000","last_clean":"2023-05-16T07:14:53.872188+0000","last_became_active":"2023-05-16T07:14:53.867069+0000","last_became_peered":"2023-05-16T07:14:53.867069+0000","last_unstale":"2023-05-16T07:14:53.872188+0000","last_undegraded":"2023-05-16T07:14:53.872188+0000","last_fullsized":"2023-05-16T07:14:53.872188+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.244977+0000","last_change":"2023-05-16T07:14:54.244977+0000","last_active":"2023-05-16T07:14:54.244977+0000","last_peered":"2023-05-16T07:14:54.244977+0000","last_clean":"2023-05-16T07:14:54.244977+0000","last_became_active":"2023-05-16T07:14:54.244717+0000","last_became_peered":"2023-05-16T07:14:54.244717+0000","last_unstale":"2023-05-16T07:14:54.244977+0000","last_undegraded":"2023-05-16T07:14:54.244977+0000","last_fullsized":"2023-05-16T07:14:54.244977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871524+0000","last_change":"2023-05-16T07:14:53.868302+0000","last_active":"2023-05-16T07:14:53.871524+0000","last_peered":"2023-05-16T07:14:53.871524+0000","last_clean":"2023-05-16T07:14:53.871524+0000","last_became_active":"2023-05-16T07:14:53.868039+0000","last_became_peered":"2023-05-16T07:14:53.868039+0000","last_unstale":"2023-05-16T07:14:53.871524+0000","last_undegraded":"2023-05-16T07:14:53.871524+0000","last_fullsized":"2023-05-16T07:14:53.871524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250672+0000","last_change":"2023-05-16T07:14:54.250672+0000","last_active":"2023-05-16T07:14:54.250672+0000","last_peered":"2023-05-16T07:14:54.250672+0000","last_clean":"2023-05-16T07:14:54.250672+0000","last_became_active":"2023-05-16T07:14:54.250403+0000","last_became_peered":"2023-05-16T07:14:54.250403+0000","last_unstale":"2023-05-16T07:14:54.250672+0000","last_undegraded":"2023-05-16T07:14:54.250672+0000","last_fullsized":"2023-05-16T07:14:54.250672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872403+0000","last_change":"2023-05-16T07:14:53.869460+0000","last_active":"2023-05-16T07:14:53.872403+0000","last_peered":"2023-05-16T07:14:53.872403+0000","last_clean":"2023-05-16T07:14:53.872403+0000","last_became_active":"2023-05-16T07:14:53.868722+0000","last_became_peered":"2023-05-16T07:14:53.868722+0000","last_unstale":"2023-05-16T07:14:53.872403+0000","last_undegraded":"2023-05-16T07:14:53.872403+0000","last_fullsized":"2023-05-16T07:14:53.872403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.244437+0000","last_change":"2023-05-16T07:14:54.244437+0000","last_active":"2023-05-16T07:14:54.244437+0000","last_peered":"2023-05-16T07:14:54.244437+0000","last_clean":"2023-05-16T07:14:54.244437+0000","last_became_active":"2023-05-16T07:14:54.244205+0000","last_became_peered":"2023-05-16T07:14:54.244205+0000","last_unstale":"2023-05-16T07:14:54.244437+0000","last_undegraded":"2023-05-16T07:14:54.244437+0000","last_fullsized":"2023-05-16T07:14:54.244437+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875111+0000","last_change":"2023-05-16T07:14:53.872924+0000","last_active":"2023-05-16T07:14:53.875111+0000","last_peered":"2023-05-16T07:14:53.875111+0000","last_clean":"2023-05-16T07:14:53.875111+0000","last_became_active":"2023-05-16T07:14:53.872635+0000","last_became_peered":"2023-05-16T07:14:53.872635+0000","last_unstale":"2023-05-16T07:14:53.875111+0000","last_undegraded":"2023-05-16T07:14:53.875111+0000","last_fullsized":"2023-05-16T07:14:53.875111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.236122+0000","last_change":"2023-05-16T07:14:54.236122+0000","last_active":"2023-05-16T07:14:54.236122+0000","last_peered":"2023-05-16T07:14:54.236122+0000","last_clean":"2023-05-16T07:14:54.236122+0000","last_became_active":"2023-05-16T07:14:54.235515+0000","last_became_peered":"2023-05-16T07:14:54.235515+0000","last_unstale":"2023-05-16T07:14:54.236122+0000","last_undegraded":"2023-05-16T07:14:54.236122+0000","last_fullsized":"2023-05-16T07:14:54.236122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871504+0000","last_change":"2023-05-16T07:14:53.867229+0000","last_active":"2023-05-16T07:14:53.871504+0000","last_peered":"2023-05-16T07:14:53.871504+0000","last_clean":"2023-05-16T07:14:53.871504+0000","last_became_active":"2023-05-16T07:14:53.866957+0000","last_became_peered":"2023-05-16T07:14:53.866957+0000","last_unstale":"2023-05-16T07:14:53.871504+0000","last_undegraded":"2023-05-16T07:14:53.871504+0000","last_fullsized":"2023-05-16T07:14:53.871504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.251231+0000","last_change":"2023-05-16T07:14:54.251231+0000","last_active":"2023-05-16T07:14:54.251231+0000","last_peered":"2023-05-16T07:14:54.251231+0000","last_clean":"2023-05-16T07:14:54.251231+0000","last_became_active":"2023-05-16T07:14:54.251039+0000","last_became_peered":"2023-05-16T07:14:54.251039+0000","last_unstale":"2023-05-16T07:14:54.251231+0000","last_undegraded":"2023-05-16T07:14:54.251231+0000","last_fullsized":"2023-05-16T07:14:54.251231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875298+0000","last_change":"2023-05-16T07:14:53.874530+0000","last_active":"2023-05-16T07:14:53.875298+0000","last_peered":"2023-05-16T07:14:53.875298+0000","last_clean":"2023-05-16T07:14:53.875298+0000","last_became_active":"2023-05-16T07:14:53.874246+0000","last_became_peered":"2023-05-16T07:14:53.874246+0000","last_unstale":"2023-05-16T07:14:53.875298+0000","last_undegraded":"2023-05-16T07:14:53.875298+0000","last_fullsized":"2023-05-16T07:14:53.875298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.252483+0000","last_change":"2023-05-16T07:14:54.252483+0000","last_active":"2023-05-16T07:14:54.252483+0000","last_peered":"2023-05-16T07:14:54.252483+0000","last_clean":"2023-05-16T07:14:54.252483+0000","last_became_active":"2023-05-16T07:14:54.252271+0000","last_became_peered":"2023-05-16T07:14:54.252271+0000","last_unstale":"2023-05-16T07:14:54.252483+0000","last_undegraded":"2023-05-16T07:14:54.252483+0000","last_fullsized":"2023-05-16T07:14:54.252483+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872568+0000","last_change":"2023-05-16T07:14:53.869071+0000","last_active":"2023-05-16T07:14:53.872568+0000","last_peered":"2023-05-16T07:14:53.872568+0000","last_clean":"2023-05-16T07:14:53.872568+0000","last_became_active":"2023-05-16T07:14:53.868759+0000","last_became_peered":"2023-05-16T07:14:53.868759+0000","last_unstale":"2023-05-16T07:14:53.872568+0000","last_undegraded":"2023-05-16T07:14:53.872568+0000","last_fullsized":"2023-05-16T07:14:53.872568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248759+0000","last_change":"2023-05-16T07:14:54.248759+0000","last_active":"2023-05-16T07:14:54.248759+0000","last_peered":"2023-05-16T07:14:54.248759+0000","last_clean":"2023-05-16T07:14:54.248759+0000","last_became_active":"2023-05-16T07:14:54.248304+0000","last_became_peered":"2023-05-16T07:14:54.248304+0000","last_unstale":"2023-05-16T07:14:54.248759+0000","last_undegraded":"2023-05-16T07:14:54.248759+0000","last_fullsized":"2023-05-16T07:14:54.248759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871512+0000","last_change":"2023-05-16T07:14:53.866808+0000","last_active":"2023-05-16T07:14:53.871512+0000","last_peered":"2023-05-16T07:14:53.871512+0000","last_clean":"2023-05-16T07:14:53.871512+0000","last_became_active":"2023-05-16T07:14:53.866510+0000","last_became_peered":"2023-05-16T07:14:53.866510+0000","last_unstale":"2023-05-16T07:14:53.871512+0000","last_undegraded":"2023-05-16T07:14:53.871512+0000","last_fullsized":"2023-05-16T07:14:53.871512+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246140+0000","last_change":"2023-05-16T07:14:54.246140+0000","last_active":"2023-05-16T07:14:54.246140+0000","last_peered":"2023-05-16T07:14:54.246140+0000","last_clean":"2023-05-16T07:14:54.246140+0000","last_became_active":"2023-05-16T07:14:54.245830+0000","last_became_peered":"2023-05-16T07:14:54.245830+0000","last_unstale":"2023-05-16T07:14:54.246140+0000","last_undegraded":"2023-05-16T07:14:54.246140+0000","last_fullsized":"2023-05-16T07:14:54.246140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.228001+0000","last_change":"2023-05-16T07:14:53.868849+0000","last_active":"2023-05-16T07:14:54.228001+0000","last_peered":"2023-05-16T07:14:54.228001+0000","last_clean":"2023-05-16T07:14:54.228001+0000","last_became_active":"2023-05-16T07:14:53.868295+0000","last_became_peered":"2023-05-16T07:14:53.868295+0000","last_unstale":"2023-05-16T07:14:54.228001+0000","last_undegraded":"2023-05-16T07:14:54.228001+0000","last_fullsized":"2023-05-16T07:14:54.228001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246494+0000","last_change":"2023-05-16T07:14:54.246494+0000","last_active":"2023-05-16T07:14:54.246494+0000","last_peered":"2023-05-16T07:14:54.246494+0000","last_clean":"2023-05-16T07:14:54.246494+0000","last_became_active":"2023-05-16T07:14:54.246204+0000","last_became_peered":"2023-05-16T07:14:54.246204+0000","last_unstale":"2023-05-16T07:14:54.246494+0000","last_undegraded":"2023-05-16T07:14:54.246494+0000","last_fullsized":"2023-05-16T07:14:54.246494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.233227+0000","last_change":"2023-05-16T07:14:53.869975+0000","last_active":"2023-05-16T07:14:54.233227+0000","last_peered":"2023-05-16T07:14:54.233227+0000","last_clean":"2023-05-16T07:14:54.233227+0000","last_became_active":"2023-05-16T07:14:53.869688+0000","last_became_peered":"2023-05-16T07:14:53.869688+0000","last_unstale":"2023-05-16T07:14:54.233227+0000","last_undegraded":"2023-05-16T07:14:54.233227+0000","last_fullsized":"2023-05-16T07:14:54.233227+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.244309+0000","last_change":"2023-05-16T07:14:54.244309+0000","last_active":"2023-05-16T07:14:54.244309+0000","last_peered":"2023-05-16T07:14:54.244309+0000","last_clean":"2023-05-16T07:14:54.244309+0000","last_became_active":"2023-05-16T07:14:54.244008+0000","last_became_peered":"2023-05-16T07:14:54.244008+0000","last_unstale":"2023-05-16T07:14:54.244309+0000","last_undegraded":"2023-05-16T07:14:54.244309+0000","last_fullsized":"2023-05-16T07:14:54.244309+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.241536+0000","last_change":"2023-05-16T07:14:53.865961+0000","last_active":"2023-05-16T07:14:54.241536+0000","last_peered":"2023-05-16T07:14:54.241536+0000","last_clean":"2023-05-16T07:14:54.241536+0000","last_became_active":"2023-05-16T07:14:53.865677+0000","last_became_peered":"2023-05-16T07:14:53.865677+0000","last_unstale":"2023-05-16T07:14:54.241536+0000","last_undegraded":"2023-05-16T07:14:54.241536+0000","last_fullsized":"2023-05-16T07:14:54.241536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.241327+0000","last_change":"2023-05-16T07:14:54.241327+0000","last_active":"2023-05-16T07:14:54.241327+0000","last_peered":"2023-05-16T07:14:54.241327+0000","last_clean":"2023-05-16T07:14:54.241327+0000","last_became_active":"2023-05-16T07:14:54.241062+0000","last_became_peered":"2023-05-16T07:14:54.241062+0000","last_unstale":"2023-05-16T07:14:54.241327+0000","last_undegraded":"2023-05-16T07:14:54.241327+0000","last_fullsized":"2023-05-16T07:14:54.241327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871895+0000","last_change":"2023-05-16T07:14:53.871086+0000","last_active":"2023-05-16T07:14:53.871895+0000","last_peered":"2023-05-16T07:14:53.871895+0000","last_clean":"2023-05-16T07:14:53.871895+0000","last_became_active":"2023-05-16T07:14:53.870873+0000","last_became_peered":"2023-05-16T07:14:53.870873+0000","last_unstale":"2023-05-16T07:14:53.871895+0000","last_undegraded":"2023-05-16T07:14:53.871895+0000","last_fullsized":"2023-05-16T07:14:53.871895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248571+0000","last_change":"2023-05-16T07:14:54.248571+0000","last_active":"2023-05-16T07:14:54.248571+0000","last_peered":"2023-05-16T07:14:54.248571+0000","last_clean":"2023-05-16T07:14:54.248571+0000","last_became_active":"2023-05-16T07:14:54.248263+0000","last_became_peered":"2023-05-16T07:14:54.248263+0000","last_unstale":"2023-05-16T07:14:54.248571+0000","last_undegraded":"2023-05-16T07:14:54.248571+0000","last_fullsized":"2023-05-16T07:14:54.248571+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881715+0000","last_change":"2023-05-16T07:14:53.876330+0000","last_active":"2023-05-16T07:14:53.881715+0000","last_peered":"2023-05-16T07:14:53.881715+0000","last_clean":"2023-05-16T07:14:53.881715+0000","last_became_active":"2023-05-16T07:14:53.875998+0000","last_became_peered":"2023-05-16T07:14:53.875998+0000","last_unstale":"2023-05-16T07:14:53.881715+0000","last_undegraded":"2023-05-16T07:14:53.881715+0000","last_fullsized":"2023-05-16T07:14:53.881715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.253365+0000","last_change":"2023-05-16T07:14:54.253365+0000","last_active":"2023-05-16T07:14:54.253365+0000","last_peered":"2023-05-16T07:14:54.253365+0000","last_clean":"2023-05-16T07:14:54.253365+0000","last_became_active":"2023-05-16T07:14:54.253095+0000","last_became_peered":"2023-05-16T07:14:54.253095+0000","last_unstale":"2023-05-16T07:14:54.253365+0000","last_undegraded":"2023-05-16T07:14:54.253365+0000","last_fullsized":"2023-05-16T07:14:54.253365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872044+0000","last_change":"2023-05-16T07:14:53.866962+0000","last_active":"2023-05-16T07:14:53.872044+0000","last_peered":"2023-05-16T07:14:53.872044+0000","last_clean":"2023-05-16T07:14:53.872044+0000","last_became_active":"2023-05-16T07:14:53.866664+0000","last_became_peered":"2023-05-16T07:14:53.866664+0000","last_unstale":"2023-05-16T07:14:53.872044+0000","last_undegraded":"2023-05-16T07:14:53.872044+0000","last_fullsized":"2023-05-16T07:14:53.872044+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240770+0000","last_change":"2023-05-16T07:14:54.240770+0000","last_active":"2023-05-16T07:14:54.240770+0000","last_peered":"2023-05-16T07:14:54.240770+0000","last_clean":"2023-05-16T07:14:54.240770+0000","last_became_active":"2023-05-16T07:14:54.240515+0000","last_became_peered":"2023-05-16T07:14:54.240515+0000","last_unstale":"2023-05-16T07:14:54.240770+0000","last_undegraded":"2023-05-16T07:14:54.240770+0000","last_fullsized":"2023-05-16T07:14:54.240770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.873687+0000","last_change":"2023-05-16T07:14:53.871224+0000","last_active":"2023-05-16T07:14:53.873687+0000","last_peered":"2023-05-16T07:14:53.873687+0000","last_clean":"2023-05-16T07:14:53.873687+0000","last_became_active":"2023-05-16T07:14:53.870914+0000","last_became_peered":"2023-05-16T07:14:53.870914+0000","last_unstale":"2023-05-16T07:14:53.873687+0000","last_undegraded":"2023-05-16T07:14:53.873687+0000","last_fullsized":"2023-05-16T07:14:53.873687+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255828+0000","last_change":"2023-05-16T07:14:54.255828+0000","last_active":"2023-05-16T07:14:54.255828+0000","last_peered":"2023-05-16T07:14:54.255828+0000","last_clean":"2023-05-16T07:14:54.255828+0000","last_became_active":"2023-05-16T07:14:54.255463+0000","last_became_peered":"2023-05-16T07:14:54.255463+0000","last_unstale":"2023-05-16T07:14:54.255828+0000","last_undegraded":"2023-05-16T07:14:54.255828+0000","last_fullsized":"2023-05-16T07:14:54.255828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875892+0000","last_change":"2023-05-16T07:14:53.872167+0000","last_active":"2023-05-16T07:14:53.875892+0000","last_peered":"2023-05-16T07:14:53.875892+0000","last_clean":"2023-05-16T07:14:53.875892+0000","last_became_active":"2023-05-16T07:14:53.871888+0000","last_became_peered":"2023-05-16T07:14:53.871888+0000","last_unstale":"2023-05-16T07:14:53.875892+0000","last_undegraded":"2023-05-16T07:14:53.875892+0000","last_fullsized":"2023-05-16T07:14:53.875892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246596+0000","last_change":"2023-05-16T07:14:54.246596+0000","last_active":"2023-05-16T07:14:54.246596+0000","last_peered":"2023-05-16T07:14:54.246596+0000","last_clean":"2023-05-16T07:14:54.246596+0000","last_became_active":"2023-05-16T07:14:54.246359+0000","last_became_peered":"2023-05-16T07:14:54.246359+0000","last_unstale":"2023-05-16T07:14:54.246596+0000","last_undegraded":"2023-05-16T07:14:54.246596+0000","last_fullsized":"2023-05-16T07:14:54.246596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875445+0000","last_change":"2023-05-16T07:14:53.872477+0000","last_active":"2023-05-16T07:14:53.875445+0000","last_peered":"2023-05-16T07:14:53.875445+0000","last_clean":"2023-05-16T07:14:53.875445+0000","last_became_active":"2023-05-16T07:14:53.872158+0000","last_became_peered":"2023-05-16T07:14:53.872158+0000","last_unstale":"2023-05-16T07:14:53.875445+0000","last_undegraded":"2023-05-16T07:14:53.875445+0000","last_fullsized":"2023-05-16T07:14:53.875445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235331+0000","last_change":"2023-05-16T07:14:54.235331+0000","last_active":"2023-05-16T07:14:54.235331+0000","last_peered":"2023-05-16T07:14:54.235331+0000","last_clean":"2023-05-16T07:14:54.235331+0000","last_became_active":"2023-05-16T07:14:54.234976+0000","last_became_peered":"2023-05-16T07:14:54.234976+0000","last_unstale":"2023-05-16T07:14:54.235331+0000","last_undegraded":"2023-05-16T07:14:54.235331+0000","last_fullsized":"2023-05-16T07:14:54.235331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.877524+0000","last_change":"2023-05-16T07:14:53.876922+0000","last_active":"2023-05-16T07:14:53.877524+0000","last_peered":"2023-05-16T07:14:53.877524+0000","last_clean":"2023-05-16T07:14:53.877524+0000","last_became_active":"2023-05-16T07:14:53.876467+0000","last_became_peered":"2023-05-16T07:14:53.876467+0000","last_unstale":"2023-05-16T07:14:53.877524+0000","last_undegraded":"2023-05-16T07:14:53.877524+0000","last_fullsized":"2023-05-16T07:14:53.877524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249968+0000","last_change":"2023-05-16T07:14:54.249968+0000","last_active":"2023-05-16T07:14:54.249968+0000","last_peered":"2023-05-16T07:14:54.249968+0000","last_clean":"2023-05-16T07:14:54.249968+0000","last_became_active":"2023-05-16T07:14:54.249645+0000","last_became_peered":"2023-05-16T07:14:54.249645+0000","last_unstale":"2023-05-16T07:14:54.249968+0000","last_undegraded":"2023-05-16T07:14:54.249968+0000","last_fullsized":"2023-05-16T07:14:54.249968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255148+0000","last_change":"2023-05-16T07:14:53.878959+0000","last_active":"2023-05-16T07:14:54.255148+0000","last_peered":"2023-05-16T07:14:54.255148+0000","last_clean":"2023-05-16T07:14:54.255148+0000","last_became_active":"2023-05-16T07:14:53.878664+0000","last_became_peered":"2023-05-16T07:14:53.878664+0000","last_unstale":"2023-05-16T07:14:54.255148+0000","last_undegraded":"2023-05-16T07:14:54.255148+0000","last_fullsized":"2023-05-16T07:14:54.255148+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.232851+0000","last_change":"2023-05-16T07:14:54.232851+0000","last_active":"2023-05-16T07:14:54.232851+0000","last_peered":"2023-05-16T07:14:54.232851+0000","last_clean":"2023-05-16T07:14:54.232851+0000","last_became_active":"2023-05-16T07:14:54.232609+0000","last_became_peered":"2023-05-16T07:14:54.232609+0000","last_unstale":"2023-05-16T07:14:54.232851+0000","last_undegraded":"2023-05-16T07:14:54.232851+0000","last_fullsized":"2023-05-16T07:14:54.232851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.256051+0000","last_change":"2023-05-16T07:14:53.870300+0000","last_active":"2023-05-16T07:14:54.256051+0000","last_peered":"2023-05-16T07:14:54.256051+0000","last_clean":"2023-05-16T07:14:54.256051+0000","last_became_active":"2023-05-16T07:14:53.868810+0000","last_became_peered":"2023-05-16T07:14:53.868810+0000","last_unstale":"2023-05-16T07:14:54.256051+0000","last_undegraded":"2023-05-16T07:14:54.256051+0000","last_fullsized":"2023-05-16T07:14:54.256051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247341+0000","last_change":"2023-05-16T07:14:54.247341+0000","last_active":"2023-05-16T07:14:54.247341+0000","last_peered":"2023-05-16T07:14:54.247341+0000","last_clean":"2023-05-16T07:14:54.247341+0000","last_became_active":"2023-05-16T07:14:54.247084+0000","last_became_peered":"2023-05-16T07:14:54.247084+0000","last_unstale":"2023-05-16T07:14:54.247341+0000","last_undegraded":"2023-05-16T07:14:54.247341+0000","last_fullsized":"2023-05-16T07:14:54.247341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.241684+0000","last_change":"2023-05-16T07:14:53.870347+0000","last_active":"2023-05-16T07:14:54.241684+0000","last_peered":"2023-05-16T07:14:54.241684+0000","last_clean":"2023-05-16T07:14:54.241684+0000","last_became_active":"2023-05-16T07:14:53.870098+0000","last_became_peered":"2023-05-16T07:14:53.870098+0000","last_unstale":"2023-05-16T07:14:54.241684+0000","last_undegraded":"2023-05-16T07:14:54.241684+0000","last_fullsized":"2023-05-16T07:14:54.241684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255366+0000","last_change":"2023-05-16T07:14:54.255366+0000","last_active":"2023-05-16T07:14:54.255366+0000","last_peered":"2023-05-16T07:14:54.255366+0000","last_clean":"2023-05-16T07:14:54.255366+0000","last_became_active":"2023-05-16T07:14:54.255089+0000","last_became_peered":"2023-05-16T07:14:54.255089+0000","last_unstale":"2023-05-16T07:14:54.255366+0000","last_undegraded":"2023-05-16T07:14:54.255366+0000","last_fullsized":"2023-05-16T07:14:54.255366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.876802+0000","last_change":"2023-05-16T07:14:53.875530+0000","last_active":"2023-05-16T07:14:53.876802+0000","last_peered":"2023-05-16T07:14:53.876802+0000","last_clean":"2023-05-16T07:14:53.876802+0000","last_became_active":"2023-05-16T07:14:53.874783+0000","last_became_peered":"2023-05-16T07:14:53.874783+0000","last_unstale":"2023-05-16T07:14:53.876802+0000","last_undegraded":"2023-05-16T07:14:53.876802+0000","last_fullsized":"2023-05-16T07:14:53.876802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249738+0000","last_change":"2023-05-16T07:14:54.249738+0000","last_active":"2023-05-16T07:14:54.249738+0000","last_peered":"2023-05-16T07:14:54.249738+0000","last_clean":"2023-05-16T07:14:54.249738+0000","last_became_active":"2023-05-16T07:14:54.247517+0000","last_became_peered":"2023-05-16T07:14:54.247517+0000","last_unstale":"2023-05-16T07:14:54.249738+0000","last_undegraded":"2023-05-16T07:14:54.249738+0000","last_fullsized":"2023-05-16T07:14:54.249738+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875342+0000","last_change":"2023-05-16T07:14:53.872604+0000","last_active":"2023-05-16T07:14:53.875342+0000","last_peered":"2023-05-16T07:14:53.875342+0000","last_clean":"2023-05-16T07:14:53.875342+0000","last_became_active":"2023-05-16T07:14:53.872275+0000","last_became_peered":"2023-05-16T07:14:53.872275+0000","last_unstale":"2023-05-16T07:14:53.875342+0000","last_undegraded":"2023-05-16T07:14:53.875342+0000","last_fullsized":"2023-05-16T07:14:53.875342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248575+0000","last_change":"2023-05-16T07:14:54.248575+0000","last_active":"2023-05-16T07:14:54.248575+0000","last_peered":"2023-05-16T07:14:54.248575+0000","last_clean":"2023-05-16T07:14:54.248575+0000","last_became_active":"2023-05-16T07:14:54.245968+0000","last_became_peered":"2023-05-16T07:14:54.245968+0000","last_unstale":"2023-05-16T07:14:54.248575+0000","last_undegraded":"2023-05-16T07:14:54.248575+0000","last_fullsized":"2023-05-16T07:14:54.248575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.256276+0000","last_change":"2023-05-16T07:14:53.872018+0000","last_active":"2023-05-16T07:14:54.256276+0000","last_peered":"2023-05-16T07:14:54.256276+0000","last_clean":"2023-05-16T07:14:54.256276+0000","last_became_active":"2023-05-16T07:14:53.871783+0000","last_became_peered":"2023-05-16T07:14:53.871783+0000","last_unstale":"2023-05-16T07:14:54.256276+0000","last_undegraded":"2023-05-16T07:14:54.256276+0000","last_fullsized":"2023-05-16T07:14:54.256276+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.248067+0000","last_change":"2023-05-16T07:14:54.248067+0000","last_active":"2023-05-16T07:14:54.248067+0000","last_peered":"2023-05-16T07:14:54.248067+0000","last_clean":"2023-05-16T07:14:54.248067+0000","last_became_active":"2023-05-16T07:14:54.245746+0000","last_became_peered":"2023-05-16T07:14:54.245746+0000","last_unstale":"2023-05-16T07:14:54.248067+0000","last_undegraded":"2023-05-16T07:14:54.248067+0000","last_fullsized":"2023-05-16T07:14:54.248067+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.242799+0000","last_change":"2023-05-16T07:14:53.876268+0000","last_active":"2023-05-16T07:14:54.242799+0000","last_peered":"2023-05-16T07:14:54.242799+0000","last_clean":"2023-05-16T07:14:54.242799+0000","last_became_active":"2023-05-16T07:14:53.875967+0000","last_became_peered":"2023-05-16T07:14:53.875967+0000","last_unstale":"2023-05-16T07:14:54.242799+0000","last_undegraded":"2023-05-16T07:14:54.242799+0000","last_fullsized":"2023-05-16T07:14:54.242799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.255438+0000","last_change":"2023-05-16T07:14:54.255438+0000","last_active":"2023-05-16T07:14:54.255438+0000","last_peered":"2023-05-16T07:14:54.255438+0000","last_clean":"2023-05-16T07:14:54.255438+0000","last_became_active":"2023-05-16T07:14:54.255171+0000","last_became_peered":"2023-05-16T07:14:54.255171+0000","last_unstale":"2023-05-16T07:14:54.255438+0000","last_undegraded":"2023-05-16T07:14:54.255438+0000","last_fullsized":"2023-05-16T07:14:54.255438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240000+0000","last_change":"2023-05-16T07:14:53.867925+0000","last_active":"2023-05-16T07:14:54.240000+0000","last_peered":"2023-05-16T07:14:54.240000+0000","last_clean":"2023-05-16T07:14:54.240000+0000","last_became_active":"2023-05-16T07:14:53.867480+0000","last_became_peered":"2023-05-16T07:14:53.867480+0000","last_unstale":"2023-05-16T07:14:54.240000+0000","last_undegraded":"2023-05-16T07:14:54.240000+0000","last_fullsized":"2023-05-16T07:14:54.240000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250769+0000","last_change":"2023-05-16T07:14:54.250769+0000","last_active":"2023-05-16T07:14:54.250769+0000","last_peered":"2023-05-16T07:14:54.250769+0000","last_clean":"2023-05-16T07:14:54.250769+0000","last_became_active":"2023-05-16T07:14:54.250435+0000","last_became_peered":"2023-05-16T07:14:54.250435+0000","last_unstale":"2023-05-16T07:14:54.250769+0000","last_undegraded":"2023-05-16T07:14:54.250769+0000","last_fullsized":"2023-05-16T07:14:54.250769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871396+0000","last_change":"2023-05-16T07:14:53.866890+0000","last_active":"2023-05-16T07:14:53.871396+0000","last_peered":"2023-05-16T07:14:53.871396+0000","last_clean":"2023-05-16T07:14:53.871396+0000","last_became_active":"2023-05-16T07:14:53.866430+0000","last_became_peered":"2023-05-16T07:14:53.866430+0000","last_unstale":"2023-05-16T07:14:53.871396+0000","last_undegraded":"2023-05-16T07:14:53.871396+0000","last_fullsized":"2023-05-16T07:14:53.871396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.239912+0000","last_change":"2023-05-16T07:14:54.239912+0000","last_active":"2023-05-16T07:14:54.239912+0000","last_peered":"2023-05-16T07:14:54.239912+0000","last_clean":"2023-05-16T07:14:54.239912+0000","last_became_active":"2023-05-16T07:14:54.239644+0000","last_became_peered":"2023-05-16T07:14:54.239644+0000","last_unstale":"2023-05-16T07:14:54.239912+0000","last_undegraded":"2023-05-16T07:14:54.239912+0000","last_fullsized":"2023-05-16T07:14:54.239912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871793+0000","last_change":"2023-05-16T07:14:53.870093+0000","last_active":"2023-05-16T07:14:53.871793+0000","last_peered":"2023-05-16T07:14:53.871793+0000","last_clean":"2023-05-16T07:14:53.871793+0000","last_became_active":"2023-05-16T07:14:53.869831+0000","last_became_peered":"2023-05-16T07:14:53.869831+0000","last_unstale":"2023-05-16T07:14:53.871793+0000","last_undegraded":"2023-05-16T07:14:53.871793+0000","last_fullsized":"2023-05-16T07:14:53.871793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.235403+0000","last_change":"2023-05-16T07:14:54.235403+0000","last_active":"2023-05-16T07:14:54.235403+0000","last_peered":"2023-05-16T07:14:54.235403+0000","last_clean":"2023-05-16T07:14:54.235403+0000","last_became_active":"2023-05-16T07:14:54.235165+0000","last_became_peered":"2023-05-16T07:14:54.235165+0000","last_unstale":"2023-05-16T07:14:54.235403+0000","last_undegraded":"2023-05-16T07:14:54.235403+0000","last_fullsized":"2023-05-16T07:14:54.235403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.233753+0000","last_change":"2023-05-16T07:14:53.868453+0000","last_active":"2023-05-16T07:14:54.233753+0000","last_peered":"2023-05-16T07:14:54.233753+0000","last_clean":"2023-05-16T07:14:54.233753+0000","last_became_active":"2023-05-16T07:14:53.867941+0000","last_became_peered":"2023-05-16T07:14:53.867941+0000","last_unstale":"2023-05-16T07:14:54.233753+0000","last_undegraded":"2023-05-16T07:14:54.233753+0000","last_fullsized":"2023-05-16T07:14:54.233753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.232135+0000","last_change":"2023-05-16T07:14:54.232135+0000","last_active":"2023-05-16T07:14:54.232135+0000","last_peered":"2023-05-16T07:14:54.232135+0000","last_clean":"2023-05-16T07:14:54.232135+0000","last_became_active":"2023-05-16T07:14:54.231800+0000","last_became_peered":"2023-05-16T07:14:54.231800+0000","last_unstale":"2023-05-16T07:14:54.232135+0000","last_undegraded":"2023-05-16T07:14:54.232135+0000","last_fullsized":"2023-05-16T07:14:54.232135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872137+0000","last_change":"2023-05-16T07:14:53.867435+0000","last_active":"2023-05-16T07:14:53.872137+0000","last_peered":"2023-05-16T07:14:53.872137+0000","last_clean":"2023-05-16T07:14:53.872137+0000","last_became_active":"2023-05-16T07:14:53.867211+0000","last_became_peered":"2023-05-16T07:14:53.867211+0000","last_unstale":"2023-05-16T07:14:53.872137+0000","last_undegraded":"2023-05-16T07:14:53.872137+0000","last_fullsized":"2023-05-16T07:14:53.872137+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250829+0000","last_change":"2023-05-16T07:14:54.250829+0000","last_active":"2023-05-16T07:14:54.250829+0000","last_peered":"2023-05-16T07:14:54.250829+0000","last_clean":"2023-05-16T07:14:54.250829+0000","last_became_active":"2023-05-16T07:14:54.250562+0000","last_became_peered":"2023-05-16T07:14:54.250562+0000","last_unstale":"2023-05-16T07:14:54.250829+0000","last_undegraded":"2023-05-16T07:14:54.250829+0000","last_fullsized":"2023-05-16T07:14:54.250829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.884559+0000","last_change":"2023-05-16T07:14:53.880118+0000","last_active":"2023-05-16T07:14:53.884559+0000","last_peered":"2023-05-16T07:14:53.884559+0000","last_clean":"2023-05-16T07:14:53.884559+0000","last_became_active":"2023-05-16T07:14:53.879846+0000","last_became_peered":"2023-05-16T07:14:53.879846+0000","last_unstale":"2023-05-16T07:14:53.884559+0000","last_undegraded":"2023-05-16T07:14:53.884559+0000","last_fullsized":"2023-05-16T07:14:53.884559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246946+0000","last_change":"2023-05-16T07:14:54.246946+0000","last_active":"2023-05-16T07:14:54.246946+0000","last_peered":"2023-05-16T07:14:54.246946+0000","last_clean":"2023-05-16T07:14:54.246946+0000","last_became_active":"2023-05-16T07:14:54.246660+0000","last_became_peered":"2023-05-16T07:14:54.246660+0000","last_unstale":"2023-05-16T07:14:54.246946+0000","last_undegraded":"2023-05-16T07:14:54.246946+0000","last_fullsized":"2023-05-16T07:14:54.246946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.871493+0000","last_change":"2023-05-16T07:14:53.864168+0000","last_active":"2023-05-16T07:14:53.871493+0000","last_peered":"2023-05-16T07:14:53.871493+0000","last_clean":"2023-05-16T07:14:53.871493+0000","last_became_active":"2023-05-16T07:14:53.863907+0000","last_became_peered":"2023-05-16T07:14:53.863907+0000","last_unstale":"2023-05-16T07:14:53.871493+0000","last_undegraded":"2023-05-16T07:14:53.871493+0000","last_fullsized":"2023-05-16T07:14:53.871493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.250704+0000","last_change":"2023-05-16T07:14:54.250704+0000","last_active":"2023-05-16T07:14:54.250704+0000","last_peered":"2023-05-16T07:14:54.250704+0000","last_clean":"2023-05-16T07:14:54.250704+0000","last_became_active":"2023-05-16T07:14:54.250447+0000","last_became_peered":"2023-05-16T07:14:54.250447+0000","last_unstale":"2023-05-16T07:14:54.250704+0000","last_undegraded":"2023-05-16T07:14:54.250704+0000","last_fullsized":"2023-05-16T07:14:54.250704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.874037+0000","last_change":"2023-05-16T07:14:53.870272+0000","last_active":"2023-05-16T07:14:53.874037+0000","last_peered":"2023-05-16T07:14:53.874037+0000","last_clean":"2023-05-16T07:14:53.874037+0000","last_became_active":"2023-05-16T07:14:53.870006+0000","last_became_peered":"2023-05-16T07:14:53.870006+0000","last_unstale":"2023-05-16T07:14:53.874037+0000","last_undegraded":"2023-05-16T07:14:53.874037+0000","last_fullsized":"2023-05-16T07:14:53.874037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.858958+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:14:53.858958+0000","last_peered":"2023-05-16T07:14:53.858958+0000","last_clean":"2023-05-16T07:14:53.858958+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:14:53.858958+0000","last_undegraded":"2023-05-16T07:14:53.858958+0000","last_fullsized":"2023-05-16T07:14:53.858958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246691+0000","last_change":"2023-05-16T07:14:54.246691+0000","last_active":"2023-05-16T07:14:54.246691+0000","last_peered":"2023-05-16T07:14:54.246691+0000","last_clean":"2023-05-16T07:14:54.246691+0000","last_became_active":"2023-05-16T07:14:54.245947+0000","last_became_peered":"2023-05-16T07:14:54.245947+0000","last_unstale":"2023-05-16T07:14:54.246691+0000","last_undegraded":"2023-05-16T07:14:54.246691+0000","last_fullsized":"2023-05-16T07:14:54.246691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.247493+0000","last_change":"2023-05-16T07:14:53.875329+0000","last_active":"2023-05-16T07:14:54.247493+0000","last_peered":"2023-05-16T07:14:54.247493+0000","last_clean":"2023-05-16T07:14:54.247493+0000","last_became_active":"2023-05-16T07:14:53.875014+0000","last_became_peered":"2023-05-16T07:14:53.875014+0000","last_unstale":"2023-05-16T07:14:54.247493+0000","last_undegraded":"2023-05-16T07:14:54.247493+0000","last_fullsized":"2023-05-16T07:14:54.247493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.857716+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:14:53.857716+0000","last_peered":"2023-05-16T07:14:53.857716+0000","last_clean":"2023-05-16T07:14:53.857716+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:14:53.857716+0000","last_undegraded":"2023-05-16T07:14:53.857716+0000","last_fullsized":"2023-05-16T07:14:53.857716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.246032+0000","last_change":"2023-05-16T07:14:54.246032+0000","last_active":"2023-05-16T07:14:54.246032+0000","last_peered":"2023-05-16T07:14:54.246032+0000","last_clean":"2023-05-16T07:14:54.246032+0000","last_became_active":"2023-05-16T07:14:54.245611+0000","last_became_peered":"2023-05-16T07:14:54.245611+0000","last_unstale":"2023-05-16T07:14:54.246032+0000","last_undegraded":"2023-05-16T07:14:54.246032+0000","last_fullsized":"2023-05-16T07:14:54.246032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881738+0000","last_change":"2023-05-16T07:14:53.875901+0000","last_active":"2023-05-16T07:14:53.881738+0000","last_peered":"2023-05-16T07:14:53.881738+0000","last_clean":"2023-05-16T07:14:53.881738+0000","last_became_active":"2023-05-16T07:14:53.874568+0000","last_became_peered":"2023-05-16T07:14:53.874568+0000","last_unstale":"2023-05-16T07:14:53.881738+0000","last_undegraded":"2023-05-16T07:14:53.881738+0000","last_fullsized":"2023-05-16T07:14:53.881738+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.855199+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:14:53.855199+0000","last_peered":"2023-05-16T07:14:53.855199+0000","last_clean":"2023-05-16T07:14:53.855199+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:14:53.855199+0000","last_undegraded":"2023-05-16T07:14:53.855199+0000","last_fullsized":"2023-05-16T07:14:53.855199+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.239967+0000","last_change":"2023-05-16T07:14:54.239967+0000","last_active":"2023-05-16T07:14:54.239967+0000","last_peered":"2023-05-16T07:14:54.239967+0000","last_clean":"2023-05-16T07:14:54.239967+0000","last_became_active":"2023-05-16T07:14:54.239601+0000","last_became_peered":"2023-05-16T07:14:54.239601+0000","last_unstale":"2023-05-16T07:14:54.239967+0000","last_undegraded":"2023-05-16T07:14:54.239967+0000","last_fullsized":"2023-05-16T07:14:54.239967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.881589+0000","last_change":"2023-05-16T07:14:53.876019+0000","last_active":"2023-05-16T07:14:53.881589+0000","last_peered":"2023-05-16T07:14:53.881589+0000","last_clean":"2023-05-16T07:14:53.881589+0000","last_became_active":"2023-05-16T07:14:53.874789+0000","last_became_peered":"2023-05-16T07:14:53.874789+0000","last_unstale":"2023-05-16T07:14:53.881589+0000","last_undegraded":"2023-05-16T07:14:53.881589+0000","last_fullsized":"2023-05-16T07:14:53.881589+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.859820+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:14:53.859820+0000","last_peered":"2023-05-16T07:14:53.859820+0000","last_clean":"2023-05-16T07:14:53.859820+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:14:53.859820+0000","last_undegraded":"2023-05-16T07:14:53.859820+0000","last_fullsized":"2023-05-16T07:14:53.859820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.864056+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:14:53.864056+0000","last_peered":"2023-05-16T07:14:53.864056+0000","last_clean":"2023-05-16T07:14:53.864056+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:14:53.864056+0000","last_undegraded":"2023-05-16T07:14:53.864056+0000","last_fullsized":"2023-05-16T07:14:53.864056+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.240778+0000","last_change":"2023-05-16T07:14:54.240778+0000","last_active":"2023-05-16T07:14:54.240778+0000","last_peered":"2023-05-16T07:14:54.240778+0000","last_clean":"2023-05-16T07:14:54.240778+0000","last_became_active":"2023-05-16T07:14:54.240521+0000","last_became_peered":"2023-05-16T07:14:54.240521+0000","last_unstale":"2023-05-16T07:14:54.240778+0000","last_undegraded":"2023-05-16T07:14:54.240778+0000","last_fullsized":"2023-05-16T07:14:54.240778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.875370+0000","last_change":"2023-05-16T07:14:53.872657+0000","last_active":"2023-05-16T07:14:53.875370+0000","last_peered":"2023-05-16T07:14:53.875370+0000","last_clean":"2023-05-16T07:14:53.875370+0000","last_became_active":"2023-05-16T07:14:53.872338+0000","last_became_peered":"2023-05-16T07:14:53.872338+0000","last_unstale":"2023-05-16T07:14:53.875370+0000","last_undegraded":"2023-05-16T07:14:53.875370+0000","last_fullsized":"2023-05-16T07:14:53.875370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.864134+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:14:53.864134+0000","last_peered":"2023-05-16T07:14:53.864134+0000","last_clean":"2023-05-16T07:14:53.864134+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:14:53.864134+0000","last_undegraded":"2023-05-16T07:14:53.864134+0000","last_fullsized":"2023-05-16T07:14:53.864134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.243135+0000","last_change":"2023-05-16T07:14:54.243135+0000","last_active":"2023-05-16T07:14:54.243135+0000","last_peered":"2023-05-16T07:14:54.243135+0000","last_clean":"2023-05-16T07:14:54.243135+0000","last_became_active":"2023-05-16T07:14:54.242882+0000","last_became_peered":"2023-05-16T07:14:54.242882+0000","last_unstale":"2023-05-16T07:14:54.243135+0000","last_undegraded":"2023-05-16T07:14:54.243135+0000","last_fullsized":"2023-05-16T07:14:54.243135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.867815+0000","last_change":"2023-05-16T07:14:53.864638+0000","last_active":"2023-05-16T07:14:53.867815+0000","last_peered":"2023-05-16T07:14:53.867815+0000","last_clean":"2023-05-16T07:14:53.867815+0000","last_became_active":"2023-05-16T07:14:53.864340+0000","last_became_peered":"2023-05-16T07:14:53.864340+0000","last_unstale":"2023-05-16T07:14:53.867815+0000","last_undegraded":"2023-05-16T07:14:53.867815+0000","last_fullsized":"2023-05-16T07:14:53.867815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.864849+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:14:53.864849+0000","last_peered":"2023-05-16T07:14:53.864849+0000","last_clean":"2023-05-16T07:14:53.864849+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:14:53.864849+0000","last_undegraded":"2023-05-16T07:14:53.864849+0000","last_fullsized":"2023-05-16T07:14:53.864849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.249122+0000","last_change":"2023-05-16T07:14:54.249122+0000","last_active":"2023-05-16T07:14:54.249122+0000","last_peered":"2023-05-16T07:14:54.249122+0000","last_clean":"2023-05-16T07:14:54.249122+0000","last_became_active":"2023-05-16T07:14:54.248025+0000","last_became_peered":"2023-05-16T07:14:54.248025+0000","last_unstale":"2023-05-16T07:14:54.249122+0000","last_undegraded":"2023-05-16T07:14:54.249122+0000","last_fullsized":"2023-05-16T07:14:54.249122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.872280+0000","last_change":"2023-05-16T07:14:53.868419+0000","last_active":"2023-05-16T07:14:53.872280+0000","last_peered":"2023-05-16T07:14:53.872280+0000","last_clean":"2023-05-16T07:14:53.872280+0000","last_became_active":"2023-05-16T07:14:53.868129+0000","last_became_peered":"2023-05-16T07:14:53.868129+0000","last_unstale":"2023-05-16T07:14:53.872280+0000","last_undegraded":"2023-05-16T07:14:53.872280+0000","last_fullsized":"2023-05-16T07:14:53.872280+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.862427+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:14:53.862427+0000","last_peered":"2023-05-16T07:14:53.862427+0000","last_clean":"2023-05-16T07:14:53.862427+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:14:53.862427+0000","last_undegraded":"2023-05-16T07:14:53.862427+0000","last_fullsized":"2023-05-16T07:14:53.862427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-05-16T07:14:54.251529+0000","last_change":"2023-05-16T07:14:54.251529+0000","last_active":"2023-05-16T07:14:54.251529+0000","last_peered":"2023-05-16T07:14:54.251529+0000","last_clean":"2023-05-16T07:14:54.251529+0000","last_became_active":"2023-05-16T07:14:54.251247+0000","last_became_peered":"2023-05-16T07:14:54.251247+0000","last_unstale":"2023-05-16T07:14:54.251529+0000","last_undegraded":"2023-05-16T07:14:54.251529+0000","last_fullsized":"2023-05-16T07:14:54.251529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:53.847870+0000","last_clean_scrub_stamp":"2023-05-16T07:14:53.847870+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.874734+0000","last_change":"2023-05-16T07:14:53.873387+0000","last_active":"2023-05-16T07:14:53.874734+0000","last_peered":"2023-05-16T07:14:53.874734+0000","last_clean":"2023-05-16T07:14:53.874734+0000","last_became_active":"2023-05-16T07:14:53.873049+0000","last_became_peered":"2023-05-16T07:14:53.873049+0000","last_unstale":"2023-05-16T07:14:53.874734+0000","last_undegraded":"2023-05-16T07:14:53.874734+0000","last_fullsized":"2023-05-16T07:14:53.874734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:14:52.843309+0000","last_clean_scrub_stamp":"2023-05-16T07:14:52.843309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-16T07:14:53.861595+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:14:53.861595+0000","last_peered":"2023-05-16T07:14:53.861595+0000","last_clean":"2023-05-16T07:14:53.861595+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:14:53.861595+0000","last_undegraded":"2023-05-16T07:14:53.861595+0000","last_fullsized":"2023-05-16T07:14:53.861595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16180,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":15488,"kb_avail":94355660,"statfs":{"total":96636764160,"available":96620195840,"internally_reserved":0,"allocated":700416,"data_stored":342532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15859712},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.84199999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.81299999999999994},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.442},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.36799999999999999},"max":{"1min":0.753,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.311},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44600000000000001,"15min":0.46700000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.81999999999999995},"last":0.37}]},{"osd":5,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.503},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.53000000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.999},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue May 16 07:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.38500000000000001,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.95099999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.77000000000000002,"15min":0.84299999999999997},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15824,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":15104,"kb_avail":94356016,"statfs":{"total":96636764160,"available":96620560384,"internally_reserved":0,"allocated":729088,"data_stored":357572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15466496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.317},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.55300000000000005},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.29699999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.80800000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.71199999999999997,"5min":0.76400000000000001,"15min":0.83699999999999997},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72399999999999998,"5min":0.75800000000000001,"15min":0.96199999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.89400000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35999999999999999,"5min":0.34699999999999998,"15min":0.32200000000000001},"max":{"1min":4.048,"5min":4.048,"15min":4.048},"last":0.48899999999999999}]},{"osd":5,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47499999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.34699999999999998,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.92300000000000004},"last":0.60899999999999999}]},{"osd":7,"last update":"Tue May 16 07:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.622},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.36699999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15952,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94355888,"statfs":{"total":96636764160,"available":96620429312,"internally_reserved":0,"allocated":729088,"data_stored":357592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.52600000000000002,"5min":0.65800000000000003,"15min":0.69899999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.65100000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.28000000000000003,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.60899999999999999,"15min":0.65500000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.55400000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.314},{"interface":"front","average":{"1min":0.37,"5min":0.39300000000000002,"15min":0.38600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.64000000000000001},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.94199999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":0.874,"15min":0.95499999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81100000000000005,"5min":0.86899999999999999,"15min":0.88900000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86799999999999999,"5min":0.90500000000000003,"15min":0.91600000000000004},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64200000000000002,"15min":0.61899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.087,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66200000000000003,"15min":0.64200000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41699999999999998},"max":{"1min":0.78200000000000003,"5min":1.204,"15min":1.204},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue May 16 07:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96899999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84399999999999997,"5min":0.876,"15min":0.91900000000000004},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10816,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":10112,"kb_avail":94361024,"statfs":{"total":96636764160,"available":96625688576,"internally_reserved":0,"allocated":712704,"data_stored":349767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10354688},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.372,"15min":0.35299999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53700000000000003,"5min":0.58899999999999997,"15min":0.60199999999999998},"last":0.254},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.38900000000000001,"15min":0.372},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.224},"max":{"1min":0.47499999999999998,"5min":0.54800000000000004,"15min":0.67100000000000004},"last":0.312}]},{"osd":2,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.49099999999999999,"5min":0.53000000000000003,"15min":0.57799999999999996},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.40799999999999997,"15min":0.39700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.66900000000000004},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.24199999999999999},"max":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42499999999999999,"15min":0.40600000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.56599999999999995,"5min":0.61199999999999999,"15min":0.64400000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.81599999999999995,"5min":0.94699999999999995,"15min":1.117},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":1.2170000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.35299999999999998},"max":{"1min":0.86499999999999999,"5min":0.93200000000000005,"15min":1.157},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.65300000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.224},"last":0.83699999999999997}]},{"osd":6,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":0.92200000000000004,"5min":0.97999999999999998,"15min":1.177},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.433,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.90000000000000002,"15min":1.0669999999999999},"last":0.82199999999999995}]},{"osd":7,"last update":"Tue May 16 07:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":0.97199999999999998,"15min":1.196},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16728,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":16000,"kb_avail":94355112,"statfs":{"total":96636764160,"available":96619634688,"internally_reserved":0,"allocated":737280,"data_stored":359038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16384000},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.35399999999999998},"min":{"1min":0.24099999999999999,"5min":0.223,"15min":0.184},"max":{"1min":0.53100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38800000000000001,"15min":0.36799999999999999},"min":{"1min":0.249,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.59399999999999997,"5min":0.626,"15min":0.626},"last":0.23000000000000001}]},{"osd":1,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.39900000000000002},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.56699999999999995,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64200000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41199999999999998,"15min":0.38300000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.22500000000000001},"max":{"1min":0.58299999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46400000000000002,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.26400000000000001},"max":{"1min":0.61899999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.309}]},{"osd":4,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.81999999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.871,"5min":1.113,"15min":1.274},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82399999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66700000000000004,"15min":0.622},"min":{"1min":0.434,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86099999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.622},"min":{"1min":0.374,"5min":0.374,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.47,"15min":1.47},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue May 16 07:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.63200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66000000000000003,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.93899999999999995,"15min":0.95899999999999996},"last":0.76500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19800,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":19072,"kb_avail":94352040,"statfs":{"total":96636764160,"available":96616488960,"internally_reserved":0,"allocated":737280,"data_stored":358168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19529728},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.37,"15min":0.34999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.60199999999999998},"last":0.223},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.38500000000000001,"15min":0.36199999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.505,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.20999999999999999}]},{"osd":1,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.38300000000000001,"15min":0.37},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.61699999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":0.61099999999999999,"15min":0.66200000000000003},"last":0.26300000000000001}]},{"osd":2,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54300000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.438,"15min":0.41899999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.375}]},{"osd":4,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64300000000000002,"15min":0.621},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.32200000000000001},"max":{"1min":0.83499999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47199999999999998,"5min":0.41399999999999998,"15min":0.39000000000000001},"max":{"1min":0.80000000000000004,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.747,"5min":0.93100000000000005,"15min":1.0489999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.51600000000000001,"5min":0.47199999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.63800000000000001},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.38700000000000001},"max":{"1min":0.79400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67800000000000005,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.98599999999999999,"15min":1.0049999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Tue May 16 07:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66500000000000004,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.879,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42399999999999999},"max":{"1min":0.92200000000000004,"5min":0.93799999999999994,"15min":1.02},"last":0.69999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16080,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94355760,"statfs":{"total":96636764160,"available":96620298240,"internally_reserved":0,"allocated":729088,"data_stored":358056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.32300000000000001,"15min":0.26200000000000001},"max":{"1min":0.76200000000000001,"5min":0.78400000000000003,"15min":0.83199999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.53900000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.81200000000000006},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.78300000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.47999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.66300000000000003}]},{"osd":2,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.35399999999999998,"15min":0.33900000000000002},"max":{"1min":0.73199999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.626},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.999},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.97999999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.629,"15min":0.60999999999999999},"min":{"1min":0.47899999999999998,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.751}]},{"osd":5,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":1.0640000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.39500000000000002,"5min":0.33700000000000002,"15min":0.318},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.84499999999999997,"15min":0.94599999999999995},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.438,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.2689999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.45500000000000002,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57299999999999995,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.89200000000000002,"15min":1.046},"last":0.82599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19060,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":18368,"kb_avail":94352780,"statfs":{"total":96636764160,"available":96617246720,"internally_reserved":0,"allocated":700416,"data_stored":342532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18808832},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.61299999999999999,"15min":0.55900000000000005},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.314},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.441},{"interface":"front","average":{"1min":0.75,"5min":0.63,"15min":0.58699999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.9930000000000003,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.498}]},{"osd":2,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.629,"15min":0.58799999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.2800000000000002,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.51500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.309},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.50800000000000001},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.313},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.372,"15min":0.33700000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.47399999999999998}]},{"osd":7,"last update":"Tue May 16 07:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.54700000000000004,"15min":0.55700000000000005},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.38100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70399999999999996,"5min":0.86599999999999999,"15min":0.878},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:14:57.553 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:14:57.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:14:57.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.694+0000 7fb9b8322700 1 Processor -- start 2023-05-16T07:14:57.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.694+0000 7fb9b8322700 1 -- start start 2023-05-16T07:14:57.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b8322700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb9b0154090 0x7fb9b00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b8322700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b8322700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9b00aa150 0x7fb9b0068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b8322700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b00aa5c0 con 0x7fb9b00aa150 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b8322700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b00c94d0 con 0x7fb9b0154090 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b8322700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00cb7e0 con 0x7fb9b0154470 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9b00aa150 0x7fb9b0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9b00aa150 0x7fb9b0068160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44212/0 (socket says 172.21.15.92:44212) 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 -- 172.21.15.92:0/1686815392 learned_addr learned my addr 172.21.15.92:0/1686815392 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 -- 172.21.15.92:0/1686815392 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 msgr2=0x7fb9b00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 --2- 172.21.15.92:0/1686815392 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b60be700 1 --2- 172.21.15.92:0/1686815392 >> v2:172.21.15.158:3300/0 conn(0x7fb9b0154090 0x7fb9b00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 -- 172.21.15.92:0/1686815392 >> v2:172.21.15.158:3300/0 conn(0x7fb9b0154090 msgr2=0x7fb9b00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 --2- 172.21.15.92:0/1686815392 >> v2:172.21.15.158:3300/0 conn(0x7fb9b0154090 0x7fb9b00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 -- 172.21.15.92:0/1686815392 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00c88c0 con 0x7fb9b00aa150 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b60be700 1 --2- 172.21.15.92:0/1686815392 >> v2:172.21.15.158:3300/0 conn(0x7fb9b0154090 0x7fb9b00aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:57.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b68bf700 1 --2- 172.21.15.92:0/1686815392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b0068160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fb9b006bd10 tx=0x7fb9ac0090f0).ready entity=mon.0 client_cookie=d0f477f1000e657d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b50bc700 1 -- 172.21.15.92:0/1686815392 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00b600 con 0x7fb9b00aa150 2023-05-16T07:14:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b50bc700 1 -- 172.21.15.92:0/1686815392 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac00b760 con 0x7fb9b00aa150 2023-05-16T07:14:57.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.695+0000 7fb9b50bc700 1 -- 172.21.15.92:0/1686815392 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00bae0 con 0x7fb9b00aa150 2023-05-16T07:14:57.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 -- 172.21.15.92:0/1686815392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 msgr2=0x7fb9b0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 --2- 172.21.15.92:0/1686815392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b0068160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fb9b006bd10 tx=0x7fb9ac0090f0).stop 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 -- 172.21.15.92:0/1686815392 shutdown_connections 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 --2- 172.21.15.92:0/1686815392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b0068160 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 --2- 172.21.15.92:0/1686815392 >> v2:172.21.15.158:3300/0 conn(0x7fb9b0154090 0x7fb9b00aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 --2- 172.21.15.92:0/1686815392 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 -- 172.21.15.92:0/1686815392 >> 172.21.15.92:0/1686815392 conn(0x7fb9b0067920 msgr2=0x7fb9b00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 -- 172.21.15.92:0/1686815392 shutdown_connections 2023-05-16T07:14:57.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.696+0000 7fb9b8322700 1 -- 172.21.15.92:0/1686815392 wait complete. 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.697+0000 7fb9b8322700 1 Processor -- start 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.697+0000 7fb9b8322700 1 -- start start 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b8322700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b8322700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9b0154090 0x7fb9b0152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b8322700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b8322700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0152550 con 0x7fb9b00aa150 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b8322700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b014c7a0 con 0x7fb9b0154090 2023-05-16T07:14:57.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b8322700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0151940 con 0x7fb9b0154470 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44214/0 (socket says 172.21.15.92:44214) 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 -- 172.21.15.92:0/3197356864 learned_addr learned my addr 172.21.15.92:0/3197356864 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b68bf700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b58bd700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9b0154090 0x7fb9b0152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 -- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 msgr2=0x7fb9b00bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 -- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9b0154090 msgr2=0x7fb9b0152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9b0154090 0x7fb9b0152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 -- 172.21.15.92:0/3197356864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac009370 con 0x7fb9b00aa150 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.698+0000 7fb9b60be700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b014c2b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fb9a800a400 tx=0x7fb9a8007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.699+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800d650 con 0x7fb9b00aa150 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.699+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a800d7b0 con 0x7fb9b00aa150 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.699+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b0151c20 con 0x7fb9b00aa150 2023-05-16T07:14:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.699+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800db30 con 0x7fb9b00aa150 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.699+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b01521e0 con 0x7fb9b00aa150 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.699+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b0066730 con 0x7fb9b00aa150 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.700+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb9a801d060 con 0x7fb9b00aa150 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.700+0000 7fb9a67fc700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb99405ed90 0x7fb994061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.700+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7fb9a807aac0 con 0x7fb9b00aa150 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.700+0000 7fb9b58bd700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb99405ed90 0x7fb994061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.701+0000 7fb9b58bd700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb99405ed90 0x7fb994061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb9b00c5490 tx=0x7fb9a0008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:57.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.704+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb9a804be30 con 0x7fb9b00aa150 2023-05-16T07:14:57.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.929+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fb9b006c350 con 0x7fb9b00aa150 2023-05-16T07:14:57.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.932+0000 7fb9a67fc700 1 -- 172.21.15.92:0/3197356864 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb9a801b030 con 0x7fb9b00aa150 2023-05-16T07:14:57.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb99405ed90 msgr2=0x7fb994061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb99405ed90 0x7fb994061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb9b00c5490 tx=0x7fb9a0008040).stop 2023-05-16T07:14:57.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 msgr2=0x7fb9b014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:57.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b014c2b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fb9a800a400 tx=0x7fb9a8007910).stop 2023-05-16T07:14:57.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 shutdown_connections 2023-05-16T07:14:57.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb99405ed90 0x7fb994061250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb9b0154470 0x7fb9b00bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9b00aa150 0x7fb9b014c2b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 --2- 172.21.15.92:0/3197356864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb9b0154090 0x7fb9b0152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:57.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 >> 172.21.15.92:0/3197356864 conn(0x7fb9b0067920 msgr2=0x7fb9b00c0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:57.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 shutdown_connections 2023-05-16T07:14:57.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:57.934+0000 7fb9b8322700 1 -- 172.21.15.92:0/3197356864 wait complete. 2023-05-16T07:14:57.940 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:14:57.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:14:58.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.056+0000 7f9fd5b03700 1 Processor -- start 2023-05-16T07:14:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.056+0000 7f9fd5b03700 1 -- start start 2023-05-16T07:14:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.057+0000 7f9fd5b03700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 0x7f9fd0154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.057+0000 7f9fd5b03700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9fd0065000 0x7f9fd0065490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.057+0000 7f9fd5b03700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd0067560 con 0x7f9fd0068c70 2023-05-16T07:14:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.057+0000 7f9fd5b03700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd0065980 con 0x7f9fd0065000 2023-05-16T07:14:58.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.057+0000 7f9fd5b03700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd00bc9b0 con 0x7f9fd0153eb0 2023-05-16T07:14:58.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 0x7f9fd0154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9fd0065000 0x7f9fd0065490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 0x7f9fd0154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:36794/0 (socket says 172.21.15.92:36794) 2023-05-16T07:14:58.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 -- 172.21.15.92:0/3775632629 learned_addr learned my addr 172.21.15.92:0/3775632629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 -- 172.21.15.92:0/3775632629 >> v2:172.21.15.158:3300/0 conn(0x7f9fd0065000 msgr2=0x7f9fd0065490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 --2- 172.21.15.92:0/3775632629 >> v2:172.21.15.158:3300/0 conn(0x7f9fd0065000 0x7f9fd0065490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 -- 172.21.15.92:0/3775632629 >> v1:172.21.15.158:6789/0 conn(0x7f9fd0068c70 legacy=0x7f9fd0066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 -- 172.21.15.92:0/3775632629 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00bcc90 con 0x7f9fd0153eb0 2023-05-16T07:14:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fcf7fe700 1 --2- 172.21.15.92:0/3775632629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 0x7f9fd0154290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f9fc0009f20 tx=0x7f9fc0009840).ready entity=mon.2 client_cookie=e14509d7720361f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fce7fc700 1 -- 172.21.15.92:0/3775632629 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000e070 con 0x7f9fd0153eb0 2023-05-16T07:14:58.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fce7fc700 1 -- 172.21.15.92:0/3775632629 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000b9d0 con 0x7f9fd0153eb0 2023-05-16T07:14:58.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.058+0000 7f9fce7fc700 1 -- 172.21.15.92:0/3775632629 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000bd50 con 0x7f9fd0153eb0 2023-05-16T07:14:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 -- 172.21.15.92:0/3775632629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 msgr2=0x7f9fd0154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/3775632629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 0x7f9fd0154290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f9fc0009f20 tx=0x7f9fc0009840).stop 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 -- 172.21.15.92:0/3775632629 shutdown_connections 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/3775632629 >> v2:172.21.15.158:3300/0 conn(0x7f9fd0065000 0x7f9fd0065490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/3775632629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0153eb0 0x7f9fd0154290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 -- 172.21.15.92:0/3775632629 >> 172.21.15.92:0/3775632629 conn(0x7f9fd00c4430 msgr2=0x7f9fd00c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 -- 172.21.15.92:0/3775632629 shutdown_connections 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.059+0000 7f9fd5b03700 1 -- 172.21.15.92:0/3775632629 wait complete. 2023-05-16T07:14:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.060+0000 7f9fd5b03700 1 Processor -- start 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.060+0000 7f9fd5b03700 1 -- start start 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fd5b03700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0065000 0x7f9fd01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fd5b03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 0x7f9fd00a8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fd5b03700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fd01465a0 0x7f9fd01552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fd5b03700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd00a9f90 con 0x7f9fd0153eb0 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fd5b03700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd00a9380 con 0x7f9fd01465a0 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fd5b03700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd00a9660 con 0x7f9fd0065000 2023-05-16T07:14:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 0x7f9fd00a8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 0x7f9fd00a8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44228/0 (socket says 172.21.15.92:44228) 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 -- 172.21.15.92:0/1699831077 learned_addr learned my addr 172.21.15.92:0/1699831077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 -- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0065000 msgr2=0x7f9fd01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0065000 0x7f9fd01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fcffff700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fd01465a0 0x7f9fd01552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 -- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fd01465a0 msgr2=0x7f9fd01552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fd01465a0 0x7f9fd01552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 -- 172.21.15.92:0/1699831077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0009ae0 con 0x7f9fd0153eb0 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fcffff700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fd01465a0 0x7f9fd01552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.061+0000 7f9fceffd700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 0x7f9fd00a8b30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f9fb800c440 tx=0x7f9fb8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.062+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8007020 con 0x7f9fd0153eb0 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.062+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8001e90 con 0x7f9fd0153eb0 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.062+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80073b0 con 0x7f9fd0153eb0 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.062+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd00a9940 con 0x7f9fd0153eb0 2023-05-16T07:14:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.062+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd00a9020 con 0x7f9fd0153eb0 2023-05-16T07:14:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.063+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd006c350 con 0x7f9fd0153eb0 2023-05-16T07:14:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.063+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9fb8010070 con 0x7f9fd0153eb0 2023-05-16T07:14:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.063+0000 7f9fc77fe700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fb005ed90 0x7f9fb0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.063+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f9fb80793e0 con 0x7f9fd0153eb0 2023-05-16T07:14:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.063+0000 7f9fcf7fe700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fb005ed90 0x7f9fb0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.064+0000 7f9fcf7fe700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fb005ed90 0x7f9fb0061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9fc000b9a0 tx=0x7f9fc0009840).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.067+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9fb804a750 con 0x7f9fd0153eb0 2023-05-16T07:14:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.271+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fd00bcf70 con 0x7f9fd0153eb0 2023-05-16T07:14:58.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.272+0000 7f9fc77fe700 1 -- 172.21.15.92:0/1699831077 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f9fb804dd70 con 0x7f9fd0153eb0 2023-05-16T07:14:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:14:58.276 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:14:54.221623+0000","modified":"2023-05-16T07:14:56.230335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/2827835616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2827835616},{"type":"v1","addr":"172.21.15.158:6837","nonce":2827835616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-05-16T07:14:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fb005ed90 msgr2=0x7f9fb0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fb005ed90 0x7f9fb0061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9fc000b9a0 tx=0x7f9fc0009840).stop 2023-05-16T07:14:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 msgr2=0x7f9fd00a8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 0x7f9fd00a8b30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f9fb800c440 tx=0x7f9fb8007d70).stop 2023-05-16T07:14:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 shutdown_connections 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9fb005ed90 0x7f9fb0061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9fd0065000 0x7f9fd01597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9fd01465a0 0x7f9fd01552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 --2- 172.21.15.92:0/1699831077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fd0153eb0 0x7f9fd00a8b30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 >> 172.21.15.92:0/1699831077 conn(0x7f9fd00c4430 msgr2=0x7f9fd00a5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 shutdown_connections 2023-05-16T07:14:58.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.274+0000 7f9fd5b03700 1 -- 172.21.15.92:0/1699831077 wait complete. 2023-05-16T07:14:58.280 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-05-16T07:14:58.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:14:54.221623+0000', 'modified': '2023-05-16T07:14:56.230335+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5748}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5748': {'gid': 5748, 'name': 'b', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.158:6837/2827835616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6836', 'nonce': 2827835616}, {'type': 'v1', 'addr': '172.21.15.158:6837', 'nonce': 2827835616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:14:58.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:14:58.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-05-16T07:14:58.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.802+0000 7f9f08cd3700 1 Processor -- start 2023-05-16T07:14:58.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.802+0000 7f9f08cd3700 1 -- start start 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f08cd3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f040696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f08cd3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f040c0660 con 0x7f9f04069b90 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f08cd3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0406a020 con 0x7f9f04154170 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f08cd3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040bfd30 con 0x7f9f04154550 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f02d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9f04069b90 0x7f9f040c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54986/0 (socket says 172.21.15.92:54986) 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f02d9d700 1 -- 172.21.15.92:0/1880486006 learned_addr learned my addr 172.21.15.92:0/1880486006 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:58.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/1880486006 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f040696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066558248 0 0) 0x7f9f040c0660 con 0x7f9f04069b90 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.803+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef00033f0 con 0x7f9f04069b90 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008461120 0 0) 0x7f9f0406a020 con 0x7f9f04154170 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f040c0660 con 0x7f9f04154170 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282393524 0 0) 0x7f9ef00033f0 con 0x7f9f04069b90 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f01d9b700 1 -- 172.21.15.92:0/1880486006 >> v1:172.21.15.158:6789/0 conn(0x7f9f04154170 legacy=0x7f9f040c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f01d9b700 1 -- 172.21.15.92:0/1880486006 >> v1:172.21.15.92:6789/0 conn(0x7f9f04069b90 legacy=0x7f9f040c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f01d9b700 1 -- 172.21.15.92:0/1880486006 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040c0010 con 0x7f9f04154550 2023-05-16T07:14:58.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/1880486006 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f040696a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9eec009790 tx=0x7f9eec009060).ready entity=mon.2 client_cookie=71065767ea91383e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc001470 con 0x7f9f04154550 2023-05-16T07:14:58.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec0090d0 con 0x7f9f04154550 2023-05-16T07:14:58.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f0159a700 1 -- 172.21.15.92:0/1880486006 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011610 con 0x7f9f04154550 2023-05-16T07:14:58.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f08cd3700 1 -- 172.21.15.92:0/1880486006 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 msgr2=0x7f9f040696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.804+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/1880486006 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f040696a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9eec009790 tx=0x7f9eec009060).stop 2023-05-16T07:14:58.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.805+0000 7f9f08cd3700 1 -- 172.21.15.92:0/1880486006 shutdown_connections 2023-05-16T07:14:58.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.805+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/1880486006 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f040696a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.805+0000 7f9f08cd3700 1 -- 172.21.15.92:0/1880486006 >> 172.21.15.92:0/1880486006 conn(0x7f9f040cb700 msgr2=0x7f9f040c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.805+0000 7f9f08cd3700 1 -- 172.21.15.92:0/1880486006 shutdown_connections 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.805+0000 7f9f08cd3700 1 -- 172.21.15.92:0/1880486006 wait complete. 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 Processor -- start 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 -- start start 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 0x7f9f04144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f04154170 0x7f9f040c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f04153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04144f40 con 0x7f9f04069b90 2023-05-16T07:14:58.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040af5f0 con 0x7f9f04154170 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.806+0000 7f9f08cd3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04148320 con 0x7f9f04154550 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 0x7f9f04144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 0x7f9f04144a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44242/0 (socket says 172.21.15.92:44242) 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 -- 172.21.15.92:0/2009337191 learned_addr learned my addr 172.21.15.92:0/2009337191 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f04154170 0x7f9f040c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 -- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 msgr2=0x7f9f04153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f04153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f04153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 -- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f04154170 msgr2=0x7f9f040c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f04154170 0x7f9f040c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 -- 172.21.15.92:0/2009337191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0092a0 con 0x7f9f04069b90 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f04154170 0x7f9f040c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f0259c700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 0x7f9f04144a50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f9ef400c770 tx=0x7f9ef4002600).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9efa7fc700 1 -- 172.21.15.92:0/2009337191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4010510 con 0x7f9f04069b90 2023-05-16T07:14:58.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9efa7fc700 1 -- 172.21.15.92:0/2009337191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4010670 con 0x7f9f04069b90 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f040aecc0 con 0x7f9f04069b90 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9efa7fc700 1 -- 172.21.15.92:0/2009337191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4010a30 con 0x7f9f04069b90 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.807+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f040af280 con 0x7f9f04069b90 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.808+0000 7f9efa7fc700 1 -- 172.21.15.92:0/2009337191 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9ef4010070 con 0x7f9f04069b90 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.809+0000 7f9efa7fc700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9ee405ed90 0x7f9ee4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.809+0000 7f9efa7fc700 1 -- 172.21.15.92:0/2009337191 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f9ef4079be0 con 0x7f9f04069b90 2023-05-16T07:14:58.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.809+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9ee405ed90 0x7f9ee4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.810+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9ee405ed90 0x7f9ee4061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9f04154930 tx=0x7f9eec016000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.811+0000 7f9f1a665b80 1 Processor -- start 2023-05-16T07:14:58.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.811+0000 7f9f1a665b80 1 -- start start 2023-05-16T07:14:58.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e48940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6dfe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6dc95e0 con 0x564fa6e04300 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6d408a0 con 0x564fa6e4d480 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa6d55900 con 0x564fa6e48560 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6dfe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6dfe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44254/0 (socket says 172.21.15.92:44254) 2023-05-16T07:14:58.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 -- 172.21.15.92:0/3716202790 learned_addr learned my addr 172.21.15.92:0/3716202790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e48940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 -- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 msgr2=0x564fa6e2bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 -- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 msgr2=0x564fa6e48940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e48940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 -- 172.21.15.92:0/3716202790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa6dc9450 con 0x564fa6e04300 2023-05-16T07:14:58.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6dfe710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f9efc0078f0 tx=0x7f9efc009040).ready entity=mon.0 client_cookie=a1a5d3731525ab3e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9ee37fe700 1 -- 172.21.15.92:0/3716202790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc007e30 con 0x564fa6e04300 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9ee37fe700 1 -- 172.21.15.92:0/3716202790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc007170 con 0x564fa6e04300 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.812+0000 7f9ee37fe700 1 -- 172.21.15.92:0/3716202790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc007530 con 0x564fa6e04300 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3716202790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 msgr2=0x564fa6dfe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6dfe710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f9efc0078f0 tx=0x7f9efc009040).stop 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3716202790 shutdown_connections 2023-05-16T07:14:58.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6dfe710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3716202790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e48940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3716202790 >> 172.21.15.92:0/3716202790 conn(0x564fa6bc0e40 msgr2=0x564fa6e4c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3716202790 shutdown_connections 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.813+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3716202790 wait complete. 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.814+0000 7f9f1a665b80 1 -- start start 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.814+0000 7f9f1a665b80 1 Processor -- start 2023-05-16T07:14:58.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e28e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6d1ced0 con 0x564fa6e04300 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e2d5a0 con 0x564fa6e4d480 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e45db0 con 0x564fa6e48560 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 10 client.0 ll_register_callbacks cb 0x564fa6dc31f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44256/0 (socket says 172.21.15.92:44256) 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 -- 172.21.15.92:0/1506331507 learned_addr learned my addr 172.21.15.92:0/1506331507 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e28e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 -- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 msgr2=0x564fa6e2b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 -- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 msgr2=0x564fa6e28e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e28e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 -- 172.21.15.92:0/1506331507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc001c80 con 0x564fa6e04300 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f0259c700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e461d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f9ef404eb20 tx=0x7f9ef400f000).ready entity=mon.0 client_cookie=6a9a534dc93e727e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9ee0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/1506331507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef404dff0 con 0x564fa6e04300 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/1506331507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f9ef404e150 con 0x564fa6e04300 2023-05-16T07:14:58.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fa6df7460 con 0x564fa6e04300 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/1506331507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef400adb0 con 0x564fa6e04300 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 10 client.5880 fetch_fsmap learned FSMap version 18 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.815+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x564fa6bf8310 con 0x564fa6e04300 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/1506331507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564fa6df7460 con 0x564fa6e04300 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/1506331507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 18) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9ef405c070 con 0x564fa6e04300 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f1a665b80 10 client.5880 fetch_fsmap finished waiting for FSMap version 18 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f1a665b80 10 client.5880 resolve_mds: resolved name 'b' to daemon mds.b 2023-05-16T07:14:58.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f1a665b80 20 client.5880 populate_metadata read hostname 'smithi092' 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e44090 0x564fa6e44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f1a665b80 4 client.5880 mds_command: new command op to 5748 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 --> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fa6bbcca0 con 0x564fa6e44090 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e44090 0x564fa6e44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9ee0ff9700 10 client.5880 ms_handle_connect on v2:172.21.15.158:6836/2827835616 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.816+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e44090 0x564fa6e44470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=93a41883a5fcc37f server_cookie=ddad53c645fc3ec1 in_seq=0 out_seq=0 2023-05-16T07:14:58.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/1506331507 <== mds.0 v2:172.21.15.158:6836/2827835616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x564fa6bbcca0 con 0x564fa6e44090 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9ee0ff9700 10 client.5880 handle_command_reply: tid=0 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 client.5880 shutdown 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 msgr2=0x564fa6e461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e461d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f9ef404eb20 tx=0x7f9ef400f000).stop 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 shutdown_connections 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e28e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e461d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e44090 0x564fa6e44470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/1506331507 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6e2b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9ee0ff9700 0 client.5880 ms_handle_reset on v2:172.21.15.158:6836/2827835616 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 >> 172.21.15.92:0/1506331507 conn(0x564fa6bc0e40 msgr2=0x564fa6e03540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 shutdown_connections 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.817+0000 7f9f1a665b80 1 -- 172.21.15.92:0/1506331507 wait complete. 2023-05-16T07:14:58.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.818+0000 7f9f1a665b80 20 client.5880 trim_cache size 0 max 16384 2023-05-16T07:14:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.834+0000 7f9f1a665b80 1 Processor -- start 2023-05-16T07:14:58.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.834+0000 7f9f1a665b80 1 -- start start 2023-05-16T07:14:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e48560 0x564fa6e29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e4d480 0x564fa6e2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e2aeb0 con 0x564fa6e04300 2023-05-16T07:14:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e2e610 con 0x564fa6e48560 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e4c520 con 0x564fa6e4d480 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e25e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44264/0 (socket says 172.21.15.92:44264) 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 -- 172.21.15.92:0/3038036999 learned_addr learned my addr 172.21.15.92:0/3038036999 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e4d480 0x564fa6e2a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e48560 0x564fa6e29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 -- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e4d480 msgr2=0x564fa6e2a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e4d480 0x564fa6e2a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 -- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e48560 msgr2=0x564fa6e29b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e48560 0x564fa6e29b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 -- 172.21.15.92:0/3038036999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa6d4e880 con 0x564fa6e04300 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e48560 0x564fa6e29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9f0259c700 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e25e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f9ef404b520 tx=0x7f9ef4054000).ready entity=mon.0 client_cookie=bebaae252916582c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9ee3fff700 1 -- 172.21.15.92:0/3038036999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4010bd0 con 0x564fa6e04300 2023-05-16T07:14:58.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9ee3fff700 1 -- 172.21.15.92:0/3038036999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef400adb0 con 0x564fa6e04300 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.835+0000 7f9ee3fff700 1 -- 172.21.15.92:0/3038036999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef404dc20 con 0x564fa6e04300 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3038036999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 msgr2=0x564fa6e25e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e25e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f9ef404b520 tx=0x7f9ef4054000).stop 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3038036999 shutdown_connections 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e4d480 0x564fa6e2a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6e25e50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/3038036999 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e48560 0x564fa6e29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3038036999 >> 172.21.15.92:0/3038036999 conn(0x564fa6bc0e40 msgr2=0x564fa6e49880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3038036999 shutdown_connections 2023-05-16T07:14:58.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 -- 172.21.15.92:0/3038036999 wait complete. 2023-05-16T07:14:58.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 -- start start 2023-05-16T07:14:58.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.836+0000 7f9f1a665b80 1 Processor -- start 2023-05-16T07:14:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6eade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6dfdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e2a860 con 0x564fa6e04300 2023-05-16T07:14:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa6dfe960 con 0x564fa6e4d480 2023-05-16T07:14:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa6e2a9d0 con 0x564fa6e48560 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f1a665b80 10 client.0 ll_register_callbacks cb 0x564fa6dc31f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6eade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6eade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44270/0 (socket says 172.21.15.92:44270) 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 -- 172.21.15.92:0/2010463252 learned_addr learned my addr 172.21.15.92:0/2010463252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6dfdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 -- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 msgr2=0x564fa6dfdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6dfdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 -- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 msgr2=0x564fa6e473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f0259c700 1 -- 172.21.15.92:0/2010463252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fa6bf8310 con 0x564fa6e04300 2023-05-16T07:14:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.837+0000 7f9f02d9d700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f0259c700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6eade40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f9ef400cec0 tx=0x7f9ef4014000).ready entity=mon.0 client_cookie=42e374433a3f2f93 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9ee0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/2010463252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef404dbd0 con 0x564fa6e04300 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/2010463252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f9ef404dd30 con 0x564fa6e04300 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9efc007170 con 0x564fa6e04300 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/2010463252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef400a500 con 0x564fa6e04300 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 10 client.5886 fetch_fsmap learned FSMap version 18 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x564fa6d4a070 con 0x564fa6e04300 2023-05-16T07:14:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/2010463252 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc007170 con 0x564fa6e04300 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/2010463252 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 18) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9ef4019020 con 0x564fa6e04300 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 10 client.5886 fetch_fsmap finished waiting for FSMap version 18 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 10 client.5886 resolve_mds: resolved name 'b' to daemon mds.b 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 20 client.5886 populate_metadata read hostname 'smithi092' 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e29a80 0x564fa6e29e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 4 client.5886 mds_command: new command op to 5748 tid=0[{"prefix": "status"}] 2023-05-16T07:14:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 --> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564fa6bf3ca0 con 0x564fa6e29a80 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.838+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e29a80 0x564fa6e29e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9ee0ff9700 10 client.5886 ms_handle_connect on v2:172.21.15.158:6836/2827835616 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f01d9b700 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e29a80 0x564fa6e29e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fea36cc73beb5f80 server_cookie=7a31597c93105912 in_seq=0 out_seq=0 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9ee0ff9700 1 -- 172.21.15.92:0/2010463252 <== mds.0 v2:172.21.15.158:6836/2827835616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564fa6bf3ca0 con 0x564fa6e29a80 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9ee0ff9700 10 client.5886 handle_command_reply: tid=0 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 client.5886 shutdown 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 msgr2=0x564fa6eade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6eade40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f9ef400cec0 tx=0x7f9ef4014000).stop 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 shutdown_connections 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x564fa6e4d480 0x564fa6e473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa6e04300 0x564fa6eade40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:6836/2827835616,v1:172.21.15.158:6837/2827835616] conn(0x564fa6e29a80 0x564fa6e29e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 --2- 172.21.15.92:0/2010463252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x564fa6e48560 0x564fa6dfdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9ee0ff9700 0 client.5886 ms_handle_reset on v2:172.21.15.158:6836/2827835616 2023-05-16T07:14:58.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 >> 172.21.15.92:0/2010463252 conn(0x564fa6bc0e40 msgr2=0x564fa6e25c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 shutdown_connections 2023-05-16T07:14:58.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 1 -- 172.21.15.92:0/2010463252 wait complete. 2023-05-16T07:14:58.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.839+0000 7f9f1a665b80 20 client.5886 trim_cache size 0 max 16384 2023-05-16T07:14:58.854 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:14:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5748, 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.6163129749999996, 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 18, 2023-05-16T07:14:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 62, 2023-05-16T07:14:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 62, 2023-05-16T07:14:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.1314686589999994 2023-05-16T07:14:58.856 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:14:58.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9ee405ed90 msgr2=0x7f9ee4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9ee405ed90 0x7f9ee4061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9f04154930 tx=0x7f9eec016000).stop 2023-05-16T07:14:58.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 msgr2=0x7f9f04144a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 0x7f9f04144a50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f9ef400c770 tx=0x7f9ef4002600).stop 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 shutdown_connections 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9ee405ed90 0x7f9ee4061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f04154550 0x7f9f04153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04069b90 0x7f9f04144a50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 --2- 172.21.15.92:0/2009337191 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f04154170 0x7f9f040c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 >> 172.21.15.92:0/2009337191 conn(0x7f9f040cb700 msgr2=0x7f9f040ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 shutdown_connections 2023-05-16T07:14:58.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.840+0000 7f9f08cd3700 1 -- 172.21.15.92:0/2009337191 wait complete. 2023-05-16T07:14:58.859 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:14:58.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.953+0000 7fdf4c76b700 1 Processor -- start 2023-05-16T07:14:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.953+0000 7fdf4c76b700 1 -- start start 2023-05-16T07:14:58.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.954+0000 7fdf4c76b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf44153dc0 0x7fdf440c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.954+0000 7fdf4c76b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf441541a0 0x7fdf440635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.954+0000 7fdf4c76b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf440aaf40 con 0x7fdf440c5c50 2023-05-16T07:14:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.954+0000 7fdf4c76b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf440c1450 con 0x7fdf44153dc0 2023-05-16T07:14:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.954+0000 7fdf4c76b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440c1730 con 0x7fdf441541a0 2023-05-16T07:14:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf44153dc0 0x7fdf440c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf44153dc0 0x7fdf440c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44278/0 (socket says 172.21.15.92:44278) 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 -- 172.21.15.92:0/3144190011 learned_addr learned my addr 172.21.15.92:0/3144190011 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf49d06700 1 --2- 172.21.15.92:0/3144190011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf441541a0 0x7fdf440635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 -- 172.21.15.92:0/3144190011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf441541a0 msgr2=0x7fdf440635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 --2- 172.21.15.92:0/3144190011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf441541a0 0x7fdf440635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 -- 172.21.15.92:0/3144190011 >> v1:172.21.15.158:6789/0 conn(0x7fdf440c5c50 legacy=0x7fdf44064a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 -- 172.21.15.92:0/3144190011 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440c1a10 con 0x7fdf44153dc0 2023-05-16T07:14:58.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf4a507700 1 --2- 172.21.15.92:0/3144190011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf440c0bc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fdf34006fd0 tx=0x7fdf340090f0).ready entity=mon.0 client_cookie=9ea330e55d34011b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf49505700 1 -- 172.21.15.92:0/3144190011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400b600 con 0x7fdf44153dc0 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf49505700 1 -- 172.21.15.92:0/3144190011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3400b760 con 0x7fdf44153dc0 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.955+0000 7fdf49505700 1 -- 172.21.15.92:0/3144190011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400bae0 con 0x7fdf44153dc0 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 -- 172.21.15.92:0/3144190011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 msgr2=0x7fdf440c0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/3144190011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf440c0bc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fdf34006fd0 tx=0x7fdf340090f0).stop 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 -- 172.21.15.92:0/3144190011 shutdown_connections 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/3144190011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf440c0bc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/3144190011 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf441541a0 0x7fdf440635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 -- 172.21.15.92:0/3144190011 >> 172.21.15.92:0/3144190011 conn(0x7fdf440caca0 msgr2=0x7fdf440cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:58.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 -- 172.21.15.92:0/3144190011 shutdown_connections 2023-05-16T07:14:58.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.956+0000 7fdf4c76b700 1 -- 172.21.15.92:0/3144190011 wait complete. 2023-05-16T07:14:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.957+0000 7fdf4c76b700 1 Processor -- start 2023-05-16T07:14:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.957+0000 7fdf4c76b700 1 -- start start 2023-05-16T07:14:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf440c5c50 0x7fdf4414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf441545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf441541a0 0x7fdf440ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440c3d40 con 0x7fdf44153dc0 2023-05-16T07:14:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440c3130 con 0x7fdf441541a0 2023-05-16T07:14:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440c3410 con 0x7fdf440c5c50 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf441545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf441545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44286/0 (socket says 172.21.15.92:44286) 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 -- 172.21.15.92:0/31831747 learned_addr learned my addr 172.21.15.92:0/31831747 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4a507700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf440c5c50 0x7fdf4414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4ad08700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf441541a0 0x7fdf440ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 -- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf440c5c50 msgr2=0x7fdf4414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf440c5c50 0x7fdf4414e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 -- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf441541a0 msgr2=0x7fdf440ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf441541a0 0x7fdf440ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 -- 172.21.15.92:0/31831747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009370 con 0x7fdf44153dc0 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4ad08700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf441541a0 0x7fdf440ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf49d06700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf441545f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fdf40006980 tx=0x7fdf40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40011590 con 0x7fdf44153dc0 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf400116f0 con 0x7fdf44153dc0 2023-05-16T07:14:58.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf440c36f0 con 0x7fdf44153dc0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40011a70 con 0x7fdf44153dc0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.958+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4414ea30 con 0x7fdf44153dc0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.959+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf440c3ef0 con 0x7fdf44153dc0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.960+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdf40012420 con 0x7fdf44153dc0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.960+0000 7fdf3a7fc700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2805ed90 0x7fdf28061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.960+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7fdf40079820 con 0x7fdf44153dc0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.960+0000 7fdf4a507700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2805ed90 0x7fdf28061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:58.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.961+0000 7fdf4a507700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2805ed90 0x7fdf28061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fdf34009130 tx=0x7fdf34017000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:58.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:58.964+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdf440c3ef0 con 0x7fdf44153dc0 2023-05-16T07:14:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.168+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf4405efa0 con 0x7fdf44153dc0 2023-05-16T07:14:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.168+0000 7fdf3a7fc700 1 -- 172.21.15.92:0/31831747 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf4004ab30 con 0x7fdf44153dc0 2023-05-16T07:14:59.173 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:14:59.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2805ed90 msgr2=0x7fdf28061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2805ed90 0x7fdf28061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fdf34009130 tx=0x7fdf34017000).stop 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 msgr2=0x7fdf441545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf441545f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fdf40006980 tx=0x7fdf40008040).stop 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 shutdown_connections 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2805ed90 0x7fdf28061250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf440c5c50 0x7fdf4414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf441541a0 0x7fdf440ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 --2- 172.21.15.92:0/31831747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf44153dc0 0x7fdf441545f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 >> 172.21.15.92:0/31831747 conn(0x7fdf440caca0 msgr2=0x7fdf44063500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:59.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.170+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 shutdown_connections 2023-05-16T07:14:59.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.171+0000 7fdf4c76b700 1 -- 172.21.15.92:0/31831747 wait complete. 2023-05-16T07:14:59.185 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:14:59.185 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:14:59.185 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:14:59.186 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:14:59.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:14:59.186 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:14:59.186 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:14:59.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:14:59.187 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:14:59.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.188 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:14:59.202 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:14:59.203 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:14:59.203 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:14:59.203 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.203 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:14:59.203 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.203 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:14:59.204 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41256sec preferred_lft 41256sec 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:14:59.205 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.206 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:14:59.207 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:14:59.207 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:14:59.207 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:14:59.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.208 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:14:59.264 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:14:59.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.266 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:14:59.321 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:14:59.321 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:14:59.321 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:14:59.321 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:14:59.322 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:14:59.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:14:59.323 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:14:59.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:14:59.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:14:59.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:14:59.324 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:14:59.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.419 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:14:59.433 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:14:59.434 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:14:59.435 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:14:59.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:14:59.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.498 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:14:59.552 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:14:59.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:14:59.635 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:14:59.635 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:14:59.677 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:14:59.677 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:14:59.679 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:14:59.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:14:59.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:14:59.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:14:59.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.893+0000 7f892082f700 1 Processor -- start 2023-05-16T07:14:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.893+0000 7f892082f700 1 -- start start 2023-05-16T07:14:59.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.893+0000 7f892082f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f891c069750 0x7f891c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.893+0000 7f892082f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f891c0aab40 0x7f891c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.893+0000 7f892082f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c06a920 con 0x7f891c0c5f70 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.893+0000 7f892082f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0c65f0 con 0x7f891c0aab40 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f892082f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0bfa30 con 0x7f891c069750 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f891c0aab40 0x7f891c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f891c0c5f70 0x7f891c153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55048/0 (socket says 172.21.15.92:55048) 2023-05-16T07:14:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891bfff700 1 -- 172.21.15.92:0/784179303 learned_addr learned my addr 172.21.15.92:0/784179303 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891b7fe700 1 --2- 172.21.15.92:0/784179303 >> v2:172.21.15.158:3300/0 conn(0x7f891c069750 0x7f891c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891affd700 1 -- 172.21.15.92:0/784179303 >> v2:172.21.15.158:3300/0 conn(0x7f891c069750 msgr2=0x7f891c069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891affd700 1 --2- 172.21.15.92:0/784179303 >> v2:172.21.15.158:3300/0 conn(0x7f891c069750 0x7f891c069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891affd700 1 -- 172.21.15.92:0/784179303 >> v1:172.21.15.92:6789/0 conn(0x7f891c0c5f70 legacy=0x7f891c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891a7fc700 1 -- 172.21.15.92:0/784179303 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78612455 0 0) 0x7f891c06a920 con 0x7f891c0c5f70 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891affd700 1 -- 172.21.15.92:0/784179303 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0c63a0 con 0x7f891c0aab40 2023-05-16T07:14:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891affd700 1 --2- 172.21.15.92:0/784179303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0aab40 0x7f891c0c5a80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f891c0c74d0 tx=0x7f890c009060).ready entity=mon.0 client_cookie=9d9b90c7aa109f06 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891a7fc700 1 -- 172.21.15.92:0/784179303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00b580 con 0x7f891c0aab40 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891a7fc700 1 -- 172.21.15.92:0/784179303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c00b6e0 con 0x7f891c0aab40 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.894+0000 7f891a7fc700 1 -- 172.21.15.92:0/784179303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00ba60 con 0x7f891c0aab40 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 -- 172.21.15.92:0/784179303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0aab40 msgr2=0x7f891c0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 --2- 172.21.15.92:0/784179303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0aab40 0x7f891c0c5a80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f891c0c74d0 tx=0x7f890c009060).stop 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 -- 172.21.15.92:0/784179303 shutdown_connections 2023-05-16T07:14:59.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 --2- 172.21.15.92:0/784179303 >> v2:172.21.15.158:3300/0 conn(0x7f891c069750 0x7f891c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 --2- 172.21.15.92:0/784179303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0aab40 0x7f891c0c5a80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 -- 172.21.15.92:0/784179303 >> 172.21.15.92:0/784179303 conn(0x7f891c0cb550 msgr2=0x7f891c0c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:14:59.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 -- 172.21.15.92:0/784179303 shutdown_connections 2023-05-16T07:14:59.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.895+0000 7f892082f700 1 -- 172.21.15.92:0/784179303 wait complete. 2023-05-16T07:14:59.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.896+0000 7f892082f700 1 Processor -- start 2023-05-16T07:14:59.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.896+0000 7f892082f700 1 -- start start 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 0x7f891c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f891c0aab40 0x7f891c0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f891c0c5f70 0x7f891c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0c4a60 con 0x7f891c069750 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0aeac0 con 0x7f891c0aab40 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0ae500 con 0x7f891c0c5f70 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 0x7f891c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 0x7f891c156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44304/0 (socket says 172.21.15.92:44304) 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 -- 172.21.15.92:0/3928984356 learned_addr learned my addr 172.21.15.92:0/3928984356 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891affd700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f891c0aab40 0x7f891c0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 -- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f891c0c5f70 msgr2=0x7f891c0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f891c0c5f70 0x7f891c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 -- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f891c0aab40 msgr2=0x7f891c0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f891c0aab40 0x7f891c0afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:14:59.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 -- 172.21.15.92:0/3928984356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0092a0 con 0x7f891c069750 2023-05-16T07:14:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f891b7fe700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 0x7f891c156470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f891000d890 tx=0x7f8910008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100125f0 con 0x7f891c069750 2023-05-16T07:14:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8910012750 con 0x7f891c069750 2023-05-16T07:14:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0ae7e0 con 0x7f891c069750 2023-05-16T07:14:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910012ad0 con 0x7f891c069750 2023-05-16T07:14:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.897+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c0ade90 con 0x7f891c069750 2023-05-16T07:14:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.898+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c0ae200 con 0x7f891c069750 2023-05-16T07:14:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.899+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8910013480 con 0x7f891c069750 2023-05-16T07:14:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.899+0000 7f89037fe700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f890805ed90 0x7f8908061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:14:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.899+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f891007a070 con 0x7f891c069750 2023-05-16T07:14:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.899+0000 7f891affd700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f890805ed90 0x7f8908061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:14:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.900+0000 7f891affd700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f890805ed90 0x7f8908061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f891c0aaf90 tx=0x7f890c017000).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:14:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:14:59.903+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f891004b380 con 0x7f891c069750 2023-05-16T07:15:00.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.090+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f891c047150 con 0x7f890805ed90 2023-05-16T07:15:00.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.092+0000 7f89037fe700 1 -- 172.21.15.92:0/3928984356 <== mgr.5275 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f891c047150 con 0x7f890805ed90 2023-05-16T07:15:00.096 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:15:00.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:15:00.096 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:15:00.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:15:00.097 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:15:00.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f890805ed90 msgr2=0x7f8908061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:00.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f890805ed90 0x7f8908061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f891c0aaf90 tx=0x7f890c017000).stop 2023-05-16T07:15:00.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 msgr2=0x7f891c156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:00.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 0x7f891c156470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f891000d890 tx=0x7f8910008040).stop 2023-05-16T07:15:00.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 shutdown_connections 2023-05-16T07:15:00.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f890805ed90 0x7f8908061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f891c0c5f70 0x7f891c0b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:15:00.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c069750 0x7f891c156470 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 --2- 172.21.15.92:0/3928984356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f891c0aab40 0x7f891c0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 >> 172.21.15.92:0/3928984356 conn(0x7f891c0cb550 msgr2=0x7f891c151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:00.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 shutdown_connections 2023-05-16T07:15:00.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.094+0000 7f892082f700 1 -- 172.21.15.92:0/3928984356 wait complete. 2023-05-16T07:15:00.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:15:00.156 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:15:00.153+0000 7f100165e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:15:00.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.218+0000 7fbc33fff700 1 Processor -- start 2023-05-16T07:15:00.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.218+0000 7fbc33fff700 1 -- start start 2023-05-16T07:15:00.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c1160 0x7fbc340c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:00.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc33fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc340c61f0 0x7fbc34154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:00.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc33fff700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc340c6660 con 0x7fbc34153f90 2023-05-16T07:15:00.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc33fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc34069f60 con 0x7fbc340c61f0 2023-05-16T07:15:00.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc33fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3406a9e0 con 0x7fbc340c1160 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc340c61f0 0x7fbc34154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc340c61f0 0x7fbc34154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44316/0 (socket says 172.21.15.92:44316) 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 -- 172.21.15.92:0/1086408663 learned_addr learned my addr 172.21.15.92:0/1086408663 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc327fc700 1 --2- 172.21.15.92:0/1086408663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c1160 0x7fbc340c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 -- 172.21.15.92:0/1086408663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c1160 msgr2=0x7fbc340c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 --2- 172.21.15.92:0/1086408663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c1160 0x7fbc340c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 -- 172.21.15.92:0/1086408663 >> v1:172.21.15.158:6789/0 conn(0x7fbc34153f90 legacy=0x7fbc340a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 -- 172.21.15.92:0/1086408663 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3406a140 con 0x7fbc340c61f0 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc337fe700 1 --2- 172.21.15.92:0/1086408663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c61f0 0x7fbc34154370 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fbc2c009f20 tx=0x7fbc2c009840).ready entity=mon.0 client_cookie=bbedef64944ab028 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc31ffb700 1 -- 172.21.15.92:0/1086408663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00e070 con 0x7fbc340c61f0 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc31ffb700 1 -- 172.21.15.92:0/1086408663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c00ba80 con 0x7fbc340c61f0 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.219+0000 7fbc31ffb700 1 -- 172.21.15.92:0/1086408663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00be00 con 0x7fbc340c61f0 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 -- 172.21.15.92:0/1086408663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c61f0 msgr2=0x7fbc34154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:00.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 --2- 172.21.15.92:0/1086408663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c61f0 0x7fbc34154370 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fbc2c009f20 tx=0x7fbc2c009840).stop 2023-05-16T07:15:00.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 -- 172.21.15.92:0/1086408663 shutdown_connections 2023-05-16T07:15:00.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 --2- 172.21.15.92:0/1086408663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c61f0 0x7fbc34154370 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 --2- 172.21.15.92:0/1086408663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c1160 0x7fbc340c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 -- 172.21.15.92:0/1086408663 >> 172.21.15.92:0/1086408663 conn(0x7fbc340cb600 msgr2=0x7fbc340c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:00.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 -- 172.21.15.92:0/1086408663 shutdown_connections 2023-05-16T07:15:00.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.220+0000 7fbc33fff700 1 -- 172.21.15.92:0/1086408663 wait complete. 2023-05-16T07:15:00.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.221+0000 7fbc33fff700 1 Processor -- start 2023-05-16T07:15:00.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.221+0000 7fbc33fff700 1 -- start start 2023-05-16T07:15:00.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 0x7fbc341503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c61f0 0x7fbc340ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc33fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc34153f90 0x7fbc34163d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc33fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc34167190 con 0x7fbc340c1160 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc33fff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340afcf0 con 0x7fbc34153f90 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc33fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340af0e0 con 0x7fbc340c61f0 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 0x7fbc341503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:00.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 0x7fbc341503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44318/0 (socket says 172.21.15.92:44318) 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 -- 172.21.15.92:0/2283985575 learned_addr learned my addr 172.21.15.92:0/2283985575 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc327fc700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c61f0 0x7fbc340ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc337fe700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc34153f90 0x7fbc34163d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 -- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c61f0 msgr2=0x7fbc340ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c61f0 0x7fbc340ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 -- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc34153f90 msgr2=0x7fbc34163d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc34153f90 0x7fbc34163d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 -- 172.21.15.92:0/2283985575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c009ae0 con 0x7fbc340c1160 2023-05-16T07:15:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc327fc700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c61f0 0x7fbc340ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:15:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc337fe700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc34153f90 0x7fbc34163d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:15:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.222+0000 7fbc32ffd700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 0x7fbc341503d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fbc2400c4c0 tx=0x7fbc24007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.223+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24007020 con 0x7fbc340c1160 2023-05-16T07:15:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.223+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24001e90 con 0x7fbc340c1160 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.223+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc340af3c0 con 0x7fbc340c1160 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.223+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240073b0 con 0x7fbc340c1160 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.223+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc340af980 con 0x7fbc340c1160 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.223+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc34083120 con 0x7fbc340c1160 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.224+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbc24010070 con 0x7fbc340c1160 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.224+0000 7fbc1affd700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc2005ed90 0x7fbc20061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.224+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7fbc240795c0 con 0x7fbc340c1160 2023-05-16T07:15:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.224+0000 7fbc327fc700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc2005ed90 0x7fbc20061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.225+0000 7fbc327fc700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc2005ed90 0x7fbc20061250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbc340c5cd0 tx=0x7fbc28008040).ready entity=mgr.5275 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.228+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbc2404a930 con 0x7fbc340c1160 2023-05-16T07:15:00.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:00.435+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fbc34047150 con 0x7fbc340c1160 2023-05-16T07:15:01.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.250+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 15) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fbc24077290 con 0x7fbc340c1160 2023-05-16T07:15:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.251+0000 7fbc1affd700 1 -- 172.21.15.92:0/2283985575 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v15) v1 ==== 92+0+0 (secure 0 0 0) 0x7fbc24071c00 con 0x7fbc340c1160 2023-05-16T07:15:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc2005ed90 msgr2=0x7fbc20061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc2005ed90 0x7fbc20061250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbc340c5cd0 tx=0x7fbc28008040).stop 2023-05-16T07:15:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 msgr2=0x7fbc341503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 0x7fbc341503d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fbc2400c4c0 tx=0x7fbc24007e00).stop 2023-05-16T07:15:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 shutdown_connections 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc2005ed90 0x7fbc20061250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc34153f90 0x7fbc34163d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc340c1160 0x7fbc341503d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 --2- 172.21.15.92:0/2283985575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc340c61f0 0x7fbc340ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 >> 172.21.15.92:0/2283985575 conn(0x7fbc340cb600 msgr2=0x7fbc34068240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 shutdown_connections 2023-05-16T07:15:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.253+0000 7fbc33fff700 1 -- 172.21.15.92:0/2283985575 wait complete. 2023-05-16T07:15:01.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:15:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.385+0000 7f2943261700 1 Processor -- start 2023-05-16T07:15:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.385+0000 7f2943261700 1 -- start start 2023-05-16T07:15:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2943261700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f293c1384d0 0x7f293c0bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2943261700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f293c0629c0 0x7f293c0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2943261700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c0c6210 con 0x7f293c064010 2023-05-16T07:15:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2943261700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c0c53a0 con 0x7f293c1384d0 2023-05-16T07:15:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2943261700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c0c5cb0 con 0x7f293c0629c0 2023-05-16T07:15:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f293c1384d0 0x7f293c0bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f293c1384d0 0x7f293c0bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44352/0 (socket says 172.21.15.92:44352) 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 -- 172.21.15.92:0/2762287033 learned_addr learned my addr 172.21.15.92:0/2762287033 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f29417fe700 1 --2- 172.21.15.92:0/2762287033 >> v2:172.21.15.158:3300/0 conn(0x7f293c0629c0 0x7f293c0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 -- 172.21.15.92:0/2762287033 >> v2:172.21.15.158:3300/0 conn(0x7f293c0629c0 msgr2=0x7f293c0c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 --2- 172.21.15.92:0/2762287033 >> v2:172.21.15.158:3300/0 conn(0x7f293c0629c0 0x7f293c0c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 -- 172.21.15.92:0/2762287033 >> v1:172.21.15.158:6789/0 conn(0x7f293c064010 legacy=0x7f293c0622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 -- 172.21.15.92:0/2762287033 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0c4de0 con 0x7f293c1384d0 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f29417fe700 1 --2- 172.21.15.92:0/2762287033 >> v2:172.21.15.158:3300/0 conn(0x7f293c0629c0 0x7f293c0c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f2940ffd700 1 --2- 172.21.15.92:0/2762287033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0bf5f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f293c0660c0 tx=0x7f29300090f0).ready entity=mon.0 client_cookie=32f7a496f4cf903a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f293b7fe700 1 -- 172.21.15.92:0/2762287033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293000b600 con 0x7f293c1384d0 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f293b7fe700 1 -- 172.21.15.92:0/2762287033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293000b760 con 0x7f293c1384d0 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.386+0000 7f293b7fe700 1 -- 172.21.15.92:0/2762287033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293000bae0 con 0x7f293c1384d0 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 -- 172.21.15.92:0/2762287033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 msgr2=0x7f293c0bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 --2- 172.21.15.92:0/2762287033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0bf5f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f293c0660c0 tx=0x7f29300090f0).stop 2023-05-16T07:15:01.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 -- 172.21.15.92:0/2762287033 shutdown_connections 2023-05-16T07:15:01.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 --2- 172.21.15.92:0/2762287033 >> v2:172.21.15.158:3300/0 conn(0x7f293c0629c0 0x7f293c0c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 --2- 172.21.15.92:0/2762287033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0bf5f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 -- 172.21.15.92:0/2762287033 >> 172.21.15.92:0/2762287033 conn(0x7f293c05e7e0 msgr2=0x7f293c0c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:01.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 -- 172.21.15.92:0/2762287033 shutdown_connections 2023-05-16T07:15:01.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.387+0000 7f2943261700 1 -- 172.21.15.92:0/2762287033 wait complete. 2023-05-16T07:15:01.395 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:01.389+0000 7f616ad4c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:15:01.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.388+0000 7f2943261700 1 Processor -- start 2023-05-16T07:15:01.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.388+0000 7f2943261700 1 -- start start 2023-05-16T07:15:01.396 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f3abc02e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:15:01.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2943261700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f293c0629c0 0x7f293c153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2943261700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f293c064010 0x7f293c0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2943261700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2943261700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c153650 con 0x7f293c1384d0 2023-05-16T07:15:01.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2943261700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c152a40 con 0x7f293c0629c0 2023-05-16T07:15:01.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2943261700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c152d20 con 0x7f293c064010 2023-05-16T07:15:01.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f29417fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f29417fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44362/0 (socket says 172.21.15.92:44362) 2023-05-16T07:15:01.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f29417fe700 1 -- 172.21.15.92:0/4266173227 learned_addr learned my addr 172.21.15.92:0/4266173227 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:01.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f2940ffd700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f293c0629c0 0x7f293c153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.389+0000 7f293bfff700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f293c064010 0x7f293c0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f29417fe700 1 -- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f293c064010 msgr2=0x7f293c0ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f29417fe700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f293c064010 0x7f293c0ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f29417fe700 1 -- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f293c0629c0 msgr2=0x7f293c153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f29417fe700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f293c0629c0 0x7f293c153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f29417fe700 1 -- 172.21.15.92:0/4266173227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2930009370 con 0x7f293c1384d0 2023-05-16T07:15:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f29417fe700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0acbc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f293400d180 tx=0x7f2934009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293400dcc0 con 0x7f293c1384d0 2023-05-16T07:15:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293400f040 con 0x7f293c1384d0 2023-05-16T07:15:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c153000 con 0x7f293c1384d0 2023-05-16T07:15:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934013600 con 0x7f293c1384d0 2023-05-16T07:15:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.390+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c1524f0 con 0x7f293c1384d0 2023-05-16T07:15:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.391+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c1537c0 con 0x7f293c1384d0 2023-05-16T07:15:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.391+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f29340139f0 con 0x7f293c1384d0 2023-05-16T07:15:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.392+0000 7f2938ff9700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f292005ede0 0x7f29200612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.392+0000 7f2940ffd700 1 -- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f292005ede0 msgr2=0x7f29200612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:15:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.392+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f293407a610 con 0x7f293c1384d0 2023-05-16T07:15:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.392+0000 7f2940ffd700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f292005ede0 0x7f29200612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:15:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.396+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f293404b980 con 0x7f293c1384d0 2023-05-16T07:15:01.455 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:01.451+0000 7f616ad4c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:15:01.458 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:01.454+0000 7f3abc02e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:15:01.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.567+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f293c06c350 con 0x7f293c1384d0 2023-05-16T07:15:01.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.571+0000 7f2938ff9700 1 -- 172.21.15.92:0/4266173227 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v7)=0 v7) v1 ==== 138+0+0 (secure 0 0 0) 0x7f293404efa0 con 0x7f293c1384d0 2023-05-16T07:15:01.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f292005ede0 msgr2=0x7f29200612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:15:01.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f292005ede0 0x7f29200612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:15:01.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 msgr2=0x7f293c0acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0acbc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f293400d180 tx=0x7f2934009a50).stop 2023-05-16T07:15:01.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 shutdown_connections 2023-05-16T07:15:01.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f292005ede0 0x7f29200612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:15:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f293c1384d0 0x7f293c0acbc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f293c0629c0 0x7f293c153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 --2- 172.21.15.92:0/4266173227 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f293c064010 0x7f293c0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 >> 172.21.15.92:0/4266173227 conn(0x7f293c05e7e0 msgr2=0x7f293c1376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:01.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 shutdown_connections 2023-05-16T07:15:01.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.573+0000 7f2943261700 1 -- 172.21.15.92:0/4266173227 wait complete. 2023-05-16T07:15:01.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:01.589 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:15:01.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-05-16T07:15:01.723 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:01.719+0000 7f616ad4c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:15:01.739 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:01.736+0000 7f3abc02e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:15:01.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.763+0000 7f9dc986d700 1 Processor -- start 2023-05-16T07:15:01.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.763+0000 7f9dc986d700 1 -- start start 2023-05-16T07:15:01.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc986d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9dc4153d00 0x7f9dc4063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc986d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9dc40622b0 0x7f9dc4062720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc986d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40bf950 con 0x7f9dc41540e0 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc986d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc4067f60 con 0x7f9dc4153d00 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc986d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc40a7c90 con 0x7f9dc40622b0 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9dc4153d00 0x7f9dc4063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9dc4153d00 0x7f9dc4063fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44410/0 (socket says 172.21.15.92:44410) 2023-05-16T07:15:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/2020762366 learned_addr learned my addr 172.21.15.92:0/2020762366 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc37fe700 1 --2- 172.21.15.92:0/2020762366 >> v2:172.21.15.158:3300/0 conn(0x7f9dc40622b0 0x7f9dc4062720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/2020762366 >> v2:172.21.15.158:3300/0 conn(0x7f9dc40622b0 msgr2=0x7f9dc4062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/2020762366 >> v2:172.21.15.158:3300/0 conn(0x7f9dc40622b0 0x7f9dc4062720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/2020762366 >> v1:172.21.15.158:6789/0 conn(0x7f9dc41540e0 legacy=0x7f9dc40bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/2020762366 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4062c10 con 0x7f9dc4153d00 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc37fe700 1 --2- 172.21.15.92:0/2020762366 >> v2:172.21.15.158:3300/0 conn(0x7f9dc40622b0 0x7f9dc4062720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:15:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.764+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/2020762366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc4063fd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f9dc405f9e0 tx=0x7f9db80090f0).ready entity=mon.0 client_cookie=5fbec76b4d331062 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:01.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc1ffb700 1 -- 172.21.15.92:0/2020762366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800b600 con 0x7f9dc4153d00 2023-05-16T07:15:01.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc1ffb700 1 -- 172.21.15.92:0/2020762366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db800b760 con 0x7f9dc4153d00 2023-05-16T07:15:01.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc1ffb700 1 -- 172.21.15.92:0/2020762366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800bae0 con 0x7f9dc4153d00 2023-05-16T07:15:01.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 -- 172.21.15.92:0/2020762366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 msgr2=0x7f9dc4063fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 --2- 172.21.15.92:0/2020762366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc4063fd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f9dc405f9e0 tx=0x7f9db80090f0).stop 2023-05-16T07:15:01.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 -- 172.21.15.92:0/2020762366 shutdown_connections 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 --2- 172.21.15.92:0/2020762366 >> v2:172.21.15.158:3300/0 conn(0x7f9dc40622b0 0x7f9dc4062720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 --2- 172.21.15.92:0/2020762366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc4063fd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 -- 172.21.15.92:0/2020762366 >> 172.21.15.92:0/2020762366 conn(0x7f9dc405e820 msgr2=0x7f9dc40c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 -- 172.21.15.92:0/2020762366 shutdown_connections 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.765+0000 7f9dc986d700 1 -- 172.21.15.92:0/2020762366 wait complete. 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 Processor -- start 2023-05-16T07:15:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 -- start start 2023-05-16T07:15:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dc40622b0 0x7f9dc4168750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc41640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9dc41540e0 0x7f9dc40c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41490b0 con 0x7f9dc4153d00 2023-05-16T07:15:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.767+0000 7f9dc986d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc414ca70 con 0x7f9dc40622b0 2023-05-16T07:15:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc986d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4151800 con 0x7f9dc41540e0 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc41640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc41640a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44416/0 (socket says 172.21.15.92:44416) 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 -- 172.21.15.92:0/278779923 learned_addr learned my addr 172.21.15.92:0/278779923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc37fe700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9dc41540e0 0x7f9dc40c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dc40622b0 0x7f9dc4168750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9dc41540e0 msgr2=0x7f9dc40c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9dc41540e0 0x7f9dc40c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dc40622b0 msgr2=0x7f9dc4168750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dc40622b0 0x7f9dc4168750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 -- 172.21.15.92:0/278779923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db8009370 con 0x7f9dc4153d00 2023-05-16T07:15:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc27fc700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc41640a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9db400c440 tx=0x7f9db4007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40123f0 con 0x7f9dc4153d00 2023-05-16T07:15:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db4001e90 con 0x7f9dc4153d00 2023-05-16T07:15:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4012780 con 0x7f9dc4153d00 2023-05-16T07:15:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc4168c40 con 0x7f9dc4153d00 2023-05-16T07:15:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.768+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc4150ed0 con 0x7f9dc4153d00 2023-05-16T07:15:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.769+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc406c350 con 0x7f9dc4153d00 2023-05-16T07:15:01.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.770+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f9db4010070 con 0x7f9dc4153d00 2023-05-16T07:15:01.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.770+0000 7f9daaffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:01.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.770+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:15:01.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.770+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6677+0+0 (secure 0 0 0) 0x7f9db4078eb0 con 0x7f9dc4153d00 2023-05-16T07:15:01.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.770+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:15:01.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.774+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9db40478f0 con 0x7f9dc4153d00 2023-05-16T07:15:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.971+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:15:01.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.971+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:15:01.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:01.975+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dc4064810 con 0x7f9db005ede0 2023-05-16T07:15:02.331 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:02.327+0000 7f616ad4c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:15:02.355 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:02.351+0000 7f3abc02e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:15:02.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:02.372+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:15:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:02.372+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:15:02.402 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:02.397+0000 7f616ad4c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:15:02.426 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:02.422+0000 7f3abc02e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:15:02.583 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:02.579+0000 7f616ad4c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:15:02.607 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:02.603+0000 7f3abc02e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:15:02.992 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:02.988+0000 7f616ad4c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:15:03.016 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.012+0000 7f3abc02e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:15:03.093 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:03.088+0000 7f616ad4c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:15:03.117 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.114+0000 7f3abc02e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:15:03.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:03.173+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:15:03.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:03.173+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:15:03.245 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:03.240+0000 7f616ad4c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:15:03.268 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.263+0000 7f3abc02e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:15:03.317 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:03.313+0000 7f616ad4c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:15:03.340 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.336+0000 7f3abc02e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:15:03.382 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:03.378+0000 7f616ad4c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:15:03.404 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.401+0000 7f3abc02e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:15:03.634 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:03.629+0000 7f616ad4c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:15:03.655 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.652+0000 7f3abc02e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:15:03.813 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:03.808+0000 7f616ad4c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:15:03.840 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:03.836+0000 7f3abc02e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:15:04.449 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:04.444+0000 7f616ad4c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:15:04.471 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:04.468+0000 7f3abc02e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:15:04.515 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:04.510+0000 7f616ad4c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:15:04.538 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:04.534+0000 7f3abc02e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:15:04.586 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:04.582+0000 7f616ad4c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:15:04.610 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:04.606+0000 7f3abc02e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:15:04.728 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:04.724+0000 7f616ad4c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:15:04.752 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:04.749+0000 7f3abc02e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:15:04.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:04.774+0000 7f9dc2ffd700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:15:04.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:04.774+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:15:04.796 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:04.792+0000 7f616ad4c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:15:04.822 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:04.818+0000 7f3abc02e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:15:04.909 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:04.904+0000 7f616ad4c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:15:04.933 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:04.930+0000 7f3abc02e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:15:05.013 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.008+0000 7f616ad4c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:15:05.039 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:05.035+0000 7f3abc02e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:15:05.298 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.294+0000 7f616ad4c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:15:05.345 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:05.341+0000 7f3abc02e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:15:05.367 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.362+0000 7f616ad4c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:15:05.414 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:15:05.411+0000 7f3abc02e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:15:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:05.432+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 16) v1 ==== 75109+0+0 (secure 0 0 0) 0x7f9db4076b80 con 0x7f9dc4153d00 2023-05-16T07:15:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:05.432+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:15:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:05.432+0000 7f9daaffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:15:05.492 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e6528700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e6528700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e6528700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e6528700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e6528700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e952e700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e952e700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.493 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e952e700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.494 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e952e700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:05.494 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:15:05.488+0000 7f60e952e700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:15:06.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.445+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 17) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f9db4076f00 con 0x7f9dc4153d00 2023-05-16T07:15:06.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.445+0000 7f9daaffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db002fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.445+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dc4064810 con 0x7f9db005ede0 2023-05-16T07:15:06.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.447+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db002fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.448+0000 7f9dc2ffd700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db002fd40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9db8009fd0 tx=0x7f9db8017000).ready entity=mgr.5593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.461+0000 7f9daaffd700 1 -- 172.21.15.92:0/278779923 <== mgr.5593 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9dc4064810 con 0x7f9db005ede0 2023-05-16T07:15:06.465 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path snap_test_dir1 2023-05-16T07:15:06.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 msgr2=0x7f9db002fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db002fd40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9db8009fd0 tx=0x7f9db8017000).stop 2023-05-16T07:15:06.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 msgr2=0x7f9dc41640a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc41640a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9db400c440 tx=0x7f9db4007db0).stop 2023-05-16T07:15:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 shutdown_connections 2023-05-16T07:15:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9db005ede0 0x7f9db002fd40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9dc41540e0 0x7f9dc40c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9dc40622b0 0x7f9dc4168750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 --2- 172.21.15.92:0/278779923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc4153d00 0x7f9dc41640a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 >> 172.21.15.92:0/278779923 conn(0x7f9dc405e820 msgr2=0x7f9dc414fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:06.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 shutdown_connections 2023-05-16T07:15:06.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.463+0000 7f9dc986d700 1 -- 172.21.15.92:0/278779923 wait complete. 2023-05-16T07:15:06.482 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-05-16T07:15:06.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-05-16T07:15:06.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.599+0000 7fad6b060700 1 Processor -- start 2023-05-16T07:15:06.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.599+0000 7fad6b060700 1 -- start start 2023-05-16T07:15:06.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad6b060700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fad64154090 0x7fad640aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad6b060700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad640c94d0 con 0x7fad64068160 2023-05-16T07:15:06.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad6b060700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad640cb7e0 con 0x7fad64154470 2023-05-16T07:15:06.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad6b060700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad640c88c0 con 0x7fad64154090 2023-05-16T07:15:06.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad695fd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad64068160 0x7fad640c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55276/0 (socket says 172.21.15.92:55276) 2023-05-16T07:15:06.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad695fd700 1 -- 172.21.15.92:0/727077109 learned_addr learned my addr 172.21.15.92:0/727077109 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad68dfc700 1 --2- 172.21.15.92:0/727077109 >> v2:172.21.15.158:3300/0 conn(0x7fad64154090 0x7fad640aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446051771 0 0) 0x7fad640c94d0 con 0x7fad64068160 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.600+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad500033f0 con 0x7fad64068160 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1386526540 0 0) 0x7fad500033f0 con 0x7fad64068160 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 >> v2:172.21.15.158:3300/0 conn(0x7fad64154090 msgr2=0x7fad640aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 --2- 172.21.15.92:0/727077109 >> v2:172.21.15.158:3300/0 conn(0x7fad64154090 0x7fad640aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 >> v1:172.21.15.158:6789/0 conn(0x7fad64154470 legacy=0x7fad640a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640c8ba0 con 0x7fad64068160 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fad5c001470 con 0x7fad64068160 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 >> v1:172.21.15.92:6789/0 conn(0x7fad64068160 legacy=0x7fad640c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad500038e0 0x7fad50003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50004690 0x7fad50004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50005000 0x7fad5000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad640c88c0 con 0x7fad50004690 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5000bf60 con 0x7fad50005000 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5000c0f0 con 0x7fad500038e0 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50004690 0x7fad50004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad68dfc700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad500038e0 0x7fad50003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad695fd700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50005000 0x7fad5000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 -- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad500038e0 msgr2=0x7fad50003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:06.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad500038e0 0x7fad50003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 -- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50005000 msgr2=0x7fad5000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50005000 0x7fad5000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 -- 172.21.15.92:0/727077109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500033f0 con 0x7fad50004690 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad695fd700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50005000 0x7fad5000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad695fd700 1 -- 172.21.15.92:0/727077109 reap_dead start 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad63fff700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50004690 0x7fad50004b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fad4c001740 tx=0x7fad4c007b90).ready entity=mon.0 client_cookie=f5968efb08b3939a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c007ea0 con 0x7fad50004690 2023-05-16T07:15:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c012400 con 0x7fad50004690 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.601+0000 7fad637fe700 1 -- 172.21.15.92:0/727077109 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0127a0 con 0x7fad50004690 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 -- 172.21.15.92:0/727077109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50004690 msgr2=0x7fad50004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50004690 0x7fad50004b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fad4c001740 tx=0x7fad4c007b90).stop 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 -- 172.21.15.92:0/727077109 shutdown_connections 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 --2- 172.21.15.92:0/727077109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50004690 0x7fad50004b10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 -- 172.21.15.92:0/727077109 >> 172.21.15.92:0/727077109 conn(0x7fad64067920 msgr2=0x7fad640cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 -- 172.21.15.92:0/727077109 shutdown_connections 2023-05-16T07:15:06.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.602+0000 7fad6b060700 1 -- 172.21.15.92:0/727077109 wait complete. 2023-05-16T07:15:06.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.603+0000 7fad6b060700 1 Processor -- start 2023-05-16T07:15:06.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.603+0000 7fad6b060700 1 -- start start 2023-05-16T07:15:06.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad6b060700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50004690 0x7fad64163710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad6b060700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 0x7fad641514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad6b060700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad64151990 0x7fad64154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad6b060700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad64152a20 con 0x7fad64152eb0 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad6b060700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad64151e10 con 0x7fad50004690 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad6b060700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad641520f0 con 0x7fad64151990 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 0x7fad641514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 0x7fad641514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44538/0 (socket says 172.21.15.92:44538) 2023-05-16T07:15:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 -- 172.21.15.92:0/3496326263 learned_addr learned my addr 172.21.15.92:0/3496326263 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:06.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad68dfc700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50004690 0x7fad64163710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad695fd700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad64151990 0x7fad64154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 -- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad64151990 msgr2=0x7fad64154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad64151990 0x7fad64154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 -- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50004690 msgr2=0x7fad64163710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50004690 0x7fad64163710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 -- 172.21.15.92:0/3496326263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad641523d0 con 0x7fad64152eb0 2023-05-16T07:15:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad68dfc700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50004690 0x7fad64163710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:15:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad63fff700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 0x7fad641514a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fad4c001d30 tx=0x7fad4c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.604+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c007ea0 con 0x7fad64152eb0 2023-05-16T07:15:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.605+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c0068b0 con 0x7fad64152eb0 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.605+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad641526b0 con 0x7fad64152eb0 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.605+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c006c70 con 0x7fad64152eb0 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.605+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad64157cc0 con 0x7fad64152eb0 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.605+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad64066730 con 0x7fad64152eb0 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.606+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fad4c010070 con 0x7fad64152eb0 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.606+0000 7fad60ff9700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fad4405ed40 0x7fad44061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.606+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6849+0+0 (secure 0 0 0) 0x7fad4c07abf0 con 0x7fad64152eb0 2023-05-16T07:15:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.606+0000 7fad68dfc700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fad4405ed40 0x7fad44061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.607+0000 7fad68dfc700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fad4405ed40 0x7fad44061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fad540094d0 tx=0x7fad5400b040).ready entity=mgr.5593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.610+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fad4c04bf50 con 0x7fad64152eb0 2023-05-16T07:15:06.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:15:06.702 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:15:06.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.801+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7fad64163dc0 con 0x7fad4405ed40 2023-05-16T07:15:06.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.806+0000 7fad60ff9700 1 -- 172.21.15.92:0/3496326263 <== mgr.5593 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fad64163dc0 con 0x7fad4405ed40 2023-05-16T07:15:06.811 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path snap_test_dir1 2023-05-16T07:15:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fad4405ed40 msgr2=0x7fad44061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fad4405ed40 0x7fad44061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fad540094d0 tx=0x7fad5400b040).stop 2023-05-16T07:15:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 msgr2=0x7fad641514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 0x7fad641514a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fad4c001d30 tx=0x7fad4c017000).stop 2023-05-16T07:15:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 shutdown_connections 2023-05-16T07:15:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fad4405ed40 0x7fad44061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fad64151990 0x7fad64154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fad50004690 0x7fad64163710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 --2- 172.21.15.92:0/3496326263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad64152eb0 0x7fad641514a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 >> 172.21.15.92:0/3496326263 conn(0x7fad64067920 msgr2=0x7fad640ab9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 shutdown_connections 2023-05-16T07:15:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.808+0000 7fad6b060700 1 -- 172.21.15.92:0/3496326263 wait complete. 2023-05-16T07:15:06.823 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-05-16T07:15:06.824 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-05-16-07_16_00 in ~54s... 2023-05-16T07:15:06.824 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-05-16-07_17_00 in ~114s... 2023-05-16T07:15:06.824 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-05-16T07:15:06.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-05-16T07:15:06.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.934+0000 7f056f2d5700 1 Processor -- start 2023-05-16T07:15:06.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.934+0000 7f056f2d5700 1 -- start start 2023-05-16T07:15:06.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056f2d5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 0x7f0568155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056f2d5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056806a140 con 0x7f05680c1160 2023-05-16T07:15:06.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056f2d5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056806a420 con 0x7f0568153f90 2023-05-16T07:15:06.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056f2d5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0568069f60 con 0x7f05680c6410 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056dad2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f05680c1160 0x7f05680c5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55296/0 (socket says 172.21.15.92:55296) 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056dad2700 1 -- 172.21.15.92:0/157895084 learned_addr learned my addr 172.21.15.92:0/157895084 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056ead4700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 0x7f0568155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.935+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503650277 0 0) 0x7f056806a140 con 0x7f05680c1160 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05500033f0 con 0x7f05680c1160 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1164629950 0 0) 0x7f05500033f0 con 0x7f05680c1160 2023-05-16T07:15:06.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 msgr2=0x7f0568155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 0x7f0568155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 >> v1:172.21.15.158:6789/0 conn(0x7f0568153f90 legacy=0x7f05680a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056806a700 con 0x7f05680c1160 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f055c001470 con 0x7f05680c1160 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 >> v1:172.21.15.92:6789/0 conn(0x7f05680c1160 legacy=0x7f05680c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 0x7f0550003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0550003e40 0x7f0550004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0550005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0568069f60 con 0x7f0550004b80 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055000b990 con 0x7f0550003e40 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055000bb50 con 0x7f05680c6410 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0550005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056e2d3700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 0x7f0550003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056dad2700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0550003e40 0x7f0550004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 -- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 msgr2=0x7f0550003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c6410 0x7f0550003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 -- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0550003e40 msgr2=0x7f0550004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0550003e40 0x7f0550004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 -- 172.21.15.92:0/157895084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05500033f0 con 0x7f0550004b80 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056dad2700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0550003e40 0x7f0550004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056ead4700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0550005000 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f05680c6810 tx=0x7f0560009060).ready entity=mon.0 client_cookie=1972e824a8026574 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560009e90 con 0x7f0550004b80 2023-05-16T07:15:06.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560009430 con 0x7f0550004b80 2023-05-16T07:15:06.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.936+0000 7f056d2d1700 1 -- 172.21.15.92:0/157895084 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000e850 con 0x7f0550004b80 2023-05-16T07:15:06.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056f2d5700 1 -- 172.21.15.92:0/157895084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 msgr2=0x7f0550005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056f2d5700 1 --2- 172.21.15.92:0/157895084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0550005000 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f05680c6810 tx=0x7f0560009060).stop 2023-05-16T07:15:06.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056ead4700 1 -- 172.21.15.92:0/157895084 reap_dead start 2023-05-16T07:15:06.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056f2d5700 1 -- 172.21.15.92:0/157895084 shutdown_connections 2023-05-16T07:15:06.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056f2d5700 1 -- 172.21.15.92:0/157895084 >> 172.21.15.92:0/157895084 conn(0x7f05680cb600 msgr2=0x7f05680c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:06.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056f2d5700 1 -- 172.21.15.92:0/157895084 shutdown_connections 2023-05-16T07:15:06.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.937+0000 7f056f2d5700 1 -- 172.21.15.92:0/157895084 wait complete. 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.938+0000 7f056f2d5700 1 Processor -- start 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.938+0000 7f056f2d5700 1 -- start start 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056f2d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0568163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056f2d5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c3570 0x7f05680c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056f2d5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0568151e50 0x7f05681522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056f2d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680adae0 con 0x7f0550004b80 2023-05-16T07:15:06.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056f2d5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681527c0 con 0x7f0568151e50 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056f2d5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0568164210 con 0x7f05680c3570 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0568163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0568163d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44558/0 (socket says 172.21.15.92:44558) 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 -- 172.21.15.92:0/660611484 learned_addr learned my addr 172.21.15.92:0/660611484 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056dad2700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c3570 0x7f05680c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056ead4700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0568151e50 0x7f05681522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 -- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c3570 msgr2=0x7f05680c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c3570 0x7f05680c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 -- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0568151e50 msgr2=0x7f05681522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0568151e50 0x7f05681522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 -- 172.21.15.92:0/660611484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680c4000 con 0x7f0550004b80 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056dad2700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c3570 0x7f05680c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.939+0000 7f056e2d3700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0568163d20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f055800e940 tx=0x7f0558007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.940+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055800ee80 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.940+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055800c7e0 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.940+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05680c42e0 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.940+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055800cb80 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.940+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05680c48a0 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.941+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0568083120 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.941+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f055801d460 con 0x7f0550004b80 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.941+0000 7f05667fc700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f055405ed10 0x7f05540611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:15:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.941+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f055807a130 con 0x7f0550004b80 2023-05-16T07:15:06.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.941+0000 7f056dad2700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f055405ed10 0x7f05540611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:15:06.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.942+0000 7f056dad2700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f055405ed10 0x7f05540611d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f055c0015c0 tx=0x7f055c009040).ready entity=mgr.5593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:15:06.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:06.945+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f055804b430 con 0x7f0550004b80 2023-05-16T07:15:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.131+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05680adc20 con 0x7f055405ed10 2023-05-16T07:15:07.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.135+0000 7f05667fc700 1 -- 172.21.15.92:0/660611484 <== mgr.5593 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f055c005870 con 0x7f055405ed10 2023-05-16T07:15:07.138 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:15:07.138 INFO:teuthology.orchestra.run.smithi092.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-05-16T07:15:07.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f055405ed10 msgr2=0x7f05540611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:07.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f055405ed10 0x7f05540611d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f055c0015c0 tx=0x7f055c009040).stop 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 msgr2=0x7f0568163d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0568163d20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f055800e940 tx=0x7f0558007f30).stop 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 shutdown_connections 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f055405ed10 0x7f05540611d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0568151e50 0x7f05681522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0550004b80 0x7f0568163d20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 --2- 172.21.15.92:0/660611484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f05680c3570 0x7f05680c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:15:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 >> 172.21.15.92:0/660611484 conn(0x7f05680cb600 msgr2=0x7f05680acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:15:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 shutdown_connections 2023-05-16T07:15:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:15:07.136+0000 7f056f2d5700 1 -- 172.21.15.92:0/660611484 wait complete. 2023-05-16T07:15:07.152 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-05-16T07:15:07.152 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-05-16T07:15:07.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:07.153 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:08.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:08.170 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:09.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:09.190 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:10.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:10.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:11.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:11.229 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:12.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:12.249 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:13.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:13.269 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:14.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:14.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:15.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:15.308 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:16.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:16.328 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:17.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:17.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:18.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:18.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:19.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:19.388 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:20.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:20.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:21.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:21.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:22.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:22.447 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:23.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:23.467 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:24.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:24.488 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:25.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:25.508 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:26.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:26.528 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:27.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:27.547 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:28.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:28.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:29.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:29.586 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:30.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:30.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:31.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:31.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:32.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:32.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:33.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:33.672 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:34.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:34.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:35.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:35.712 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:36.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:36.732 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:36.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:15:36.793 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:15:37.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:37.752 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:38.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:38.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:39.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:39.790 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:40.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:40.810 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:41.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:41.829 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:42.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:42.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:43.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:43.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:44.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:44.888 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:45.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:45.908 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:46.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:46.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:47.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:47.948 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:48.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:48.967 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:49.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:49.987 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:51.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:51.007 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:52.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:52.027 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:53.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:53.047 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:54.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:54.067 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:55.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:55.086 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:56.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:56.106 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:57.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:57.126 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:58.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:58.146 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:15:59.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:15:59.165 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:00.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:00.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:16:00.810+0000 7f60d8d0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:16:00.810+0000 7f60d8d0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:16:00.810+0000 7f60d8d0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:16:00.810+0000 7f60d8d0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:16:00.810+0000 7f60d8d0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:16:01.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:01.206 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:01.224 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:01.225 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-05-16-07_16_00_UTC'] 2023-05-16T07:16:01.225 DEBUG:tasks.cephfs.test_snap_schedules:exec=1684221306.8239524, now = 1684221361.225644, timo = 54 2023-05-16T07:16:02.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:02.226 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:02.244 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:03.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:03.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:03.265 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:04.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:04.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:04.285 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:05.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:05.286 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:05.304 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:06.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:06.306 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:06.325 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:06.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:16:06.862 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:16:07.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:07.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:07.344 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:08.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:08.346 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:08.364 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:09.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:09.366 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:09.384 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:10.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:10.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:10.404 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:11.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:11.406 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:11.424 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:12.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:12.426 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:12.446 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:13.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:13.447 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:13.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:14.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:14.468 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:14.486 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:15.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:15.488 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:15.507 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:16.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:16.509 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:16.527 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:17.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:17.529 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:17.546 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:18.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:18.550 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:18.569 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:19.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:19.570 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:19.588 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:20.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:20.590 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:20.609 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:21.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:21.610 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:21.629 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:22.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:22.630 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:22.649 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:23.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:23.650 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:23.669 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:24.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:24.670 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:24.688 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:25.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:25.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:25.710 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:26.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:26.712 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:26.731 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:27.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:27.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:28.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:28.753 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:28.771 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:29.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:29.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:29.792 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:30.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:30.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:30.813 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:31.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:31.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:32.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:32.836 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:32.854 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:33.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:33.856 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:33.877 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:34.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:34.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:34.897 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:35.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:35.899 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:35.918 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:36.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:36.920 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:36.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:16:36.924 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:16:36.938 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:37.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:37.940 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:37.959 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:38.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:38.961 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:38.981 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:39.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:39.983 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:41.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:41.004 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:41.023 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:42.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:42.024 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:42.043 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:43.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:43.045 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:43.063 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:44.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:44.065 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:44.083 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:45.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:45.085 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:45.103 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:46.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:46.106 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:46.123 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:47.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:47.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:47.143 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:48.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:48.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:48.163 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:49.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:49.166 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:49.183 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:50.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:50.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:50.203 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:51.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:51.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:51.224 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:52.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:52.226 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:52.244 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:53.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:53.245 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:53.264 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:54.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:54.265 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:54.284 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:55.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:55.286 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:56.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:56.306 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:56.324 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:57.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:57.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:57.346 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:58.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:58.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:58.366 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:16:59.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:16:59.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:17:00.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:00.389 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:17:01.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:01.409 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:17:01.426 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:17:01.426 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_17_00_UTC 2023-05-16T07:17:01.427 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-05-16-07_17_00_UTC'] 2023-05-16T07:17:01.428 DEBUG:tasks.cephfs.test_snap_schedules:exec=1684221306.8239524, now = 1684221421.4281607, timo = 114 2023-05-16T07:17:02.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-05-16T07:17:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.558+0000 7f3d19615700 1 Processor -- start 2023-05-16T07:17:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.558+0000 7f3d19615700 1 -- start start 2023-05-16T07:17:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.559+0000 7f3d19615700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3d14065680 0x7f3d140bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.559+0000 7f3d19615700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d14150890 con 0x7f3d14064b90 2023-05-16T07:17:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.559+0000 7f3d19615700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d140bcf30 con 0x7f3d1406a410 2023-05-16T07:17:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.559+0000 7f3d19615700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d14069930 con 0x7f3d14065680 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.559+0000 7f3d12ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3d14064b90 0x7f3d14064f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55314/0 (socket says 172.21.15.92:55314) 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.559+0000 7f3d12ffd700 1 -- 172.21.15.92:0/748364928 learned_addr learned my addr 172.21.15.92:0/748364928 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d137fe700 1 --2- 172.21.15.92:0/748364928 >> v2:172.21.15.158:3300/0 conn(0x7f3d14065680 0x7f3d140bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230731967 0 0) 0x7f3d14150890 con 0x7f3d14064b90 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cfc0033f0 con 0x7f3d14064b90 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1644397840 0 0) 0x7f3cfc0033f0 con 0x7f3d14064b90 2023-05-16T07:17:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 >> v2:172.21.15.158:3300/0 conn(0x7f3d14065680 msgr2=0x7f3d140bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 --2- 172.21.15.92:0/748364928 >> v2:172.21.15.158:3300/0 conn(0x7f3d14065680 0x7f3d140bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 >> v1:172.21.15.158:6789/0 conn(0x7f3d1406a410 legacy=0x7f3d140c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14069c10 con 0x7f3d14064b90 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f3d040017f0 con 0x7f3d14064b90 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 >> v1:172.21.15.92:6789/0 conn(0x7f3d14064b90 legacy=0x7f3d14064f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3cfc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cfc004690 0x7f3cfc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3cfc005000 0x7f3cfc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14069930 con 0x7f3cfc0038e0 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc00bf60 con 0x7f3cfc004690 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc00c0f0 con 0x7f3cfc005000 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3cfc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d127fc700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cfc004690 0x7f3cfc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d137fe700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3cfc005000 0x7f3cfc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d12ffd700 1 -- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3cfc005000 msgr2=0x7f3cfc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3cfc005000 0x7f3cfc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d12ffd700 1 -- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cfc004690 msgr2=0x7f3cfc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cfc004690 0x7f3cfc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d12ffd700 1 -- 172.21.15.92:0/748364928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0033f0 con 0x7f3cfc0038e0 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.560+0000 7f3d137fe700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3cfc005000 0x7f3cfc00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d137fe700 1 -- 172.21.15.92:0/748364928 reap_dead start 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3cfc003d00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f3d0400c480 tx=0x7f3d0400c790).ready entity=mon.0 client_cookie=8ec8c6a783b380ec server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04002030 con 0x7f3cfc0038e0 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0400ce20 con 0x7f3cfc0038e0 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d11ffb700 1 -- 172.21.15.92:0/748364928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04005d30 con 0x7f3cfc0038e0 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 -- 172.21.15.92:0/748364928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 msgr2=0x7f3cfc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3cfc003d00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f3d0400c480 tx=0x7f3d0400c790).stop 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 -- 172.21.15.92:0/748364928 shutdown_connections 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 --2- 172.21.15.92:0/748364928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3cfc003d00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 -- 172.21.15.92:0/748364928 >> 172.21.15.92:0/748364928 conn(0x7f3d140c4450 msgr2=0x7f3d14066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 -- 172.21.15.92:0/748364928 shutdown_connections 2023-05-16T07:17:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.561+0000 7f3d19615700 1 -- 172.21.15.92:0/748364928 wait complete. 2023-05-16T07:17:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.562+0000 7f3d19615700 1 Processor -- start 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 -- start start 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3d141431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d141651e0 0x7f3d141395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d14141d20 0x7f3d141421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14151c00 con 0x7f3cfc0038e0 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14142690 con 0x7f3d14141d20 2023-05-16T07:17:02.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d19615700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14150ff0 con 0x7f3d141651e0 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d12ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3d141431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d12ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3d141431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44580/0 (socket says 172.21.15.92:44580) 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d12ffd700 1 -- 172.21.15.92:0/2879223328 learned_addr learned my addr 172.21.15.92:0/2879223328 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d127fc700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d141651e0 0x7f3d141395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.563+0000 7f3d137fe700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d14141d20 0x7f3d141421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d12ffd700 1 -- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d141651e0 msgr2=0x7f3d141395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d141651e0 0x7f3d141395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d12ffd700 1 -- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d14141d20 msgr2=0x7f3d141421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d14141d20 0x7f3d141421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d12ffd700 1 -- 172.21.15.92:0/2879223328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d141512d0 con 0x7f3cfc0038e0 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d12ffd700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3d141431d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f3d0400cae0 tx=0x7f3d04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d040074f0 con 0x7f3cfc0038e0 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d04007650 con 0x7f3cfc0038e0 2023-05-16T07:17:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d141515b0 con 0x7f3cfc0038e0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04007a10 con 0x7f3cfc0038e0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.564+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14165660 con 0x7f3cfc0038e0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.565+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d14151ee0 con 0x7f3cfc0038e0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.565+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f3d04010070 con 0x7f3cfc0038e0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.566+0000 7f3cfaffd700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d0005ee70 0x7f3d00061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.566+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6965+0+0 (secure 0 0 0) 0x7f3d0407b040 con 0x7f3cfc0038e0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.566+0000 7f3d127fc700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d0005ee70 0x7f3d00061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.566+0000 7f3d127fc700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d0005ee70 0x7f3d00061330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3d141447d0 tx=0x7f3d08007c00).ready entity=mgr.5593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:02.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.570+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3d0404c270 con 0x7f3cfc0038e0 2023-05-16T07:17:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.757+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d14151a50 con 0x7f3d0005ee70 2023-05-16T07:17:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.762+0000 7f3cfaffd700 1 -- 172.21.15.92:0/2879223328 <== mgr.5593 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3d14151a50 con 0x7f3d0005ee70 2023-05-16T07:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path snap_test_dir1 2023-05-16T07:17:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d0005ee70 msgr2=0x7f3d00061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d0005ee70 0x7f3d00061330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3d141447d0 tx=0x7f3d08007c00).stop 2023-05-16T07:17:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 msgr2=0x7f3d141431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3d141431d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f3d0400cae0 tx=0x7f3d04015000).stop 2023-05-16T07:17:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 shutdown_connections 2023-05-16T07:17:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d0005ee70 0x7f3d00061330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d14141d20 0x7f3d141421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cfc0038e0 0x7f3d141431d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 --2- 172.21.15.92:0/2879223328 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d141651e0 0x7f3d141395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 >> 172.21.15.92:0/2879223328 conn(0x7f3d140c4450 msgr2=0x7f3d140c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 shutdown_connections 2023-05-16T07:17:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:02.763+0000 7f3d19615700 1 -- 172.21.15.92:0/2879223328 wait complete. 2023-05-16T07:17:02.779 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-05-16T07:17:02.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:02.780 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_17_00_UTC 2023-05-16T07:17:02.795 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-05-16-07_16_00_UTC 2023-05-16T07:17:02.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:02.795 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-05-16-07_16_00_UTC) 2023-05-16T07:17:02.853 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-05-16-07_17_00_UTC 2023-05-16T07:17:02.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:02.854 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-05-16-07_17_00_UTC) 2023-05-16T07:17:02.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:02.913 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-05-16T07:17:02.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:17:03.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 Processor -- start 2023-05-16T07:17:03.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 -- start start 2023-05-16T07:17:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefe0153bb0 0x7fefe00698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefe0153f90 0x7fefe0067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe005e760 con 0x7fefe00bd810 2023-05-16T07:17:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe0069ef0 con 0x7fefe0153bb0 2023-05-16T07:17:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.117+0000 7fefe5950700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe00bf540 con 0x7fefe0153f90 2023-05-16T07:17:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe494e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefe0153bb0 0x7fefe00698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe514f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fefe00bd810 0x7fefe00634d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55338/0 (socket says 172.21.15.92:55338) 2023-05-16T07:17:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe514f700 1 -- 172.21.15.92:0/2077357328 learned_addr learned my addr 172.21.15.92:0/2077357328 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe494e700 1 -- 172.21.15.92:0/2077357328 >> v2:172.21.15.158:3300/0 conn(0x7fefe0153f90 msgr2=0x7fefe0067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefdffff700 1 --2- 172.21.15.92:0/2077357328 >> v2:172.21.15.158:3300/0 conn(0x7fefe0153f90 0x7fefe0067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe494e700 1 --2- 172.21.15.92:0/2077357328 >> v2:172.21.15.158:3300/0 conn(0x7fefe0153f90 0x7fefe0067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe494e700 1 -- 172.21.15.92:0/2077357328 >> v1:172.21.15.92:6789/0 conn(0x7fefe00bd810 legacy=0x7fefe00634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefdf7fe700 1 -- 172.21.15.92:0/2077357328 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293228047 0 0) 0x7fefe005e760 con 0x7fefe00bd810 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe494e700 1 -- 172.21.15.92:0/2077357328 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0068450 con 0x7fefe0153bb0 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefdffff700 1 --2- 172.21.15.92:0/2077357328 >> v2:172.21.15.158:3300/0 conn(0x7fefe0153f90 0x7fefe0067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefe494e700 1 --2- 172.21.15.92:0/2077357328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe00698c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fefe00be7f0 tx=0x7fefd0009060).ready entity=mon.0 client_cookie=e90f7b96cabb7e33 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefdf7fe700 1 -- 172.21.15.92:0/2077357328 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000b580 con 0x7fefe0153bb0 2023-05-16T07:17:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefdf7fe700 1 -- 172.21.15.92:0/2077357328 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000b6e0 con 0x7fefe0153bb0 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.118+0000 7fefdf7fe700 1 -- 172.21.15.92:0/2077357328 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000ba60 con 0x7fefe0153bb0 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 -- 172.21.15.92:0/2077357328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 msgr2=0x7fefe00698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 --2- 172.21.15.92:0/2077357328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe00698c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fefe00be7f0 tx=0x7fefd0009060).stop 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 -- 172.21.15.92:0/2077357328 shutdown_connections 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 --2- 172.21.15.92:0/2077357328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe00698c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 --2- 172.21.15.92:0/2077357328 >> v2:172.21.15.158:3300/0 conn(0x7fefe0153f90 0x7fefe0067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 -- 172.21.15.92:0/2077357328 >> 172.21.15.92:0/2077357328 conn(0x7fefe0062d80 msgr2=0x7fefe005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 -- 172.21.15.92:0/2077357328 shutdown_connections 2023-05-16T07:17:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.119+0000 7fefe5950700 1 -- 172.21.15.92:0/2077357328 wait complete. 2023-05-16T07:17:03.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.120+0000 7fefe5950700 1 Processor -- start 2023-05-16T07:17:03.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.120+0000 7fefe5950700 1 -- start start 2023-05-16T07:17:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefe00bd810 0x7fefe00c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefe0153f90 0x7fefe01563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0163f20 con 0x7fefe0153bb0 2023-05-16T07:17:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0163bd0 con 0x7fefe00bd810 2023-05-16T07:17:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00c1300 con 0x7fefe0153f90 2023-05-16T07:17:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe0152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44594/0 (socket says 172.21.15.92:44594) 2023-05-16T07:17:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 -- 172.21.15.92:0/4017592626 learned_addr learned my addr 172.21.15.92:0/4017592626 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe494e700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefe00bd810 0x7fefe00c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe514f700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefe0153f90 0x7fefe01563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 -- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefe0153f90 msgr2=0x7fefe01563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefe0153f90 0x7fefe01563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 -- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefe00bd810 msgr2=0x7fefe00c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefe00bd810 0x7fefe00c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 -- 172.21.15.92:0/4017592626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd00092a0 con 0x7fefe0153bb0 2023-05-16T07:17:03.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdffff700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe0152440 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fefc800c670 tx=0x7fefc8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:03.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8015070 con 0x7fefe0153bb0 2023-05-16T07:17:03.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8010580 con 0x7fefe0153bb0 2023-05-16T07:17:03.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe00c15e0 con 0x7fefe0153bb0 2023-05-16T07:17:03.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8010900 con 0x7fefe0153bb0 2023-05-16T07:17:03.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.121+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00c0fc0 con 0x7fefe0153bb0 2023-05-16T07:17:03.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.122+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe0047150 con 0x7fefe0153bb0 2023-05-16T07:17:03.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.123+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fefc8010d30 con 0x7fefe0153bb0 2023-05-16T07:17:03.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.123+0000 7fefdcff9700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefc005ee70 0x7fefc0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:03.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.123+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6965+0+0 (secure 0 0 0) 0x7fefc8079c80 con 0x7fefe0153bb0 2023-05-16T07:17:03.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.123+0000 7fefe494e700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefc005ee70 0x7fefc0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:03.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.124+0000 7fefe494e700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefc005ee70 0x7fefc0061330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fefd0009fd0 tx=0x7fefd0017000).ready entity=mgr.5593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:03.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.128+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fefc804aeb0 con 0x7fefe0153bb0 2023-05-16T07:17:03.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.333+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fefe0158460 con 0x7fefe0153bb0 2023-05-16T07:17:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.895+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fefc8077810 con 0x7fefe0153bb0 2023-05-16T07:17:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.896+0000 7fefdcff9700 1 -- 172.21.15.92:0/4017592626 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v21) v1 ==== 93+0+0 (secure 0 0 0) 0x7fefc8072660 con 0x7fefe0153bb0 2023-05-16T07:17:03.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.899+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefc005ee70 msgr2=0x7fefc0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:03.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.899+0000 7fefe5950700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefc005ee70 0x7fefc0061330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fefd0009fd0 tx=0x7fefd0017000).stop 2023-05-16T07:17:03.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.899+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 msgr2=0x7fefe0152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:03.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.899+0000 7fefe5950700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe0152440 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fefc800c670 tx=0x7fefc8008040).stop 2023-05-16T07:17:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 shutdown_connections 2023-05-16T07:17:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefc005ee70 0x7fefc0061330 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefe0153f90 0x7fefe01563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefe00bd810 0x7fefe00c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 --2- 172.21.15.92:0/4017592626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe0153bb0 0x7fefe0152440 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:03.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 >> 172.21.15.92:0/4017592626 conn(0x7fefe0062d80 msgr2=0x7fefe0153340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:03.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 shutdown_connections 2023-05-16T07:17:03.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:03.904+0000 7fefe5950700 1 -- 172.21.15.92:0/4017592626 wait complete. 2023-05-16T07:17:03.925 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:17:03.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:03.955 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:03.992 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:17:03.992 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:17:03.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:03.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:04.034 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:17:04.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:04.035 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:17:04.049 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:04.045+0000 7f9f3ba80680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:04.051 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:04.048+0000 7f3e3f92f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:04.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:04.113 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:04.109+0000 7f3e3f92f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:04.114 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:04.110+0000 7f9f3ba80680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:04.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.215+0000 7fc722a32700 1 Processor -- start 2023-05-16T07:17:04.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.215+0000 7fc722a32700 1 -- start start 2023-05-16T07:17:04.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc722a32700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc71c154550 0x7fc71c0aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc722a32700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc71c0c5cb0 0x7fc71c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc722a32700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0c6650 con 0x7fc71c154170 2023-05-16T07:17:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc722a32700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c0ab200 con 0x7fc71c154550 2023-05-16T07:17:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc722a32700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c0c08b0 con 0x7fc71c0c5cb0 2023-05-16T07:17:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc71c154550 0x7fc71c0aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc71c154550 0x7fc71c0aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44662/0 (socket says 172.21.15.92:44662) 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 -- 172.21.15.92:0/1832944759 learned_addr learned my addr 172.21.15.92:0/1832944759 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc71c154170 0x7fc71c0a9d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55408/0 (socket says 172.21.15.92:55408) 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc720fcf700 1 --2- 172.21.15.92:0/1832944759 >> v2:172.21.15.158:3300/0 conn(0x7fc71c0c5cb0 0x7fc71c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 -- 172.21.15.92:0/1832944759 >> v2:172.21.15.158:3300/0 conn(0x7fc71c0c5cb0 msgr2=0x7fc71c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 --2- 172.21.15.92:0/1832944759 >> v2:172.21.15.158:3300/0 conn(0x7fc71c0c5cb0 0x7fc71c0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 -- 172.21.15.92:0/1832944759 >> v1:172.21.15.92:6789/0 conn(0x7fc71c154170 legacy=0x7fc71c0a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71affd700 1 -- 172.21.15.92:0/1832944759 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982552554 0 0) 0x7fc710001110 con 0x7fc71c154170 2023-05-16T07:17:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 -- 172.21.15.92:0/1832944759 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0bfca0 con 0x7fc71c154550 2023-05-16T07:17:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc720fcf700 1 --2- 172.21.15.92:0/1832944759 >> v2:172.21.15.158:3300/0 conn(0x7fc71c0c5cb0 0x7fc71c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71b7fe700 1 --2- 172.21.15.92:0/1832944759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c0aad10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fc71c0c0820 tx=0x7fc70c0090f0).ready entity=mon.0 client_cookie=d45cced4e0ae32e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.217+0000 7fc71affd700 1 -- 172.21.15.92:0/1832944759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00b600 con 0x7fc71c154550 2023-05-16T07:17:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc71affd700 1 -- 172.21.15.92:0/1832944759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c00b760 con 0x7fc71c154550 2023-05-16T07:17:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc71affd700 1 -- 172.21.15.92:0/1832944759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00bae0 con 0x7fc71c154550 2023-05-16T07:17:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 -- 172.21.15.92:0/1832944759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 msgr2=0x7fc71c0aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 --2- 172.21.15.92:0/1832944759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c0aad10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fc71c0c0820 tx=0x7fc70c0090f0).stop 2023-05-16T07:17:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 -- 172.21.15.92:0/1832944759 shutdown_connections 2023-05-16T07:17:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 --2- 172.21.15.92:0/1832944759 >> v2:172.21.15.158:3300/0 conn(0x7fc71c0c5cb0 0x7fc71c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 --2- 172.21.15.92:0/1832944759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c0aad10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 -- 172.21.15.92:0/1832944759 >> 172.21.15.92:0/1832944759 conn(0x7fc71c0cb6d0 msgr2=0x7fc71c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 -- 172.21.15.92:0/1832944759 shutdown_connections 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.218+0000 7fc722a32700 1 -- 172.21.15.92:0/1832944759 wait complete. 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.220+0000 7fc722a32700 1 Processor -- start 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.220+0000 7fc722a32700 1 -- start start 2023-05-16T07:17:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc71c0c5cb0 0x7fc71c1649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc71c154170 0x7fc71c159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c164ec0 con 0x7fc71c154550 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0c2780 con 0x7fc71c154170 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0c2d40 con 0x7fc71c0c5cb0 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44670/0 (socket says 172.21.15.92:44670) 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 -- 172.21.15.92:0/1934142488 learned_addr learned my addr 172.21.15.92:0/1934142488 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc71b7fe700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc71c154170 0x7fc71c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc71bfff700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc71c0c5cb0 0x7fc71c1649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 -- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc71c0c5cb0 msgr2=0x7fc71c1649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc71c0c5cb0 0x7fc71c1649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 -- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc71c154170 msgr2=0x7fc71c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc71c154170 0x7fc71c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 -- 172.21.15.92:0/1934142488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c009370 con 0x7fc71c154550 2023-05-16T07:17:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc720fcf700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c156c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc71400d180 tx=0x7fc714009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400dc60 con 0x7fc71c154550 2023-05-16T07:17:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71400f040 con 0x7fc71c154550 2023-05-16T07:17:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c1576f0 con 0x7fc71c154550 2023-05-16T07:17:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.222+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140133f0 con 0x7fc71c154550 2023-05-16T07:17:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.223+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c0c29d0 con 0x7fc71c154550 2023-05-16T07:17:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.224+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fc7140137e0 con 0x7fc71c154550 2023-05-16T07:17:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.224+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c0666c0 con 0x7fc71c154550 2023-05-16T07:17:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.224+0000 7fc703fff700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 0x7fc7080612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.224+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6965+0+0 (secure 0 0 0) 0x7fc71407a620 con 0x7fc71c154550 2023-05-16T07:17:04.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.224+0000 7fc71bfff700 1 -- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 msgr2=0x7fc7080612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:04.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.224+0000 7fc71bfff700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 0x7fc7080612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:04.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.228+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc71404b7f0 con 0x7fc71c154550 2023-05-16T07:17:04.383 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:04.379+0000 7f3e3f92f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:04.406 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:04.402+0000 7f9f3ba80680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:04.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.425+0000 7fc71bfff700 1 -- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 msgr2=0x7fc7080612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:04.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.425+0000 7fc71bfff700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 0x7fc7080612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:04.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.449+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc71c06c350 con 0x7fc71c154550 2023-05-16T07:17:04.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.449+0000 7fc703fff700 1 -- 172.21.15.92:0/1934142488 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fc71404ee10 con 0x7fc71c154550 2023-05-16T07:17:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:04.453 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:14:54.221623+0000","modified":"2023-05-16T07:14:56.230335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/2827835616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2827835616},{"type":"v1","addr":"172.21.15.158:6837","nonce":2827835616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-05-16T07:17:04.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 msgr2=0x7fc7080612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:04.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 0x7fc7080612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:04.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 msgr2=0x7fc71c156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c156c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc71400d180 tx=0x7fc714009a50).stop 2023-05-16T07:17:04.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 shutdown_connections 2023-05-16T07:17:04.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc70805ee20 0x7fc7080612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:04.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc71c0c5cb0 0x7fc71c1649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c154550 0x7fc71c156c70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 --2- 172.21.15.92:0/1934142488 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc71c154170 0x7fc71c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 >> 172.21.15.92:0/1934142488 conn(0x7fc71c0cb6d0 msgr2=0x7fc71c15ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:04.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 shutdown_connections 2023-05-16T07:17:04.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.451+0000 7fc722a32700 1 -- 172.21.15.92:0/1934142488 wait complete. 2023-05-16T07:17:04.458 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-05-16T07:17:04.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.617+0000 7f14bb514700 1 Processor -- start 2023-05-16T07:17:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.617+0000 7f14bb514700 1 -- start start 2023-05-16T07:17:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14bb514700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f14b4150230 0x7f14b4066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14bb514700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f14b40bc3d0 0x7f14b40c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14bb514700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b4066e50 con 0x7f14b4150610 2023-05-16T07:17:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14bb514700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b40676f0 con 0x7f14b4150230 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14bb514700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b4065be0 con 0x7f14b40bc3d0 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f14b4150230 0x7f14b4066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f14b4150230 0x7f14b4066420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44720/0 (socket says 172.21.15.92:44720) 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 -- 172.21.15.92:0/3658111836 learned_addr learned my addr 172.21.15.92:0/3658111836 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14bad13700 1 --2- 172.21.15.92:0/3658111836 >> v2:172.21.15.158:3300/0 conn(0x7f14b40bc3d0 0x7f14b40c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 -- 172.21.15.92:0/3658111836 >> v2:172.21.15.158:3300/0 conn(0x7f14b40bc3d0 msgr2=0x7f14b40c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:04.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 --2- 172.21.15.92:0/3658111836 >> v2:172.21.15.158:3300/0 conn(0x7f14b40bc3d0 0x7f14b40c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 -- 172.21.15.92:0/3658111836 >> v1:172.21.15.158:6789/0 conn(0x7f14b4150610 legacy=0x7f14b4065430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 -- 172.21.15.92:0/3658111836 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40670a0 con 0x7f14b4150230 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14ba512700 1 --2- 172.21.15.92:0/3658111836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b4066420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f14b4060090 tx=0x7f14b000b370).ready entity=mon.0 client_cookie=7b3ee37e4e9944bb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.618+0000 7f14b9510700 1 -- 172.21.15.92:0/3658111836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b000dc40 con 0x7f14b4150230 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14b9510700 1 -- 172.21.15.92:0/3658111836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b000dda0 con 0x7f14b4150230 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14b9510700 1 -- 172.21.15.92:0/3658111836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0015600 con 0x7f14b4150230 2023-05-16T07:17:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 -- 172.21.15.92:0/3658111836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 msgr2=0x7f14b4066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 --2- 172.21.15.92:0/3658111836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b4066420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f14b4060090 tx=0x7f14b000b370).stop 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 -- 172.21.15.92:0/3658111836 shutdown_connections 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 --2- 172.21.15.92:0/3658111836 >> v2:172.21.15.158:3300/0 conn(0x7f14b40bc3d0 0x7f14b40c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 --2- 172.21.15.92:0/3658111836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b4066420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 -- 172.21.15.92:0/3658111836 >> 172.21.15.92:0/3658111836 conn(0x7f14b4069940 msgr2=0x7f14b40c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 -- 172.21.15.92:0/3658111836 shutdown_connections 2023-05-16T07:17:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.619+0000 7f14bb514700 1 -- 172.21.15.92:0/3658111836 wait complete. 2023-05-16T07:17:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.621+0000 7f14bb514700 1 Processor -- start 2023-05-16T07:17:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.621+0000 7f14bb514700 1 -- start start 2023-05-16T07:17:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bb514700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14b40bc3d0 0x7f14b4144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bb514700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bb514700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14b4150610 0x7f14b40aa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bb514700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40c5050 con 0x7f14b4150230 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bb514700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b414f040 con 0x7f14b40bc3d0 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bb514700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b414e7a0 con 0x7f14b4150610 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b414dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44726/0 (socket says 172.21.15.92:44726) 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 -- 172.21.15.92:0/4102104641 learned_addr learned my addr 172.21.15.92:0/4102104641 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14ba512700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14b40bc3d0 0x7f14b4144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14bad13700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14b4150610 0x7f14b40aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 -- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14b4150610 msgr2=0x7f14b40aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14b4150610 0x7f14b40aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 -- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14b40bc3d0 msgr2=0x7f14b4144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14b40bc3d0 0x7f14b4144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.622+0000 7f14b9d11700 1 -- 172.21.15.92:0/4102104641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b000b6a0 con 0x7f14b4150230 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.623+0000 7f14b9d11700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b414dd50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f14a400c4c0 tx=0x7f14a4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.623+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400ce70 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.623+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4001e90 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.623+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b414ea80 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.623+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4007270 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.623+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b414d6d0 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.624+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b4047150 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.624+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f14a4010070 con 0x7f14b4150230 2023-05-16T07:17:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.624+0000 7f14aa7fc700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f149805ee20 0x7f14980612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.624+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6965+0+0 (secure 0 0 0) 0x7f14a40795b0 con 0x7f14b4150230 2023-05-16T07:17:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.624+0000 7f14ba512700 1 -- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f149805ee20 msgr2=0x7f14980612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.624+0000 7f14ba512700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f149805ee20 0x7f14980612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.629+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f14a404a7e0 con 0x7f14b4150230 2023-05-16T07:17:04.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.797+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14b40a9ab0 con 0x7f14b4150230 2023-05-16T07:17:04.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.798+0000 7f14aa7fc700 1 -- 172.21.15.92:0/4102104641 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18250 (secure 0 0 0) 0x7f14a404de00 con 0x7f14b4150230 2023-05-16T07:17:04.802 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:04.802 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":65,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:16:06.721313+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:14:52.750879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-05-16T07:14:53.182846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]},{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/1803957237":"2023-05-17T07:15:05.377224+0000","172.21.15.158:0/1247324037":"2023-05-17T07:15:05.377224+0000","172.21.15.158:6833/86301":"2023-05-17T07:15:05.377224+0000","172.21.15.158:6832/86301":"2023-05-17T07:15:05.377224+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:04.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f149805ee20 msgr2=0x7f14980612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:04.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f149805ee20 0x7f14980612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:04.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 msgr2=0x7f14b414dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b414dd50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f14a400c4c0 tx=0x7f14a4007e00).stop 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 shutdown_connections 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f149805ee20 0x7f14980612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14b4150610 0x7f14b40aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14b40bc3d0 0x7f14b4144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 --2- 172.21.15.92:0/4102104641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b4150230 0x7f14b414dd50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 >> 172.21.15.92:0/4102104641 conn(0x7f14b4069940 msgr2=0x7f14b40c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:04.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 shutdown_connections 2023-05-16T07:17:04.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.800+0000 7f14bb514700 1 -- 172.21.15.92:0/4102104641 wait complete. 2023-05-16T07:17:04.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:14:52.750879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:14:53.182846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:17:04.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:17:04.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.936+0000 7f0868b2c700 1 Processor -- start 2023-05-16T07:17:04.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.936+0000 7f0868b2c700 1 -- start start 2023-05-16T07:17:04.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0868b2c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0864150d90 0x7f0864069b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0868b2c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f086406a070 0x7f08640c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0868b2c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086406a4e0 con 0x7f08641509b0 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0868b2c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08640bc1b0 con 0x7f086406a070 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0868b2c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08640bc490 con 0x7f0864150d90 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0862d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f086406a070 0x7f08640c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0862d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f086406a070 0x7f08640c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44736/0 (socket says 172.21.15.92:44736) 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0862d9d700 1 -- 172.21.15.92:0/753607189 learned_addr learned my addr 172.21.15.92:0/753607189 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.937+0000 7f0861d9b700 1 --2- 172.21.15.92:0/753607189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0864150d90 0x7f0864069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0862d9d700 1 -- 172.21.15.92:0/753607189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0864150d90 msgr2=0x7f0864069b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0862d9d700 1 --2- 172.21.15.92:0/753607189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0864150d90 0x7f0864069b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0862d9d700 1 -- 172.21.15.92:0/753607189 >> v1:172.21.15.158:6789/0 conn(0x7f08641509b0 legacy=0x7f0864065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0862d9d700 1 -- 172.21.15.92:0/753607189 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640bc770 con 0x7f086406a070 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0862d9d700 1 --2- 172.21.15.92:0/753607189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08640c16b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f085c009f20 tx=0x7f085c009840).ready entity=mon.0 client_cookie=519f0fd77d17351d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f086159a700 1 -- 172.21.15.92:0/753607189 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00e070 con 0x7f086406a070 2023-05-16T07:17:04.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f086159a700 1 -- 172.21.15.92:0/753607189 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c00ba80 con 0x7f086406a070 2023-05-16T07:17:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f086159a700 1 -- 172.21.15.92:0/753607189 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00be00 con 0x7f086406a070 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0868b2c700 1 -- 172.21.15.92:0/753607189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 msgr2=0x7f08640c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.938+0000 7f0868b2c700 1 --2- 172.21.15.92:0/753607189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08640c16b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f085c009f20 tx=0x7f085c009840).stop 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.939+0000 7f0868b2c700 1 -- 172.21.15.92:0/753607189 shutdown_connections 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.939+0000 7f0868b2c700 1 --2- 172.21.15.92:0/753607189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08640c16b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.939+0000 7f0868b2c700 1 --2- 172.21.15.92:0/753607189 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0864150d90 0x7f0864069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.939+0000 7f0868b2c700 1 -- 172.21.15.92:0/753607189 >> 172.21.15.92:0/753607189 conn(0x7f08640c4620 msgr2=0x7f0864066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.939+0000 7f0868b2c700 1 -- 172.21.15.92:0/753607189 shutdown_connections 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.939+0000 7f0868b2c700 1 -- 172.21.15.92:0/753607189 wait complete. 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 Processor -- start 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 -- start start 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08641548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f08641509b0 0x7f08641562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0864150d90 0x7f08640c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641567c0 con 0x7f086406a070 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0864155b10 con 0x7f0864150d90 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.940+0000 7f0868b2c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0864154f90 con 0x7f08641509b0 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08641548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08641548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44738/0 (socket says 172.21.15.92:44738) 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 -- 172.21.15.92:0/589191757 learned_addr learned my addr 172.21.15.92:0/589191757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f0862d9d700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0864150d90 0x7f08640c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f0861d9b700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f08641509b0 0x7f08641562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 -- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f08641509b0 msgr2=0x7f08641562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f08641509b0 0x7f08641562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 -- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0864150d90 msgr2=0x7f08640c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0864150d90 0x7f08640c1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 -- 172.21.15.92:0/589191757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c009ae0 con 0x7f086406a070 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f0861d9b700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f08641509b0 0x7f08641562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f086259c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08641548d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f085800c4c0 tx=0x7f0858007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858007020 con 0x7f086406a070 2023-05-16T07:17:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858001e90 con 0x7f086406a070 2023-05-16T07:17:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580073b0 con 0x7f086406a070 2023-05-16T07:17:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0864155270 con 0x7f086406a070 2023-05-16T07:17:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.941+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0864155830 con 0x7f086406a070 2023-05-16T07:17:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.942+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f0858010070 con 0x7f086406a070 2023-05-16T07:17:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.942+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086406c350 con 0x7f086406a070 2023-05-16T07:17:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.943+0000 7f08527fc700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 0x7f08440612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.943+0000 7f0861d9b700 1 -- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 msgr2=0x7f08440612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.943+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f08580797c0 con 0x7f086406a070 2023-05-16T07:17:04.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.943+0000 7f0861d9b700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 0x7f08440612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:04.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:04.947+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f085804a940 con 0x7f086406a070 2023-05-16T07:17:04.991 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:04.987+0000 7f3e3f92f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:05.049 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:05.045+0000 7f9f3ba80680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:05.064 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:05.060+0000 7f3e3f92f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:05.119 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:05.115+0000 7f9f3ba80680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.144+0000 7f0861d9b700 1 -- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 msgr2=0x7f08440612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:05.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.144+0000 7f0861d9b700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 0x7f08440612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:05.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.144+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f086405de90 con 0x7f086406a070 2023-05-16T07:17:05.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.144+0000 7f08527fc700 1 -- 172.21.15.92:0/589191757 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7f085804df60 con 0x7f086406a070 2023-05-16T07:17:05.149 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:17:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 msgr2=0x7f08440612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 0x7f08440612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 msgr2=0x7f08641548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08641548d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f085800c4c0 tx=0x7f0858007e00).stop 2023-05-16T07:17:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 shutdown_connections 2023-05-16T07:17:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f084405ee20 0x7f08440612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0864150d90 0x7f08640c1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086406a070 0x7f08641548d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 --2- 172.21.15.92:0/589191757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f08641509b0 0x7f08641562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.146+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 >> 172.21.15.92:0/589191757 conn(0x7f08640c4620 msgr2=0x7f08640c4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.147+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 shutdown_connections 2023-05-16T07:17:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.147+0000 7f0868b2c700 1 -- 172.21.15.92:0/589191757 wait complete. 2023-05-16T07:17:05.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:17:05.249 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:05.245+0000 7f3e3f92f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:05.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.278+0000 7f6234987700 1 Processor -- start 2023-05-16T07:17:05.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.278+0000 7f6234987700 1 -- start start 2023-05-16T07:17:05.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f6234987700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f6234987700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62300c6110 0x7f6230069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f6234987700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6230153aa0 0x7f6230153f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f6234987700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300c8830 con 0x7f62300c6110 2023-05-16T07:17:05.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f6234987700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300c8b10 con 0x7f6230153aa0 2023-05-16T07:17:05.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f6234987700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c8df0 con 0x7f62300c5d30 2023-05-16T07:17:05.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62300c6110 0x7f6230069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62300c6110 0x7f6230069ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44748/0 (socket says 172.21.15.92:44748) 2023-05-16T07:17:05.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 -- 172.21.15.92:0/470273349 learned_addr learned my addr 172.21.15.92:0/470273349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622e59c700 1 --2- 172.21.15.92:0/470273349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622ed9d700 1 --2- 172.21.15.92:0/470273349 >> v2:172.21.15.158:3300/0 conn(0x7f6230153aa0 0x7f6230153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 -- 172.21.15.92:0/470273349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 msgr2=0x7f62300aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 --2- 172.21.15.92:0/470273349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 -- 172.21.15.92:0/470273349 >> v2:172.21.15.158:3300/0 conn(0x7f6230153aa0 msgr2=0x7f6230153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 --2- 172.21.15.92:0/470273349 >> v2:172.21.15.158:3300/0 conn(0x7f6230153aa0 0x7f6230153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 -- 172.21.15.92:0/470273349 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300c90d0 con 0x7f62300c6110 2023-05-16T07:17:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622ed9d700 1 --2- 172.21.15.92:0/470273349 >> v2:172.21.15.158:3300/0 conn(0x7f6230153aa0 0x7f6230153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622dd9b700 1 --2- 172.21.15.92:0/470273349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f6230069ff0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f623006c040 tx=0x7f62240090f0).ready entity=mon.0 client_cookie=b67c844677b3ea7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.279+0000 7f622d59a700 1 -- 172.21.15.92:0/470273349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400b520 con 0x7f62300c6110 2023-05-16T07:17:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f622d59a700 1 -- 172.21.15.92:0/470273349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622400b680 con 0x7f62300c6110 2023-05-16T07:17:05.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f622d59a700 1 -- 172.21.15.92:0/470273349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400ba00 con 0x7f62300c6110 2023-05-16T07:17:05.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 -- 172.21.15.92:0/470273349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 msgr2=0x7f6230069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 --2- 172.21.15.92:0/470273349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f6230069ff0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f623006c040 tx=0x7f62240090f0).stop 2023-05-16T07:17:05.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 -- 172.21.15.92:0/470273349 shutdown_connections 2023-05-16T07:17:05.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 --2- 172.21.15.92:0/470273349 >> v2:172.21.15.158:3300/0 conn(0x7f6230153aa0 0x7f6230153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 --2- 172.21.15.92:0/470273349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 --2- 172.21.15.92:0/470273349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f6230069ff0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 -- 172.21.15.92:0/470273349 >> 172.21.15.92:0/470273349 conn(0x7f62300cb700 msgr2=0x7f62300c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:05.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 -- 172.21.15.92:0/470273349 shutdown_connections 2023-05-16T07:17:05.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.280+0000 7f6234987700 1 -- 172.21.15.92:0/470273349 wait complete. 2023-05-16T07:17:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.281+0000 7f6234987700 1 Processor -- start 2023-05-16T07:17:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.281+0000 7f6234987700 1 -- start start 2023-05-16T07:17:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f6234987700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f6234987700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f62300c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f6234987700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6230153aa0 0x7f6230151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f6234987700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c2b90 con 0x7f62300c6110 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f6234987700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c2e50 con 0x7f6230153aa0 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f6234987700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c3130 con 0x7f62300c5d30 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f62300c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f62300c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44756/0 (socket says 172.21.15.92:44756) 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 -- 172.21.15.92:0/1219798888 learned_addr learned my addr 172.21.15.92:0/1219798888 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622ed9d700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6230153aa0 0x7f6230151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622e59c700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 -- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 msgr2=0x7f62300c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 -- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6230153aa0 msgr2=0x7f6230151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6230153aa0 0x7f6230151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.282+0000 7f622dd9b700 1 -- 172.21.15.92:0/1219798888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224009370 con 0x7f62300c6110 2023-05-16T07:17:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.283+0000 7f622dd9b700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f62300c26a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f6224011000 tx=0x7f6224017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.283+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224000bc0 con 0x7f62300c6110 2023-05-16T07:17:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.283+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622400b7f0 con 0x7f62300c6110 2023-05-16T07:17:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.283+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62300c3410 con 0x7f62300c6110 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.283+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224000d20 con 0x7f62300c6110 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.283+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6230163c80 con 0x7f62300c6110 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.284+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62300add30 con 0x7f62300c6110 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.284+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f622402c070 con 0x7f62300c6110 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.284+0000 7f621e7fc700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f621005ee20 0x7f62100612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.284+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f622407b890 con 0x7f62300c6110 2023-05-16T07:17:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.284+0000 7f622e59c700 1 -- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f621005ee20 msgr2=0x7f62100612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.284+0000 7f622e59c700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f621005ee20 0x7f62100612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.288+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f62300add30 con 0x7f62300c6110 2023-05-16T07:17:05.315 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:05.311+0000 7f9f3ba80680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:05.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.456+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f62300add30 con 0x7f62300c6110 2023-05-16T07:17:05.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.457+0000 7f621e7fc700 1 -- 172.21.15.92:0/1219798888 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f62300add30 con 0x7f62300c6110 2023-05-16T07:17:05.461 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:17:05.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f621005ee20 msgr2=0x7f62100612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:05.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f621005ee20 0x7f62100612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:05.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 msgr2=0x7f62300c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f62300c26a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f6224011000 tx=0x7f6224017000).stop 2023-05-16T07:17:05.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 shutdown_connections 2023-05-16T07:17:05.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f621005ee20 0x7f62100612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:05.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62300c5d30 0x7f62300c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6230153aa0 0x7f6230151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 --2- 172.21.15.92:0/1219798888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62300c6110 0x7f62300c26a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 >> 172.21.15.92:0/1219798888 conn(0x7f62300cb700 msgr2=0x7f62300c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:05.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 shutdown_connections 2023-05-16T07:17:05.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.458+0000 7f6234987700 1 -- 172.21.15.92:0/1219798888 wait complete. 2023-05-16T07:17:05.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:17:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.589+0000 7f19d6ccd700 1 Processor -- start 2023-05-16T07:17:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.589+0000 7f19d6ccd700 1 -- start start 2023-05-16T07:17:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d6ccd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d0069b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d6ccd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d006a070 0x7f19d00c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d6ccd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d006a4e0 con 0x7f19d01509b0 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d6ccd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d00bc1b0 con 0x7f19d006a070 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d6ccd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d00bc490 con 0x7f19d0150d90 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d006a070 0x7f19d00c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d006a070 0x7f19d00c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44800/0 (socket says 172.21.15.92:44800) 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 -- 172.21.15.92:0/1087406880 learned_addr learned my addr 172.21.15.92:0/1087406880 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19cffff700 1 --2- 172.21.15.92:0/1087406880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d0069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 -- 172.21.15.92:0/1087406880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 msgr2=0x7f19d0069b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 --2- 172.21.15.92:0/1087406880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d0069b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 -- 172.21.15.92:0/1087406880 >> v1:172.21.15.92:6789/0 conn(0x7f19d01509b0 legacy=0x7f19d0065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 -- 172.21.15.92:0/1087406880 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d00bc770 con 0x7f19d006a070 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19cf7fe700 1 -- 172.21.15.92:0/1087406880 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484305937 0 0) 0x7f19d006a4e0 con 0x7f19d01509b0 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.590+0000 7f19d526a700 1 --2- 172.21.15.92:0/1087406880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d00c16b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f19c800d1a0 tx=0x7f19c800ba50).ready entity=mon.0 client_cookie=14513a1af036f34c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19cf7fe700 1 -- 172.21.15.92:0/1087406880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8010070 con 0x7f19d006a070 2023-05-16T07:17:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19cf7fe700 1 -- 172.21.15.92:0/1087406880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c800de10 con 0x7f19d006a070 2023-05-16T07:17:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19cf7fe700 1 -- 172.21.15.92:0/1087406880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c800b2e0 con 0x7f19d006a070 2023-05-16T07:17:05.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 -- 172.21.15.92:0/1087406880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 msgr2=0x7f19d00c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/1087406880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d00c16b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f19c800d1a0 tx=0x7f19c800ba50).stop 2023-05-16T07:17:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 -- 172.21.15.92:0/1087406880 shutdown_connections 2023-05-16T07:17:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/1087406880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d00c16b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/1087406880 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d0069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 -- 172.21.15.92:0/1087406880 >> 172.21.15.92:0/1087406880 conn(0x7f19d00c4620 msgr2=0x7f19d0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 -- 172.21.15.92:0/1087406880 shutdown_connections 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.591+0000 7f19d6ccd700 1 -- 172.21.15.92:0/1087406880 wait complete. 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.592+0000 7f19d6ccd700 1 Processor -- start 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 -- start start 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d01509b0 0x7f19d00c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d00c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d00c19c0 con 0x7f19d006a070 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d0160140 con 0x7f19d01509b0 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d6ccd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d0161360 con 0x7f19d0150d90 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d4a69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d4a69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d014deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44802/0 (socket says 172.21.15.92:44802) 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d4a69700 1 -- 172.21.15.92:0/3500953651 learned_addr learned my addr 172.21.15.92:0/3500953651 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19d526a700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d00c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.593+0000 7f19cffff700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d01509b0 0x7f19d00c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d4a69700 1 -- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 msgr2=0x7f19d00c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d4a69700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d00c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d4a69700 1 -- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d01509b0 msgr2=0x7f19d00c0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d4a69700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d01509b0 0x7f19d00c0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d4a69700 1 -- 172.21.15.92:0/3500953651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c800bcf0 con 0x7f19d006a070 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d4a69700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d014deb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f19c400c4e0 tx=0x7f19c4007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400ce50 con 0x7f19d006a070 2023-05-16T07:17:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400f470 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d014e3a0 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400f830 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.594+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d00c1ea0 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.595+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d0161640 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.595+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f19c4019030 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.595+0000 7f19ccff9700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19b405edf0 0x7f19b40612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.596+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f19c4079190 con 0x7f19d006a070 2023-05-16T07:17:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.596+0000 7f19cffff700 1 -- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19b405edf0 msgr2=0x7f19b40612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:05.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.596+0000 7f19cffff700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19b405edf0 0x7f19b40612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:05.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.600+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f19c40488f0 con 0x7f19d006a070 2023-05-16T07:17:05.665 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:05.662+0000 7f3e3f92f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:05.742 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:05.738+0000 7f9f3ba80680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:05.766 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:05.763+0000 7f3e3f92f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:05.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.768+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19d006c350 con 0x7f19d006a070 2023-05-16T07:17:05.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.768+0000 7f19ccff9700 1 -- 172.21.15.92:0/3500953651 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19c404a440 con 0x7f19d006a070 2023-05-16T07:17:05.773 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:17:05.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19b405edf0 msgr2=0x7f19b40612b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:05.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19b405edf0 0x7f19b40612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:05.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 msgr2=0x7f19d014deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d014deb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f19c400c4e0 tx=0x7f19c4007c70).stop 2023-05-16T07:17:05.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 shutdown_connections 2023-05-16T07:17:05.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19b405edf0 0x7f19b40612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:05.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19d0150d90 0x7f19d00c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d006a070 0x7f19d014deb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 --2- 172.21.15.92:0/3500953651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d01509b0 0x7f19d00c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 >> 172.21.15.92:0/3500953651 conn(0x7f19d00c4620 msgr2=0x7f19d0065a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:05.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 shutdown_connections 2023-05-16T07:17:05.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.770+0000 7f19d6ccd700 1 -- 172.21.15.92:0/3500953651 wait complete. 2023-05-16T07:17:05.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:17:05.848 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:05.845+0000 7f9f3ba80680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:05.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.898+0000 7fdbf7fa9700 1 Processor -- start 2023-05-16T07:17:05.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.898+0000 7fdbf7fa9700 1 -- start start 2023-05-16T07:17:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf7fa9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdbf0154090 0x7fdbf00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf7fa9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf00c94d0 con 0x7fdbf0068160 2023-05-16T07:17:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf7fa9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf00cb7e0 con 0x7fdbf0154470 2023-05-16T07:17:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf7fa9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf00c88c0 con 0x7fdbf0154090 2023-05-16T07:17:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf5d45700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdbf0154090 0x7fdbf00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf5d45700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdbf0154090 0x7fdbf00aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44810/0 (socket says 172.21.15.92:44810) 2023-05-16T07:17:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.899+0000 7fdbf5d45700 1 -- 172.21.15.92:0/2256548649 learned_addr learned my addr 172.21.15.92:0/2256548649 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf5d45700 1 -- 172.21.15.92:0/2256548649 >> v1:172.21.15.158:6789/0 conn(0x7fdbf0154470 legacy=0x7fdbf00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf5d45700 1 -- 172.21.15.92:0/2256548649 >> v1:172.21.15.92:6789/0 conn(0x7fdbf0068160 legacy=0x7fdbf00c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf5d45700 1 -- 172.21.15.92:0/2256548649 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00c8ba0 con 0x7fdbf0154090 2023-05-16T07:17:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf5d45700 1 --2- 172.21.15.92:0/2256548649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00aaca0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fdbec006fd0 tx=0x7fdbec0090f0).ready entity=mon.0 client_cookie=456c37f471eb97fc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf4d43700 1 -- 172.21.15.92:0/2256548649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00b600 con 0x7fdbf0154090 2023-05-16T07:17:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf4d43700 1 -- 172.21.15.92:0/2256548649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec00b760 con 0x7fdbf0154090 2023-05-16T07:17:05.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.900+0000 7fdbf4d43700 1 -- 172.21.15.92:0/2256548649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00bae0 con 0x7fdbf0154090 2023-05-16T07:17:05.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/2256548649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 msgr2=0x7fdbf00aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/2256548649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00aaca0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fdbec006fd0 tx=0x7fdbec0090f0).stop 2023-05-16T07:17:05.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/2256548649 shutdown_connections 2023-05-16T07:17:05.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/2256548649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00aaca0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/2256548649 >> 172.21.15.92:0/2256548649 conn(0x7fdbf0067920 msgr2=0x7fdbf00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/2256548649 shutdown_connections 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.901+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/2256548649 wait complete. 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.902+0000 7fdbf7fa9700 1 Processor -- start 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.902+0000 7fdbf7fa9700 1 -- start start 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf7fa9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 0x7fdbf0152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf7fa9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00c0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf7fa9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbf0154470 0x7fdbf00c0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf7fa9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0156190 con 0x7fdbf0154090 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf7fa9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00c0a40 con 0x7fdbf0154470 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf7fa9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0167e30 con 0x7fdbf0068160 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5d45700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 0x7fdbf0152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00c0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5d45700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 0x7fdbf0152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:37388/0 (socket says 172.21.15.92:37388) 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf6546700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbf0154470 0x7fdbf00c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00c0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44818/0 (socket says 172.21.15.92:44818) 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5d45700 1 -- 172.21.15.92:0/3473102306 learned_addr learned my addr 172.21.15.92:0/3473102306 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 -- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 msgr2=0x7fdbf0152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 0x7fdbf0152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 -- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbf0154470 msgr2=0x7fdbf00c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbf0154470 0x7fdbf00c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5544700 1 -- 172.21.15.92:0/3473102306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec009370 con 0x7fdbf0154090 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf6546700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbf0154470 0x7fdbf00c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.903+0000 7fdbf5d45700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 0x7fdbf0152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.904+0000 7fdbf5544700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00c0060 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fdbe400c440 tx=0x7fdbe4007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.905+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4007020 con 0x7fdbf0154090 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.905+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf0168970 con 0x7fdbf0154090 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.905+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe4007390 con 0x7fdbf0154090 2023-05-16T07:17:05.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.905+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4007500 con 0x7fdbf0154090 2023-05-16T07:17:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.905+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf0168320 con 0x7fdbf0154090 2023-05-16T07:17:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.906+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf0066730 con 0x7fdbf0154090 2023-05-16T07:17:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.906+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fdbe4010070 con 0x7fdbf0154090 2023-05-16T07:17:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.907+0000 7fdbe1ffb700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbd405ee20 0x7fdbd40612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.907+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7fdbe4078fd0 con 0x7fdbf0154090 2023-05-16T07:17:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.907+0000 7fdbf5d45700 1 -- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbd405ee20 msgr2=0x7fdbd40612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.907+0000 7fdbf5d45700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbd405ee20 0x7fdbd40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:05.914+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdbe40478f0 con 0x7fdbf0154090 2023-05-16T07:17:05.920 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:05.916+0000 7f3e3f92f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:05.993 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:05.989+0000 7f3e3f92f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:06.005 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:06.002+0000 7f9f3ba80680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:06.058 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:06.054+0000 7f3e3f92f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:06.080 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:06.076+0000 7f9f3ba80680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:06.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.081+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbf00c8eb0 con 0x7fdbf0154090 2023-05-16T07:17:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.081+0000 7fdbe1ffb700 1 -- 172.21.15.92:0/3473102306 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbe404a1d0 con 0x7fdbf0154090 2023-05-16T07:17:06.085 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:17:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbd405ee20 msgr2=0x7fdbd40612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbd405ee20 0x7fdbd40612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 msgr2=0x7fdbf00c0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00c0060 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fdbe400c440 tx=0x7fdbe4007d40).stop 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 shutdown_connections 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbd405ee20 0x7fdbd40612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdbf0068160 0x7fdbf0152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdbf0154470 0x7fdbf00c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 --2- 172.21.15.92:0/3473102306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0154090 0x7fdbf00c0060 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.083+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 >> 172.21.15.92:0/3473102306 conn(0x7fdbf0067920 msgr2=0x7fdbf00aa040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.084+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 shutdown_connections 2023-05-16T07:17:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.084+0000 7fdbf7fa9700 1 -- 172.21.15.92:0/3473102306 wait complete. 2023-05-16T07:17:06.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:06.151 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:06.147+0000 7f9f3ba80680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:06.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.208+0000 7f7349074700 1 Processor -- start 2023-05-16T07:17:06.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.208+0000 7f7349074700 1 -- start start 2023-05-16T07:17:06.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7349074700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7344154040 0x7f7344069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7349074700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 0x7f7344062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7349074700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7344062960 con 0x7f7344063720 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7349074700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7344154420 con 0x7f7344154040 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7349074700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7344069fb0 con 0x7f73440bf4f0 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7342d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7344154040 0x7f7344069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f734259c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 0x7f7344062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f734359e700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f7344063720 0x7f73440bda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:59174/0 (socket says 172.21.15.92:59174) 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f7342d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7344154040 0x7f7344069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:48460/0 (socket says 172.21.15.92:48460) 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f734259c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 0x7f7344062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:37396/0 (socket says 172.21.15.92:37396) 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.209+0000 7f734359e700 1 -- 172.21.15.92:0/1844703810 learned_addr learned my addr 172.21.15.92:0/1844703810 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7342d9d700 1 -- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 msgr2=0x7f7344062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7342d9d700 1 --2- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 0x7f7344062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7342d9d700 1 -- 172.21.15.92:0/1844703810 >> v1:172.21.15.158:6789/0 conn(0x7f7344063720 legacy=0x7f73440bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7342d9d700 1 -- 172.21.15.92:0/1844703810 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7344062350 con 0x7f7344154040 2023-05-16T07:17:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f734259c700 1 --2- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 0x7f7344062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7342d9d700 1 --2- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7344154040 0x7f7344069ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f732c009760 tx=0x7f732c009060).ready entity=mon.1 client_cookie=109e6e8360598f36 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7341d9b700 1 -- 172.21.15.92:0/1844703810 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00f510 con 0x7f7344154040 2023-05-16T07:17:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7341d9b700 1 -- 172.21.15.92:0/1844703810 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c00f670 con 0x7f7344154040 2023-05-16T07:17:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.210+0000 7f7341d9b700 1 -- 172.21.15.92:0/1844703810 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00f9f0 con 0x7f7344154040 2023-05-16T07:17:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 -- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7344154040 msgr2=0x7f7344069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 --2- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7344154040 0x7f7344069ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f732c009760 tx=0x7f732c009060).stop 2023-05-16T07:17:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 -- 172.21.15.92:0/1844703810 shutdown_connections 2023-05-16T07:17:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 --2- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7344154040 0x7f7344069ac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 --2- 172.21.15.92:0/1844703810 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73440bf4f0 0x7f7344062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 -- 172.21.15.92:0/1844703810 >> 172.21.15.92:0/1844703810 conn(0x7f73440c8aa0 msgr2=0x7f73440c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 -- 172.21.15.92:0/1844703810 shutdown_connections 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.211+0000 7f7349074700 1 -- 172.21.15.92:0/1844703810 wait complete. 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.212+0000 7f7349074700 1 Processor -- start 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.212+0000 7f7349074700 1 -- start start 2023-05-16T07:17:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 0x7f73440c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73440bf4f0 0x7f73440c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7344154040 0x7f73440c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73440aebf0 con 0x7f7344063720 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73440c10d0 con 0x7f73440bf4f0 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73440adfe0 con 0x7f7344154040 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 0x7f73440c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 0x7f73440c0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44828/0 (socket says 172.21.15.92:44828) 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 -- 172.21.15.92:0/2525061577 learned_addr learned my addr 172.21.15.92:0/2525061577 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f734359e700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7344154040 0x7f73440c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f734259c700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73440bf4f0 0x7f73440c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 -- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7344154040 msgr2=0x7f73440c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7344154040 0x7f73440c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 -- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73440bf4f0 msgr2=0x7f73440c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73440bf4f0 0x7f73440c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 -- 172.21.15.92:0/2525061577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340092a0 con 0x7f7344063720 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7342d9d700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 0x7f73440c0be0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f732c009e00 tx=0x7f732c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c01a4d0 con 0x7f7344063720 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c0092a0 con 0x7f7344063720 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c01e070 con 0x7f7344063720 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c01a8a0 con 0x7f7344063720 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.213+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73440ae5a0 con 0x7f7344063720 2023-05-16T07:17:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.215+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f732c00f070 con 0x7f7344063720 2023-05-16T07:17:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.215+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734406c350 con 0x7f7344063720 2023-05-16T07:17:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.215+0000 7f733affd700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 0x7f73240612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.215+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f732c07b0c0 con 0x7f7344063720 2023-05-16T07:17:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.215+0000 7f734259c700 1 -- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 msgr2=0x7f73240612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.215+0000 7f734259c700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 0x7f73240612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.219+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f732c04c260 con 0x7f7344063720 2023-05-16T07:17:06.312 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:06.309+0000 7f3e3f92f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:06.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.416+0000 7f734259c700 1 -- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 msgr2=0x7f73240612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.416+0000 7f734259c700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 0x7f73240612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:06.423 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:06.420+0000 7f9f3ba80680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.445+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73440c3fc0 con 0x7f7344063720 2023-05-16T07:17:06.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.445+0000 7f733affd700 1 -- 172.21.15.92:0/2525061577 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f732c028070 con 0x7f7344063720 2023-05-16T07:17:06.449 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:06.449 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:14:54.221623+0000","modified":"2023-05-16T07:14:56.230335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/2827835616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2827835616},{"type":"v1","addr":"172.21.15.158:6837","nonce":2827835616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-05-16T07:17:06.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 msgr2=0x7f73240612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:06.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 0x7f73240612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 msgr2=0x7f73440c0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 0x7f73440c0be0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f732c009e00 tx=0x7f732c017000).stop 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 shutdown_connections 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f732405ee20 0x7f73240612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7344154040 0x7f73440c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7344063720 0x7f73440c0be0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 --2- 172.21.15.92:0/2525061577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73440bf4f0 0x7f73440c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 >> 172.21.15.92:0/2525061577 conn(0x7f73440c8aa0 msgr2=0x7f73440c47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 shutdown_connections 2023-05-16T07:17:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.447+0000 7f7349074700 1 -- 172.21.15.92:0/2525061577 wait complete. 2023-05-16T07:17:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-05-16T07:17:06.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:06.497 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:06.494+0000 7f3e3f92f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:06.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.577+0000 7fb666f84700 1 Processor -- start 2023-05-16T07:17:06.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.577+0000 7fb666f84700 1 -- start start 2023-05-16T07:17:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb666f84700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb660154170 0x7fb6600c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb666f84700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb660154550 0x7fb6600c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb666f84700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb660069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb666f84700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6600c0660 con 0x7fb660154550 2023-05-16T07:17:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb666f84700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66006a020 con 0x7fb660154170 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb666f84700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600bfd30 con 0x7fb6600696a0 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb660154550 0x7fb6600c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb660154550 0x7fb6600c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44838/0 (socket says 172.21.15.92:44838) 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 -- 172.21.15.92:0/45944471 learned_addr learned my addr 172.21.15.92:0/45944471 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb664d20700 1 --2- 172.21.15.92:0/45944471 >> v2:172.21.15.158:3300/0 conn(0x7fb660154170 0x7fb6600c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb665521700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb660069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 -- 172.21.15.92:0/45944471 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 msgr2=0x7fb660069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb660069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 -- 172.21.15.92:0/45944471 >> v2:172.21.15.158:3300/0 conn(0x7fb660154170 msgr2=0x7fb6600c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 --2- 172.21.15.92:0/45944471 >> v2:172.21.15.158:3300/0 conn(0x7fb660154170 0x7fb6600c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 -- 172.21.15.92:0/45944471 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600c0010 con 0x7fb660154550 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb665521700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb660069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb664d20700 1 --2- 172.21.15.92:0/45944471 >> v2:172.21.15.158:3300/0 conn(0x7fb660154170 0x7fb6600c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.578+0000 7fb65ffff700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154550 0x7fb6600c6160 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb660083030 tx=0x7fb64c0090f0).ready entity=mon.0 client_cookie=4a757ea125c45083 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb65f7fe700 1 -- 172.21.15.92:0/45944471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00b600 con 0x7fb660154550 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb65f7fe700 1 -- 172.21.15.92:0/45944471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c00b760 con 0x7fb660154550 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb65f7fe700 1 -- 172.21.15.92:0/45944471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00bae0 con 0x7fb660154550 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 -- 172.21.15.92:0/45944471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154550 msgr2=0x7fb6600c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154550 0x7fb6600c6160 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb660083030 tx=0x7fb64c0090f0).stop 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 -- 172.21.15.92:0/45944471 shutdown_connections 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb660069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 --2- 172.21.15.92:0/45944471 >> v2:172.21.15.158:3300/0 conn(0x7fb660154170 0x7fb6600c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 --2- 172.21.15.92:0/45944471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154550 0x7fb6600c6160 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 -- 172.21.15.92:0/45944471 >> 172.21.15.92:0/45944471 conn(0x7fb6600cb750 msgr2=0x7fb6600c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 -- 172.21.15.92:0/45944471 shutdown_connections 2023-05-16T07:17:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.579+0000 7fb666f84700 1 -- 172.21.15.92:0/45944471 wait complete. 2023-05-16T07:17:06.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.580+0000 7fb666f84700 1 Processor -- start 2023-05-16T07:17:06.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 -- start start 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb6600aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 0x7fb6600c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb660154550 0x7fb6601510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600c4fa0 con 0x7fb660154170 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600c2c50 con 0x7fb660154550 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb666f84700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600af1e0 con 0x7fb6600696a0 2023-05-16T07:17:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb65ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 0x7fb6600c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb65ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 0x7fb6600c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44844/0 (socket says 172.21.15.92:44844) 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb65ffff700 1 -- 172.21.15.92:0/2222664367 learned_addr learned my addr 172.21.15.92:0/2222664367 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb664d20700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb6600aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.581+0000 7fb665521700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb660154550 0x7fb6601510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65ffff700 1 -- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 msgr2=0x7fb6600aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65ffff700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb6600aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65ffff700 1 -- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb660154550 msgr2=0x7fb6601510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65ffff700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb660154550 0x7fb6601510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65ffff700 1 -- 172.21.15.92:0/2222664367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c009370 con 0x7fb660154170 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65ffff700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 0x7fb6600c4ab0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb64c011000 tx=0x7fb64c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00bea0 con 0x7fb660154170 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c00b8d0 con 0x7fb660154170 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01c670 con 0x7fb660154170 2023-05-16T07:17:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6600af4c0 con 0x7fb660154170 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.582+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6600afa80 con 0x7fb660154170 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.583+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fb64c00f070 con 0x7fb660154170 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.583+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6600666c0 con 0x7fb660154170 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.584+0000 7fb65cff9700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb64405ee20 0x7fb6440612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.584+0000 7fb664d20700 1 -- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb64405ee20 msgr2=0x7fb6440612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.584+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7fb64c07b440 con 0x7fb660154170 2023-05-16T07:17:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.584+0000 7fb664d20700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb64405ee20 0x7fb6440612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.588+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb64c04c6f0 con 0x7fb660154170 2023-05-16T07:17:06.612 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:06.608+0000 7f9f3ba80680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.756+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb66006c350 con 0x7fb660154170 2023-05-16T07:17:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.757+0000 7fb65cff9700 1 -- 172.21.15.92:0/2222664367 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fb64c020020 con 0x7fb660154170 2023-05-16T07:17:06.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:06.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:04.899733+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:14:52.750879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-05-16T07:14:53.182846+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]},{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/1803957237":"2023-05-17T07:15:05.377224+0000","172.21.15.158:0/1247324037":"2023-05-17T07:15:05.377224+0000","172.21.15.158:6833/86301":"2023-05-17T07:15:05.377224+0000","172.21.15.158:6832/86301":"2023-05-17T07:15:05.377224+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb64405ee20 msgr2=0x7fb6440612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb64405ee20 0x7fb6440612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 msgr2=0x7fb6600c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 0x7fb6600c4ab0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb64c011000 tx=0x7fb64c017000).stop 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 shutdown_connections 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb64405ee20 0x7fb6440612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb6600696a0 0x7fb6600aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb660154550 0x7fb6601510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 --2- 172.21.15.92:0/2222664367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb660154170 0x7fb6600c4ab0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 >> 172.21.15.92:0/2222664367 conn(0x7fb6600cb750 msgr2=0x7fb6600ca160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 shutdown_connections 2023-05-16T07:17:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.759+0000 7fb666f84700 1 -- 172.21.15.92:0/2222664367 wait complete. 2023-05-16T07:17:06.775 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:17:06.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.886+0000 7f678af05700 1 Processor -- start 2023-05-16T07:17:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.886+0000 7f678af05700 1 -- start start 2023-05-16T07:17:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f678af05700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f678af05700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6784065cd0 0x7f67840c2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f678af05700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6784066130 con 0x7f6784150a40 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f678af05700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6784065350 con 0x7f6784065cd0 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f678af05700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6784151240 con 0x7f6784150e20 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789f03700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6784150a40 0x7f67840a7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55598/0 (socket says 172.21.15.92:55598) 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789f03700 1 -- 172.21.15.92:0/4224631955 learned_addr learned my addr 172.21.15.92:0/4224631955 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f678a704700 1 --2- 172.21.15.92:0/4224631955 >> v2:172.21.15.158:3300/0 conn(0x7f6784065cd0 0x7f67840c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789702700 1 --2- 172.21.15.92:0/4224631955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6788f01700 1 -- 172.21.15.92:0/4224631955 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223569222 0 0) 0x7f6780001110 con 0x7f6784150a40 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6788f01700 1 -- 172.21.15.92:0/4224631955 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67700033f0 con 0x7f6784150a40 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789702700 1 -- 172.21.15.92:0/4224631955 >> v2:172.21.15.158:3300/0 conn(0x7f6784065cd0 msgr2=0x7f67840c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789702700 1 --2- 172.21.15.92:0/4224631955 >> v2:172.21.15.158:3300/0 conn(0x7f6784065cd0 0x7f67840c2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789702700 1 -- 172.21.15.92:0/4224631955 >> v1:172.21.15.92:6789/0 conn(0x7f6784150a40 legacy=0x7f67840a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789702700 1 -- 172.21.15.92:0/4224631955 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840c2e00 con 0x7f6784150e20 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.887+0000 7f6789702700 1 --2- 172.21.15.92:0/4224631955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840657e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f6774009760 tx=0x7f6774009060).ready entity=mon.2 client_cookie=4d491e0b4183a60d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f6788f01700 1 -- 172.21.15.92:0/4224631955 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400d4a0 con 0x7f6784150e20 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f6788f01700 1 -- 172.21.15.92:0/4224631955 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677400d600 con 0x7f6784150e20 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f6788f01700 1 -- 172.21.15.92:0/4224631955 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400d980 con 0x7f6784150e20 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 -- 172.21.15.92:0/4224631955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 msgr2=0x7f67840657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 --2- 172.21.15.92:0/4224631955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840657e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f6774009760 tx=0x7f6774009060).stop 2023-05-16T07:17:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 -- 172.21.15.92:0/4224631955 shutdown_connections 2023-05-16T07:17:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 --2- 172.21.15.92:0/4224631955 >> v2:172.21.15.158:3300/0 conn(0x7f6784065cd0 0x7f67840c2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 --2- 172.21.15.92:0/4224631955 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840657e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 -- 172.21.15.92:0/4224631955 >> 172.21.15.92:0/4224631955 conn(0x7f6784067f60 msgr2=0x7f6784068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 -- 172.21.15.92:0/4224631955 shutdown_connections 2023-05-16T07:17:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.888+0000 7f678af05700 1 -- 172.21.15.92:0/4224631955 wait complete. 2023-05-16T07:17:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.889+0000 7f678af05700 1 Processor -- start 2023-05-16T07:17:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.889+0000 7f678af05700 1 -- start start 2023-05-16T07:17:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678af05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 0x7f6784066e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678af05700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784150a40 0x7f6784067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678af05700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678af05700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6784067830 con 0x7f6784065cd0 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678af05700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840a5f70 con 0x7f6784150a40 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678af05700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67840c8340 con 0x7f6784150e20 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f6789f03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 0x7f6784066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f6789f03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 0x7f6784066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44856/0 (socket says 172.21.15.92:44856) 2023-05-16T07:17:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f6789f03700 1 -- 172.21.15.92:0/2628457418 learned_addr learned my addr 172.21.15.92:0/2628457418 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f6789702700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784150a40 0x7f6784067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.890+0000 7f678a704700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6789f03700 1 -- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 msgr2=0x7f67840a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6789f03700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6789f03700 1 -- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784150a40 msgr2=0x7f6784067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6789f03700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784150a40 0x7f6784067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6789f03700 1 -- 172.21.15.92:0/2628457418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740092a0 con 0x7f6784065cd0 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6789f03700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 0x7f6784066e50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f678000c470 tx=0x7f6780007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800070d0 con 0x7f6784065cd0 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67840a79d0 con 0x7f6784065cd0 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780007230 con 0x7f6784065cd0 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800075f0 con 0x7f6784065cd0 2023-05-16T07:17:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.891+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67840a7f90 con 0x7f6784065cd0 2023-05-16T07:17:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.892+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67840a85e0 con 0x7f6784065cd0 2023-05-16T07:17:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.892+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f6780010070 con 0x7f6784065cd0 2023-05-16T07:17:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.892+0000 7f6779ffb700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 0x7f67680612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.893+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f67800791a0 con 0x7f6784065cd0 2023-05-16T07:17:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.893+0000 7f6789702700 1 -- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 msgr2=0x7f67680612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.893+0000 7f6789702700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 0x7f67680612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:06.896+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f678004a3a0 con 0x7f6784065cd0 2023-05-16T07:17:06.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:17:06.989 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:17:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.094+0000 7f6789702700 1 -- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 msgr2=0x7f67680612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.094+0000 7f6789702700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 0x7f67680612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.112+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6784066130 con 0x7f6784065cd0 2023-05-16T07:17:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.112+0000 7f6779ffb700 1 -- 172.21.15.92:0/2628457418 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f6784066130 con 0x7f6784065cd0 2023-05-16T07:17:07.116 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:07.116 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:14:54.221623+0000","modified":"2023-05-16T07:14:56.230335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/2827835616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2827835616},{"type":"v1","addr":"172.21.15.158:6837","nonce":2827835616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-05-16T07:17:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 msgr2=0x7f67680612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 0x7f67680612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 msgr2=0x7f6784066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 0x7f6784066e50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f678000c470 tx=0x7f6780007ab0).stop 2023-05-16T07:17:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 shutdown_connections 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f676805ee20 0x7f67680612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784150e20 0x7f67840a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784065cd0 0x7f6784066e50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 --2- 172.21.15.92:0/2628457418 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784150a40 0x7f6784067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 >> 172.21.15.92:0/2628457418 conn(0x7f6784067f60 msgr2=0x7f67841513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 shutdown_connections 2023-05-16T07:17:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.114+0000 7f678af05700 1 -- 172.21.15.92:0/2628457418 wait complete. 2023-05-16T07:17:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-05-16T07:17:07.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:07.142 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.139+0000 7f3e3f92f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:07.209 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.206+0000 7f3e3f92f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.243+0000 7f1984b04700 1 Processor -- start 2023-05-16T07:17:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.244+0000 7f1984b04700 1 -- start start 2023-05-16T07:17:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.244+0000 7f1984b04700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19800663a0 0x7f1980151de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.244+0000 7f1984b04700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f19800bc760 0x7f1980155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.244+0000 7f1984b04700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19800a6ce0 con 0x7f1980151a00 2023-05-16T07:17:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.244+0000 7f1984b04700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19800c4b20 con 0x7f19800663a0 2023-05-16T07:17:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.244+0000 7f1984b04700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1980066a10 con 0x7f19800bc760 2023-05-16T07:17:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19800663a0 0x7f1980151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19800663a0 0x7f1980151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44892/0 (socket says 172.21.15.92:44892) 2023-05-16T07:17:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 -- 172.21.15.92:0/1777600274 learned_addr learned my addr 172.21.15.92:0/1777600274 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197ed9d700 1 --2- 172.21.15.92:0/1777600274 >> v2:172.21.15.158:3300/0 conn(0x7f19800bc760 0x7f1980155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 -- 172.21.15.92:0/1777600274 >> v2:172.21.15.158:3300/0 conn(0x7f19800bc760 msgr2=0x7f1980155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 --2- 172.21.15.92:0/1777600274 >> v2:172.21.15.158:3300/0 conn(0x7f19800bc760 0x7f1980155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 -- 172.21.15.92:0/1777600274 >> v1:172.21.15.92:6789/0 conn(0x7f1980151a00 legacy=0x7f19800bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 -- 172.21.15.92:0/1777600274 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19800c4560 con 0x7f19800663a0 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197dd9b700 1 --2- 172.21.15.92:0/1777600274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f1980151de0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f198005eca0 tx=0x7f19700090f0).ready entity=mon.0 client_cookie=5cebdc5c8df2afcc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197d59a700 1 -- 172.21.15.92:0/1777600274 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000b520 con 0x7f19800663a0 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197d59a700 1 -- 172.21.15.92:0/1777600274 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000b680 con 0x7f19800663a0 2023-05-16T07:17:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.245+0000 7f197d59a700 1 -- 172.21.15.92:0/1777600274 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000ba20 con 0x7f19800663a0 2023-05-16T07:17:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 -- 172.21.15.92:0/1777600274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 msgr2=0x7f1980151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 --2- 172.21.15.92:0/1777600274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f1980151de0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f198005eca0 tx=0x7f19700090f0).stop 2023-05-16T07:17:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 -- 172.21.15.92:0/1777600274 shutdown_connections 2023-05-16T07:17:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 --2- 172.21.15.92:0/1777600274 >> v2:172.21.15.158:3300/0 conn(0x7f19800bc760 0x7f1980155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 --2- 172.21.15.92:0/1777600274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f1980151de0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 -- 172.21.15.92:0/1777600274 >> 172.21.15.92:0/1777600274 conn(0x7f19800c7bc0 msgr2=0x7f19800c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 -- 172.21.15.92:0/1777600274 shutdown_connections 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.246+0000 7f1984b04700 1 -- 172.21.15.92:0/1777600274 wait complete. 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.247+0000 7f1984b04700 1 Processor -- start 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.247+0000 7f1984b04700 1 -- start start 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f1984b04700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f198014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f1984b04700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19800bc760 0x7f19800aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f1984b04700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1980151a00 0x7f19800a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f1984b04700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800a0470 con 0x7f19800663a0 2023-05-16T07:17:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f1984b04700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800a0710 con 0x7f19800bc760 2023-05-16T07:17:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f1984b04700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800a09f0 con 0x7f1980151a00 2023-05-16T07:17:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f198014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f198014f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44896/0 (socket says 172.21.15.92:44896) 2023-05-16T07:17:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 -- 172.21.15.92:0/2606392586 learned_addr learned my addr 172.21.15.92:0/2606392586 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197dd9b700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19800bc760 0x7f19800aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197ed9d700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1980151a00 0x7f19800a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 -- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1980151a00 msgr2=0x7f19800a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1980151a00 0x7f19800a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 -- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19800bc760 msgr2=0x7f19800aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19800bc760 0x7f19800aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 -- 172.21.15.92:0/2606392586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970009370 con 0x7f19800663a0 2023-05-16T07:17:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.248+0000 7f197e59c700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f198014f9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f197400c440 tx=0x7f1974007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.249+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740123f0 con 0x7f19800663a0 2023-05-16T07:17:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.249+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974001e90 con 0x7f19800663a0 2023-05-16T07:17:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.249+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19800a0cd0 con 0x7f19800663a0 2023-05-16T07:17:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.249+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974012780 con 0x7f19800663a0 2023-05-16T07:17:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.249+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800a1290 con 0x7f19800663a0 2023-05-16T07:17:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.249+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198006c350 con 0x7f19800663a0 2023-05-16T07:17:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.250+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f1974010070 con 0x7f19800663a0 2023-05-16T07:17:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.250+0000 7f196e7fc700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 0x7f19600612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.250+0000 7f197dd9b700 1 -- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 msgr2=0x7f19600612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.250+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f1974079010 con 0x7f19800663a0 2023-05-16T07:17:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.250+0000 7f197dd9b700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 0x7f19600612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.254+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f19740478f0 con 0x7f19800663a0 2023-05-16T07:17:07.279 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.275+0000 7f9f3ba80680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:07.283 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.280+0000 7f3e3f92f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:07.347 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.343+0000 7f9f3ba80680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:07.422 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.419+0000 7f9f3ba80680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:07.425 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.422+0000 7f3e3f92f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.451+0000 7f197dd9b700 1 -- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 msgr2=0x7f19600612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.451+0000 7f197dd9b700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 0x7f19600612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:07.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.463+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19800c42a0 con 0x7f19800663a0 2023-05-16T07:17:07.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.463+0000 7f196e7fc700 1 -- 172.21.15.92:0/2606392586 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f197404a210 con 0x7f19800663a0 2023-05-16T07:17:07.468 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:07.468 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:14:54.221623+0000","modified":"2023-05-16T07:14:56.230335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/2827835616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2827835616},{"type":"v1","addr":"172.21.15.158:6837","nonce":2827835616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-05-16T07:17:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 msgr2=0x7f19600612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:07.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 0x7f19600612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:07.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 msgr2=0x7f198014f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f198014f9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f197400c440 tx=0x7f1974007db0).stop 2023-05-16T07:17:07.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 shutdown_connections 2023-05-16T07:17:07.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f196005ee20 0x7f19600612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:07.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1980151a00 0x7f19800a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19800663a0 0x7f198014f9c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 --2- 172.21.15.92:0/2606392586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19800bc760 0x7f19800aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.465+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 >> 172.21.15.92:0/2606392586 conn(0x7f19800c7bc0 msgr2=0x7f19800a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.466+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 shutdown_connections 2023-05-16T07:17:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.466+0000 7f1984b04700 1 -- 172.21.15.92:0/2606392586 wait complete. 2023-05-16T07:17:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2023-05-16T07:17:07.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:07.494 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.491+0000 7f3e3f92f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:07.573 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.569+0000 7f9f3ba80680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:07.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.592+0000 7f5aa2b1c700 1 Processor -- start 2023-05-16T07:17:07.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.592+0000 7f5aa2b1c700 1 -- start start 2023-05-16T07:17:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa2b1c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c138840 0x7f5a9c066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa2b1c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a9c0be4e0 0x7f5a9c065000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa2b1c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5a9c0654f0 0x7f5a9c0622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa2b1c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c0659a0 con 0x7f5a9c0be4e0 2023-05-16T07:17:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa2b1c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c0bda10 con 0x7f5a9c0654f0 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa2b1c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c068be0 con 0x7f5a9c138840 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a9c0be4e0 0x7f5a9c065000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a9c0be4e0 0x7f5a9c065000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44904/0 (socket says 172.21.15.92:44904) 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 -- 172.21.15.92:0/2474386457 learned_addr learned my addr 172.21.15.92:0/2474386457 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa10b9700 1 --2- 172.21.15.92:0/2474386457 >> v2:172.21.15.158:3300/0 conn(0x7f5a9c0654f0 0x7f5a9c0622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5aa08b8700 1 --2- 172.21.15.92:0/2474386457 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c138840 0x7f5a9c066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 -- 172.21.15.92:0/2474386457 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c138840 msgr2=0x7f5a9c066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 --2- 172.21.15.92:0/2474386457 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c138840 0x7f5a9c066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 -- 172.21.15.92:0/2474386457 >> v2:172.21.15.158:3300/0 conn(0x7f5a9c0654f0 msgr2=0x7f5a9c0622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.593+0000 7f5a9bfff700 1 --2- 172.21.15.92:0/2474386457 >> v2:172.21.15.158:3300/0 conn(0x7f5a9c0654f0 0x7f5a9c0622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5a9bfff700 1 -- 172.21.15.92:0/2474386457 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c068ec0 con 0x7f5a9c0be4e0 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5a9bfff700 1 --2- 172.21.15.92:0/2474386457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c0be4e0 0x7f5a9c065000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f5a9c064f90 tx=0x7f5a880090f0).ready entity=mon.0 client_cookie=fbcd806ef223c182 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/2474386457 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800f510 con 0x7f5a9c0be4e0 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/2474386457 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8800f670 con 0x7f5a9c0be4e0 2023-05-16T07:17:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/2474386457 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800f9f0 con 0x7f5a9c0be4e0 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/2474386457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c0be4e0 msgr2=0x7f5a9c065000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/2474386457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c0be4e0 0x7f5a9c065000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f5a9c064f90 tx=0x7f5a880090f0).stop 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/2474386457 shutdown_connections 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/2474386457 >> v2:172.21.15.158:3300/0 conn(0x7f5a9c0654f0 0x7f5a9c0622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/2474386457 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c138840 0x7f5a9c066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/2474386457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c0be4e0 0x7f5a9c065000 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/2474386457 >> 172.21.15.92:0/2474386457 conn(0x7f5a9c0c4430 msgr2=0x7f5a9c0c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/2474386457 shutdown_connections 2023-05-16T07:17:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.594+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/2474386457 wait complete. 2023-05-16T07:17:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.595+0000 7f5aa2b1c700 1 Processor -- start 2023-05-16T07:17:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 -- start start 2023-05-16T07:17:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 0x7f5a9c0aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c0be4e0 0x7f5a9c14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 0x7f5a9c0c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c2a00 con 0x7f5a9c138840 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c154680 con 0x7f5a9c0654f0 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.596+0000 7f5aa2b1c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c20d0 con 0x7f5a9c0be4e0 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 0x7f5a9c0c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5a9bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c0be4e0 0x7f5a9c14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa08b8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 0x7f5a9c0aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 0x7f5a9c0c0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44914/0 (socket says 172.21.15.92:44914) 2023-05-16T07:17:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa08b8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 0x7f5a9c0aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:48546/0 (socket says 172.21.15.92:48546) 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 -- 172.21.15.92:0/1165825492 learned_addr learned my addr 172.21.15.92:0/1165825492 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5a9bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c0be4e0 0x7f5a9c14f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:37484/0 (socket says 172.21.15.92:37484) 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 -- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c0be4e0 msgr2=0x7f5a9c14f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c0be4e0 0x7f5a9c14f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 -- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 msgr2=0x7f5a9c0aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 0x7f5a9c0aa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 -- 172.21.15.92:0/1165825492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009370 con 0x7f5a9c138840 2023-05-16T07:17:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa08b8700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 0x7f5a9c0aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa10b9700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 0x7f5a9c0c0e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f5a94009df0 tx=0x7f5a9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400caa0 con 0x7f5a9c138840 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9400f040 con 0x7f5a9c138840 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c0c23b0 con 0x7f5a9c138840 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400ce10 con 0x7f5a9c138840 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.597+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c0aaeb0 con 0x7f5a9c138840 2023-05-16T07:17:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.598+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c0c2b70 con 0x7f5a9c138840 2023-05-16T07:17:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.599+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f5a94013610 con 0x7f5a9c138840 2023-05-16T07:17:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.599+0000 7f5a98ff9700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5a8005ee20 0x7f5a800612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.599+0000 7f5aa08b8700 1 -- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5a8005ee20 msgr2=0x7f5a800612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.599+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f5a9407a7a0 con 0x7f5a9c138840 2023-05-16T07:17:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.599+0000 7f5aa08b8700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5a8005ee20 0x7f5a800612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:07.609 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.604+0000 7f3e3f92f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.603+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5a9404b9a0 con 0x7f5a9c138840 2023-05-16T07:17:07.646 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.643+0000 7f9f3ba80680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:07.713 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:07.710+0000 7f3e3f92f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:07.761 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.757+0000 7f9f3ba80680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:07.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.773+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a9c06c350 con 0x7f5a9c138840 2023-05-16T07:17:07.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.774+0000 7f5a98ff9700 1 -- 172.21.15.92:0/1165825492 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f5a9404efc0 con 0x7f5a9c138840 2023-05-16T07:17:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:07.779 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:04.899733+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:14:52.750879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-05-16T07:14:53.182846+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]},{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/1803957237":"2023-05-17T07:15:05.377224+0000","172.21.15.158:0/1247324037":"2023-05-17T07:15:05.377224+0000","172.21.15.158:6833/86301":"2023-05-17T07:15:05.377224+0000","172.21.15.158:6832/86301":"2023-05-17T07:15:05.377224+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:07.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5a8005ee20 msgr2=0x7f5a800612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5a8005ee20 0x7f5a800612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 msgr2=0x7f5a9c0c0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 0x7f5a9c0c0e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f5a94009df0 tx=0x7f5a9400b040).stop 2023-05-16T07:17:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 shutdown_connections 2023-05-16T07:17:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5a8005ee20 0x7f5a800612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c138840 0x7f5a9c0c0e20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a9c0654f0 0x7f5a9c0aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 --2- 172.21.15.92:0/1165825492 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a9c0be4e0 0x7f5a9c14f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 >> 172.21.15.92:0/1165825492 conn(0x7f5a9c0c4430 msgr2=0x7f5a9c135ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 shutdown_connections 2023-05-16T07:17:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.777+0000 7f5aa2b1c700 1 -- 172.21.15.92:0/1165825492 wait complete. 2023-05-16T07:17:07.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:17:07.873 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:07.869+0000 7f9f3ba80680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.905+0000 7f25e756c700 1 Processor -- start 2023-05-16T07:17:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.905+0000 7f25e756c700 1 -- start start 2023-05-16T07:17:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e756c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25e0150ac0 0x7f25e00c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e756c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e00bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e756c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e00c2240 con 0x7f25e01547b0 2023-05-16T07:17:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e756c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e00c31a0 con 0x7f25e0150ac0 2023-05-16T07:17:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e756c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01550c0 con 0x7f25e0154c30 2023-05-16T07:17:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e5308700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25e0150ac0 0x7f25e00c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e5308700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25e0150ac0 0x7f25e00c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44918/0 (socket says 172.21.15.92:44918) 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e5308700 1 -- 172.21.15.92:0/849494348 learned_addr learned my addr 172.21.15.92:0/849494348 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.906+0000 7f25e5b09700 1 --2- 172.21.15.92:0/849494348 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e00bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25e5308700 1 -- 172.21.15.92:0/849494348 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 msgr2=0x7f25e00bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25e5308700 1 --2- 172.21.15.92:0/849494348 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e00bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25e5308700 1 -- 172.21.15.92:0/849494348 >> v1:172.21.15.158:6789/0 conn(0x7f25e01547b0 legacy=0x7f25e00bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25e5308700 1 -- 172.21.15.92:0/849494348 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00c4680 con 0x7f25e0150ac0 2023-05-16T07:17:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25e5308700 1 --2- 172.21.15.92:0/849494348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e0150ac0 0x7f25e00c2cb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f25d0006fd0 tx=0x7f25d00090f0).ready entity=mon.0 client_cookie=1e598553151aa56c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25dffff700 1 -- 172.21.15.92:0/849494348 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000b580 con 0x7f25e0150ac0 2023-05-16T07:17:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25dffff700 1 -- 172.21.15.92:0/849494348 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d000b6e0 con 0x7f25e0150ac0 2023-05-16T07:17:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.907+0000 7f25dffff700 1 -- 172.21.15.92:0/849494348 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000ba60 con 0x7f25e0150ac0 2023-05-16T07:17:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 -- 172.21.15.92:0/849494348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e0150ac0 msgr2=0x7f25e00c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 --2- 172.21.15.92:0/849494348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e0150ac0 0x7f25e00c2cb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f25d0006fd0 tx=0x7f25d00090f0).stop 2023-05-16T07:17:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 -- 172.21.15.92:0/849494348 shutdown_connections 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 --2- 172.21.15.92:0/849494348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e0150ac0 0x7f25e00c2cb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 --2- 172.21.15.92:0/849494348 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e00bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 -- 172.21.15.92:0/849494348 >> 172.21.15.92:0/849494348 conn(0x7f25e00c7bc0 msgr2=0x7f25e00c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 -- 172.21.15.92:0/849494348 shutdown_connections 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.908+0000 7f25e756c700 1 -- 172.21.15.92:0/849494348 wait complete. 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.909+0000 7f25e756c700 1 Processor -- start 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.909+0000 7f25e756c700 1 -- start start 2023-05-16T07:17:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e756c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25e0150ac0 0x7f25e00acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e756c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 0x7f25e0142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e756c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e0160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e756c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00ab3f0 con 0x7f25e01547b0 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e756c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00ab0f0 con 0x7f25e0150ac0 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e756c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01431a0 con 0x7f25e0154c30 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 0x7f25e0142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 0x7f25e0142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44926/0 (socket says 172.21.15.92:44926) 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 -- 172.21.15.92:0/4176695008 learned_addr learned my addr 172.21.15.92:0/4176695008 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e5308700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25e0150ac0 0x7f25e00acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e5b09700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e0160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 msgr2=0x7f25e0160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e0160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25e0150ac0 msgr2=0x7f25e00acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25e0150ac0 0x7f25e00acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.910+0000 7f25e4b07700 1 -- 172.21.15.92:0/4176695008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0009370 con 0x7f25e01547b0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.911+0000 7f25e4b07700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 0x7f25e0142cb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f25d400c440 tx=0x7f25d4007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.911+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4012400 con 0x7f25e01547b0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.911+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4012560 con 0x7f25e01547b0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.911+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e01617c0 con 0x7f25e01547b0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.911+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4012900 con 0x7f25e01547b0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.911+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e0161d80 con 0x7f25e01547b0 2023-05-16T07:17:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.912+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e006c350 con 0x7f25e01547b0 2023-05-16T07:17:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.912+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f25d4019030 con 0x7f25e01547b0 2023-05-16T07:17:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.912+0000 7f25dd7fa700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 0x7f25cc0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.912+0000 7f25e5308700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 msgr2=0x7f25cc0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:07.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.912+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f25d40790b0 con 0x7f25e01547b0 2023-05-16T07:17:07.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.912+0000 7f25e5308700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 0x7f25cc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:07.920+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f25d40488f0 con 0x7f25e01547b0 2023-05-16T07:17:08.003 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.000+0000 7f3e3f92f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:08.072 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.068+0000 7f3e3f92f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.113+0000 7f25e5308700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 msgr2=0x7f25cc0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.113+0000 7f25e5308700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 0x7f25cc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:08.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.129+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f25e00c24b0 con 0x7f25e01547b0 2023-05-16T07:17:08.168 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:08.164+0000 7f9f3ba80680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:08.236 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:08.232+0000 7f9f3ba80680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:08.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.514+0000 7f25e5308700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 msgr2=0x7f25cc0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:08.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.514+0000 7f25e5308700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 0x7f25cc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:17:08.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.913+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 22) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f25d4010070 con 0x7f25e01547b0 2023-05-16T07:17:08.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.913+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 msgr2=0x7f25cc0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:08.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:08.913+0000 7f25dd7fa700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 0x7f25cc0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:08.965 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dbf113700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.965 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dbf113700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.965 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dbf113700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.965 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dbf113700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.965 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dbf113700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.966 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dc291a700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.966 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dc291a700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.966 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dc291a700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.966 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dc291a700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.966 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:08.961+0000 7f3dc291a700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:08.967 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:17:08.964+0000 7f0ffde57700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:08.967 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:17:08.964+0000 7f0ffde57700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:08.968 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:17:08.964+0000 7f0ffde57700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:17:08.981 INFO:tasks.ceph.mds.b.smithi158.stderr: -4> 2023-05-16T07:17:08.964+0000 7f0ffde57700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:08.982 INFO:tasks.ceph.mds.b.smithi158.stderr: -3> 2023-05-16T07:17:08.964+0000 7f0ffde57700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:08.982 INFO:tasks.ceph.mds.b.smithi158.stderr: -1> 2023-05-16T07:17:08.964+0000 7f0ffde57700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:17:08.999 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:17:09.087 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:17:09.084+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:17:09.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.086+0000 7f25dd7fa700 1 -- 172.21.15.92:0/4176695008 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f25d4072d60 con 0x7f25e01547b0 2023-05-16T07:17:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.088+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 msgr2=0x7f25e0142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.088+0000 7f25e756c700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 0x7f25e0142cb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f25d400c440 tx=0x7f25d4007dc0).stop 2023-05-16T07:17:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 shutdown_connections 2023-05-16T07:17:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25cc05ee20 0x7f25cc0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25e0154c30 0x7f25e0160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25e0150ac0 0x7f25e00acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 --2- 172.21.15.92:0/4176695008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25e01547b0 0x7f25e0142cb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 >> 172.21.15.92:0/4176695008 conn(0x7f25e00c7bc0 msgr2=0x7f25e014ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 shutdown_connections 2023-05-16T07:17:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.089+0000 7f25e756c700 1 -- 172.21.15.92:0/4176695008 wait complete. 2023-05-16T07:17:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:17:09.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:17:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.214+0000 7f63d18ed700 1 Processor -- start 2023-05-16T07:17:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.214+0000 7f63d18ed700 1 -- start start 2023-05-16T07:17:09.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63d18ed700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154090 0x7f63cc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:09.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63d18ed700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63cc068160 0x7f63cc068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63d18ed700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc0c94d0 con 0x7f63cc154470 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63d18ed700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0cb7e0 con 0x7f63cc068160 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63d18ed700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc068a70 con 0x7f63cc154090 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63cb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63cc068160 0x7f63cc068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63ca7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f63cc154470 0x7f63cc0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55692/0 (socket says 172.21.15.92:55692) 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.215+0000 7f63ca7fc700 1 -- 172.21.15.92:0/1188060377 learned_addr learned my addr 172.21.15.92:0/1188060377 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:09.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63caffd700 1 --2- 172.21.15.92:0/1188060377 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154090 0x7f63cc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1188060377 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154090 msgr2=0x7f63cc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1188060377 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154090 0x7f63cc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1188060377 >> v1:172.21.15.92:6789/0 conn(0x7f63cc154470 legacy=0x7f63cc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1188060377 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0c88c0 con 0x7f63cc068160 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63c9ffb700 1 -- 172.21.15.92:0/1188060377 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468438990 0 0) 0x7f63cc0c94d0 con 0x7f63cc154470 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1188060377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc068160 0x7f63cc068580 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f63c4009760 tx=0x7f63c4009060).ready entity=mon.0 client_cookie=2a783fafcf59e617 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:09.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63c9ffb700 1 -- 172.21.15.92:0/1188060377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c400b580 con 0x7f63cc068160 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63c9ffb700 1 -- 172.21.15.92:0/1188060377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c400b6e0 con 0x7f63cc068160 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63c9ffb700 1 -- 172.21.15.92:0/1188060377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c400ba60 con 0x7f63cc068160 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63d18ed700 1 -- 172.21.15.92:0/1188060377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc068160 msgr2=0x7f63cc068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63d18ed700 1 --2- 172.21.15.92:0/1188060377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc068160 0x7f63cc068580 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f63c4009760 tx=0x7f63c4009060).stop 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63d18ed700 1 -- 172.21.15.92:0/1188060377 shutdown_connections 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63d18ed700 1 --2- 172.21.15.92:0/1188060377 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154090 0x7f63cc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63d18ed700 1 --2- 172.21.15.92:0/1188060377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc068160 0x7f63cc068580 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.216+0000 7f63d18ed700 1 -- 172.21.15.92:0/1188060377 >> 172.21.15.92:0/1188060377 conn(0x7f63cc067920 msgr2=0x7f63cc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:09.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.217+0000 7f63d18ed700 1 -- 172.21.15.92:0/1188060377 shutdown_connections 2023-05-16T07:17:09.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.217+0000 7f63d18ed700 1 -- 172.21.15.92:0/1188060377 wait complete. 2023-05-16T07:17:09.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.218+0000 7f63d18ed700 1 Processor -- start 2023-05-16T07:17:09.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.218+0000 7f63d18ed700 1 -- start start 2023-05-16T07:17:09.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.218+0000 7f63d18ed700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc068160 0x7f63cc14ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:09.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.218+0000 7f63d18ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 0x7f63cc1665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:09.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.218+0000 7f63d18ed700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154470 0x7f63cc166a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:09.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63d18ed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0c3320 con 0x7f63cc154090 2023-05-16T07:17:09.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63d18ed700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1567a0 con 0x7f63cc068160 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63d18ed700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc156a80 con 0x7f63cc154470 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 0x7f63cc1665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 0x7f63cc1665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44950/0 (socket says 172.21.15.92:44950) 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 -- 172.21.15.92:0/3042137608 learned_addr learned my addr 172.21.15.92:0/3042137608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154470 0x7f63cc166a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63caffd700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc068160 0x7f63cc14ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 -- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154470 msgr2=0x7f63cc166a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:09.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154470 0x7f63cc166a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 -- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc068160 msgr2=0x7f63cc14ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc068160 0x7f63cc14ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 -- 172.21.15.92:0/3042137608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c40092a0 con 0x7f63cc154090 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63ca7fc700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 0x7f63cc1665a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f63b4001eb0 tx=0x7f63b4007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40070d0 con 0x7f63cc154090 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b4007230 con 0x7f63cc154090 2023-05-16T07:17:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc156d60 con 0x7f63cc154090 2023-05-16T07:17:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40075d0 con 0x7f63cc154090 2023-05-16T07:17:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.219+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc14f220 con 0x7f63cc154090 2023-05-16T07:17:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.220+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc0c3490 con 0x7f63cc154090 2023-05-16T07:17:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.221+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 22) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f63b4010070 con 0x7f63cc154090 2023-05-16T07:17:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.221+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7251+0+0 (secure 0 0 0) 0x7f63b4079210 con 0x7f63cc154090 2023-05-16T07:17:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.225+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f63cc0c3490 con 0x7f63cc154090 2023-05-16T07:17:09.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.427+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f63cc0c3490 con 0x7f63cc154090 2023-05-16T07:17:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.916+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 23) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f63b4076d40 con 0x7f63cc154090 2023-05-16T07:17:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.916+0000 7f63c2ffd700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63ac0301b0 0x7f63ac067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:09.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.918+0000 7f63caffd700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63ac0301b0 0x7f63ac067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:09.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:09.920+0000 7f63caffd700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63ac0301b0 0x7f63ac067e90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f63bc009fd0 tx=0x7f63bc009880).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:10.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.088+0000 7f63c2ffd700 1 -- 172.21.15.92:0/3042137608 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f63b4071be0 con 0x7f63cc154090 2023-05-16T07:17:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63ac0301b0 msgr2=0x7f63ac067e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63ac0301b0 0x7f63ac067e90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f63bc009fd0 tx=0x7f63bc009880).stop 2023-05-16T07:17:10.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 msgr2=0x7f63cc1665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 0x7f63cc1665a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f63b4001eb0 tx=0x7f63b4007b90).stop 2023-05-16T07:17:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 shutdown_connections 2023-05-16T07:17:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63ac0301b0 0x7f63ac067e90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc154470 0x7f63cc166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc068160 0x7f63cc14ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 --2- 172.21.15.92:0/3042137608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc154090 0x7f63cc1665a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 >> 172.21.15.92:0/3042137608 conn(0x7f63cc067920 msgr2=0x7f63cc0abc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 shutdown_connections 2023-05-16T07:17:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.090+0000 7f63d18ed700 1 -- 172.21.15.92:0/3042137608 wait complete. 2023-05-16T07:17:10.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:17:10.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.218+0000 7f6e731db700 1 Processor -- start 2023-05-16T07:17:10.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.218+0000 7f6e731db700 1 -- start start 2023-05-16T07:17:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.219+0000 7f6e731db700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e6c0c1160 0x7f6e6c0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.219+0000 7f6e731db700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c06a140 con 0x7f6e6c0c61f0 2023-05-16T07:17:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.219+0000 7f6e731db700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c06a420 con 0x7f6e6c153f90 2023-05-16T07:17:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.219+0000 7f6e731db700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c069f60 con 0x7f6e6c0c1160 2023-05-16T07:17:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e6c0c1160 0x7f6e6c0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e6c0c1160 0x7f6e6c0c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44976/0 (socket says 172.21.15.92:44976) 2023-05-16T07:17:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 -- 172.21.15.92:0/878187211 learned_addr learned my addr 172.21.15.92:0/878187211 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e729da700 1 --1- 172.21.15.92:0/878187211 >> v1:172.21.15.92:6789/0 conn(0x7f6e6c0c61f0 0x7f6e6c154370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55722/0 (socket says 172.21.15.92:55722) 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 -- 172.21.15.92:0/878187211 >> v1:172.21.15.158:6789/0 conn(0x7f6e6c153f90 legacy=0x7f6e6c0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 -- 172.21.15.92:0/878187211 >> v1:172.21.15.92:6789/0 conn(0x7f6e6c0c61f0 legacy=0x7f6e6c154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 -- 172.21.15.92:0/878187211 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c06a700 con 0x7f6e6c0c1160 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e711d7700 1 -- 172.21.15.92:0/878187211 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592799559 0 0) 0x7f6e6c06a140 con 0x7f6e6c0c61f0 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e719d8700 1 --2- 172.21.15.92:0/878187211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c0c1160 0x7f6e6c0c5d00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f6e60006fd0 tx=0x7f6e600090f0).ready entity=mon.0 client_cookie=58927d5b907f7066 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e711d7700 1 -- 172.21.15.92:0/878187211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000b600 con 0x7f6e6c0c1160 2023-05-16T07:17:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e711d7700 1 -- 172.21.15.92:0/878187211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6000b760 con 0x7f6e6c0c1160 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.220+0000 7f6e711d7700 1 -- 172.21.15.92:0/878187211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000bae0 con 0x7f6e6c0c1160 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 -- 172.21.15.92:0/878187211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c0c1160 msgr2=0x7f6e6c0c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 --2- 172.21.15.92:0/878187211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c0c1160 0x7f6e6c0c5d00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f6e60006fd0 tx=0x7f6e600090f0).stop 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 -- 172.21.15.92:0/878187211 shutdown_connections 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 --2- 172.21.15.92:0/878187211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c0c1160 0x7f6e6c0c5d00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 -- 172.21.15.92:0/878187211 >> 172.21.15.92:0/878187211 conn(0x7f6e6c0cb600 msgr2=0x7f6e6c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:10.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 -- 172.21.15.92:0/878187211 shutdown_connections 2023-05-16T07:17:10.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.221+0000 7f6e731db700 1 -- 172.21.15.92:0/878187211 wait complete. 2023-05-16T07:17:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 Processor -- start 2023-05-16T07:17:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 -- start start 2023-05-16T07:17:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e6c0c1160 0x7f6e6c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e6c0c61f0 0x7f6e6c0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 0x7f6e6c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0c4830 con 0x7f6e6c153f90 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c168510 con 0x7f6e6c0c61f0 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.222+0000 7f6e731db700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0c9280 con 0x7f6e6c0c1160 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 0x7f6e6c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 0x7f6e6c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44980/0 (socket says 172.21.15.92:44980) 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 -- 172.21.15.92:0/8594207 learned_addr learned my addr 172.21.15.92:0/8594207 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e721d9700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e6c0c1160 0x7f6e6c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e719d8700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e6c0c61f0 0x7f6e6c0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 -- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e6c0c1160 msgr2=0x7f6e6c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e6c0c1160 0x7f6e6c0b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 -- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e6c0c61f0 msgr2=0x7f6e6c0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e6c0c61f0 0x7f6e6c0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 -- 172.21.15.92:0/8594207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e60009370 con 0x7f6e6c153f90 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e729da700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 0x7f6e6c0c8d90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f6e6400c4c0 tx=0x7f6e64007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640072e0 con 0x7f6e6c153f90 2023-05-16T07:17:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64007440 con 0x7f6e6c153f90 2023-05-16T07:17:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c163fa0 con 0x7f6e6c153f90 2023-05-16T07:17:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640058f0 con 0x7f6e6c153f90 2023-05-16T07:17:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.223+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c1641f0 con 0x7f6e6c153f90 2023-05-16T07:17:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.224+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c083120 con 0x7f6e6c153f90 2023-05-16T07:17:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.224+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 23) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f6e64010070 con 0x7f6e6c153f90 2023-05-16T07:17:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.225+0000 7f6e6a7fc700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5805ecf0 0x7f6e580611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:10.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.225+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7251+0+0 (secure 0 0 0) 0x7f6e64079470 con 0x7f6e6c153f90 2023-05-16T07:17:10.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.225+0000 7f6e721d9700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5805ecf0 0x7f6e580611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:10.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.225+0000 7f6e721d9700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5805ecf0 0x7f6e580611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6e5c003e00 tx=0x7f6e5c009040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:10.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.229+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6e6404a640 con 0x7f6e6c153f90 2023-05-16T07:17:10.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:10.398+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e6c047150 con 0x7f6e6c153f90 2023-05-16T07:17:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.092+0000 7f6e6a7fc700 1 -- 172.21.15.92:0/8594207 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6e6404dc60 con 0x7f6e6c153f90 2023-05-16T07:17:11.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5805ecf0 msgr2=0x7f6e580611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:11.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5805ecf0 0x7f6e580611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6e5c003e00 tx=0x7f6e5c009040).stop 2023-05-16T07:17:11.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 msgr2=0x7f6e6c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 0x7f6e6c0c8d90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f6e6400c4c0 tx=0x7f6e64007be0).stop 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 shutdown_connections 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5805ecf0 0x7f6e580611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e6c0c1160 0x7f6e6c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e6c153f90 0x7f6e6c0c8d90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 --2- 172.21.15.92:0/8594207 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e6c0c61f0 0x7f6e6c0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 >> 172.21.15.92:0/8594207 conn(0x7f6e6c0cb600 msgr2=0x7f6e6c151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:11.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 shutdown_connections 2023-05-16T07:17:11.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.095+0000 7f6e731db700 1 -- 172.21.15.92:0/8594207 wait complete. 2023-05-16T07:17:11.101 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:17:11.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:17:11.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.228+0000 7fb5f8cb7700 1 Processor -- start 2023-05-16T07:17:11.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.229+0000 7fb5f8cb7700 1 -- start start 2023-05-16T07:17:11.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.229+0000 7fb5f8cb7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f4064f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:11.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.229+0000 7fb5f8cb7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb5f4066e50 0x7fb5f4067250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:11.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.229+0000 7fb5f8cb7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f4065410 con 0x7fb5f41540b0 2023-05-16T07:17:11.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.229+0000 7fb5f8cb7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40a7370 con 0x7fb5f4066e50 2023-05-16T07:17:11.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.229+0000 7fb5f8cb7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40a6760 con 0x7fb5f4153cd0 2023-05-16T07:17:11.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f3fff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb5f4066e50 0x7fb5f4067250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:11.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f37fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f4064f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:11.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f2ffd700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fb5f41540b0 0x7fb5f40a4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:59334/0 (socket says 172.21.15.92:59334) 2023-05-16T07:17:11.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f2ffd700 1 -- 172.21.15.92:0/434063873 learned_addr learned my addr 172.21.15.92:0/434063873 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:11.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f27fc700 1 -- 172.21.15.92:0/434063873 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045318542 0 0) 0x7fb5f4065410 con 0x7fb5f41540b0 2023-05-16T07:17:11.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f27fc700 1 -- 172.21.15.92:0/434063873 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc0033f0 con 0x7fb5f41540b0 2023-05-16T07:17:11.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f37fe700 1 -- 172.21.15.92:0/434063873 >> v2:172.21.15.158:3300/0 conn(0x7fb5f4066e50 msgr2=0x7fb5f4067250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:11.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f37fe700 1 --2- 172.21.15.92:0/434063873 >> v2:172.21.15.158:3300/0 conn(0x7fb5f4066e50 0x7fb5f4067250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f37fe700 1 -- 172.21.15.92:0/434063873 >> v1:172.21.15.158:6789/0 conn(0x7fb5f41540b0 legacy=0x7fb5f40a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f37fe700 1 -- 172.21.15.92:0/434063873 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40a6a40 con 0x7fb5f4153cd0 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/434063873 >> v2:172.21.15.158:3300/0 conn(0x7fb5f4066e50 0x7fb5f4067250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f37fe700 1 --2- 172.21.15.92:0/434063873 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f4064f20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb5e4009e60 tx=0x7fb5e4009780).ready entity=mon.2 client_cookie=71d1f81058a3e344 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f27fc700 1 -- 172.21.15.92:0/434063873 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400e070 con 0x7fb5f4153cd0 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f27fc700 1 -- 172.21.15.92:0/434063873 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e400bab0 con 0x7fb5f4153cd0 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.230+0000 7fb5f27fc700 1 -- 172.21.15.92:0/434063873 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400be30 con 0x7fb5f4153cd0 2023-05-16T07:17:11.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/434063873 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 msgr2=0x7fb5f4064f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/434063873 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f4064f20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb5e4009e60 tx=0x7fb5e4009780).stop 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/434063873 shutdown_connections 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/434063873 >> v2:172.21.15.158:3300/0 conn(0x7fb5f4066e50 0x7fb5f4067250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/434063873 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f4064f20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/434063873 >> 172.21.15.92:0/434063873 conn(0x7fb5f4068a30 msgr2=0x7fb5f40c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/434063873 shutdown_connections 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.231+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/434063873 wait complete. 2023-05-16T07:17:11.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.232+0000 7fb5f8cb7700 1 Processor -- start 2023-05-16T07:17:11.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.232+0000 7fb5f8cb7700 1 -- start start 2023-05-16T07:17:11.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f8cb7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5f4066e50 0x7fb5f4164510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f8cb7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f41492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f8cb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 0x7fb5f40ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f8cb7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40a7930 con 0x7fb5f41540b0 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f8cb7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4164db0 con 0x7fb5f4066e50 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f8cb7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4165090 con 0x7fb5f4153cd0 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 0x7fb5f40ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 0x7fb5f40ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44994/0 (socket says 172.21.15.92:44994) 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 -- 172.21.15.92:0/887130213 learned_addr learned my addr 172.21.15.92:0/887130213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f37fe700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5f4066e50 0x7fb5f4164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f2ffd700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f41492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 -- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 msgr2=0x7fb5f41492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f41492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 -- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5f4066e50 msgr2=0x7fb5f4164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5f4066e50 0x7fb5f4164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 -- 172.21.15.92:0/887130213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0092a0 con 0x7fb5f41540b0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.233+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 0x7fb5f40ab8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fb5ec00e980 tx=0x7fb5ec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.234+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00c610 con 0x7fb5f41540b0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.234+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec00c770 con 0x7fb5f41540b0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.234+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00cb10 con 0x7fb5f41540b0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.234+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e4009a20 con 0x7fb5f41540b0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.234+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f40a75c0 con 0x7fb5f41540b0 2023-05-16T07:17:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.235+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f4047150 con 0x7fb5f41540b0 2023-05-16T07:17:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.235+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb5ec022070 con 0x7fb5f41540b0 2023-05-16T07:17:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.235+0000 7fb5db7fe700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e005ed60 0x7fb5e0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.235+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6755+0+0 (secure 0 0 0) 0x7fb5ec07a180 con 0x7fb5f41540b0 2023-05-16T07:17:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.235+0000 7fb5f37fe700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e005ed60 0x7fb5e0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.236+0000 7fb5f37fe700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e005ed60 0x7fb5e0061220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb5e40097c0 tx=0x7fb5f4157a40).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.239+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb5ec04b4a0 con 0x7fb5f41540b0 2023-05-16T07:17:11.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:11.401+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb5f40c8480 con 0x7fb5f41540b0 2023-05-16T07:17:12.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.103+0000 7fb5db7fe700 1 -- 172.21.15.92:0/887130213 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb5ec016020 con 0x7fb5f41540b0 2023-05-16T07:17:12.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.107+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e005ed60 msgr2=0x7fb5e0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.107+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e005ed60 0x7fb5e0061220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb5e40097c0 tx=0x7fb5f4157a40).stop 2023-05-16T07:17:12.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.107+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 msgr2=0x7fb5f40ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.107+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 0x7fb5f40ab8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fb5ec00e980 tx=0x7fb5ec009060).stop 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.108+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 shutdown_connections 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.108+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb5e005ed60 0x7fb5e0061220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.108+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f41540b0 0x7fb5f40ab8f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.108+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb5f4066e50 0x7fb5f4164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.108+0000 7fb5f8cb7700 1 --2- 172.21.15.92:0/887130213 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb5f4153cd0 0x7fb5f41492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.108+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 >> 172.21.15.92:0/887130213 conn(0x7fb5f4068a30 msgr2=0x7fb5f40c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:12.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.109+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 shutdown_connections 2023-05-16T07:17:12.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.110+0000 7fb5f8cb7700 1 -- 172.21.15.92:0/887130213 wait complete. 2023-05-16T07:17:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:17:12.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:17:12.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.248+0000 7f0bb483f700 1 Processor -- start 2023-05-16T07:17:12.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.248+0000 7f0bb483f700 1 -- start start 2023-05-16T07:17:12.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0bb483f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 0x7f0bb0150380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0bb483f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0bb00bc290 0x7f0bb0069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0bb483f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00bc770 con 0x7f0bb00bd810 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0bb483f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb00bca50 con 0x7f0bb00bc290 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0bb483f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00bcd30 con 0x7f0bb014ffa0 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0baed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0bb00bd810 0x7f0bb0153eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55748/0 (socket says 172.21.15.92:55748) 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.249+0000 7f0baed9d700 1 -- 172.21.15.92:0/4138710855 learned_addr learned my addr 172.21.15.92:0/4138710855 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0ba5d9b700 1 --2- 172.21.15.92:0/4138710855 >> v2:172.21.15.158:3300/0 conn(0x7f0bb00bc290 0x7f0bb0069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0bae59c700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 0x7f0bb0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136498062 0 0) 0x7f0bb00bc770 con 0x7f0bb00bd810 2023-05-16T07:17:12.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b9c003310 con 0x7f0bb00bd810 2023-05-16T07:17:12.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2816832353 0 0) 0x7f0b9c003310 con 0x7f0bb00bd810 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 msgr2=0x7f0bb0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 0x7f0bb0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 >> v2:172.21.15.158:3300/0 conn(0x7f0bb00bc290 msgr2=0x7f0bb0069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 --2- 172.21.15.92:0/4138710855 >> v2:172.21.15.158:3300/0 conn(0x7f0bb00bc290 0x7f0bb0069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00bd010 con 0x7f0bb00bd810 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f0ba80017f0 con 0x7f0bb00bd810 2023-05-16T07:17:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 >> v1:172.21.15.92:6789/0 conn(0x7f0bb00bd810 legacy=0x7f0bb0153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b9c003a60 0x7f0b9c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 0x7f0b9c00a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b9c00ad40 0x7f0b9c00d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00bca50 con 0x7f0b9c00ad40 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00bcd30 con 0x7f0b9c003a60 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c00d9e0 con 0x7f0bb014ffa0 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0baed9d700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b9c00ad40 0x7f0b9c00d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0bae59c700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b9c003a60 0x7f0b9c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0baed9d700 1 -- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 msgr2=0x7f0b9c00a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.250+0000 7f0baed9d700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb014ffa0 0x7f0b9c00a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0baed9d700 1 -- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b9c003a60 msgr2=0x7f0b9c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0baed9d700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b9c003a60 0x7f0b9c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0baed9d700 1 -- 172.21.15.92:0/4138710855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c003310 con 0x7f0b9c00ad40 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0baed9d700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b9c00ad40 0x7f0b9c00d1d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f0ba800c530 tx=0x7f0ba8001d90).ready entity=mon.0 client_cookie=437ef5337a2d8738 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800ce60 con 0x7f0b9c00ad40 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8005b20 con 0x7f0b9c00ad40 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0badd9b700 1 -- 172.21.15.92:0/4138710855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80070d0 con 0x7f0b9c00ad40 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0bb483f700 1 -- 172.21.15.92:0/4138710855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b9c00ad40 msgr2=0x7f0b9c00d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0bb483f700 1 --2- 172.21.15.92:0/4138710855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b9c00ad40 0x7f0b9c00d1d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f0ba800c530 tx=0x7f0ba8001d90).stop 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0baed9d700 1 -- 172.21.15.92:0/4138710855 reap_dead start 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0bb483f700 1 -- 172.21.15.92:0/4138710855 shutdown_connections 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0bb483f700 1 -- 172.21.15.92:0/4138710855 >> 172.21.15.92:0/4138710855 conn(0x7f0bb00c4620 msgr2=0x7f0bb0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0bb483f700 1 -- 172.21.15.92:0/4138710855 shutdown_connections 2023-05-16T07:17:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.251+0000 7f0bb483f700 1 -- 172.21.15.92:0/4138710855 wait complete. 2023-05-16T07:17:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.253+0000 7f0bb483f700 1 Processor -- start 2023-05-16T07:17:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.253+0000 7f0bb483f700 1 -- start start 2023-05-16T07:17:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bb483f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb00ac610 0x7f0bb00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bb483f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bb00ac9f0 0x7f0bb014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bb483f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 0x7f0bb014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bb483f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00a9130 con 0x7f0bb00aa920 2023-05-16T07:17:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bb483f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00a95c0 con 0x7f0bb00ac9f0 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bb483f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb014ba30 con 0x7f0bb00ac610 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 0x7f0bb014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 0x7f0bb014e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45012/0 (socket says 172.21.15.92:45012) 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 -- 172.21.15.92:0/1664555797 learned_addr learned my addr 172.21.15.92:0/1664555797 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bae59c700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb00ac610 0x7f0bb00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0ba5d9b700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bb00ac9f0 0x7f0bb014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 -- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb00ac610 msgr2=0x7f0bb00aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb00ac610 0x7f0bb00aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 -- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bb00ac9f0 msgr2=0x7f0bb014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bb00ac9f0 0x7f0bb014dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0baed9d700 1 -- 172.21.15.92:0/1664555797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb014bd10 con 0x7f0bb00aa920 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.254+0000 7f0bae59c700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb00ac610 0x7f0bb00aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.255+0000 7f0baed9d700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 0x7f0bb014e4a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f0ba800a000 tx=0x7f0ba8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.255+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80070d0 con 0x7f0bb00aa920 2023-05-16T07:17:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.255+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba80017b0 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.255+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8013870 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.255+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb00a8dc0 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.255+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb014b690 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.256+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb014bea0 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.256+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0ba801d070 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.256+0000 7f0ba6ffd700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b9005ed90 0x7f0b90061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.256+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f0ba807a930 con 0x7f0bb00aa920 2023-05-16T07:17:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.256+0000 7f0bae59c700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b9005ed90 0x7f0b90061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.257+0000 7f0bae59c700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b9005ed90 0x7f0b90061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0ba00143a0 tx=0x7f0ba0008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.260+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0ba804be30 con 0x7f0bb00aa920 2023-05-16T07:17:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.425+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0bb006c350 con 0x7f0bb00aa920 2023-05-16T07:17:12.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.428+0000 7f0ba6ffd700 1 -- 172.21.15.92:0/1664555797 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ba801a020 con 0x7f0bb00aa920 2023-05-16T07:17:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b9005ed90 msgr2=0x7f0b90061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b9005ed90 0x7f0b90061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0ba00143a0 tx=0x7f0ba0008040).stop 2023-05-16T07:17:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 msgr2=0x7f0bb014e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 0x7f0bb014e4a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f0ba800a000 tx=0x7f0ba8015000).stop 2023-05-16T07:17:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 shutdown_connections 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0b9005ed90 0x7f0b90061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bb00ac610 0x7f0bb00aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb00aa920 0x7f0bb014e4a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 --2- 172.21.15.92:0/1664555797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bb00ac9f0 0x7f0bb014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 >> 172.21.15.92:0/1664555797 conn(0x7f0bb00c4620 msgr2=0x7f0bb00c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 shutdown_connections 2023-05-16T07:17:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.430+0000 7f0bb483f700 1 -- 172.21.15.92:0/1664555797 wait complete. 2023-05-16T07:17:12.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:17:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.554+0000 7fe47e1a9700 1 Processor -- start 2023-05-16T07:17:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 -- start start 2023-05-16T07:17:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe47814f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe47805e460 0x7fe478064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe478152bc0 0x7fe4780a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47805ec20 con 0x7fe47805e460 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4780c3170 con 0x7fe478152bc0 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.555+0000 7fe47e1a9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47805e940 con 0x7fe47814f3b0 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe47805e460 0x7fe478064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe47805e460 0x7fe478064150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45020/0 (socket says 172.21.15.92:45020) 2023-05-16T07:17:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 -- 172.21.15.92:0/2771486421 learned_addr learned my addr 172.21.15.92:0/2771486421 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47d9a8700 1 --2- 172.21.15.92:0/2771486421 >> v2:172.21.15.158:3300/0 conn(0x7fe478152bc0 0x7fe4780a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47d1a7700 1 --2- 172.21.15.92:0/2771486421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe47814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 -- 172.21.15.92:0/2771486421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 msgr2=0x7fe47814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 --2- 172.21.15.92:0/2771486421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe47814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 -- 172.21.15.92:0/2771486421 >> v2:172.21.15.158:3300/0 conn(0x7fe478152bc0 msgr2=0x7fe4780a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 --2- 172.21.15.92:0/2771486421 >> v2:172.21.15.158:3300/0 conn(0x7fe478152bc0 0x7fe4780a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 -- 172.21.15.92:0/2771486421 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478063670 con 0x7fe47805e460 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47d9a8700 1 --2- 172.21.15.92:0/2771486421 >> v2:172.21.15.158:3300/0 conn(0x7fe478152bc0 0x7fe4780a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47c9a6700 1 --2- 172.21.15.92:0/2771486421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe478064150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fe47814fc80 tx=0x7fe46c0090f0).ready entity=mon.0 client_cookie=8fd8c519818764f7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe477fff700 1 -- 172.21.15.92:0/2771486421 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00b600 con 0x7fe47805e460 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe477fff700 1 -- 172.21.15.92:0/2771486421 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c00b760 con 0x7fe47805e460 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe477fff700 1 -- 172.21.15.92:0/2771486421 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00bae0 con 0x7fe47805e460 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47e1a9700 1 -- 172.21.15.92:0/2771486421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 msgr2=0x7fe478064150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.556+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/2771486421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe478064150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fe47814fc80 tx=0x7fe46c0090f0).stop 2023-05-16T07:17:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 -- 172.21.15.92:0/2771486421 shutdown_connections 2023-05-16T07:17:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/2771486421 >> v2:172.21.15.158:3300/0 conn(0x7fe478152bc0 0x7fe4780a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/2771486421 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe47814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/2771486421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe478064150 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 -- 172.21.15.92:0/2771486421 >> 172.21.15.92:0/2771486421 conn(0x7fe478062d80 msgr2=0x7fe47805dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 -- 172.21.15.92:0/2771486421 shutdown_connections 2023-05-16T07:17:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.557+0000 7fe47e1a9700 1 -- 172.21.15.92:0/2771486421 wait complete. 2023-05-16T07:17:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.558+0000 7fe47e1a9700 1 Processor -- start 2023-05-16T07:17:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.558+0000 7fe47e1a9700 1 -- start start 2023-05-16T07:17:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe4781463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe4780a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe478152bc0 0x7fe4780a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780a35b0 con 0x7fe47805e460 2023-05-16T07:17:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780a2400 con 0x7fe478152bc0 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780a1360 con 0x7fe47814f3b0 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe4781463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe4781463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45024/0 (socket says 172.21.15.92:45024) 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 -- 172.21.15.92:0/1213827706 learned_addr learned my addr 172.21.15.92:0/1213827706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d9a8700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe478152bc0 0x7fe4780a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47c9a6700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe4780a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 -- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 msgr2=0x7fe4780a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe4780a2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 -- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe478152bc0 msgr2=0x7fe4780a30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe478152bc0 0x7fe4780a30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 -- 172.21.15.92:0/1213827706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c009370 con 0x7fe47805e460 2023-05-16T07:17:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d9a8700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe478152bc0 0x7fe4780a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47d1a7700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe4781463f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fe46800cfc0 tx=0x7fe468007890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468016400 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe468016560 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4780a1640 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4680168e0 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.559+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4780a1c00 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.560+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe478047150 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.561+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe46801d060 con 0x7fe47805e460 2023-05-16T07:17:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.561+0000 7fe4757fa700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe46405ed90 0x7fe464061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.561+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6283+0+0 (secure 0 0 0) 0x7fe46807a6d0 con 0x7fe47805e460 2023-05-16T07:17:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.561+0000 7fe47c9a6700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe46405ed90 0x7fe464061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.562+0000 7fe47c9a6700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe46405ed90 0x7fe464061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe4780659e0 tx=0x7fe46c017000).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.565+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe46804bb70 con 0x7fe47805e460 2023-05-16T07:17:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.729+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe4780a2240 con 0x7fe47805e460 2023-05-16T07:17:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.732+0000 7fe4757fa700 1 -- 172.21.15.92:0/1213827706 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v9) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe4780a2240 con 0x7fe47805e460 2023-05-16T07:17:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe46405ed90 msgr2=0x7fe464061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe46405ed90 0x7fe464061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe4780659e0 tx=0x7fe46c017000).stop 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 msgr2=0x7fe4781463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe4781463f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fe46800cfc0 tx=0x7fe468007890).stop 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 shutdown_connections 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe46405ed90 0x7fe464061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe478152bc0 0x7fe4780a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe47805e460 0x7fe4781463f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 --2- 172.21.15.92:0/1213827706 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe47814f3b0 0x7fe4780a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 >> 172.21.15.92:0/1213827706 conn(0x7fe478062d80 msgr2=0x7fe4780c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 shutdown_connections 2023-05-16T07:17:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.734+0000 7fe47e1a9700 1 -- 172.21.15.92:0/1213827706 wait complete. 2023-05-16T07:17:12.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-05-16T07:17:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.856+0000 7f2fb2f58700 1 Processor -- start 2023-05-16T07:17:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.856+0000 7f2fb2f58700 1 -- start start 2023-05-16T07:17:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb2f58700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2fac154170 0x7f2fac0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb2f58700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154550 0x7f2fac0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb2f58700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac0c0660 con 0x7f2fac0696a0 2023-05-16T07:17:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb2f58700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac0c1b00 con 0x7f2fac154170 2023-05-16T07:17:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb2f58700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0bfd30 con 0x7f2fac154550 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb0cf4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2fac154170 0x7f2fac0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb14f5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2fac0696a0 0x7f2fac157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55780/0 (socket says 172.21.15.92:55780) 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb0cf4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2fac154170 0x7f2fac0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45032/0 (socket says 172.21.15.92:45032) 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb14f5700 1 -- 172.21.15.92:0/91656567 learned_addr learned my addr 172.21.15.92:0/91656567 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fabfff700 1 --2- 172.21.15.92:0/91656567 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154550 0x7f2fac0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/91656567 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154550 msgr2=0x7f2fac0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb0cf4700 1 --2- 172.21.15.92:0/91656567 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154550 0x7f2fac0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/91656567 >> v1:172.21.15.92:6789/0 conn(0x7f2fac0696a0 legacy=0x7f2fac157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/91656567 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac0c0010 con 0x7f2fac154170 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.857+0000 7f2fab7fe700 1 -- 172.21.15.92:0/91656567 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373419730 0 0) 0x7f2fac0c0660 con 0x7f2fac0696a0 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb0cf4700 1 --2- 172.21.15.92:0/91656567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac154170 0x7f2fac0c5c70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f2f9c009760 tx=0x7f2f9c009060).ready entity=mon.0 client_cookie=1b874246ac03db6a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fab7fe700 1 -- 172.21.15.92:0/91656567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0090d0 con 0x7f2fac154170 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fab7fe700 1 -- 172.21.15.92:0/91656567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c00b580 con 0x7f2fac154170 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fab7fe700 1 -- 172.21.15.92:0/91656567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00b900 con 0x7f2fac154170 2023-05-16T07:17:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 -- 172.21.15.92:0/91656567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac154170 msgr2=0x7f2fac0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/91656567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac154170 0x7f2fac0c5c70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f2f9c009760 tx=0x7f2f9c009060).stop 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 -- 172.21.15.92:0/91656567 shutdown_connections 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/91656567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac154170 0x7f2fac0c5c70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/91656567 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154550 0x7f2fac0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 -- 172.21.15.92:0/91656567 >> 172.21.15.92:0/91656567 conn(0x7f2fac0cb700 msgr2=0x7f2fac0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 -- 172.21.15.92:0/91656567 shutdown_connections 2023-05-16T07:17:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.858+0000 7f2fb2f58700 1 -- 172.21.15.92:0/91656567 wait complete. 2023-05-16T07:17:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.859+0000 7f2fb2f58700 1 Processor -- start 2023-05-16T07:17:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 -- start start 2023-05-16T07:17:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 0x7f2fac0c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154170 0x7f2fac0c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fac154550 0x7f2fac0c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0c4d60 con 0x7f2fac0696a0 2023-05-16T07:17:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0ae030 con 0x7f2fac154550 2023-05-16T07:17:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb2f58700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac153900 con 0x7f2fac154170 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb0cf4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 0x7f2fac0c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb0cf4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 0x7f2fac0c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45036/0 (socket says 172.21.15.92:45036) 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/2230972990 learned_addr learned my addr 172.21.15.92:0/2230972990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fabfff700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154170 0x7f2fac0c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.860+0000 7f2fb14f5700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fac154550 0x7f2fac0c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154170 msgr2=0x7f2fac0c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb0cf4700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154170 0x7f2fac0c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fac154550 msgr2=0x7f2fac0c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb0cf4700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fac154550 0x7f2fac0c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb0cf4700 1 -- 172.21.15.92:0/2230972990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0092a0 con 0x7f2fac0696a0 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb0cf4700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 0x7f2fac0c2790 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f2f9c009eb0 tx=0x7f2f9c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00b900 con 0x7f2fac0696a0 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c006970 con 0x7f2fac0696a0 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac0ae280 con 0x7f2fac0696a0 2023-05-16T07:17:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c000c90 con 0x7f2fac0696a0 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.861+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac0ae840 con 0x7f2fac0696a0 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.862+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac158db0 con 0x7f2fac0696a0 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.862+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2f9c00f070 con 0x7f2fac0696a0 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.862+0000 7f2fa8ff9700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f9805ed90 0x7f2f98061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.863+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f2f9c07b4c0 con 0x7f2fac0696a0 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.863+0000 7f2fabfff700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f9805ed90 0x7f2f98061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.863+0000 7f2fabfff700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f9805ed90 0x7f2f98061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2fac0c8b40 tx=0x7f2fa0008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:12.866+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2f9c04c960 con 0x7f2fac0696a0 2023-05-16T07:17:13.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.094+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f2fac0666c0 con 0x7f2fac0696a0 2023-05-16T07:17:13.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.104+0000 7f2fa8ff9700 1 -- 172.21.15.92:0/2230972990 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v524) v1 ==== 154+0+0 (secure 0 0 0) 0x7f2f9c04ff80 con 0x7f2fac0696a0 2023-05-16T07:17:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f9805ed90 msgr2=0x7f2f98061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f9805ed90 0x7f2f98061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2fac0c8b40 tx=0x7f2fa0008040).stop 2023-05-16T07:17:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 msgr2=0x7f2fac0c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 0x7f2fac0c2790 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f2f9c009eb0 tx=0x7f2f9c017000).stop 2023-05-16T07:17:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 shutdown_connections 2023-05-16T07:17:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2f9805ed90 0x7f2f98061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fac154550 0x7f2fac0c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fac0696a0 0x7f2fac0c2790 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 --2- 172.21.15.92:0/2230972990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fac154170 0x7f2fac0c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 >> 172.21.15.92:0/2230972990 conn(0x7f2fac0cb700 msgr2=0x7f2fac0a9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 shutdown_connections 2023-05-16T07:17:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.106+0000 7f2fb2f58700 1 -- 172.21.15.92:0/2230972990 wait complete. 2023-05-16T07:17:13.121 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-05-16T07:17:13.122 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-05-16T07:17:13.122 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:17:13.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-05-16T07:17:13.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.230+0000 7f7e8e106700 1 Processor -- start 2023-05-16T07:17:13.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.230+0000 7f7e8e106700 1 -- start start 2023-05-16T07:17:13.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.230+0000 7f7e8e106700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e88063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.230+0000 7f7e8e106700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e88062b10 con 0x7f7e88062fa0 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e8e106700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e88068340 con 0x7f7e88153da0 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e8e106700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e88068620 con 0x7f7e88154180 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e87fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7e88062fa0 0x7f7e880bda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55796/0 (socket says 172.21.15.92:55796) 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e87fff700 1 -- 172.21.15.92:0/1390681550 learned_addr learned my addr 172.21.15.92:0/1390681550 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e88063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830483401 0 0) 0x7f7e88062b10 con 0x7f7e88062fa0 2023-05-16T07:17:13.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e700033f0 con 0x7f7e88062fa0 2023-05-16T07:17:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182099654 0 0) 0x7f7e88068340 con 0x7f7e88153da0 2023-05-16T07:17:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e88062b10 con 0x7f7e88153da0 2023-05-16T07:17:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474607885 0 0) 0x7f7e700033f0 con 0x7f7e88062fa0 2023-05-16T07:17:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 msgr2=0x7f7e88063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e88063720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 >> v1:172.21.15.158:6789/0 conn(0x7f7e88153da0 legacy=0x7f7e88069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880624c0 con 0x7f7e88062fa0 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f7e780017f0 con 0x7f7e88062fa0 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 >> v1:172.21.15.92:6789/0 conn(0x7f7e88062fa0 legacy=0x7f7e880bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e70003aa0 0x7f7e70003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e70004690 0x7f7e70004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e70005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e88068620 con 0x7f7e70004690 2023-05-16T07:17:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7000b990 con 0x7f7e70003aa0 2023-05-16T07:17:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7000bb20 con 0x7f7e88154180 2023-05-16T07:17:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e70004690 0x7f7e70004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e877fe700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e70003aa0 0x7f7e70003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.231+0000 7f7e87fff700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e70005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 msgr2=0x7f7e70005020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e70005020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e70003aa0 msgr2=0x7f7e70003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e70003aa0 0x7f7e70003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1390681550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700033f0 con 0x7f7e70004690 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e87fff700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e88154180 0x7f7e70005020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e877fe700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e70003aa0 0x7f7e70003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:13.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e70004690 0x7f7e70004b10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f7e7c009a60 tx=0x7f7e7c009060).ready entity=mon.0 client_cookie=9e41f9c581e4b177 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c009ea0 con 0x7f7e70004690 2023-05-16T07:17:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c009430 con 0x7f7e70004690 2023-05-16T07:17:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e867fc700 1 -- 172.21.15.92:0/1390681550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00e870 con 0x7f7e70004690 2023-05-16T07:17:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e8e106700 1 -- 172.21.15.92:0/1390681550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e70004690 msgr2=0x7f7e70004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1390681550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e70004690 0x7f7e70004b10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f7e7c009a60 tx=0x7f7e7c009060).stop 2023-05-16T07:17:13.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e87fff700 1 -- 172.21.15.92:0/1390681550 reap_dead start 2023-05-16T07:17:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e8e106700 1 -- 172.21.15.92:0/1390681550 shutdown_connections 2023-05-16T07:17:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e8e106700 1 -- 172.21.15.92:0/1390681550 >> 172.21.15.92:0/1390681550 conn(0x7f7e880c8aa0 msgr2=0x7f7e880c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e8e106700 1 -- 172.21.15.92:0/1390681550 shutdown_connections 2023-05-16T07:17:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.232+0000 7f7e8e106700 1 -- 172.21.15.92:0/1390681550 wait complete. 2023-05-16T07:17:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 Processor -- start 2023-05-16T07:17:13.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 -- start start 2023-05-16T07:17:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e70004690 0x7f7e880c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 0x7f7e8814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e8814da20 0x7f7e880c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880c7090 con 0x7f7e880c3730 2023-05-16T07:17:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8814dea0 con 0x7f7e8814da20 2023-05-16T07:17:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.234+0000 7f7e8e106700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880c64a0 con 0x7f7e70004690 2023-05-16T07:17:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 0x7f7e8814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 0x7f7e8814d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45058/0 (socket says 172.21.15.92:45058) 2023-05-16T07:17:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1965531169 learned_addr learned my addr 172.21.15.92:0/1965531169 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e877fe700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e70004690 0x7f7e880c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e87fff700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e8814da20 0x7f7e880c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e70004690 msgr2=0x7f7e880c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e70004690 0x7f7e880c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e8814da20 msgr2=0x7f7e880c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e8814da20 0x7f7e880c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 -- 172.21.15.92:0/1965531169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880c6760 con 0x7f7e880c3730 2023-05-16T07:17:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e877fe700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e70004690 0x7f7e880c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:13.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 0x7f7e8814d530 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f7e7c01b520 tx=0x7f7e7c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c020070 con 0x7f7e880c3730 2023-05-16T07:17:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c01ba90 con 0x7f7e880c3730 2023-05-16T07:17:13.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e880c6a40 con 0x7f7e880c3730 2023-05-16T07:17:13.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c01be50 con 0x7f7e880c3730 2023-05-16T07:17:13.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.235+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e880c7eb0 con 0x7f7e880c3730 2023-05-16T07:17:13.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.236+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e880c7200 con 0x7f7e880c3730 2023-05-16T07:17:13.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.236+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7e7c02b030 con 0x7f7e880c3730 2023-05-16T07:17:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.237+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7e7405edc0 0x7f7e74061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.237+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f7e7c07ae30 con 0x7f7e880c3730 2023-05-16T07:17:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.237+0000 7f7e877fe700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7e7405edc0 0x7f7e74061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.237+0000 7f7e877fe700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7e7405edc0 0x7f7e74061280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7e80000c00 tx=0x7f7e80009040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.241+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7e7c04c330 con 0x7f7e880c3730 2023-05-16T07:17:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:13.465+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f7e8806c350 con 0x7f7e880c3730 2023-05-16T07:17:14.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.107+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1965531169 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v525) v1 ==== 159+0+0 (secure 0 0 0) 0x7f7e7c029070 con 0x7f7e880c3730 2023-05-16T07:17:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7e7405edc0 msgr2=0x7f7e74061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7e7405edc0 0x7f7e74061280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7e80000c00 tx=0x7f7e80009040).stop 2023-05-16T07:17:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 msgr2=0x7f7e8814d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 0x7f7e8814d530 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f7e7c01b520 tx=0x7f7e7c01a000).stop 2023-05-16T07:17:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 shutdown_connections 2023-05-16T07:17:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7e7405edc0 0x7f7e74061280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7e70004690 0x7f7e880c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7e8814da20 0x7f7e880c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 --2- 172.21.15.92:0/1965531169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e880c3730 0x7f7e8814d530 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 >> 172.21.15.92:0/1965531169 conn(0x7f7e880c8aa0 msgr2=0x7f7e880c47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:14.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 shutdown_connections 2023-05-16T07:17:14.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.109+0000 7f7e8e106700 1 -- 172.21.15.92:0/1965531169 wait complete. 2023-05-16T07:17:14.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:17:14.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.235+0000 7f8f2359e700 1 Processor -- start 2023-05-16T07:17:14.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.235+0000 7f8f2359e700 1 -- start start 2023-05-16T07:17:14.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.236+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 0x7f8f2415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.236+0000 7f8f2359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240bfcc0 con 0x7f8f24157710 2023-05-16T07:17:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.236+0000 7f8f2359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240aa410 con 0x7f8f240c1160 2023-05-16T07:17:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.236+0000 7f8f2359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2406a9a0 con 0x7f8f240c62c0 2023-05-16T07:17:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.236+0000 7f8f2259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8f24157710 0x7f8f240a9d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55812/0 (socket says 172.21.15.92:55812) 2023-05-16T07:17:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2259c700 1 -- 172.21.15.92:0/417667360 learned_addr learned my addr 172.21.15.92:0/417667360 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f22d9d700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 0x7f8f2415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748583573 0 0) 0x7f8f240bfcc0 con 0x7f8f24157710 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f100033f0 con 0x7f8f24157710 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041184377 0 0) 0x7f8f100033f0 con 0x7f8f24157710 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 msgr2=0x7f8f2415ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 0x7f8f2415ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 >> v1:172.21.15.158:6789/0 conn(0x7f8f240c1160 legacy=0x7f8f240c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2406a140 con 0x7f8f24157710 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f8f140017f0 con 0x7f8f24157710 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 >> v1:172.21.15.92:6789/0 conn(0x7f8f24157710 legacy=0x7f8f240a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10003910 0x7f8f10003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f10004690 0x7f8f10004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 0x7f8f10005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2406a9a0 con 0x7f8f10004690 2023-05-16T07:17:14.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1000b8e0 con 0x7f8f10003910 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1000ba50 con 0x7f8f240c62c0 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f21d9b700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f10004690 0x7f8f10004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2259c700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10003910 0x7f8f10003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f22d9d700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 0x7f8f10005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f21d9b700 1 -- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 msgr2=0x7f8f10005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f21d9b700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f240c62c0 0x7f8f10005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f21d9b700 1 -- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10003910 msgr2=0x7f8f10003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f21d9b700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10003910 0x7f8f10003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f21d9b700 1 -- 172.21.15.92:0/417667360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100033f0 con 0x7f8f10004690 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.237+0000 7f8f2259c700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10003910 0x7f8f10003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f21d9b700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f10004690 0x7f8f10004b10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f8f0c0017e0 tx=0x7f8f0c007b90).ready entity=mon.0 client_cookie=1319a513a4e30761 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c007ea0 con 0x7f8f10004690 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c012400 con 0x7f8f10004690 2023-05-16T07:17:14.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2159a700 1 -- 172.21.15.92:0/417667360 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0127a0 con 0x7f8f10004690 2023-05-16T07:17:14.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2359e700 1 -- 172.21.15.92:0/417667360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f10004690 msgr2=0x7f8f10004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2359e700 1 --2- 172.21.15.92:0/417667360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f10004690 0x7f8f10004b10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f8f0c0017e0 tx=0x7f8f0c007b90).stop 2023-05-16T07:17:14.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f22d9d700 1 -- 172.21.15.92:0/417667360 reap_dead start 2023-05-16T07:17:14.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2359e700 1 -- 172.21.15.92:0/417667360 shutdown_connections 2023-05-16T07:17:14.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2359e700 1 -- 172.21.15.92:0/417667360 >> 172.21.15.92:0/417667360 conn(0x7f8f240cb600 msgr2=0x7f8f240c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:14.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2359e700 1 -- 172.21.15.92:0/417667360 shutdown_connections 2023-05-16T07:17:14.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.238+0000 7f8f2359e700 1 -- 172.21.15.92:0/417667360 wait complete. 2023-05-16T07:17:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.239+0000 7f8f2359e700 1 Processor -- start 2023-05-16T07:17:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.239+0000 7f8f2359e700 1 -- start start 2023-05-16T07:17:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10004690 0x7f8f240c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f2414c6f0 0x7f8f241516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 0x7f8f24151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2414e800 con 0x7f8f2414cb70 2023-05-16T07:17:14.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24152090 con 0x7f8f10004690 2023-05-16T07:17:14.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2414db80 con 0x7f8f2414c6f0 2023-05-16T07:17:14.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 0x7f8f24151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 0x7f8f24151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45080/0 (socket says 172.21.15.92:45080) 2023-05-16T07:17:14.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 -- 172.21.15.92:0/1217828776 learned_addr learned my addr 172.21.15.92:0/1217828776 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:14.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f2259c700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10004690 0x7f8f240c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f21d9b700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f2414c6f0 0x7f8f241516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 -- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f2414c6f0 msgr2=0x7f8f241516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f2414c6f0 0x7f8f241516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 -- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10004690 msgr2=0x7f8f240c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10004690 0x7f8f240c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.240+0000 7f8f22d9d700 1 -- 172.21.15.92:0/1217828776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2414de60 con 0x7f8f2414cb70 2023-05-16T07:17:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f21d9b700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f2414c6f0 0x7f8f241516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f22d9d700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 0x7f8f24151ba0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f8f24083990 tx=0x7f8f1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c015480 con 0x7f8f2414cb70 2023-05-16T07:17:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c0155e0 con 0x7f8f2414cb70 2023-05-16T07:17:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2414e140 con 0x7f8f2414cb70 2023-05-16T07:17:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c015980 con 0x7f8f2414cb70 2023-05-16T07:17:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.241+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f240c20c0 con 0x7f8f2414cb70 2023-05-16T07:17:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.242+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2414e940 con 0x7f8f2414cb70 2023-05-16T07:17:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.242+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8f1c012040 con 0x7f8f2414cb70 2023-05-16T07:17:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.242+0000 7f8f1a7fc700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f0405ed90 0x7f8f04061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.242+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f8f1c079ee0 con 0x7f8f2414cb70 2023-05-16T07:17:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.242+0000 7f8f2259c700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f0405ed90 0x7f8f04061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.243+0000 7f8f2259c700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f0405ed90 0x7f8f04061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8f14007e30 tx=0x7f8f140013e0).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.246+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8f2414e940 con 0x7f8f2414cb70 2023-05-16T07:17:14.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.409+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8f2414e940 con 0x7f8f2414cb70 2023-05-16T07:17:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.413+0000 7f8f1a7fc700 1 -- 172.21.15.92:0/1217828776 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8f2414e940 con 0x7f8f2414cb70 2023-05-16T07:17:14.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f0405ed90 msgr2=0x7f8f04061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f0405ed90 0x7f8f04061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8f14007e30 tx=0x7f8f140013e0).stop 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 msgr2=0x7f8f24151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 0x7f8f24151ba0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f8f24083990 tx=0x7f8f1c008040).stop 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 shutdown_connections 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8f0405ed90 0x7f8f04061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f2414cb70 0x7f8f24151ba0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f10004690 0x7f8f240c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 --2- 172.21.15.92:0/1217828776 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8f2414c6f0 0x7f8f241516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 >> 172.21.15.92:0/1217828776 conn(0x7f8f240cb600 msgr2=0x7f8f240c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:14.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 shutdown_connections 2023-05-16T07:17:14.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.415+0000 7f8f2359e700 1 -- 172.21.15.92:0/1217828776 wait complete. 2023-05-16T07:17:14.430 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:17:14.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:14.456 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:14.489 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:17:14.490 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:17:14.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.603+0000 7efd6fde6700 1 Processor -- start 2023-05-16T07:17:14.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.603+0000 7efd6fde6700 1 -- start start 2023-05-16T07:17:14.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6fde6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 0x7efd68150f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6fde6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd680a61d0 con 0x7efd68150b50 2023-05-16T07:17:14.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6fde6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd68069410 con 0x7efd680bc8f0 2023-05-16T07:17:14.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6fde6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd680696f0 con 0x7efd68068f10 2023-05-16T07:17:14.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6db82700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd68150b50 0x7efd680a5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55832/0 (socket says 172.21.15.92:55832) 2023-05-16T07:17:14.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6db82700 1 -- 172.21.15.92:0/2289228682 learned_addr learned my addr 172.21.15.92:0/2289228682 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:14.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6d381700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 0x7efd68150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289099845 0 0) 0x7efd680a61d0 con 0x7efd68150b50 2023-05-16T07:17:14.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd540033f0 con 0x7efd68150b50 2023-05-16T07:17:14.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3776191265 0 0) 0x7efd540033f0 con 0x7efd68150b50 2023-05-16T07:17:14.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 msgr2=0x7efd68150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:14.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 0x7efd68150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 >> v1:172.21.15.158:6789/0 conn(0x7efd680bc8f0 legacy=0x7efd68065320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680699d0 con 0x7efd68150b50 2023-05-16T07:17:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7efd60001470 con 0x7efd68150b50 2023-05-16T07:17:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 >> v1:172.21.15.92:6789/0 conn(0x7efd68150b50 legacy=0x7efd680a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.604+0000 7efd6cb80700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003910 0x7efd54003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 0x7efd54004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54004b80 0x7efd54005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd680696f0 con 0x7efd54003910 2023-05-16T07:17:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5400b910 con 0x7efd54004b80 2023-05-16T07:17:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5400bad0 con 0x7efd68068f10 2023-05-16T07:17:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003910 0x7efd54003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6e383700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54004b80 0x7efd54005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6d381700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 0x7efd54004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 -- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 msgr2=0x7efd54004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68068f10 0x7efd54004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 -- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54004b80 msgr2=0x7efd54005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54004b80 0x7efd54005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 -- 172.21.15.92:0/2289228682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540033f0 con 0x7efd54003910 2023-05-16T07:17:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6db82700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003910 0x7efd54003d30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7efd6000c190 tx=0x7efd60001e50).ready entity=mon.0 client_cookie=57259f99f18d5081 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60007a20 con 0x7efd54003910 2023-05-16T07:17:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6000cac0 con 0x7efd54003910 2023-05-16T07:17:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.605+0000 7efd6cb80700 1 -- 172.21.15.92:0/2289228682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6000ce40 con 0x7efd54003910 2023-05-16T07:17:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6fde6700 1 -- 172.21.15.92:0/2289228682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003910 msgr2=0x7efd54003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2289228682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003910 0x7efd54003d30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7efd6000c190 tx=0x7efd60001e50).stop 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6e383700 1 -- 172.21.15.92:0/2289228682 reap_dead start 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6fde6700 1 -- 172.21.15.92:0/2289228682 shutdown_connections 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6fde6700 1 -- 172.21.15.92:0/2289228682 >> 172.21.15.92:0/2289228682 conn(0x7efd680c7350 msgr2=0x7efd680c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6fde6700 1 -- 172.21.15.92:0/2289228682 shutdown_connections 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.606+0000 7efd6fde6700 1 -- 172.21.15.92:0/2289228682 wait complete. 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 Processor -- start 2023-05-16T07:17:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 -- start start 2023-05-16T07:17:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54003910 0x7efd6813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68156710 0x7efd680abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 0x7efd680c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd68166640 con 0x7efd680ac390 2023-05-16T07:17:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6813ff20 con 0x7efd54003910 2023-05-16T07:17:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.607+0000 7efd6fde6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd68165a30 con 0x7efd68156710 2023-05-16T07:17:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 0x7efd680c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 0x7efd680c0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45100/0 (socket says 172.21.15.92:45100) 2023-05-16T07:17:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 -- 172.21.15.92:0/2806952969 learned_addr learned my addr 172.21.15.92:0/2806952969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6db82700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54003910 0x7efd6813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6d381700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68156710 0x7efd680abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 -- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68156710 msgr2=0x7efd680abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68156710 0x7efd680abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 -- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54003910 msgr2=0x7efd6813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54003910 0x7efd6813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 -- 172.21.15.92:0/2806952969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68165d10 con 0x7efd680ac390 2023-05-16T07:17:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6e383700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 0x7efd680c0de0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7efd6806bf40 tx=0x7efd64007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64010520 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd640106a0 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64010a60 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd68165ff0 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.608+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd68149b30 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.609+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6806c350 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.610+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7efd6401b030 con 0x7efd680ac390 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.610+0000 7efd5dffb700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4c05ed90 0x7efd4c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:14.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.610+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6283+0+0 (secure 0 0 0) 0x7efd640794c0 con 0x7efd680ac390 2023-05-16T07:17:14.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.610+0000 7efd6db82700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4c05ed90 0x7efd4c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:14.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.610+0000 7efd6db82700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4c05ed90 0x7efd4c061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7efd60001ed0 tx=0x7efd60015000).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:14.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.614+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7efd6404a9c0 con 0x7efd680ac390 2023-05-16T07:17:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.815+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd68064e00 con 0x7efd680ac390 2023-05-16T07:17:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.816+0000 7efd5dffb700 1 -- 172.21.15.92:0/2806952969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3532 (secure 0 0 0) 0x7efd6404dfe0 con 0x7efd680ac390 2023-05-16T07:17:14.820 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:14.820 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/322004454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":322004454},{"type":"v1","addr":"172.21.15.158:6835","nonce":322004454}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2447960904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2447960904},{"type":"v1","addr":"172.21.15.92:6834","nonce":2447960904}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1846977934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1846977934},{"type":"v1","addr":"172.21.15.92:6835","nonce":1846977934}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1276988519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1276988519},{"type":"v1","addr":"172.21.15.158:6837","nonce":1276988519}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-05-16T07:17:14.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.817+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4c05ed90 msgr2=0x7efd4c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.817+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4c05ed90 0x7efd4c061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7efd60001ed0 tx=0x7efd60015000).stop 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.817+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 msgr2=0x7efd680c0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.817+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 0x7efd680c0de0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7efd6806bf40 tx=0x7efd64007f30).stop 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 shutdown_connections 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7efd4c05ed90 0x7efd4c061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd680ac390 0x7efd680c0de0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efd54003910 0x7efd6813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 --2- 172.21.15.92:0/2806952969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efd68156710 0x7efd680abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 >> 172.21.15.92:0/2806952969 conn(0x7efd680c7350 msgr2=0x7efd68065760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 shutdown_connections 2023-05-16T07:17:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:14.818+0000 7efd6fde6700 1 -- 172.21.15.92:0/2806952969 wait complete. 2023-05-16T07:17:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2023-05-16T07:17:14.834 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:17:14.834 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:17:14.834 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:17:14.835 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:14.835 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:17:14.835 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:17:14.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:17:14.836 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:14.836 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:17:14.836 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:17:14.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:17:14.837 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:14.837 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:17:14.837 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:17:14.837 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:17:14.837 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:14.843 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:17:14.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:17:14.844 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:17:14.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:17:14.847 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:17:14.848 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:17:14.848 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:17:14.849 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:17:14.885 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:17:14.886 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:14.886 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:17:14.887 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:14.889 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:17:14.889 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:14.890 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:17:14.890 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:14.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:17:14.956 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:17:14.956 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:17:14.972 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:17:14.973 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:17:15.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.007+0000 7ff357872700 1 Processor -- start 2023-05-16T07:17:15.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.007+0000 7ff357872700 1 -- start start 2023-05-16T07:17:15.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff357872700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff3500bd9a0 0x7ff35015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:15.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff357872700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff350154b80 con 0x7ff350068060 2023-05-16T07:17:15.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff357872700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff350154e60 con 0x7ff35013a5a0 2023-05-16T07:17:15.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff357872700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff350155140 con 0x7ff3500bd9a0 2023-05-16T07:17:15.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff354e0d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff350068060 0x7ff3500bd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55878/0 (socket says 172.21.15.92:55878) 2023-05-16T07:17:15.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff354e0d700 1 -- 172.21.15.92:0/1733441079 learned_addr learned my addr 172.21.15.92:0/1733441079 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:15.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.008+0000 7ff355e0f700 1 --2- 172.21.15.92:0/1733441079 >> v2:172.21.15.92:3300/0 conn(0x7ff3500bd9a0 0x7ff35015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff343fff700 1 -- 172.21.15.92:0/1733441079 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285972969 0 0) 0x7ff350154b80 con 0x7ff350068060 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff343fff700 1 -- 172.21.15.92:0/1733441079 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff33c0033f0 con 0x7ff350068060 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff355e0f700 1 -- 172.21.15.92:0/1733441079 >> v1:172.21.15.158:6789/0 conn(0x7ff35013a5a0 legacy=0x7ff350068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff355e0f700 1 -- 172.21.15.92:0/1733441079 >> v1:172.21.15.92:6789/0 conn(0x7ff350068060 legacy=0x7ff3500bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff355e0f700 1 -- 172.21.15.92:0/1733441079 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350155420 con 0x7ff3500bd9a0 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff355e0f700 1 --2- 172.21.15.92:0/1733441079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3500bd9a0 0x7ff35015bd80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7ff348009e60 tx=0x7ff348009780).ready entity=mon.0 client_cookie=c7e22aad033a3a87 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff343fff700 1 -- 172.21.15.92:0/1733441079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800e070 con 0x7ff3500bd9a0 2023-05-16T07:17:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff343fff700 1 -- 172.21.15.92:0/1733441079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34800bb20 con 0x7ff3500bd9a0 2023-05-16T07:17:15.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff343fff700 1 -- 172.21.15.92:0/1733441079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800bea0 con 0x7ff3500bd9a0 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff357872700 1 -- 172.21.15.92:0/1733441079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3500bd9a0 msgr2=0x7ff35015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.009+0000 7ff357872700 1 --2- 172.21.15.92:0/1733441079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3500bd9a0 0x7ff35015bd80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7ff348009e60 tx=0x7ff348009780).stop 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.010+0000 7ff357872700 1 -- 172.21.15.92:0/1733441079 shutdown_connections 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.010+0000 7ff357872700 1 --2- 172.21.15.92:0/1733441079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3500bd9a0 0x7ff35015bd80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.010+0000 7ff357872700 1 -- 172.21.15.92:0/1733441079 >> 172.21.15.92:0/1733441079 conn(0x7ff3500c4430 msgr2=0x7ff3500c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.010+0000 7ff357872700 1 -- 172.21.15.92:0/1733441079 shutdown_connections 2023-05-16T07:17:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.010+0000 7ff357872700 1 -- 172.21.15.92:0/1733441079 wait complete. 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.011+0000 7ff357872700 1 Processor -- start 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.011+0000 7ff357872700 1 -- start start 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff350068060 0x7ff3500ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3500bd9a0 0x7ff3501387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 0x7ff3500ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff350138cc0 con 0x7ff35013a5a0 2023-05-16T07:17:15.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500ac530 con 0x7ff3500bd9a0 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500ab920 con 0x7ff350068060 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 0x7ff3500ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 0x7ff3500ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45138/0 (socket says 172.21.15.92:45138) 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 -- 172.21.15.92:0/1058044582 learned_addr learned my addr 172.21.15.92:0/1058044582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 -- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff350068060 msgr2=0x7ff3500ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff350068060 0x7ff3500ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 -- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3500bd9a0 msgr2=0x7ff3501387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:15.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3500bd9a0 0x7ff3501387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 -- 172.21.15.92:0/1058044582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348009a20 con 0x7ff35013a5a0 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff355e0f700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 0x7ff3500ac9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7ff34800b6c0 tx=0x7ff348009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34801d630 con 0x7ff35013a5a0 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34801d790 con 0x7ff35013a5a0 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34801db30 con 0x7ff35013a5a0 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3500abc00 con 0x7ff35013a5a0 2023-05-16T07:17:15.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.012+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3500ac1c0 con 0x7ff35013a5a0 2023-05-16T07:17:15.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.013+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35006c350 con 0x7ff35013a5a0 2023-05-16T07:17:15.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.014+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff348026410 con 0x7ff35013a5a0 2023-05-16T07:17:15.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.014+0000 7ff3417fa700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff33405ed90 0x7ff334061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:15.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.014+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6283+0+0 (secure 0 0 0) 0x7ff34807bd40 con 0x7ff35013a5a0 2023-05-16T07:17:15.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.014+0000 7ff35560e700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff33405ed90 0x7ff334061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:15.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.014+0000 7ff35560e700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff33405ed90 0x7ff334061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff344002510 tx=0x7ff3440079a0).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:15.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.018+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff34804d1e0 con 0x7ff35013a5a0 2023-05-16T07:17:15.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:15.195+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff350068e70 con 0x7ff35013a5a0 2023-05-16T07:17:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.119+0000 7ff3417fa700 1 -- 172.21.15.92:0/1058044582 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff348050800 con 0x7ff35013a5a0 2023-05-16T07:17:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff33405ed90 msgr2=0x7ff334061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff33405ed90 0x7ff334061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff344002510 tx=0x7ff3440079a0).stop 2023-05-16T07:17:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 msgr2=0x7ff3500ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 0x7ff3500ac9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7ff34800b6c0 tx=0x7ff348009780).stop 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 shutdown_connections 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff33405ed90 0x7ff334061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff350068060 0x7ff3500ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff35013a5a0 0x7ff3500ac9c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 --2- 172.21.15.92:0/1058044582 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3500bd9a0 0x7ff3501387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 >> 172.21.15.92:0/1058044582 conn(0x7ff3500c4430 msgr2=0x7ff350139460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.121+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 shutdown_connections 2023-05-16T07:17:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.122+0000 7ff357872700 1 -- 172.21.15.92:0/1058044582 wait complete. 2023-05-16T07:17:16.127 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:17:16.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:17:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.253+0000 7fcd17581700 1 Processor -- start 2023-05-16T07:17:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.253+0000 7fcd17581700 1 -- start start 2023-05-16T07:17:16.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd17581700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd10153eb0 0x7fcd10154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd17581700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10068c70 0x7fcd10066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd17581700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcd10065000 0x7fcd100654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd17581700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100659a0 con 0x7fcd10153eb0 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd17581700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100bc9b0 con 0x7fcd10065000 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd17581700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100bcc90 con 0x7fcd10068c70 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd1531d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd10153eb0 0x7fcd10154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd1531d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd10153eb0 0x7fcd10154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45146/0 (socket says 172.21.15.92:45146) 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.254+0000 7fcd1531d700 1 -- 172.21.15.92:0/2653272757 learned_addr learned my addr 172.21.15.92:0/2653272757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd15b1e700 1 --2- 172.21.15.92:0/2653272757 >> v2:172.21.15.158:3300/0 conn(0x7fcd10065000 0x7fcd100654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd14b1c700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10068c70 0x7fcd10066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd1531d700 1 -- 172.21.15.92:0/2653272757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10068c70 msgr2=0x7fcd10066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd1531d700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10068c70 0x7fcd10066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd1531d700 1 -- 172.21.15.92:0/2653272757 >> v2:172.21.15.158:3300/0 conn(0x7fcd10065000 msgr2=0x7fcd100654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:16.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd1531d700 1 --2- 172.21.15.92:0/2653272757 >> v2:172.21.15.158:3300/0 conn(0x7fcd10065000 0x7fcd100654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd1531d700 1 -- 172.21.15.92:0/2653272757 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100bcf70 con 0x7fcd10153eb0 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd15b1e700 1 --2- 172.21.15.92:0/2653272757 >> v2:172.21.15.158:3300/0 conn(0x7fcd10065000 0x7fcd100654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd14b1c700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10068c70 0x7fcd10066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd1531d700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10153eb0 0x7fcd10154290 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fcd1005f9e0 tx=0x7fcd00009840).ready entity=mon.0 client_cookie=2d980abf7b73cbec server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd0ffff700 1 -- 172.21.15.92:0/2653272757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000e070 con 0x7fcd10153eb0 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd0ffff700 1 -- 172.21.15.92:0/2653272757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0000ba50 con 0x7fcd10153eb0 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd0ffff700 1 -- 172.21.15.92:0/2653272757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000bdd0 con 0x7fcd10153eb0 2023-05-16T07:17:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 -- 172.21.15.92:0/2653272757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10153eb0 msgr2=0x7fcd10154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10153eb0 0x7fcd10154290 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fcd1005f9e0 tx=0x7fcd00009840).stop 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 -- 172.21.15.92:0/2653272757 shutdown_connections 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 --2- 172.21.15.92:0/2653272757 >> v2:172.21.15.158:3300/0 conn(0x7fcd10065000 0x7fcd100654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10153eb0 0x7fcd10154290 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 --2- 172.21.15.92:0/2653272757 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10068c70 0x7fcd10066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.255+0000 7fcd17581700 1 -- 172.21.15.92:0/2653272757 >> 172.21.15.92:0/2653272757 conn(0x7fcd100c4430 msgr2=0x7fcd100c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.256+0000 7fcd17581700 1 -- 172.21.15.92:0/2653272757 shutdown_connections 2023-05-16T07:17:16.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.256+0000 7fcd17581700 1 -- 172.21.15.92:0/2653272757 wait complete. 2023-05-16T07:17:16.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.257+0000 7fcd17581700 1 Processor -- start 2023-05-16T07:17:16.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.257+0000 7fcd17581700 1 -- start start 2023-05-16T07:17:16.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 0x7fcd100be710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10153eb0 0x7fcd100ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcd10148450 0x7fcd100c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100ad540 con 0x7fcd10065000 2023-05-16T07:17:16.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100aca00 con 0x7fcd10148450 2023-05-16T07:17:16.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100acca0 con 0x7fcd10153eb0 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 0x7fcd100be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 0x7fcd100be710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45152/0 (socket says 172.21.15.92:45152) 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 -- 172.21.15.92:0/4060155361 learned_addr learned my addr 172.21.15.92:0/4060155361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd14b1c700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10153eb0 0x7fcd100ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 -- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10153eb0 msgr2=0x7fcd100ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10153eb0 0x7fcd100ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 -- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcd10148450 msgr2=0x7fcd100c0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcd10148450 0x7fcd100c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 -- 172.21.15.92:0/4060155361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009ae0 con 0x7fcd10065000 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd1531d700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 0x7fcd100be710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fcd00009fd0 tx=0x7fcd00009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000bdd0 con 0x7fcd10065000 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00001010 con 0x7fcd10065000 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0001d670 con 0x7fcd10065000 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd100acf80 con 0x7fcd10065000 2023-05-16T07:17:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.258+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd100beeb0 con 0x7fcd10065000 2023-05-16T07:17:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.259+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd100bed10 con 0x7fcd10065000 2023-05-16T07:17:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.259+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcd00024030 con 0x7fcd10065000 2023-05-16T07:17:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.260+0000 7fcd0d7fa700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fccfc05ed90 0x7fccfc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.260+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcd0007ae80 con 0x7fcd10065000 2023-05-16T07:17:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.260+0000 7fcd14b1c700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fccfc05ed90 0x7fccfc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.260+0000 7fcd14b1c700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fccfc05ed90 0x7fccfc061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcd040088c0 tx=0x7fcd04008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.264+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcd100bed10 con 0x7fcd10065000 2023-05-16T07:17:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.489+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcd100bed10 con 0x7fcd10065000 2023-05-16T07:17:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.490+0000 7fcd0d7fa700 1 -- 172.21.15.92:0/4060155361 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fcd100bed10 con 0x7fcd10065000 2023-05-16T07:17:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:16.495 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:17:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:17:16.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:17:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:17:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.496 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.497 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:16.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:17:16.500 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:17:16.503 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.504 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:17:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:17:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:17:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:17:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:17:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:17:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:17:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:17:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:17:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:17:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:17:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:17:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:17:16.522 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:17:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fccfc05ed90 msgr2=0x7fccfc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fccfc05ed90 0x7fccfc061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcd040088c0 tx=0x7fcd04008040).stop 2023-05-16T07:17:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 msgr2=0x7fcd100be710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 0x7fcd100be710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fcd00009fd0 tx=0x7fcd00009840).stop 2023-05-16T07:17:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 shutdown_connections 2023-05-16T07:17:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fccfc05ed90 0x7fccfc061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcd10148450 0x7fcd100c0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd10065000 0x7fcd100be710 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 --2- 172.21.15.92:0/4060155361 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcd10153eb0 0x7fcd100ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 >> 172.21.15.92:0/4060155361 conn(0x7fcd100c4430 msgr2=0x7fcd100a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:16.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 shutdown_connections 2023-05-16T07:17:16.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.492+0000 7fcd17581700 1 -- 172.21.15.92:0/4060155361 wait complete. 2023-05-16T07:17:16.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:16.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.636+0000 7fdf4bfff700 1 Processor -- start 2023-05-16T07:17:16.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.636+0000 7fdf4bfff700 1 -- start start 2023-05-16T07:17:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf4c0ad260 0x7fdf4c14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4bfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c158970 con 0x7fdf4c0c7760 2023-05-16T07:17:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4bfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0ad6e0 con 0x7fdf4c0ad260 2023-05-16T07:17:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4bfff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c155340 con 0x7fdf4c0c7380 2023-05-16T07:17:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf4c0ad260 0x7fdf4c14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4a7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdf4c0c7760 0x7fdf4c0acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55910/0 (socket says 172.21.15.92:55910) 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4a7fc700 1 -- 172.21.15.92:0/3772987981 learned_addr learned my addr 172.21.15.92:0/3772987981 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4affd700 1 --2- 172.21.15.92:0/3772987981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3772987981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 msgr2=0x7fdf4c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4b7fe700 1 --2- 172.21.15.92:0/3772987981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3772987981 >> v1:172.21.15.92:6789/0 conn(0x7fdf4c0c7760 legacy=0x7fdf4c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf49ffb700 1 -- 172.21.15.92:0/3772987981 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229096662 0 0) 0x7fdf4c158970 con 0x7fdf4c0c7760 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3772987981 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c157d60 con 0x7fdf4c0ad260 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf4b7fe700 1 --2- 172.21.15.92:0/3772987981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c14dd70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fdf44009760 tx=0x7fdf44009060).ready entity=mon.0 client_cookie=76a749a6e70d813a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf49ffb700 1 -- 172.21.15.92:0/3772987981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400b580 con 0x7fdf4c0ad260 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.637+0000 7fdf49ffb700 1 -- 172.21.15.92:0/3772987981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b6e0 con 0x7fdf4c0ad260 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf49ffb700 1 -- 172.21.15.92:0/3772987981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400ba60 con 0x7fdf4c0ad260 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3772987981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0ad260 msgr2=0x7fdf4c14dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3772987981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c14dd70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fdf44009760 tx=0x7fdf44009060).stop 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3772987981 shutdown_connections 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3772987981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3772987981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c14dd70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3772987981 >> 172.21.15.92:0/3772987981 conn(0x7fdf4c068330 msgr2=0x7fdf4c068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3772987981 shutdown_connections 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.638+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3772987981 wait complete. 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.639+0000 7fdf4bfff700 1 Processor -- start 2023-05-16T07:17:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.639+0000 7fdf4bfff700 1 -- start start 2023-05-16T07:17:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4bfff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 0x7fdf4c149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4bfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0ae950 con 0x7fdf4c0c7760 2023-05-16T07:17:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4bfff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0adaa0 con 0x7fdf4c0ad260 2023-05-16T07:17:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4bfff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0add40 con 0x7fdf4c0c7380 2023-05-16T07:17:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 0x7fdf4c149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 0x7fdf4c149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45168/0 (socket says 172.21.15.92:45168) 2023-05-16T07:17:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3768140045 learned_addr learned my addr 172.21.15.92:0/3768140045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4affd700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4a7fc700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 msgr2=0x7fdf4c146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf4c0ad260 msgr2=0x7fdf4c0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c0b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 -- 172.21.15.92:0/3768140045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440092a0 con 0x7fdf4c0c7760 2023-05-16T07:17:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.640+0000 7fdf4b7fe700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 0x7fdf4c149550 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fdf4c083bd0 tx=0x7fdf44017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.641+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400b850 con 0x7fdf4c0c7760 2023-05-16T07:17:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.641+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b9b0 con 0x7fdf4c0c7760 2023-05-16T07:17:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.641+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c0ae020 con 0x7fdf4c0c7760 2023-05-16T07:17:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.641+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4401b620 con 0x7fdf4c0c7760 2023-05-16T07:17:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.641+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c0ae5e0 con 0x7fdf4c0c7760 2023-05-16T07:17:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.641+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c047150 con 0x7fdf4c0c7760 2023-05-16T07:17:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.642+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdf4402b070 con 0x7fdf4c0c7760 2023-05-16T07:17:16.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.642+0000 7fdf42ffd700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2c05ed90 0x7fdf2c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.642+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdf4407ba90 con 0x7fdf4c0c7760 2023-05-16T07:17:16.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.642+0000 7fdf4affd700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2c05ed90 0x7fdf2c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.643+0000 7fdf4affd700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2c05ed90 0x7fdf2c061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdf3c0075a0 tx=0x7fdf3c0097a0).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.646+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdf4404d050 con 0x7fdf4c0c7760 2023-05-16T07:17:16.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.810+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf4c0bfb90 con 0x7fdf4c0c7760 2023-05-16T07:17:16.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.811+0000 7fdf42ffd700 1 -- 172.21.15.92:0/3768140045 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+14642 (secure 0 0 0) 0x7fdf44029020 con 0x7fdf4c0c7760 2023-05-16T07:17:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:16.815 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:16.108683+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2c05ed90 msgr2=0x7fdf2c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2c05ed90 0x7fdf2c061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdf3c0075a0 tx=0x7fdf3c0097a0).stop 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 msgr2=0x7fdf4c149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 0x7fdf4c149550 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fdf4c083bd0 tx=0x7fdf44017000).stop 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 shutdown_connections 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdf2c05ed90 0x7fdf2c061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf4c0c7760 0x7fdf4c149550 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf4c0ad260 0x7fdf4c0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 --2- 172.21.15.92:0/3768140045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdf4c0c7380 0x7fdf4c146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 >> 172.21.15.92:0/3768140045 conn(0x7fdf4c068330 msgr2=0x7fdf4c0c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 shutdown_connections 2023-05-16T07:17:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.813+0000 7fdf4bfff700 1 -- 172.21.15.92:0/3768140045 wait complete. 2023-05-16T07:17:16.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:17:16.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:17:16.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.942+0000 7f159ffff700 1 Processor -- start 2023-05-16T07:17:16.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.942+0000 7f159ffff700 1 -- start start 2023-05-16T07:17:16.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159ffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0150920 0x7f15a00a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15a0150d00 0x7f15a01519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f15a0151ea0 0x7f15a0062980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159ffff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a0151520 con 0x7f15a0150d00 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159ffff700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00a7aa0 con 0x7f15a0151ea0 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159ffff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00a8480 con 0x7f15a0150920 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15a0150d00 0x7f15a01519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15a0150d00 0x7f15a01519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45174/0 (socket says 172.21.15.92:45174) 2023-05-16T07:17:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 -- 172.21.15.92:0/1985640951 learned_addr learned my addr 172.21.15.92:0/1985640951 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159effd700 1 --2- 172.21.15.92:0/1985640951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0150920 0x7f15a00a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159f7fe700 1 --2- 172.21.15.92:0/1985640951 >> v2:172.21.15.158:3300/0 conn(0x7f15a0151ea0 0x7f15a0062980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 -- 172.21.15.92:0/1985640951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0150920 msgr2=0x7f15a00a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 --2- 172.21.15.92:0/1985640951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0150920 0x7f15a00a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 -- 172.21.15.92:0/1985640951 >> v2:172.21.15.158:3300/0 conn(0x7f15a0151ea0 msgr2=0x7f15a0062980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 --2- 172.21.15.92:0/1985640951 >> v2:172.21.15.158:3300/0 conn(0x7f15a0151ea0 0x7f15a0062980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 -- 172.21.15.92:0/1985640951 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00a8760 con 0x7f15a0150d00 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f1597fff700 1 --2- 172.21.15.92:0/1985640951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a01519b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f15a0064650 tx=0x7f15880090f0).ready entity=mon.0 client_cookie=cfd4f6200d84eaec server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159e7fc700 1 -- 172.21.15.92:0/1985640951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800b600 con 0x7f15a0150d00 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159e7fc700 1 -- 172.21.15.92:0/1985640951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158800b760 con 0x7f15a0150d00 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.943+0000 7f159e7fc700 1 -- 172.21.15.92:0/1985640951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800bae0 con 0x7f15a0150d00 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 -- 172.21.15.92:0/1985640951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 msgr2=0x7f15a01519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 --2- 172.21.15.92:0/1985640951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a01519b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f15a0064650 tx=0x7f15880090f0).stop 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 -- 172.21.15.92:0/1985640951 shutdown_connections 2023-05-16T07:17:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 --2- 172.21.15.92:0/1985640951 >> v2:172.21.15.158:3300/0 conn(0x7f15a0151ea0 0x7f15a0062980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 --2- 172.21.15.92:0/1985640951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0150920 0x7f15a00a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 --2- 172.21.15.92:0/1985640951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a01519b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 -- 172.21.15.92:0/1985640951 >> 172.21.15.92:0/1985640951 conn(0x7f15a0068a30 msgr2=0x7f15a00c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 -- 172.21.15.92:0/1985640951 shutdown_connections 2023-05-16T07:17:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.944+0000 7f159ffff700 1 -- 172.21.15.92:0/1985640951 wait complete. 2023-05-16T07:17:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.945+0000 7f159ffff700 1 Processor -- start 2023-05-16T07:17:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.945+0000 7f159ffff700 1 -- start start 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a0150920 0x7f15a01425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a0147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0151ea0 0x7f15a0145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0146ab0 con 0x7f15a0150d00 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0146d90 con 0x7f15a0150920 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0147070 con 0x7f15a0151ea0 2023-05-16T07:17:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a0147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a0147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45180/0 (socket says 172.21.15.92:45180) 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 -- 172.21.15.92:0/206808815 learned_addr learned my addr 172.21.15.92:0/206808815 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159effd700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a0150920 0x7f15a01425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159f7fe700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0151ea0 0x7f15a0145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 -- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0151ea0 msgr2=0x7f15a0145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0151ea0 0x7f15a0145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 -- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a0150920 msgr2=0x7f15a01425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a0150920 0x7f15a01425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 -- 172.21.15.92:0/206808815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1588009370 con 0x7f15a0150d00 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159effd700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a0150920 0x7f15a01425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1597fff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a0147b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f1588009130 tx=0x7f1588017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800b8d0 con 0x7f15a0150d00 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158800ba50 con 0x7f15a0150d00 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a0147350 con 0x7f15a0150d00 2023-05-16T07:17:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158801c670 con 0x7f15a0150d00 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.946+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a0142a90 con 0x7f15a0150d00 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.947+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a0047150 con 0x7f15a0150d00 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.948+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f158802c070 con 0x7f15a0150d00 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.948+0000 7f1596ffd700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f158005ed90 0x7f1580061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.948+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f158807b600 con 0x7f15a0150d00 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.948+0000 7f159effd700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f158005ed90 0x7f1580061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.948+0000 7f159effd700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f158005ed90 0x7f1580061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f1590006fd0 tx=0x7f1590008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:16.952+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f15880506e0 con 0x7f15a0150d00 2023-05-16T07:17:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.112+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f15a00a8a40 con 0x7f15a0150d00 2023-05-16T07:17:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.112+0000 7f1596ffd700 1 -- 172.21.15.92:0/206808815 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v75) v1 ==== 110+0+10 (secure 0 0 0) 0x7f158802a0b0 con 0x7f15a0150d00 2023-05-16T07:17:17.116 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:17:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f158005ed90 msgr2=0x7f1580061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f158005ed90 0x7f1580061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f1590006fd0 tx=0x7f1590008040).stop 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 msgr2=0x7f15a0147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a0147b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f1588009130 tx=0x7f1588017000).stop 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 shutdown_connections 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f158005ed90 0x7f1580061250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a0151ea0 0x7f15a0145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a0150920 0x7f15a01425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 --2- 172.21.15.92:0/206808815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a0150d00 0x7f15a0147b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 >> 172.21.15.92:0/206808815 conn(0x7f15a0068a30 msgr2=0x7f15a0065690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 shutdown_connections 2023-05-16T07:17:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.114+0000 7f159ffff700 1 -- 172.21.15.92:0/206808815 wait complete. 2023-05-16T07:17:17.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:17:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.241+0000 7fb70bc2a700 1 Processor -- start 2023-05-16T07:17:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.241+0000 7fb70bc2a700 1 -- start start 2023-05-16T07:17:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70bc2a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb704154090 0x7fb7040aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70bc2a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb704154470 0x7fb7040a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70bc2a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7040c94d0 con 0x7fb7040aa150 2023-05-16T07:17:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70bc2a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7040aa5a0 con 0x7fb704154470 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70bc2a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7040cb7e0 con 0x7fb704154090 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7091c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb704154470 0x7fb7040a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70a1c7700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb7040aa150 0x7fb704068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55938/0 (socket says 172.21.15.92:55938) 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb70a1c7700 1 -- 172.21.15.92:0/3388774298 learned_addr learned my addr 172.21.15.92:0/3388774298 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7099c6700 1 --2- 172.21.15.92:0/3388774298 >> v2:172.21.15.158:3300/0 conn(0x7fb704154090 0x7fb7040aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7091c5700 1 -- 172.21.15.92:0/3388774298 >> v2:172.21.15.158:3300/0 conn(0x7fb704154090 msgr2=0x7fb7040aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7091c5700 1 --2- 172.21.15.92:0/3388774298 >> v2:172.21.15.158:3300/0 conn(0x7fb704154090 0x7fb7040aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7091c5700 1 -- 172.21.15.92:0/3388774298 >> v1:172.21.15.92:6789/0 conn(0x7fb7040aa150 legacy=0x7fb704068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7091c5700 1 -- 172.21.15.92:0/3388774298 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040c88c0 con 0x7fb704154470 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7089c4700 1 -- 172.21.15.92:0/3388774298 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894014304 0 0) 0x7fb7040c94d0 con 0x7fb7040aa150 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7099c6700 1 --2- 172.21.15.92:0/3388774298 >> v2:172.21.15.158:3300/0 conn(0x7fb704154090 0x7fb7040aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.242+0000 7fb7091c5700 1 --2- 172.21.15.92:0/3388774298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb704154470 0x7fb7040a9c60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7fb70406bd10 tx=0x7fb6f4009060).ready entity=mon.0 client_cookie=51315dc53b428579 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb7089c4700 1 -- 172.21.15.92:0/3388774298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400b580 con 0x7fb704154470 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb7089c4700 1 -- 172.21.15.92:0/3388774298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f400b6e0 con 0x7fb704154470 2023-05-16T07:17:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb7089c4700 1 -- 172.21.15.92:0/3388774298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400ba60 con 0x7fb704154470 2023-05-16T07:17:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 -- 172.21.15.92:0/3388774298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb704154470 msgr2=0x7fb7040a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/3388774298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb704154470 0x7fb7040a9c60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7fb70406bd10 tx=0x7fb6f4009060).stop 2023-05-16T07:17:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 -- 172.21.15.92:0/3388774298 shutdown_connections 2023-05-16T07:17:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/3388774298 >> v2:172.21.15.158:3300/0 conn(0x7fb704154090 0x7fb7040aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/3388774298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb704154470 0x7fb7040a9c60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 -- 172.21.15.92:0/3388774298 >> 172.21.15.92:0/3388774298 conn(0x7fb704067920 msgr2=0x7fb7040cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 -- 172.21.15.92:0/3388774298 shutdown_connections 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.243+0000 7fb70bc2a700 1 -- 172.21.15.92:0/3388774298 wait complete. 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.244+0000 7fb70bc2a700 1 Processor -- start 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.244+0000 7fb70bc2a700 1 -- start start 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70bc2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 0x7fb70414d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70bc2a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb704154090 0x7fb7040adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70bc2a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb704154470 0x7fb7040c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70bc2a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040ae5f0 con 0x7fb7040aa150 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70bc2a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040ae8d0 con 0x7fb704154470 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70bc2a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040aebb0 con 0x7fb704154090 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7099c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 0x7fb70414d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7099c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 0x7fb70414d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45192/0 (socket says 172.21.15.92:45192) 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7099c6700 1 -- 172.21.15.92:0/1665001847 learned_addr learned my addr 172.21.15.92:0/1665001847 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7091c5700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb704154090 0x7fb7040adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb70a1c7700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb704154470 0x7fb7040c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7099c6700 1 -- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb704154090 msgr2=0x7fb7040adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7099c6700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb704154090 0x7fb7040adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.245+0000 7fb7099c6700 1 -- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb704154470 msgr2=0x7fb7040c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb7099c6700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb704154470 0x7fb7040c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb7099c6700 1 -- 172.21.15.92:0/1665001847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40092a0 con 0x7fb7040aa150 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb7091c5700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb704154090 0x7fb7040adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb7099c6700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 0x7fb70414d970 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fb6fc00a400 tx=0x7fb6fc007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ae20 con 0x7fb7040aa150 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00d650 con 0x7fb7040aa150 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7040aee90 con 0x7fb7040aa150 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00d9d0 con 0x7fb7040aa150 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.246+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70414de60 con 0x7fb7040aa150 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.247+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7040c2fc0 con 0x7fb7040aa150 2023-05-16T07:17:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.247+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb6fc01d060 con 0x7fb7040aa150 2023-05-16T07:17:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.247+0000 7fb6f9ffb700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6e805ed90 0x7fb6e8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.247+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb6fc07a750 con 0x7fb7040aa150 2023-05-16T07:17:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.247+0000 7fb7091c5700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6e805ed90 0x7fb6e8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.248+0000 7fb7091c5700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6e805ed90 0x7fb6e8061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb6f4009fd0 tx=0x7fb6f4017000).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.251+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb6fc04be00 con 0x7fb7040aa150 2023-05-16T07:17:17.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.412+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb704066730 con 0x7fb7040aa150 2023-05-16T07:17:17.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.412+0000 7fb6f9ffb700 1 -- 172.21.15.92:0/1665001847 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb6fc01b020 con 0x7fb7040aa150 2023-05-16T07:17:17.416 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:17:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6e805ed90 msgr2=0x7fb6e8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6e805ed90 0x7fb6e8061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb6f4009fd0 tx=0x7fb6f4017000).stop 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 msgr2=0x7fb70414d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 0x7fb70414d970 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fb6fc00a400 tx=0x7fb6fc007910).stop 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 shutdown_connections 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6e805ed90 0x7fb6e8061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb704154470 0x7fb7040c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7040aa150 0x7fb70414d970 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 --2- 172.21.15.92:0/1665001847 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb704154090 0x7fb7040adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 >> 172.21.15.92:0/1665001847 conn(0x7fb704067920 msgr2=0x7fb70414e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 shutdown_connections 2023-05-16T07:17:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.414+0000 7fb70bc2a700 1 -- 172.21.15.92:0/1665001847 wait complete. 2023-05-16T07:17:17.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:17:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.535+0000 7ff9d60be700 1 Processor -- start 2023-05-16T07:17:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.536+0000 7ff9d60be700 1 -- start start 2023-05-16T07:17:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.536+0000 7ff9d60be700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9d0154e50 0x7ff9d00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.536+0000 7ff9d60be700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff9d00ad040 0x7ff9d014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.536+0000 7ff9d60be700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d0158970 con 0x7ff9d00c7380 2023-05-16T07:17:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.536+0000 7ff9d60be700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d0155300 con 0x7ff9d0154e50 2023-05-16T07:17:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.536+0000 7ff9d60be700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d0157d60 con 0x7ff9d00ad040 2023-05-16T07:17:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d48bb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9d0154e50 0x7ff9d00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d50bc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff9d00c7380 0x7ff9d00c7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55948/0 (socket says 172.21.15.92:55948) 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d50bc700 1 -- 172.21.15.92:0/1996386852 learned_addr learned my addr 172.21.15.92:0/1996386852 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d58bd700 1 --2- 172.21.15.92:0/1996386852 >> v2:172.21.15.158:3300/0 conn(0x7ff9d00ad040 0x7ff9d014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d48bb700 1 -- 172.21.15.92:0/1996386852 >> v2:172.21.15.158:3300/0 conn(0x7ff9d00ad040 msgr2=0x7ff9d014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d48bb700 1 --2- 172.21.15.92:0/1996386852 >> v2:172.21.15.158:3300/0 conn(0x7ff9d00ad040 0x7ff9d014dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d48bb700 1 -- 172.21.15.92:0/1996386852 >> v1:172.21.15.92:6789/0 conn(0x7ff9d00c7380 legacy=0x7ff9d00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d48bb700 1 -- 172.21.15.92:0/1996386852 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0158040 con 0x7ff9d0154e50 2023-05-16T07:17:17.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d58bd700 1 --2- 172.21.15.92:0/1996386852 >> v2:172.21.15.158:3300/0 conn(0x7ff9d00ad040 0x7ff9d014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:17.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9d48bb700 1 --2- 172.21.15.92:0/1996386852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00acb50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7ff9d005eea0 tx=0x7ff9c4009060).ready entity=mon.0 client_cookie=14b2ad2f6a45b51d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9cffff700 1 -- 172.21.15.92:0/1996386852 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400b580 con 0x7ff9d0154e50 2023-05-16T07:17:17.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9cffff700 1 -- 172.21.15.92:0/1996386852 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c400b6e0 con 0x7ff9d0154e50 2023-05-16T07:17:17.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.537+0000 7ff9cffff700 1 -- 172.21.15.92:0/1996386852 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400ba60 con 0x7ff9d0154e50 2023-05-16T07:17:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 -- 172.21.15.92:0/1996386852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 msgr2=0x7ff9d00acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 --2- 172.21.15.92:0/1996386852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00acb50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7ff9d005eea0 tx=0x7ff9c4009060).stop 2023-05-16T07:17:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 -- 172.21.15.92:0/1996386852 shutdown_connections 2023-05-16T07:17:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 --2- 172.21.15.92:0/1996386852 >> v2:172.21.15.158:3300/0 conn(0x7ff9d00ad040 0x7ff9d014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 --2- 172.21.15.92:0/1996386852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00acb50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 -- 172.21.15.92:0/1996386852 >> 172.21.15.92:0/1996386852 conn(0x7ff9d0068330 msgr2=0x7ff9d0068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 -- 172.21.15.92:0/1996386852 shutdown_connections 2023-05-16T07:17:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.538+0000 7ff9d60be700 1 -- 172.21.15.92:0/1996386852 wait complete. 2023-05-16T07:17:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.539+0000 7ff9d60be700 1 Processor -- start 2023-05-16T07:17:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.539+0000 7ff9d60be700 1 -- start start 2023-05-16T07:17:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d60be700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff9d00ad040 0x7ff9d00c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d60be700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff9d00c7380 0x7ff9d0164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d60be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d60be700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00adb40 con 0x7ff9d0154e50 2023-05-16T07:17:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d60be700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00ade20 con 0x7ff9d00c7380 2023-05-16T07:17:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d60be700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00ae100 con 0x7ff9d00ad040 2023-05-16T07:17:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45210/0 (socket says 172.21.15.92:45210) 2023-05-16T07:17:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 -- 172.21.15.92:0/4241999391 learned_addr learned my addr 172.21.15.92:0/4241999391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 -- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff9d00ad040 msgr2=0x7ff9d00c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d50bc700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff9d00ad040 0x7ff9d00c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d48bb700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff9d00c7380 0x7ff9d0164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff9d00ad040 0x7ff9d00c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 -- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff9d00c7380 msgr2=0x7ff9d0164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff9d00c7380 0x7ff9d0164e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 -- 172.21.15.92:0/4241999391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40092a0 con 0x7ff9d0154e50 2023-05-16T07:17:17.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9d58bd700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00c3120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7ff9c800d860 tx=0x7ff9c8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.540+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80125f0 con 0x7ff9d0154e50 2023-05-16T07:17:17.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.541+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8012750 con 0x7ff9d0154e50 2023-05-16T07:17:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.541+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d00ae3e0 con 0x7ff9d0154e50 2023-05-16T07:17:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.541+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8012ad0 con 0x7ff9d0154e50 2023-05-16T07:17:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.541+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d0165330 con 0x7ff9d0154e50 2023-05-16T07:17:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.542+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d0047150 con 0x7ff9d0154e50 2023-05-16T07:17:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.542+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff9c8013480 con 0x7ff9d0154e50 2023-05-16T07:17:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.542+0000 7ff9cd7fa700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff9bc05ed90 0x7ff9bc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.542+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff9c80799b0 con 0x7ff9d0154e50 2023-05-16T07:17:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.542+0000 7ff9d50bc700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff9bc05ed90 0x7ff9bc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.543+0000 7ff9d50bc700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff9bc05ed90 0x7ff9bc061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ff9c0007fd0 tx=0x7ff9c0009040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.546+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff9c804b000 con 0x7ff9d0154e50 2023-05-16T07:17:17.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.712+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9d00666c0 con 0x7ff9d0154e50 2023-05-16T07:17:17.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.712+0000 7ff9cd7fa700 1 -- 172.21.15.92:0/4241999391 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff9c804e620 con 0x7ff9d0154e50 2023-05-16T07:17:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:17.717 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:17:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:17:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:17:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:17:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:17:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:17.718 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:17:17.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff9bc05ed90 msgr2=0x7ff9bc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff9bc05ed90 0x7ff9bc061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ff9c0007fd0 tx=0x7ff9c0009040).stop 2023-05-16T07:17:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 msgr2=0x7ff9d00c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00c3120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7ff9c800d860 tx=0x7ff9c8008040).stop 2023-05-16T07:17:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 shutdown_connections 2023-05-16T07:17:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff9bc05ed90 0x7ff9bc061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff9d00ad040 0x7ff9d00c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9d0154e50 0x7ff9d00c3120 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 --2- 172.21.15.92:0/4241999391 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff9d00c7380 0x7ff9d0164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 >> 172.21.15.92:0/4241999391 conn(0x7ff9d0068330 msgr2=0x7ff9d0152020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 shutdown_connections 2023-05-16T07:17:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.714+0000 7ff9d60be700 1 -- 172.21.15.92:0/4241999391 wait complete. 2023-05-16T07:17:17.729 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:17:17.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:17:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.843+0000 7fa405dd3700 1 Processor -- start 2023-05-16T07:17:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.843+0000 7fa405dd3700 1 -- start start 2023-05-16T07:17:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa405dd3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa400064a50 0x7fa400068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa405dd3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa400069c60 con 0x7fa400155840 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa405dd3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa400069050 con 0x7fa40006a0f0 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa405dd3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa400135380 con 0x7fa400064a50 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3ff7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa400064a50 0x7fa400068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3fffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa400155840 0x7fa4000bc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55966/0 (socket says 172.21.15.92:55966) 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3fffff700 1 -- 172.21.15.92:0/1987578255 learned_addr learned my addr 172.21.15.92:0/1987578255 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3ff7fe700 1 -- 172.21.15.92:0/1987578255 >> v1:172.21.15.158:6789/0 conn(0x7fa40006a0f0 legacy=0x7fa400066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3ff7fe700 1 -- 172.21.15.92:0/1987578255 >> v1:172.21.15.92:6789/0 conn(0x7fa400155840 legacy=0x7fa4000bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3fe7fc700 1 -- 172.21.15.92:0/1987578255 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493320526 0 0) 0x7fa400069c60 con 0x7fa400155840 2023-05-16T07:17:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.844+0000 7fa3ff7fe700 1 -- 172.21.15.92:0/1987578255 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400069330 con 0x7fa400064a50 2023-05-16T07:17:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa3ff7fe700 1 --2- 172.21.15.92:0/1987578255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400064a50 0x7fa400068a70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fa3f0009e60 tx=0x7fa3f0009780).ready entity=mon.0 client_cookie=88c314745bdd9d5b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa3fe7fc700 1 -- 172.21.15.92:0/1987578255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000e070 con 0x7fa400064a50 2023-05-16T07:17:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa3fe7fc700 1 -- 172.21.15.92:0/1987578255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f000bc30 con 0x7fa400064a50 2023-05-16T07:17:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa3fe7fc700 1 -- 172.21.15.92:0/1987578255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0012410 con 0x7fa400064a50 2023-05-16T07:17:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 -- 172.21.15.92:0/1987578255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400064a50 msgr2=0x7fa400068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 --2- 172.21.15.92:0/1987578255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400064a50 0x7fa400068a70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fa3f0009e60 tx=0x7fa3f0009780).stop 2023-05-16T07:17:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 -- 172.21.15.92:0/1987578255 shutdown_connections 2023-05-16T07:17:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 --2- 172.21.15.92:0/1987578255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400064a50 0x7fa400068a70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 -- 172.21.15.92:0/1987578255 >> 172.21.15.92:0/1987578255 conn(0x7fa4000c4620 msgr2=0x7fa400066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 -- 172.21.15.92:0/1987578255 shutdown_connections 2023-05-16T07:17:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.845+0000 7fa405dd3700 1 -- 172.21.15.92:0/1987578255 wait complete. 2023-05-16T07:17:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.846+0000 7fa405dd3700 1 Processor -- start 2023-05-16T07:17:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.846+0000 7fa405dd3700 1 -- start start 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa405dd3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa400064a50 0x7fa4001627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa405dd3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 0x7fa4000aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa405dd3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa400155840 0x7fa400137a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa405dd3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000ab700 con 0x7fa40006a0f0 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa405dd3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa400162ca0 con 0x7fa400155840 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa405dd3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000aaaf0 con 0x7fa400064a50 2023-05-16T07:17:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa3feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 0x7fa4000aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa3feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 0x7fa4000aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45222/0 (socket says 172.21.15.92:45222) 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa3feffd700 1 -- 172.21.15.92:0/3152247561 learned_addr learned my addr 172.21.15.92:0/3152247561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa3fffff700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa400155840 0x7fa400137a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.847+0000 7fa3ff7fe700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa400064a50 0x7fa4001627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3feffd700 1 -- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa400064a50 msgr2=0x7fa4001627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3feffd700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa400064a50 0x7fa4001627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3feffd700 1 -- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa400155840 msgr2=0x7fa400137a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3feffd700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa400155840 0x7fa400137a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3feffd700 1 -- 172.21.15.92:0/3152247561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009a20 con 0x7fa40006a0f0 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3ff7fe700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa400064a50 0x7fa4001627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3feffd700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 0x7fa4000aa2a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fa3f400c490 tx=0x7fa3f4007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400f470 con 0x7fa40006a0f0 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f400f5d0 con 0x7fa40006a0f0 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4000aadd0 con 0x7fa40006a0f0 2023-05-16T07:17:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400f950 con 0x7fa40006a0f0 2023-05-16T07:17:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.848+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4000ab390 con 0x7fa40006a0f0 2023-05-16T07:17:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.849+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40006c350 con 0x7fa40006a0f0 2023-05-16T07:17:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.849+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa3f4019030 con 0x7fa40006a0f0 2023-05-16T07:17:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.849+0000 7fa3e77fe700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa3ec05ed90 0x7fa3ec061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.849+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa3f4078d50 con 0x7fa40006a0f0 2023-05-16T07:17:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.850+0000 7fa3ff7fe700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa3ec05ed90 0x7fa3ec061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.850+0000 7fa3ff7fe700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa3ec05ed90 0x7fa3ec061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa400151660 tx=0x7fa3f0009780).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:17.853+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa3f40488f0 con 0x7fa40006a0f0 2023-05-16T07:17:18.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.020+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fa40006bb10 con 0x7fa40006a0f0 2023-05-16T07:17:18.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.123+0000 7fa3e77fe700 1 -- 172.21.15.92:0/3152247561 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 152+0+0 (secure 0 0 0) 0x7fa3f404a450 con 0x7fa40006a0f0 2023-05-16T07:17:18.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.127+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa3ec05ed90 msgr2=0x7fa3ec061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:18.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.127+0000 7fa405dd3700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa3ec05ed90 0x7fa3ec061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa400151660 tx=0x7fa3f0009780).stop 2023-05-16T07:17:18.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.127+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 msgr2=0x7fa4000aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:18.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.127+0000 7fa405dd3700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 0x7fa4000aa2a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fa3f400c490 tx=0x7fa3f4007de0).stop 2023-05-16T07:17:18.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.130+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 shutdown_connections 2023-05-16T07:17:18.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.130+0000 7fa405dd3700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa3ec05ed90 0x7fa3ec061250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.130+0000 7fa405dd3700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa400064a50 0x7fa4001627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.130+0000 7fa405dd3700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa400155840 0x7fa400137a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.130+0000 7fa405dd3700 1 --2- 172.21.15.92:0/3152247561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40006a0f0 0x7fa4000aa2a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.130+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 >> 172.21.15.92:0/3152247561 conn(0x7fa4000c4620 msgr2=0x7fa4000c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:18.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.132+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 shutdown_connections 2023-05-16T07:17:18.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.132+0000 7fa405dd3700 1 -- 172.21.15.92:0/3152247561 wait complete. 2023-05-16T07:17:18.137 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:17:18.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:17:18.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.263+0000 7f2715d3a700 1 Processor -- start 2023-05-16T07:17:18.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2715d3a700 1 -- start start 2023-05-16T07:17:18.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2715d3a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f27100a75b0 0x7f2710150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2715d3a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 0x7f2710062e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2715d3a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2710151520 con 0x7f2710150920 2023-05-16T07:17:18.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2715d3a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2710063300 con 0x7f27100a75b0 2023-05-16T07:17:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2715d3a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27101511f0 con 0x7f2710062980 2023-05-16T07:17:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2714d38700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2710150920 0x7f27101519b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55976/0 (socket says 172.21.15.92:55976) 2023-05-16T07:17:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.264+0000 7f2714d38700 1 -- 172.21.15.92:0/4200299411 learned_addr learned my addr 172.21.15.92:0/4200299411 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270ffff700 1 --2- 172.21.15.92:0/4200299411 >> v2:172.21.15.158:3300/0 conn(0x7f27100a75b0 0x7f2710150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2715539700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 0x7f2710062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866304981 0 0) 0x7f2710151520 con 0x7f2710150920 2023-05-16T07:17:18.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26fc0033f0 con 0x7f2710150920 2023-05-16T07:17:18.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089363147 0 0) 0x7f26fc0033f0 con 0x7f2710150920 2023-05-16T07:17:18.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 msgr2=0x7f2710062e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:18.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 0x7f2710062e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 >> v2:172.21.15.158:3300/0 conn(0x7f27100a75b0 msgr2=0x7f2710150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 --2- 172.21.15.92:0/4200299411 >> v2:172.21.15.158:3300/0 conn(0x7f27100a75b0 0x7f2710150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27101520c0 con 0x7f2710150920 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f2700001470 con 0x7f2710150920 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 >> v1:172.21.15.92:6789/0 conn(0x7f2710150920 legacy=0x7f27101519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc0038e0 0x7f26fc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 0x7f26fc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710063300 con 0x7f26fc0038e0 2023-05-16T07:17:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27101511f0 con 0x7f26fc004690 2023-05-16T07:17:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc00b8e0 con 0x7f2710062980 2023-05-16T07:17:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2714d38700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc0038e0 0x7f26fc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270ffff700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2715539700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 0x7f26fc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2714d38700 1 -- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 msgr2=0x7f26fc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2714d38700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710062980 0x7f26fc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2714d38700 1 -- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 msgr2=0x7f26fc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2714d38700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f2714d38700 1 -- 172.21.15.92:0/4200299411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0033f0 con 0x7f26fc0038e0 2023-05-16T07:17:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.265+0000 7f270ffff700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2714d38700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc0038e0 0x7f26fc003d00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f270000c210 tx=0x7f2700001e90).ready entity=mon.0 client_cookie=bab74b75449bbd2e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700007a60 con 0x7f26fc0038e0 2023-05-16T07:17:18.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270000cae0 con 0x7f26fc0038e0 2023-05-16T07:17:18.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f270f7fe700 1 -- 172.21.15.92:0/4200299411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000cea0 con 0x7f26fc0038e0 2023-05-16T07:17:18.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715d3a700 1 -- 172.21.15.92:0/4200299411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc0038e0 msgr2=0x7f26fc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:18.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715d3a700 1 --2- 172.21.15.92:0/4200299411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc0038e0 0x7f26fc003d00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f270000c210 tx=0x7f2700001e90).stop 2023-05-16T07:17:18.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715539700 1 -- 172.21.15.92:0/4200299411 reap_dead start 2023-05-16T07:17:18.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715d3a700 1 -- 172.21.15.92:0/4200299411 shutdown_connections 2023-05-16T07:17:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715d3a700 1 -- 172.21.15.92:0/4200299411 >> 172.21.15.92:0/4200299411 conn(0x7f2710068a30 msgr2=0x7f27100c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715d3a700 1 -- 172.21.15.92:0/4200299411 shutdown_connections 2023-05-16T07:17:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.266+0000 7f2715d3a700 1 -- 172.21.15.92:0/4200299411 wait complete. 2023-05-16T07:17:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.267+0000 7f2715d3a700 1 Processor -- start 2023-05-16T07:17:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 -- start start 2023-05-16T07:17:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f27100c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 0x7f27100a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2710164690 0x7f2710164b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710165000 con 0x7f27100a52d0 2023-05-16T07:17:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100c4e80 con 0x7f2710164690 2023-05-16T07:17:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715d3a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27100a4880 con 0x7f26fc0038e0 2023-05-16T07:17:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f270ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 0x7f27100a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f270ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 0x7f27100a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45242/0 (socket says 172.21.15.92:45242) 2023-05-16T07:17:18.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f270ffff700 1 -- 172.21.15.92:0/923811951 learned_addr learned my addr 172.21.15.92:0/923811951 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:18.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2714d38700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f27100c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.268+0000 7f2715539700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2710164690 0x7f2710164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270ffff700 1 -- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 msgr2=0x7f27100c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270ffff700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f27100c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270ffff700 1 -- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2710164690 msgr2=0x7f2710164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270ffff700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2710164690 0x7f2710164b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270ffff700 1 -- 172.21.15.92:0/923811951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100a4e40 con 0x7f27100a52d0 2023-05-16T07:17:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f2715539700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2710164690 0x7f2710164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270ffff700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 0x7f27100a5750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f26f8010570 tx=0x7f26f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f800e030 con 0x7f27100a52d0 2023-05-16T07:17:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f800f040 con 0x7f27100a52d0 2023-05-16T07:17:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27101651f0 con 0x7f27100a52d0 2023-05-16T07:17:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8015690 con 0x7f27100a52d0 2023-05-16T07:17:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.269+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100a4ad0 con 0x7f27100a52d0 2023-05-16T07:17:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.270+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27100a68f0 con 0x7f27100a52d0 2023-05-16T07:17:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.270+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f26f8015ac0 con 0x7f27100a52d0 2023-05-16T07:17:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.271+0000 7f270cff9700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed60 0x7f26f0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.271+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6223+0+0 (secure 0 0 0) 0x7f26f807a740 con 0x7f27100a52d0 2023-05-16T07:17:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.271+0000 7f2714d38700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed60 0x7f26f0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.271+0000 7f2714d38700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed60 0x7f26f0061220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2700001ed0 tx=0x7f2700015000).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.275+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f27100a68f0 con 0x7f27100a52d0 2023-05-16T07:17:18.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:18.441+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f27100a68f0 con 0x7f27100a52d0 2023-05-16T07:17:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.126+0000 7f270cff9700 1 -- 172.21.15.92:0/923811951 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7f27100a68f0 con 0x7f27100a52d0 2023-05-16T07:17:19.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed60 msgr2=0x7f26f0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed60 0x7f26f0061220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2700001ed0 tx=0x7f2700015000).stop 2023-05-16T07:17:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 msgr2=0x7f27100a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 0x7f27100a5750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f26f8010570 tx=0x7f26f800b040).stop 2023-05-16T07:17:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 shutdown_connections 2023-05-16T07:17:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed60 0x7f26f0061220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f27100c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2710164690 0x7f2710164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 --2- 172.21.15.92:0/923811951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27100a52d0 0x7f27100a5750 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.133+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 >> 172.21.15.92:0/923811951 conn(0x7f2710068a30 msgr2=0x7f27101518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.135+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 shutdown_connections 2023-05-16T07:17:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.135+0000 7f2715d3a700 1 -- 172.21.15.92:0/923811951 wait complete. 2023-05-16T07:17:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:17:19.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:17:19.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 Processor -- start 2023-05-16T07:17:19.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 -- start start 2023-05-16T07:17:19.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c064c90 0x7fbb3c066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbb3c152e90 0x7fbb3c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c068160 con 0x7fbb3c14c700 2023-05-16T07:17:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c067340 con 0x7fbb3c152e90 2023-05-16T07:17:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.279+0000 7fbb41521700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c153320 con 0x7fbb3c064c90 2023-05-16T07:17:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbb3c152e90 0x7fbb3c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c064c90 0x7fbb3c066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbb3c152e90 0x7fbb3c15bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:48886/0 (socket says 172.21.15.92:48886) 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb3bfff700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fbb3c14c700 0x7fbb3c064580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:59596/0 (socket says 172.21.15.92:59596) 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 -- 172.21.15.92:0/2153131210 learned_addr learned my addr 172.21.15.92:0/2153131210 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 -- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c064c90 msgr2=0x7fbb3c066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 --2- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c064c90 0x7fbb3c066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 -- 172.21.15.92:0/2153131210 >> v1:172.21.15.158:6789/0 conn(0x7fbb3c14c700 legacy=0x7fbb3c064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 -- 172.21.15.92:0/2153131210 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c067550 con 0x7fbb3c152e90 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb33fff700 1 --2- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c064c90 0x7fbb3c066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.280+0000 7fbb40d20700 1 --2- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c152e90 0x7fbb3c15bdc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fbb34009f20 tx=0x7fbb34009840).ready entity=mon.1 client_cookie=2725f752be8960cd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/2153131210 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400e070 con 0x7fbb3c152e90 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/2153131210 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3400b9d0 con 0x7fbb3c152e90 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/2153131210 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400bd50 con 0x7fbb3c152e90 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 -- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c152e90 msgr2=0x7fbb3c15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 --2- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c152e90 0x7fbb3c15bdc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fbb34009f20 tx=0x7fbb34009840).stop 2023-05-16T07:17:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 -- 172.21.15.92:0/2153131210 shutdown_connections 2023-05-16T07:17:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 --2- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c152e90 0x7fbb3c15bdc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 --2- 172.21.15.92:0/2153131210 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c064c90 0x7fbb3c066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 -- 172.21.15.92:0/2153131210 >> 172.21.15.92:0/2153131210 conn(0x7fbb3c0c7a80 msgr2=0x7fbb3c0c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 -- 172.21.15.92:0/2153131210 shutdown_connections 2023-05-16T07:17:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.281+0000 7fbb41521700 1 -- 172.21.15.92:0/2153131210 wait complete. 2023-05-16T07:17:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.282+0000 7fbb41521700 1 Processor -- start 2023-05-16T07:17:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.282+0000 7fbb41521700 1 -- start start 2023-05-16T07:17:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb41521700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 0x7fbb3c0a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb41521700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c14c700 0x7fbb3c0a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb41521700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c152e90 0x7fbb3c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb41521700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0a8460 con 0x7fbb3c064c90 2023-05-16T07:17:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb41521700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0a8fe0 con 0x7fbb3c14c700 2023-05-16T07:17:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb41521700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c156790 con 0x7fbb3c152e90 2023-05-16T07:17:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 0x7fbb3c0a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 0x7fbb3c0a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45254/0 (socket says 172.21.15.92:45254) 2023-05-16T07:17:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 -- 172.21.15.92:0/2944188264 learned_addr learned my addr 172.21.15.92:0/2944188264 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb33fff700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c14c700 0x7fbb3c0a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb40d20700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c152e90 0x7fbb3c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 -- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c152e90 msgr2=0x7fbb3c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c152e90 0x7fbb3c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 -- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c14c700 msgr2=0x7fbb3c0a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c14c700 0x7fbb3c0a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 -- 172.21.15.92:0/2944188264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240092a0 con 0x7fbb3c064c90 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb40d20700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c152e90 0x7fbb3c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.283+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 0x7fbb3c0a7f70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fbb2c007fd0 tx=0x7fbb2c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.284+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0072e0 con 0x7fbb3c064c90 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.284+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c007440 con 0x7fbb3c064c90 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.284+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0058f0 con 0x7fbb3c064c90 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.284+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb34009ae0 con 0x7fbb3c064c90 2023-05-16T07:17:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.284+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0a8760 con 0x7fbb3c064c90 2023-05-16T07:17:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.284+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c0a92c0 con 0x7fbb3c064c90 2023-05-16T07:17:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.285+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbb2c010070 con 0x7fbb3c064c90 2023-05-16T07:17:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.285+0000 7fbb38ff9700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1c05ed90 0x7fbb1c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.285+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fbb2c0792f0 con 0x7fbb3c064c90 2023-05-16T07:17:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.285+0000 7fbb33fff700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1c05ed90 0x7fbb1c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.286+0000 7fbb33fff700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1c05ed90 0x7fbb1c061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fbb3c138740 tx=0x7fbb24009060).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.289+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbb2c04a6b0 con 0x7fbb3c064c90 2023-05-16T07:17:19.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.492+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbb3c0471c0 con 0x7fbb3c064c90 2023-05-16T07:17:19.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.498+0000 7fbb38ff9700 1 -- 172.21.15.92:0/2944188264 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v22) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbb3c0471c0 con 0x7fbb3c064c90 2023-05-16T07:17:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.503+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1c05ed90 msgr2=0x7fbb1c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.503+0000 7fbb41521700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1c05ed90 0x7fbb1c061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fbb3c138740 tx=0x7fbb24009060).stop 2023-05-16T07:17:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.503+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 msgr2=0x7fbb3c0a7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.503+0000 7fbb41521700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 0x7fbb3c0a7f70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fbb2c007fd0 tx=0x7fbb2c007df0).stop 2023-05-16T07:17:19.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.504+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 shutdown_connections 2023-05-16T07:17:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.504+0000 7fbb41521700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb1c05ed90 0x7fbb1c061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.504+0000 7fbb41521700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c152e90 0x7fbb3c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.504+0000 7fbb41521700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c064c90 0x7fbb3c0a7f70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.504+0000 7fbb41521700 1 --2- 172.21.15.92:0/2944188264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c14c700 0x7fbb3c0a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.504+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 >> 172.21.15.92:0/2944188264 conn(0x7fbb3c0c7a80 msgr2=0x7fbb3c0ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.510+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 shutdown_connections 2023-05-16T07:17:19.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.510+0000 7fbb41521700 1 -- 172.21.15.92:0/2944188264 wait complete. 2023-05-16T07:17:19.515 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 9 and data pool 10 2023-05-16T07:17:19.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:19.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.637+0000 7fa6610a6700 1 Processor -- start 2023-05-16T07:17:19.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.637+0000 7fa6610a6700 1 -- start start 2023-05-16T07:17:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa6610a6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa65c154550 0x7fa65c0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa6610a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa65c069b90 0x7fa65c0c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa6610a6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c0c0660 con 0x7fa65c154170 2023-05-16T07:17:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa6610a6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c069fc0 con 0x7fa65c069b90 2023-05-16T07:17:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa6610a6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c0bfd30 con 0x7fa65c154550 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa65c069b90 0x7fa65c0c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa65c069b90 0x7fa65c0c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45278/0 (socket says 172.21.15.92:45278) 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 -- 172.21.15.92:0/469475501 learned_addr learned my addr 172.21.15.92:0/469475501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65259c700 1 --2- 172.21.15.92:0/469475501 >> v2:172.21.15.158:3300/0 conn(0x7fa65c154550 0x7fa65c0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 -- 172.21.15.92:0/469475501 >> v2:172.21.15.158:3300/0 conn(0x7fa65c154550 msgr2=0x7fa65c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 --2- 172.21.15.92:0/469475501 >> v2:172.21.15.158:3300/0 conn(0x7fa65c154550 0x7fa65c0696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 -- 172.21.15.92:0/469475501 >> v1:172.21.15.158:6789/0 conn(0x7fa65c154170 legacy=0x7fa65c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65b59e700 1 -- 172.21.15.92:0/469475501 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0c0010 con 0x7fa65c069b90 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.638+0000 7fa65259c700 1 --2- 172.21.15.92:0/469475501 >> v2:172.21.15.158:3300/0 conn(0x7fa65c154550 0x7fa65c0696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa65b59e700 1 --2- 172.21.15.92:0/469475501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0c0af0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fa65c0844e0 tx=0x7fa654009840).ready entity=mon.0 client_cookie=6a138f84ac19feda server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa65a59c700 1 -- 172.21.15.92:0/469475501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400e070 con 0x7fa65c069b90 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa65a59c700 1 -- 172.21.15.92:0/469475501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400ba50 con 0x7fa65c069b90 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa65a59c700 1 -- 172.21.15.92:0/469475501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400bdd0 con 0x7fa65c069b90 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 -- 172.21.15.92:0/469475501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 msgr2=0x7fa65c0c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 --2- 172.21.15.92:0/469475501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0c0af0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fa65c0844e0 tx=0x7fa654009840).stop 2023-05-16T07:17:19.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 -- 172.21.15.92:0/469475501 shutdown_connections 2023-05-16T07:17:19.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 --2- 172.21.15.92:0/469475501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0c0af0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 --2- 172.21.15.92:0/469475501 >> v2:172.21.15.158:3300/0 conn(0x7fa65c154550 0x7fa65c0696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 -- 172.21.15.92:0/469475501 >> 172.21.15.92:0/469475501 conn(0x7fa65c0cb700 msgr2=0x7fa65c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:19.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 -- 172.21.15.92:0/469475501 shutdown_connections 2023-05-16T07:17:19.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.639+0000 7fa6610a6700 1 -- 172.21.15.92:0/469475501 wait complete. 2023-05-16T07:17:19.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.640+0000 7fa6610a6700 1 Processor -- start 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.640+0000 7fa6610a6700 1 -- start start 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa6610a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa6610a6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65c154170 0x7fa65c144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa6610a6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa65c154550 0x7fa65c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa6610a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c144c00 con 0x7fa65c069b90 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa6610a6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c1452d0 con 0x7fa65c154550 2023-05-16T07:17:19.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa6610a6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c145b70 con 0x7fa65c154170 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa65ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa65ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45282/0 (socket says 172.21.15.92:45282) 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa65ad9d700 1 -- 172.21.15.92:0/800564116 learned_addr learned my addr 172.21.15.92:0/800564116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa65259c700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65c154170 0x7fa65c144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.641+0000 7fa65b59e700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa65c154550 0x7fa65c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65ad9d700 1 -- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65c154170 msgr2=0x7fa65c144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65ad9d700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65c154170 0x7fa65c144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65ad9d700 1 -- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa65c154550 msgr2=0x7fa65c146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65ad9d700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa65c154550 0x7fa65c146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65ad9d700 1 -- 172.21.15.92:0/800564116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009ae0 con 0x7fa65c069b90 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65b59e700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa65c154550 0x7fa65c146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa65ad9d700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0afc90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fa64c00c470 tx=0x7fa64c007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00ce90 con 0x7fa65c069b90 2023-05-16T07:17:19.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c00f470 con 0x7fa65c069b90 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c144650 con 0x7fa65c069b90 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00f810 con 0x7fa65c069b90 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.642+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c145520 con 0x7fa65c069b90 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.643+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c145e50 con 0x7fa65c069b90 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.643+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa64c019030 con 0x7fa65c069b90 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.643+0000 7fa6537fe700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ed90 0x7fa63c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.644+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fa64c079030 con 0x7fa65c069b90 2023-05-16T07:17:19.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.644+0000 7fa65259c700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ed90 0x7fa63c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.644+0000 7fa65259c700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ed90 0x7fa63c061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa65c0acf30 tx=0x7fa644008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.648+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa64c0488f0 con 0x7fa65c069b90 2023-05-16T07:17:19.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.810+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa65c0666c0 con 0x7fa65c069b90 2023-05-16T07:17:19.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.811+0000 7fa6537fe700 1 -- 172.21.15.92:0/800564116 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17873 (secure 0 0 0) 0x7fa64c017030 con 0x7fa65c069b90 2023-05-16T07:17:19.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:19.815 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:19.493560+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:18.044089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:18.464963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:19.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.812+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ed90 msgr2=0x7fa63c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.812+0000 7fa6610a6700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ed90 0x7fa63c061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa65c0acf30 tx=0x7fa644008040).stop 2023-05-16T07:17:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.812+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 msgr2=0x7fa65c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.812+0000 7fa6610a6700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0afc90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fa64c00c470 tx=0x7fa64c007dc0).stop 2023-05-16T07:17:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 shutdown_connections 2023-05-16T07:17:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa63c05ed90 0x7fa63c061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa65c154550 0x7fa65c146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c069b90 0x7fa65c0afc90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 --2- 172.21.15.92:0/800564116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa65c154170 0x7fa65c144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 >> 172.21.15.92:0/800564116 conn(0x7fa65c0cb700 msgr2=0x7fa65c0aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:19.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 shutdown_connections 2023-05-16T07:17:19.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.813+0000 7fa6610a6700 1 -- 172.21.15.92:0/800564116 wait complete. 2023-05-16T07:17:19.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:19.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.935+0000 7fed2c86a700 1 Processor -- start 2023-05-16T07:17:19.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.935+0000 7fed2c86a700 1 -- start start 2023-05-16T07:17:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed2c86a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28068f10 0x7fed28150f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed2c86a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed280bc8f0 0x7fed280bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed2c86a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed280a61d0 con 0x7fed28150b50 2023-05-16T07:17:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed2c86a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed28066330 con 0x7fed280bc8f0 2023-05-16T07:17:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed2c86a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280bd270 con 0x7fed28068f10 2023-05-16T07:17:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed26d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed280bc8f0 0x7fed280bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed26d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed280bc8f0 0x7fed280bcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45294/0 (socket says 172.21.15.92:45294) 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.936+0000 7fed26d9d700 1 -- 172.21.15.92:0/3050451736 learned_addr learned my addr 172.21.15.92:0/3050451736 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed25d9b700 1 --2- 172.21.15.92:0/3050451736 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28068f10 0x7fed28150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed26d9d700 1 -- 172.21.15.92:0/3050451736 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28068f10 msgr2=0x7fed28150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed26d9d700 1 --2- 172.21.15.92:0/3050451736 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28068f10 0x7fed28150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed26d9d700 1 -- 172.21.15.92:0/3050451736 >> v1:172.21.15.158:6789/0 conn(0x7fed28150b50 legacy=0x7fed280a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed26d9d700 1 -- 172.21.15.92:0/3050451736 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28069410 con 0x7fed280bc8f0 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed26d9d700 1 --2- 172.21.15.92:0/3050451736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280bc8f0 0x7fed280bcd80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fed20006fd0 tx=0x7fed2000b3b0).ready entity=mon.0 client_cookie=8b3023045dd7c71e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2559a700 1 -- 172.21.15.92:0/3050451736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2000db40 con 0x7fed280bc8f0 2023-05-16T07:17:19.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2559a700 1 -- 172.21.15.92:0/3050451736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2000dca0 con 0x7fed280bc8f0 2023-05-16T07:17:19.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2559a700 1 -- 172.21.15.92:0/3050451736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20010680 con 0x7fed280bc8f0 2023-05-16T07:17:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2c86a700 1 -- 172.21.15.92:0/3050451736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280bc8f0 msgr2=0x7fed280bcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3050451736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280bc8f0 0x7fed280bcd80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fed20006fd0 tx=0x7fed2000b3b0).stop 2023-05-16T07:17:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2c86a700 1 -- 172.21.15.92:0/3050451736 shutdown_connections 2023-05-16T07:17:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3050451736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280bc8f0 0x7fed280bcd80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.937+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3050451736 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28068f10 0x7fed28150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.938+0000 7fed2c86a700 1 -- 172.21.15.92:0/3050451736 >> 172.21.15.92:0/3050451736 conn(0x7fed280c7350 msgr2=0x7fed280c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.938+0000 7fed2c86a700 1 -- 172.21.15.92:0/3050451736 shutdown_connections 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.938+0000 7fed2c86a700 1 -- 172.21.15.92:0/3050451736 wait complete. 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 Processor -- start 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 -- start start 2023-05-16T07:17:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 0x7fed280ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed280bc8f0 0x7fed280bff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28150b50 0x7fed280c0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed280abd60 con 0x7fed28068f10 2023-05-16T07:17:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed280c0950 con 0x7fed280bc8f0 2023-05-16T07:17:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2c86a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280ab150 con 0x7fed28150b50 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.939+0000 7fed2659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 0x7fed280ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 0x7fed280ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45296/0 (socket says 172.21.15.92:45296) 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 -- 172.21.15.92:0/3192446198 learned_addr learned my addr 172.21.15.92:0/3192446198 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed25d9b700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed280bc8f0 0x7fed280bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed26d9d700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28150b50 0x7fed280c0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 -- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28150b50 msgr2=0x7fed280c0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28150b50 0x7fed280c0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 -- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed280bc8f0 msgr2=0x7fed280bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:19.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed280bc8f0 0x7fed280bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:19.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 -- 172.21.15.92:0/3192446198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2000b610 con 0x7fed28068f10 2023-05-16T07:17:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed26d9d700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28150b50 0x7fed280c0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2659c700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 0x7fed280ac1f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fed1800c7f0 tx=0x7fed180015c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18006a20 con 0x7fed28068f10 2023-05-16T07:17:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed18006b80 con 0x7fed28068f10 2023-05-16T07:17:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed280ab430 con 0x7fed28068f10 2023-05-16T07:17:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18007020 con 0x7fed28068f10 2023-05-16T07:17:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.940+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed280ab9f0 con 0x7fed28068f10 2023-05-16T07:17:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.941+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed280ac040 con 0x7fed28068f10 2023-05-16T07:17:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.941+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fed18018020 con 0x7fed28068f10 2023-05-16T07:17:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.942+0000 7fed1e7fc700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed0805edc0 0x7fed08061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.942+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fed18079780 con 0x7fed28068f10 2023-05-16T07:17:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.942+0000 7fed25d9b700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed0805edc0 0x7fed08061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.942+0000 7fed25d9b700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed0805edc0 0x7fed08061280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fed1000bbc0 tx=0x7fed10008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:19.946+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fed1804aad0 con 0x7fed28068f10 2023-05-16T07:17:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.108+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed2806c350 con 0x7fed28068f10 2023-05-16T07:17:20.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.109+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3192446198 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17873 (secure 0 0 0) 0x7fed1804e0f0 con 0x7fed28068f10 2023-05-16T07:17:20.113 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:20.114 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:19.493560+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:18.044089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:18.464963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed0805edc0 msgr2=0x7fed08061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed0805edc0 0x7fed08061280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fed1000bbc0 tx=0x7fed10008040).stop 2023-05-16T07:17:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 msgr2=0x7fed280ac1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 0x7fed280ac1f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fed1800c7f0 tx=0x7fed180015c0).stop 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 shutdown_connections 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed0805edc0 0x7fed08061280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed28150b50 0x7fed280c0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28068f10 0x7fed280ac1f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 --2- 172.21.15.92:0/3192446198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed280bc8f0 0x7fed280bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.111+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 >> 172.21.15.92:0/3192446198 conn(0x7fed280c7350 msgr2=0x7fed280c1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.112+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 shutdown_connections 2023-05-16T07:17:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.112+0000 7fed2c86a700 1 -- 172.21.15.92:0/3192446198 wait complete. 2023-05-16T07:17:20.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:17:20.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.236+0000 7f6143443700 1 Processor -- start 2023-05-16T07:17:20.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.236+0000 7f6143443700 1 -- start start 2023-05-16T07:17:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f6143443700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f613c067080 0x7f613c0bd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f6143443700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f613c0bd7f0 0x7f613c0a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f6143443700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c0bc8b0 con 0x7f613c155b90 2023-05-16T07:17:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f6143443700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0bdca0 con 0x7f613c0bd7f0 2023-05-16T07:17:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f6143443700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c1523e0 con 0x7f613c067080 2023-05-16T07:17:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f61419e0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f613c0bd7f0 0x7f613c0a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f61411df700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f613c155b90 0x7f613c064e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56052/0 (socket says 172.21.15.92:56052) 2023-05-16T07:17:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.237+0000 7f61411df700 1 -- 172.21.15.92:0/614483761 learned_addr learned my addr 172.21.15.92:0/614483761 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f61419e0700 1 -- 172.21.15.92:0/614483761 >> v2:172.21.15.158:3300/0 conn(0x7f613c067080 msgr2=0x7f613c0bd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f61409de700 1 --2- 172.21.15.92:0/614483761 >> v2:172.21.15.158:3300/0 conn(0x7f613c067080 0x7f613c0bd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f61419e0700 1 --2- 172.21.15.92:0/614483761 >> v2:172.21.15.158:3300/0 conn(0x7f613c067080 0x7f613c0bd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f613bfff700 1 -- 172.21.15.92:0/614483761 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233200460 0 0) 0x7f613c0bc8b0 con 0x7f613c155b90 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f61419e0700 1 -- 172.21.15.92:0/614483761 >> v1:172.21.15.92:6789/0 conn(0x7f613c155b90 legacy=0x7f613c064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f61419e0700 1 -- 172.21.15.92:0/614483761 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c068820 con 0x7f613c0bd7f0 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f61419e0700 1 --2- 172.21.15.92:0/614483761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0bd7f0 0x7f613c0a6910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f613c0aa190 tx=0x7f6134009060).ready entity=mon.0 client_cookie=95ab5e51f4cf9ec5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f613bfff700 1 -- 172.21.15.92:0/614483761 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400b580 con 0x7f613c0bd7f0 2023-05-16T07:17:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f613bfff700 1 -- 172.21.15.92:0/614483761 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613400b6e0 con 0x7f613c0bd7f0 2023-05-16T07:17:20.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f613bfff700 1 -- 172.21.15.92:0/614483761 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400ba60 con 0x7f613c0bd7f0 2023-05-16T07:17:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f6143443700 1 -- 172.21.15.92:0/614483761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0bd7f0 msgr2=0x7f613c0a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f6143443700 1 --2- 172.21.15.92:0/614483761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0bd7f0 0x7f613c0a6910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f613c0aa190 tx=0x7f6134009060).stop 2023-05-16T07:17:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f6143443700 1 -- 172.21.15.92:0/614483761 shutdown_connections 2023-05-16T07:17:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f6143443700 1 --2- 172.21.15.92:0/614483761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0bd7f0 0x7f613c0a6910 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f6143443700 1 --2- 172.21.15.92:0/614483761 >> v2:172.21.15.158:3300/0 conn(0x7f613c067080 0x7f613c0bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.238+0000 7f6143443700 1 -- 172.21.15.92:0/614483761 >> 172.21.15.92:0/614483761 conn(0x7f613c0c8420 msgr2=0x7f613c0c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.239+0000 7f6143443700 1 -- 172.21.15.92:0/614483761 shutdown_connections 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.239+0000 7f6143443700 1 -- 172.21.15.92:0/614483761 wait complete. 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.240+0000 7f6143443700 1 Processor -- start 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.240+0000 7f6143443700 1 -- start start 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.240+0000 7f6143443700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 0x7f613c0bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.240+0000 7f6143443700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f613c0bd7f0 0x7f613c0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f6143443700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f613c155b90 0x7f613c14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f6143443700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0aa900 con 0x7f613c067080 2023-05-16T07:17:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f6143443700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0aabe0 con 0x7f613c0bd7f0 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f6143443700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0c3140 con 0x7f613c155b90 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 0x7f613c0bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 0x7f613c0bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45310/0 (socket says 172.21.15.92:45310) 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 -- 172.21.15.92:0/984439742 learned_addr learned my addr 172.21.15.92:0/984439742 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61419e0700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f613c155b90 0x7f613c14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61409de700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f613c0bd7f0 0x7f613c0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 -- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f613c155b90 msgr2=0x7f613c14d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f613c155b90 0x7f613c14d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 -- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f613c0bd7f0 msgr2=0x7f613c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f613c0bd7f0 0x7f613c0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 -- 172.21.15.92:0/984439742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340092a0 con 0x7f613c067080 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61409de700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f613c0bd7f0 0x7f613c0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61411df700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 0x7f613c0bfb90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f612c00c810 tx=0x7f612c001570).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c005d30 con 0x7f613c067080 2023-05-16T07:17:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c005e90 con 0x7f613c067080 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c007340 con 0x7f613c067080 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c0c3420 con 0x7f613c067080 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.241+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c0c2e10 con 0x7f613c067080 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.242+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c063520 con 0x7f613c067080 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.243+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f612c010030 con 0x7f613c067080 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.243+0000 7f61397fa700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f612805ed90 0x7f6128061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.243+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f612c079c20 con 0x7f613c067080 2023-05-16T07:17:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.243+0000 7f61409de700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f612805ed90 0x7f6128061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.243+0000 7f61409de700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f612805ed90 0x7f6128061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f613000aa10 tx=0x7f6130008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.247+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f612c04af70 con 0x7f613c067080 2023-05-16T07:17:20.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.450+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f613c06c350 con 0x7f613c067080 2023-05-16T07:17:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.497+0000 7f61397fa700 1 -- 172.21.15.92:0/984439742 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f612c079020 con 0x7f613c067080 2023-05-16T07:17:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f612805ed90 msgr2=0x7f6128061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f612805ed90 0x7f6128061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f613000aa10 tx=0x7f6130008040).stop 2023-05-16T07:17:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 msgr2=0x7f613c0bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 0x7f613c0bfb90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f612c00c810 tx=0x7f612c001570).stop 2023-05-16T07:17:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 shutdown_connections 2023-05-16T07:17:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f612805ed90 0x7f6128061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f613c155b90 0x7f613c14d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c067080 0x7f613c0bfb90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 --2- 172.21.15.92:0/984439742 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f613c0bd7f0 0x7f613c0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 >> 172.21.15.92:0/984439742 conn(0x7f613c0c8420 msgr2=0x7f613c0c6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 shutdown_connections 2023-05-16T07:17:20.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.499+0000 7f6143443700 1 -- 172.21.15.92:0/984439742 wait complete. 2023-05-16T07:17:20.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:17:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.624+0000 7fbfefda2700 1 Processor -- start 2023-05-16T07:17:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.624+0000 7fbfefda2700 1 -- start start 2023-05-16T07:17:20.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfefda2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfefda2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbfe8068160 0x7fbfe8068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfefda2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80c94d0 con 0x7fbfe8154470 2023-05-16T07:17:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfefda2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe80cb7e0 con 0x7fbfe8068160 2023-05-16T07:17:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfefda2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8068a70 con 0x7fbfe8154090 2023-05-16T07:17:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfee33f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbfe8068160 0x7fbfe8068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfee33f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbfe8068160 0x7fbfe8068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45320/0 (socket says 172.21.15.92:45320) 2023-05-16T07:17:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfee33f700 1 -- 172.21.15.92:0/145887597 learned_addr learned my addr 172.21.15.92:0/145887597 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfedb3e700 1 --2- 172.21.15.92:0/145887597 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.625+0000 7fbfee33f700 1 -- 172.21.15.92:0/145887597 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 msgr2=0x7fbfe80aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfee33f700 1 --2- 172.21.15.92:0/145887597 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfee33f700 1 -- 172.21.15.92:0/145887597 >> v1:172.21.15.158:6789/0 conn(0x7fbfe8154470 legacy=0x7fbfe80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfee33f700 1 -- 172.21.15.92:0/145887597 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80c88c0 con 0x7fbfe8068160 2023-05-16T07:17:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfee33f700 1 --2- 172.21.15.92:0/145887597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8068160 0x7fbfe8068580 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fbfe0006fd0 tx=0x7fbfe00090f0).ready entity=mon.0 client_cookie=477f9017f19d3ee0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfecb3c700 1 -- 172.21.15.92:0/145887597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000b600 con 0x7fbfe8068160 2023-05-16T07:17:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfecb3c700 1 -- 172.21.15.92:0/145887597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000b760 con 0x7fbfe8068160 2023-05-16T07:17:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfecb3c700 1 -- 172.21.15.92:0/145887597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000bae0 con 0x7fbfe8068160 2023-05-16T07:17:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 -- 172.21.15.92:0/145887597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8068160 msgr2=0x7fbfe8068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 --2- 172.21.15.92:0/145887597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8068160 0x7fbfe8068580 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fbfe0006fd0 tx=0x7fbfe00090f0).stop 2023-05-16T07:17:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 -- 172.21.15.92:0/145887597 shutdown_connections 2023-05-16T07:17:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 --2- 172.21.15.92:0/145887597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8068160 0x7fbfe8068580 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 --2- 172.21.15.92:0/145887597 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 -- 172.21.15.92:0/145887597 >> 172.21.15.92:0/145887597 conn(0x7fbfe8067920 msgr2=0x7fbfe80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 -- 172.21.15.92:0/145887597 shutdown_connections 2023-05-16T07:17:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.626+0000 7fbfefda2700 1 -- 172.21.15.92:0/145887597 wait complete. 2023-05-16T07:17:20.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.627+0000 7fbfefda2700 1 Processor -- start 2023-05-16T07:17:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.627+0000 7fbfefda2700 1 -- start start 2023-05-16T07:17:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfefda2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbfe8068160 0x7fbfe80adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfefda2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfefda2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 0x7fbfe80c3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfefda2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8156240 con 0x7fbfe8154470 2023-05-16T07:17:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfefda2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8155950 con 0x7fbfe8068160 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfefda2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80b0c20 con 0x7fbfe8154090 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfee33f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 0x7fbfe80c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfee33f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 0x7fbfe80c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45328/0 (socket says 172.21.15.92:45328) 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfee33f700 1 -- 172.21.15.92:0/2629021472 learned_addr learned my addr 172.21.15.92:0/2629021472 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfedb3e700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbfe8068160 0x7fbfe80adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.628+0000 7fbfed33d700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfee33f700 1 -- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 msgr2=0x7fbfe80b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfee33f700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfee33f700 1 -- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbfe8068160 msgr2=0x7fbfe80adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfee33f700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbfe8068160 0x7fbfe80adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfee33f700 1 -- 172.21.15.92:0/2629021472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0009370 con 0x7fbfe8154470 2023-05-16T07:17:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfed33d700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfee33f700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 0x7fbfe80c3810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fbfe806bb30 tx=0x7fbfe0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000b600 con 0x7fbfe8154470 2023-05-16T07:17:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000bea0 con 0x7fbfe8154470 2023-05-16T07:17:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe814ff20 con 0x7fbfe8154470 2023-05-16T07:17:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000bae0 con 0x7fbfe8154470 2023-05-16T07:17:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.629+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe8155ed0 con 0x7fbfe8154470 2023-05-16T07:17:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.630+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe8066730 con 0x7fbfe8154470 2023-05-16T07:17:20.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.630+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbfe002b070 con 0x7fbfe8154470 2023-05-16T07:17:20.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.631+0000 7fbfe5ffb700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbfd405ed90 0x7fbfd4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:20.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.631+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fbfe007b560 con 0x7fbfe8154470 2023-05-16T07:17:20.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.631+0000 7fbfedb3e700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbfd405ed90 0x7fbfd4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:20.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.631+0000 7fbfedb3e700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbfd405ed90 0x7fbfd4061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fbfd8007640 tx=0x7fbfd8009850).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:20.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.635+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbfe00502e0 con 0x7fbfe8154470 2023-05-16T07:17:20.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:20.842+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbfe806c350 con 0x7fbfe8154470 2023-05-16T07:17:21.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.503+0000 7fbfe5ffb700 1 -- 172.21.15.92:0/2629021472 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbfe0029070 con 0x7fbfe8154470 2023-05-16T07:17:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbfd405ed90 msgr2=0x7fbfd4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbfd405ed90 0x7fbfd4061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fbfd8007640 tx=0x7fbfd8009850).stop 2023-05-16T07:17:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 msgr2=0x7fbfe80c3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 0x7fbfe80c3810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fbfe806bb30 tx=0x7fbfe0017000).stop 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 shutdown_connections 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbfd405ed90 0x7fbfd4061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe8154470 0x7fbfe80c3810 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbfe8068160 0x7fbfe80adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 --2- 172.21.15.92:0/2629021472 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbfe8154090 0x7fbfe80b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 >> 172.21.15.92:0/2629021472 conn(0x7fbfe8067920 msgr2=0x7fbfe8152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 shutdown_connections 2023-05-16T07:17:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.505+0000 7fbfefda2700 1 -- 172.21.15.92:0/2629021472 wait complete. 2023-05-16T07:17:21.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:21.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.630+0000 7f7b84aa0700 1 Processor -- start 2023-05-16T07:17:21.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.630+0000 7f7b84aa0700 1 -- start start 2023-05-16T07:17:21.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b84aa0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7b80154e50 0x7f7b800acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b84aa0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 0x7f7b8014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b84aa0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b80158970 con 0x7f7b800c7380 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b84aa0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b80155300 con 0x7f7b80154e50 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b84aa0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80157d60 con 0x7f7b800ad040 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7b800c7380 0x7f7b800c7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56080/0 (socket says 172.21.15.92:56080) 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7f7fe700 1 -- 172.21.15.92:0/4157669228 learned_addr learned my addr 172.21.15.92:0/4157669228 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 0x7f7b8014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7effd700 1 --2- 172.21.15.92:0/4157669228 >> v2:172.21.15.158:3300/0 conn(0x7f7b80154e50 0x7f7b800acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428133551 0 0) 0x7f7b80158970 con 0x7f7b800c7380 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b680033f0 con 0x7f7b800c7380 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1884596929 0 0) 0x7f7b680033f0 con 0x7f7b800c7380 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 msgr2=0x7f7b8014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 0x7f7b8014dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 >> v2:172.21.15.158:3300/0 conn(0x7f7b80154e50 msgr2=0x7f7b800acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 --2- 172.21.15.92:0/4157669228 >> v2:172.21.15.158:3300/0 conn(0x7f7b80154e50 0x7f7b800acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80158040 con 0x7f7b800c7380 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f7b700017f0 con 0x7f7b800c7380 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 >> v1:172.21.15.92:6789/0 conn(0x7f7b800c7380 legacy=0x7f7b800c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b680038e0 0x7f7b68003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 0x7f7b68004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b68004b80 0x7f7b68005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80155300 con 0x7f7b68004b80 2023-05-16T07:17:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80157d60 con 0x7f7b680038e0 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.631+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6800b8e0 con 0x7f7b800ad040 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b68004b80 0x7f7b68005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7effd700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 0x7f7b68004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7f7fe700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b680038e0 0x7f7b68003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 -- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 msgr2=0x7f7b68004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b800ad040 0x7f7b68004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 -- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b680038e0 msgr2=0x7f7b68003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b680038e0 0x7f7b68003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 -- 172.21.15.92:0/4157669228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680033f0 con 0x7f7b68004b80 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b68004b80 0x7f7b68005000 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f7b78009a80 tx=0x7f7b78009060).ready entity=mon.0 client_cookie=2f30144b3e48f261 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009430 con 0x7f7b68004b80 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7800e640 con 0x7f7b68004b80 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b7e7fc700 1 -- 172.21.15.92:0/4157669228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800e9c0 con 0x7f7b68004b80 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b84aa0700 1 -- 172.21.15.92:0/4157669228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b68004b80 msgr2=0x7f7b68005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.632+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/4157669228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b68004b80 0x7f7b68005000 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f7b78009a80 tx=0x7f7b78009060).stop 2023-05-16T07:17:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.633+0000 7f7b7ffff700 1 -- 172.21.15.92:0/4157669228 reap_dead start 2023-05-16T07:17:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.633+0000 7f7b84aa0700 1 -- 172.21.15.92:0/4157669228 shutdown_connections 2023-05-16T07:17:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.633+0000 7f7b84aa0700 1 -- 172.21.15.92:0/4157669228 >> 172.21.15.92:0/4157669228 conn(0x7f7b80068330 msgr2=0x7f7b80068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.633+0000 7f7b84aa0700 1 -- 172.21.15.92:0/4157669228 shutdown_connections 2023-05-16T07:17:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.633+0000 7f7b84aa0700 1 -- 172.21.15.92:0/4157669228 wait complete. 2023-05-16T07:17:21.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.634+0000 7f7b84aa0700 1 Processor -- start 2023-05-16T07:17:21.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.634+0000 7f7b84aa0700 1 -- start start 2023-05-16T07:17:21.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b801531b0 0x7f7b800c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 0x7f7b800c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b800cb0b0 0x7f7b80140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b800cac20 con 0x7f7b80153590 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b800ca010 con 0x7f7b800cb0b0 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b800ca2f0 con 0x7f7b801531b0 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 0x7f7b800c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 0x7f7b800c1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45346/0 (socket says 172.21.15.92:45346) 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 -- 172.21.15.92:0/2725661904 learned_addr learned my addr 172.21.15.92:0/2725661904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 -- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b801531b0 msgr2=0x7f7b800c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b801531b0 0x7f7b800c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 -- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b800cb0b0 msgr2=0x7f7b80140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b800cb0b0 0x7f7b80140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 -- 172.21.15.92:0/2725661904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800ca5d0 con 0x7f7b80153590 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b7effd700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 0x7f7b800c1c60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f7b740119c0 tx=0x7f7b74009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:21.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400f940 con 0x7f7b80153590 2023-05-16T07:17:21.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7400faa0 con 0x7f7b80153590 2023-05-16T07:17:21.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400fe20 con 0x7f7b80153590 2023-05-16T07:17:21.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b800ca8b0 con 0x7f7b80153590 2023-05-16T07:17:21.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.635+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b800cb510 con 0x7f7b80153590 2023-05-16T07:17:21.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.636+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b800caf00 con 0x7f7b80153590 2023-05-16T07:17:21.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.636+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7b74024070 con 0x7f7b80153590 2023-05-16T07:17:21.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.637+0000 7f7b677fe700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6c05ed90 0x7f7b6c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.637+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f7b7407a7a0 con 0x7f7b80153590 2023-05-16T07:17:21.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.637+0000 7f7b7f7fe700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6c05ed90 0x7f7b6c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.638+0000 7f7b7f7fe700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6c05ed90 0x7f7b6c061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7b70001560 tx=0x7f7b801543d0).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:21.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.641+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7b7404baf0 con 0x7f7b80153590 2023-05-16T07:17:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.843+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b800cad60 con 0x7f7b80153590 2023-05-16T07:17:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.844+0000 7f7b677fe700 1 -- 172.21.15.92:0/2725661904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f7b800cad60 con 0x7f7b80153590 2023-05-16T07:17:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:21.848 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:19.493924+0000","modified":"2023-05-16T07:17:21.499636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1922547921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1922547921},{"type":"v1","addr":"172.21.15.92:6835","nonce":1922547921}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-05-16T07:17:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6c05ed90 msgr2=0x7f7b6c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6c05ed90 0x7f7b6c061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7b70001560 tx=0x7f7b801543d0).stop 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 msgr2=0x7f7b800c1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 0x7f7b800c1c60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f7b740119c0 tx=0x7f7b74009060).stop 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 shutdown_connections 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6c05ed90 0x7f7b6c061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b801531b0 0x7f7b800c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b800cb0b0 0x7f7b80140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 --2- 172.21.15.92:0/2725661904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b80153590 0x7f7b800c1c60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 >> 172.21.15.92:0/2725661904 conn(0x7f7b80068330 msgr2=0x7f7b8014f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 shutdown_connections 2023-05-16T07:17:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.846+0000 7f7b84aa0700 1 -- 172.21.15.92:0/2725661904 wait complete. 2023-05-16T07:17:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-05-16T07:17:21.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:21.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.968+0000 7fb283913700 1 Processor -- start 2023-05-16T07:17:21.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.968+0000 7fb283913700 1 -- start start 2023-05-16T07:17:21.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb283913700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb27c157c20 0x7fb27c158000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb283913700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27c068400 con 0x7fb27c0c1790 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb283913700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27c0be060 con 0x7fb27c0c3770 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb283913700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27c067560 con 0x7fb27c157c20 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb2816af700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb27c157c20 0x7fb27c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb281eb0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb27c0c1790 0x7fb27c15de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56104/0 (socket says 172.21.15.92:56104) 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.969+0000 7fb281eb0700 1 -- 172.21.15.92:0/1506505432 learned_addr learned my addr 172.21.15.92:0/1506505432 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb2816af700 1 -- 172.21.15.92:0/1506505432 >> v1:172.21.15.158:6789/0 conn(0x7fb27c0c3770 legacy=0x7fb27c066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb2816af700 1 -- 172.21.15.92:0/1506505432 >> v1:172.21.15.92:6789/0 conn(0x7fb27c0c1790 legacy=0x7fb27c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb2816af700 1 -- 172.21.15.92:0/1506505432 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0677f0 con 0x7fb27c157c20 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb27bfff700 1 -- 172.21.15.92:0/1506505432 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888510470 0 0) 0x7fb27c068400 con 0x7fb27c0c1790 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb2816af700 1 --2- 172.21.15.92:0/1506505432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c157c20 0x7fb27c158000 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7fb26c009e60 tx=0x7fb26c009780).ready entity=mon.0 client_cookie=db81ec646f893433 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb27bfff700 1 -- 172.21.15.92:0/1506505432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c00e070 con 0x7fb27c157c20 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb27bfff700 1 -- 172.21.15.92:0/1506505432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c00bbf0 con 0x7fb27c157c20 2023-05-16T07:17:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb27bfff700 1 -- 172.21.15.92:0/1506505432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c012410 con 0x7fb27c157c20 2023-05-16T07:17:21.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 -- 172.21.15.92:0/1506505432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c157c20 msgr2=0x7fb27c158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 --2- 172.21.15.92:0/1506505432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c157c20 0x7fb27c158000 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7fb26c009e60 tx=0x7fb26c009780).stop 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 -- 172.21.15.92:0/1506505432 shutdown_connections 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 --2- 172.21.15.92:0/1506505432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c157c20 0x7fb27c158000 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 -- 172.21.15.92:0/1506505432 >> 172.21.15.92:0/1506505432 conn(0x7fb27c0c7bc0 msgr2=0x7fb27c0c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 -- 172.21.15.92:0/1506505432 shutdown_connections 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.970+0000 7fb283913700 1 -- 172.21.15.92:0/1506505432 wait complete. 2023-05-16T07:17:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.971+0000 7fb283913700 1 Processor -- start 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 -- start start 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb27c0c1790 0x7fb27c1683a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 0x7fb27c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb27c157c20 0x7fb27c151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c1566e0 con 0x7fb27c0c3770 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c1518c0 con 0x7fb27c157c20 2023-05-16T07:17:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.972+0000 7fb283913700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c155db0 con 0x7fb27c0c1790 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 0x7fb27c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 0x7fb27c13d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45360/0 (socket says 172.21.15.92:45360) 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 -- 172.21.15.92:0/187973714 learned_addr learned my addr 172.21.15.92:0/187973714 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb2816af700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb27c0c1790 0x7fb27c1683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb281eb0700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb27c157c20 0x7fb27c151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 -- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb27c0c1790 msgr2=0x7fb27c1683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb27c0c1790 0x7fb27c1683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 -- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb27c157c20 msgr2=0x7fb27c151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb27c157c20 0x7fb27c151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 -- 172.21.15.92:0/187973714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c009a20 con 0x7fb27c0c3770 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb280eae700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 0x7fb27c13d1e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fb27000c440 tx=0x7fb270007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000cea0 con 0x7fb27c0c3770 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27000f470 con 0x7fb27c0c3770 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000f810 con 0x7fb27c0c3770 2023-05-16T07:17:21.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c156090 con 0x7fb27c0c3770 2023-05-16T07:17:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.973+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c156b70 con 0x7fb27c0c3770 2023-05-16T07:17:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.974+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c06c350 con 0x7fb27c0c3770 2023-05-16T07:17:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.974+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb270019030 con 0x7fb27c0c3770 2023-05-16T07:17:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.975+0000 7fb2797fa700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb26805ed90 0x7fb268061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.975+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fb270079070 con 0x7fb27c0c3770 2023-05-16T07:17:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.975+0000 7fb2816af700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb26805ed90 0x7fb268061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.975+0000 7fb2816af700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb26805ed90 0x7fb268061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb26c0097c0 tx=0x7fb27c068370).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:21.979+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb2700488f0 con 0x7fb27c0c3770 2023-05-16T07:17:22.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.142+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb27c13a8b0 con 0x7fb27c0c3770 2023-05-16T07:17:22.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.143+0000 7fb2797fa700 1 -- 172.21.15.92:0/187973714 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17860 (secure 0 0 0) 0x7fb27004a470 con 0x7fb27c0c3770 2023-05-16T07:17:22.147 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:22.147 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":79,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:20.495712+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:18.044089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:18.464963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:22.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb26805ed90 msgr2=0x7fb268061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb26805ed90 0x7fb268061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb26c0097c0 tx=0x7fb27c068370).stop 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 msgr2=0x7fb27c13d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 0x7fb27c13d1e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fb27000c440 tx=0x7fb270007de0).stop 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 shutdown_connections 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb26805ed90 0x7fb268061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb27c0c1790 0x7fb27c1683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb27c157c20 0x7fb27c151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 --2- 172.21.15.92:0/187973714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb27c0c3770 0x7fb27c13d1e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 >> 172.21.15.92:0/187973714 conn(0x7fb27c0c7bc0 msgr2=0x7fb27c136a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:22.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 shutdown_connections 2023-05-16T07:17:22.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.145+0000 7fb283913700 1 -- 172.21.15.92:0/187973714 wait complete. 2023-05-16T07:17:22.161 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:17:22.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:17:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.269+0000 7f4f71491700 1 Processor -- start 2023-05-16T07:17:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.269+0000 7f4f71491700 1 -- start start 2023-05-16T07:17:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f71491700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4f6c065ef0 0x7f4f6c0c2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f71491700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c065350 con 0x7f4f6c150e20 2023-05-16T07:17:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f71491700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c151240 con 0x7f4f6c150a40 2023-05-16T07:17:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f71491700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0c2e00 con 0x7f4f6c065ef0 2023-05-16T07:17:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f63fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4f6c150e20 0x7f4f6c0657e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56116/0 (socket says 172.21.15.92:56116) 2023-05-16T07:17:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f63fff700 1 -- 172.21.15.92:0/1475674606 learned_addr learned my addr 172.21.15.92:0/1475674606 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f70c90700 1 --2- 172.21.15.92:0/1475674606 >> v2:172.21.15.158:3300/0 conn(0x7f4f6c065ef0 0x7f4f6c0c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598673415 0 0) 0x7f4f6c065350 con 0x7f4f6c150e20 2023-05-16T07:17:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f580033f0 con 0x7f4f6c150e20 2023-05-16T07:17:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.270+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 953691029 0 0) 0x7f4f580033f0 con 0x7f4f6c150e20 2023-05-16T07:17:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 >> v2:172.21.15.158:3300/0 conn(0x7f4f6c065ef0 msgr2=0x7f4f6c0c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 --2- 172.21.15.92:0/1475674606 >> v2:172.21.15.158:3300/0 conn(0x7f4f6c065ef0 0x7f4f6c0c2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 >> v1:172.21.15.158:6789/0 conn(0x7f4f6c150a40 legacy=0x7f4f6c0a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0c30c0 con 0x7f4f6c150e20 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4f54001470 con 0x7f4f6c150e20 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 >> v1:172.21.15.92:6789/0 conn(0x7f4f6c150e20 legacy=0x7f4f6c0657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f580038e0 0x7f4f58003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f58004690 0x7f4f58004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58005000 0x7f4f5800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0c2e00 con 0x7f4f58004690 2023-05-16T07:17:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5800beb0 con 0x7f4f580038e0 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5800c070 con 0x7f4f58005000 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f58004690 0x7f4f58004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f580038e0 0x7f4f58003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f70c90700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58005000 0x7f4f5800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 -- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58005000 msgr2=0x7f4f5800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58005000 0x7f4f5800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 -- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f580038e0 msgr2=0x7f4f58003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f580038e0 0x7f4f58003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 -- 172.21.15.92:0/1475674606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580033f0 con 0x7f4f58004690 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f70c90700 1 -- 172.21.15.92:0/1475674606 reap_dead start 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f63fff700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f58004690 0x7f4f58004b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f4f5400c190 tx=0x7f4f54001e50).ready entity=mon.0 client_cookie=58f0067c08b4b5ab server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54007a20 con 0x7f4f58004690 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400cac0 con 0x7f4f58004690 2023-05-16T07:17:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.271+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/1475674606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400ce40 con 0x7f4f58004690 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 -- 172.21.15.92:0/1475674606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f58004690 msgr2=0x7f4f58004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f58004690 0x7f4f58004b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f4f5400c190 tx=0x7f4f54001e50).stop 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 -- 172.21.15.92:0/1475674606 shutdown_connections 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 --2- 172.21.15.92:0/1475674606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f58004690 0x7f4f58004b10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 -- 172.21.15.92:0/1475674606 >> 172.21.15.92:0/1475674606 conn(0x7f4f6c067f60 msgr2=0x7f4f6c068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 -- 172.21.15.92:0/1475674606 shutdown_connections 2023-05-16T07:17:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.272+0000 7f4f71491700 1 -- 172.21.15.92:0/1475674606 wait complete. 2023-05-16T07:17:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.273+0000 7f4f71491700 1 Processor -- start 2023-05-16T07:17:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.273+0000 7f4f71491700 1 -- start start 2023-05-16T07:17:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58004690 0x7f4f6c14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f6c0c5a50 0x7f4f6c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 0x7f4f6c0c0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c14e020 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0c8750 con 0x7f4f6c0c5a50 2023-05-16T07:17:22.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0ab830 con 0x7f4f58004690 2023-05-16T07:17:22.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 0x7f4f6c0c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 0x7f4f6c0c0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45382/0 (socket says 172.21.15.92:45382) 2023-05-16T07:17:22.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 -- 172.21.15.92:0/2885427968 learned_addr learned my addr 172.21.15.92:0/2885427968 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:22.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f63fff700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f6c0c5a50 0x7f4f6c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58004690 0x7f4f6c14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 -- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58004690 msgr2=0x7f4f6c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58004690 0x7f4f6c14db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 -- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f6c0c5a50 msgr2=0x7f4f6c0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f6c0c5a50 0x7f4f6c0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 -- 172.21.15.92:0/2885427968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0ac3b0 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f63fff700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f6c0c5a50 0x7f4f6c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:22.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f70c90700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 0x7f4f6c0c0260 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f4f64009fd0 tx=0x7f4f6400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6400e070 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64011c00 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c0c6e10 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64016450 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.274+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c0abd60 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.275+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c047150 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.276+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4f6401d030 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.276+0000 7f4f68ff9700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4f4c05ed90 0x7f4f4c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.276+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f4f6407a0c0 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.276+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4f4c05ed90 0x7f4f4c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.277+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4f4c05ed90 0x7f4f4c061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4f5c000c00 tx=0x7f4f5c009040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.280+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4f6404b4c0 con 0x7f4f6c0bfde0 2023-05-16T07:17:22.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.440+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f6c0aa160 con 0x7f4f4c05ed90 2023-05-16T07:17:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.452+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2885427968 <== mgr.5330 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361226 (secure 0 0 0) 0x7f4f6c0aa160 con 0x7f4f4c05ed90 2023-05-16T07:17:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:22.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4f4c05ed90 msgr2=0x7f4f4c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4f4c05ed90 0x7f4f4c061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4f5c000c00 tx=0x7f4f5c009040).stop 2023-05-16T07:17:22.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 msgr2=0x7f4f6c0c0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 0x7f4f6c0c0260 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f4f64009fd0 tx=0x7f4f6400b040).stop 2023-05-16T07:17:22.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 shutdown_connections 2023-05-16T07:17:22.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4f4c05ed90 0x7f4f4c061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f58004690 0x7f4f6c14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0bfde0 0x7f4f6c0c0260 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 --2- 172.21.15.92:0/2885427968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f6c0c5a50 0x7f4f6c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 >> 172.21.15.92:0/2885427968 conn(0x7f4f6c067f60 msgr2=0x7f4f6c151de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.455+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 shutdown_connections 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.456+0000 7f4f71491700 1 -- 172.21.15.92:0/2885427968 wait complete. 2023-05-16T07:17:22.463 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:17:22.471 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:17:20.923221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":135752,"kb_used_data":7304,"kb_used_omap":0,"kb_used_meta":128384,"kb_avail":754838968,"statfs":{"total":773094113280,"available":772955103232,"internally_reserved":0,"allocated":7479296,"data_stored":3777083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":131465216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28002,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-78,"num_write_kb":-63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.586570"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515982+0000","last_change":"2023-05-16T07:17:19.515982+0000","last_active":"2023-05-16T07:17:19.515982+0000","last_peered":"2023-05-16T07:17:19.515982+0000","last_clean":"2023-05-16T07:17:19.515982+0000","last_became_active":"2023-05-16T07:17:19.513374+0000","last_became_peered":"2023-05-16T07:17:19.513374+0000","last_unstale":"2023-05-16T07:17:19.515982+0000","last_undegraded":"2023-05-16T07:17:19.515982+0000","last_fullsized":"2023-05-16T07:17:19.515982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"78'3","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516249+0000","last_change":"2023-05-16T07:17:19.140747+0000","last_active":"2023-05-16T07:17:19.516249+0000","last_peered":"2023-05-16T07:17:19.516249+0000","last_clean":"2023-05-16T07:17:19.516249+0000","last_became_active":"2023-05-16T07:17:19.140467+0000","last_became_peered":"2023-05-16T07:17:19.140467+0000","last_unstale":"2023-05-16T07:17:19.516249+0000","last_undegraded":"2023-05-16T07:17:19.516249+0000","last_fullsized":"2023-05-16T07:17:19.516249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518056+0000","last_change":"2023-05-16T07:17:19.518056+0000","last_active":"2023-05-16T07:17:19.518056+0000","last_peered":"2023-05-16T07:17:19.518056+0000","last_clean":"2023-05-16T07:17:19.518056+0000","last_became_active":"2023-05-16T07:17:19.517784+0000","last_became_peered":"2023-05-16T07:17:19.517784+0000","last_unstale":"2023-05-16T07:17:19.518056+0000","last_undegraded":"2023-05-16T07:17:19.518056+0000","last_fullsized":"2023-05-16T07:17:19.518056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146025+0000","last_change":"2023-05-16T07:17:19.139074+0000","last_active":"2023-05-16T07:17:19.146025+0000","last_peered":"2023-05-16T07:17:19.146025+0000","last_clean":"2023-05-16T07:17:19.146025+0000","last_became_active":"2023-05-16T07:17:19.138616+0000","last_became_peered":"2023-05-16T07:17:19.138616+0000","last_unstale":"2023-05-16T07:17:19.146025+0000","last_undegraded":"2023-05-16T07:17:19.146025+0000","last_fullsized":"2023-05-16T07:17:19.146025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513813+0000","last_change":"2023-05-16T07:17:19.513813+0000","last_active":"2023-05-16T07:17:19.513813+0000","last_peered":"2023-05-16T07:17:19.513813+0000","last_clean":"2023-05-16T07:17:19.513813+0000","last_became_active":"2023-05-16T07:17:19.513537+0000","last_became_peered":"2023-05-16T07:17:19.513537+0000","last_unstale":"2023-05-16T07:17:19.513813+0000","last_undegraded":"2023-05-16T07:17:19.513813+0000","last_fullsized":"2023-05-16T07:17:19.513813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151456+0000","last_change":"2023-05-16T07:17:19.147500+0000","last_active":"2023-05-16T07:17:19.151456+0000","last_peered":"2023-05-16T07:17:19.151456+0000","last_clean":"2023-05-16T07:17:19.151456+0000","last_became_active":"2023-05-16T07:17:19.147222+0000","last_became_peered":"2023-05-16T07:17:19.147222+0000","last_unstale":"2023-05-16T07:17:19.151456+0000","last_undegraded":"2023-05-16T07:17:19.151456+0000","last_fullsized":"2023-05-16T07:17:19.151456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518191+0000","last_change":"2023-05-16T07:17:19.518191+0000","last_active":"2023-05-16T07:17:19.518191+0000","last_peered":"2023-05-16T07:17:19.518191+0000","last_clean":"2023-05-16T07:17:19.518191+0000","last_became_active":"2023-05-16T07:17:19.517822+0000","last_became_peered":"2023-05-16T07:17:19.517822+0000","last_unstale":"2023-05-16T07:17:19.518191+0000","last_undegraded":"2023-05-16T07:17:19.518191+0000","last_fullsized":"2023-05-16T07:17:19.518191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517484+0000","last_change":"2023-05-16T07:17:19.144182+0000","last_active":"2023-05-16T07:17:19.517484+0000","last_peered":"2023-05-16T07:17:19.517484+0000","last_clean":"2023-05-16T07:17:19.517484+0000","last_became_active":"2023-05-16T07:17:19.143901+0000","last_became_peered":"2023-05-16T07:17:19.143901+0000","last_unstale":"2023-05-16T07:17:19.517484+0000","last_undegraded":"2023-05-16T07:17:19.517484+0000","last_fullsized":"2023-05-16T07:17:19.517484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514124+0000","last_change":"2023-05-16T07:17:19.514124+0000","last_active":"2023-05-16T07:17:19.514124+0000","last_peered":"2023-05-16T07:17:19.514124+0000","last_clean":"2023-05-16T07:17:19.514124+0000","last_became_active":"2023-05-16T07:17:19.513819+0000","last_became_peered":"2023-05-16T07:17:19.513819+0000","last_unstale":"2023-05-16T07:17:19.514124+0000","last_undegraded":"2023-05-16T07:17:19.514124+0000","last_fullsized":"2023-05-16T07:17:19.514124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.498672+0000","last_change":"2023-05-16T07:17:19.147823+0000","last_active":"2023-05-16T07:17:19.498672+0000","last_peered":"2023-05-16T07:17:19.498672+0000","last_clean":"2023-05-16T07:17:19.498672+0000","last_became_active":"2023-05-16T07:17:19.147560+0000","last_became_peered":"2023-05-16T07:17:19.147560+0000","last_unstale":"2023-05-16T07:17:19.498672+0000","last_undegraded":"2023-05-16T07:17:19.498672+0000","last_fullsized":"2023-05-16T07:17:19.498672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.509160+0000","last_change":"2023-05-16T07:17:19.509160+0000","last_active":"2023-05-16T07:17:19.509160+0000","last_peered":"2023-05-16T07:17:19.509160+0000","last_clean":"2023-05-16T07:17:19.509160+0000","last_became_active":"2023-05-16T07:17:19.508876+0000","last_became_peered":"2023-05-16T07:17:19.508876+0000","last_unstale":"2023-05-16T07:17:19.509160+0000","last_undegraded":"2023-05-16T07:17:19.509160+0000","last_fullsized":"2023-05-16T07:17:19.509160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.502295+0000","last_change":"2023-05-16T07:17:19.142468+0000","last_active":"2023-05-16T07:17:19.502295+0000","last_peered":"2023-05-16T07:17:19.502295+0000","last_clean":"2023-05-16T07:17:19.502295+0000","last_became_active":"2023-05-16T07:17:19.142190+0000","last_became_peered":"2023-05-16T07:17:19.142190+0000","last_unstale":"2023-05-16T07:17:19.502295+0000","last_undegraded":"2023-05-16T07:17:19.502295+0000","last_fullsized":"2023-05-16T07:17:19.502295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510578+0000","last_change":"2023-05-16T07:17:19.510578+0000","last_active":"2023-05-16T07:17:19.510578+0000","last_peered":"2023-05-16T07:17:19.510578+0000","last_clean":"2023-05-16T07:17:19.510578+0000","last_became_active":"2023-05-16T07:17:19.510331+0000","last_became_peered":"2023-05-16T07:17:19.510331+0000","last_unstale":"2023-05-16T07:17:19.510578+0000","last_undegraded":"2023-05-16T07:17:19.510578+0000","last_fullsized":"2023-05-16T07:17:19.510578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145763+0000","last_change":"2023-05-16T07:17:19.141920+0000","last_active":"2023-05-16T07:17:19.145763+0000","last_peered":"2023-05-16T07:17:19.145763+0000","last_clean":"2023-05-16T07:17:19.145763+0000","last_became_active":"2023-05-16T07:17:19.140680+0000","last_became_peered":"2023-05-16T07:17:19.140680+0000","last_unstale":"2023-05-16T07:17:19.145763+0000","last_undegraded":"2023-05-16T07:17:19.145763+0000","last_fullsized":"2023-05-16T07:17:19.145763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513257+0000","last_change":"2023-05-16T07:17:19.513257+0000","last_active":"2023-05-16T07:17:19.513257+0000","last_peered":"2023-05-16T07:17:19.513257+0000","last_clean":"2023-05-16T07:17:19.513257+0000","last_became_active":"2023-05-16T07:17:19.512948+0000","last_became_peered":"2023-05-16T07:17:19.512948+0000","last_unstale":"2023-05-16T07:17:19.513257+0000","last_undegraded":"2023-05-16T07:17:19.513257+0000","last_fullsized":"2023-05-16T07:17:19.513257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.140344+0000","last_change":"2023-05-16T07:17:19.136135+0000","last_active":"2023-05-16T07:17:19.140344+0000","last_peered":"2023-05-16T07:17:19.140344+0000","last_clean":"2023-05-16T07:17:19.140344+0000","last_became_active":"2023-05-16T07:17:19.135867+0000","last_became_peered":"2023-05-16T07:17:19.135867+0000","last_unstale":"2023-05-16T07:17:19.140344+0000","last_undegraded":"2023-05-16T07:17:19.140344+0000","last_fullsized":"2023-05-16T07:17:19.140344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514137+0000","last_change":"2023-05-16T07:17:19.514137+0000","last_active":"2023-05-16T07:17:19.514137+0000","last_peered":"2023-05-16T07:17:19.514137+0000","last_clean":"2023-05-16T07:17:19.514137+0000","last_became_active":"2023-05-16T07:17:19.513839+0000","last_became_peered":"2023-05-16T07:17:19.513839+0000","last_unstale":"2023-05-16T07:17:19.514137+0000","last_undegraded":"2023-05-16T07:17:19.514137+0000","last_fullsized":"2023-05-16T07:17:19.514137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145554+0000","last_change":"2023-05-16T07:17:19.141077+0000","last_active":"2023-05-16T07:17:19.145554+0000","last_peered":"2023-05-16T07:17:19.145554+0000","last_clean":"2023-05-16T07:17:19.145554+0000","last_became_active":"2023-05-16T07:17:19.140664+0000","last_became_peered":"2023-05-16T07:17:19.140664+0000","last_unstale":"2023-05-16T07:17:19.145554+0000","last_undegraded":"2023-05-16T07:17:19.145554+0000","last_fullsized":"2023-05-16T07:17:19.145554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.522880+0000","last_change":"2023-05-16T07:17:19.522880+0000","last_active":"2023-05-16T07:17:19.522880+0000","last_peered":"2023-05-16T07:17:19.522880+0000","last_clean":"2023-05-16T07:17:19.522880+0000","last_became_active":"2023-05-16T07:17:19.520002+0000","last_became_peered":"2023-05-16T07:17:19.520002+0000","last_unstale":"2023-05-16T07:17:19.522880+0000","last_undegraded":"2023-05-16T07:17:19.522880+0000","last_fullsized":"2023-05-16T07:17:19.522880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147750+0000","last_change":"2023-05-16T07:17:19.143468+0000","last_active":"2023-05-16T07:17:19.147750+0000","last_peered":"2023-05-16T07:17:19.147750+0000","last_clean":"2023-05-16T07:17:19.147750+0000","last_became_active":"2023-05-16T07:17:19.143182+0000","last_became_peered":"2023-05-16T07:17:19.143182+0000","last_unstale":"2023-05-16T07:17:19.147750+0000","last_undegraded":"2023-05-16T07:17:19.147750+0000","last_fullsized":"2023-05-16T07:17:19.147750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512442+0000","last_change":"2023-05-16T07:17:19.512442+0000","last_active":"2023-05-16T07:17:19.512442+0000","last_peered":"2023-05-16T07:17:19.512442+0000","last_clean":"2023-05-16T07:17:19.512442+0000","last_became_active":"2023-05-16T07:17:19.512181+0000","last_became_peered":"2023-05-16T07:17:19.512181+0000","last_unstale":"2023-05-16T07:17:19.512442+0000","last_undegraded":"2023-05-16T07:17:19.512442+0000","last_fullsized":"2023-05-16T07:17:19.512442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147778+0000","last_change":"2023-05-16T07:17:19.140034+0000","last_active":"2023-05-16T07:17:19.147778+0000","last_peered":"2023-05-16T07:17:19.147778+0000","last_clean":"2023-05-16T07:17:19.147778+0000","last_became_active":"2023-05-16T07:17:19.139466+0000","last_became_peered":"2023-05-16T07:17:19.139466+0000","last_unstale":"2023-05-16T07:17:19.147778+0000","last_undegraded":"2023-05-16T07:17:19.147778+0000","last_fullsized":"2023-05-16T07:17:19.147778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.525033+0000","last_change":"2023-05-16T07:17:19.525033+0000","last_active":"2023-05-16T07:17:19.525033+0000","last_peered":"2023-05-16T07:17:19.525033+0000","last_clean":"2023-05-16T07:17:19.525033+0000","last_became_active":"2023-05-16T07:17:19.524755+0000","last_became_peered":"2023-05-16T07:17:19.524755+0000","last_unstale":"2023-05-16T07:17:19.525033+0000","last_undegraded":"2023-05-16T07:17:19.525033+0000","last_fullsized":"2023-05-16T07:17:19.525033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"78'4","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519246+0000","last_change":"2023-05-16T07:17:19.146276+0000","last_active":"2023-05-16T07:17:19.519246+0000","last_peered":"2023-05-16T07:17:19.519246+0000","last_clean":"2023-05-16T07:17:19.519246+0000","last_became_active":"2023-05-16T07:17:19.145276+0000","last_became_peered":"2023-05-16T07:17:19.145276+0000","last_unstale":"2023-05-16T07:17:19.519246+0000","last_undegraded":"2023-05-16T07:17:19.519246+0000","last_fullsized":"2023-05-16T07:17:19.519246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506703+0000","last_change":"2023-05-16T07:17:19.506703+0000","last_active":"2023-05-16T07:17:19.506703+0000","last_peered":"2023-05-16T07:17:19.506703+0000","last_clean":"2023-05-16T07:17:19.506703+0000","last_became_active":"2023-05-16T07:17:19.505520+0000","last_became_peered":"2023-05-16T07:17:19.505520+0000","last_unstale":"2023-05-16T07:17:19.506703+0000","last_undegraded":"2023-05-16T07:17:19.506703+0000","last_fullsized":"2023-05-16T07:17:19.506703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.500203+0000","last_change":"2023-05-16T07:17:19.143837+0000","last_active":"2023-05-16T07:17:19.500203+0000","last_peered":"2023-05-16T07:17:19.500203+0000","last_clean":"2023-05-16T07:17:19.500203+0000","last_became_active":"2023-05-16T07:17:19.142940+0000","last_became_peered":"2023-05-16T07:17:19.142940+0000","last_unstale":"2023-05-16T07:17:19.500203+0000","last_undegraded":"2023-05-16T07:17:19.500203+0000","last_fullsized":"2023-05-16T07:17:19.500203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512595+0000","last_change":"2023-05-16T07:17:19.512595+0000","last_active":"2023-05-16T07:17:19.512595+0000","last_peered":"2023-05-16T07:17:19.512595+0000","last_clean":"2023-05-16T07:17:19.512595+0000","last_became_active":"2023-05-16T07:17:19.512303+0000","last_became_peered":"2023-05-16T07:17:19.512303+0000","last_unstale":"2023-05-16T07:17:19.512595+0000","last_undegraded":"2023-05-16T07:17:19.512595+0000","last_fullsized":"2023-05-16T07:17:19.512595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145602+0000","last_change":"2023-05-16T07:17:19.141413+0000","last_active":"2023-05-16T07:17:19.145602+0000","last_peered":"2023-05-16T07:17:19.145602+0000","last_clean":"2023-05-16T07:17:19.145602+0000","last_became_active":"2023-05-16T07:17:19.141125+0000","last_became_peered":"2023-05-16T07:17:19.141125+0000","last_unstale":"2023-05-16T07:17:19.145602+0000","last_undegraded":"2023-05-16T07:17:19.145602+0000","last_fullsized":"2023-05-16T07:17:19.145602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518530+0000","last_change":"2023-05-16T07:17:19.518530+0000","last_active":"2023-05-16T07:17:19.518530+0000","last_peered":"2023-05-16T07:17:19.518530+0000","last_clean":"2023-05-16T07:17:19.518530+0000","last_became_active":"2023-05-16T07:17:19.518238+0000","last_became_peered":"2023-05-16T07:17:19.518238+0000","last_unstale":"2023-05-16T07:17:19.518530+0000","last_undegraded":"2023-05-16T07:17:19.518530+0000","last_fullsized":"2023-05-16T07:17:19.518530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.142035+0000","last_change":"2023-05-16T07:17:19.137259+0000","last_active":"2023-05-16T07:17:19.142035+0000","last_peered":"2023-05-16T07:17:19.142035+0000","last_clean":"2023-05-16T07:17:19.142035+0000","last_became_active":"2023-05-16T07:17:19.136939+0000","last_became_peered":"2023-05-16T07:17:19.136939+0000","last_unstale":"2023-05-16T07:17:19.142035+0000","last_undegraded":"2023-05-16T07:17:19.142035+0000","last_fullsized":"2023-05-16T07:17:19.142035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518926+0000","last_change":"2023-05-16T07:17:19.518926+0000","last_active":"2023-05-16T07:17:19.518926+0000","last_peered":"2023-05-16T07:17:19.518926+0000","last_clean":"2023-05-16T07:17:19.518926+0000","last_became_active":"2023-05-16T07:17:19.518655+0000","last_became_peered":"2023-05-16T07:17:19.518655+0000","last_unstale":"2023-05-16T07:17:19.518926+0000","last_undegraded":"2023-05-16T07:17:19.518926+0000","last_fullsized":"2023-05-16T07:17:19.518926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"78'4","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517390+0000","last_change":"2023-05-16T07:17:19.141753+0000","last_active":"2023-05-16T07:17:19.517390+0000","last_peered":"2023-05-16T07:17:19.517390+0000","last_clean":"2023-05-16T07:17:19.517390+0000","last_became_active":"2023-05-16T07:17:19.141451+0000","last_became_peered":"2023-05-16T07:17:19.141451+0000","last_unstale":"2023-05-16T07:17:19.517390+0000","last_undegraded":"2023-05-16T07:17:19.517390+0000","last_fullsized":"2023-05-16T07:17:19.517390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519742+0000","last_change":"2023-05-16T07:17:19.519742+0000","last_active":"2023-05-16T07:17:19.519742+0000","last_peered":"2023-05-16T07:17:19.519742+0000","last_clean":"2023-05-16T07:17:19.519742+0000","last_became_active":"2023-05-16T07:17:19.519424+0000","last_became_peered":"2023-05-16T07:17:19.519424+0000","last_unstale":"2023-05-16T07:17:19.519742+0000","last_undegraded":"2023-05-16T07:17:19.519742+0000","last_fullsized":"2023-05-16T07:17:19.519742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147183+0000","last_change":"2023-05-16T07:17:19.143901+0000","last_active":"2023-05-16T07:17:19.147183+0000","last_peered":"2023-05-16T07:17:19.147183+0000","last_clean":"2023-05-16T07:17:19.147183+0000","last_became_active":"2023-05-16T07:17:19.143630+0000","last_became_peered":"2023-05-16T07:17:19.143630+0000","last_unstale":"2023-05-16T07:17:19.147183+0000","last_undegraded":"2023-05-16T07:17:19.147183+0000","last_fullsized":"2023-05-16T07:17:19.147183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513462+0000","last_change":"2023-05-16T07:17:19.513462+0000","last_active":"2023-05-16T07:17:19.513462+0000","last_peered":"2023-05-16T07:17:19.513462+0000","last_clean":"2023-05-16T07:17:19.513462+0000","last_became_active":"2023-05-16T07:17:19.513140+0000","last_became_peered":"2023-05-16T07:17:19.513140+0000","last_unstale":"2023-05-16T07:17:19.513462+0000","last_undegraded":"2023-05-16T07:17:19.513462+0000","last_fullsized":"2023-05-16T07:17:19.513462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.153503+0000","last_change":"2023-05-16T07:17:19.152221+0000","last_active":"2023-05-16T07:17:19.153503+0000","last_peered":"2023-05-16T07:17:19.153503+0000","last_clean":"2023-05-16T07:17:19.153503+0000","last_became_active":"2023-05-16T07:17:19.151948+0000","last_became_peered":"2023-05-16T07:17:19.151948+0000","last_unstale":"2023-05-16T07:17:19.153503+0000","last_undegraded":"2023-05-16T07:17:19.153503+0000","last_fullsized":"2023-05-16T07:17:19.153503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.520774+0000","last_change":"2023-05-16T07:17:19.520774+0000","last_active":"2023-05-16T07:17:19.520774+0000","last_peered":"2023-05-16T07:17:19.520774+0000","last_clean":"2023-05-16T07:17:19.520774+0000","last_became_active":"2023-05-16T07:17:19.520496+0000","last_became_peered":"2023-05-16T07:17:19.520496+0000","last_unstale":"2023-05-16T07:17:19.520774+0000","last_undegraded":"2023-05-16T07:17:19.520774+0000","last_fullsized":"2023-05-16T07:17:19.520774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.498483+0000","last_change":"2023-05-16T07:17:19.145077+0000","last_active":"2023-05-16T07:17:19.498483+0000","last_peered":"2023-05-16T07:17:19.498483+0000","last_clean":"2023-05-16T07:17:19.498483+0000","last_became_active":"2023-05-16T07:17:19.144763+0000","last_became_peered":"2023-05-16T07:17:19.144763+0000","last_unstale":"2023-05-16T07:17:19.498483+0000","last_undegraded":"2023-05-16T07:17:19.498483+0000","last_fullsized":"2023-05-16T07:17:19.498483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.524609+0000","last_change":"2023-05-16T07:17:19.524609+0000","last_active":"2023-05-16T07:17:19.524609+0000","last_peered":"2023-05-16T07:17:19.524609+0000","last_clean":"2023-05-16T07:17:19.524609+0000","last_became_active":"2023-05-16T07:17:19.524344+0000","last_became_peered":"2023-05-16T07:17:19.524344+0000","last_unstale":"2023-05-16T07:17:19.524609+0000","last_undegraded":"2023-05-16T07:17:19.524609+0000","last_fullsized":"2023-05-16T07:17:19.524609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151386+0000","last_change":"2023-05-16T07:17:19.146790+0000","last_active":"2023-05-16T07:17:19.151386+0000","last_peered":"2023-05-16T07:17:19.151386+0000","last_clean":"2023-05-16T07:17:19.151386+0000","last_became_active":"2023-05-16T07:17:19.146479+0000","last_became_peered":"2023-05-16T07:17:19.146479+0000","last_unstale":"2023-05-16T07:17:19.151386+0000","last_undegraded":"2023-05-16T07:17:19.151386+0000","last_fullsized":"2023-05-16T07:17:19.151386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510767+0000","last_change":"2023-05-16T07:17:19.510767+0000","last_active":"2023-05-16T07:17:19.510767+0000","last_peered":"2023-05-16T07:17:19.510767+0000","last_clean":"2023-05-16T07:17:19.510767+0000","last_became_active":"2023-05-16T07:17:19.510450+0000","last_became_peered":"2023-05-16T07:17:19.510450+0000","last_unstale":"2023-05-16T07:17:19.510767+0000","last_undegraded":"2023-05-16T07:17:19.510767+0000","last_fullsized":"2023-05-16T07:17:19.510767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506712+0000","last_change":"2023-05-16T07:17:19.149426+0000","last_active":"2023-05-16T07:17:19.506712+0000","last_peered":"2023-05-16T07:17:19.506712+0000","last_clean":"2023-05-16T07:17:19.506712+0000","last_became_active":"2023-05-16T07:17:19.149069+0000","last_became_peered":"2023-05-16T07:17:19.149069+0000","last_unstale":"2023-05-16T07:17:19.506712+0000","last_undegraded":"2023-05-16T07:17:19.506712+0000","last_fullsized":"2023-05-16T07:17:19.506712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513104+0000","last_change":"2023-05-16T07:17:19.513104+0000","last_active":"2023-05-16T07:17:19.513104+0000","last_peered":"2023-05-16T07:17:19.513104+0000","last_clean":"2023-05-16T07:17:19.513104+0000","last_became_active":"2023-05-16T07:17:19.512813+0000","last_became_peered":"2023-05-16T07:17:19.512813+0000","last_unstale":"2023-05-16T07:17:19.513104+0000","last_undegraded":"2023-05-16T07:17:19.513104+0000","last_fullsized":"2023-05-16T07:17:19.513104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147229+0000","last_change":"2023-05-16T07:17:19.143353+0000","last_active":"2023-05-16T07:17:19.147229+0000","last_peered":"2023-05-16T07:17:19.147229+0000","last_clean":"2023-05-16T07:17:19.147229+0000","last_became_active":"2023-05-16T07:17:19.142906+0000","last_became_peered":"2023-05-16T07:17:19.142906+0000","last_unstale":"2023-05-16T07:17:19.147229+0000","last_undegraded":"2023-05-16T07:17:19.147229+0000","last_fullsized":"2023-05-16T07:17:19.147229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513341+0000","last_change":"2023-05-16T07:17:19.513341+0000","last_active":"2023-05-16T07:17:19.513341+0000","last_peered":"2023-05-16T07:17:19.513341+0000","last_clean":"2023-05-16T07:17:19.513341+0000","last_became_active":"2023-05-16T07:17:19.512786+0000","last_became_peered":"2023-05-16T07:17:19.512786+0000","last_unstale":"2023-05-16T07:17:19.513341+0000","last_undegraded":"2023-05-16T07:17:19.513341+0000","last_fullsized":"2023-05-16T07:17:19.513341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.153456+0000","last_change":"2023-05-16T07:17:19.152043+0000","last_active":"2023-05-16T07:17:19.153456+0000","last_peered":"2023-05-16T07:17:19.153456+0000","last_clean":"2023-05-16T07:17:19.153456+0000","last_became_active":"2023-05-16T07:17:19.151779+0000","last_became_peered":"2023-05-16T07:17:19.151779+0000","last_unstale":"2023-05-16T07:17:19.153456+0000","last_undegraded":"2023-05-16T07:17:19.153456+0000","last_fullsized":"2023-05-16T07:17:19.153456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518210+0000","last_change":"2023-05-16T07:17:19.518210+0000","last_active":"2023-05-16T07:17:19.518210+0000","last_peered":"2023-05-16T07:17:19.518210+0000","last_clean":"2023-05-16T07:17:19.518210+0000","last_became_active":"2023-05-16T07:17:19.517913+0000","last_became_peered":"2023-05-16T07:17:19.517913+0000","last_unstale":"2023-05-16T07:17:19.518210+0000","last_undegraded":"2023-05-16T07:17:19.518210+0000","last_fullsized":"2023-05-16T07:17:19.518210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145415+0000","last_change":"2023-05-16T07:17:19.142419+0000","last_active":"2023-05-16T07:17:19.145415+0000","last_peered":"2023-05-16T07:17:19.145415+0000","last_clean":"2023-05-16T07:17:19.145415+0000","last_became_active":"2023-05-16T07:17:19.141893+0000","last_became_peered":"2023-05-16T07:17:19.141893+0000","last_unstale":"2023-05-16T07:17:19.145415+0000","last_undegraded":"2023-05-16T07:17:19.145415+0000","last_fullsized":"2023-05-16T07:17:19.145415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518269+0000","last_change":"2023-05-16T07:17:19.518269+0000","last_active":"2023-05-16T07:17:19.518269+0000","last_peered":"2023-05-16T07:17:19.518269+0000","last_clean":"2023-05-16T07:17:19.518269+0000","last_became_active":"2023-05-16T07:17:19.518075+0000","last_became_peered":"2023-05-16T07:17:19.518075+0000","last_unstale":"2023-05-16T07:17:19.518269+0000","last_undegraded":"2023-05-16T07:17:19.518269+0000","last_fullsized":"2023-05-16T07:17:19.518269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151474+0000","last_change":"2023-05-16T07:17:19.147071+0000","last_active":"2023-05-16T07:17:19.151474+0000","last_peered":"2023-05-16T07:17:19.151474+0000","last_clean":"2023-05-16T07:17:19.151474+0000","last_became_active":"2023-05-16T07:17:19.146537+0000","last_became_peered":"2023-05-16T07:17:19.146537+0000","last_unstale":"2023-05-16T07:17:19.151474+0000","last_undegraded":"2023-05-16T07:17:19.151474+0000","last_fullsized":"2023-05-16T07:17:19.151474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519990+0000","last_change":"2023-05-16T07:17:19.519990+0000","last_active":"2023-05-16T07:17:19.519990+0000","last_peered":"2023-05-16T07:17:19.519990+0000","last_clean":"2023-05-16T07:17:19.519990+0000","last_became_active":"2023-05-16T07:17:19.519439+0000","last_became_peered":"2023-05-16T07:17:19.519439+0000","last_unstale":"2023-05-16T07:17:19.519990+0000","last_undegraded":"2023-05-16T07:17:19.519990+0000","last_fullsized":"2023-05-16T07:17:19.519990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.150325+0000","last_change":"2023-05-16T07:17:19.146861+0000","last_active":"2023-05-16T07:17:19.150325+0000","last_peered":"2023-05-16T07:17:19.150325+0000","last_clean":"2023-05-16T07:17:19.150325+0000","last_became_active":"2023-05-16T07:17:19.146593+0000","last_became_peered":"2023-05-16T07:17:19.146593+0000","last_unstale":"2023-05-16T07:17:19.150325+0000","last_undegraded":"2023-05-16T07:17:19.150325+0000","last_fullsized":"2023-05-16T07:17:19.150325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511941+0000","last_change":"2023-05-16T07:17:19.511941+0000","last_active":"2023-05-16T07:17:19.511941+0000","last_peered":"2023-05-16T07:17:19.511941+0000","last_clean":"2023-05-16T07:17:19.511941+0000","last_became_active":"2023-05-16T07:17:19.511686+0000","last_became_peered":"2023-05-16T07:17:19.511686+0000","last_unstale":"2023-05-16T07:17:19.511941+0000","last_undegraded":"2023-05-16T07:17:19.511941+0000","last_fullsized":"2023-05-16T07:17:19.511941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146167+0000","last_change":"2023-05-16T07:17:19.141747+0000","last_active":"2023-05-16T07:17:19.146167+0000","last_peered":"2023-05-16T07:17:19.146167+0000","last_clean":"2023-05-16T07:17:19.146167+0000","last_became_active":"2023-05-16T07:17:19.141440+0000","last_became_peered":"2023-05-16T07:17:19.141440+0000","last_unstale":"2023-05-16T07:17:19.146167+0000","last_undegraded":"2023-05-16T07:17:19.146167+0000","last_fullsized":"2023-05-16T07:17:19.146167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515727+0000","last_change":"2023-05-16T07:17:19.515727+0000","last_active":"2023-05-16T07:17:19.515727+0000","last_peered":"2023-05-16T07:17:19.515727+0000","last_clean":"2023-05-16T07:17:19.515727+0000","last_became_active":"2023-05-16T07:17:19.515452+0000","last_became_peered":"2023-05-16T07:17:19.515452+0000","last_unstale":"2023-05-16T07:17:19.515727+0000","last_undegraded":"2023-05-16T07:17:19.515727+0000","last_fullsized":"2023-05-16T07:17:19.515727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151377+0000","last_change":"2023-05-16T07:17:19.149065+0000","last_active":"2023-05-16T07:17:19.151377+0000","last_peered":"2023-05-16T07:17:19.151377+0000","last_clean":"2023-05-16T07:17:19.151377+0000","last_became_active":"2023-05-16T07:17:19.148774+0000","last_became_peered":"2023-05-16T07:17:19.148774+0000","last_unstale":"2023-05-16T07:17:19.151377+0000","last_undegraded":"2023-05-16T07:17:19.151377+0000","last_fullsized":"2023-05-16T07:17:19.151377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515477+0000","last_change":"2023-05-16T07:17:19.515477+0000","last_active":"2023-05-16T07:17:19.515477+0000","last_peered":"2023-05-16T07:17:19.515477+0000","last_clean":"2023-05-16T07:17:19.515477+0000","last_became_active":"2023-05-16T07:17:19.515214+0000","last_became_peered":"2023-05-16T07:17:19.515214+0000","last_unstale":"2023-05-16T07:17:19.515477+0000","last_undegraded":"2023-05-16T07:17:19.515477+0000","last_fullsized":"2023-05-16T07:17:19.515477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147343+0000","last_change":"2023-05-16T07:17:19.145511+0000","last_active":"2023-05-16T07:17:19.147343+0000","last_peered":"2023-05-16T07:17:19.147343+0000","last_clean":"2023-05-16T07:17:19.147343+0000","last_became_active":"2023-05-16T07:17:19.145243+0000","last_became_peered":"2023-05-16T07:17:19.145243+0000","last_unstale":"2023-05-16T07:17:19.147343+0000","last_undegraded":"2023-05-16T07:17:19.147343+0000","last_fullsized":"2023-05-16T07:17:19.147343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515984+0000","last_change":"2023-05-16T07:17:19.515984+0000","last_active":"2023-05-16T07:17:19.515984+0000","last_peered":"2023-05-16T07:17:19.515984+0000","last_clean":"2023-05-16T07:17:19.515984+0000","last_became_active":"2023-05-16T07:17:19.515690+0000","last_became_peered":"2023-05-16T07:17:19.515690+0000","last_unstale":"2023-05-16T07:17:19.515984+0000","last_undegraded":"2023-05-16T07:17:19.515984+0000","last_fullsized":"2023-05-16T07:17:19.515984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.149684+0000","last_change":"2023-05-16T07:17:19.146729+0000","last_active":"2023-05-16T07:17:19.149684+0000","last_peered":"2023-05-16T07:17:19.149684+0000","last_clean":"2023-05-16T07:17:19.149684+0000","last_became_active":"2023-05-16T07:17:19.146474+0000","last_became_peered":"2023-05-16T07:17:19.146474+0000","last_unstale":"2023-05-16T07:17:19.149684+0000","last_undegraded":"2023-05-16T07:17:19.149684+0000","last_fullsized":"2023-05-16T07:17:19.149684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515316+0000","last_change":"2023-05-16T07:17:19.515316+0000","last_active":"2023-05-16T07:17:19.515316+0000","last_peered":"2023-05-16T07:17:19.515316+0000","last_clean":"2023-05-16T07:17:19.515316+0000","last_became_active":"2023-05-16T07:17:19.515022+0000","last_became_peered":"2023-05-16T07:17:19.515022+0000","last_unstale":"2023-05-16T07:17:19.515316+0000","last_undegraded":"2023-05-16T07:17:19.515316+0000","last_fullsized":"2023-05-16T07:17:19.515316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151425+0000","last_change":"2023-05-16T07:17:19.150026+0000","last_active":"2023-05-16T07:17:19.151425+0000","last_peered":"2023-05-16T07:17:19.151425+0000","last_clean":"2023-05-16T07:17:19.151425+0000","last_became_active":"2023-05-16T07:17:19.149733+0000","last_became_peered":"2023-05-16T07:17:19.149733+0000","last_unstale":"2023-05-16T07:17:19.151425+0000","last_undegraded":"2023-05-16T07:17:19.151425+0000","last_fullsized":"2023-05-16T07:17:19.151425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.524598+0000","last_change":"2023-05-16T07:17:19.524598+0000","last_active":"2023-05-16T07:17:19.524598+0000","last_peered":"2023-05-16T07:17:19.524598+0000","last_clean":"2023-05-16T07:17:19.524598+0000","last_became_active":"2023-05-16T07:17:19.524321+0000","last_became_peered":"2023-05-16T07:17:19.524321+0000","last_unstale":"2023-05-16T07:17:19.524598+0000","last_undegraded":"2023-05-16T07:17:19.524598+0000","last_fullsized":"2023-05-16T07:17:19.524598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.152140+0000","last_change":"2023-05-16T07:17:19.148333+0000","last_active":"2023-05-16T07:17:19.152140+0000","last_peered":"2023-05-16T07:17:19.152140+0000","last_clean":"2023-05-16T07:17:19.152140+0000","last_became_active":"2023-05-16T07:17:19.148064+0000","last_became_peered":"2023-05-16T07:17:19.148064+0000","last_unstale":"2023-05-16T07:17:19.152140+0000","last_undegraded":"2023-05-16T07:17:19.152140+0000","last_fullsized":"2023-05-16T07:17:19.152140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518476+0000","last_change":"2023-05-16T07:17:19.518476+0000","last_active":"2023-05-16T07:17:19.518476+0000","last_peered":"2023-05-16T07:17:19.518476+0000","last_clean":"2023-05-16T07:17:19.518476+0000","last_became_active":"2023-05-16T07:17:19.518199+0000","last_became_peered":"2023-05-16T07:17:19.518199+0000","last_unstale":"2023-05-16T07:17:19.518476+0000","last_undegraded":"2023-05-16T07:17:19.518476+0000","last_fullsized":"2023-05-16T07:17:19.518476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.507701+0000","last_change":"2023-05-16T07:17:19.148411+0000","last_active":"2023-05-16T07:17:19.507701+0000","last_peered":"2023-05-16T07:17:19.507701+0000","last_clean":"2023-05-16T07:17:19.507701+0000","last_became_active":"2023-05-16T07:17:19.146937+0000","last_became_peered":"2023-05-16T07:17:19.146937+0000","last_unstale":"2023-05-16T07:17:19.507701+0000","last_undegraded":"2023-05-16T07:17:19.507701+0000","last_fullsized":"2023-05-16T07:17:19.507701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519739+0000","last_change":"2023-05-16T07:17:19.519739+0000","last_active":"2023-05-16T07:17:19.519739+0000","last_peered":"2023-05-16T07:17:19.519739+0000","last_clean":"2023-05-16T07:17:19.519739+0000","last_became_active":"2023-05-16T07:17:19.516776+0000","last_became_peered":"2023-05-16T07:17:19.516776+0000","last_unstale":"2023-05-16T07:17:19.519739+0000","last_undegraded":"2023-05-16T07:17:19.519739+0000","last_fullsized":"2023-05-16T07:17:19.519739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.505122+0000","last_change":"2023-05-16T07:17:19.147103+0000","last_active":"2023-05-16T07:17:19.505122+0000","last_peered":"2023-05-16T07:17:19.505122+0000","last_clean":"2023-05-16T07:17:19.505122+0000","last_became_active":"2023-05-16T07:17:19.146097+0000","last_became_peered":"2023-05-16T07:17:19.146097+0000","last_unstale":"2023-05-16T07:17:19.505122+0000","last_undegraded":"2023-05-16T07:17:19.505122+0000","last_fullsized":"2023-05-16T07:17:19.505122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511185+0000","last_change":"2023-05-16T07:17:19.511185+0000","last_active":"2023-05-16T07:17:19.511185+0000","last_peered":"2023-05-16T07:17:19.511185+0000","last_clean":"2023-05-16T07:17:19.511185+0000","last_became_active":"2023-05-16T07:17:19.510914+0000","last_became_peered":"2023-05-16T07:17:19.510914+0000","last_unstale":"2023-05-16T07:17:19.511185+0000","last_undegraded":"2023-05-16T07:17:19.511185+0000","last_fullsized":"2023-05-16T07:17:19.511185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518798+0000","last_change":"2023-05-16T07:17:19.144810+0000","last_active":"2023-05-16T07:17:19.518798+0000","last_peered":"2023-05-16T07:17:19.518798+0000","last_clean":"2023-05-16T07:17:19.518798+0000","last_became_active":"2023-05-16T07:17:19.144540+0000","last_became_peered":"2023-05-16T07:17:19.144540+0000","last_unstale":"2023-05-16T07:17:19.518798+0000","last_undegraded":"2023-05-16T07:17:19.518798+0000","last_fullsized":"2023-05-16T07:17:19.518798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516844+0000","last_change":"2023-05-16T07:17:19.516844+0000","last_active":"2023-05-16T07:17:19.516844+0000","last_peered":"2023-05-16T07:17:19.516844+0000","last_clean":"2023-05-16T07:17:19.516844+0000","last_became_active":"2023-05-16T07:17:19.516529+0000","last_became_peered":"2023-05-16T07:17:19.516529+0000","last_unstale":"2023-05-16T07:17:19.516844+0000","last_undegraded":"2023-05-16T07:17:19.516844+0000","last_fullsized":"2023-05-16T07:17:19.516844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.152209+0000","last_change":"2023-05-16T07:17:19.150631+0000","last_active":"2023-05-16T07:17:19.152209+0000","last_peered":"2023-05-16T07:17:19.152209+0000","last_clean":"2023-05-16T07:17:19.152209+0000","last_became_active":"2023-05-16T07:17:19.150358+0000","last_became_peered":"2023-05-16T07:17:19.150358+0000","last_unstale":"2023-05-16T07:17:19.152209+0000","last_undegraded":"2023-05-16T07:17:19.152209+0000","last_fullsized":"2023-05-16T07:17:19.152209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.509924+0000","last_change":"2023-05-16T07:17:19.509924+0000","last_active":"2023-05-16T07:17:19.509924+0000","last_peered":"2023-05-16T07:17:19.509924+0000","last_clean":"2023-05-16T07:17:19.509924+0000","last_became_active":"2023-05-16T07:17:19.509634+0000","last_became_peered":"2023-05-16T07:17:19.509634+0000","last_unstale":"2023-05-16T07:17:19.509924+0000","last_undegraded":"2023-05-16T07:17:19.509924+0000","last_fullsized":"2023-05-16T07:17:19.509924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151438+0000","last_change":"2023-05-16T07:17:19.146889+0000","last_active":"2023-05-16T07:17:19.151438+0000","last_peered":"2023-05-16T07:17:19.151438+0000","last_clean":"2023-05-16T07:17:19.151438+0000","last_became_active":"2023-05-16T07:17:19.146563+0000","last_became_peered":"2023-05-16T07:17:19.146563+0000","last_unstale":"2023-05-16T07:17:19.151438+0000","last_undegraded":"2023-05-16T07:17:19.151438+0000","last_fullsized":"2023-05-16T07:17:19.151438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513585+0000","last_change":"2023-05-16T07:17:19.513585+0000","last_active":"2023-05-16T07:17:19.513585+0000","last_peered":"2023-05-16T07:17:19.513585+0000","last_clean":"2023-05-16T07:17:19.513585+0000","last_became_active":"2023-05-16T07:17:19.513289+0000","last_became_peered":"2023-05-16T07:17:19.513289+0000","last_unstale":"2023-05-16T07:17:19.513585+0000","last_undegraded":"2023-05-16T07:17:19.513585+0000","last_fullsized":"2023-05-16T07:17:19.513585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147210+0000","last_change":"2023-05-16T07:17:19.143176+0000","last_active":"2023-05-16T07:17:19.147210+0000","last_peered":"2023-05-16T07:17:19.147210+0000","last_clean":"2023-05-16T07:17:19.147210+0000","last_became_active":"2023-05-16T07:17:19.142570+0000","last_became_peered":"2023-05-16T07:17:19.142570+0000","last_unstale":"2023-05-16T07:17:19.147210+0000","last_undegraded":"2023-05-16T07:17:19.147210+0000","last_fullsized":"2023-05-16T07:17:19.147210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517329+0000","last_change":"2023-05-16T07:17:19.517329+0000","last_active":"2023-05-16T07:17:19.517329+0000","last_peered":"2023-05-16T07:17:19.517329+0000","last_clean":"2023-05-16T07:17:19.517329+0000","last_became_active":"2023-05-16T07:17:19.516706+0000","last_became_peered":"2023-05-16T07:17:19.516706+0000","last_unstale":"2023-05-16T07:17:19.517329+0000","last_undegraded":"2023-05-16T07:17:19.517329+0000","last_fullsized":"2023-05-16T07:17:19.517329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146193+0000","last_change":"2023-05-16T07:17:19.139471+0000","last_active":"2023-05-16T07:17:19.146193+0000","last_peered":"2023-05-16T07:17:19.146193+0000","last_clean":"2023-05-16T07:17:19.146193+0000","last_became_active":"2023-05-16T07:17:19.138683+0000","last_became_peered":"2023-05-16T07:17:19.138683+0000","last_unstale":"2023-05-16T07:17:19.146193+0000","last_undegraded":"2023-05-16T07:17:19.146193+0000","last_fullsized":"2023-05-16T07:17:19.146193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512399+0000","last_change":"2023-05-16T07:17:19.512399+0000","last_active":"2023-05-16T07:17:19.512399+0000","last_peered":"2023-05-16T07:17:19.512399+0000","last_clean":"2023-05-16T07:17:19.512399+0000","last_became_active":"2023-05-16T07:17:19.512115+0000","last_became_peered":"2023-05-16T07:17:19.512115+0000","last_unstale":"2023-05-16T07:17:19.512399+0000","last_undegraded":"2023-05-16T07:17:19.512399+0000","last_fullsized":"2023-05-16T07:17:19.512399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.149732+0000","last_change":"2023-05-16T07:17:19.146828+0000","last_active":"2023-05-16T07:17:19.149732+0000","last_peered":"2023-05-16T07:17:19.149732+0000","last_clean":"2023-05-16T07:17:19.149732+0000","last_became_active":"2023-05-16T07:17:19.146553+0000","last_became_peered":"2023-05-16T07:17:19.146553+0000","last_unstale":"2023-05-16T07:17:19.149732+0000","last_undegraded":"2023-05-16T07:17:19.149732+0000","last_fullsized":"2023-05-16T07:17:19.149732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512096+0000","last_change":"2023-05-16T07:17:19.512096+0000","last_active":"2023-05-16T07:17:19.512096+0000","last_peered":"2023-05-16T07:17:19.512096+0000","last_clean":"2023-05-16T07:17:19.512096+0000","last_became_active":"2023-05-16T07:17:19.511858+0000","last_became_peered":"2023-05-16T07:17:19.511858+0000","last_unstale":"2023-05-16T07:17:19.512096+0000","last_undegraded":"2023-05-16T07:17:19.512096+0000","last_fullsized":"2023-05-16T07:17:19.512096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.154015+0000","last_change":"2023-05-16T07:17:19.153507+0000","last_active":"2023-05-16T07:17:19.154015+0000","last_peered":"2023-05-16T07:17:19.154015+0000","last_clean":"2023-05-16T07:17:19.154015+0000","last_became_active":"2023-05-16T07:17:19.153218+0000","last_became_peered":"2023-05-16T07:17:19.153218+0000","last_unstale":"2023-05-16T07:17:19.154015+0000","last_undegraded":"2023-05-16T07:17:19.154015+0000","last_fullsized":"2023-05-16T07:17:19.154015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516452+0000","last_change":"2023-05-16T07:17:19.516452+0000","last_active":"2023-05-16T07:17:19.516452+0000","last_peered":"2023-05-16T07:17:19.516452+0000","last_clean":"2023-05-16T07:17:19.516452+0000","last_became_active":"2023-05-16T07:17:19.516242+0000","last_became_peered":"2023-05-16T07:17:19.516242+0000","last_unstale":"2023-05-16T07:17:19.516452+0000","last_undegraded":"2023-05-16T07:17:19.516452+0000","last_fullsized":"2023-05-16T07:17:19.516452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.144812+0000","last_change":"2023-05-16T07:17:19.138064+0000","last_active":"2023-05-16T07:17:19.144812+0000","last_peered":"2023-05-16T07:17:19.144812+0000","last_clean":"2023-05-16T07:17:19.144812+0000","last_became_active":"2023-05-16T07:17:19.137794+0000","last_became_peered":"2023-05-16T07:17:19.137794+0000","last_unstale":"2023-05-16T07:17:19.144812+0000","last_undegraded":"2023-05-16T07:17:19.144812+0000","last_fullsized":"2023-05-16T07:17:19.144812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.521925+0000","last_change":"2023-05-16T07:17:19.521925+0000","last_active":"2023-05-16T07:17:19.521925+0000","last_peered":"2023-05-16T07:17:19.521925+0000","last_clean":"2023-05-16T07:17:19.521925+0000","last_became_active":"2023-05-16T07:17:19.521635+0000","last_became_peered":"2023-05-16T07:17:19.521635+0000","last_unstale":"2023-05-16T07:17:19.521925+0000","last_undegraded":"2023-05-16T07:17:19.521925+0000","last_fullsized":"2023-05-16T07:17:19.521925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"78'3","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516181+0000","last_change":"2023-05-16T07:17:19.149599+0000","last_active":"2023-05-16T07:17:19.516181+0000","last_peered":"2023-05-16T07:17:19.516181+0000","last_clean":"2023-05-16T07:17:19.516181+0000","last_became_active":"2023-05-16T07:17:19.149216+0000","last_became_peered":"2023-05-16T07:17:19.149216+0000","last_unstale":"2023-05-16T07:17:19.516181+0000","last_undegraded":"2023-05-16T07:17:19.516181+0000","last_fullsized":"2023-05-16T07:17:19.516181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517208+0000","last_change":"2023-05-16T07:17:19.517208+0000","last_active":"2023-05-16T07:17:19.517208+0000","last_peered":"2023-05-16T07:17:19.517208+0000","last_clean":"2023-05-16T07:17:19.517208+0000","last_became_active":"2023-05-16T07:17:19.516916+0000","last_became_peered":"2023-05-16T07:17:19.516916+0000","last_unstale":"2023-05-16T07:17:19.517208+0000","last_undegraded":"2023-05-16T07:17:19.517208+0000","last_fullsized":"2023-05-16T07:17:19.517208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.524140+0000","last_change":"2023-05-16T07:17:19.140024+0000","last_active":"2023-05-16T07:17:19.524140+0000","last_peered":"2023-05-16T07:17:19.524140+0000","last_clean":"2023-05-16T07:17:19.524140+0000","last_became_active":"2023-05-16T07:17:19.139570+0000","last_became_peered":"2023-05-16T07:17:19.139570+0000","last_unstale":"2023-05-16T07:17:19.524140+0000","last_undegraded":"2023-05-16T07:17:19.524140+0000","last_fullsized":"2023-05-16T07:17:19.524140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.522144+0000","last_change":"2023-05-16T07:17:19.522144+0000","last_active":"2023-05-16T07:17:19.522144+0000","last_peered":"2023-05-16T07:17:19.522144+0000","last_clean":"2023-05-16T07:17:19.522144+0000","last_became_active":"2023-05-16T07:17:19.521859+0000","last_became_peered":"2023-05-16T07:17:19.521859+0000","last_unstale":"2023-05-16T07:17:19.522144+0000","last_undegraded":"2023-05-16T07:17:19.522144+0000","last_fullsized":"2023-05-16T07:17:19.522144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506821+0000","last_change":"2023-05-16T07:17:19.152636+0000","last_active":"2023-05-16T07:17:19.506821+0000","last_peered":"2023-05-16T07:17:19.506821+0000","last_clean":"2023-05-16T07:17:19.506821+0000","last_became_active":"2023-05-16T07:17:19.151206+0000","last_became_peered":"2023-05-16T07:17:19.151206+0000","last_unstale":"2023-05-16T07:17:19.506821+0000","last_undegraded":"2023-05-16T07:17:19.506821+0000","last_fullsized":"2023-05-16T07:17:19.506821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511571+0000","last_change":"2023-05-16T07:17:19.511571+0000","last_active":"2023-05-16T07:17:19.511571+0000","last_peered":"2023-05-16T07:17:19.511571+0000","last_clean":"2023-05-16T07:17:19.511571+0000","last_became_active":"2023-05-16T07:17:19.511285+0000","last_became_peered":"2023-05-16T07:17:19.511285+0000","last_unstale":"2023-05-16T07:17:19.511571+0000","last_undegraded":"2023-05-16T07:17:19.511571+0000","last_fullsized":"2023-05-16T07:17:19.511571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.150832+0000","last_change":"2023-05-16T07:17:19.149874+0000","last_active":"2023-05-16T07:17:19.150832+0000","last_peered":"2023-05-16T07:17:19.150832+0000","last_clean":"2023-05-16T07:17:19.150832+0000","last_became_active":"2023-05-16T07:17:19.149595+0000","last_became_peered":"2023-05-16T07:17:19.149595+0000","last_unstale":"2023-05-16T07:17:19.150832+0000","last_undegraded":"2023-05-16T07:17:19.150832+0000","last_fullsized":"2023-05-16T07:17:19.150832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510410+0000","last_change":"2023-05-16T07:17:19.510410+0000","last_active":"2023-05-16T07:17:19.510410+0000","last_peered":"2023-05-16T07:17:19.510410+0000","last_clean":"2023-05-16T07:17:19.510410+0000","last_became_active":"2023-05-16T07:17:19.509417+0000","last_became_peered":"2023-05-16T07:17:19.509417+0000","last_unstale":"2023-05-16T07:17:19.510410+0000","last_undegraded":"2023-05-16T07:17:19.510410+0000","last_fullsized":"2023-05-16T07:17:19.510410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151381+0000","last_change":"2023-05-16T07:17:19.148341+0000","last_active":"2023-05-16T07:17:19.151381+0000","last_peered":"2023-05-16T07:17:19.151381+0000","last_clean":"2023-05-16T07:17:19.151381+0000","last_became_active":"2023-05-16T07:17:19.148060+0000","last_became_peered":"2023-05-16T07:17:19.148060+0000","last_unstale":"2023-05-16T07:17:19.151381+0000","last_undegraded":"2023-05-16T07:17:19.151381+0000","last_fullsized":"2023-05-16T07:17:19.151381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515607+0000","last_change":"2023-05-16T07:17:19.515607+0000","last_active":"2023-05-16T07:17:19.515607+0000","last_peered":"2023-05-16T07:17:19.515607+0000","last_clean":"2023-05-16T07:17:19.515607+0000","last_became_active":"2023-05-16T07:17:19.515347+0000","last_became_peered":"2023-05-16T07:17:19.515347+0000","last_unstale":"2023-05-16T07:17:19.515607+0000","last_undegraded":"2023-05-16T07:17:19.515607+0000","last_fullsized":"2023-05-16T07:17:19.515607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"78'4","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.521939+0000","last_change":"2023-05-16T07:17:19.141719+0000","last_active":"2023-05-16T07:17:19.521939+0000","last_peered":"2023-05-16T07:17:19.521939+0000","last_clean":"2023-05-16T07:17:19.521939+0000","last_became_active":"2023-05-16T07:17:19.140418+0000","last_became_peered":"2023-05-16T07:17:19.140418+0000","last_unstale":"2023-05-16T07:17:19.521939+0000","last_undegraded":"2023-05-16T07:17:19.521939+0000","last_fullsized":"2023-05-16T07:17:19.521939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.521707+0000","last_change":"2023-05-16T07:17:19.521707+0000","last_active":"2023-05-16T07:17:19.521707+0000","last_peered":"2023-05-16T07:17:19.521707+0000","last_clean":"2023-05-16T07:17:19.521707+0000","last_became_active":"2023-05-16T07:17:19.521396+0000","last_became_peered":"2023-05-16T07:17:19.521396+0000","last_unstale":"2023-05-16T07:17:19.521707+0000","last_undegraded":"2023-05-16T07:17:19.521707+0000","last_fullsized":"2023-05-16T07:17:19.521707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515843+0000","last_change":"2023-05-16T07:17:19.146483+0000","last_active":"2023-05-16T07:17:19.515843+0000","last_peered":"2023-05-16T07:17:19.515843+0000","last_clean":"2023-05-16T07:17:19.515843+0000","last_became_active":"2023-05-16T07:17:19.146137+0000","last_became_peered":"2023-05-16T07:17:19.146137+0000","last_unstale":"2023-05-16T07:17:19.515843+0000","last_undegraded":"2023-05-16T07:17:19.515843+0000","last_fullsized":"2023-05-16T07:17:19.515843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511259+0000","last_change":"2023-05-16T07:17:19.511259+0000","last_active":"2023-05-16T07:17:19.511259+0000","last_peered":"2023-05-16T07:17:19.511259+0000","last_clean":"2023-05-16T07:17:19.511259+0000","last_became_active":"2023-05-16T07:17:19.510981+0000","last_became_peered":"2023-05-16T07:17:19.510981+0000","last_unstale":"2023-05-16T07:17:19.511259+0000","last_undegraded":"2023-05-16T07:17:19.511259+0000","last_fullsized":"2023-05-16T07:17:19.511259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.148446+0000","last_change":"2023-05-16T07:17:19.144404+0000","last_active":"2023-05-16T07:17:19.148446+0000","last_peered":"2023-05-16T07:17:19.148446+0000","last_clean":"2023-05-16T07:17:19.148446+0000","last_became_active":"2023-05-16T07:17:19.144084+0000","last_became_peered":"2023-05-16T07:17:19.144084+0000","last_unstale":"2023-05-16T07:17:19.148446+0000","last_undegraded":"2023-05-16T07:17:19.148446+0000","last_fullsized":"2023-05-16T07:17:19.148446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514210+0000","last_change":"2023-05-16T07:17:19.514210+0000","last_active":"2023-05-16T07:17:19.514210+0000","last_peered":"2023-05-16T07:17:19.514210+0000","last_clean":"2023-05-16T07:17:19.514210+0000","last_became_active":"2023-05-16T07:17:19.513894+0000","last_became_peered":"2023-05-16T07:17:19.513894+0000","last_unstale":"2023-05-16T07:17:19.514210+0000","last_undegraded":"2023-05-16T07:17:19.514210+0000","last_fullsized":"2023-05-16T07:17:19.514210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145473+0000","last_change":"2023-05-16T07:17:19.137803+0000","last_active":"2023-05-16T07:17:19.145473+0000","last_peered":"2023-05-16T07:17:19.145473+0000","last_clean":"2023-05-16T07:17:19.145473+0000","last_became_active":"2023-05-16T07:17:19.137474+0000","last_became_peered":"2023-05-16T07:17:19.137474+0000","last_unstale":"2023-05-16T07:17:19.145473+0000","last_undegraded":"2023-05-16T07:17:19.145473+0000","last_fullsized":"2023-05-16T07:17:19.145473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517508+0000","last_change":"2023-05-16T07:17:19.517508+0000","last_active":"2023-05-16T07:17:19.517508+0000","last_peered":"2023-05-16T07:17:19.517508+0000","last_clean":"2023-05-16T07:17:19.517508+0000","last_became_active":"2023-05-16T07:17:19.517019+0000","last_became_peered":"2023-05-16T07:17:19.517019+0000","last_unstale":"2023-05-16T07:17:19.517508+0000","last_undegraded":"2023-05-16T07:17:19.517508+0000","last_fullsized":"2023-05-16T07:17:19.517508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.148515+0000","last_change":"2023-05-16T07:17:19.139835+0000","last_active":"2023-05-16T07:17:19.148515+0000","last_peered":"2023-05-16T07:17:19.148515+0000","last_clean":"2023-05-16T07:17:19.148515+0000","last_became_active":"2023-05-16T07:17:19.139214+0000","last_became_peered":"2023-05-16T07:17:19.139214+0000","last_unstale":"2023-05-16T07:17:19.148515+0000","last_undegraded":"2023-05-16T07:17:19.148515+0000","last_fullsized":"2023-05-16T07:17:19.148515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512598+0000","last_change":"2023-05-16T07:17:19.512598+0000","last_active":"2023-05-16T07:17:19.512598+0000","last_peered":"2023-05-16T07:17:19.512598+0000","last_clean":"2023-05-16T07:17:19.512598+0000","last_became_active":"2023-05-16T07:17:19.512331+0000","last_became_peered":"2023-05-16T07:17:19.512331+0000","last_unstale":"2023-05-16T07:17:19.512598+0000","last_undegraded":"2023-05-16T07:17:19.512598+0000","last_fullsized":"2023-05-16T07:17:19.512598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151343+0000","last_change":"2023-05-16T07:17:19.148092+0000","last_active":"2023-05-16T07:17:19.151343+0000","last_peered":"2023-05-16T07:17:19.151343+0000","last_clean":"2023-05-16T07:17:19.151343+0000","last_became_active":"2023-05-16T07:17:19.147750+0000","last_became_peered":"2023-05-16T07:17:19.147750+0000","last_unstale":"2023-05-16T07:17:19.151343+0000","last_undegraded":"2023-05-16T07:17:19.151343+0000","last_fullsized":"2023-05-16T07:17:19.151343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.520746+0000","last_change":"2023-05-16T07:17:19.520746+0000","last_active":"2023-05-16T07:17:19.520746+0000","last_peered":"2023-05-16T07:17:19.520746+0000","last_clean":"2023-05-16T07:17:19.520746+0000","last_became_active":"2023-05-16T07:17:19.520457+0000","last_became_peered":"2023-05-16T07:17:19.520457+0000","last_unstale":"2023-05-16T07:17:19.520746+0000","last_undegraded":"2023-05-16T07:17:19.520746+0000","last_fullsized":"2023-05-16T07:17:19.520746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151584+0000","last_change":"2023-05-16T07:17:19.149907+0000","last_active":"2023-05-16T07:17:19.151584+0000","last_peered":"2023-05-16T07:17:19.151584+0000","last_clean":"2023-05-16T07:17:19.151584+0000","last_became_active":"2023-05-16T07:17:19.149599+0000","last_became_peered":"2023-05-16T07:17:19.149599+0000","last_unstale":"2023-05-16T07:17:19.151584+0000","last_undegraded":"2023-05-16T07:17:19.151584+0000","last_fullsized":"2023-05-16T07:17:19.151584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514675+0000","last_change":"2023-05-16T07:17:19.514675+0000","last_active":"2023-05-16T07:17:19.514675+0000","last_peered":"2023-05-16T07:17:19.514675+0000","last_clean":"2023-05-16T07:17:19.514675+0000","last_became_active":"2023-05-16T07:17:19.514456+0000","last_became_peered":"2023-05-16T07:17:19.514456+0000","last_unstale":"2023-05-16T07:17:19.514675+0000","last_undegraded":"2023-05-16T07:17:19.514675+0000","last_fullsized":"2023-05-16T07:17:19.514675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146172+0000","last_change":"2023-05-16T07:17:19.138961+0000","last_active":"2023-05-16T07:17:19.146172+0000","last_peered":"2023-05-16T07:17:19.146172+0000","last_clean":"2023-05-16T07:17:19.146172+0000","last_became_active":"2023-05-16T07:17:19.138304+0000","last_became_peered":"2023-05-16T07:17:19.138304+0000","last_unstale":"2023-05-16T07:17:19.146172+0000","last_undegraded":"2023-05-16T07:17:19.146172+0000","last_fullsized":"2023-05-16T07:17:19.146172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506518+0000","last_change":"2023-05-16T07:17:19.506518+0000","last_active":"2023-05-16T07:17:19.506518+0000","last_peered":"2023-05-16T07:17:19.506518+0000","last_clean":"2023-05-16T07:17:19.506518+0000","last_became_active":"2023-05-16T07:17:19.506309+0000","last_became_peered":"2023-05-16T07:17:19.506309+0000","last_unstale":"2023-05-16T07:17:19.506518+0000","last_undegraded":"2023-05-16T07:17:19.506518+0000","last_fullsized":"2023-05-16T07:17:19.506518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147984+0000","last_change":"2023-05-16T07:17:19.140206+0000","last_active":"2023-05-16T07:17:19.147984+0000","last_peered":"2023-05-16T07:17:19.147984+0000","last_clean":"2023-05-16T07:17:19.147984+0000","last_became_active":"2023-05-16T07:17:19.139903+0000","last_became_peered":"2023-05-16T07:17:19.139903+0000","last_unstale":"2023-05-16T07:17:19.147984+0000","last_undegraded":"2023-05-16T07:17:19.147984+0000","last_fullsized":"2023-05-16T07:17:19.147984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.525240+0000","last_change":"2023-05-16T07:17:19.525240+0000","last_active":"2023-05-16T07:17:19.525240+0000","last_peered":"2023-05-16T07:17:19.525240+0000","last_clean":"2023-05-16T07:17:19.525240+0000","last_became_active":"2023-05-16T07:17:19.524369+0000","last_became_peered":"2023-05-16T07:17:19.524369+0000","last_unstale":"2023-05-16T07:17:19.525240+0000","last_undegraded":"2023-05-16T07:17:19.525240+0000","last_fullsized":"2023-05-16T07:17:19.525240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.505034+0000","last_change":"2023-05-16T07:17:19.150196+0000","last_active":"2023-05-16T07:17:19.505034+0000","last_peered":"2023-05-16T07:17:19.505034+0000","last_clean":"2023-05-16T07:17:19.505034+0000","last_became_active":"2023-05-16T07:17:19.149926+0000","last_became_peered":"2023-05-16T07:17:19.149926+0000","last_unstale":"2023-05-16T07:17:19.505034+0000","last_undegraded":"2023-05-16T07:17:19.505034+0000","last_fullsized":"2023-05-16T07:17:19.505034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.126574+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:17:19.126574+0000","last_peered":"2023-05-16T07:17:19.126574+0000","last_clean":"2023-05-16T07:17:19.126574+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:17:19.126574+0000","last_undegraded":"2023-05-16T07:17:19.126574+0000","last_fullsized":"2023-05-16T07:17:19.126574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510225+0000","last_change":"2023-05-16T07:17:19.510225+0000","last_active":"2023-05-16T07:17:19.510225+0000","last_peered":"2023-05-16T07:17:19.510225+0000","last_clean":"2023-05-16T07:17:19.510225+0000","last_became_active":"2023-05-16T07:17:19.509046+0000","last_became_peered":"2023-05-16T07:17:19.509046+0000","last_unstale":"2023-05-16T07:17:19.510225+0000","last_undegraded":"2023-05-16T07:17:19.510225+0000","last_fullsized":"2023-05-16T07:17:19.510225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145795+0000","last_change":"2023-05-16T07:17:19.140774+0000","last_active":"2023-05-16T07:17:19.145795+0000","last_peered":"2023-05-16T07:17:19.145795+0000","last_clean":"2023-05-16T07:17:19.145795+0000","last_became_active":"2023-05-16T07:17:19.140517+0000","last_became_peered":"2023-05-16T07:17:19.140517+0000","last_unstale":"2023-05-16T07:17:19.145795+0000","last_undegraded":"2023-05-16T07:17:19.145795+0000","last_fullsized":"2023-05-16T07:17:19.145795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.136728+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:17:19.136728+0000","last_peered":"2023-05-16T07:17:19.136728+0000","last_clean":"2023-05-16T07:17:19.136728+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:17:19.136728+0000","last_undegraded":"2023-05-16T07:17:19.136728+0000","last_fullsized":"2023-05-16T07:17:19.136728+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518065+0000","last_change":"2023-05-16T07:17:19.518065+0000","last_active":"2023-05-16T07:17:19.518065+0000","last_peered":"2023-05-16T07:17:19.518065+0000","last_clean":"2023-05-16T07:17:19.518065+0000","last_became_active":"2023-05-16T07:17:19.517796+0000","last_became_peered":"2023-05-16T07:17:19.517796+0000","last_unstale":"2023-05-16T07:17:19.518065+0000","last_undegraded":"2023-05-16T07:17:19.518065+0000","last_fullsized":"2023-05-16T07:17:19.518065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.150814+0000","last_change":"2023-05-16T07:17:19.149475+0000","last_active":"2023-05-16T07:17:19.150814+0000","last_peered":"2023-05-16T07:17:19.150814+0000","last_clean":"2023-05-16T07:17:19.150814+0000","last_became_active":"2023-05-16T07:17:19.148603+0000","last_became_peered":"2023-05-16T07:17:19.148603+0000","last_unstale":"2023-05-16T07:17:19.150814+0000","last_undegraded":"2023-05-16T07:17:19.150814+0000","last_fullsized":"2023-05-16T07:17:19.150814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.135945+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:17:19.135945+0000","last_peered":"2023-05-16T07:17:19.135945+0000","last_clean":"2023-05-16T07:17:19.135945+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:17:19.135945+0000","last_undegraded":"2023-05-16T07:17:19.135945+0000","last_fullsized":"2023-05-16T07:17:19.135945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515057+0000","last_change":"2023-05-16T07:17:19.515057+0000","last_active":"2023-05-16T07:17:19.515057+0000","last_peered":"2023-05-16T07:17:19.515057+0000","last_clean":"2023-05-16T07:17:19.515057+0000","last_became_active":"2023-05-16T07:17:19.514754+0000","last_became_peered":"2023-05-16T07:17:19.514754+0000","last_unstale":"2023-05-16T07:17:19.515057+0000","last_undegraded":"2023-05-16T07:17:19.515057+0000","last_fullsized":"2023-05-16T07:17:19.515057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147345+0000","last_change":"2023-05-16T07:17:19.143563+0000","last_active":"2023-05-16T07:17:19.147345+0000","last_peered":"2023-05-16T07:17:19.147345+0000","last_clean":"2023-05-16T07:17:19.147345+0000","last_became_active":"2023-05-16T07:17:19.143253+0000","last_became_peered":"2023-05-16T07:17:19.143253+0000","last_unstale":"2023-05-16T07:17:19.147345+0000","last_undegraded":"2023-05-16T07:17:19.147345+0000","last_fullsized":"2023-05-16T07:17:19.147345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.141353+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:17:19.141353+0000","last_peered":"2023-05-16T07:17:19.141353+0000","last_clean":"2023-05-16T07:17:19.141353+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:17:19.141353+0000","last_undegraded":"2023-05-16T07:17:19.141353+0000","last_fullsized":"2023-05-16T07:17:19.141353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.139213+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:17:19.139213+0000","last_peered":"2023-05-16T07:17:19.139213+0000","last_clean":"2023-05-16T07:17:19.139213+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:17:19.139213+0000","last_undegraded":"2023-05-16T07:17:19.139213+0000","last_fullsized":"2023-05-16T07:17:19.139213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515264+0000","last_change":"2023-05-16T07:17:19.515264+0000","last_active":"2023-05-16T07:17:19.515264+0000","last_peered":"2023-05-16T07:17:19.515264+0000","last_clean":"2023-05-16T07:17:19.515264+0000","last_became_active":"2023-05-16T07:17:19.515018+0000","last_became_peered":"2023-05-16T07:17:19.515018+0000","last_unstale":"2023-05-16T07:17:19.515264+0000","last_undegraded":"2023-05-16T07:17:19.515264+0000","last_fullsized":"2023-05-16T07:17:19.515264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.502067+0000","last_change":"2023-05-16T07:17:19.139087+0000","last_active":"2023-05-16T07:17:19.502067+0000","last_peered":"2023-05-16T07:17:19.502067+0000","last_clean":"2023-05-16T07:17:19.502067+0000","last_became_active":"2023-05-16T07:17:19.138776+0000","last_became_peered":"2023-05-16T07:17:19.138776+0000","last_unstale":"2023-05-16T07:17:19.502067+0000","last_undegraded":"2023-05-16T07:17:19.502067+0000","last_fullsized":"2023-05-16T07:17:19.502067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.139390+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:17:19.139390+0000","last_peered":"2023-05-16T07:17:19.139390+0000","last_clean":"2023-05-16T07:17:19.139390+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:17:19.139390+0000","last_undegraded":"2023-05-16T07:17:19.139390+0000","last_fullsized":"2023-05-16T07:17:19.139390+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510943+0000","last_change":"2023-05-16T07:17:19.510943+0000","last_active":"2023-05-16T07:17:19.510943+0000","last_peered":"2023-05-16T07:17:19.510943+0000","last_clean":"2023-05-16T07:17:19.510943+0000","last_became_active":"2023-05-16T07:17:19.510681+0000","last_became_peered":"2023-05-16T07:17:19.510681+0000","last_unstale":"2023-05-16T07:17:19.510943+0000","last_undegraded":"2023-05-16T07:17:19.510943+0000","last_fullsized":"2023-05-16T07:17:19.510943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.522415+0000","last_change":"2023-05-16T07:17:19.150924+0000","last_active":"2023-05-16T07:17:19.522415+0000","last_peered":"2023-05-16T07:17:19.522415+0000","last_clean":"2023-05-16T07:17:19.522415+0000","last_became_active":"2023-05-16T07:17:19.150672+0000","last_became_peered":"2023-05-16T07:17:19.150672+0000","last_unstale":"2023-05-16T07:17:19.522415+0000","last_undegraded":"2023-05-16T07:17:19.522415+0000","last_fullsized":"2023-05-16T07:17:19.522415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.140288+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:17:19.140288+0000","last_peered":"2023-05-16T07:17:19.140288+0000","last_clean":"2023-05-16T07:17:19.140288+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:17:19.140288+0000","last_undegraded":"2023-05-16T07:17:19.140288+0000","last_fullsized":"2023-05-16T07:17:19.140288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.520237+0000","last_change":"2023-05-16T07:17:19.520237+0000","last_active":"2023-05-16T07:17:19.520237+0000","last_peered":"2023-05-16T07:17:19.520237+0000","last_clean":"2023-05-16T07:17:19.520237+0000","last_became_active":"2023-05-16T07:17:19.517683+0000","last_became_peered":"2023-05-16T07:17:19.517683+0000","last_unstale":"2023-05-16T07:17:19.520237+0000","last_undegraded":"2023-05-16T07:17:19.520237+0000","last_fullsized":"2023-05-16T07:17:19.520237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151494+0000","last_change":"2023-05-16T07:17:19.150106+0000","last_active":"2023-05-16T07:17:19.151494+0000","last_peered":"2023-05-16T07:17:19.151494+0000","last_clean":"2023-05-16T07:17:19.151494+0000","last_became_active":"2023-05-16T07:17:19.149841+0000","last_became_peered":"2023-05-16T07:17:19.149841+0000","last_unstale":"2023-05-16T07:17:19.151494+0000","last_undegraded":"2023-05-16T07:17:19.151494+0000","last_fullsized":"2023-05-16T07:17:19.151494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.126669+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:17:19.126669+0000","last_peered":"2023-05-16T07:17:19.126669+0000","last_clean":"2023-05-16T07:17:19.126669+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:17:19.126669+0000","last_undegraded":"2023-05-16T07:17:19.126669+0000","last_fullsized":"2023-05-16T07:17:19.126669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517317+0000","last_change":"2023-05-16T07:17:19.517317+0000","last_active":"2023-05-16T07:17:19.517317+0000","last_peered":"2023-05-16T07:17:19.517317+0000","last_clean":"2023-05-16T07:17:19.517317+0000","last_became_active":"2023-05-16T07:17:19.516610+0000","last_became_peered":"2023-05-16T07:17:19.516610+0000","last_unstale":"2023-05-16T07:17:19.517317+0000","last_undegraded":"2023-05-16T07:17:19.517317+0000","last_fullsized":"2023-05-16T07:17:19.517317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147203+0000","last_change":"2023-05-16T07:17:19.142570+0000","last_active":"2023-05-16T07:17:19.147203+0000","last_peered":"2023-05-16T07:17:19.147203+0000","last_clean":"2023-05-16T07:17:19.147203+0000","last_became_active":"2023-05-16T07:17:19.142285+0000","last_became_peered":"2023-05-16T07:17:19.142285+0000","last_unstale":"2023-05-16T07:17:19.147203+0000","last_undegraded":"2023-05-16T07:17:19.147203+0000","last_fullsized":"2023-05-16T07:17:19.147203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.129769+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:17:19.129769+0000","last_peered":"2023-05-16T07:17:19.129769+0000","last_clean":"2023-05-16T07:17:19.129769+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:17:19.129769+0000","last_undegraded":"2023-05-16T07:17:19.129769+0000","last_fullsized":"2023-05-16T07:17:19.129769+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17068,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":16128,"kb_avail":94354772,"statfs":{"total":96636764160,"available":96619286528,"internally_reserved":0,"allocated":954368,"data_stored":479471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16515072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.439,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.751,"5min":0.81399999999999995,"15min":0.84199999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.47399999999999998,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":0.873},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.752,"5min":0.876,"15min":0.876},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.48599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63}]},{"osd":4,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66600000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.752,"15min":0.81999999999999995},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68500000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.40899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.999},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.95099999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16328,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":15424,"kb_avail":94355512,"statfs":{"total":96636764160,"available":96620044288,"internally_reserved":0,"allocated":917504,"data_stored":463429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15794176},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.26400000000000001},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.47799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77700000000000002,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.47399999999999998,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.37},{"interface":"front","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.38300000000000001,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.67900000000000005}]},{"osd":2,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.51100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.50900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.86299999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.497},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.32400000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61699999999999999,"5min":0.749,"15min":0.89400000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.628,"5min":4.048,"15min":4.048},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61499999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.311},"max":{"1min":0.63700000000000001,"5min":0.74399999999999999,"15min":0.92300000000000004},"last":0.749}]},{"osd":7,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67400000000000004,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.69299999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16456,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94355384,"statfs":{"total":96636764160,"available":96619913216,"internally_reserved":0,"allocated":917504,"data_stored":463429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.53300000000000003,"5min":0.53400000000000003,"15min":0.69899999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.34799999999999998}]},{"osd":2,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.375,"15min":0.376},"min":{"1min":0.27300000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":0.59699999999999998,"15min":0.65100000000000002},"last":0.313},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.40100000000000002},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.33600000000000002}]},{"osd":3,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42499999999999999},"min":{"1min":0.31900000000000001,"5min":0.29399999999999998,"15min":0.26300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64000000000000001},"last":0.251}]},{"osd":4,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.622,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.623,"15min":0.622},"min":{"1min":0.435,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.51300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85299999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.47399999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.087},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.41699999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":1.204},"last":0.53400000000000003}]},{"osd":7,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.53100000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11616,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":10688,"kb_avail":94360224,"statfs":{"total":96636764160,"available":96624869376,"internally_reserved":0,"allocated":942080,"data_stored":478455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10944512},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34899999999999998,"15min":0.35699999999999998},"min":{"1min":0.246,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.60199999999999998},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.224},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.316}]},{"osd":2,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.36399999999999999,"15min":0.373},"min":{"1min":0.28899999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.40200000000000002},"min":{"1min":0.32100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.35599999999999998}]},{"osd":3,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.30299999999999999,"5min":0.253,"15min":0.24199999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41599999999999998},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.64400000000000002},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.621,"15min":0.63500000000000001},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.117},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.55000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.2170000000000001},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45900000000000002,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.86699999999999999,"5min":0.90000000000000002,"15min":1.157},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.97199999999999998,"15min":1.224},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.177},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.93700000000000006,"15min":1.196},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.56699999999999995,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.90600000000000003,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.442}]}]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17448,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":16512,"kb_avail":94354392,"statfs":{"total":96636764160,"available":96618897408,"internally_reserved":0,"allocated":950272,"data_stored":479079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16908288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36199999999999999},"min":{"1min":0.23899999999999999,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.54100000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.27400000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.28799999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.40899999999999997},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.23499999999999999},"max":{"1min":0.54000000000000004,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.435},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.434},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.626,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.377}]},{"osd":3,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.248,"15min":0.22500000000000001},"max":{"1min":0.504,"5min":0.64000000000000001,"15min":0.64700000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44700000000000001,"15min":0.438},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64900000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41699999999999998}]},{"osd":4,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.41999999999999998,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":1.274},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.628},"min":{"1min":0.47099999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80400000000000005,"5min":0.92900000000000005,"15min":0.94899999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.89000000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.37},"max":{"1min":0.872,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67900000000000005,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80800000000000005,"5min":1.47,"15min":1.47},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20372,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":19456,"kb_avail":94351468,"statfs":{"total":96636764160,"available":96615903232,"internally_reserved":0,"allocated":929792,"data_stored":470692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19922944},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.437},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.29799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.40899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42199999999999999,"15min":0.41499999999999998},"min":{"1min":0.33100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.33800000000000002,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.626,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.70199999999999996}]},{"osd":5,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77500000000000002,"5min":0.93100000000000005,"15min":1.0489999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83799999999999997,"5min":0.93799999999999994,"15min":1.0409999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.504,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.97399999999999998,"15min":1.0780000000000001},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":1.0049999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.44,"15min":0.40100000000000002},"max":{"1min":0.75800000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":1.02},"last":0.74299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16680,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":15744,"kb_avail":94355160,"statfs":{"total":96636764160,"available":96619683840,"internally_reserved":0,"allocated":950272,"data_stored":479099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16121856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.83199999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.81200000000000006},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.33900000000000002},"max":{"1min":0.73699999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.505},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.999},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.84599999999999997,"15min":0.97999999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.93100000000000005},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.32200000000000001,"5min":0.314,"15min":0.308},"max":{"1min":0.69899999999999995,"5min":0.79500000000000004,"15min":1.0640000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52300000000000002,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.79200000000000004,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":1.2689999999999999},"last":0.43099999999999999}]},{"osd":7,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":1.48,"15min":1.48},"last":0.375},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":0.82599999999999996,"15min":1.046},"last":0.46300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575028,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19784,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":18880,"kb_avail":94352056,"statfs":{"total":96636764160,"available":96616505344,"internally_reserved":0,"allocated":917504,"data_stored":463429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19333120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60899999999999999,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.69599999999999995,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59199999999999997,"15min":0.56299999999999994},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.314},"max":{"1min":0.747,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.44,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71999999999999997,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.38700000000000001}]},{"osd":2,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":2.278,"15min":2.278},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.62,"15min":0.58699999999999997},"min":{"1min":0.48799999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77500000000000002,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.51100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75800000000000001,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.435},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.51800000000000002,"5min":0.435,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58199999999999996,"5min":1.333,"15min":1.333},"last":0.499},{"interface":"front","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.309},"max":{"1min":0.78400000000000003,"5min":2.081,"15min":2.081},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.69499999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":1.024,"15min":1.024},"last":0.53600000000000003}]},{"osd":7,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81699999999999995,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.76700000000000002,"5min":0.86599999999999999,"15min":0.878},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:17:22.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:17:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.618+0000 7f5d73b8d700 1 Processor -- start 2023-05-16T07:17:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.618+0000 7f5d73b8d700 1 -- start start 2023-05-16T07:17:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d73b8d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d6c154170 0x7f5d6c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d73b8d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5d6c154550 0x7f5d6c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d73b8d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d73b8d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c0c0660 con 0x7f5d6c154170 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d73b8d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c06a020 con 0x7f5d6c154550 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d73b8d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0bfd30 con 0x7f5d6c0696a0 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d71929700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d6c154170 0x7f5d6c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d71929700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d6c154170 0x7f5d6c0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45386/0 (socket says 172.21.15.92:45386) 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.619+0000 7f5d71929700 1 -- 172.21.15.92:0/1459738442 learned_addr learned my addr 172.21.15.92:0/1459738442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71128700 1 --2- 172.21.15.92:0/1459738442 >> v2:172.21.15.158:3300/0 conn(0x7f5d6c154550 0x7f5d6c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71929700 1 -- 172.21.15.92:0/1459738442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 msgr2=0x7f5d6c069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d7212a700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71929700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71929700 1 -- 172.21.15.92:0/1459738442 >> v2:172.21.15.158:3300/0 conn(0x7f5d6c154550 msgr2=0x7f5d6c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71929700 1 --2- 172.21.15.92:0/1459738442 >> v2:172.21.15.158:3300/0 conn(0x7f5d6c154550 0x7f5d6c0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71929700 1 -- 172.21.15.92:0/1459738442 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c0c0010 con 0x7f5d6c154170 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71128700 1 --2- 172.21.15.92:0/1459738442 >> v2:172.21.15.158:3300/0 conn(0x7f5d6c154550 0x7f5d6c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d7212a700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d71929700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c0c5c70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f5d6c0844e0 tx=0x7f5d5c009840).ready entity=mon.0 client_cookie=bc28deffcdf04432 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d70927700 1 -- 172.21.15.92:0/1459738442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00e070 con 0x7f5d6c154170 2023-05-16T07:17:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d70927700 1 -- 172.21.15.92:0/1459738442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5c00ba50 con 0x7f5d6c154170 2023-05-16T07:17:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d70927700 1 -- 172.21.15.92:0/1459738442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00bdd0 con 0x7f5d6c154170 2023-05-16T07:17:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 -- 172.21.15.92:0/1459738442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 msgr2=0x7f5d6c0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c0c5c70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f5d6c0844e0 tx=0x7f5d5c009840).stop 2023-05-16T07:17:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 -- 172.21.15.92:0/1459738442 shutdown_connections 2023-05-16T07:17:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/1459738442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c0c5c70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/1459738442 >> v2:172.21.15.158:3300/0 conn(0x7f5d6c154550 0x7f5d6c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 -- 172.21.15.92:0/1459738442 >> 172.21.15.92:0/1459738442 conn(0x7f5d6c0cb700 msgr2=0x7f5d6c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 -- 172.21.15.92:0/1459738442 shutdown_connections 2023-05-16T07:17:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.620+0000 7f5d73b8d700 1 -- 172.21.15.92:0/1459738442 wait complete. 2023-05-16T07:17:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.621+0000 7f5d73b8d700 1 Processor -- start 2023-05-16T07:17:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 -- start start 2023-05-16T07:17:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5d6c154550 0x7f5d6c1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c14d210 con 0x7f5d6c154170 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c14c390 con 0x7f5d6c154550 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d73b8d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c145dc0 con 0x7f5d6c0696a0 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d71128700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d71128700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c1453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45394/0 (socket says 172.21.15.92:45394) 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d71128700 1 -- 172.21.15.92:0/4063212120 learned_addr learned my addr 172.21.15.92:0/4063212120 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.622+0000 7f5d71929700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d7212a700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5d6c154550 0x7f5d6c1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d71128700 1 -- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 msgr2=0x7f5d6c14bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d71128700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c14bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d71128700 1 -- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5d6c154550 msgr2=0x7f5d6c1458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d71128700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5d6c154550 0x7f5d6c1458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d71128700 1 -- 172.21.15.92:0/4063212120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c009ae0 con 0x7f5d6c154170 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d7212a700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5d6c154550 0x7f5d6c1458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d71128700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c1453e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f5d6000c8b0 tx=0x7f5d60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010580 con 0x7f5d6c154170 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d600106e0 con 0x7f5d6c154170 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c14c600 con 0x7f5d6c154170 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010a60 con 0x7f5d6c154170 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.623+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c14cbc0 con 0x7f5d6c154170 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.624+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c0666c0 con 0x7f5d6c154170 2023-05-16T07:17:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.624+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5d6001b460 con 0x7f5d6c154170 2023-05-16T07:17:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.625+0000 7f5d69ffb700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5d5805edc0 0x7f5d58061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.625+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f5d60079e40 con 0x7f5d6c154170 2023-05-16T07:17:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.625+0000 7f5d71929700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5d5805edc0 0x7f5d58061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.625+0000 7f5d71929700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5d5805edc0 0x7f5d58061280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5d5c009930 tx=0x7f5d5c009840).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.629+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5d6004b130 con 0x7f5d6c154170 2023-05-16T07:17:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.788+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d6c14d350 con 0x7f5d5805edc0 2023-05-16T07:17:22.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.799+0000 7f5d69ffb700 1 -- 172.21.15.92:0/4063212120 <== mgr.5330 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361226 (secure 0 0 0) 0x7f5d5c00bc50 con 0x7f5d5805edc0 2023-05-16T07:17:22.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5d5805edc0 msgr2=0x7f5d58061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5d5805edc0 0x7f5d58061280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5d5c009930 tx=0x7f5d5c009840).stop 2023-05-16T07:17:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 msgr2=0x7f5d6c1453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c1453e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f5d6000c8b0 tx=0x7f5d60008040).stop 2023-05-16T07:17:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 shutdown_connections 2023-05-16T07:17:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5d5805edc0 0x7f5d58061280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5d6c0696a0 0x7f5d6c14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5d6c154550 0x7f5d6c1458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 --2- 172.21.15.92:0/4063212120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d6c154170 0x7f5d6c1453e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 >> 172.21.15.92:0/4063212120 conn(0x7f5d6c0cb700 msgr2=0x7f5d6c0c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 shutdown_connections 2023-05-16T07:17:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.802+0000 7f5d73b8d700 1 -- 172.21.15.92:0/4063212120 wait complete. 2023-05-16T07:17:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:17:22.817 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:17:20.923221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":135752,"kb_used_data":7304,"kb_used_omap":0,"kb_used_meta":128384,"kb_avail":754838968,"statfs":{"total":773094113280,"available":772955103232,"internally_reserved":0,"allocated":7479296,"data_stored":3777083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":131465216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28002,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-78,"num_write_kb":-63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.586570"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515982+0000","last_change":"2023-05-16T07:17:19.515982+0000","last_active":"2023-05-16T07:17:19.515982+0000","last_peered":"2023-05-16T07:17:19.515982+0000","last_clean":"2023-05-16T07:17:19.515982+0000","last_became_active":"2023-05-16T07:17:19.513374+0000","last_became_peered":"2023-05-16T07:17:19.513374+0000","last_unstale":"2023-05-16T07:17:19.515982+0000","last_undegraded":"2023-05-16T07:17:19.515982+0000","last_fullsized":"2023-05-16T07:17:19.515982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"78'3","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516249+0000","last_change":"2023-05-16T07:17:19.140747+0000","last_active":"2023-05-16T07:17:19.516249+0000","last_peered":"2023-05-16T07:17:19.516249+0000","last_clean":"2023-05-16T07:17:19.516249+0000","last_became_active":"2023-05-16T07:17:19.140467+0000","last_became_peered":"2023-05-16T07:17:19.140467+0000","last_unstale":"2023-05-16T07:17:19.516249+0000","last_undegraded":"2023-05-16T07:17:19.516249+0000","last_fullsized":"2023-05-16T07:17:19.516249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518056+0000","last_change":"2023-05-16T07:17:19.518056+0000","last_active":"2023-05-16T07:17:19.518056+0000","last_peered":"2023-05-16T07:17:19.518056+0000","last_clean":"2023-05-16T07:17:19.518056+0000","last_became_active":"2023-05-16T07:17:19.517784+0000","last_became_peered":"2023-05-16T07:17:19.517784+0000","last_unstale":"2023-05-16T07:17:19.518056+0000","last_undegraded":"2023-05-16T07:17:19.518056+0000","last_fullsized":"2023-05-16T07:17:19.518056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146025+0000","last_change":"2023-05-16T07:17:19.139074+0000","last_active":"2023-05-16T07:17:19.146025+0000","last_peered":"2023-05-16T07:17:19.146025+0000","last_clean":"2023-05-16T07:17:19.146025+0000","last_became_active":"2023-05-16T07:17:19.138616+0000","last_became_peered":"2023-05-16T07:17:19.138616+0000","last_unstale":"2023-05-16T07:17:19.146025+0000","last_undegraded":"2023-05-16T07:17:19.146025+0000","last_fullsized":"2023-05-16T07:17:19.146025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513813+0000","last_change":"2023-05-16T07:17:19.513813+0000","last_active":"2023-05-16T07:17:19.513813+0000","last_peered":"2023-05-16T07:17:19.513813+0000","last_clean":"2023-05-16T07:17:19.513813+0000","last_became_active":"2023-05-16T07:17:19.513537+0000","last_became_peered":"2023-05-16T07:17:19.513537+0000","last_unstale":"2023-05-16T07:17:19.513813+0000","last_undegraded":"2023-05-16T07:17:19.513813+0000","last_fullsized":"2023-05-16T07:17:19.513813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151456+0000","last_change":"2023-05-16T07:17:19.147500+0000","last_active":"2023-05-16T07:17:19.151456+0000","last_peered":"2023-05-16T07:17:19.151456+0000","last_clean":"2023-05-16T07:17:19.151456+0000","last_became_active":"2023-05-16T07:17:19.147222+0000","last_became_peered":"2023-05-16T07:17:19.147222+0000","last_unstale":"2023-05-16T07:17:19.151456+0000","last_undegraded":"2023-05-16T07:17:19.151456+0000","last_fullsized":"2023-05-16T07:17:19.151456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518191+0000","last_change":"2023-05-16T07:17:19.518191+0000","last_active":"2023-05-16T07:17:19.518191+0000","last_peered":"2023-05-16T07:17:19.518191+0000","last_clean":"2023-05-16T07:17:19.518191+0000","last_became_active":"2023-05-16T07:17:19.517822+0000","last_became_peered":"2023-05-16T07:17:19.517822+0000","last_unstale":"2023-05-16T07:17:19.518191+0000","last_undegraded":"2023-05-16T07:17:19.518191+0000","last_fullsized":"2023-05-16T07:17:19.518191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517484+0000","last_change":"2023-05-16T07:17:19.144182+0000","last_active":"2023-05-16T07:17:19.517484+0000","last_peered":"2023-05-16T07:17:19.517484+0000","last_clean":"2023-05-16T07:17:19.517484+0000","last_became_active":"2023-05-16T07:17:19.143901+0000","last_became_peered":"2023-05-16T07:17:19.143901+0000","last_unstale":"2023-05-16T07:17:19.517484+0000","last_undegraded":"2023-05-16T07:17:19.517484+0000","last_fullsized":"2023-05-16T07:17:19.517484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514124+0000","last_change":"2023-05-16T07:17:19.514124+0000","last_active":"2023-05-16T07:17:19.514124+0000","last_peered":"2023-05-16T07:17:19.514124+0000","last_clean":"2023-05-16T07:17:19.514124+0000","last_became_active":"2023-05-16T07:17:19.513819+0000","last_became_peered":"2023-05-16T07:17:19.513819+0000","last_unstale":"2023-05-16T07:17:19.514124+0000","last_undegraded":"2023-05-16T07:17:19.514124+0000","last_fullsized":"2023-05-16T07:17:19.514124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.498672+0000","last_change":"2023-05-16T07:17:19.147823+0000","last_active":"2023-05-16T07:17:19.498672+0000","last_peered":"2023-05-16T07:17:19.498672+0000","last_clean":"2023-05-16T07:17:19.498672+0000","last_became_active":"2023-05-16T07:17:19.147560+0000","last_became_peered":"2023-05-16T07:17:19.147560+0000","last_unstale":"2023-05-16T07:17:19.498672+0000","last_undegraded":"2023-05-16T07:17:19.498672+0000","last_fullsized":"2023-05-16T07:17:19.498672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.509160+0000","last_change":"2023-05-16T07:17:19.509160+0000","last_active":"2023-05-16T07:17:19.509160+0000","last_peered":"2023-05-16T07:17:19.509160+0000","last_clean":"2023-05-16T07:17:19.509160+0000","last_became_active":"2023-05-16T07:17:19.508876+0000","last_became_peered":"2023-05-16T07:17:19.508876+0000","last_unstale":"2023-05-16T07:17:19.509160+0000","last_undegraded":"2023-05-16T07:17:19.509160+0000","last_fullsized":"2023-05-16T07:17:19.509160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.502295+0000","last_change":"2023-05-16T07:17:19.142468+0000","last_active":"2023-05-16T07:17:19.502295+0000","last_peered":"2023-05-16T07:17:19.502295+0000","last_clean":"2023-05-16T07:17:19.502295+0000","last_became_active":"2023-05-16T07:17:19.142190+0000","last_became_peered":"2023-05-16T07:17:19.142190+0000","last_unstale":"2023-05-16T07:17:19.502295+0000","last_undegraded":"2023-05-16T07:17:19.502295+0000","last_fullsized":"2023-05-16T07:17:19.502295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510578+0000","last_change":"2023-05-16T07:17:19.510578+0000","last_active":"2023-05-16T07:17:19.510578+0000","last_peered":"2023-05-16T07:17:19.510578+0000","last_clean":"2023-05-16T07:17:19.510578+0000","last_became_active":"2023-05-16T07:17:19.510331+0000","last_became_peered":"2023-05-16T07:17:19.510331+0000","last_unstale":"2023-05-16T07:17:19.510578+0000","last_undegraded":"2023-05-16T07:17:19.510578+0000","last_fullsized":"2023-05-16T07:17:19.510578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145763+0000","last_change":"2023-05-16T07:17:19.141920+0000","last_active":"2023-05-16T07:17:19.145763+0000","last_peered":"2023-05-16T07:17:19.145763+0000","last_clean":"2023-05-16T07:17:19.145763+0000","last_became_active":"2023-05-16T07:17:19.140680+0000","last_became_peered":"2023-05-16T07:17:19.140680+0000","last_unstale":"2023-05-16T07:17:19.145763+0000","last_undegraded":"2023-05-16T07:17:19.145763+0000","last_fullsized":"2023-05-16T07:17:19.145763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513257+0000","last_change":"2023-05-16T07:17:19.513257+0000","last_active":"2023-05-16T07:17:19.513257+0000","last_peered":"2023-05-16T07:17:19.513257+0000","last_clean":"2023-05-16T07:17:19.513257+0000","last_became_active":"2023-05-16T07:17:19.512948+0000","last_became_peered":"2023-05-16T07:17:19.512948+0000","last_unstale":"2023-05-16T07:17:19.513257+0000","last_undegraded":"2023-05-16T07:17:19.513257+0000","last_fullsized":"2023-05-16T07:17:19.513257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.140344+0000","last_change":"2023-05-16T07:17:19.136135+0000","last_active":"2023-05-16T07:17:19.140344+0000","last_peered":"2023-05-16T07:17:19.140344+0000","last_clean":"2023-05-16T07:17:19.140344+0000","last_became_active":"2023-05-16T07:17:19.135867+0000","last_became_peered":"2023-05-16T07:17:19.135867+0000","last_unstale":"2023-05-16T07:17:19.140344+0000","last_undegraded":"2023-05-16T07:17:19.140344+0000","last_fullsized":"2023-05-16T07:17:19.140344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514137+0000","last_change":"2023-05-16T07:17:19.514137+0000","last_active":"2023-05-16T07:17:19.514137+0000","last_peered":"2023-05-16T07:17:19.514137+0000","last_clean":"2023-05-16T07:17:19.514137+0000","last_became_active":"2023-05-16T07:17:19.513839+0000","last_became_peered":"2023-05-16T07:17:19.513839+0000","last_unstale":"2023-05-16T07:17:19.514137+0000","last_undegraded":"2023-05-16T07:17:19.514137+0000","last_fullsized":"2023-05-16T07:17:19.514137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145554+0000","last_change":"2023-05-16T07:17:19.141077+0000","last_active":"2023-05-16T07:17:19.145554+0000","last_peered":"2023-05-16T07:17:19.145554+0000","last_clean":"2023-05-16T07:17:19.145554+0000","last_became_active":"2023-05-16T07:17:19.140664+0000","last_became_peered":"2023-05-16T07:17:19.140664+0000","last_unstale":"2023-05-16T07:17:19.145554+0000","last_undegraded":"2023-05-16T07:17:19.145554+0000","last_fullsized":"2023-05-16T07:17:19.145554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.522880+0000","last_change":"2023-05-16T07:17:19.522880+0000","last_active":"2023-05-16T07:17:19.522880+0000","last_peered":"2023-05-16T07:17:19.522880+0000","last_clean":"2023-05-16T07:17:19.522880+0000","last_became_active":"2023-05-16T07:17:19.520002+0000","last_became_peered":"2023-05-16T07:17:19.520002+0000","last_unstale":"2023-05-16T07:17:19.522880+0000","last_undegraded":"2023-05-16T07:17:19.522880+0000","last_fullsized":"2023-05-16T07:17:19.522880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147750+0000","last_change":"2023-05-16T07:17:19.143468+0000","last_active":"2023-05-16T07:17:19.147750+0000","last_peered":"2023-05-16T07:17:19.147750+0000","last_clean":"2023-05-16T07:17:19.147750+0000","last_became_active":"2023-05-16T07:17:19.143182+0000","last_became_peered":"2023-05-16T07:17:19.143182+0000","last_unstale":"2023-05-16T07:17:19.147750+0000","last_undegraded":"2023-05-16T07:17:19.147750+0000","last_fullsized":"2023-05-16T07:17:19.147750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512442+0000","last_change":"2023-05-16T07:17:19.512442+0000","last_active":"2023-05-16T07:17:19.512442+0000","last_peered":"2023-05-16T07:17:19.512442+0000","last_clean":"2023-05-16T07:17:19.512442+0000","last_became_active":"2023-05-16T07:17:19.512181+0000","last_became_peered":"2023-05-16T07:17:19.512181+0000","last_unstale":"2023-05-16T07:17:19.512442+0000","last_undegraded":"2023-05-16T07:17:19.512442+0000","last_fullsized":"2023-05-16T07:17:19.512442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147778+0000","last_change":"2023-05-16T07:17:19.140034+0000","last_active":"2023-05-16T07:17:19.147778+0000","last_peered":"2023-05-16T07:17:19.147778+0000","last_clean":"2023-05-16T07:17:19.147778+0000","last_became_active":"2023-05-16T07:17:19.139466+0000","last_became_peered":"2023-05-16T07:17:19.139466+0000","last_unstale":"2023-05-16T07:17:19.147778+0000","last_undegraded":"2023-05-16T07:17:19.147778+0000","last_fullsized":"2023-05-16T07:17:19.147778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.525033+0000","last_change":"2023-05-16T07:17:19.525033+0000","last_active":"2023-05-16T07:17:19.525033+0000","last_peered":"2023-05-16T07:17:19.525033+0000","last_clean":"2023-05-16T07:17:19.525033+0000","last_became_active":"2023-05-16T07:17:19.524755+0000","last_became_peered":"2023-05-16T07:17:19.524755+0000","last_unstale":"2023-05-16T07:17:19.525033+0000","last_undegraded":"2023-05-16T07:17:19.525033+0000","last_fullsized":"2023-05-16T07:17:19.525033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"78'4","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519246+0000","last_change":"2023-05-16T07:17:19.146276+0000","last_active":"2023-05-16T07:17:19.519246+0000","last_peered":"2023-05-16T07:17:19.519246+0000","last_clean":"2023-05-16T07:17:19.519246+0000","last_became_active":"2023-05-16T07:17:19.145276+0000","last_became_peered":"2023-05-16T07:17:19.145276+0000","last_unstale":"2023-05-16T07:17:19.519246+0000","last_undegraded":"2023-05-16T07:17:19.519246+0000","last_fullsized":"2023-05-16T07:17:19.519246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506703+0000","last_change":"2023-05-16T07:17:19.506703+0000","last_active":"2023-05-16T07:17:19.506703+0000","last_peered":"2023-05-16T07:17:19.506703+0000","last_clean":"2023-05-16T07:17:19.506703+0000","last_became_active":"2023-05-16T07:17:19.505520+0000","last_became_peered":"2023-05-16T07:17:19.505520+0000","last_unstale":"2023-05-16T07:17:19.506703+0000","last_undegraded":"2023-05-16T07:17:19.506703+0000","last_fullsized":"2023-05-16T07:17:19.506703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.500203+0000","last_change":"2023-05-16T07:17:19.143837+0000","last_active":"2023-05-16T07:17:19.500203+0000","last_peered":"2023-05-16T07:17:19.500203+0000","last_clean":"2023-05-16T07:17:19.500203+0000","last_became_active":"2023-05-16T07:17:19.142940+0000","last_became_peered":"2023-05-16T07:17:19.142940+0000","last_unstale":"2023-05-16T07:17:19.500203+0000","last_undegraded":"2023-05-16T07:17:19.500203+0000","last_fullsized":"2023-05-16T07:17:19.500203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512595+0000","last_change":"2023-05-16T07:17:19.512595+0000","last_active":"2023-05-16T07:17:19.512595+0000","last_peered":"2023-05-16T07:17:19.512595+0000","last_clean":"2023-05-16T07:17:19.512595+0000","last_became_active":"2023-05-16T07:17:19.512303+0000","last_became_peered":"2023-05-16T07:17:19.512303+0000","last_unstale":"2023-05-16T07:17:19.512595+0000","last_undegraded":"2023-05-16T07:17:19.512595+0000","last_fullsized":"2023-05-16T07:17:19.512595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145602+0000","last_change":"2023-05-16T07:17:19.141413+0000","last_active":"2023-05-16T07:17:19.145602+0000","last_peered":"2023-05-16T07:17:19.145602+0000","last_clean":"2023-05-16T07:17:19.145602+0000","last_became_active":"2023-05-16T07:17:19.141125+0000","last_became_peered":"2023-05-16T07:17:19.141125+0000","last_unstale":"2023-05-16T07:17:19.145602+0000","last_undegraded":"2023-05-16T07:17:19.145602+0000","last_fullsized":"2023-05-16T07:17:19.145602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518530+0000","last_change":"2023-05-16T07:17:19.518530+0000","last_active":"2023-05-16T07:17:19.518530+0000","last_peered":"2023-05-16T07:17:19.518530+0000","last_clean":"2023-05-16T07:17:19.518530+0000","last_became_active":"2023-05-16T07:17:19.518238+0000","last_became_peered":"2023-05-16T07:17:19.518238+0000","last_unstale":"2023-05-16T07:17:19.518530+0000","last_undegraded":"2023-05-16T07:17:19.518530+0000","last_fullsized":"2023-05-16T07:17:19.518530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.142035+0000","last_change":"2023-05-16T07:17:19.137259+0000","last_active":"2023-05-16T07:17:19.142035+0000","last_peered":"2023-05-16T07:17:19.142035+0000","last_clean":"2023-05-16T07:17:19.142035+0000","last_became_active":"2023-05-16T07:17:19.136939+0000","last_became_peered":"2023-05-16T07:17:19.136939+0000","last_unstale":"2023-05-16T07:17:19.142035+0000","last_undegraded":"2023-05-16T07:17:19.142035+0000","last_fullsized":"2023-05-16T07:17:19.142035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518926+0000","last_change":"2023-05-16T07:17:19.518926+0000","last_active":"2023-05-16T07:17:19.518926+0000","last_peered":"2023-05-16T07:17:19.518926+0000","last_clean":"2023-05-16T07:17:19.518926+0000","last_became_active":"2023-05-16T07:17:19.518655+0000","last_became_peered":"2023-05-16T07:17:19.518655+0000","last_unstale":"2023-05-16T07:17:19.518926+0000","last_undegraded":"2023-05-16T07:17:19.518926+0000","last_fullsized":"2023-05-16T07:17:19.518926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"78'4","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517390+0000","last_change":"2023-05-16T07:17:19.141753+0000","last_active":"2023-05-16T07:17:19.517390+0000","last_peered":"2023-05-16T07:17:19.517390+0000","last_clean":"2023-05-16T07:17:19.517390+0000","last_became_active":"2023-05-16T07:17:19.141451+0000","last_became_peered":"2023-05-16T07:17:19.141451+0000","last_unstale":"2023-05-16T07:17:19.517390+0000","last_undegraded":"2023-05-16T07:17:19.517390+0000","last_fullsized":"2023-05-16T07:17:19.517390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519742+0000","last_change":"2023-05-16T07:17:19.519742+0000","last_active":"2023-05-16T07:17:19.519742+0000","last_peered":"2023-05-16T07:17:19.519742+0000","last_clean":"2023-05-16T07:17:19.519742+0000","last_became_active":"2023-05-16T07:17:19.519424+0000","last_became_peered":"2023-05-16T07:17:19.519424+0000","last_unstale":"2023-05-16T07:17:19.519742+0000","last_undegraded":"2023-05-16T07:17:19.519742+0000","last_fullsized":"2023-05-16T07:17:19.519742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147183+0000","last_change":"2023-05-16T07:17:19.143901+0000","last_active":"2023-05-16T07:17:19.147183+0000","last_peered":"2023-05-16T07:17:19.147183+0000","last_clean":"2023-05-16T07:17:19.147183+0000","last_became_active":"2023-05-16T07:17:19.143630+0000","last_became_peered":"2023-05-16T07:17:19.143630+0000","last_unstale":"2023-05-16T07:17:19.147183+0000","last_undegraded":"2023-05-16T07:17:19.147183+0000","last_fullsized":"2023-05-16T07:17:19.147183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513462+0000","last_change":"2023-05-16T07:17:19.513462+0000","last_active":"2023-05-16T07:17:19.513462+0000","last_peered":"2023-05-16T07:17:19.513462+0000","last_clean":"2023-05-16T07:17:19.513462+0000","last_became_active":"2023-05-16T07:17:19.513140+0000","last_became_peered":"2023-05-16T07:17:19.513140+0000","last_unstale":"2023-05-16T07:17:19.513462+0000","last_undegraded":"2023-05-16T07:17:19.513462+0000","last_fullsized":"2023-05-16T07:17:19.513462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.153503+0000","last_change":"2023-05-16T07:17:19.152221+0000","last_active":"2023-05-16T07:17:19.153503+0000","last_peered":"2023-05-16T07:17:19.153503+0000","last_clean":"2023-05-16T07:17:19.153503+0000","last_became_active":"2023-05-16T07:17:19.151948+0000","last_became_peered":"2023-05-16T07:17:19.151948+0000","last_unstale":"2023-05-16T07:17:19.153503+0000","last_undegraded":"2023-05-16T07:17:19.153503+0000","last_fullsized":"2023-05-16T07:17:19.153503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.520774+0000","last_change":"2023-05-16T07:17:19.520774+0000","last_active":"2023-05-16T07:17:19.520774+0000","last_peered":"2023-05-16T07:17:19.520774+0000","last_clean":"2023-05-16T07:17:19.520774+0000","last_became_active":"2023-05-16T07:17:19.520496+0000","last_became_peered":"2023-05-16T07:17:19.520496+0000","last_unstale":"2023-05-16T07:17:19.520774+0000","last_undegraded":"2023-05-16T07:17:19.520774+0000","last_fullsized":"2023-05-16T07:17:19.520774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.498483+0000","last_change":"2023-05-16T07:17:19.145077+0000","last_active":"2023-05-16T07:17:19.498483+0000","last_peered":"2023-05-16T07:17:19.498483+0000","last_clean":"2023-05-16T07:17:19.498483+0000","last_became_active":"2023-05-16T07:17:19.144763+0000","last_became_peered":"2023-05-16T07:17:19.144763+0000","last_unstale":"2023-05-16T07:17:19.498483+0000","last_undegraded":"2023-05-16T07:17:19.498483+0000","last_fullsized":"2023-05-16T07:17:19.498483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.524609+0000","last_change":"2023-05-16T07:17:19.524609+0000","last_active":"2023-05-16T07:17:19.524609+0000","last_peered":"2023-05-16T07:17:19.524609+0000","last_clean":"2023-05-16T07:17:19.524609+0000","last_became_active":"2023-05-16T07:17:19.524344+0000","last_became_peered":"2023-05-16T07:17:19.524344+0000","last_unstale":"2023-05-16T07:17:19.524609+0000","last_undegraded":"2023-05-16T07:17:19.524609+0000","last_fullsized":"2023-05-16T07:17:19.524609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151386+0000","last_change":"2023-05-16T07:17:19.146790+0000","last_active":"2023-05-16T07:17:19.151386+0000","last_peered":"2023-05-16T07:17:19.151386+0000","last_clean":"2023-05-16T07:17:19.151386+0000","last_became_active":"2023-05-16T07:17:19.146479+0000","last_became_peered":"2023-05-16T07:17:19.146479+0000","last_unstale":"2023-05-16T07:17:19.151386+0000","last_undegraded":"2023-05-16T07:17:19.151386+0000","last_fullsized":"2023-05-16T07:17:19.151386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510767+0000","last_change":"2023-05-16T07:17:19.510767+0000","last_active":"2023-05-16T07:17:19.510767+0000","last_peered":"2023-05-16T07:17:19.510767+0000","last_clean":"2023-05-16T07:17:19.510767+0000","last_became_active":"2023-05-16T07:17:19.510450+0000","last_became_peered":"2023-05-16T07:17:19.510450+0000","last_unstale":"2023-05-16T07:17:19.510767+0000","last_undegraded":"2023-05-16T07:17:19.510767+0000","last_fullsized":"2023-05-16T07:17:19.510767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506712+0000","last_change":"2023-05-16T07:17:19.149426+0000","last_active":"2023-05-16T07:17:19.506712+0000","last_peered":"2023-05-16T07:17:19.506712+0000","last_clean":"2023-05-16T07:17:19.506712+0000","last_became_active":"2023-05-16T07:17:19.149069+0000","last_became_peered":"2023-05-16T07:17:19.149069+0000","last_unstale":"2023-05-16T07:17:19.506712+0000","last_undegraded":"2023-05-16T07:17:19.506712+0000","last_fullsized":"2023-05-16T07:17:19.506712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513104+0000","last_change":"2023-05-16T07:17:19.513104+0000","last_active":"2023-05-16T07:17:19.513104+0000","last_peered":"2023-05-16T07:17:19.513104+0000","last_clean":"2023-05-16T07:17:19.513104+0000","last_became_active":"2023-05-16T07:17:19.512813+0000","last_became_peered":"2023-05-16T07:17:19.512813+0000","last_unstale":"2023-05-16T07:17:19.513104+0000","last_undegraded":"2023-05-16T07:17:19.513104+0000","last_fullsized":"2023-05-16T07:17:19.513104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147229+0000","last_change":"2023-05-16T07:17:19.143353+0000","last_active":"2023-05-16T07:17:19.147229+0000","last_peered":"2023-05-16T07:17:19.147229+0000","last_clean":"2023-05-16T07:17:19.147229+0000","last_became_active":"2023-05-16T07:17:19.142906+0000","last_became_peered":"2023-05-16T07:17:19.142906+0000","last_unstale":"2023-05-16T07:17:19.147229+0000","last_undegraded":"2023-05-16T07:17:19.147229+0000","last_fullsized":"2023-05-16T07:17:19.147229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513341+0000","last_change":"2023-05-16T07:17:19.513341+0000","last_active":"2023-05-16T07:17:19.513341+0000","last_peered":"2023-05-16T07:17:19.513341+0000","last_clean":"2023-05-16T07:17:19.513341+0000","last_became_active":"2023-05-16T07:17:19.512786+0000","last_became_peered":"2023-05-16T07:17:19.512786+0000","last_unstale":"2023-05-16T07:17:19.513341+0000","last_undegraded":"2023-05-16T07:17:19.513341+0000","last_fullsized":"2023-05-16T07:17:19.513341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.153456+0000","last_change":"2023-05-16T07:17:19.152043+0000","last_active":"2023-05-16T07:17:19.153456+0000","last_peered":"2023-05-16T07:17:19.153456+0000","last_clean":"2023-05-16T07:17:19.153456+0000","last_became_active":"2023-05-16T07:17:19.151779+0000","last_became_peered":"2023-05-16T07:17:19.151779+0000","last_unstale":"2023-05-16T07:17:19.153456+0000","last_undegraded":"2023-05-16T07:17:19.153456+0000","last_fullsized":"2023-05-16T07:17:19.153456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518210+0000","last_change":"2023-05-16T07:17:19.518210+0000","last_active":"2023-05-16T07:17:19.518210+0000","last_peered":"2023-05-16T07:17:19.518210+0000","last_clean":"2023-05-16T07:17:19.518210+0000","last_became_active":"2023-05-16T07:17:19.517913+0000","last_became_peered":"2023-05-16T07:17:19.517913+0000","last_unstale":"2023-05-16T07:17:19.518210+0000","last_undegraded":"2023-05-16T07:17:19.518210+0000","last_fullsized":"2023-05-16T07:17:19.518210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145415+0000","last_change":"2023-05-16T07:17:19.142419+0000","last_active":"2023-05-16T07:17:19.145415+0000","last_peered":"2023-05-16T07:17:19.145415+0000","last_clean":"2023-05-16T07:17:19.145415+0000","last_became_active":"2023-05-16T07:17:19.141893+0000","last_became_peered":"2023-05-16T07:17:19.141893+0000","last_unstale":"2023-05-16T07:17:19.145415+0000","last_undegraded":"2023-05-16T07:17:19.145415+0000","last_fullsized":"2023-05-16T07:17:19.145415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518269+0000","last_change":"2023-05-16T07:17:19.518269+0000","last_active":"2023-05-16T07:17:19.518269+0000","last_peered":"2023-05-16T07:17:19.518269+0000","last_clean":"2023-05-16T07:17:19.518269+0000","last_became_active":"2023-05-16T07:17:19.518075+0000","last_became_peered":"2023-05-16T07:17:19.518075+0000","last_unstale":"2023-05-16T07:17:19.518269+0000","last_undegraded":"2023-05-16T07:17:19.518269+0000","last_fullsized":"2023-05-16T07:17:19.518269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151474+0000","last_change":"2023-05-16T07:17:19.147071+0000","last_active":"2023-05-16T07:17:19.151474+0000","last_peered":"2023-05-16T07:17:19.151474+0000","last_clean":"2023-05-16T07:17:19.151474+0000","last_became_active":"2023-05-16T07:17:19.146537+0000","last_became_peered":"2023-05-16T07:17:19.146537+0000","last_unstale":"2023-05-16T07:17:19.151474+0000","last_undegraded":"2023-05-16T07:17:19.151474+0000","last_fullsized":"2023-05-16T07:17:19.151474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519990+0000","last_change":"2023-05-16T07:17:19.519990+0000","last_active":"2023-05-16T07:17:19.519990+0000","last_peered":"2023-05-16T07:17:19.519990+0000","last_clean":"2023-05-16T07:17:19.519990+0000","last_became_active":"2023-05-16T07:17:19.519439+0000","last_became_peered":"2023-05-16T07:17:19.519439+0000","last_unstale":"2023-05-16T07:17:19.519990+0000","last_undegraded":"2023-05-16T07:17:19.519990+0000","last_fullsized":"2023-05-16T07:17:19.519990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.150325+0000","last_change":"2023-05-16T07:17:19.146861+0000","last_active":"2023-05-16T07:17:19.150325+0000","last_peered":"2023-05-16T07:17:19.150325+0000","last_clean":"2023-05-16T07:17:19.150325+0000","last_became_active":"2023-05-16T07:17:19.146593+0000","last_became_peered":"2023-05-16T07:17:19.146593+0000","last_unstale":"2023-05-16T07:17:19.150325+0000","last_undegraded":"2023-05-16T07:17:19.150325+0000","last_fullsized":"2023-05-16T07:17:19.150325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511941+0000","last_change":"2023-05-16T07:17:19.511941+0000","last_active":"2023-05-16T07:17:19.511941+0000","last_peered":"2023-05-16T07:17:19.511941+0000","last_clean":"2023-05-16T07:17:19.511941+0000","last_became_active":"2023-05-16T07:17:19.511686+0000","last_became_peered":"2023-05-16T07:17:19.511686+0000","last_unstale":"2023-05-16T07:17:19.511941+0000","last_undegraded":"2023-05-16T07:17:19.511941+0000","last_fullsized":"2023-05-16T07:17:19.511941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146167+0000","last_change":"2023-05-16T07:17:19.141747+0000","last_active":"2023-05-16T07:17:19.146167+0000","last_peered":"2023-05-16T07:17:19.146167+0000","last_clean":"2023-05-16T07:17:19.146167+0000","last_became_active":"2023-05-16T07:17:19.141440+0000","last_became_peered":"2023-05-16T07:17:19.141440+0000","last_unstale":"2023-05-16T07:17:19.146167+0000","last_undegraded":"2023-05-16T07:17:19.146167+0000","last_fullsized":"2023-05-16T07:17:19.146167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515727+0000","last_change":"2023-05-16T07:17:19.515727+0000","last_active":"2023-05-16T07:17:19.515727+0000","last_peered":"2023-05-16T07:17:19.515727+0000","last_clean":"2023-05-16T07:17:19.515727+0000","last_became_active":"2023-05-16T07:17:19.515452+0000","last_became_peered":"2023-05-16T07:17:19.515452+0000","last_unstale":"2023-05-16T07:17:19.515727+0000","last_undegraded":"2023-05-16T07:17:19.515727+0000","last_fullsized":"2023-05-16T07:17:19.515727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151377+0000","last_change":"2023-05-16T07:17:19.149065+0000","last_active":"2023-05-16T07:17:19.151377+0000","last_peered":"2023-05-16T07:17:19.151377+0000","last_clean":"2023-05-16T07:17:19.151377+0000","last_became_active":"2023-05-16T07:17:19.148774+0000","last_became_peered":"2023-05-16T07:17:19.148774+0000","last_unstale":"2023-05-16T07:17:19.151377+0000","last_undegraded":"2023-05-16T07:17:19.151377+0000","last_fullsized":"2023-05-16T07:17:19.151377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515477+0000","last_change":"2023-05-16T07:17:19.515477+0000","last_active":"2023-05-16T07:17:19.515477+0000","last_peered":"2023-05-16T07:17:19.515477+0000","last_clean":"2023-05-16T07:17:19.515477+0000","last_became_active":"2023-05-16T07:17:19.515214+0000","last_became_peered":"2023-05-16T07:17:19.515214+0000","last_unstale":"2023-05-16T07:17:19.515477+0000","last_undegraded":"2023-05-16T07:17:19.515477+0000","last_fullsized":"2023-05-16T07:17:19.515477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147343+0000","last_change":"2023-05-16T07:17:19.145511+0000","last_active":"2023-05-16T07:17:19.147343+0000","last_peered":"2023-05-16T07:17:19.147343+0000","last_clean":"2023-05-16T07:17:19.147343+0000","last_became_active":"2023-05-16T07:17:19.145243+0000","last_became_peered":"2023-05-16T07:17:19.145243+0000","last_unstale":"2023-05-16T07:17:19.147343+0000","last_undegraded":"2023-05-16T07:17:19.147343+0000","last_fullsized":"2023-05-16T07:17:19.147343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515984+0000","last_change":"2023-05-16T07:17:19.515984+0000","last_active":"2023-05-16T07:17:19.515984+0000","last_peered":"2023-05-16T07:17:19.515984+0000","last_clean":"2023-05-16T07:17:19.515984+0000","last_became_active":"2023-05-16T07:17:19.515690+0000","last_became_peered":"2023-05-16T07:17:19.515690+0000","last_unstale":"2023-05-16T07:17:19.515984+0000","last_undegraded":"2023-05-16T07:17:19.515984+0000","last_fullsized":"2023-05-16T07:17:19.515984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.149684+0000","last_change":"2023-05-16T07:17:19.146729+0000","last_active":"2023-05-16T07:17:19.149684+0000","last_peered":"2023-05-16T07:17:19.149684+0000","last_clean":"2023-05-16T07:17:19.149684+0000","last_became_active":"2023-05-16T07:17:19.146474+0000","last_became_peered":"2023-05-16T07:17:19.146474+0000","last_unstale":"2023-05-16T07:17:19.149684+0000","last_undegraded":"2023-05-16T07:17:19.149684+0000","last_fullsized":"2023-05-16T07:17:19.149684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515316+0000","last_change":"2023-05-16T07:17:19.515316+0000","last_active":"2023-05-16T07:17:19.515316+0000","last_peered":"2023-05-16T07:17:19.515316+0000","last_clean":"2023-05-16T07:17:19.515316+0000","last_became_active":"2023-05-16T07:17:19.515022+0000","last_became_peered":"2023-05-16T07:17:19.515022+0000","last_unstale":"2023-05-16T07:17:19.515316+0000","last_undegraded":"2023-05-16T07:17:19.515316+0000","last_fullsized":"2023-05-16T07:17:19.515316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151425+0000","last_change":"2023-05-16T07:17:19.150026+0000","last_active":"2023-05-16T07:17:19.151425+0000","last_peered":"2023-05-16T07:17:19.151425+0000","last_clean":"2023-05-16T07:17:19.151425+0000","last_became_active":"2023-05-16T07:17:19.149733+0000","last_became_peered":"2023-05-16T07:17:19.149733+0000","last_unstale":"2023-05-16T07:17:19.151425+0000","last_undegraded":"2023-05-16T07:17:19.151425+0000","last_fullsized":"2023-05-16T07:17:19.151425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.524598+0000","last_change":"2023-05-16T07:17:19.524598+0000","last_active":"2023-05-16T07:17:19.524598+0000","last_peered":"2023-05-16T07:17:19.524598+0000","last_clean":"2023-05-16T07:17:19.524598+0000","last_became_active":"2023-05-16T07:17:19.524321+0000","last_became_peered":"2023-05-16T07:17:19.524321+0000","last_unstale":"2023-05-16T07:17:19.524598+0000","last_undegraded":"2023-05-16T07:17:19.524598+0000","last_fullsized":"2023-05-16T07:17:19.524598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.152140+0000","last_change":"2023-05-16T07:17:19.148333+0000","last_active":"2023-05-16T07:17:19.152140+0000","last_peered":"2023-05-16T07:17:19.152140+0000","last_clean":"2023-05-16T07:17:19.152140+0000","last_became_active":"2023-05-16T07:17:19.148064+0000","last_became_peered":"2023-05-16T07:17:19.148064+0000","last_unstale":"2023-05-16T07:17:19.152140+0000","last_undegraded":"2023-05-16T07:17:19.152140+0000","last_fullsized":"2023-05-16T07:17:19.152140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518476+0000","last_change":"2023-05-16T07:17:19.518476+0000","last_active":"2023-05-16T07:17:19.518476+0000","last_peered":"2023-05-16T07:17:19.518476+0000","last_clean":"2023-05-16T07:17:19.518476+0000","last_became_active":"2023-05-16T07:17:19.518199+0000","last_became_peered":"2023-05-16T07:17:19.518199+0000","last_unstale":"2023-05-16T07:17:19.518476+0000","last_undegraded":"2023-05-16T07:17:19.518476+0000","last_fullsized":"2023-05-16T07:17:19.518476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.507701+0000","last_change":"2023-05-16T07:17:19.148411+0000","last_active":"2023-05-16T07:17:19.507701+0000","last_peered":"2023-05-16T07:17:19.507701+0000","last_clean":"2023-05-16T07:17:19.507701+0000","last_became_active":"2023-05-16T07:17:19.146937+0000","last_became_peered":"2023-05-16T07:17:19.146937+0000","last_unstale":"2023-05-16T07:17:19.507701+0000","last_undegraded":"2023-05-16T07:17:19.507701+0000","last_fullsized":"2023-05-16T07:17:19.507701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.519739+0000","last_change":"2023-05-16T07:17:19.519739+0000","last_active":"2023-05-16T07:17:19.519739+0000","last_peered":"2023-05-16T07:17:19.519739+0000","last_clean":"2023-05-16T07:17:19.519739+0000","last_became_active":"2023-05-16T07:17:19.516776+0000","last_became_peered":"2023-05-16T07:17:19.516776+0000","last_unstale":"2023-05-16T07:17:19.519739+0000","last_undegraded":"2023-05-16T07:17:19.519739+0000","last_fullsized":"2023-05-16T07:17:19.519739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.505122+0000","last_change":"2023-05-16T07:17:19.147103+0000","last_active":"2023-05-16T07:17:19.505122+0000","last_peered":"2023-05-16T07:17:19.505122+0000","last_clean":"2023-05-16T07:17:19.505122+0000","last_became_active":"2023-05-16T07:17:19.146097+0000","last_became_peered":"2023-05-16T07:17:19.146097+0000","last_unstale":"2023-05-16T07:17:19.505122+0000","last_undegraded":"2023-05-16T07:17:19.505122+0000","last_fullsized":"2023-05-16T07:17:19.505122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511185+0000","last_change":"2023-05-16T07:17:19.511185+0000","last_active":"2023-05-16T07:17:19.511185+0000","last_peered":"2023-05-16T07:17:19.511185+0000","last_clean":"2023-05-16T07:17:19.511185+0000","last_became_active":"2023-05-16T07:17:19.510914+0000","last_became_peered":"2023-05-16T07:17:19.510914+0000","last_unstale":"2023-05-16T07:17:19.511185+0000","last_undegraded":"2023-05-16T07:17:19.511185+0000","last_fullsized":"2023-05-16T07:17:19.511185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518798+0000","last_change":"2023-05-16T07:17:19.144810+0000","last_active":"2023-05-16T07:17:19.518798+0000","last_peered":"2023-05-16T07:17:19.518798+0000","last_clean":"2023-05-16T07:17:19.518798+0000","last_became_active":"2023-05-16T07:17:19.144540+0000","last_became_peered":"2023-05-16T07:17:19.144540+0000","last_unstale":"2023-05-16T07:17:19.518798+0000","last_undegraded":"2023-05-16T07:17:19.518798+0000","last_fullsized":"2023-05-16T07:17:19.518798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516844+0000","last_change":"2023-05-16T07:17:19.516844+0000","last_active":"2023-05-16T07:17:19.516844+0000","last_peered":"2023-05-16T07:17:19.516844+0000","last_clean":"2023-05-16T07:17:19.516844+0000","last_became_active":"2023-05-16T07:17:19.516529+0000","last_became_peered":"2023-05-16T07:17:19.516529+0000","last_unstale":"2023-05-16T07:17:19.516844+0000","last_undegraded":"2023-05-16T07:17:19.516844+0000","last_fullsized":"2023-05-16T07:17:19.516844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.152209+0000","last_change":"2023-05-16T07:17:19.150631+0000","last_active":"2023-05-16T07:17:19.152209+0000","last_peered":"2023-05-16T07:17:19.152209+0000","last_clean":"2023-05-16T07:17:19.152209+0000","last_became_active":"2023-05-16T07:17:19.150358+0000","last_became_peered":"2023-05-16T07:17:19.150358+0000","last_unstale":"2023-05-16T07:17:19.152209+0000","last_undegraded":"2023-05-16T07:17:19.152209+0000","last_fullsized":"2023-05-16T07:17:19.152209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.509924+0000","last_change":"2023-05-16T07:17:19.509924+0000","last_active":"2023-05-16T07:17:19.509924+0000","last_peered":"2023-05-16T07:17:19.509924+0000","last_clean":"2023-05-16T07:17:19.509924+0000","last_became_active":"2023-05-16T07:17:19.509634+0000","last_became_peered":"2023-05-16T07:17:19.509634+0000","last_unstale":"2023-05-16T07:17:19.509924+0000","last_undegraded":"2023-05-16T07:17:19.509924+0000","last_fullsized":"2023-05-16T07:17:19.509924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151438+0000","last_change":"2023-05-16T07:17:19.146889+0000","last_active":"2023-05-16T07:17:19.151438+0000","last_peered":"2023-05-16T07:17:19.151438+0000","last_clean":"2023-05-16T07:17:19.151438+0000","last_became_active":"2023-05-16T07:17:19.146563+0000","last_became_peered":"2023-05-16T07:17:19.146563+0000","last_unstale":"2023-05-16T07:17:19.151438+0000","last_undegraded":"2023-05-16T07:17:19.151438+0000","last_fullsized":"2023-05-16T07:17:19.151438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.513585+0000","last_change":"2023-05-16T07:17:19.513585+0000","last_active":"2023-05-16T07:17:19.513585+0000","last_peered":"2023-05-16T07:17:19.513585+0000","last_clean":"2023-05-16T07:17:19.513585+0000","last_became_active":"2023-05-16T07:17:19.513289+0000","last_became_peered":"2023-05-16T07:17:19.513289+0000","last_unstale":"2023-05-16T07:17:19.513585+0000","last_undegraded":"2023-05-16T07:17:19.513585+0000","last_fullsized":"2023-05-16T07:17:19.513585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147210+0000","last_change":"2023-05-16T07:17:19.143176+0000","last_active":"2023-05-16T07:17:19.147210+0000","last_peered":"2023-05-16T07:17:19.147210+0000","last_clean":"2023-05-16T07:17:19.147210+0000","last_became_active":"2023-05-16T07:17:19.142570+0000","last_became_peered":"2023-05-16T07:17:19.142570+0000","last_unstale":"2023-05-16T07:17:19.147210+0000","last_undegraded":"2023-05-16T07:17:19.147210+0000","last_fullsized":"2023-05-16T07:17:19.147210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517329+0000","last_change":"2023-05-16T07:17:19.517329+0000","last_active":"2023-05-16T07:17:19.517329+0000","last_peered":"2023-05-16T07:17:19.517329+0000","last_clean":"2023-05-16T07:17:19.517329+0000","last_became_active":"2023-05-16T07:17:19.516706+0000","last_became_peered":"2023-05-16T07:17:19.516706+0000","last_unstale":"2023-05-16T07:17:19.517329+0000","last_undegraded":"2023-05-16T07:17:19.517329+0000","last_fullsized":"2023-05-16T07:17:19.517329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146193+0000","last_change":"2023-05-16T07:17:19.139471+0000","last_active":"2023-05-16T07:17:19.146193+0000","last_peered":"2023-05-16T07:17:19.146193+0000","last_clean":"2023-05-16T07:17:19.146193+0000","last_became_active":"2023-05-16T07:17:19.138683+0000","last_became_peered":"2023-05-16T07:17:19.138683+0000","last_unstale":"2023-05-16T07:17:19.146193+0000","last_undegraded":"2023-05-16T07:17:19.146193+0000","last_fullsized":"2023-05-16T07:17:19.146193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512399+0000","last_change":"2023-05-16T07:17:19.512399+0000","last_active":"2023-05-16T07:17:19.512399+0000","last_peered":"2023-05-16T07:17:19.512399+0000","last_clean":"2023-05-16T07:17:19.512399+0000","last_became_active":"2023-05-16T07:17:19.512115+0000","last_became_peered":"2023-05-16T07:17:19.512115+0000","last_unstale":"2023-05-16T07:17:19.512399+0000","last_undegraded":"2023-05-16T07:17:19.512399+0000","last_fullsized":"2023-05-16T07:17:19.512399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.149732+0000","last_change":"2023-05-16T07:17:19.146828+0000","last_active":"2023-05-16T07:17:19.149732+0000","last_peered":"2023-05-16T07:17:19.149732+0000","last_clean":"2023-05-16T07:17:19.149732+0000","last_became_active":"2023-05-16T07:17:19.146553+0000","last_became_peered":"2023-05-16T07:17:19.146553+0000","last_unstale":"2023-05-16T07:17:19.149732+0000","last_undegraded":"2023-05-16T07:17:19.149732+0000","last_fullsized":"2023-05-16T07:17:19.149732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512096+0000","last_change":"2023-05-16T07:17:19.512096+0000","last_active":"2023-05-16T07:17:19.512096+0000","last_peered":"2023-05-16T07:17:19.512096+0000","last_clean":"2023-05-16T07:17:19.512096+0000","last_became_active":"2023-05-16T07:17:19.511858+0000","last_became_peered":"2023-05-16T07:17:19.511858+0000","last_unstale":"2023-05-16T07:17:19.512096+0000","last_undegraded":"2023-05-16T07:17:19.512096+0000","last_fullsized":"2023-05-16T07:17:19.512096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.154015+0000","last_change":"2023-05-16T07:17:19.153507+0000","last_active":"2023-05-16T07:17:19.154015+0000","last_peered":"2023-05-16T07:17:19.154015+0000","last_clean":"2023-05-16T07:17:19.154015+0000","last_became_active":"2023-05-16T07:17:19.153218+0000","last_became_peered":"2023-05-16T07:17:19.153218+0000","last_unstale":"2023-05-16T07:17:19.154015+0000","last_undegraded":"2023-05-16T07:17:19.154015+0000","last_fullsized":"2023-05-16T07:17:19.154015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516452+0000","last_change":"2023-05-16T07:17:19.516452+0000","last_active":"2023-05-16T07:17:19.516452+0000","last_peered":"2023-05-16T07:17:19.516452+0000","last_clean":"2023-05-16T07:17:19.516452+0000","last_became_active":"2023-05-16T07:17:19.516242+0000","last_became_peered":"2023-05-16T07:17:19.516242+0000","last_unstale":"2023-05-16T07:17:19.516452+0000","last_undegraded":"2023-05-16T07:17:19.516452+0000","last_fullsized":"2023-05-16T07:17:19.516452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.144812+0000","last_change":"2023-05-16T07:17:19.138064+0000","last_active":"2023-05-16T07:17:19.144812+0000","last_peered":"2023-05-16T07:17:19.144812+0000","last_clean":"2023-05-16T07:17:19.144812+0000","last_became_active":"2023-05-16T07:17:19.137794+0000","last_became_peered":"2023-05-16T07:17:19.137794+0000","last_unstale":"2023-05-16T07:17:19.144812+0000","last_undegraded":"2023-05-16T07:17:19.144812+0000","last_fullsized":"2023-05-16T07:17:19.144812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.521925+0000","last_change":"2023-05-16T07:17:19.521925+0000","last_active":"2023-05-16T07:17:19.521925+0000","last_peered":"2023-05-16T07:17:19.521925+0000","last_clean":"2023-05-16T07:17:19.521925+0000","last_became_active":"2023-05-16T07:17:19.521635+0000","last_became_peered":"2023-05-16T07:17:19.521635+0000","last_unstale":"2023-05-16T07:17:19.521925+0000","last_undegraded":"2023-05-16T07:17:19.521925+0000","last_fullsized":"2023-05-16T07:17:19.521925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"78'3","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.516181+0000","last_change":"2023-05-16T07:17:19.149599+0000","last_active":"2023-05-16T07:17:19.516181+0000","last_peered":"2023-05-16T07:17:19.516181+0000","last_clean":"2023-05-16T07:17:19.516181+0000","last_became_active":"2023-05-16T07:17:19.149216+0000","last_became_peered":"2023-05-16T07:17:19.149216+0000","last_unstale":"2023-05-16T07:17:19.516181+0000","last_undegraded":"2023-05-16T07:17:19.516181+0000","last_fullsized":"2023-05-16T07:17:19.516181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517208+0000","last_change":"2023-05-16T07:17:19.517208+0000","last_active":"2023-05-16T07:17:19.517208+0000","last_peered":"2023-05-16T07:17:19.517208+0000","last_clean":"2023-05-16T07:17:19.517208+0000","last_became_active":"2023-05-16T07:17:19.516916+0000","last_became_peered":"2023-05-16T07:17:19.516916+0000","last_unstale":"2023-05-16T07:17:19.517208+0000","last_undegraded":"2023-05-16T07:17:19.517208+0000","last_fullsized":"2023-05-16T07:17:19.517208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.524140+0000","last_change":"2023-05-16T07:17:19.140024+0000","last_active":"2023-05-16T07:17:19.524140+0000","last_peered":"2023-05-16T07:17:19.524140+0000","last_clean":"2023-05-16T07:17:19.524140+0000","last_became_active":"2023-05-16T07:17:19.139570+0000","last_became_peered":"2023-05-16T07:17:19.139570+0000","last_unstale":"2023-05-16T07:17:19.524140+0000","last_undegraded":"2023-05-16T07:17:19.524140+0000","last_fullsized":"2023-05-16T07:17:19.524140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.522144+0000","last_change":"2023-05-16T07:17:19.522144+0000","last_active":"2023-05-16T07:17:19.522144+0000","last_peered":"2023-05-16T07:17:19.522144+0000","last_clean":"2023-05-16T07:17:19.522144+0000","last_became_active":"2023-05-16T07:17:19.521859+0000","last_became_peered":"2023-05-16T07:17:19.521859+0000","last_unstale":"2023-05-16T07:17:19.522144+0000","last_undegraded":"2023-05-16T07:17:19.522144+0000","last_fullsized":"2023-05-16T07:17:19.522144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506821+0000","last_change":"2023-05-16T07:17:19.152636+0000","last_active":"2023-05-16T07:17:19.506821+0000","last_peered":"2023-05-16T07:17:19.506821+0000","last_clean":"2023-05-16T07:17:19.506821+0000","last_became_active":"2023-05-16T07:17:19.151206+0000","last_became_peered":"2023-05-16T07:17:19.151206+0000","last_unstale":"2023-05-16T07:17:19.506821+0000","last_undegraded":"2023-05-16T07:17:19.506821+0000","last_fullsized":"2023-05-16T07:17:19.506821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511571+0000","last_change":"2023-05-16T07:17:19.511571+0000","last_active":"2023-05-16T07:17:19.511571+0000","last_peered":"2023-05-16T07:17:19.511571+0000","last_clean":"2023-05-16T07:17:19.511571+0000","last_became_active":"2023-05-16T07:17:19.511285+0000","last_became_peered":"2023-05-16T07:17:19.511285+0000","last_unstale":"2023-05-16T07:17:19.511571+0000","last_undegraded":"2023-05-16T07:17:19.511571+0000","last_fullsized":"2023-05-16T07:17:19.511571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.150832+0000","last_change":"2023-05-16T07:17:19.149874+0000","last_active":"2023-05-16T07:17:19.150832+0000","last_peered":"2023-05-16T07:17:19.150832+0000","last_clean":"2023-05-16T07:17:19.150832+0000","last_became_active":"2023-05-16T07:17:19.149595+0000","last_became_peered":"2023-05-16T07:17:19.149595+0000","last_unstale":"2023-05-16T07:17:19.150832+0000","last_undegraded":"2023-05-16T07:17:19.150832+0000","last_fullsized":"2023-05-16T07:17:19.150832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510410+0000","last_change":"2023-05-16T07:17:19.510410+0000","last_active":"2023-05-16T07:17:19.510410+0000","last_peered":"2023-05-16T07:17:19.510410+0000","last_clean":"2023-05-16T07:17:19.510410+0000","last_became_active":"2023-05-16T07:17:19.509417+0000","last_became_peered":"2023-05-16T07:17:19.509417+0000","last_unstale":"2023-05-16T07:17:19.510410+0000","last_undegraded":"2023-05-16T07:17:19.510410+0000","last_fullsized":"2023-05-16T07:17:19.510410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151381+0000","last_change":"2023-05-16T07:17:19.148341+0000","last_active":"2023-05-16T07:17:19.151381+0000","last_peered":"2023-05-16T07:17:19.151381+0000","last_clean":"2023-05-16T07:17:19.151381+0000","last_became_active":"2023-05-16T07:17:19.148060+0000","last_became_peered":"2023-05-16T07:17:19.148060+0000","last_unstale":"2023-05-16T07:17:19.151381+0000","last_undegraded":"2023-05-16T07:17:19.151381+0000","last_fullsized":"2023-05-16T07:17:19.151381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515607+0000","last_change":"2023-05-16T07:17:19.515607+0000","last_active":"2023-05-16T07:17:19.515607+0000","last_peered":"2023-05-16T07:17:19.515607+0000","last_clean":"2023-05-16T07:17:19.515607+0000","last_became_active":"2023-05-16T07:17:19.515347+0000","last_became_peered":"2023-05-16T07:17:19.515347+0000","last_unstale":"2023-05-16T07:17:19.515607+0000","last_undegraded":"2023-05-16T07:17:19.515607+0000","last_fullsized":"2023-05-16T07:17:19.515607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"78'4","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.521939+0000","last_change":"2023-05-16T07:17:19.141719+0000","last_active":"2023-05-16T07:17:19.521939+0000","last_peered":"2023-05-16T07:17:19.521939+0000","last_clean":"2023-05-16T07:17:19.521939+0000","last_became_active":"2023-05-16T07:17:19.140418+0000","last_became_peered":"2023-05-16T07:17:19.140418+0000","last_unstale":"2023-05-16T07:17:19.521939+0000","last_undegraded":"2023-05-16T07:17:19.521939+0000","last_fullsized":"2023-05-16T07:17:19.521939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.521707+0000","last_change":"2023-05-16T07:17:19.521707+0000","last_active":"2023-05-16T07:17:19.521707+0000","last_peered":"2023-05-16T07:17:19.521707+0000","last_clean":"2023-05-16T07:17:19.521707+0000","last_became_active":"2023-05-16T07:17:19.521396+0000","last_became_peered":"2023-05-16T07:17:19.521396+0000","last_unstale":"2023-05-16T07:17:19.521707+0000","last_undegraded":"2023-05-16T07:17:19.521707+0000","last_fullsized":"2023-05-16T07:17:19.521707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515843+0000","last_change":"2023-05-16T07:17:19.146483+0000","last_active":"2023-05-16T07:17:19.515843+0000","last_peered":"2023-05-16T07:17:19.515843+0000","last_clean":"2023-05-16T07:17:19.515843+0000","last_became_active":"2023-05-16T07:17:19.146137+0000","last_became_peered":"2023-05-16T07:17:19.146137+0000","last_unstale":"2023-05-16T07:17:19.515843+0000","last_undegraded":"2023-05-16T07:17:19.515843+0000","last_fullsized":"2023-05-16T07:17:19.515843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.511259+0000","last_change":"2023-05-16T07:17:19.511259+0000","last_active":"2023-05-16T07:17:19.511259+0000","last_peered":"2023-05-16T07:17:19.511259+0000","last_clean":"2023-05-16T07:17:19.511259+0000","last_became_active":"2023-05-16T07:17:19.510981+0000","last_became_peered":"2023-05-16T07:17:19.510981+0000","last_unstale":"2023-05-16T07:17:19.511259+0000","last_undegraded":"2023-05-16T07:17:19.511259+0000","last_fullsized":"2023-05-16T07:17:19.511259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.148446+0000","last_change":"2023-05-16T07:17:19.144404+0000","last_active":"2023-05-16T07:17:19.148446+0000","last_peered":"2023-05-16T07:17:19.148446+0000","last_clean":"2023-05-16T07:17:19.148446+0000","last_became_active":"2023-05-16T07:17:19.144084+0000","last_became_peered":"2023-05-16T07:17:19.144084+0000","last_unstale":"2023-05-16T07:17:19.148446+0000","last_undegraded":"2023-05-16T07:17:19.148446+0000","last_fullsized":"2023-05-16T07:17:19.148446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514210+0000","last_change":"2023-05-16T07:17:19.514210+0000","last_active":"2023-05-16T07:17:19.514210+0000","last_peered":"2023-05-16T07:17:19.514210+0000","last_clean":"2023-05-16T07:17:19.514210+0000","last_became_active":"2023-05-16T07:17:19.513894+0000","last_became_peered":"2023-05-16T07:17:19.513894+0000","last_unstale":"2023-05-16T07:17:19.514210+0000","last_undegraded":"2023-05-16T07:17:19.514210+0000","last_fullsized":"2023-05-16T07:17:19.514210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145473+0000","last_change":"2023-05-16T07:17:19.137803+0000","last_active":"2023-05-16T07:17:19.145473+0000","last_peered":"2023-05-16T07:17:19.145473+0000","last_clean":"2023-05-16T07:17:19.145473+0000","last_became_active":"2023-05-16T07:17:19.137474+0000","last_became_peered":"2023-05-16T07:17:19.137474+0000","last_unstale":"2023-05-16T07:17:19.145473+0000","last_undegraded":"2023-05-16T07:17:19.145473+0000","last_fullsized":"2023-05-16T07:17:19.145473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517508+0000","last_change":"2023-05-16T07:17:19.517508+0000","last_active":"2023-05-16T07:17:19.517508+0000","last_peered":"2023-05-16T07:17:19.517508+0000","last_clean":"2023-05-16T07:17:19.517508+0000","last_became_active":"2023-05-16T07:17:19.517019+0000","last_became_peered":"2023-05-16T07:17:19.517019+0000","last_unstale":"2023-05-16T07:17:19.517508+0000","last_undegraded":"2023-05-16T07:17:19.517508+0000","last_fullsized":"2023-05-16T07:17:19.517508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.148515+0000","last_change":"2023-05-16T07:17:19.139835+0000","last_active":"2023-05-16T07:17:19.148515+0000","last_peered":"2023-05-16T07:17:19.148515+0000","last_clean":"2023-05-16T07:17:19.148515+0000","last_became_active":"2023-05-16T07:17:19.139214+0000","last_became_peered":"2023-05-16T07:17:19.139214+0000","last_unstale":"2023-05-16T07:17:19.148515+0000","last_undegraded":"2023-05-16T07:17:19.148515+0000","last_fullsized":"2023-05-16T07:17:19.148515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.512598+0000","last_change":"2023-05-16T07:17:19.512598+0000","last_active":"2023-05-16T07:17:19.512598+0000","last_peered":"2023-05-16T07:17:19.512598+0000","last_clean":"2023-05-16T07:17:19.512598+0000","last_became_active":"2023-05-16T07:17:19.512331+0000","last_became_peered":"2023-05-16T07:17:19.512331+0000","last_unstale":"2023-05-16T07:17:19.512598+0000","last_undegraded":"2023-05-16T07:17:19.512598+0000","last_fullsized":"2023-05-16T07:17:19.512598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151343+0000","last_change":"2023-05-16T07:17:19.148092+0000","last_active":"2023-05-16T07:17:19.151343+0000","last_peered":"2023-05-16T07:17:19.151343+0000","last_clean":"2023-05-16T07:17:19.151343+0000","last_became_active":"2023-05-16T07:17:19.147750+0000","last_became_peered":"2023-05-16T07:17:19.147750+0000","last_unstale":"2023-05-16T07:17:19.151343+0000","last_undegraded":"2023-05-16T07:17:19.151343+0000","last_fullsized":"2023-05-16T07:17:19.151343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.520746+0000","last_change":"2023-05-16T07:17:19.520746+0000","last_active":"2023-05-16T07:17:19.520746+0000","last_peered":"2023-05-16T07:17:19.520746+0000","last_clean":"2023-05-16T07:17:19.520746+0000","last_became_active":"2023-05-16T07:17:19.520457+0000","last_became_peered":"2023-05-16T07:17:19.520457+0000","last_unstale":"2023-05-16T07:17:19.520746+0000","last_undegraded":"2023-05-16T07:17:19.520746+0000","last_fullsized":"2023-05-16T07:17:19.520746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151584+0000","last_change":"2023-05-16T07:17:19.149907+0000","last_active":"2023-05-16T07:17:19.151584+0000","last_peered":"2023-05-16T07:17:19.151584+0000","last_clean":"2023-05-16T07:17:19.151584+0000","last_became_active":"2023-05-16T07:17:19.149599+0000","last_became_peered":"2023-05-16T07:17:19.149599+0000","last_unstale":"2023-05-16T07:17:19.151584+0000","last_undegraded":"2023-05-16T07:17:19.151584+0000","last_fullsized":"2023-05-16T07:17:19.151584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.514675+0000","last_change":"2023-05-16T07:17:19.514675+0000","last_active":"2023-05-16T07:17:19.514675+0000","last_peered":"2023-05-16T07:17:19.514675+0000","last_clean":"2023-05-16T07:17:19.514675+0000","last_became_active":"2023-05-16T07:17:19.514456+0000","last_became_peered":"2023-05-16T07:17:19.514456+0000","last_unstale":"2023-05-16T07:17:19.514675+0000","last_undegraded":"2023-05-16T07:17:19.514675+0000","last_fullsized":"2023-05-16T07:17:19.514675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.146172+0000","last_change":"2023-05-16T07:17:19.138961+0000","last_active":"2023-05-16T07:17:19.146172+0000","last_peered":"2023-05-16T07:17:19.146172+0000","last_clean":"2023-05-16T07:17:19.146172+0000","last_became_active":"2023-05-16T07:17:19.138304+0000","last_became_peered":"2023-05-16T07:17:19.138304+0000","last_unstale":"2023-05-16T07:17:19.146172+0000","last_undegraded":"2023-05-16T07:17:19.146172+0000","last_fullsized":"2023-05-16T07:17:19.146172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.506518+0000","last_change":"2023-05-16T07:17:19.506518+0000","last_active":"2023-05-16T07:17:19.506518+0000","last_peered":"2023-05-16T07:17:19.506518+0000","last_clean":"2023-05-16T07:17:19.506518+0000","last_became_active":"2023-05-16T07:17:19.506309+0000","last_became_peered":"2023-05-16T07:17:19.506309+0000","last_unstale":"2023-05-16T07:17:19.506518+0000","last_undegraded":"2023-05-16T07:17:19.506518+0000","last_fullsized":"2023-05-16T07:17:19.506518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147984+0000","last_change":"2023-05-16T07:17:19.140206+0000","last_active":"2023-05-16T07:17:19.147984+0000","last_peered":"2023-05-16T07:17:19.147984+0000","last_clean":"2023-05-16T07:17:19.147984+0000","last_became_active":"2023-05-16T07:17:19.139903+0000","last_became_peered":"2023-05-16T07:17:19.139903+0000","last_unstale":"2023-05-16T07:17:19.147984+0000","last_undegraded":"2023-05-16T07:17:19.147984+0000","last_fullsized":"2023-05-16T07:17:19.147984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.525240+0000","last_change":"2023-05-16T07:17:19.525240+0000","last_active":"2023-05-16T07:17:19.525240+0000","last_peered":"2023-05-16T07:17:19.525240+0000","last_clean":"2023-05-16T07:17:19.525240+0000","last_became_active":"2023-05-16T07:17:19.524369+0000","last_became_peered":"2023-05-16T07:17:19.524369+0000","last_unstale":"2023-05-16T07:17:19.525240+0000","last_undegraded":"2023-05-16T07:17:19.525240+0000","last_fullsized":"2023-05-16T07:17:19.525240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.505034+0000","last_change":"2023-05-16T07:17:19.150196+0000","last_active":"2023-05-16T07:17:19.505034+0000","last_peered":"2023-05-16T07:17:19.505034+0000","last_clean":"2023-05-16T07:17:19.505034+0000","last_became_active":"2023-05-16T07:17:19.149926+0000","last_became_peered":"2023-05-16T07:17:19.149926+0000","last_unstale":"2023-05-16T07:17:19.505034+0000","last_undegraded":"2023-05-16T07:17:19.505034+0000","last_fullsized":"2023-05-16T07:17:19.505034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.126574+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:17:19.126574+0000","last_peered":"2023-05-16T07:17:19.126574+0000","last_clean":"2023-05-16T07:17:19.126574+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:17:19.126574+0000","last_undegraded":"2023-05-16T07:17:19.126574+0000","last_fullsized":"2023-05-16T07:17:19.126574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510225+0000","last_change":"2023-05-16T07:17:19.510225+0000","last_active":"2023-05-16T07:17:19.510225+0000","last_peered":"2023-05-16T07:17:19.510225+0000","last_clean":"2023-05-16T07:17:19.510225+0000","last_became_active":"2023-05-16T07:17:19.509046+0000","last_became_peered":"2023-05-16T07:17:19.509046+0000","last_unstale":"2023-05-16T07:17:19.510225+0000","last_undegraded":"2023-05-16T07:17:19.510225+0000","last_fullsized":"2023-05-16T07:17:19.510225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.145795+0000","last_change":"2023-05-16T07:17:19.140774+0000","last_active":"2023-05-16T07:17:19.145795+0000","last_peered":"2023-05-16T07:17:19.145795+0000","last_clean":"2023-05-16T07:17:19.145795+0000","last_became_active":"2023-05-16T07:17:19.140517+0000","last_became_peered":"2023-05-16T07:17:19.140517+0000","last_unstale":"2023-05-16T07:17:19.145795+0000","last_undegraded":"2023-05-16T07:17:19.145795+0000","last_fullsized":"2023-05-16T07:17:19.145795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.136728+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:17:19.136728+0000","last_peered":"2023-05-16T07:17:19.136728+0000","last_clean":"2023-05-16T07:17:19.136728+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:17:19.136728+0000","last_undegraded":"2023-05-16T07:17:19.136728+0000","last_fullsized":"2023-05-16T07:17:19.136728+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.518065+0000","last_change":"2023-05-16T07:17:19.518065+0000","last_active":"2023-05-16T07:17:19.518065+0000","last_peered":"2023-05-16T07:17:19.518065+0000","last_clean":"2023-05-16T07:17:19.518065+0000","last_became_active":"2023-05-16T07:17:19.517796+0000","last_became_peered":"2023-05-16T07:17:19.517796+0000","last_unstale":"2023-05-16T07:17:19.518065+0000","last_undegraded":"2023-05-16T07:17:19.518065+0000","last_fullsized":"2023-05-16T07:17:19.518065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.150814+0000","last_change":"2023-05-16T07:17:19.149475+0000","last_active":"2023-05-16T07:17:19.150814+0000","last_peered":"2023-05-16T07:17:19.150814+0000","last_clean":"2023-05-16T07:17:19.150814+0000","last_became_active":"2023-05-16T07:17:19.148603+0000","last_became_peered":"2023-05-16T07:17:19.148603+0000","last_unstale":"2023-05-16T07:17:19.150814+0000","last_undegraded":"2023-05-16T07:17:19.150814+0000","last_fullsized":"2023-05-16T07:17:19.150814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.135945+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:17:19.135945+0000","last_peered":"2023-05-16T07:17:19.135945+0000","last_clean":"2023-05-16T07:17:19.135945+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:17:19.135945+0000","last_undegraded":"2023-05-16T07:17:19.135945+0000","last_fullsized":"2023-05-16T07:17:19.135945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515057+0000","last_change":"2023-05-16T07:17:19.515057+0000","last_active":"2023-05-16T07:17:19.515057+0000","last_peered":"2023-05-16T07:17:19.515057+0000","last_clean":"2023-05-16T07:17:19.515057+0000","last_became_active":"2023-05-16T07:17:19.514754+0000","last_became_peered":"2023-05-16T07:17:19.514754+0000","last_unstale":"2023-05-16T07:17:19.515057+0000","last_undegraded":"2023-05-16T07:17:19.515057+0000","last_fullsized":"2023-05-16T07:17:19.515057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147345+0000","last_change":"2023-05-16T07:17:19.143563+0000","last_active":"2023-05-16T07:17:19.147345+0000","last_peered":"2023-05-16T07:17:19.147345+0000","last_clean":"2023-05-16T07:17:19.147345+0000","last_became_active":"2023-05-16T07:17:19.143253+0000","last_became_peered":"2023-05-16T07:17:19.143253+0000","last_unstale":"2023-05-16T07:17:19.147345+0000","last_undegraded":"2023-05-16T07:17:19.147345+0000","last_fullsized":"2023-05-16T07:17:19.147345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.141353+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:17:19.141353+0000","last_peered":"2023-05-16T07:17:19.141353+0000","last_clean":"2023-05-16T07:17:19.141353+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:17:19.141353+0000","last_undegraded":"2023-05-16T07:17:19.141353+0000","last_fullsized":"2023-05-16T07:17:19.141353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.139213+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:17:19.139213+0000","last_peered":"2023-05-16T07:17:19.139213+0000","last_clean":"2023-05-16T07:17:19.139213+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:17:19.139213+0000","last_undegraded":"2023-05-16T07:17:19.139213+0000","last_fullsized":"2023-05-16T07:17:19.139213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.515264+0000","last_change":"2023-05-16T07:17:19.515264+0000","last_active":"2023-05-16T07:17:19.515264+0000","last_peered":"2023-05-16T07:17:19.515264+0000","last_clean":"2023-05-16T07:17:19.515264+0000","last_became_active":"2023-05-16T07:17:19.515018+0000","last_became_peered":"2023-05-16T07:17:19.515018+0000","last_unstale":"2023-05-16T07:17:19.515264+0000","last_undegraded":"2023-05-16T07:17:19.515264+0000","last_fullsized":"2023-05-16T07:17:19.515264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"78'1","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.502067+0000","last_change":"2023-05-16T07:17:19.139087+0000","last_active":"2023-05-16T07:17:19.502067+0000","last_peered":"2023-05-16T07:17:19.502067+0000","last_clean":"2023-05-16T07:17:19.502067+0000","last_became_active":"2023-05-16T07:17:19.138776+0000","last_became_peered":"2023-05-16T07:17:19.138776+0000","last_unstale":"2023-05-16T07:17:19.502067+0000","last_undegraded":"2023-05-16T07:17:19.502067+0000","last_fullsized":"2023-05-16T07:17:19.502067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.139390+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:17:19.139390+0000","last_peered":"2023-05-16T07:17:19.139390+0000","last_clean":"2023-05-16T07:17:19.139390+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:17:19.139390+0000","last_undegraded":"2023-05-16T07:17:19.139390+0000","last_fullsized":"2023-05-16T07:17:19.139390+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.510943+0000","last_change":"2023-05-16T07:17:19.510943+0000","last_active":"2023-05-16T07:17:19.510943+0000","last_peered":"2023-05-16T07:17:19.510943+0000","last_clean":"2023-05-16T07:17:19.510943+0000","last_became_active":"2023-05-16T07:17:19.510681+0000","last_became_peered":"2023-05-16T07:17:19.510681+0000","last_unstale":"2023-05-16T07:17:19.510943+0000","last_undegraded":"2023-05-16T07:17:19.510943+0000","last_fullsized":"2023-05-16T07:17:19.510943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"78'2","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.522415+0000","last_change":"2023-05-16T07:17:19.150924+0000","last_active":"2023-05-16T07:17:19.522415+0000","last_peered":"2023-05-16T07:17:19.522415+0000","last_clean":"2023-05-16T07:17:19.522415+0000","last_became_active":"2023-05-16T07:17:19.150672+0000","last_became_peered":"2023-05-16T07:17:19.150672+0000","last_unstale":"2023-05-16T07:17:19.522415+0000","last_undegraded":"2023-05-16T07:17:19.522415+0000","last_fullsized":"2023-05-16T07:17:19.522415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.140288+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:17:19.140288+0000","last_peered":"2023-05-16T07:17:19.140288+0000","last_clean":"2023-05-16T07:17:19.140288+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:17:19.140288+0000","last_undegraded":"2023-05-16T07:17:19.140288+0000","last_fullsized":"2023-05-16T07:17:19.140288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.520237+0000","last_change":"2023-05-16T07:17:19.520237+0000","last_active":"2023-05-16T07:17:19.520237+0000","last_peered":"2023-05-16T07:17:19.520237+0000","last_clean":"2023-05-16T07:17:19.520237+0000","last_became_active":"2023-05-16T07:17:19.517683+0000","last_became_peered":"2023-05-16T07:17:19.517683+0000","last_unstale":"2023-05-16T07:17:19.520237+0000","last_undegraded":"2023-05-16T07:17:19.520237+0000","last_fullsized":"2023-05-16T07:17:19.520237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.151494+0000","last_change":"2023-05-16T07:17:19.150106+0000","last_active":"2023-05-16T07:17:19.151494+0000","last_peered":"2023-05-16T07:17:19.151494+0000","last_clean":"2023-05-16T07:17:19.151494+0000","last_became_active":"2023-05-16T07:17:19.149841+0000","last_became_peered":"2023-05-16T07:17:19.149841+0000","last_unstale":"2023-05-16T07:17:19.151494+0000","last_undegraded":"2023-05-16T07:17:19.151494+0000","last_fullsized":"2023-05-16T07:17:19.151494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.126669+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:17:19.126669+0000","last_peered":"2023-05-16T07:17:19.126669+0000","last_clean":"2023-05-16T07:17:19.126669+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:17:19.126669+0000","last_undegraded":"2023-05-16T07:17:19.126669+0000","last_fullsized":"2023-05-16T07:17:19.126669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.517317+0000","last_change":"2023-05-16T07:17:19.517317+0000","last_active":"2023-05-16T07:17:19.517317+0000","last_peered":"2023-05-16T07:17:19.517317+0000","last_clean":"2023-05-16T07:17:19.517317+0000","last_became_active":"2023-05-16T07:17:19.516610+0000","last_became_peered":"2023-05-16T07:17:19.516610+0000","last_unstale":"2023-05-16T07:17:19.517317+0000","last_undegraded":"2023-05-16T07:17:19.517317+0000","last_fullsized":"2023-05-16T07:17:19.517317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:19.121584+0000","last_clean_scrub_stamp":"2023-05-16T07:17:19.121584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.147203+0000","last_change":"2023-05-16T07:17:19.142570+0000","last_active":"2023-05-16T07:17:19.147203+0000","last_peered":"2023-05-16T07:17:19.147203+0000","last_clean":"2023-05-16T07:17:19.147203+0000","last_became_active":"2023-05-16T07:17:19.142285+0000","last_became_peered":"2023-05-16T07:17:19.142285+0000","last_unstale":"2023-05-16T07:17:19.147203+0000","last_undegraded":"2023-05-16T07:17:19.147203+0000","last_fullsized":"2023-05-16T07:17:19.147203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:18.119627+0000","last_clean_scrub_stamp":"2023-05-16T07:17:18.119627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-05-16T07:17:19.129769+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:17:19.129769+0000","last_peered":"2023-05-16T07:17:19.129769+0000","last_clean":"2023-05-16T07:17:19.129769+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:17:19.129769+0000","last_undegraded":"2023-05-16T07:17:19.129769+0000","last_fullsized":"2023-05-16T07:17:19.129769+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17068,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":16128,"kb_avail":94354772,"statfs":{"total":96636764160,"available":96619286528,"internally_reserved":0,"allocated":954368,"data_stored":479471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16515072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.439,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.751,"5min":0.81399999999999995,"15min":0.84199999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.47399999999999998,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":0.873},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.752,"5min":0.876,"15min":0.876},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.48599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63}]},{"osd":4,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66600000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.752,"15min":0.81999999999999995},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68500000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.40899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.999},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.95099999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16328,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":15424,"kb_avail":94355512,"statfs":{"total":96636764160,"available":96620044288,"internally_reserved":0,"allocated":917504,"data_stored":463429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15794176},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.26400000000000001},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.47799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77700000000000002,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.47399999999999998,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.37},{"interface":"front","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.38300000000000001,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.67900000000000005}]},{"osd":2,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.51100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.50900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.86299999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.497},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.32400000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61699999999999999,"5min":0.749,"15min":0.89400000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.628,"5min":4.048,"15min":4.048},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61499999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.311},"max":{"1min":0.63700000000000001,"5min":0.74399999999999999,"15min":0.92300000000000004},"last":0.749}]},{"osd":7,"last update":"Tue May 16 07:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67400000000000004,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.69299999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16456,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94355384,"statfs":{"total":96636764160,"available":96619913216,"internally_reserved":0,"allocated":917504,"data_stored":463429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.53300000000000003,"5min":0.53400000000000003,"15min":0.69899999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.34799999999999998}]},{"osd":2,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.375,"15min":0.376},"min":{"1min":0.27300000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":0.59699999999999998,"15min":0.65100000000000002},"last":0.313},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.40100000000000002},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.33600000000000002}]},{"osd":3,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42499999999999999},"min":{"1min":0.31900000000000001,"5min":0.29399999999999998,"15min":0.26300000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.55400000000000005,"5min":0.622,"15min":0.64000000000000001},"last":0.251}]},{"osd":4,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.622,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.623,"15min":0.622},"min":{"1min":0.435,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.51300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85299999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.47399999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.087},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.41699999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":1.204},"last":0.53400000000000003}]},{"osd":7,"last update":"Tue May 16 07:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.53100000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11616,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":10688,"kb_avail":94360224,"statfs":{"total":96636764160,"available":96624869376,"internally_reserved":0,"allocated":942080,"data_stored":478455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10944512},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34899999999999998,"15min":0.35699999999999998},"min":{"1min":0.246,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.60199999999999998},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.224},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.316}]},{"osd":2,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.36399999999999999,"15min":0.373},"min":{"1min":0.28899999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.40200000000000002},"min":{"1min":0.32100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.35599999999999998}]},{"osd":3,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.40500000000000003},"min":{"1min":0.30299999999999999,"5min":0.253,"15min":0.24199999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41599999999999998},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.64400000000000002},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.621,"15min":0.63500000000000001},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.117},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.55000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.2170000000000001},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45900000000000002,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.86699999999999999,"5min":0.90000000000000002,"15min":1.157},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.97199999999999998,"15min":1.224},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.177},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue May 16 07:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.93700000000000006,"15min":1.196},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.56699999999999995,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.90600000000000003,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.442}]}]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17448,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":16512,"kb_avail":94354392,"statfs":{"total":96636764160,"available":96618897408,"internally_reserved":0,"allocated":950272,"data_stored":479079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16908288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36199999999999999},"min":{"1min":0.23899999999999999,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.54100000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.27400000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.28799999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.40899999999999997},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.23499999999999999},"max":{"1min":0.54000000000000004,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.435},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.434},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.626,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.377}]},{"osd":3,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.248,"15min":0.22500000000000001},"max":{"1min":0.504,"5min":0.64000000000000001,"15min":0.64700000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44700000000000001,"15min":0.438},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64900000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41699999999999998}]},{"osd":4,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.41999999999999998,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":1.274},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.628},"min":{"1min":0.47099999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80400000000000005,"5min":0.92900000000000005,"15min":0.94899999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.89000000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.37},"max":{"1min":0.872,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67900000000000005,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80800000000000005,"5min":1.47,"15min":1.47},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue May 16 07:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20372,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":19456,"kb_avail":94351468,"statfs":{"total":96636764160,"available":96615903232,"internally_reserved":0,"allocated":929792,"data_stored":470692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19922944},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.437},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.29799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.40899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42199999999999999,"15min":0.41499999999999998},"min":{"1min":0.33100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.33800000000000002,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.626,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.70199999999999996}]},{"osd":5,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77500000000000002,"5min":0.93100000000000005,"15min":1.0489999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83799999999999997,"5min":0.93799999999999994,"15min":1.0409999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.504,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.97399999999999998,"15min":1.0780000000000001},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":1.0049999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.44,"15min":0.40100000000000002},"max":{"1min":0.75800000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":1.02},"last":0.74299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16680,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":15744,"kb_avail":94355160,"statfs":{"total":96636764160,"available":96619683840,"internally_reserved":0,"allocated":950272,"data_stored":479099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16121856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.83199999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.81200000000000006},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.33900000000000002},"max":{"1min":0.73699999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.505},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.999},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.84599999999999997,"15min":0.97999999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.93100000000000005},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.32200000000000001,"5min":0.314,"15min":0.308},"max":{"1min":0.69899999999999995,"5min":0.79500000000000004,"15min":1.0640000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52300000000000002,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.79200000000000004,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":1.2689999999999999},"last":0.43099999999999999}]},{"osd":7,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":1.48,"15min":1.48},"last":0.375},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":0.82599999999999996,"15min":1.046},"last":0.46300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575028,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19784,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":18880,"kb_avail":94352056,"statfs":{"total":96636764160,"available":96616505344,"internally_reserved":0,"allocated":917504,"data_stored":463429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19333120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60899999999999999,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.69599999999999995,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59199999999999997,"15min":0.56299999999999994},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.314},"max":{"1min":0.747,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.44,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":0.79200000000000004,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.47499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71999999999999997,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.38700000000000001}]},{"osd":2,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":2.278,"15min":2.278},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.62,"15min":0.58699999999999997},"min":{"1min":0.48799999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77500000000000002,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.51100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75800000000000001,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.435},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.51800000000000002,"5min":0.435,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58199999999999996,"5min":1.333,"15min":1.333},"last":0.499},{"interface":"front","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.309},"max":{"1min":0.78400000000000003,"5min":2.081,"15min":2.081},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.69499999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.66200000000000003,"5min":1.024,"15min":1.024},"last":0.53600000000000003}]},{"osd":7,"last update":"Tue May 16 07:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81699999999999995,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.76700000000000002,"5min":0.86599999999999999,"15min":0.878},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:17:22.829 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:17:22.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:17:22.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.971+0000 7fe527f5c700 1 Processor -- start 2023-05-16T07:17:22.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.971+0000 7fe527f5c700 1 -- start start 2023-05-16T07:17:22.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe527f5c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 0x7fe5200aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe527f5c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200c94d0 con 0x7fe520154470 2023-05-16T07:17:22.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe527f5c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200cb7e0 con 0x7fe520068160 2023-05-16T07:17:22.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe527f5c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c88c0 con 0x7fe520154090 2023-05-16T07:17:22.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe5254f7700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe520154470 0x7fe5200a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56150/0 (socket says 172.21.15.92:56150) 2023-05-16T07:17:22.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe5254f7700 1 -- 172.21.15.92:0/2101247414 learned_addr learned my addr 172.21.15.92:0/2101247414 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:22.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.972+0000 7fe525cf8700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 0x7fe5200aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315675865 0 0) 0x7fe5200c94d0 con 0x7fe520154470 2023-05-16T07:17:22.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5080033f0 con 0x7fe520154470 2023-05-16T07:17:22.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066657008 0 0) 0x7fe5080033f0 con 0x7fe520154470 2023-05-16T07:17:22.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 msgr2=0x7fe5200aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 0x7fe5200aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 >> v1:172.21.15.158:6789/0 conn(0x7fe520068160 legacy=0x7fe520068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200c8ba0 con 0x7fe520154470 2023-05-16T07:17:22.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fe514001470 con 0x7fe520154470 2023-05-16T07:17:22.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 >> v1:172.21.15.92:6789/0 conn(0x7fe520154470 legacy=0x7fe5200a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe508003910 0x7fe508003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 0x7fe508004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508004b80 0x7fe508005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c88c0 con 0x7fe508003910 2023-05-16T07:17:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50800b910 con 0x7fe508004b80 2023-05-16T07:17:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50800bad0 con 0x7fe520154090 2023-05-16T07:17:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe508003910 0x7fe508003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe5264f9700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508004b80 0x7fe508005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe5254f7700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 0x7fe508004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 -- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 msgr2=0x7fe508004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe520154090 0x7fe508004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 -- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508004b80 msgr2=0x7fe508005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:22.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508004b80 0x7fe508005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 -- 172.21.15.92:0/2101247414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080033f0 con 0x7fe508003910 2023-05-16T07:17:22.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe525cf8700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe508003910 0x7fe508003d30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fe51000e1c0 tx=0x7fe51000b040).ready entity=mon.0 client_cookie=39275b1928ff1fcc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.973+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510009ac0 con 0x7fe508003910 2023-05-16T07:17:22.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51000f040 con 0x7fe508003910 2023-05-16T07:17:22.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe524cf6700 1 -- 172.21.15.92:0/2101247414 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000eca0 con 0x7fe508003910 2023-05-16T07:17:22.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe527f5c700 1 -- 172.21.15.92:0/2101247414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe508003910 msgr2=0x7fe508003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe527f5c700 1 --2- 172.21.15.92:0/2101247414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe508003910 0x7fe508003d30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fe51000e1c0 tx=0x7fe51000b040).stop 2023-05-16T07:17:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe5264f9700 1 -- 172.21.15.92:0/2101247414 reap_dead start 2023-05-16T07:17:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe527f5c700 1 -- 172.21.15.92:0/2101247414 shutdown_connections 2023-05-16T07:17:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe527f5c700 1 -- 172.21.15.92:0/2101247414 >> 172.21.15.92:0/2101247414 conn(0x7fe520067920 msgr2=0x7fe5200cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe527f5c700 1 -- 172.21.15.92:0/2101247414 shutdown_connections 2023-05-16T07:17:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.974+0000 7fe527f5c700 1 -- 172.21.15.92:0/2101247414 wait complete. 2023-05-16T07:17:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.975+0000 7fe527f5c700 1 Processor -- start 2023-05-16T07:17:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.975+0000 7fe527f5c700 1 -- start start 2023-05-16T07:17:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe527f5c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508003910 0x7fe52013c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe527f5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 0x7fe5200c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe527f5c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5200adbe0 0x7fe5200ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe527f5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52013c880 con 0x7fe5200bfb90 2023-05-16T07:17:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe527f5c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c11f0 con 0x7fe508003910 2023-05-16T07:17:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe527f5c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200ae550 con 0x7fe5200adbe0 2023-05-16T07:17:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 0x7fe5200c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 0x7fe5200c0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45414/0 (socket says 172.21.15.92:45414) 2023-05-16T07:17:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 -- 172.21.15.92:0/1574789358 learned_addr learned my addr 172.21.15.92:0/1574789358 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe525cf8700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508003910 0x7fe52013c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5264f9700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5200adbe0 0x7fe5200ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 -- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5200adbe0 msgr2=0x7fe5200ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5200adbe0 0x7fe5200ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 -- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508003910 msgr2=0x7fe52013c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508003910 0x7fe52013c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 -- 172.21.15.92:0/1574789358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200c05e0 con 0x7fe5200bfb90 2023-05-16T07:17:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.976+0000 7fe5254f7700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 0x7fe5200c0010 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fe51400e860 tx=0x7fe514000f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.977+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514010520 con 0x7fe5200bfb90 2023-05-16T07:17:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.977+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514010680 con 0x7fe5200bfb90 2023-05-16T07:17:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.977+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514010a00 con 0x7fe5200bfb90 2023-05-16T07:17:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.977+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5200c08c0 con 0x7fe5200bfb90 2023-05-16T07:17:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.977+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5200c0e80 con 0x7fe5200bfb90 2023-05-16T07:17:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.978+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200c13a0 con 0x7fe5200bfb90 2023-05-16T07:17:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.978+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe514010d70 con 0x7fe5200bfb90 2023-05-16T07:17:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.978+0000 7fe51dffb700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe50c05ed90 0x7fe50c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.978+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fe514079e30 con 0x7fe5200bfb90 2023-05-16T07:17:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.978+0000 7fe525cf8700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe50c05ed90 0x7fe50c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.979+0000 7fe525cf8700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe50c05ed90 0x7fe50c061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe5200ad940 tx=0x7fe51001a000).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:22.982+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe5200c13a0 con 0x7fe5200bfb90 2023-05-16T07:17:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.205+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fe520066730 con 0x7fe5200bfb90 2023-05-16T07:17:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.209+0000 7fe51dffb700 1 -- 172.21.15.92:0/1574789358 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe51404ebb0 con 0x7fe5200bfb90 2023-05-16T07:17:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe50c05ed90 msgr2=0x7fe50c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe50c05ed90 0x7fe50c061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe5200ad940 tx=0x7fe51001a000).stop 2023-05-16T07:17:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 msgr2=0x7fe5200c0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 0x7fe5200c0010 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fe51400e860 tx=0x7fe514000f60).stop 2023-05-16T07:17:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 shutdown_connections 2023-05-16T07:17:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe50c05ed90 0x7fe50c061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe5200adbe0 0x7fe5200ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508003910 0x7fe52013c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 --2- 172.21.15.92:0/1574789358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5200bfb90 0x7fe5200c0010 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 >> 172.21.15.92:0/1574789358 conn(0x7fe520067920 msgr2=0x7fe5200c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 shutdown_connections 2023-05-16T07:17:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.211+0000 7fe527f5c700 1 -- 172.21.15.92:0/1574789358 wait complete. 2023-05-16T07:17:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:17:23.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:23.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.333+0000 7fe9cc210700 1 Processor -- start 2023-05-16T07:17:23.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.333+0000 7fe9cc210700 1 -- start start 2023-05-16T07:17:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9cc210700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe9c4154550 0x7fe9c40696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9cc210700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c40c0660 con 0x7fe9c4154170 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9cc210700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c4069fc0 con 0x7fe9c4069b90 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9cc210700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c40bfd30 con 0x7fe9c4154550 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9c9fac700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe9c4154170 0x7fe9c40c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56168/0 (socket says 172.21.15.92:56168) 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9c9fac700 1 -- 172.21.15.92:0/754423320 learned_addr learned my addr 172.21.15.92:0/754423320 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/754423320 >> v2:172.21.15.158:3300/0 conn(0x7fe9c4154550 0x7fe9c40696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.334+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045171416 0 0) 0x7fe9c40c0660 con 0x7fe9c4154170 2023-05-16T07:17:23.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ac0033f0 con 0x7fe9c4154170 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436162673 0 0) 0x7fe9ac0033f0 con 0x7fe9c4154170 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 >> v2:172.21.15.158:3300/0 conn(0x7fe9c4154550 msgr2=0x7fe9c40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 --2- 172.21.15.92:0/754423320 >> v2:172.21.15.158:3300/0 conn(0x7fe9c4154550 0x7fe9c40696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 >> v1:172.21.15.158:6789/0 conn(0x7fe9c4069b90 legacy=0x7fe9c40c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40c0010 con 0x7fe9c4154170 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fe9b40017f0 con 0x7fe9c4154170 2023-05-16T07:17:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 >> v1:172.21.15.92:6789/0 conn(0x7fe9c4154170 legacy=0x7fe9c40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9ac003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9ac004690 0x7fe9ac004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9ac005000 0x7fe9ac00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40bfd30 con 0x7fe9ac0038e0 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac00bf30 con 0x7fe9ac004690 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac00c0f0 con 0x7fe9ac005000 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9ac003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9ac004690 0x7fe9ac004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9ca7ad700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9ac005000 0x7fe9ac00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 -- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9ac005000 msgr2=0x7fe9ac00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9ac005000 0x7fe9ac00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 -- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9ac004690 msgr2=0x7fe9ac004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9ac004690 0x7fe9ac004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9ca7ad700 1 -- 172.21.15.92:0/754423320 reap_dead start 2023-05-16T07:17:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 -- 172.21.15.92:0/754423320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0033f0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9ac004690 0x7fe9ac004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9ac003cc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fe9b400c5f0 tx=0x7fe9b4001d90).ready entity=mon.0 client_cookie=b9b079defb63231f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4001fc0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4005b20 con 0x7fe9ac0038e0 2023-05-16T07:17:23.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.335+0000 7fe9c8faa700 1 -- 172.21.15.92:0/754423320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4005ea0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 -- 172.21.15.92:0/754423320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 msgr2=0x7fe9ac003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9ac003cc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fe9b400c5f0 tx=0x7fe9b4001d90).stop 2023-05-16T07:17:23.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 -- 172.21.15.92:0/754423320 shutdown_connections 2023-05-16T07:17:23.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 --2- 172.21.15.92:0/754423320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9ac003cc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 -- 172.21.15.92:0/754423320 >> 172.21.15.92:0/754423320 conn(0x7fe9c40cb700 msgr2=0x7fe9c40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:23.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 -- 172.21.15.92:0/754423320 shutdown_connections 2023-05-16T07:17:23.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.336+0000 7fe9cc210700 1 -- 172.21.15.92:0/754423320 wait complete. 2023-05-16T07:17:23.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.337+0000 7fe9cc210700 1 Processor -- start 2023-05-16T07:17:23.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.337+0000 7fe9cc210700 1 -- start start 2023-05-16T07:17:23.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9cc210700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9c4164240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9cc210700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9c414f210 0x7fe9c40af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9cc210700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9c40afb50 0x7fe9c4153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9cc210700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40c3000 con 0x7fe9ac0038e0 2023-05-16T07:17:23.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9cc210700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40affd0 con 0x7fe9c414f210 2023-05-16T07:17:23.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9cc210700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40c3210 con 0x7fe9c40afb50 2023-05-16T07:17:23.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9c4164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9c4164240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45432/0 (socket says 172.21.15.92:45432) 2023-05-16T07:17:23.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 -- 172.21.15.92:0/2580356537 learned_addr learned my addr 172.21.15.92:0/2580356537 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:23.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9ca7ad700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9c40afb50 0x7fe9c4153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9c414f210 0x7fe9c40af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 -- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9c40afb50 msgr2=0x7fe9c4153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9c40afb50 0x7fe9c4153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 -- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9c414f210 msgr2=0x7fe9c40af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9c414f210 0x7fe9c40af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 -- 172.21.15.92:0/2580356537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40c34f0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9c414f210 0x7fe9c40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:23.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.338+0000 7fe9c9fac700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9c4164240 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fe9b400cbe0 tx=0x7fe9b4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:23.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.339+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4001a90 con 0x7fe9ac0038e0 2023-05-16T07:17:23.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.339+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4000f40 con 0x7fe9ac0038e0 2023-05-16T07:17:23.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.339+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c40c37d0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.339+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4004c80 con 0x7fe9ac0038e0 2023-05-16T07:17:23.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.339+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c40c3d90 con 0x7fe9ac0038e0 2023-05-16T07:17:23.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.340+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c40666c0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.340+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe9b4028030 con 0x7fe9ac0038e0 2023-05-16T07:17:23.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.340+0000 7fe9c1ffb700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe9b005ed90 0x7fe9b0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:23.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.340+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fe9b407b390 con 0x7fe9ac0038e0 2023-05-16T07:17:23.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.341+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe9b005ed90 0x7fe9b0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:23.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.341+0000 7fe9c97ab700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe9b005ed90 0x7fe9b0061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe9b8009270 tx=0x7fe9b8008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:23.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.345+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe9b404c6e0 con 0x7fe9ac0038e0 2023-05-16T07:17:23.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.550+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9c406c350 con 0x7fe9ac0038e0 2023-05-16T07:17:23.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.551+0000 7fe9c1ffb700 1 -- 172.21.15.92:0/2580356537 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fe9b404fd00 con 0x7fe9ac0038e0 2023-05-16T07:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:19.493924+0000","modified":"2023-05-16T07:17:21.499636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1922547921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1922547921},{"type":"v1","addr":"172.21.15.92:6835","nonce":1922547921}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-05-16T07:17:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe9b005ed90 msgr2=0x7fe9b0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe9b005ed90 0x7fe9b0061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe9b8009270 tx=0x7fe9b8008040).stop 2023-05-16T07:17:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 msgr2=0x7fe9c4164240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9c4164240 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fe9b400cbe0 tx=0x7fe9b4015000).stop 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 shutdown_connections 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe9b005ed90 0x7fe9b0061250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe9c40afb50 0x7fe9c4153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ac0038e0 0x7fe9c4164240 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 --2- 172.21.15.92:0/2580356537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe9c414f210 0x7fe9c40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.552+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 >> 172.21.15.92:0/2580356537 conn(0x7fe9c40cb700 msgr2=0x7fe9c40c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:23.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.553+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 shutdown_connections 2023-05-16T07:17:23.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:23.553+0000 7fe9cc210700 1 -- 172.21.15.92:0/2580356537 wait complete. 2023-05-16T07:17:23.558 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-05-16T07:17:23.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:17:19.493924+0000', 'modified': '2023-05-16T07:17:21.499636+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6165}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6165': {'gid': 6165, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/1922547921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6833', 'nonce': 1922547921}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1922547921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:17:23.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:17:23.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-16T07:17:24.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.081+0000 7f63ab3bd700 1 Processor -- start 2023-05-16T07:17:24.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.081+0000 7f63ab3bd700 1 -- start start 2023-05-16T07:17:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63ab3bd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63a4154170 0x7f63a40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63ab3bd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f63a4154550 0x7f63a40c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63ab3bd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a40c0660 con 0x7f63a40696a0 2023-05-16T07:17:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63ab3bd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a40c1b00 con 0x7f63a4154170 2023-05-16T07:17:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63ab3bd700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a40bfd30 con 0x7f63a4154550 2023-05-16T07:17:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63a4154170 0x7f63a40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63a4154170 0x7f63a40c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45440/0 (socket says 172.21.15.92:45440) 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 -- 172.21.15.92:0/3135459497 learned_addr learned my addr 172.21.15.92:0/3135459497 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a8958700 1 --2- 172.21.15.92:0/3135459497 >> v2:172.21.15.158:3300/0 conn(0x7f63a4154550 0x7f63a40c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 -- 172.21.15.92:0/3135459497 >> v2:172.21.15.158:3300/0 conn(0x7f63a4154550 msgr2=0x7f63a40c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 --2- 172.21.15.92:0/3135459497 >> v2:172.21.15.158:3300/0 conn(0x7f63a4154550 0x7f63a40c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 -- 172.21.15.92:0/3135459497 >> v1:172.21.15.158:6789/0 conn(0x7f63a40696a0 legacy=0x7f63a4157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.082+0000 7f63a9159700 1 -- 172.21.15.92:0/3135459497 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40c0010 con 0x7f63a4154170 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63a8958700 1 --2- 172.21.15.92:0/3135459497 >> v2:172.21.15.158:3300/0 conn(0x7f63a4154550 0x7f63a40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63a9159700 1 --2- 172.21.15.92:0/3135459497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a4154170 0x7f63a40c5c70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f63a40844e0 tx=0x7f6394009840).ready entity=mon.0 client_cookie=88078c87af1a4cbe server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63a3fff700 1 -- 172.21.15.92:0/3135459497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e070 con 0x7f63a4154170 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63a3fff700 1 -- 172.21.15.92:0/3135459497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639400ba10 con 0x7f63a4154170 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63a3fff700 1 -- 172.21.15.92:0/3135459497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400bd90 con 0x7f63a4154170 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 -- 172.21.15.92:0/3135459497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a4154170 msgr2=0x7f63a40c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/3135459497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a4154170 0x7f63a40c5c70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f63a40844e0 tx=0x7f6394009840).stop 2023-05-16T07:17:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 -- 172.21.15.92:0/3135459497 shutdown_connections 2023-05-16T07:17:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/3135459497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a4154170 0x7f63a40c5c70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/3135459497 >> v2:172.21.15.158:3300/0 conn(0x7f63a4154550 0x7f63a40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 -- 172.21.15.92:0/3135459497 >> 172.21.15.92:0/3135459497 conn(0x7f63a40cb700 msgr2=0x7f63a40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 -- 172.21.15.92:0/3135459497 shutdown_connections 2023-05-16T07:17:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.083+0000 7f63ab3bd700 1 -- 172.21.15.92:0/3135459497 wait complete. 2023-05-16T07:17:24.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.084+0000 7f63ab3bd700 1 Processor -- start 2023-05-16T07:17:24.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.084+0000 7f63ab3bd700 1 -- start start 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63ab3bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 0x7f63a4154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63ab3bd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63a4154170 0x7f63a413d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63ab3bd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63a4154550 0x7f63a40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63ab3bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4157c60 con 0x7f63a40696a0 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63ab3bd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4157090 con 0x7f63a4154170 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63ab3bd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4157330 con 0x7f63a4154550 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63a9159700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 0x7f63a4154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63a9159700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 0x7f63a4154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45444/0 (socket says 172.21.15.92:45444) 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63a9159700 1 -- 172.21.15.92:0/2466392016 learned_addr learned my addr 172.21.15.92:0/2466392016 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.085+0000 7f63a8958700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63a4154170 0x7f63a413d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a995a700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63a4154550 0x7f63a40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a9159700 1 -- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63a4154550 msgr2=0x7f63a40ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a9159700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63a4154550 0x7f63a40ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a9159700 1 -- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63a4154170 msgr2=0x7f63a413d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a9159700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63a4154170 0x7f63a413d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a9159700 1 -- 172.21.15.92:0/2466392016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009ae0 con 0x7f63a40696a0 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a995a700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63a4154550 0x7f63a40ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a9159700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 0x7f63a4154960 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f6394009930 tx=0x7f6394009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a17fa700 1 -- 172.21.15.92:0/2466392016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394013d10 con 0x7f63a40696a0 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a17fa700 1 -- 172.21.15.92:0/2466392016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394013e70 con 0x7f63a40696a0 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4157610 con 0x7f63a40696a0 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63a17fa700 1 -- 172.21.15.92:0/2466392016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639401e6a0 con 0x7f63a40696a0 2023-05-16T07:17:24.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.086+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a4154e50 con 0x7f63a40696a0 2023-05-16T07:17:24.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.087+0000 7f63a17fa700 1 -- 172.21.15.92:0/2466392016 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6394025030 con 0x7f63a40696a0 2023-05-16T07:17:24.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.088+0000 7f63a17fa700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f639005ed90 0x7f6390061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.088+0000 7f63a17fa700 1 -- 172.21.15.92:0/2466392016 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f639407b2f0 con 0x7f63a40696a0 2023-05-16T07:17:24.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.088+0000 7f63a8958700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f639005ed90 0x7f6390061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.088+0000 7f63a8958700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f639005ed90 0x7f6390061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f63a40ca020 tx=0x7f6398008040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 Processor -- start 2023-05-16T07:17:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 -- start start 2023-05-16T07:17:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e4a480 0x55a581e45940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 0x55a581dfb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581dc65e0 con 0x55a581e01300 2023-05-16T07:17:24.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d3d8a0 con 0x55a581e4a480 2023-05-16T07:17:24.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.090+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a581d52900 con 0x55a581e45560 2023-05-16T07:17:24.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 0x55a581dfb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 0x55a581dfb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45456/0 (socket says 172.21.15.92:45456) 2023-05-16T07:17:24.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 -- 172.21.15.92:0/158810096 learned_addr learned my addr 172.21.15.92:0/158810096 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a9159700 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a8958700 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e4a480 0x55a581e45940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 -- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 msgr2=0x55a581e28c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e28c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 -- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e4a480 msgr2=0x55a581e45940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e4a480 0x55a581e45940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 -- 172.21.15.92:0/158810096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a581dc6450 con 0x55a581e01300 2023-05-16T07:17:24.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a9159700 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e28c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:24.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f63a995a700 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 0x55a581dfb710 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f639c0025d0 tx=0x7f639c009040).ready entity=mon.0 client_cookie=231a7a66b24a7c6f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f638a7fc700 1 -- 172.21.15.92:0/158810096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c010520 con 0x55a581e01300 2023-05-16T07:17:24.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f638a7fc700 1 -- 172.21.15.92:0/158810096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c010680 con 0x55a581e01300 2023-05-16T07:17:24.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.091+0000 7f638a7fc700 1 -- 172.21.15.92:0/158810096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c010a00 con 0x55a581e01300 2023-05-16T07:17:24.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/158810096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 msgr2=0x55a581dfb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 0x55a581dfb710 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f639c0025d0 tx=0x7f639c009040).stop 2023-05-16T07:17:24.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/158810096 shutdown_connections 2023-05-16T07:17:24.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e28c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e01300 0x55a581dfb710 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/158810096 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e4a480 0x55a581e45940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/158810096 >> 172.21.15.92:0/158810096 conn(0x55a581bbde40 msgr2=0x55a581e499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/158810096 shutdown_connections 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/158810096 wait complete. 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.092+0000 7f63bcd4fb80 1 -- start start 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 Processor -- start 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e23cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581dfb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581dfb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581d19ed0 con 0x55a581e4a480 2023-05-16T07:17:24.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581dfbb50 con 0x55a581e01300 2023-05-16T07:17:24.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a581e2c530 con 0x55a581e45560 2023-05-16T07:17:24.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63bcd4fb80 10 client.0 ll_register_callbacks cb 0x55a581dc01f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:17:24.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63a995a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581dfb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63a995a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581dfb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45462/0 (socket says 172.21.15.92:45462) 2023-05-16T07:17:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63a995a700 1 -- 172.21.15.92:0/2528510926 learned_addr learned my addr 172.21.15.92:0/2528510926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63a9159700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e23cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.093+0000 7f63a8958700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581dfb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63a995a700 1 -- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 msgr2=0x55a581dfb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63a995a700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581dfb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63a995a700 1 -- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 msgr2=0x55a581e23cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63a995a700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e23cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63a995a700 1 -- 172.21.15.92:0/2528510926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c00f040 con 0x55a581e4a480 2023-05-16T07:17:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63a995a700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581dfb660 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f639c007bd0 tx=0x7f639c009040).ready entity=mon.0 client_cookie=9dfd30863b0747fb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63837fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:17:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63837fe700 1 -- 172.21.15.92:0/2528510926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c007040 con 0x55a581e4a480 2023-05-16T07:17:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63837fe700 1 -- 172.21.15.92:0/2528510926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f639c0071a0 con 0x55a581e4a480 2023-05-16T07:17:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a581df4460 con 0x55a581e4a480 2023-05-16T07:17:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63837fe700 1 -- 172.21.15.92:0/2528510926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f639c007570 con 0x55a581e4a480 2023-05-16T07:17:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 10 client.6318 fetch_fsmap learned FSMap version 24 2023-05-16T07:17:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a581bf5310 con 0x55a581e4a480 2023-05-16T07:17:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63837fe700 1 -- 172.21.15.92:0/2528510926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a581df4460 con 0x55a581e4a480 2023-05-16T07:17:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63837fe700 1 -- 172.21.15.92:0/2528510926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f639c0066a0 con 0x55a581e4a480 2023-05-16T07:17:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 10 client.6318 fetch_fsmap finished waiting for FSMap version 24 2023-05-16T07:17:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 10 client.6318 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:17:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 20 client.6318 populate_metadata read hostname 'smithi092' 2023-05-16T07:17:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e58030 0x55a581e58410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 4 client.6318 mds_command: new command op to 6165 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:17:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.094+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 --> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a581bb9ca0 con 0x55a581e58030 2023-05-16T07:17:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63a9159700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e58030 0x55a581e58410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63837fe700 10 client.6318 ms_handle_connect on v2:172.21.15.92:6833/1922547921 2023-05-16T07:17:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63a9159700 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e58030 0x55a581e58410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=331fd05c8ed247fa server_cookie=59ed159ed099ffcd in_seq=0 out_seq=0 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63837fe700 1 -- 172.21.15.92:0/2528510926 <== mds.0 v2:172.21.15.92:6833/1922547921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55a581bb9ca0 con 0x55a581e58030 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63837fe700 10 client.6318 handle_command_reply: tid=0 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 client.6318 shutdown 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 msgr2=0x55a581dfb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581dfb660 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f639c007bd0 tx=0x7f639c009040).stop 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 shutdown_connections 2023-05-16T07:17:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e58030 0x55a581e58410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581dfb660 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e23cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63837fe700 0 client.6318 ms_handle_reset on v2:172.21.15.92:6833/1922547921 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/2528510926 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581dfb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.095+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 >> 172.21.15.92:0/2528510926 conn(0x55a581bbde40 msgr2=0x55a581e462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.096+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 shutdown_connections 2023-05-16T07:17:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.096+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/2528510926 wait complete. 2023-05-16T07:17:24.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.096+0000 7f63bcd4fb80 20 client.6318 trim_cache size 0 max 16384 2023-05-16T07:17:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 Processor -- start 2023-05-16T07:17:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 -- start start 2023-05-16T07:17:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e28f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 0x55a581e528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e4a480 0x55a581e40480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581e2eed0 con 0x55a581e45560 2023-05-16T07:17:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581df4120 con 0x55a581e01300 2023-05-16T07:17:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.112+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a581e297d0 con 0x55a581e4a480 2023-05-16T07:17:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 0x55a581e528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 0x55a581e528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45468/0 (socket says 172.21.15.92:45468) 2023-05-16T07:17:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 -- 172.21.15.92:0/720303187 learned_addr learned my addr 172.21.15.92:0/720303187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a9159700 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e28f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a995a700 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e4a480 0x55a581e40480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 -- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e4a480 msgr2=0x55a581e40480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e4a480 0x55a581e40480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 -- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 msgr2=0x55a581e28f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e28f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 -- 172.21.15.92:0/720303187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a581e2f430 con 0x55a581e45560 2023-05-16T07:17:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a995a700 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e4a480 0x55a581e40480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f63a8958700 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 0x55a581e528a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f639800fbf0 tx=0x7f6398015000).ready entity=mon.0 client_cookie=4e30543bd5146425 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f638affd700 1 -- 172.21.15.92:0/720303187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398013880 con 0x55a581e45560 2023-05-16T07:17:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f638affd700 1 -- 172.21.15.92:0/720303187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398023070 con 0x55a581e45560 2023-05-16T07:17:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.113+0000 7f638affd700 1 -- 172.21.15.92:0/720303187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63980139e0 con 0x55a581e45560 2023-05-16T07:17:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/720303187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 msgr2=0x55a581e528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 0x55a581e528a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f639800fbf0 tx=0x7f6398015000).stop 2023-05-16T07:17:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/720303187 shutdown_connections 2023-05-16T07:17:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e4a480 0x55a581e40480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581e28f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/720303187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e45560 0x55a581e528a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/720303187 >> 172.21.15.92:0/720303187 conn(0x55a581bbde40 msgr2=0x55a581df7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/720303187 shutdown_connections 2023-05-16T07:17:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/720303187 wait complete. 2023-05-16T07:17:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.114+0000 7f63bcd4fb80 1 -- start start 2023-05-16T07:17:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 Processor -- start 2023-05-16T07:17:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581ea8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581e467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581e46c90 con 0x55a581e4a480 2023-05-16T07:17:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a581e27890 con 0x55a581e01300 2023-05-16T07:17:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a581dfa740 con 0x55a581e45560 2023-05-16T07:17:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63bcd4fb80 10 client.0 ll_register_callbacks cb 0x55a581dc01f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:17:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63a995a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581e467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63a995a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581e467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45476/0 (socket says 172.21.15.92:45476) 2023-05-16T07:17:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63a995a700 1 -- 172.21.15.92:0/1961119993 learned_addr learned my addr 172.21.15.92:0/1961119993 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63a9159700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581ea8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.115+0000 7f63a8958700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a995a700 1 -- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 msgr2=0x55a581e462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a995a700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a995a700 1 -- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 msgr2=0x55a581ea8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a995a700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581ea8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a995a700 1 -- 172.21.15.92:0/1961119993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639800f980 con 0x55a581e4a480 2023-05-16T07:17:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a995a700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581e467a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x55a581e00f90 tx=0x7f639c01f000).ready entity=mon.0 client_cookie=cbfd96523b3ccb6f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63837fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63837fe700 1 -- 172.21.15.92:0/1961119993 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c006bb0 con 0x55a581e4a480 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63837fe700 1 -- 172.21.15.92:0/1961119993 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f639c007570 con 0x55a581e4a480 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f639c010680 con 0x55a581e4a480 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63837fe700 1 -- 172.21.15.92:0/1961119993 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f639c000bc0 con 0x55a581e4a480 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 10 client.6324 fetch_fsmap learned FSMap version 24 2023-05-16T07:17:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a581d4b880 con 0x55a581e4a480 2023-05-16T07:17:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63837fe700 1 -- 172.21.15.92:0/1961119993 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c010680 con 0x55a581e4a480 2023-05-16T07:17:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63837fe700 1 -- 172.21.15.92:0/1961119993 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f639c000d20 con 0x55a581e4a480 2023-05-16T07:17:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 10 client.6324 fetch_fsmap finished waiting for FSMap version 24 2023-05-16T07:17:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 10 client.6324 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:17:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 20 client.6324 populate_metadata read hostname 'smithi092' 2023-05-16T07:17:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e26aa0 0x55a581e26e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 4 client.6324 mds_command: new command op to 6165 tid=0[{"prefix": "status"}] 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 --> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a581bf0ca0 con 0x55a581e26aa0 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.116+0000 7f63a9159700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e26aa0 0x55a581e26e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63837fe700 10 client.6324 ms_handle_connect on v2:172.21.15.92:6833/1922547921 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63a9159700 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e26aa0 0x55a581e26e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d0fb43ccfc59faf server_cookie=5e895c131dd3ffb5 in_seq=0 out_seq=0 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63837fe700 1 -- 172.21.15.92:0/1961119993 <== mds.0 v2:172.21.15.92:6833/1922547921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a581bf0ca0 con 0x55a581e26aa0 2023-05-16T07:17:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63837fe700 10 client.6324 handle_command_reply: tid=0 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 client.6324 shutdown 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 msgr2=0x55a581e467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581e467a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x55a581e00f90 tx=0x7f639c01f000).stop 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 shutdown_connections 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:6833/1922547921,v1:172.21.15.92:6835/1922547921] conn(0x55a581e26aa0 0x55a581e26e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a581e4a480 0x55a581e467a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55a581e01300 0x55a581ea8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 --2- 172.21.15.92:0/1961119993 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55a581e45560 0x55a581e462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63837fe700 0 client.6324 ms_handle_reset on v2:172.21.15.92:6833/1922547921 2023-05-16T07:17:24.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 >> 172.21.15.92:0/1961119993 conn(0x55a581bbde40 msgr2=0x55a581df7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 shutdown_connections 2023-05-16T07:17:24.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 1 -- 172.21.15.92:0/1961119993 wait complete. 2023-05-16T07:17:24.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.117+0000 7f63bcd4fb80 20 client.6324 trim_cache size 0 max 16384 2023-05-16T07:17:24.142 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:17:24.142 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:17:24.142 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:17:24.142 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6165, 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.6224328870000004, 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 24, 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 79, 2023-05-16T07:17:24.143 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 79, 2023-05-16T07:17:24.144 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.1630909290000009 2023-05-16T07:17:24.144 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:17:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f639005ed90 msgr2=0x7f6390061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f639005ed90 0x7f6390061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f63a40ca020 tx=0x7f6398008040).stop 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 msgr2=0x7f63a4154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 0x7f63a4154960 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f6394009930 tx=0x7f6394009840).stop 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 shutdown_connections 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f639005ed90 0x7f6390061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63a4154550 0x7f63a40ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63a40696a0 0x7f63a4154960 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 --2- 172.21.15.92:0/2466392016 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63a4154170 0x7f63a413d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 >> 172.21.15.92:0/2466392016 conn(0x7f63a40cb700 msgr2=0x7f63a40ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 shutdown_connections 2023-05-16T07:17:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.118+0000 7f63ab3bd700 1 -- 172.21.15.92:0/2466392016 wait complete. 2023-05-16T07:17:24.147 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:17:24.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 Processor -- start 2023-05-16T07:17:24.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 -- start start 2023-05-16T07:17:24.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f740c13b570 0x7f740c0be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c066780 0x7f740c0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0c3ba0 con 0x7f740c13b950 2023-05-16T07:17:24.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0bf0c0 con 0x7f740c13b570 2023-05-16T07:17:24.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.243+0000 7f74131de700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0bf3a0 con 0x7f740c066780 2023-05-16T07:17:24.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f7410f7a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f740c13b570 0x7f740c0be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f740bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f740c13b950 0x7f740c066070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56228/0 (socket says 172.21.15.92:56228) 2023-05-16T07:17:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f740bfff700 1 -- 172.21.15.92:0/4190991975 learned_addr learned my addr 172.21.15.92:0/4190991975 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f741177b700 1 --2- 172.21.15.92:0/4190991975 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c066780 0x7f740c0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f7410f7a700 1 -- 172.21.15.92:0/4190991975 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c066780 msgr2=0x7f740c0c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f7410f7a700 1 --2- 172.21.15.92:0/4190991975 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c066780 0x7f740c0c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f7410f7a700 1 -- 172.21.15.92:0/4190991975 >> v1:172.21.15.92:6789/0 conn(0x7f740c13b950 legacy=0x7f740c066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f740b7fe700 1 -- 172.21.15.92:0/4190991975 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273266858 0 0) 0x7f740c0c3ba0 con 0x7f740c13b950 2023-05-16T07:17:24.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f7410f7a700 1 -- 172.21.15.92:0/4190991975 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0c3550 con 0x7f740c13b570 2023-05-16T07:17:24.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f7410f7a700 1 --2- 172.21.15.92:0/4190991975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b570 0x7f740c0be650 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f7404009e60 tx=0x7f7404009780).ready entity=mon.0 client_cookie=7de6bba0f6b768ed server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f740b7fe700 1 -- 172.21.15.92:0/4190991975 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400e070 con 0x7f740c13b570 2023-05-16T07:17:24.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f740b7fe700 1 -- 172.21.15.92:0/4190991975 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400bc30 con 0x7f740c13b570 2023-05-16T07:17:24.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.244+0000 7f740b7fe700 1 -- 172.21.15.92:0/4190991975 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404012410 con 0x7f740c13b570 2023-05-16T07:17:24.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 -- 172.21.15.92:0/4190991975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b570 msgr2=0x7f740c0be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 --2- 172.21.15.92:0/4190991975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b570 0x7f740c0be650 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f7404009e60 tx=0x7f7404009780).stop 2023-05-16T07:17:24.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 -- 172.21.15.92:0/4190991975 shutdown_connections 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 --2- 172.21.15.92:0/4190991975 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c066780 0x7f740c0c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 --2- 172.21.15.92:0/4190991975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b570 0x7f740c0be650 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 -- 172.21.15.92:0/4190991975 >> 172.21.15.92:0/4190991975 conn(0x7f740c0c8b10 msgr2=0x7f740c0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 -- 172.21.15.92:0/4190991975 shutdown_connections 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.245+0000 7f74131de700 1 -- 172.21.15.92:0/4190991975 wait complete. 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.246+0000 7f74131de700 1 Processor -- start 2023-05-16T07:17:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.246+0000 7f74131de700 1 -- start start 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f740c066780 0x7f740c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c13b570 0x7f740c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 0x7f740c138010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c155610 con 0x7f740c13b950 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c154a00 con 0x7f740c066780 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c154ce0 con 0x7f740c13b570 2023-05-16T07:17:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 0x7f740c138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 0x7f740c138010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45490/0 (socket says 172.21.15.92:45490) 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 -- 172.21.15.92:0/2738074078 learned_addr learned my addr 172.21.15.92:0/2738074078 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f740bfff700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c13b570 0x7f740c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f7410f7a700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f740c066780 0x7f740c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 -- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c13b570 msgr2=0x7f740c1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c13b570 0x7f740c1530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 -- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f740c066780 msgr2=0x7f740c1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f740c066780 0x7f740c1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 -- 172.21.15.92:0/2738074078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009a20 con 0x7f740c13b950 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f7410f7a700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f740c066780 0x7f740c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f741177b700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 0x7f740c138010 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f73f400c620 tx=0x7f73f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4010580 con 0x7f740c13b950 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f40106e0 con 0x7f740c13b950 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4010a60 con 0x7f740c13b950 2023-05-16T07:17:24.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c154fc0 con 0x7f740c13b950 2023-05-16T07:17:24.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.247+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c1535c0 con 0x7f740c13b950 2023-05-16T07:17:24.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.248+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c0a8510 con 0x7f740c13b950 2023-05-16T07:17:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.249+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f73f401b460 con 0x7f740c13b950 2023-05-16T07:17:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.249+0000 7f7408ff9700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73ec05ed90 0x7f73ec061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.249+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f73f4079de0 con 0x7f740c13b950 2023-05-16T07:17:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.249+0000 7f7410f7a700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73ec05ed90 0x7f73ec061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.250+0000 7f7410f7a700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73ec05ed90 0x7f73ec061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7404009e60 tx=0x7f7404009780).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.253+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f73f404b0d0 con 0x7f740c13b950 2023-05-16T07:17:24.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.457+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f740c05efa0 con 0x7f740c13b950 2023-05-16T07:17:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.457+0000 7f7408ff9700 1 -- 172.21.15.92:0/2738074078 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f73f404e6f0 con 0x7f740c13b950 2023-05-16T07:17:24.461 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:17:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73ec05ed90 msgr2=0x7f73ec061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73ec05ed90 0x7f73ec061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7404009e60 tx=0x7f7404009780).stop 2023-05-16T07:17:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 msgr2=0x7f740c138010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 0x7f740c138010 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f73f400c620 tx=0x7f73f4008040).stop 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 shutdown_connections 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73ec05ed90 0x7f73ec061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f740c13b950 0x7f740c138010 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f740c066780 0x7f740c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 --2- 172.21.15.92:0/2738074078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f740c13b570 0x7f740c1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 >> 172.21.15.92:0/2738074078 conn(0x7f740c0c8b10 msgr2=0x7f740c139860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 shutdown_connections 2023-05-16T07:17:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:24.459+0000 7f74131de700 1 -- 172.21.15.92:0/2738074078 wait complete. 2023-05-16T07:17:24.477 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:17:24.478 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:17:24.478 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:17:24.479 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:17:24.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:24.479 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:17:24.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:17:24.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:17:24.480 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:17:24.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.481 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:17:24.496 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:17:24.498 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:17:24.498 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:17:24.499 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.499 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:17:24.499 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.499 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:17:24.499 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:24.500 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:17:24.500 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:24.500 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:17:24.500 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:24.501 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:17:24.501 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:24.501 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:17:24.501 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41110sec preferred_lft 41110sec 2023-05-16T07:17:24.502 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:17:24.502 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.502 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:17:24.502 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:24.502 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:17:24.503 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.503 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:17:24.503 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.503 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:17:24.504 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:17:24.504 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:17:24.504 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.504 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:17:24.504 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:17:24.506 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:17:24.506 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:24.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.507 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:17:24.560 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:17:24.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.561 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:17:24.617 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:17:24.617 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:17:24.617 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:17:24.617 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:17:24.617 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:17:24.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.618 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:17:24.619 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:17:24.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:17:24.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:17:24.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:17:24.620 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:17:24.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.715 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:17:24.730 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:17:24.730 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:17:24.730 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:17:24.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:17:24.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.795 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:24.849 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:17:24.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:17:24.927 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:17:24.928 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:17:24.970 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:17:24.970 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:17:24.971 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:17:24.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:24.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:25.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:17:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.182+0000 7f29b2b61700 1 Processor -- start 2023-05-16T07:17:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.182+0000 7f29b2b61700 1 -- start start 2023-05-16T07:17:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b2b61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f29ac069db0 0x7f29ac15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b2b61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ac0c0660 con 0x7f29ac154170 2023-05-16T07:17:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b2b61700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ac0c1b00 con 0x7f29ac154550 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b2b61700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ac0bfd30 con 0x7f29ac069db0 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b10fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f29ac069db0 0x7f29ac15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b08fd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f29ac154170 0x7f29ac0c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56248/0 (socket says 172.21.15.92:56248) 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.183+0000 7f29b08fd700 1 -- 172.21.15.92:0/1152305366 learned_addr learned my addr 172.21.15.92:0/1152305366 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b10fe700 1 -- 172.21.15.92:0/1152305366 >> v1:172.21.15.158:6789/0 conn(0x7f29ac154550 legacy=0x7f29ac0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b10fe700 1 -- 172.21.15.92:0/1152305366 >> v1:172.21.15.92:6789/0 conn(0x7f29ac154170 legacy=0x7f29ac0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29ab7fe700 1 -- 172.21.15.92:0/1152305366 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738890404 0 0) 0x7f29ac0c0660 con 0x7f29ac154170 2023-05-16T07:17:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b10fe700 1 -- 172.21.15.92:0/1152305366 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac0c0010 con 0x7f29ac069db0 2023-05-16T07:17:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b10fe700 1 --2- 172.21.15.92:0/1152305366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac069db0 0x7f29ac15f200 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f29a4009e60 tx=0x7f29a4009780).ready entity=mon.0 client_cookie=343512483fb31b4f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29ab7fe700 1 -- 172.21.15.92:0/1152305366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a400e070 con 0x7f29ac069db0 2023-05-16T07:17:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29ab7fe700 1 -- 172.21.15.92:0/1152305366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a400bc30 con 0x7f29ac069db0 2023-05-16T07:17:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29ab7fe700 1 -- 172.21.15.92:0/1152305366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4012410 con 0x7f29ac069db0 2023-05-16T07:17:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b2b61700 1 -- 172.21.15.92:0/1152305366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac069db0 msgr2=0x7f29ac15f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b2b61700 1 --2- 172.21.15.92:0/1152305366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac069db0 0x7f29ac15f200 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f29a4009e60 tx=0x7f29a4009780).stop 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b2b61700 1 -- 172.21.15.92:0/1152305366 shutdown_connections 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b2b61700 1 --2- 172.21.15.92:0/1152305366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac069db0 0x7f29ac15f200 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.184+0000 7f29b2b61700 1 -- 172.21.15.92:0/1152305366 >> 172.21.15.92:0/1152305366 conn(0x7f29ac0cb700 msgr2=0x7f29ac0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.185+0000 7f29b2b61700 1 -- 172.21.15.92:0/1152305366 shutdown_connections 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.185+0000 7f29b2b61700 1 -- 172.21.15.92:0/1152305366 wait complete. 2023-05-16T07:17:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.186+0000 7f29b2b61700 1 Processor -- start 2023-05-16T07:17:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.186+0000 7f29b2b61700 1 -- start start 2023-05-16T07:17:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.186+0000 7f29b2b61700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29ac069db0 0x7f29ac0c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.186+0000 7f29b2b61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 0x7f29ac147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.186+0000 7f29b2b61700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29ac154550 0x7f29ac0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b2b61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac0c4570 con 0x7f29ac154170 2023-05-16T07:17:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b2b61700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac0affb0 con 0x7f29ac154550 2023-05-16T07:17:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b2b61700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac0af3a0 con 0x7f29ac069db0 2023-05-16T07:17:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 0x7f29ac147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 0x7f29ac147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45506/0 (socket says 172.21.15.92:45506) 2023-05-16T07:17:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 -- 172.21.15.92:0/368006066 learned_addr learned my addr 172.21.15.92:0/368006066 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b10fe700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29ac154550 0x7f29ac0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b08fd700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29ac069db0 0x7f29ac0c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 -- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29ac069db0 msgr2=0x7f29ac0c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29ac069db0 0x7f29ac0c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 -- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29ac154550 msgr2=0x7f29ac0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29ac154550 0x7f29ac0c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 -- 172.21.15.92:0/368006066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a4009a20 con 0x7f29ac154170 2023-05-16T07:17:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29abfff700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 0x7f29ac147b10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f29a000c440 tx=0x7f29a0007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000f470 con 0x7f29ac154170 2023-05-16T07:17:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000f5d0 con 0x7f29ac154170 2023-05-16T07:17:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac0af680 con 0x7f29ac154170 2023-05-16T07:17:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000f970 con 0x7f29ac154170 2023-05-16T07:17:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.187+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac0afcc0 con 0x7f29ac154170 2023-05-16T07:17:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.188+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac0c3e40 con 0x7f29ac154170 2023-05-16T07:17:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.188+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f29a0010070 con 0x7f29ac154170 2023-05-16T07:17:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.189+0000 7f29a8ff9700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299805ed60 0x7f2998061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.189+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f29a0079210 con 0x7f29ac154170 2023-05-16T07:17:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.189+0000 7f29b08fd700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299805ed60 0x7f2998061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.189+0000 7f29b08fd700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299805ed60 0x7f2998061220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f299c0011e0 tx=0x7f299c009040).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.193+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f29a00478f0 con 0x7f29ac154170 2023-05-16T07:17:25.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.380+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f29ac0666c0 con 0x7f299805ed60 2023-05-16T07:17:25.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.383+0000 7f29a8ff9700 1 -- 172.21.15.92:0/368006066 <== mgr.5330 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f29ac0666c0 con 0x7f299805ed60 2023-05-16T07:17:25.387 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:17:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:17:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:25.388 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:17:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299805ed60 msgr2=0x7f2998061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299805ed60 0x7f2998061220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f299c0011e0 tx=0x7f299c009040).stop 2023-05-16T07:17:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 msgr2=0x7f29ac147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 0x7f29ac147b10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f29a000c440 tx=0x7f29a0007d80).stop 2023-05-16T07:17:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 shutdown_connections 2023-05-16T07:17:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f299805ed60 0x7f2998061220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f29ac069db0 0x7f29ac0c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29ac154550 0x7f29ac0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 --2- 172.21.15.92:0/368006066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ac154170 0x7f29ac147b10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 >> 172.21.15.92:0/368006066 conn(0x7f29ac0cb700 msgr2=0x7f29ac0696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 shutdown_connections 2023-05-16T07:17:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.384+0000 7f29b2b61700 1 -- 172.21.15.92:0/368006066 wait complete. 2023-05-16T07:17:25.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:17:25.423 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:17:25.419+0000 7f0b0236c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:17:25.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 Processor -- start 2023-05-16T07:17:25.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 -- start start 2023-05-16T07:17:25.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac153f90 0x7f53ac0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53ac0c5d00 0x7f53ac0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ac0c6670 con 0x7f53ac0c0150 2023-05-16T07:17:25.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac069f60 con 0x7f53ac0c5d00 2023-05-16T07:17:25.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.509+0000 7f53b12ea700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac06a9e0 con 0x7f53ac153f90 2023-05-16T07:17:25.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53ac0c5d00 0x7f53ac0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53b0ae9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f53ac0c0150 0x7f53ac0c05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56266/0 (socket says 172.21.15.92:56266) 2023-05-16T07:17:25.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53b0ae9700 1 -- 172.21.15.92:0/2581377427 learned_addr learned my addr 172.21.15.92:0/2581377427 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:25.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53abfff700 1 --2- 172.21.15.92:0/2581377427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac153f90 0x7f53ac0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53a3fff700 1 -- 172.21.15.92:0/2581377427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac153f90 msgr2=0x7f53ac0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:25.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53a3fff700 1 --2- 172.21.15.92:0/2581377427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac153f90 0x7f53ac0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53a3fff700 1 -- 172.21.15.92:0/2581377427 >> v1:172.21.15.92:6789/0 conn(0x7f53ac0c0150 legacy=0x7f53ac0c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53a3fff700 1 -- 172.21.15.92:0/2581377427 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac06a140 con 0x7f53ac0c5d00 2023-05-16T07:17:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53ab7fe700 1 -- 172.21.15.92:0/2581377427 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011502327 0 0) 0x7f53ac0c6670 con 0x7f53ac0c0150 2023-05-16T07:17:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53a3fff700 1 --2- 172.21.15.92:0/2581377427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c6180 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f5394009760 tx=0x7f5394009060).ready entity=mon.0 client_cookie=e6c02d230aa9b3f2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53ab7fe700 1 -- 172.21.15.92:0/2581377427 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400f510 con 0x7f53ac0c5d00 2023-05-16T07:17:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53ab7fe700 1 -- 172.21.15.92:0/2581377427 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400f670 con 0x7f53ac0c5d00 2023-05-16T07:17:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.510+0000 7f53ab7fe700 1 -- 172.21.15.92:0/2581377427 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400f9f0 con 0x7f53ac0c5d00 2023-05-16T07:17:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 -- 172.21.15.92:0/2581377427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 msgr2=0x7f53ac0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 --2- 172.21.15.92:0/2581377427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c6180 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f5394009760 tx=0x7f5394009060).stop 2023-05-16T07:17:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 -- 172.21.15.92:0/2581377427 shutdown_connections 2023-05-16T07:17:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 --2- 172.21.15.92:0/2581377427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac153f90 0x7f53ac0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 --2- 172.21.15.92:0/2581377427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c6180 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 -- 172.21.15.92:0/2581377427 >> 172.21.15.92:0/2581377427 conn(0x7f53ac0cb600 msgr2=0x7f53ac0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:25.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 -- 172.21.15.92:0/2581377427 shutdown_connections 2023-05-16T07:17:25.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.511+0000 7f53b12ea700 1 -- 172.21.15.92:0/2581377427 wait complete. 2023-05-16T07:17:25.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.512+0000 7f53b12ea700 1 Processor -- start 2023-05-16T07:17:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.512+0000 7f53b12ea700 1 -- start start 2023-05-16T07:17:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b12ea700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac0c0150 0x7f53ac167680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b12ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b12ea700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53ac153f90 0x7f53ac0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b12ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0c25b0 con 0x7f53ac0c5d00 2023-05-16T07:17:25.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b12ea700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0c33c0 con 0x7f53ac153f90 2023-05-16T07:17:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b12ea700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac168340 con 0x7f53ac0c0150 2023-05-16T07:17:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45522/0 (socket says 172.21.15.92:45522) 2023-05-16T07:17:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 -- 172.21.15.92:0/4035696537 learned_addr learned my addr 172.21.15.92:0/4035696537 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53abfff700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac0c0150 0x7f53ac167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53b0ae9700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53ac153f90 0x7f53ac0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 -- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac0c0150 msgr2=0x7f53ac167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac0c0150 0x7f53ac167680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 -- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53ac153f90 msgr2=0x7f53ac0c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53ac153f90 0x7f53ac0c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 -- 172.21.15.92:0/4035696537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940092a0 con 0x7f53ac0c5d00 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.513+0000 7f53a3fff700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c1bd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f5394011000 tx=0x7f5394017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:25.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.514+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400f7e0 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.514+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400f940 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.514+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac0c27b0 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.514+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539401b660 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.514+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac0c2d70 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.515+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac083120 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.516+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f539400f070 con 0x7f53ac0c5d00 2023-05-16T07:17:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.516+0000 7f53a8ff9700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f538c05edc0 0x7f538c061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:25.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.516+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f539407b590 con 0x7f53ac0c5d00 2023-05-16T07:17:25.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.516+0000 7f53abfff700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f538c05edc0 0x7f538c061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:25.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.517+0000 7f53abfff700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f538c05edc0 0x7f538c061280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f539c009b70 tx=0x7f539c009060).ready entity=mgr.5330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:25.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.521+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f539404c8e0 con 0x7f53ac0c5d00 2023-05-16T07:17:25.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:25.726+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f53ac047150 con 0x7f53ac0c5d00 2023-05-16T07:17:26.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.526+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 25) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f5394079240 con 0x7f53ac0c5d00 2023-05-16T07:17:26.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.527+0000 7f53a8ff9700 1 -- 172.21.15.92:0/4035696537 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v25) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5394055cd0 con 0x7f53ac0c5d00 2023-05-16T07:17:26.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f538c05edc0 msgr2=0x7f538c061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f538c05edc0 0x7f538c061280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f539c009b70 tx=0x7f539c009060).stop 2023-05-16T07:17:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 msgr2=0x7f53ac0c1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c1bd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f5394011000 tx=0x7f5394017000).stop 2023-05-16T07:17:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 shutdown_connections 2023-05-16T07:17:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f538c05edc0 0x7f538c061280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f53ac0c0150 0x7f53ac167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f53ac153f90 0x7f53ac0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 --2- 172.21.15.92:0/4035696537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53ac0c5d00 0x7f53ac0c1bd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 >> 172.21.15.92:0/4035696537 conn(0x7f53ac0cb600 msgr2=0x7f53ac151220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 shutdown_connections 2023-05-16T07:17:26.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.530+0000 7f53b12ea700 1 -- 172.21.15.92:0/4035696537 wait complete. 2023-05-16T07:17:26.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:17:26.671 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:26.667+0000 7f1e6877b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:26.675 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:26.672+0000 7fa8b0b9a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.704+0000 7f170df38700 1 Processor -- start 2023-05-16T07:17:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.704+0000 7f170df38700 1 -- start start 2023-05-16T07:17:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f170df38700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17080cb420 0x7f1708154500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f170df38700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17080c6320 con 0x7f1708155580 2023-05-16T07:17:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f170df38700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17081550f0 con 0x7f1708154120 2023-05-16T07:17:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f170df38700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17080c6840 con 0x7f17080cb420 2023-05-16T07:17:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f1706ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17080cb420 0x7f1708154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f1706ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17080cb420 0x7f1708154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45554/0 (socket says 172.21.15.92:45554) 2023-05-16T07:17:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f1706ffd700 1 -- 172.21.15.92:0/2629947655 learned_addr learned my addr 172.21.15.92:0/2629947655 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f1706ffd700 1 -- 172.21.15.92:0/2629947655 >> v1:172.21.15.158:6789/0 conn(0x7f1708154120 legacy=0x7f17080c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f1706ffd700 1 -- 172.21.15.92:0/2629947655 >> v1:172.21.15.92:6789/0 conn(0x7f1708155580 legacy=0x7f1708068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.705+0000 7f1706ffd700 1 -- 172.21.15.92:0/2629947655 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080c6560 con 0x7f17080cb420 2023-05-16T07:17:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f1706ffd700 1 --2- 172.21.15.92:0/2629947655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708154500 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f16fc006fd0 tx=0x7f16fc0090f0).ready entity=mon.0 client_cookie=266a347af38ec742 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f17067fc700 1 -- 172.21.15.92:0/2629947655 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00b600 con 0x7f17080cb420 2023-05-16T07:17:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f17067fc700 1 -- 172.21.15.92:0/2629947655 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00b760 con 0x7f17080cb420 2023-05-16T07:17:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f17067fc700 1 -- 172.21.15.92:0/2629947655 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00bae0 con 0x7f17080cb420 2023-05-16T07:17:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 -- 172.21.15.92:0/2629947655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 msgr2=0x7f1708154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 --2- 172.21.15.92:0/2629947655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708154500 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f16fc006fd0 tx=0x7f16fc0090f0).stop 2023-05-16T07:17:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 -- 172.21.15.92:0/2629947655 shutdown_connections 2023-05-16T07:17:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 --2- 172.21.15.92:0/2629947655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708154500 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 -- 172.21.15.92:0/2629947655 >> 172.21.15.92:0/2629947655 conn(0x7f17080678e0 msgr2=0x7f17080cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 -- 172.21.15.92:0/2629947655 shutdown_connections 2023-05-16T07:17:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.706+0000 7f170df38700 1 -- 172.21.15.92:0/2629947655 wait complete. 2023-05-16T07:17:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 Processor -- start 2023-05-16T07:17:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 -- start start 2023-05-16T07:17:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708165040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1708154120 0x7f17080aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1708155580 0x7f1708151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080a1de0 con 0x7f17080cb420 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1708165530 con 0x7f1708155580 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.708+0000 7f170df38700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080add10 con 0x7f1708154120 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708165040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708165040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45558/0 (socket says 172.21.15.92:45558) 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 -- 172.21.15.92:0/1505679063 learned_addr learned my addr 172.21.15.92:0/1505679063 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f1706ffd700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1708154120 0x7f17080aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f1707fff700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1708155580 0x7f1708151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 -- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1708154120 msgr2=0x7f17080aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1708154120 0x7f17080aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 -- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1708155580 msgr2=0x7f1708151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1708155580 0x7f1708151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 -- 172.21.15.92:0/1505679063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009370 con 0x7f17080cb420 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f17077fe700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708165040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f16f800c4b0 tx=0x7f16f8007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f800ce60 con 0x7f17080cb420 2023-05-16T07:17:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f800f470 con 0x7f17080cb420 2023-05-16T07:17:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f800f7f0 con 0x7f17080cb420 2023-05-16T07:17:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17080adff0 con 0x7f17080cb420 2023-05-16T07:17:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.709+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17080ae5b0 con 0x7f17080cb420 2023-05-16T07:17:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.710+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f16f8019030 con 0x7f17080cb420 2023-05-16T07:17:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.710+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1708066730 con 0x7f17080cb420 2023-05-16T07:17:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.711+0000 7f16ef7fe700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f16f405ede0 0x7f16f40612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.711+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f16f8079010 con 0x7f17080cb420 2023-05-16T07:17:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.711+0000 7f1706ffd700 1 -- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f16f405ede0 msgr2=0x7f16f40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.711+0000 7f1706ffd700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f16f405ede0 0x7f16f40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.715+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f16f80488f0 con 0x7f17080cb420 2023-05-16T07:17:26.733 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:26.730+0000 7f1e6877b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:26.742 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:26.738+0000 7fa8b0b9a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:26.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.885+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1708154aa0 con 0x7f17080cb420 2023-05-16T07:17:26.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.890+0000 7f16ef7fe700 1 -- 172.21.15.92:0/1505679063 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f16f804a410 con 0x7f17080cb420 2023-05-16T07:17:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f16f405ede0 msgr2=0x7f16f40612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f16f405ede0 0x7f16f40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 msgr2=0x7f1708165040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708165040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f16f800c4b0 tx=0x7f16f8007e30).stop 2023-05-16T07:17:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 shutdown_connections 2023-05-16T07:17:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f16f405ede0 0x7f16f40612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1708155580 0x7f1708151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17080cb420 0x7f1708165040 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 --2- 172.21.15.92:0/1505679063 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1708154120 0x7f17080aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 >> 172.21.15.92:0/1505679063 conn(0x7f17080678e0 msgr2=0x7f17080ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 shutdown_connections 2023-05-16T07:17:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:26.892+0000 7f170df38700 1 -- 172.21.15.92:0/1505679063 wait complete. 2023-05-16T07:17:26.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:26.908 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:17:26.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-05-16T07:17:27.006 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:27.003+0000 7f1e6877b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:27.025 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:27.021+0000 7fa8b0b9a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:27.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.082+0000 7f2fe94d5700 1 Processor -- start 2023-05-16T07:17:27.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.082+0000 7f2fe94d5700 1 -- start start 2023-05-16T07:17:27.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe94d5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2fe414ea00 0x7f2fe4154760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:27.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe94d5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe4154c50 0x7f2fe40c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:27.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe94d5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe4155100 con 0x7f2fe414e620 2023-05-16T07:17:27.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe94d5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe40c4830 con 0x7f2fe414ea00 2023-05-16T07:17:27.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe94d5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4065bf0 con 0x7f2fe4154c50 2023-05-16T07:17:27.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2fe414e620 0x7f2fe40c4120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56330/0 (socket says 172.21.15.92:56330) 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe3fff700 1 -- 172.21.15.92:0/3981277516 learned_addr learned my addr 172.21.15.92:0/3981277516 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe37fe700 1 --2- 172.21.15.92:0/3981277516 >> v2:172.21.15.158:3300/0 conn(0x7f2fe414ea00 0x7f2fe4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe8cd4700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe4154c50 0x7f2fe40c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe2ffd700 1 -- 172.21.15.92:0/3981277516 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058541603 0 0) 0x7f2fd8001110 con 0x7f2fe414e620 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe2ffd700 1 -- 172.21.15.92:0/3981277516 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fcc0033f0 con 0x7f2fe414e620 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe37fe700 1 -- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe4154c50 msgr2=0x7f2fe40c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:27.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe2ffd700 1 -- 172.21.15.92:0/3981277516 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1470390654 0 0) 0x7f2fcc0033f0 con 0x7f2fe414e620 2023-05-16T07:17:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe37fe700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe4154c50 0x7f2fe40c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe37fe700 1 -- 172.21.15.92:0/3981277516 >> v1:172.21.15.92:6789/0 conn(0x7f2fe414e620 legacy=0x7f2fe40c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe37fe700 1 -- 172.21.15.92:0/3981277516 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4065ed0 con 0x7f2fe414ea00 2023-05-16T07:17:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.083+0000 7f2fe8cd4700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe4154c50 0x7f2fe40c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe37fe700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414ea00 0x7f2fe4154760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f2fd4009760 tx=0x7f2fd4009060).ready entity=mon.1 client_cookie=11bcc5b539bd5ad0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:27.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe2ffd700 1 -- 172.21.15.92:0/3981277516 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8001600 con 0x7f2fe414ea00 2023-05-16T07:17:27.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe2ffd700 1 -- 172.21.15.92:0/3981277516 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4011400 con 0x7f2fe414ea00 2023-05-16T07:17:27.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe2ffd700 1 -- 172.21.15.92:0/3981277516 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4011780 con 0x7f2fe414ea00 2023-05-16T07:17:27.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 -- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414ea00 msgr2=0x7f2fe4154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414ea00 0x7f2fe4154760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f2fd4009760 tx=0x7f2fd4009060).stop 2023-05-16T07:17:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 -- 172.21.15.92:0/3981277516 shutdown_connections 2023-05-16T07:17:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe4154c50 0x7f2fe40c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 --2- 172.21.15.92:0/3981277516 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414ea00 0x7f2fe4154760 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 -- 172.21.15.92:0/3981277516 >> 172.21.15.92:0/3981277516 conn(0x7f2fe40c7a80 msgr2=0x7f2fe40c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 -- 172.21.15.92:0/3981277516 shutdown_connections 2023-05-16T07:17:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.084+0000 7f2fe94d5700 1 -- 172.21.15.92:0/3981277516 wait complete. 2023-05-16T07:17:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.085+0000 7f2fe94d5700 1 Processor -- start 2023-05-16T07:17:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 -- start start 2023-05-16T07:17:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414e620 0x7f2fe4164920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe414ea00 0x7f2fe40a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fe4154c50 0x7f2fe40a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4167fb0 con 0x7f2fe4154c50 2023-05-16T07:17:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40a6350 con 0x7f2fe414e620 2023-05-16T07:17:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe94d5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40aa4e0 con 0x7f2fe414ea00 2023-05-16T07:17:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe8cd4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fe4154c50 0x7f2fe40a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.086+0000 7f2fe8cd4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fe4154c50 0x7f2fe40a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45588/0 (socket says 172.21.15.92:45588) 2023-05-16T07:17:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 -- 172.21.15.92:0/1243381839 learned_addr learned my addr 172.21.15.92:0/1243381839 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe37fe700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe414ea00 0x7f2fe40a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414e620 0x7f2fe4164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe414ea00 msgr2=0x7f2fe40a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe414ea00 0x7f2fe40a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414e620 msgr2=0x7f2fe4164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414e620 0x7f2fe4164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 -- 172.21.15.92:0/1243381839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009a20 con 0x7f2fe4154c50 2023-05-16T07:17:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe37fe700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2fe414ea00 0x7f2fe40a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2fe414e620 0x7f2fe4164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe8cd4700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fe4154c50 0x7f2fe40a9ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2fdc009fc0 tx=0x7f2fdc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00cbd0 con 0x7f2fe4154c50 2023-05-16T07:17:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe94d5700 1 -- 172.21.15.92:0/1243381839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd40092a0 con 0x7f2fe4154c50 2023-05-16T07:17:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc00f040 con 0x7f2fe4154c50 2023-05-16T07:17:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc013400 con 0x7f2fe4154c50 2023-05-16T07:17:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.087+0000 7f2fe94d5700 1 -- 172.21.15.92:0/1243381839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe41651e0 con 0x7f2fe4154c50 2023-05-16T07:17:27.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.088+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f2fdc0137f0 con 0x7f2fe4154c50 2023-05-16T07:17:27.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.088+0000 7f2fe94d5700 1 -- 172.21.15.92:0/1243381839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe41680f0 con 0x7f2fe4154c50 2023-05-16T07:17:27.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.089+0000 7f2fcbfff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:27.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.089+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f2fdc07a8d0 con 0x7f2fe4154c50 2023-05-16T07:17:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.089+0000 7f2fe3fff700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 msgr2=0x7f2fd00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.089+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:27.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.093+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2fe41680f0 con 0x7f2fe4154c50 2023-05-16T07:17:27.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.289+0000 7f2fe94d5700 1 -- 172.21.15.92:0/1243381839 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe41680f0 con 0x7f2fd005ede0 2023-05-16T07:17:27.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.290+0000 7f2fe3fff700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 msgr2=0x7f2fd00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:27.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.290+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:27.619 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:27.616+0000 7f1e6877b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:27.644 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:27.640+0000 7fa8b0b9a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:27.691 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:27.688+0000 7f1e6877b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.691+0000 7f2fe3fff700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 msgr2=0x7f2fd00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:27.691+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:17:27.714 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:27.711+0000 7fa8b0b9a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:27.875 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:27.872+0000 7f1e6877b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:27.896 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:27.892+0000 7fa8b0b9a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:28.289 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:28.286+0000 7f1e6877b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:28.308 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:28.303+0000 7fa8b0b9a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:28.391 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:28.388+0000 7f1e6877b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:28.408 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:28.404+0000 7fa8b0b9a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:28.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:28.492+0000 7f2fe3fff700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 msgr2=0x7f2fd00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:28.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:28.492+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:17:28.543 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:28.540+0000 7f1e6877b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:28.558 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:28.554+0000 7fa8b0b9a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:28.616 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:28.613+0000 7f1e6877b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:28.631 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:28.627+0000 7fa8b0b9a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:28.683 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:28.680+0000 7f1e6877b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:28.697 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:28.693+0000 7fa8b0b9a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:28.936 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:28.933+0000 7f1e6877b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:28.949 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:28.946+0000 7fa8b0b9a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:29.120 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:29.117+0000 7f1e6877b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:29.128 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:29.125+0000 7fa8b0b9a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:29.761 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:29.757+0000 7fa8b0b9a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:29.761 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:29.757+0000 7f1e6877b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:29.827 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:29.823+0000 7fa8b0b9a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:29.827 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:29.824+0000 7f1e6877b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:29.899 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:29.896+0000 7fa8b0b9a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:29.901 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:29.898+0000 7f1e6877b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:30.041 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:30.037+0000 7fa8b0b9a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:30.045 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.041+0000 7f1e6877b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:30.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:30.094+0000 7f2fe3fff700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 msgr2=0x7f2fd00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:30.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:30.094+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:17:30.109 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:30.106+0000 7fa8b0b9a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:30.116 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.111+0000 7f1e6877b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:30.221 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:30.218+0000 7fa8b0b9a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:30.228 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.225+0000 7f1e6877b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:30.327 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:30.323+0000 7fa8b0b9a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:30.335 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.331+0000 7f1e6877b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:30.611 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:30.608+0000 7fa8b0b9a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:30.623 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.619+0000 7f1e6877b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:30.680 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:30.676+0000 7fa8b0b9a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:30.691 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.688+0000 7f1e6877b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:30.745+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 26) v1 ==== 75109+0+0 (secure 0 0 0) 0x7f2fdc078580 con 0x7f2fe4154c50 2023-05-16T07:17:30.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:30.745+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 msgr2=0x7f2fd00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:30.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:30.745+0000 7f2fcbfff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:30.808 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de6f5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.808 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de6f5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de6f5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de6f5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de6f5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de3f57700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de3f57700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de3f57700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.809 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de3f57700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:30.810 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:30.805+0000 7f1de3f57700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:31.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.757+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 27) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f2fdc078900 con 0x7f2fe4154c50 2023-05-16T07:17:31.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.757+0000 7f2fcbfff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd002fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.757+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe41680f0 con 0x7f2fd005ede0 2023-05-16T07:17:31.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.759+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd002fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.760+0000 7f2fe3fff700 1 --2- 172.21.15.92:0/1243381839 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2fd005ede0 0x7f2fd002fd40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f2fd80017b0 tx=0x7f2fd8001dd0).ready entity=mgr.5582 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:31.766 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:31.762+0000 7f1e16206700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-05-16T07:17:31.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.762+0000 7f2fcbfff700 1 -- 172.21.15.92:0/1243381839 <== mgr.5582 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f2fe41680f0 con 0x7f2fd005ede0 2023-05-16T07:17:31.768 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-05-16T07:17:31.771 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-05-16T07:17:31.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:31.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-05-16T07:17:31.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:17:31.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.944+0000 7fe88d98b700 1 Processor -- start 2023-05-16T07:17:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.944+0000 7fe88d98b700 1 -- start start 2023-05-16T07:17:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe88d98b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe8880aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe88d98b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe888154470 0x7fe8880a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe88d98b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8880aa570 con 0x7fe8880aa150 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe88d98b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8880c94d0 con 0x7fe888154470 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe88d98b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880cb7e0 con 0x7fe888154090 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe8867fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe888154470 0x7fe8880a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe8867fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe888154470 0x7fe8880a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45700/0 (socket says 172.21.15.92:45700) 2023-05-16T07:17:31.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.945+0000 7fe8867fc700 1 -- 172.21.15.92:0/3767031756 learned_addr learned my addr 172.21.15.92:0/3767031756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:31.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe886ffd700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe8880aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe8867fc700 1 -- 172.21.15.92:0/3767031756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 msgr2=0x7fe8880aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:31.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe8867fc700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe8880aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:31.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe8867fc700 1 -- 172.21.15.92:0/3767031756 >> v1:172.21.15.92:6789/0 conn(0x7fe8880aa150 legacy=0x7fe888068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:31.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe8867fc700 1 -- 172.21.15.92:0/3767031756 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8880c88c0 con 0x7fe888154470 2023-05-16T07:17:31.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe886ffd700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe8880aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe8867fc700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe8880a9c60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fe870006fd0 tx=0x7fe8700090f0).ready entity=mon.0 client_cookie=60c0246bca2f350f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe885ffb700 1 -- 172.21.15.92:0/3767031756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000b600 con 0x7fe888154470 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe885ffb700 1 -- 172.21.15.92:0/3767031756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000b760 con 0x7fe888154470 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe885ffb700 1 -- 172.21.15.92:0/3767031756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000bae0 con 0x7fe888154470 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe88d98b700 1 -- 172.21.15.92:0/3767031756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 msgr2=0x7fe8880a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe88d98b700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe8880a9c60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fe870006fd0 tx=0x7fe8700090f0).stop 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe88d98b700 1 -- 172.21.15.92:0/3767031756 shutdown_connections 2023-05-16T07:17:31.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.946+0000 7fe88d98b700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe8880aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.947+0000 7fe88d98b700 1 --2- 172.21.15.92:0/3767031756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe8880a9c60 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.947+0000 7fe88d98b700 1 -- 172.21.15.92:0/3767031756 >> 172.21.15.92:0/3767031756 conn(0x7fe888067920 msgr2=0x7fe8880cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.947+0000 7fe88d98b700 1 -- 172.21.15.92:0/3767031756 shutdown_connections 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.947+0000 7fe88d98b700 1 -- 172.21.15.92:0/3767031756 wait complete. 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.948+0000 7fe88d98b700 1 Processor -- start 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.948+0000 7fe88d98b700 1 -- start start 2023-05-16T07:17:31.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8880aa150 0x7fe888165210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe88813f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe888151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88813fe80 con 0x7fe888154470 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88814f9f0 con 0x7fe8880aa150 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe888151e70 con 0x7fe888154090 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe888151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe888151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45706/0 (socket says 172.21.15.92:45706) 2023-05-16T07:17:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 -- 172.21.15.92:0/2488207822 learned_addr learned my addr 172.21.15.92:0/2488207822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:31.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8867fc700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe88813f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe886ffd700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8880aa150 0x7fe888165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 -- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 msgr2=0x7fe88813f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:31.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe88813f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 -- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8880aa150 msgr2=0x7fe888165210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8880aa150 0x7fe888165210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 -- 172.21.15.92:0/2488207822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870009370 con 0x7fe888154470 2023-05-16T07:17:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe886ffd700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8880aa150 0x7fe888165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:31.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe8877fe700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe888151620 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fe88000c440 tx=0x7fe880007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:31.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000cea0 con 0x7fe888154470 2023-05-16T07:17:31.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880001e90 con 0x7fe888154470 2023-05-16T07:17:31.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe888152150 con 0x7fe888154470 2023-05-16T07:17:31.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000f680 con 0x7fe888154470 2023-05-16T07:17:31.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.949+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88814fc00 con 0x7fe888154470 2023-05-16T07:17:31.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.950+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe888066730 con 0x7fe888154470 2023-05-16T07:17:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.951+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 27) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fe880010070 con 0x7fe888154470 2023-05-16T07:17:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.951+0000 7fe87effd700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe86805ed40 0x7fe868061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.951+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fe880078e00 con 0x7fe888154470 2023-05-16T07:17:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.951+0000 7fe886ffd700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe86805ed40 0x7fe868061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.951+0000 7fe886ffd700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe86805ed40 0x7fe868061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe8780076e0 tx=0x7fe8780098f0).ready entity=mgr.5582 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:31.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:31.955+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe8800478f0 con 0x7fe888154470 2023-05-16T07:17:32.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.160+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe88814fe50 con 0x7fe888154470 2023-05-16T07:17:32.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.761+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fe880076ab0 con 0x7fe888154470 2023-05-16T07:17:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.763+0000 7fe87effd700 1 -- 172.21.15.92:0/2488207822 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe8800742b0 con 0x7fe888154470 2023-05-16T07:17:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe86805ed40 msgr2=0x7fe868061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe86805ed40 0x7fe868061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe8780076e0 tx=0x7fe8780098f0).stop 2023-05-16T07:17:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 msgr2=0x7fe888151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe888151620 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fe88000c440 tx=0x7fe880007d70).stop 2023-05-16T07:17:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 shutdown_connections 2023-05-16T07:17:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe86805ed40 0x7fe868061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe888154470 0x7fe888151620 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8880aa150 0x7fe888165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 --2- 172.21.15.92:0/2488207822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe888154090 0x7fe88813f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 >> 172.21.15.92:0/2488207822 conn(0x7fe888067920 msgr2=0x7fe8880c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 shutdown_connections 2023-05-16T07:17:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:32.766+0000 7fe88d98b700 1 -- 172.21.15.92:0/2488207822 wait complete. 2023-05-16T07:17:32.783 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:17:32.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:32.813 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:32.841 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:17:32.842 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:17:32.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:17:32.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:32.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:17:32.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:32.899 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:17:32.907 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:32.902+0000 7f079daea680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:32.911 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:32.908+0000 7f03e447c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:32.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:32.971 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:32.967+0000 7f079daea680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:32.972 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:32.969+0000 7f03e447c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.064+0000 7fc10874a700 1 Processor -- start 2023-05-16T07:17:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.064+0000 7fc10874a700 1 -- start start 2023-05-16T07:17:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc10874a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc100154040 0x7fc100069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc10874a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc100063e30 0x7fc100062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc10874a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc100062960 con 0x7fc1000bf4f0 2023-05-16T07:17:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc10874a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc100154420 con 0x7fc100063e30 2023-05-16T07:17:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc10874a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc100069fb0 con 0x7fc100154040 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc100063e30 0x7fc100062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc100063e30 0x7fc100062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45778/0 (socket says 172.21.15.92:45778) 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 -- 172.21.15.92:0/2270727930 learned_addr learned my addr 172.21.15.92:0/2270727930 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc1064e6700 1 --2- 172.21.15.92:0/2270727930 >> v2:172.21.15.158:3300/0 conn(0x7fc100154040 0x7fc100069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 -- 172.21.15.92:0/2270727930 >> v2:172.21.15.158:3300/0 conn(0x7fc100154040 msgr2=0x7fc100069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 --2- 172.21.15.92:0/2270727930 >> v2:172.21.15.158:3300/0 conn(0x7fc100154040 0x7fc100069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 -- 172.21.15.92:0/2270727930 >> v1:172.21.15.158:6789/0 conn(0x7fc1000bf4f0 legacy=0x7fc100063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 -- 172.21.15.92:0/2270727930 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100062350 con 0x7fc100063e30 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc106ce7700 1 --2- 172.21.15.92:0/2270727930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc100062df0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fc1000c89b0 tx=0x7fc0f8009840).ready entity=mon.0 client_cookie=351f5778e4d2edc4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.065+0000 7fc1054e4700 1 -- 172.21.15.92:0/2270727930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800e070 con 0x7fc100063e30 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc1054e4700 1 -- 172.21.15.92:0/2270727930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f800ba10 con 0x7fc100063e30 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc1054e4700 1 -- 172.21.15.92:0/2270727930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800bd90 con 0x7fc100063e30 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 -- 172.21.15.92:0/2270727930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 msgr2=0x7fc100062df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 --2- 172.21.15.92:0/2270727930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc100062df0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fc1000c89b0 tx=0x7fc0f8009840).stop 2023-05-16T07:17:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 -- 172.21.15.92:0/2270727930 shutdown_connections 2023-05-16T07:17:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 --2- 172.21.15.92:0/2270727930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc100062df0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 --2- 172.21.15.92:0/2270727930 >> v2:172.21.15.158:3300/0 conn(0x7fc100154040 0x7fc100069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 -- 172.21.15.92:0/2270727930 >> 172.21.15.92:0/2270727930 conn(0x7fc1000c8aa0 msgr2=0x7fc1000c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 -- 172.21.15.92:0/2270727930 shutdown_connections 2023-05-16T07:17:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.066+0000 7fc10874a700 1 -- 172.21.15.92:0/2270727930 wait complete. 2023-05-16T07:17:33.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.067+0000 7fc10874a700 1 Processor -- start 2023-05-16T07:17:33.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.067+0000 7fc10874a700 1 -- start start 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc10874a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc1000a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc10874a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1000bf4f0 0x7fc100164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc10874a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc100154040 0x7fc100156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc10874a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000a8400 con 0x7fc100063e30 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc10874a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000ad8d0 con 0x7fc100154040 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc10874a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc100164750 con 0x7fc1000bf4f0 2023-05-16T07:17:33.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc1000a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc1000a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45780/0 (socket says 172.21.15.92:45780) 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 -- 172.21.15.92:0/1307511822 learned_addr learned my addr 172.21.15.92:0/1307511822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc106ce7700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc100154040 0x7fc100156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc105ce5700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1000bf4f0 0x7fc100164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 -- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1000bf4f0 msgr2=0x7fc100164260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1000bf4f0 0x7fc100164260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 -- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc100154040 msgr2=0x7fc100156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc100154040 0x7fc100156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 -- 172.21.15.92:0/1307511822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009ae0 con 0x7fc100063e30 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.068+0000 7fc1064e6700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc1000a7f10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fc0f000d8c0 tx=0x7fc0f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.069+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0011590 con 0x7fc100063e30 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.069+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f00116f0 con 0x7fc100063e30 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.069+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1000abee0 con 0x7fc100063e30 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.069+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0011a70 con 0x7fc100063e30 2023-05-16T07:17:33.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.069+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1000ac4a0 con 0x7fc100063e30 2023-05-16T07:17:33.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.069+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10006c350 con 0x7fc100063e30 2023-05-16T07:17:33.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.070+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fc0f001c460 con 0x7fc100063e30 2023-05-16T07:17:33.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.070+0000 7fc0f67fc700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 0x7fc0e40612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.070+0000 7fc105ce5700 1 -- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 msgr2=0x7fc0e40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:33.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.070+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fc0f0079d60 con 0x7fc100063e30 2023-05-16T07:17:33.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.070+0000 7fc105ce5700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 0x7fc0e40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:33.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.074+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc0f004aee0 con 0x7fc100063e30 2023-05-16T07:17:33.240 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:33.237+0000 7f03e447c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:33.251 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:33.246+0000 7f079daea680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.271+0000 7fc105ce5700 1 -- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 msgr2=0x7fc0e40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.271+0000 7fc105ce5700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 0x7fc0e40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.285+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc100157270 con 0x7fc100063e30 2023-05-16T07:17:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.285+0000 7fc0f67fc700 1 -- 172.21.15.92:0/1307511822 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fc0f004e500 con 0x7fc100063e30 2023-05-16T07:17:33.290 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:33.290 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:19.493924+0000","modified":"2023-05-16T07:17:21.499636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1922547921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1922547921},{"type":"v1","addr":"172.21.15.92:6835","nonce":1922547921}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-05-16T07:17:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 msgr2=0x7fc0e40612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 0x7fc0e40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 msgr2=0x7fc1000a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc1000a7f10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fc0f000d8c0 tx=0x7fc0f0008040).stop 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 shutdown_connections 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc0e405ede0 0x7fc0e40612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc100154040 0x7fc100156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc100063e30 0x7fc1000a7f10 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 --2- 172.21.15.92:0/1307511822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc1000bf4f0 0x7fc100164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 >> 172.21.15.92:0/1307511822 conn(0x7fc1000c8aa0 msgr2=0x7fc10014f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:33.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 shutdown_connections 2023-05-16T07:17:33.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.287+0000 7fc10874a700 1 -- 172.21.15.92:0/1307511822 wait complete. 2023-05-16T07:17:33.295 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-05-16T07:17:33.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.456+0000 7fa2ca338700 1 Processor -- start 2023-05-16T07:17:33.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.456+0000 7fa2ca338700 1 -- start start 2023-05-16T07:17:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2ca338700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154170 0x7fa2c40aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2ca338700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40c08b0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2ca338700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40bfca0 con 0x7fa2c4154550 2023-05-16T07:17:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2ca338700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40ab200 con 0x7fa2c4154170 2023-05-16T07:17:33.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c88d5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa2c40c5cb0 0x7fa2c40c0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56586/0 (socket says 172.21.15.92:56586) 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c88d5700 1 -- 172.21.15.92:0/2325435537 learned_addr learned my addr 172.21.15.92:0/2325435537 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c3fff700 1 --2- 172.21.15.92:0/2325435537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154170 0x7fa2c40aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858219700 0 0) 0x7fa2c40c08b0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac0033f0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242886777 0 0) 0x7fa2c40bfca0 con 0x7fa2c4154550 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c40c08b0 con 0x7fa2c4154550 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956814877 0 0) 0x7fa2ac0033f0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c3fff700 1 -- 172.21.15.92:0/2325435537 >> v1:172.21.15.158:6789/0 conn(0x7fa2c4154550 legacy=0x7fa2c40a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c3fff700 1 -- 172.21.15.92:0/2325435537 >> v1:172.21.15.92:6789/0 conn(0x7fa2c40c5cb0 legacy=0x7fa2c40c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c3fff700 1 -- 172.21.15.92:0/2325435537 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40bff80 con 0x7fa2c4154170 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.457+0000 7fa2c3fff700 1 --2- 172.21.15.92:0/2325435537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154170 0x7fa2c40aad10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa2b4009790 tx=0x7fa2b4009060).ready entity=mon.2 client_cookie=95044c3a46425ccf server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0017f0 con 0x7fa2c4154170 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4011400 con 0x7fa2c4154170 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2c2ffd700 1 -- 172.21.15.92:0/2325435537 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011780 con 0x7fa2c4154170 2023-05-16T07:17:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 -- 172.21.15.92:0/2325435537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154170 msgr2=0x7fa2c40aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 --2- 172.21.15.92:0/2325435537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154170 0x7fa2c40aad10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa2b4009790 tx=0x7fa2b4009060).stop 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 -- 172.21.15.92:0/2325435537 shutdown_connections 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 --2- 172.21.15.92:0/2325435537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154170 0x7fa2c40aad10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 -- 172.21.15.92:0/2325435537 >> 172.21.15.92:0/2325435537 conn(0x7fa2c40cb6d0 msgr2=0x7fa2c40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 -- 172.21.15.92:0/2325435537 shutdown_connections 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.458+0000 7fa2ca338700 1 -- 172.21.15.92:0/2325435537 wait complete. 2023-05-16T07:17:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.459+0000 7fa2ca338700 1 Processor -- start 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.459+0000 7fa2ca338700 1 -- start start 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2ca338700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 0x7fa2c40c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2ca338700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2c4154170 0x7fa2c4151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2ca338700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154550 0x7fa2c4167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2ca338700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41530d0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2ca338700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41524c0 con 0x7fa2c4154170 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2ca338700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41527a0 con 0x7fa2c4154550 2023-05-16T07:17:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 0x7fa2c40c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 0x7fa2c40c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45840/0 (socket says 172.21.15.92:45840) 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2c3fff700 1 -- 172.21.15.92:0/3484532512 learned_addr learned my addr 172.21.15.92:0/3484532512 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2c88d5700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154550 0x7fa2c4167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.460+0000 7fa2c37fe700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2c4154170 0x7fa2c4151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c3fff700 1 -- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154550 msgr2=0x7fa2c4167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c3fff700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154550 0x7fa2c4167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c3fff700 1 -- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2c4154170 msgr2=0x7fa2c4151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c3fff700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2c4154170 0x7fa2c4151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c3fff700 1 -- 172.21.15.92:0/3484532512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40092a0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c88d5700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154550 0x7fa2c4167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2c3fff700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 0x7fa2c40c4440 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fa2c4084220 tx=0x7fa2b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4010840 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b40109a0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c4152a80 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4010d40 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.461+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c4151a60 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.462+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c40666c0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.462+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fa2b400f070 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.463+0000 7fa2abfff700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2b005ede0 0x7fa2b00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.463+0000 7fa2c37fe700 1 -- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2b005ede0 msgr2=0x7fa2b00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.463+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fa2b407b3d0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.463+0000 7fa2c37fe700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2b005ede0 0x7fa2b00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.467+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa2b404c5b0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.637+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2c406c350 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.638+0000 7fa2abfff700 1 -- 172.21.15.92:0/3484532512 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18107 (secure 0 0 0) 0x7fa2b404fbd0 con 0x7fa2c40c5cb0 2023-05-16T07:17:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:33.643 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":80,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:30.702127+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:18.044089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:18.464963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3054196167":"2023-05-17T07:17:30.702082+0000","172.21.15.158:0/2925301946":"2023-05-17T07:17:30.702082+0000","172.21.15.158:6833/86301":"2023-05-17T07:17:30.702082+0000","172.21.15.158:6832/86301":"2023-05-17T07:17:30.702082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:33.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2b005ede0 msgr2=0x7fa2b00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:33.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2b005ede0 0x7fa2b00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:33.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 msgr2=0x7fa2c40c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 0x7fa2c40c4440 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fa2c4084220 tx=0x7fa2b4016000).stop 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 shutdown_connections 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2b005ede0 0x7fa2b00612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2c4154550 0x7fa2c4167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40c5cb0 0x7fa2c40c4440 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 --2- 172.21.15.92:0/3484532512 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2c4154170 0x7fa2c4151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 >> 172.21.15.92:0/3484532512 conn(0x7fa2c40cb6d0 msgr2=0x7fa2c40abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 shutdown_connections 2023-05-16T07:17:33.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.640+0000 7fa2ca338700 1 -- 172.21.15.92:0/3484532512 wait complete. 2023-05-16T07:17:33.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:17:18.044089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:17:18.464963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:17:33.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:17:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.772+0000 7f091dd42700 1 Processor -- start 2023-05-16T07:17:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.772+0000 7f091dd42700 1 -- start start 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f091dd42700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0918154120 0x7f0918154500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f091dd42700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09180c6320 con 0x7f09180cb420 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f091dd42700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09181550f0 con 0x7f0918155580 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f091dd42700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09180c6840 con 0x7f0918154120 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f0916ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09180cb420 0x7f09180c5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56596/0 (socket says 172.21.15.92:56596) 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f0916ffd700 1 -- 172.21.15.92:0/3609308583 learned_addr learned my addr 172.21.15.92:0/3609308583 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f09177fe700 1 --2- 172.21.15.92:0/3609308583 >> v2:172.21.15.158:3300/0 conn(0x7f0918154120 0x7f0918154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f09167fc700 1 -- 172.21.15.92:0/3609308583 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769183890 0 0) 0x7f090c001110 con 0x7f09180cb420 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.773+0000 7f09167fc700 1 -- 172.21.15.92:0/3609308583 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09000033f0 con 0x7f09180cb420 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09177fe700 1 -- 172.21.15.92:0/3609308583 >> v1:172.21.15.158:6789/0 conn(0x7f0918155580 legacy=0x7f0918068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09167fc700 1 -- 172.21.15.92:0/3609308583 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167379891 0 0) 0x7f09181550f0 con 0x7f0918155580 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09177fe700 1 -- 172.21.15.92:0/3609308583 >> v1:172.21.15.92:6789/0 conn(0x7f09180cb420 legacy=0x7f09180c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09177fe700 1 -- 172.21.15.92:0/3609308583 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180c6560 con 0x7f0918154120 2023-05-16T07:17:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09177fe700 1 --2- 172.21.15.92:0/3609308583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0918154120 0x7f0918154500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f0908006fd0 tx=0x7f0908006f40).ready entity=mon.1 client_cookie=bef6c82e7bc87a03 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09167fc700 1 -- 172.21.15.92:0/3609308583 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908005880 con 0x7f0918154120 2023-05-16T07:17:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09167fc700 1 -- 172.21.15.92:0/3609308583 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09080059e0 con 0x7f0918154120 2023-05-16T07:17:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f09167fc700 1 -- 172.21.15.92:0/3609308583 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908005d60 con 0x7f0918154120 2023-05-16T07:17:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.774+0000 7f091dd42700 1 -- 172.21.15.92:0/3609308583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0918154120 msgr2=0x7f0918154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.775+0000 7f091dd42700 1 --2- 172.21.15.92:0/3609308583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0918154120 0x7f0918154500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f0908006fd0 tx=0x7f0908006f40).stop 2023-05-16T07:17:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.775+0000 7f091dd42700 1 -- 172.21.15.92:0/3609308583 shutdown_connections 2023-05-16T07:17:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.775+0000 7f091dd42700 1 --2- 172.21.15.92:0/3609308583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0918154120 0x7f0918154500 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.775+0000 7f091dd42700 1 -- 172.21.15.92:0/3609308583 >> 172.21.15.92:0/3609308583 conn(0x7f09180678e0 msgr2=0x7f09180cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.775+0000 7f091dd42700 1 -- 172.21.15.92:0/3609308583 shutdown_connections 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.775+0000 7f091dd42700 1 -- 172.21.15.92:0/3609308583 wait complete. 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.776+0000 7f091dd42700 1 Processor -- start 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.776+0000 7f091dd42700 1 -- start start 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09180cb420 0x7f09180ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 0x7f0918149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0918155580 0x7f0918158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180acb90 con 0x7f0918154120 2023-05-16T07:17:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091814a390 con 0x7f09180cb420 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09180adfa0 con 0x7f0918155580 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 0x7f0918149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 0x7f0918149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45854/0 (socket says 172.21.15.92:45854) 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 -- 172.21.15.92:0/448105281 learned_addr learned my addr 172.21.15.92:0/448105281 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f09177fe700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09180cb420 0x7f09180ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0917fff700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0918155580 0x7f0918158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 -- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0918155580 msgr2=0x7f0918158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0918155580 0x7f0918158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 -- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09180cb420 msgr2=0x7f09180ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09180cb420 0x7f09180ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 -- 172.21.15.92:0/448105281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908009250 con 0x7f0918154120 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f09177fe700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09180cb420 0x7f09180ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f0916ffd700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 0x7f0918149930 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f090c00c810 tx=0x7f090c001a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c007170 con 0x7f0918154120 2023-05-16T07:17:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c0072d0 con 0x7f0918154120 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c0048c0 con 0x7f0918154120 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.777+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09180ae280 con 0x7f0918154120 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.778+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09180ae840 con 0x7f0918154120 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.778+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0918066730 con 0x7f0918154120 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.779+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f090c01b070 con 0x7f0918154120 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.779+0000 7f08ff7fe700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 0x7f09040612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.779+0000 7f09177fe700 1 -- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 msgr2=0x7f09040612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.779+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f090c079570 con 0x7f0918154120 2023-05-16T07:17:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.779+0000 7f09177fe700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 0x7f09040612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.783+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f090c04a750 con 0x7f0918154120 2023-05-16T07:17:33.845 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:33.841+0000 7f03e447c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:33.883 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:33.878+0000 7f079daea680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:33.916 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:33.913+0000 7f03e447c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:33.954 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:33.950+0000 7f079daea680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:33.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.980+0000 7f09177fe700 1 -- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 msgr2=0x7f09040612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.980+0000 7f09177fe700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 0x7f09040612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:33.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.984+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f091806c350 con 0x7f0918154120 2023-05-16T07:17:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.985+0000 7f08ff7fe700 1 -- 172.21.15.92:0/448105281 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7f090c019020 con 0x7f0918154120 2023-05-16T07:17:33.990 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:17:33.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 msgr2=0x7f09040612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:33.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 0x7f09040612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:33.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 msgr2=0x7f0918149930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:33.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 0x7f0918149930 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f090c00c810 tx=0x7f090c001a10).stop 2023-05-16T07:17:33.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 shutdown_connections 2023-05-16T07:17:33.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f090405ede0 0x7f09040612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:33.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0918155580 0x7f0918158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09180cb420 0x7f09180ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 --2- 172.21.15.92:0/448105281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0918154120 0x7f0918149930 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 >> 172.21.15.92:0/448105281 conn(0x7f09180678e0 msgr2=0x7f0918155960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 shutdown_connections 2023-05-16T07:17:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:33.987+0000 7f091dd42700 1 -- 172.21.15.92:0/448105281 wait complete. 2023-05-16T07:17:34.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:17:34.099 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:34.095+0000 7f03e447c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.118+0000 7f6a93d22700 1 Processor -- start 2023-05-16T07:17:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.118+0000 7f6a93d22700 1 -- start start 2023-05-16T07:17:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a93d22700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c150a90 0x7f6a8c150e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a93d22700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6a8c0c2db0 0x7f6a8c0bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a93d22700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c155740 con 0x7f6a8c0bc890 2023-05-16T07:17:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a93d22700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c0bd200 con 0x7f6a8c0c2db0 2023-05-16T07:17:34.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a93d22700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c154b30 con 0x7f6a8c150a90 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a912bd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6a8c0bc890 0x7f6a8c0c26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56610/0 (socket says 172.21.15.92:56610) 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.119+0000 7f6a912bd700 1 -- 172.21.15.92:0/570667838 learned_addr learned my addr 172.21.15.92:0/570667838 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a922bf700 1 --2- 172.21.15.92:0/570667838 >> v2:172.21.15.158:3300/0 conn(0x7f6a8c0c2db0 0x7f6a8c0bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a91abe700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c150a90 0x7f6a8c150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334893975 0 0) 0x7f6a8c155740 con 0x7f6a8c0bc890 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a780033f0 con 0x7f6a8c0bc890 2023-05-16T07:17:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 571776541 0 0) 0x7f6a780033f0 con 0x7f6a8c0bc890 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c150a90 msgr2=0x7f6a8c150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c150a90 0x7f6a8c150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 >> v2:172.21.15.158:3300/0 conn(0x7f6a8c0c2db0 msgr2=0x7f6a8c0bcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 --2- 172.21.15.92:0/570667838 >> v2:172.21.15.158:3300/0 conn(0x7f6a8c0c2db0 0x7f6a8c0bcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c154e10 con 0x7f6a8c0bc890 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6a88001470 con 0x7f6a8c0bc890 2023-05-16T07:17:34.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 >> v1:172.21.15.92:6789/0 conn(0x7f6a8c0bc890 legacy=0x7f6a8c0c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a780038e0 0x7f6a78003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a78004690 0x7f6a78004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78005000 0x7f6a7800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0bd200 con 0x7f6a78005000 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c154b30 con 0x7f6a780038e0 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7800bdd0 con 0x7f6a78004690 2023-05-16T07:17:34.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a922bf700 1 --2- 172.21.15.92:0/570667838 >> v2:172.21.15.158:3300/0 conn(0x7f6a8c0c2db0 0x7f6a8c0bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a91abe700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c150a90 0x7f6a8c150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a912bd700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a78004690 0x7f6a78004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a922bf700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78005000 0x7f6a7800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a91abe700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a780038e0 0x7f6a78003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a912bd700 1 -- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a780038e0 msgr2=0x7f6a78003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a912bd700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a780038e0 0x7f6a78003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a912bd700 1 -- 172.21.15.92:0/570667838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78005000 msgr2=0x7f6a7800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:34.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a912bd700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78005000 0x7f6a7800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a912bd700 1 -- 172.21.15.92:0/570667838 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780033f0 con 0x7f6a78004690 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.120+0000 7f6a922bf700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78005000 0x7f6a7800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a912bd700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a78004690 0x7f6a78004b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6a8800c1e0 tx=0x7f6a88001e90).ready entity=mon.2 client_cookie=2292593c61f8261a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88007a60 con 0x7f6a78004690 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8800cab0 con 0x7f6a78004690 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a90abc700 1 -- 172.21.15.92:0/570667838 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800ce50 con 0x7f6a78004690 2023-05-16T07:17:34.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a93d22700 1 -- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a78004690 msgr2=0x7f6a78004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a93d22700 1 --2- 172.21.15.92:0/570667838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a78004690 0x7f6a78004b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6a8800c1e0 tx=0x7f6a88001e90).stop 2023-05-16T07:17:34.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a922bf700 1 -- 172.21.15.92:0/570667838 reap_dead start 2023-05-16T07:17:34.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a93d22700 1 -- 172.21.15.92:0/570667838 shutdown_connections 2023-05-16T07:17:34.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a93d22700 1 -- 172.21.15.92:0/570667838 >> 172.21.15.92:0/570667838 conn(0x7f6a8c0c3d90 msgr2=0x7f6a8c0c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:34.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a93d22700 1 -- 172.21.15.92:0/570667838 shutdown_connections 2023-05-16T07:17:34.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.121+0000 7f6a93d22700 1 -- 172.21.15.92:0/570667838 wait complete. 2023-05-16T07:17:34.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.122+0000 7f6a93d22700 1 Processor -- start 2023-05-16T07:17:34.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.122+0000 7f6a93d22700 1 -- start start 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a93d22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 0x7f6a8c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a93d22700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c13e9a0 0x7f6a8c0bea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a93d22700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a8c153620 0x7f6a8c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a93d22700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0a6eb0 con 0x7f6a78004690 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a93d22700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c150e70 con 0x7f6a8c153620 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a93d22700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c151110 con 0x7f6a8c13e9a0 2023-05-16T07:17:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a91abe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 0x7f6a8c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a91abe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 0x7f6a8c152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45872/0 (socket says 172.21.15.92:45872) 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a91abe700 1 -- 172.21.15.92:0/2584631764 learned_addr learned my addr 172.21.15.92:0/2584631764 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a922bf700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a8c153620 0x7f6a8c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.123+0000 7f6a912bd700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c13e9a0 0x7f6a8c0bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a91abe700 1 -- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c13e9a0 msgr2=0x7f6a8c0bea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a91abe700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c13e9a0 0x7f6a8c0bea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a91abe700 1 -- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a8c153620 msgr2=0x7f6a8c153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a91abe700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a8c153620 0x7f6a8c153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a91abe700 1 -- 172.21.15.92:0/2584631764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c0a6580 con 0x7f6a78004690 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a91abe700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 0x7f6a8c152ef0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f6a7c011af0 tx=0x7f6a7c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00f8c0 con 0x7f6a78004690 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00fa20 con 0x7f6a78004690 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c0a6860 con 0x7f6a78004690 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00fda0 con 0x7f6a78004690 2023-05-16T07:17:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.124+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c0bef30 con 0x7f6a78004690 2023-05-16T07:17:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.125+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c13ed80 con 0x7f6a78004690 2023-05-16T07:17:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.125+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f6a7c024070 con 0x7f6a78004690 2023-05-16T07:17:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.126+0000 7f6a81ffb700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a7005ede0 0x7f6a700612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.126+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f6a7c07a720 con 0x7f6a78004690 2023-05-16T07:17:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.126+0000 7f6a912bd700 1 -- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a7005ede0 msgr2=0x7f6a700612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.126+0000 7f6a912bd700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a7005ede0 0x7f6a700612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.133+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6a7c04b900 con 0x7f6a78004690 2023-05-16T07:17:34.149 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:34.145+0000 7f079daea680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.303+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6a8c15dd80 con 0x7f6a78004690 2023-05-16T07:17:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.303+0000 7f6a81ffb700 1 -- 172.21.15.92:0/2584631764 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6a7c022070 con 0x7f6a78004690 2023-05-16T07:17:34.308 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:17:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a7005ede0 msgr2=0x7f6a700612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a7005ede0 0x7f6a700612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 msgr2=0x7f6a8c152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 0x7f6a8c152ef0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f6a7c011af0 tx=0x7f6a7c009060).stop 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 shutdown_connections 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6a7005ede0 0x7f6a700612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a8c153620 0x7f6a8c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a78004690 0x7f6a8c152ef0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 --2- 172.21.15.92:0/2584631764 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a8c13e9a0 0x7f6a8c0bea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 >> 172.21.15.92:0/2584631764 conn(0x7f6a8c0c3d90 msgr2=0x7f6a8c0bb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 shutdown_connections 2023-05-16T07:17:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.305+0000 7f6a93d22700 1 -- 172.21.15.92:0/2584631764 wait complete. 2023-05-16T07:17:34.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:17:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.437+0000 7f5494ab6700 1 Processor -- start 2023-05-16T07:17:34.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.437+0000 7f5494ab6700 1 -- start start 2023-05-16T07:17:34.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f5494ab6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5490153d00 0x7f5490063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f5494ab6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54901540e0 0x7f54900bf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f5494ab6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900a7c90 con 0x7f54900622b0 2023-05-16T07:17:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f5494ab6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5490067f60 con 0x7f54901540e0 2023-05-16T07:17:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f5494ab6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900a7f00 con 0x7f5490153d00 2023-05-16T07:17:34.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54901540e0 0x7f54900bf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54901540e0 0x7f54900bf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45914/0 (socket says 172.21.15.92:45914) 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 -- 172.21.15.92:0/2297649732 learned_addr learned my addr 172.21.15.92:0/2297649732 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548e59c700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5490153d00 0x7f5490063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 -- 172.21.15.92:0/2297649732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5490153d00 msgr2=0x7f5490063fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5490153d00 0x7f5490063fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 -- 172.21.15.92:0/2297649732 >> v1:172.21.15.158:6789/0 conn(0x7f54900622b0 legacy=0x7f5490157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 -- 172.21.15.92:0/2297649732 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900a81e0 con 0x7f54901540e0 2023-05-16T07:17:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548e59c700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5490153d00 0x7f5490063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.438+0000 7f548dd9b700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54901540e0 0x7f54900bf240 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f5480006fd0 tx=0x7f54800090f0).ready entity=mon.0 client_cookie=c246daf06a877124 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f548d59a700 1 -- 172.21.15.92:0/2297649732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000b600 con 0x7f54901540e0 2023-05-16T07:17:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f548d59a700 1 -- 172.21.15.92:0/2297649732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548000b760 con 0x7f54901540e0 2023-05-16T07:17:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f548d59a700 1 -- 172.21.15.92:0/2297649732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000bae0 con 0x7f54901540e0 2023-05-16T07:17:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 -- 172.21.15.92:0/2297649732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54901540e0 msgr2=0x7f54900bf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54901540e0 0x7f54900bf240 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f5480006fd0 tx=0x7f54800090f0).stop 2023-05-16T07:17:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 -- 172.21.15.92:0/2297649732 shutdown_connections 2023-05-16T07:17:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5490153d00 0x7f5490063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 --2- 172.21.15.92:0/2297649732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54901540e0 0x7f54900bf240 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 -- 172.21.15.92:0/2297649732 >> 172.21.15.92:0/2297649732 conn(0x7f549005e820 msgr2=0x7f54900c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 -- 172.21.15.92:0/2297649732 shutdown_connections 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.439+0000 7f5494ab6700 1 -- 172.21.15.92:0/2297649732 wait complete. 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.440+0000 7f5494ab6700 1 Processor -- start 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.440+0000 7f5494ab6700 1 -- start start 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f5494ab6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 0x7f54900aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f5494ab6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5490153d00 0x7f549014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f5494ab6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54901540e0 0x7f54900a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f5494ab6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54900a65d0 con 0x7f54900622b0 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f5494ab6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549014e7d0 con 0x7f5490153d00 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f5494ab6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900aa410 con 0x7f54901540e0 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f548e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 0x7f54900aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f548e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 0x7f54900aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45918/0 (socket says 172.21.15.92:45918) 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f548e59c700 1 -- 172.21.15.92:0/3757998667 learned_addr learned my addr 172.21.15.92:0/3757998667 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f548dd9b700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5490153d00 0x7f549014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.441+0000 7f548ed9d700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54901540e0 0x7f54900a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f548e59c700 1 -- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54901540e0 msgr2=0x7f54900a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f548e59c700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54901540e0 0x7f54900a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f548e59c700 1 -- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5490153d00 msgr2=0x7f549014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f548e59c700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5490153d00 0x7f549014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f548e59c700 1 -- 172.21.15.92:0/3757998667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009370 con 0x7f54900622b0 2023-05-16T07:17:34.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f548e59c700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 0x7f54900aae70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f548400d860 tx=0x7f5484008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548400f520 con 0x7f54900622b0 2023-05-16T07:17:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548400f680 con 0x7f54900622b0 2023-05-16T07:17:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54900a5ca0 con 0x7f54900622b0 2023-05-16T07:17:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548400fa00 con 0x7f54900622b0 2023-05-16T07:17:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.442+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54900a6260 con 0x7f54900622b0 2023-05-16T07:17:34.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.443+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549006c350 con 0x7f54900622b0 2023-05-16T07:17:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.443+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f548401a030 con 0x7f54900622b0 2023-05-16T07:17:34.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.444+0000 7f547e7fc700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f547005ede0 0x7f54700612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.444+0000 7f548dd9b700 1 -- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f547005ede0 msgr2=0x7f54700612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.444+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f5484079a40 con 0x7f54900622b0 2023-05-16T07:17:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.444+0000 7f548dd9b700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f547005ede0 0x7f54700612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:34.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.448+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f548404ac20 con 0x7f54900622b0 2023-05-16T07:17:34.513 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:34.509+0000 7f03e447c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:34.584 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:34.577+0000 7f079daea680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:34.613 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:34.610+0000 7f03e447c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.618+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5490064560 con 0x7f54900622b0 2023-05-16T07:17:34.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.618+0000 7f547e7fc700 1 -- 172.21.15.92:0/3757998667 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f548404e240 con 0x7f54900622b0 2023-05-16T07:17:34.624 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:17:34.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f547005ede0 msgr2=0x7f54700612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:34.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f547005ede0 0x7f54700612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:34.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 msgr2=0x7f54900aae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 0x7f54900aae70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f548400d860 tx=0x7f5484008040).stop 2023-05-16T07:17:34.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 shutdown_connections 2023-05-16T07:17:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f547005ede0 0x7f54700612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54901540e0 0x7f54900a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54900622b0 0x7f54900aae70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 --2- 172.21.15.92:0/3757998667 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5490153d00 0x7f549014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 >> 172.21.15.92:0/3757998667 conn(0x7f549005e820 msgr2=0x7f549014f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:34.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 shutdown_connections 2023-05-16T07:17:34.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.620+0000 7f5494ab6700 1 -- 172.21.15.92:0/3757998667 wait complete. 2023-05-16T07:17:34.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:17:34.691 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:34.686+0000 7f079daea680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.750+0000 7f412dfdb700 1 Processor -- start 2023-05-16T07:17:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.750+0000 7f412dfdb700 1 -- start start 2023-05-16T07:17:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.750+0000 7f412dfdb700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f412814e6f0 0x7f4128155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.750+0000 7f412dfdb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41280c4d40 0x7f41280622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f412dfdb700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412814ead0 con 0x7f41280627a0 2023-05-16T07:17:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f412dfdb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4128154cb0 con 0x7f41280c4d40 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f412dfdb700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280c6020 con 0x7f412814e6f0 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41280c4d40 0x7f41280622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41280c4d40 0x7f41280622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45926/0 (socket says 172.21.15.92:45926) 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 -- 172.21.15.92:0/2110185363 learned_addr learned my addr 172.21.15.92:0/2110185363 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f41277fe700 1 --2- 172.21.15.92:0/2110185363 >> v2:172.21.15.158:3300/0 conn(0x7f412814e6f0 0x7f4128155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 -- 172.21.15.92:0/2110185363 >> v2:172.21.15.158:3300/0 conn(0x7f412814e6f0 msgr2=0x7f4128155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 --2- 172.21.15.92:0/2110185363 >> v2:172.21.15.158:3300/0 conn(0x7f412814e6f0 0x7f4128155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 -- 172.21.15.92:0/2110185363 >> v1:172.21.15.158:6789/0 conn(0x7f41280627a0 legacy=0x7f4128063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 -- 172.21.15.92:0/2110185363 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128154660 con 0x7f41280c4d40 2023-05-16T07:17:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f4126ffd700 1 --2- 172.21.15.92:0/2110185363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f41280622b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f41280c9b00 tx=0x7f411c0090f0).ready entity=mon.0 client_cookie=323069e2c82e7b66 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f41267fc700 1 -- 172.21.15.92:0/2110185363 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00b520 con 0x7f41280c4d40 2023-05-16T07:17:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f41267fc700 1 -- 172.21.15.92:0/2110185363 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00b680 con 0x7f41280c4d40 2023-05-16T07:17:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.751+0000 7f41267fc700 1 -- 172.21.15.92:0/2110185363 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00ba00 con 0x7f41280c4d40 2023-05-16T07:17:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 -- 172.21.15.92:0/2110185363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 msgr2=0x7f41280622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 --2- 172.21.15.92:0/2110185363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f41280622b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f41280c9b00 tx=0x7f411c0090f0).stop 2023-05-16T07:17:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 -- 172.21.15.92:0/2110185363 shutdown_connections 2023-05-16T07:17:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 --2- 172.21.15.92:0/2110185363 >> v2:172.21.15.158:3300/0 conn(0x7f412814e6f0 0x7f4128155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 --2- 172.21.15.92:0/2110185363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f41280622b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 -- 172.21.15.92:0/2110185363 >> 172.21.15.92:0/2110185363 conn(0x7f412805e820 msgr2=0x7f41280c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 -- 172.21.15.92:0/2110185363 shutdown_connections 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.752+0000 7f412dfdb700 1 -- 172.21.15.92:0/2110185363 wait complete. 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.753+0000 7f412dfdb700 1 Processor -- start 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.753+0000 7f412dfdb700 1 -- start start 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f412dfdb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f41280627a0 0x7f41280aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f412dfdb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f412814c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f412dfdb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f412814e6f0 0x7f41280c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f412dfdb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280aa980 con 0x7f41280c4d40 2023-05-16T07:17:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f412dfdb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280bfe50 con 0x7f412814e6f0 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f412dfdb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41280a9d70 con 0x7f41280627a0 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f412814c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f412814c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45934/0 (socket says 172.21.15.92:45934) 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 -- 172.21.15.92:0/3413742029 learned_addr learned my addr 172.21.15.92:0/3413742029 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4127fff700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f412814e6f0 0x7f41280c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f41277fe700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f41280627a0 0x7f41280aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 -- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f41280627a0 msgr2=0x7f41280aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f41280627a0 0x7f41280aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 -- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f412814e6f0 msgr2=0x7f41280c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f412814e6f0 0x7f41280c0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.754+0000 7f4126ffd700 1 -- 172.21.15.92:0/3413742029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009370 con 0x7f41280c4d40 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.755+0000 7f4126ffd700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f412814c8c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f411c009130 tx=0x7f411c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.755+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c000bc0 con 0x7f41280c4d40 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.755+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00b7f0 con 0x7f41280c4d40 2023-05-16T07:17:34.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.755+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41280aa050 con 0x7f41280c4d40 2023-05-16T07:17:34.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.755+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c000d20 con 0x7f41280c4d40 2023-05-16T07:17:34.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.755+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41280aa610 con 0x7f41280c4d40 2023-05-16T07:17:34.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.756+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412806c350 con 0x7f41280c4d40 2023-05-16T07:17:34.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.756+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f411c02c070 con 0x7f41280c4d40 2023-05-16T07:17:34.766 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:34.761+0000 7f03e447c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:34.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.756+0000 7f410f7fe700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f411405ede0 0x7f41140612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.757+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f411c07bb90 con 0x7f41280c4d40 2023-05-16T07:17:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.757+0000 7f41277fe700 1 -- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f411405ede0 msgr2=0x7f41140612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.757+0000 7f41277fe700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f411405ede0 0x7f41140612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.761+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f411c04cd70 con 0x7f41280c4d40 2023-05-16T07:17:34.838 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:34.835+0000 7f03e447c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:34.848 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:34.843+0000 7f079daea680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:34.904 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:34.900+0000 7f03e447c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:34.923 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:34.918+0000 7f079daea680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:34.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.929+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41280c66a0 con 0x7f41280c4d40 2023-05-16T07:17:34.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.930+0000 7f410f7fe700 1 -- 172.21.15.92:0/3413742029 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f411c02a070 con 0x7f41280c4d40 2023-05-16T07:17:34.935 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:17:34.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f411405ede0 msgr2=0x7f41140612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:34.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f411405ede0 0x7f41140612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:34.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 msgr2=0x7f412814c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:34.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f412814c8c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f411c009130 tx=0x7f411c017000).stop 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 shutdown_connections 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f411405ede0 0x7f41140612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f41280627a0 0x7f41280aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f412814e6f0 0x7f41280c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 --2- 172.21.15.92:0/3413742029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41280c4d40 0x7f412814c8c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 >> 172.21.15.92:0/3413742029 conn(0x7f412805e820 msgr2=0x7f41280c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 shutdown_connections 2023-05-16T07:17:34.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:34.932+0000 7f412dfdb700 1 -- 172.21.15.92:0/3413742029 wait complete. 2023-05-16T07:17:34.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:34.995 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:34.990+0000 7f079daea680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.060+0000 7fd30ebde700 1 Processor -- start 2023-05-16T07:17:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.060+0000 7fd30ebde700 1 -- start start 2023-05-16T07:17:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.060+0000 7fd30ebde700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd308069560 0x7fd308065c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.060+0000 7fd30ebde700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3080a4890 con 0x7fd3081541f0 2023-05-16T07:17:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30ebde700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30806a650 con 0x7fd308065880 2023-05-16T07:17:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30ebde700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3080c2150 con 0x7fd308069560 2023-05-16T07:17:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30e3dd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd308069560 0x7fd308065c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30d3db700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd3081541f0 0x7fd3080c3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56690/0 (socket says 172.21.15.92:56690) 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30d3db700 1 -- 172.21.15.92:0/1986253096 learned_addr learned my addr 172.21.15.92:0/1986253096 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30e3dd700 1 -- 172.21.15.92:0/1986253096 >> v1:172.21.15.158:6789/0 conn(0x7fd308065880 legacy=0x7fd308153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30e3dd700 1 -- 172.21.15.92:0/1986253096 >> v1:172.21.15.92:6789/0 conn(0x7fd3081541f0 legacy=0x7fd3080c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30e3dd700 1 -- 172.21.15.92:0/1986253096 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308069ad0 con 0x7fd308069560 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30cbda700 1 -- 172.21.15.92:0/1986253096 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772418584 0 0) 0x7fd3080a4890 con 0x7fd3081541f0 2023-05-16T07:17:35.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30e3dd700 1 --2- 172.21.15.92:0/1986253096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd308069560 0x7fd308065c60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fd300009e60 tx=0x7fd300009780).ready entity=mon.0 client_cookie=7b397fc20b00fab8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30cbda700 1 -- 172.21.15.92:0/1986253096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000e070 con 0x7fd308069560 2023-05-16T07:17:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30cbda700 1 -- 172.21.15.92:0/1986253096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30000bc30 con 0x7fd308069560 2023-05-16T07:17:35.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.061+0000 7fd30cbda700 1 -- 172.21.15.92:0/1986253096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300012410 con 0x7fd308069560 2023-05-16T07:17:35.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 -- 172.21.15.92:0/1986253096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd308069560 msgr2=0x7fd308065c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 --2- 172.21.15.92:0/1986253096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd308069560 0x7fd308065c60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fd300009e60 tx=0x7fd300009780).stop 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 -- 172.21.15.92:0/1986253096 shutdown_connections 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 --2- 172.21.15.92:0/1986253096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd308069560 0x7fd308065c60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 -- 172.21.15.92:0/1986253096 >> 172.21.15.92:0/1986253096 conn(0x7fd308067f60 msgr2=0x7fd308068370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 -- 172.21.15.92:0/1986253096 shutdown_connections 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.062+0000 7fd30ebde700 1 -- 172.21.15.92:0/1986253096 wait complete. 2023-05-16T07:17:35.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.063+0000 7fd30ebde700 1 Processor -- start 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.063+0000 7fd30ebde700 1 -- start start 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd308065880 0x7fd3080c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd308069560 0x7fd308150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 0x7fd3080bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080aac40 con 0x7fd3081541f0 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080aaf20 con 0x7fd308065880 2023-05-16T07:17:35.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080ab200 con 0x7fd308069560 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 0x7fd3080bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 0x7fd3080bffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45950/0 (socket says 172.21.15.92:45950) 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 -- 172.21.15.92:0/113283241 learned_addr learned my addr 172.21.15.92:0/113283241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30d3db700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd308069560 0x7fd308150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30dbdc700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd308065880 0x7fd3080c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 -- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd308069560 msgr2=0x7fd308150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd308069560 0x7fd308150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 -- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd308065880 msgr2=0x7fd3080c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd308065880 0x7fd3080c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 -- 172.21.15.92:0/113283241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300009a20 con 0x7fd3081541f0 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30d3db700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd308069560 0x7fd308150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30dbdc700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd308065880 0x7fd3080c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30e3dd700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 0x7fd3080bffe0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fd30000b6c0 tx=0x7fd300009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300006590 con 0x7fd3081541f0 2023-05-16T07:17:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000066f0 con 0x7fd3081541f0 2023-05-16T07:17:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3080ab4e0 con 0x7fd3081541f0 2023-05-16T07:17:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.064+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300006a90 con 0x7fd3081541f0 2023-05-16T07:17:35.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.065+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3080aa860 con 0x7fd3081541f0 2023-05-16T07:17:35.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.065+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3080ab980 con 0x7fd3081541f0 2023-05-16T07:17:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.066+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fd30001d630 con 0x7fd3081541f0 2023-05-16T07:17:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.066+0000 7fd2fdffb700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 0x7fd2ec0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.066+0000 7fd30dbdc700 1 -- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 msgr2=0x7fd2ec0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.066+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fd30007c770 con 0x7fd3081541f0 2023-05-16T07:17:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.067+0000 7fd30dbdc700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 0x7fd2ec0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.070+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd30004d840 con 0x7fd3081541f0 2023-05-16T07:17:35.161 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:35.157+0000 7f03e447c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:35.263 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:35.258+0000 7f079daea680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:35.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.268+0000 7fd30dbdc700 1 -- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 msgr2=0x7fd2ec0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:35.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.268+0000 7fd30dbdc700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 0x7fd2ec0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.285+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd308069db0 con 0x7fd3081541f0 2023-05-16T07:17:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.285+0000 7fd2fdffb700 1 -- 172.21.15.92:0/113283241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fd300050e60 con 0x7fd3081541f0 2023-05-16T07:17:35.290 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:35.291 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:19.493924+0000","modified":"2023-05-16T07:17:21.499636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1922547921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1922547921},{"type":"v1","addr":"172.21.15.92:6835","nonce":1922547921}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-05-16T07:17:35.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 msgr2=0x7fd2ec0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:35.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 0x7fd2ec0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:35.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 msgr2=0x7fd3080bffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 0x7fd3080bffe0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fd30000b6c0 tx=0x7fd300009780).stop 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 shutdown_connections 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2ec05ede0 0x7fd2ec0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3081541f0 0x7fd3080bffe0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd308065880 0x7fd3080c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 --2- 172.21.15.92:0/113283241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd308069560 0x7fd308150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 >> 172.21.15.92:0/113283241 conn(0x7fd308067f60 msgr2=0x7fd3080c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 shutdown_connections 2023-05-16T07:17:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.287+0000 7fd30ebde700 1 -- 172.21.15.92:0/113283241 wait complete. 2023-05-16T07:17:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-05-16T07:17:35.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:35.356 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:35.353+0000 7f03e447c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.419+0000 7f4c4dc8c700 1 Processor -- start 2023-05-16T07:17:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.419+0000 7f4c4dc8c700 1 -- start start 2023-05-16T07:17:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c4dc8c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4c4814e5e0 0x7f4c48157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c4dc8c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c48157f40 0x7f4c4815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c4dc8c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480683d0 con 0x7f4c4814e200 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c4dc8c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c48137470 con 0x7f4c48157f40 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c4dc8c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c480c3520 con 0x7f4c4814e5e0 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c48157f40 0x7f4c4815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c477fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4c4814e200 0x7f4c48066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56704/0 (socket says 172.21.15.92:56704) 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c477fe700 1 -- 172.21.15.92:0/2943726415 learned_addr learned my addr 172.21.15.92:0/2943726415 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c3effd700 1 --2- 172.21.15.92:0/2943726415 >> v2:172.21.15.158:3300/0 conn(0x7f4c4814e5e0 0x7f4c48157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c47fff700 1 -- 172.21.15.92:0/2943726415 >> v2:172.21.15.158:3300/0 conn(0x7f4c4814e5e0 msgr2=0x7f4c48157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:35.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c47fff700 1 --2- 172.21.15.92:0/2943726415 >> v2:172.21.15.158:3300/0 conn(0x7f4c4814e5e0 0x7f4c48157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c47fff700 1 -- 172.21.15.92:0/2943726415 >> v1:172.21.15.92:6789/0 conn(0x7f4c4814e200 legacy=0x7f4c48066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c46ffd700 1 -- 172.21.15.92:0/2943726415 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642705502 0 0) 0x7f4c480683d0 con 0x7f4c4814e200 2023-05-16T07:17:35.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c47fff700 1 -- 172.21.15.92:0/2943726415 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480be060 con 0x7f4c48157f40 2023-05-16T07:17:35.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.420+0000 7f4c47fff700 1 --2- 172.21.15.92:0/2943726415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4815ddf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f4c480a6be0 tx=0x7f4c40009780).ready entity=mon.0 client_cookie=77f9c9f900909049 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:35.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c46ffd700 1 -- 172.21.15.92:0/2943726415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000e070 con 0x7f4c48157f40 2023-05-16T07:17:35.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c46ffd700 1 -- 172.21.15.92:0/2943726415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4000bbf0 con 0x7f4c48157f40 2023-05-16T07:17:35.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c46ffd700 1 -- 172.21.15.92:0/2943726415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40012410 con 0x7f4c48157f40 2023-05-16T07:17:35.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2943726415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 msgr2=0x7f4c4815ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2943726415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4815ddf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f4c480a6be0 tx=0x7f4c40009780).stop 2023-05-16T07:17:35.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2943726415 shutdown_connections 2023-05-16T07:17:35.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2943726415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4815ddf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2943726415 >> v2:172.21.15.158:3300/0 conn(0x7f4c4814e5e0 0x7f4c48157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2943726415 >> 172.21.15.92:0/2943726415 conn(0x7f4c480c7bc0 msgr2=0x7f4c480c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:35.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2943726415 shutdown_connections 2023-05-16T07:17:35.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.421+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2943726415 wait complete. 2023-05-16T07:17:35.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.422+0000 7f4c4dc8c700 1 Processor -- start 2023-05-16T07:17:35.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.422+0000 7f4c4dc8c700 1 -- start start 2023-05-16T07:17:35.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c4dc8c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c4814e200 0x7f4c48135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c4dc8c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c4814e5e0 0x7f4c4813aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c4dc8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4813d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c4dc8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48135840 con 0x7f4c48157f40 2023-05-16T07:17:35.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c4dc8c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4813b4d0 con 0x7f4c4814e5e0 2023-05-16T07:17:35.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c4dc8c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4813d7e0 con 0x7f4c4814e200 2023-05-16T07:17:35.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4813d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c47fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4813d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45966/0 (socket says 172.21.15.92:45966) 2023-05-16T07:17:35.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c47fff700 1 -- 172.21.15.92:0/2657905731 learned_addr learned my addr 172.21.15.92:0/2657905731 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:35.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c3effd700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c4814e5e0 0x7f4c4813aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.423+0000 7f4c477fe700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c4814e200 0x7f4c48135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c47fff700 1 -- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c4814e200 msgr2=0x7f4c48135ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c47fff700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c4814e200 0x7f4c48135ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c47fff700 1 -- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c4814e5e0 msgr2=0x7f4c4813aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c47fff700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c4814e5e0 0x7f4c4813aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c47fff700 1 -- 172.21.15.92:0/2657905731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c40009a20 con 0x7f4c48157f40 2023-05-16T07:17:35.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c47fff700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4813d170 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f4c4000b6c0 tx=0x7f4c40009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:35.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4001d630 con 0x7f4c48157f40 2023-05-16T07:17:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4001d790 con 0x7f4c48157f40 2023-05-16T07:17:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4813b720 con 0x7f4c48157f40 2023-05-16T07:17:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4001db10 con 0x7f4c48157f40 2023-05-16T07:17:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.424+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c48135c40 con 0x7f4c48157f40 2023-05-16T07:17:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.425+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c48064240 con 0x7f4c48157f40 2023-05-16T07:17:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.425+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f4c40026410 con 0x7f4c48157f40 2023-05-16T07:17:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.426+0000 7f4c3ffff700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4c2805ede0 0x7f4c280612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.426+0000 7f4c477fe700 1 -- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4c2805ede0 msgr2=0x7f4c280612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.426+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f4c4007c310 con 0x7f4c48157f40 2023-05-16T07:17:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.426+0000 7f4c477fe700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4c2805ede0 0x7f4c280612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:35.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.430+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4c4004d490 con 0x7f4c48157f40 2023-05-16T07:17:35.453 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:35.448+0000 7f079daea680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:35.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.603+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c4806c350 con 0x7f4c48157f40 2023-05-16T07:17:35.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.604+0000 7f4c3ffff700 1 -- 172.21.15.92:0/2657905731 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18107 (secure 0 0 0) 0x7f4c40050ab0 con 0x7f4c48157f40 2023-05-16T07:17:35.609 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:35.609 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":80,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:30.702127+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:18.044089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:18.464963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3054196167":"2023-05-17T07:17:30.702082+0000","172.21.15.158:0/2925301946":"2023-05-17T07:17:30.702082+0000","172.21.15.158:6833/86301":"2023-05-17T07:17:30.702082+0000","172.21.15.158:6832/86301":"2023-05-17T07:17:30.702082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:35.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4c2805ede0 msgr2=0x7f4c280612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:35.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4c2805ede0 0x7f4c280612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:35.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 msgr2=0x7f4c4813d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4813d170 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f4c4000b6c0 tx=0x7f4c40009780).stop 2023-05-16T07:17:35.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 shutdown_connections 2023-05-16T07:17:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4c2805ede0 0x7f4c280612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4c4814e200 0x7f4c48135ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c48157f40 0x7f4c4813d170 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 --2- 172.21.15.92:0/2657905731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c4814e5e0 0x7f4c4813aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 >> 172.21.15.92:0/2657905731 conn(0x7f4c480c7bc0 msgr2=0x7f4c48066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 shutdown_connections 2023-05-16T07:17:35.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.606+0000 7f4c4dc8c700 1 -- 172.21.15.92:0/2657905731 wait complete. 2023-05-16T07:17:35.623 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:17:35.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.736+0000 7f6e6f59e700 1 Processor -- start 2023-05-16T07:17:35.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.736+0000 7f6e6f59e700 1 -- start start 2023-05-16T07:17:35.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e700c76e0 0x7f6e700c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6f59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6e700a9b70 0x7f6e700a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e70158900 con 0x7f6e70153820 2023-05-16T07:17:35.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6f59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e70157cf0 con 0x7f6e700a9b70 2023-05-16T07:17:35.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70157fd0 con 0x7f6e700c76e0 2023-05-16T07:17:35.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6e70153820 0x7f6e70153c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56722/0 (socket says 172.21.15.92:56722) 2023-05-16T07:17:35.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6ed9d700 1 -- 172.21.15.92:0/2824814439 learned_addr learned my addr 172.21.15.92:0/2824814439 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:35.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6e59c700 1 --2- 172.21.15.92:0/2824814439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e700c76e0 0x7f6e700c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6dd9b700 1 --2- 172.21.15.92:0/2824814439 >> v2:172.21.15.158:3300/0 conn(0x7f6e700a9b70 0x7f6e700a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6d59a700 1 -- 172.21.15.92:0/2824814439 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146402797 0 0) 0x7f6e70158900 con 0x7f6e70153820 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.737+0000 7f6e6d59a700 1 -- 172.21.15.92:0/2824814439 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e580033f0 con 0x7f6e70153820 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6e59c700 1 -- 172.21.15.92:0/2824814439 >> v2:172.21.15.158:3300/0 conn(0x7f6e700a9b70 msgr2=0x7f6e700a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6e59c700 1 --2- 172.21.15.92:0/2824814439 >> v2:172.21.15.158:3300/0 conn(0x7f6e700a9b70 0x7f6e700a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6e59c700 1 -- 172.21.15.92:0/2824814439 >> v1:172.21.15.92:6789/0 conn(0x7f6e70153820 legacy=0x7f6e70153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6d59a700 1 -- 172.21.15.92:0/2824814439 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1537991142 0 0) 0x7f6e580033f0 con 0x7f6e70153820 2023-05-16T07:17:35.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6e59c700 1 -- 172.21.15.92:0/2824814439 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e701582b0 con 0x7f6e700c76e0 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6dd9b700 1 --2- 172.21.15.92:0/2824814439 >> v2:172.21.15.158:3300/0 conn(0x7f6e700a9b70 0x7f6e700a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6e59c700 1 --2- 172.21.15.92:0/2824814439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e700c76e0 0x7f6e700c7ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f6e60009760 tx=0x7f6e60009060).ready entity=mon.2 client_cookie=9375b00d591e2f34 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6d59a700 1 -- 172.21.15.92:0/2824814439 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680017f0 con 0x7f6e700c76e0 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6d59a700 1 -- 172.21.15.92:0/2824814439 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60011400 con 0x7f6e700c76e0 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6d59a700 1 -- 172.21.15.92:0/2824814439 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60011780 con 0x7f6e700c76e0 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6f59e700 1 -- 172.21.15.92:0/2824814439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e700c76e0 msgr2=0x7f6e700c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.738+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/2824814439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e700c76e0 0x7f6e700c7ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f6e60009760 tx=0x7f6e60009060).stop 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.739+0000 7f6e6f59e700 1 -- 172.21.15.92:0/2824814439 shutdown_connections 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.739+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/2824814439 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e700c76e0 0x7f6e700c7ac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.739+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/2824814439 >> v2:172.21.15.158:3300/0 conn(0x7f6e700a9b70 0x7f6e700a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.739+0000 7f6e6f59e700 1 -- 172.21.15.92:0/2824814439 >> 172.21.15.92:0/2824814439 conn(0x7f6e70068330 msgr2=0x7f6e70068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.739+0000 7f6e6f59e700 1 -- 172.21.15.92:0/2824814439 shutdown_connections 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.739+0000 7f6e6f59e700 1 -- 172.21.15.92:0/2824814439 wait complete. 2023-05-16T07:17:35.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 Processor -- start 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 -- start start 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 0x7f6e701444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e700c76e0 0x7f6e700c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e70153820 0x7f6e700c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c9c00 con 0x7f6e700a9b70 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c2ad0 con 0x7f6e700c76e0 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c1ec0 con 0x7f6e70153820 2023-05-16T07:17:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 0x7f6e701444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.740+0000 7f6e6e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 0x7f6e701444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45976/0 (socket says 172.21.15.92:45976) 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 -- 172.21.15.92:0/69878648 learned_addr learned my addr 172.21.15.92:0/69878648 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6ed9d700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e70153820 0x7f6e700c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6dd9b700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e700c76e0 0x7f6e700c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 -- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e70153820 msgr2=0x7f6e700c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e70153820 0x7f6e700c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 -- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e700c76e0 msgr2=0x7f6e700c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e700c76e0 0x7f6e700c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 -- 172.21.15.92:0/69878648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640092a0 con 0x7f6e700a9b70 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6e59c700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 0x7f6e701444b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f6e60009d50 tx=0x7f6e60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600106c0 con 0x7f6e700a9b70 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60010820 con 0x7f6e700a9b70 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e600092a0 con 0x7f6e700a9b70 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60010be0 con 0x7f6e700a9b70 2023-05-16T07:17:35.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.741+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e700c2480 con 0x7f6e700a9b70 2023-05-16T07:17:35.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.742+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e700c62b0 con 0x7f6e700a9b70 2023-05-16T07:17:35.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.742+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f6e60024030 con 0x7f6e700a9b70 2023-05-16T07:17:35.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.743+0000 7f6e5e7fc700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 0x7f6e500612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:35.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.743+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f6e6007a9d0 con 0x7f6e700a9b70 2023-05-16T07:17:35.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.743+0000 7f6e6dd9b700 1 -- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 msgr2=0x7f6e500612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:35.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.743+0000 7f6e6dd9b700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 0x7f6e500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:35.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.750+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6e6004bbb0 con 0x7f6e700a9b70 2023-05-16T07:17:35.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.944+0000 7f6e6dd9b700 1 -- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 msgr2=0x7f6e500612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:35.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.944+0000 7f6e6dd9b700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 0x7f6e500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:35.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.967+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e70158590 con 0x7f6e700a9b70 2023-05-16T07:17:35.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.967+0000 7f6e5e7fc700 1 -- 172.21.15.92:0/69878648 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f6e60022070 con 0x7f6e700a9b70 2023-05-16T07:17:35.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:35.973 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:19.493924+0000","modified":"2023-05-16T07:17:21.499636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1922547921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1922547921},{"type":"v1","addr":"172.21.15.92:6835","nonce":1922547921}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-05-16T07:17:35.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 msgr2=0x7f6e500612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:35.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 0x7f6e500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 msgr2=0x7f6e701444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 0x7f6e701444b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f6e60009d50 tx=0x7f6e60016000).stop 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 shutdown_connections 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e5005ede0 0x7f6e500612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.969+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e70153820 0x7f6e700c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.970+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e700a9b70 0x7f6e701444b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.970+0000 7f6e6f59e700 1 --2- 172.21.15.92:0/69878648 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e700c76e0 0x7f6e700c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.970+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 >> 172.21.15.92:0/69878648 conn(0x7f6e70068330 msgr2=0x7f6e700acac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.970+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 shutdown_connections 2023-05-16T07:17:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:35.970+0000 7f6e6f59e700 1 -- 172.21.15.92:0/69878648 wait complete. 2023-05-16T07:17:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-05-16T07:17:35.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:36.000 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:35.997+0000 7f03e447c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:36.067 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.063+0000 7f03e447c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.099+0000 7f7b81772700 1 Processor -- start 2023-05-16T07:17:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.099+0000 7f7b81772700 1 -- start start 2023-05-16T07:17:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b81772700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 0x7f7b7c0c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b81772700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c154c30 0x7f7b7c0bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b81772700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0c2240 con 0x7f7b7c1547b0 2023-05-16T07:17:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b81772700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7c0c31a0 con 0x7f7b7c150ac0 2023-05-16T07:17:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b81772700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c1550c0 con 0x7f7b7c154c30 2023-05-16T07:17:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7affd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 0x7f7b7c0c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7b7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c154c30 0x7f7b7c0bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7affd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 0x7f7b7c0c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:49666/0 (socket says 172.21.15.92:49666) 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7affd700 1 -- 172.21.15.92:0/857542272 learned_addr learned my addr 172.21.15.92:0/857542272 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7a7fc700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f7b7c1547b0 0x7f7b7c0bc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:60378/0 (socket says 172.21.15.92:60378) 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/857542272 >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 msgr2=0x7f7b7c0c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7b7fe700 1 --2- 172.21.15.92:0/857542272 >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 0x7f7b7c0c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/857542272 >> v1:172.21.15.158:6789/0 conn(0x7f7b7c1547b0 legacy=0x7f7b7c0bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b79ffb700 1 -- 172.21.15.92:0/857542272 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432191404 0 0) 0x7f7b7c0c2240 con 0x7f7b7c1547b0 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/857542272 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0c4680 con 0x7f7b7c154c30 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.101+0000 7f7b7affd700 1 --2- 172.21.15.92:0/857542272 >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 0x7f7b7c0c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b7b7fe700 1 --2- 172.21.15.92:0/857542272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c154c30 0x7f7b7c0bc760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f7b74009e60 tx=0x7f7b74009780).ready entity=mon.2 client_cookie=c3d9ca460f79a39e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b79ffb700 1 -- 172.21.15.92:0/857542272 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400e070 con 0x7f7b7c154c30 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b79ffb700 1 -- 172.21.15.92:0/857542272 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7400bb60 con 0x7f7b7c154c30 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b79ffb700 1 -- 172.21.15.92:0/857542272 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74012410 con 0x7f7b7c154c30 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 -- 172.21.15.92:0/857542272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c154c30 msgr2=0x7f7b7c0bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 --2- 172.21.15.92:0/857542272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c154c30 0x7f7b7c0bc760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f7b74009e60 tx=0x7f7b74009780).stop 2023-05-16T07:17:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 -- 172.21.15.92:0/857542272 shutdown_connections 2023-05-16T07:17:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 --2- 172.21.15.92:0/857542272 >> v2:172.21.15.158:3300/0 conn(0x7f7b7c150ac0 0x7f7b7c0c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 --2- 172.21.15.92:0/857542272 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c154c30 0x7f7b7c0bc760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 -- 172.21.15.92:0/857542272 >> 172.21.15.92:0/857542272 conn(0x7f7b7c0c7bc0 msgr2=0x7f7b7c0c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 -- 172.21.15.92:0/857542272 shutdown_connections 2023-05-16T07:17:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.102+0000 7f7b81772700 1 -- 172.21.15.92:0/857542272 wait complete. 2023-05-16T07:17:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.103+0000 7f7b81772700 1 Processor -- start 2023-05-16T07:17:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 -- start start 2023-05-16T07:17:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b7c150ac0 0x7f7b7c0ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c1547b0 0x7f7b7c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 0x7f7b7c160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c0aa080 con 0x7f7b7c154c30 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c0aa920 con 0x7f7b7c150ac0 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b81772700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c160da0 con 0x7f7b7c1547b0 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 0x7f7b7c160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 0x7f7b7c160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46038/0 (socket says 172.21.15.92:46038) 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/1700100053 learned_addr learned my addr 172.21.15.92:0/1700100053 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b7affd700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b7c150ac0 0x7f7b7c0ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.104+0000 7f7b7a7fc700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c1547b0 0x7f7b7c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c1547b0 msgr2=0x7f7b7c1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b7b7fe700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c1547b0 0x7f7b7c1608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b7c150ac0 msgr2=0x7f7b7c0ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b7b7fe700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b7c150ac0 0x7f7b7c0ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b7b7fe700 1 -- 172.21.15.92:0/1700100053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009370 con 0x7f7b7c154c30 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b7b7fe700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 0x7f7b7c160160 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f7b740094c0 tx=0x7f7b74009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74012410 con 0x7f7b7c154c30 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7401d630 con 0x7f7b7c154c30 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b74009a20 con 0x7f7b7c154c30 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7401d9f0 con 0x7f7b7c154c30 2023-05-16T07:17:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.105+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c0aa2d0 con 0x7f7b7c154c30 2023-05-16T07:17:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.106+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7c06c350 con 0x7f7b7c154c30 2023-05-16T07:17:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.106+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f7b7401dd60 con 0x7f7b7c154c30 2023-05-16T07:17:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.107+0000 7f7b62ffd700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 0x7f7b680612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.107+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f7b7407c100 con 0x7f7b7c154c30 2023-05-16T07:17:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.107+0000 7f7b7affd700 1 -- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 msgr2=0x7f7b680612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.107+0000 7f7b7affd700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 0x7f7b680612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.111+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7b7404d2e0 con 0x7f7b7c154c30 2023-05-16T07:17:36.131 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.126+0000 7f079daea680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:36.140 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.137+0000 7f03e447c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:36.200 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.196+0000 7f079daea680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:36.275 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.270+0000 7f079daea680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:36.283 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.279+0000 7f03e447c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:36.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.308+0000 7f7b7affd700 1 -- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 msgr2=0x7f7b680612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:36.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.308+0000 7f7b7affd700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 0x7f7b680612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.322+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b7c0c24b0 con 0x7f7b7c154c30 2023-05-16T07:17:36.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.322+0000 7f7b62ffd700 1 -- 172.21.15.92:0/1700100053 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f7b74050900 con 0x7f7b7c154c30 2023-05-16T07:17:36.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:36.328 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:19.493924+0000","modified":"2023-05-16T07:17:21.499636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1922547921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1922547921},{"type":"v1","addr":"172.21.15.92:6835","nonce":1922547921}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-05-16T07:17:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.324+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 msgr2=0x7f7b680612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.324+0000 7f7b81772700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 0x7f7b680612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 msgr2=0x7f7b7c160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 0x7f7b7c160160 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f7b740094c0 tx=0x7f7b74009780).stop 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 shutdown_connections 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7b6805ede0 0x7f7b680612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b7c154c30 0x7f7b7c160160 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b7c150ac0 0x7f7b7c0ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 --2- 172.21.15.92:0/1700100053 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b7c1547b0 0x7f7b7c1608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 >> 172.21.15.92:0/1700100053 conn(0x7f7b7c0c7bc0 msgr2=0x7f7b7c0a8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 shutdown_connections 2023-05-16T07:17:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.325+0000 7f7b81772700 1 -- 172.21.15.92:0/1700100053 wait complete. 2023-05-16T07:17:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-05-16T07:17:36.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:36.351 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.348+0000 7f03e447c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:36.426 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.421+0000 7f079daea680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:36.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.452+0000 7f0f2c5f7700 1 Processor -- start 2023-05-16T07:17:36.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.452+0000 7f0f2c5f7700 1 -- start start 2023-05-16T07:17:36.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2c5f7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f240c63c0 0x7f0f240aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2c5f7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f240c08b0 con 0x7f0f24154550 2023-05-16T07:17:36.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2c5f7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f240bfca0 con 0x7f0f24154170 2023-05-16T07:17:36.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2c5f7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f240ab200 con 0x7f0f240c63c0 2023-05-16T07:17:36.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f240c63c0 0x7f0f240aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f240c63c0 0x7f0f240aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46048/0 (socket says 172.21.15.92:46048) 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f29b92700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0f24154550 0x7f0f240c5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56792/0 (socket says 172.21.15.92:56792) 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 -- 172.21.15.92:0/3817106266 learned_addr learned my addr 172.21.15.92:0/3817106266 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 -- 172.21.15.92:0/3817106266 >> v1:172.21.15.158:6789/0 conn(0x7f0f24154170 legacy=0x7f0f240a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 -- 172.21.15.92:0/3817106266 >> v1:172.21.15.92:6789/0 conn(0x7f0f24154550 legacy=0x7f0f240c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f29391700 1 -- 172.21.15.92:0/3817106266 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397896475 0 0) 0x7f0f240c08b0 con 0x7f0f24154550 2023-05-16T07:17:36.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 -- 172.21.15.92:0/3817106266 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f240bff80 con 0x7f0f240c63c0 2023-05-16T07:17:36.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f2ab94700 1 --2- 172.21.15.92:0/3817106266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f240c63c0 0x7f0f240aad10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f0f1c009f20 tx=0x7f0f1c009840).ready entity=mon.0 client_cookie=e0b645b4ba5f3c4f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:36.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f29391700 1 -- 172.21.15.92:0/3817106266 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00e070 con 0x7f0f240c63c0 2023-05-16T07:17:36.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f29391700 1 -- 172.21.15.92:0/3817106266 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00ba80 con 0x7f0f240c63c0 2023-05-16T07:17:36.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.453+0000 7f0f29391700 1 -- 172.21.15.92:0/3817106266 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00be00 con 0x7f0f240c63c0 2023-05-16T07:17:36.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/3817106266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f240c63c0 msgr2=0x7f0f240aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/3817106266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f240c63c0 0x7f0f240aad10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f0f1c009f20 tx=0x7f0f1c009840).stop 2023-05-16T07:17:36.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/3817106266 shutdown_connections 2023-05-16T07:17:36.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/3817106266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f240c63c0 0x7f0f240aad10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/3817106266 >> 172.21.15.92:0/3817106266 conn(0x7f0f240cb6d0 msgr2=0x7f0f240c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:36.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/3817106266 shutdown_connections 2023-05-16T07:17:36.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.454+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/3817106266 wait complete. 2023-05-16T07:17:36.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.455+0000 7f0f2c5f7700 1 Processor -- start 2023-05-16T07:17:36.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.455+0000 7f0f2c5f7700 1 -- start start 2023-05-16T07:17:36.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2c5f7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f240c63c0 0x7f0f2413f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2c5f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 0x7f0f240c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2c5f7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f24154550 0x7f0f240c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2c5f7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240ca000 con 0x7f0f24154170 2023-05-16T07:17:36.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2c5f7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24167a10 con 0x7f0f240c63c0 2023-05-16T07:17:36.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2c5f7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24168270 con 0x7f0f24154550 2023-05-16T07:17:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 0x7f0f240c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 0x7f0f240c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46052/0 (socket says 172.21.15.92:46052) 2023-05-16T07:17:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 -- 172.21.15.92:0/4190146295 learned_addr learned my addr 172.21.15.92:0/4190146295 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2a393700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f240c63c0 0x7f0f2413f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f2ab94700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f24154550 0x7f0f240c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 -- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f24154550 msgr2=0x7f0f240c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f24154550 0x7f0f240c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 -- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f240c63c0 msgr2=0x7f0f2413f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f240c63c0 0x7f0f2413f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 -- 172.21.15.92:0/4190146295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c009ae0 con 0x7f0f24154170 2023-05-16T07:17:36.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.456+0000 7f0f29b92700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 0x7f0f240c9620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f0f1400c490 tx=0x7f0f14007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:36.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.457+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f140070d0 con 0x7f0f24154170 2023-05-16T07:17:36.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.457+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14007250 con 0x7f0f24154170 2023-05-16T07:17:36.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.457+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f241651a0 con 0x7f0f24154170 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.457+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f140056e0 con 0x7f0f24154170 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.457+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f24167f00 con 0x7f0f24154170 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.457+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f24168420 con 0x7f0f24154170 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.458+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f0f1401a060 con 0x7f0f24154170 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.458+0000 7f0f1a7fc700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f0805ede0 0x7f0f080612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.458+0000 7f0f2a393700 1 -- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f0805ede0 msgr2=0x7f0f080612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.458+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f0f14079760 con 0x7f0f24154170 2023-05-16T07:17:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.458+0000 7f0f2a393700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f0805ede0 0x7f0f080612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:36.475 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.463+0000 7f03e447c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.466+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0f24168420 con 0x7f0f24154170 2023-05-16T07:17:36.502 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.497+0000 7f079daea680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:36.572 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.568+0000 7f03e447c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:36.617 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.612+0000 7f079daea680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.633+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f240666c0 con 0x7f0f24154170 2023-05-16T07:17:36.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.634+0000 7f0f1a7fc700 1 -- 172.21.15.92:0/4190146295 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18107 (secure 0 0 0) 0x7f0f14018020 con 0x7f0f24154170 2023-05-16T07:17:36.639 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:36.639 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":80,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:30.702127+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:18.044089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:18.464963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3054196167":"2023-05-17T07:17:30.702082+0000","172.21.15.158:0/2925301946":"2023-05-17T07:17:30.702082+0000","172.21.15.158:6833/86301":"2023-05-17T07:17:30.702082+0000","172.21.15.158:6832/86301":"2023-05-17T07:17:30.702082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:36.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f0805ede0 msgr2=0x7f0f080612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:36.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f0805ede0 0x7f0f080612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:36.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 msgr2=0x7f0f240c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 0x7f0f240c9620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f0f1400c490 tx=0x7f0f14007db0).stop 2023-05-16T07:17:36.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 shutdown_connections 2023-05-16T07:17:36.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f0805ede0 0x7f0f080612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:36.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f24154550 0x7f0f240c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f240c63c0 0x7f0f2413f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 --2- 172.21.15.92:0/4190146295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f24154170 0x7f0f240c9620 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 >> 172.21.15.92:0/4190146295 conn(0x7f0f240cb6d0 msgr2=0x7f0f240ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:36.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 shutdown_connections 2023-05-16T07:17:36.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.636+0000 7f0f2c5f7700 1 -- 172.21.15.92:0/4190146295 wait complete. 2023-05-16T07:17:36.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:17:36.728 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:36.724+0000 7f079daea680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:36.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.763+0000 7f2c73d75700 1 Processor -- start 2023-05-16T07:17:36.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.763+0000 7f2c73d75700 1 -- start start 2023-05-16T07:17:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.764+0000 7f2c73d75700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c6c153eb0 0x7f2c6c154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c73d75700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c068c70 0x7f2c6c066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c73d75700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2c6c065000 0x7f2c6c0654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c73d75700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0659a0 con 0x7f2c6c153eb0 2023-05-16T07:17:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c73d75700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0bc9b0 con 0x7f2c6c065000 2023-05-16T07:17:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c73d75700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0bcc90 con 0x7f2c6c068c70 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c6c153eb0 0x7f2c6c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c6c153eb0 0x7f2c6c154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46058/0 (socket says 172.21.15.92:46058) 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 -- 172.21.15.92:0/3033057261 learned_addr learned my addr 172.21.15.92:0/3033057261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71310700 1 --2- 172.21.15.92:0/3033057261 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c068c70 0x7f2c6c066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c72312700 1 --2- 172.21.15.92:0/3033057261 >> v2:172.21.15.158:3300/0 conn(0x7f2c6c065000 0x7f2c6c0654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 -- 172.21.15.92:0/3033057261 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c068c70 msgr2=0x7f2c6c066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 --2- 172.21.15.92:0/3033057261 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c068c70 0x7f2c6c066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 -- 172.21.15.92:0/3033057261 >> v2:172.21.15.158:3300/0 conn(0x7f2c6c065000 msgr2=0x7f2c6c0654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 --2- 172.21.15.92:0/3033057261 >> v2:172.21.15.158:3300/0 conn(0x7f2c6c065000 0x7f2c6c0654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 -- 172.21.15.92:0/3033057261 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c0bcf70 con 0x7f2c6c153eb0 2023-05-16T07:17:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.765+0000 7f2c71b11700 1 --2- 172.21.15.92:0/3033057261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c154290 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f2c6c05f9e0 tx=0x7f2c680090f0).ready entity=mon.0 client_cookie=2f75f45414205043 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.766+0000 7f2c70b0f700 1 -- 172.21.15.92:0/3033057261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800b520 con 0x7f2c6c153eb0 2023-05-16T07:17:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.766+0000 7f2c70b0f700 1 -- 172.21.15.92:0/3033057261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6800b680 con 0x7f2c6c153eb0 2023-05-16T07:17:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.766+0000 7f2c70b0f700 1 -- 172.21.15.92:0/3033057261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800ba00 con 0x7f2c6c153eb0 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 -- 172.21.15.92:0/3033057261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c153eb0 msgr2=0x7f2c6c154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 --2- 172.21.15.92:0/3033057261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c154290 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f2c6c05f9e0 tx=0x7f2c680090f0).stop 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 -- 172.21.15.92:0/3033057261 shutdown_connections 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 --2- 172.21.15.92:0/3033057261 >> v2:172.21.15.158:3300/0 conn(0x7f2c6c065000 0x7f2c6c0654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 --2- 172.21.15.92:0/3033057261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c154290 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 --2- 172.21.15.92:0/3033057261 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c068c70 0x7f2c6c066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 -- 172.21.15.92:0/3033057261 >> 172.21.15.92:0/3033057261 conn(0x7f2c6c0c4430 msgr2=0x7f2c6c0c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 -- 172.21.15.92:0/3033057261 shutdown_connections 2023-05-16T07:17:36.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.767+0000 7f2c73d75700 1 -- 172.21.15.92:0/3033057261 wait complete. 2023-05-16T07:17:36.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.768+0000 7f2c73d75700 1 Processor -- start 2023-05-16T07:17:36.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.768+0000 7f2c73d75700 1 -- start start 2023-05-16T07:17:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c73d75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 0x7f2c6c167d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c73d75700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c0a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c73d75700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c0bfd80 0x7f2c6c0c0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c73d75700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0a9a10 con 0x7f2c6c065000 2023-05-16T07:17:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c73d75700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0a8b30 con 0x7f2c6c153eb0 2023-05-16T07:17:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c73d75700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0a8e00 con 0x7f2c6c0bfd80 2023-05-16T07:17:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c71b11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 0x7f2c6c167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c71b11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 0x7f2c6c167d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46064/0 (socket says 172.21.15.92:46064) 2023-05-16T07:17:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c71b11700 1 -- 172.21.15.92:0/4003730176 learned_addr learned my addr 172.21.15.92:0/4003730176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c72312700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c0bfd80 0x7f2c6c0c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c71310700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c0a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:36.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.769+0000 7f2c71b11700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c0bfd80 msgr2=0x7f2c6c0c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c71b11700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c0bfd80 0x7f2c6c0c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c71b11700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2c6c153eb0 msgr2=0x7f2c6c0a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:36.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c71b11700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c0a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:36.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c71b11700 1 -- 172.21.15.92:0/4003730176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68009370 con 0x7f2c6c065000 2023-05-16T07:17:36.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c71b11700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 0x7f2c6c167d30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f2c6c064390 tx=0x7f2c68017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:36.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68000bc0 con 0x7f2c6c065000 2023-05-16T07:17:36.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6800b7f0 con 0x7f2c6c065000 2023-05-16T07:17:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c0a90e0 con 0x7f2c6c065000 2023-05-16T07:17:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68000d20 con 0x7f2c6c065000 2023-05-16T07:17:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.770+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c0a96a0 con 0x7f2c6c065000 2023-05-16T07:17:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.771+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c06c350 con 0x7f2c6c065000 2023-05-16T07:17:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.772+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 28) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f2c6800f070 con 0x7f2c6c065000 2023-05-16T07:17:36.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.772+0000 7f2c61ffb700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 0x7f2c500612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:36.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.772+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f2c6807b590 con 0x7f2c6c065000 2023-05-16T07:17:36.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.772+0000 7f2c71310700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 msgr2=0x7f2c500612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:36.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.772+0000 7f2c71310700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 0x7f2c500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.776+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2c6804c770 con 0x7f2c6c065000 2023-05-16T07:17:36.861 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.857+0000 7f03e447c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:36.929 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:36.926+0000 7f03e447c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:36.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.973+0000 7f2c71310700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 msgr2=0x7f2c500612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:36.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.973+0000 7f2c71310700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 0x7f2c500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:36.991+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2c6c0bd250 con 0x7f2c6c065000 2023-05-16T07:17:37.027 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:37.022+0000 7f079daea680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:37.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:17:37.052 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:17:37.098 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:37.093+0000 7f079daea680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:37.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.374+0000 7f2c71310700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 msgr2=0x7f2c500612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:37.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.374+0000 7f2c71310700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 0x7f2c500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:17:37.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.773+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 29) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f2c680790d0 con 0x7f2c6c065000 2023-05-16T07:17:37.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.773+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 msgr2=0x7f2c500612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:37.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.773+0000 7f2c61ffb700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 0x7f2c500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:37.827 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0363c60700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.856 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0363c60700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.856 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0363c60700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.857 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0363c60700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.857 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0363c60700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.857 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0367467700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.857 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0367467700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.857 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0367467700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.858 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0367467700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.858 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:37.824+0000 7f0367467700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:37.859 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:17:37.825+0000 7f0afeb65700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:37.859 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:17:37.825+0000 7f0afeb65700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:37.859 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:17:37.825+0000 7f0afeb65700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:17:37.859 INFO:tasks.ceph.mds.a.smithi092.stderr: -2078> 2023-05-16T07:17:25.419+0000 7f0b0236c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:17:37.860 INFO:tasks.ceph.mds.a.smithi092.stderr: -4> 2023-05-16T07:17:37.825+0000 7f0afeb65700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:37.860 INFO:tasks.ceph.mds.a.smithi092.stderr: -3> 2023-05-16T07:17:37.825+0000 7f0afeb65700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:17:37.860 INFO:tasks.ceph.mds.a.smithi092.stderr: -1> 2023-05-16T07:17:37.825+0000 7f0afeb65700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:17:37.860 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:17:37.944 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:17:37.940+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:17:37.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.942+0000 7f2c61ffb700 1 -- 172.21.15.92:0/4003730176 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2c680551d0 con 0x7f2c6c065000 2023-05-16T07:17:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 msgr2=0x7f2c6c167d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 0x7f2c6c167d30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f2c6c064390 tx=0x7f2c68017000).stop 2023-05-16T07:17:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 shutdown_connections 2023-05-16T07:17:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2c5005ede0 0x7f2c500612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2c6c0bfd80 0x7f2c6c0c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c6c065000 0x7f2c6c167d30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 --2- 172.21.15.92:0/4003730176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2c6c153eb0 0x7f2c6c0a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 >> 172.21.15.92:0/4003730176 conn(0x7f2c6c0c4430 msgr2=0x7f2c6c0a60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 shutdown_connections 2023-05-16T07:17:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:37.944+0000 7f2c73d75700 1 -- 172.21.15.92:0/4003730176 wait complete. 2023-05-16T07:17:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:17:37.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:17:38.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.071+0000 7fed8fdcb700 1 Processor -- start 2023-05-16T07:17:38.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.071+0000 7fed8fdcb700 1 -- start start 2023-05-16T07:17:38.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed880aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:38.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed88068160 0x7fed88068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:38.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880c94d0 con 0x7fed88154470 2023-05-16T07:17:38.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880cb7e0 con 0x7fed88068160 2023-05-16T07:17:38.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed88068a70 con 0x7fed88154090 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed88068160 0x7fed88068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed88068160 0x7fed88068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46094/0 (socket says 172.21.15.92:46094) 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 -- 172.21.15.92:0/2913894737 learned_addr learned my addr 172.21.15.92:0/2913894737 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8db67700 1 --2- 172.21.15.92:0/2913894737 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed880aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 -- 172.21.15.92:0/2913894737 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 msgr2=0x7fed880aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 --2- 172.21.15.92:0/2913894737 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed880aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 -- 172.21.15.92:0/2913894737 >> v1:172.21.15.158:6789/0 conn(0x7fed88154470 legacy=0x7fed880a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 -- 172.21.15.92:0/2913894737 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880c88c0 con 0x7fed88068160 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8e368700 1 --2- 172.21.15.92:0/2913894737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88068160 0x7fed88068580 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fed80006fd0 tx=0x7fed800090f0).ready entity=mon.0 client_cookie=deee6ed373a5b8cc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8cb65700 1 -- 172.21.15.92:0/2913894737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000b600 con 0x7fed88068160 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8cb65700 1 -- 172.21.15.92:0/2913894737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000b760 con 0x7fed88068160 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8cb65700 1 -- 172.21.15.92:0/2913894737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000bae0 con 0x7fed88068160 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 -- 172.21.15.92:0/2913894737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88068160 msgr2=0x7fed88068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/2913894737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88068160 0x7fed88068580 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fed80006fd0 tx=0x7fed800090f0).stop 2023-05-16T07:17:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 -- 172.21.15.92:0/2913894737 shutdown_connections 2023-05-16T07:17:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/2913894737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88068160 0x7fed88068580 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/2913894737 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed880aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.073+0000 7fed8fdcb700 1 -- 172.21.15.92:0/2913894737 >> 172.21.15.92:0/2913894737 conn(0x7fed88067920 msgr2=0x7fed880cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.074+0000 7fed8fdcb700 1 -- 172.21.15.92:0/2913894737 shutdown_connections 2023-05-16T07:17:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.074+0000 7fed8fdcb700 1 -- 172.21.15.92:0/2913894737 wait complete. 2023-05-16T07:17:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 Processor -- start 2023-05-16T07:17:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 -- start start 2023-05-16T07:17:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed88068160 0x7fed881675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed88165780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 0x7fed881645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed88165c70 con 0x7fed88154470 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880ae0b0 con 0x7fed88068160 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8fdcb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed881650e0 con 0x7fed88154090 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.075+0000 7fed8e368700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 0x7fed881645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 0x7fed881645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46102/0 (socket says 172.21.15.92:46102) 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 -- 172.21.15.92:0/604813144 learned_addr learned my addr 172.21.15.92:0/604813144 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8db67700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed88068160 0x7fed881675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8d366700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed88165780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 -- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 msgr2=0x7fed88165780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed88165780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 -- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed88068160 msgr2=0x7fed881675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed88068160 0x7fed881675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 -- 172.21.15.92:0/604813144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80009370 con 0x7fed88154470 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8e368700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 0x7fed881645b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fed80009f50 tx=0x7fed80017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000b600 con 0x7fed88154470 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000bea0 con 0x7fed88154470 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed881653c0 con 0x7fed88154470 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000bae0 con 0x7fed88154470 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.076+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880ae5e0 con 0x7fed88154470 2023-05-16T07:17:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.077+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed88066730 con 0x7fed88154470 2023-05-16T07:17:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.077+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 29) v1 ==== 75092+0+0 (secure 0 0 0) 0x7fed8000f070 con 0x7fed88154470 2023-05-16T07:17:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.078+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fed8007bbc0 con 0x7fed88154470 2023-05-16T07:17:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.082+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fed8004cd80 con 0x7fed88154470 2023-05-16T07:17:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.283+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fed8806c350 con 0x7fed88154470 2023-05-16T07:17:38.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.777+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 30) v1 ==== 75162+0+0 (secure 0 0 0) 0x7fed80079740 con 0x7fed88154470 2023-05-16T07:17:38.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.777+0000 7fed85ffb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed740301b0 0x7fed74067ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.779+0000 7fed8db67700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed740301b0 0x7fed74067ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:38.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.780+0000 7fed8db67700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed740301b0 0x7fed74067ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fed880c5570 tx=0x7fed78009850).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.944+0000 7fed85ffb700 1 -- 172.21.15.92:0/604813144 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed80053780 con 0x7fed88154470 2023-05-16T07:17:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed740301b0 msgr2=0x7fed74067ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed740301b0 0x7fed74067ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fed880c5570 tx=0x7fed78009850).stop 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 msgr2=0x7fed881645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 0x7fed881645b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fed80009f50 tx=0x7fed80017000).stop 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 shutdown_connections 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fed740301b0 0x7fed74067ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed88154470 0x7fed881645b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fed88068160 0x7fed881675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 --2- 172.21.15.92:0/604813144 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fed88154090 0x7fed88165780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 >> 172.21.15.92:0/604813144 conn(0x7fed88067920 msgr2=0x7fed88156860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 shutdown_connections 2023-05-16T07:17:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:38.946+0000 7fed8fdcb700 1 -- 172.21.15.92:0/604813144 wait complete. 2023-05-16T07:17:38.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:17:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.075+0000 7f73dc261700 1 Processor -- start 2023-05-16T07:17:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.075+0000 7f73dc261700 1 -- start start 2023-05-16T07:17:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73dc261700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f73d4153ed0 0x7f73d40bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73dc261700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 0x7f73d40622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73dc261700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d4069070 con 0x7f73d4062e80 2023-05-16T07:17:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73dc261700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d40c3b40 con 0x7f73d4153ed0 2023-05-16T07:17:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73dc261700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4068460 con 0x7f73d4067f60 2023-05-16T07:17:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73d9ffd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f73d4153ed0 0x7f73d40bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.076+0000 7f73da7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 0x7f73d40622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73da7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 0x7f73d40622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:38698/0 (socket says 172.21.15.92:38698) 2023-05-16T07:17:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d9ffd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f73d4153ed0 0x7f73d40bf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:49760/0 (socket says 172.21.15.92:49760) 2023-05-16T07:17:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d97fc700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f73d4062e80 0x7f73d40c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:60470/0 (socket says 172.21.15.92:60470) 2023-05-16T07:17:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73da7fe700 1 -- 172.21.15.92:0/2883524534 learned_addr learned my addr 172.21.15.92:0/2883524534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d9ffd700 1 -- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 msgr2=0x7f73d40622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d9ffd700 1 --2- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 0x7f73d40622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d9ffd700 1 -- 172.21.15.92:0/2883524534 >> v1:172.21.15.158:6789/0 conn(0x7f73d4062e80 legacy=0x7f73d40c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d9ffd700 1 -- 172.21.15.92:0/2883524534 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4068740 con 0x7f73d4153ed0 2023-05-16T07:17:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d8ffb700 1 -- 172.21.15.92:0/2883524534 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812164338 0 0) 0x7f73d4069070 con 0x7f73d4062e80 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73da7fe700 1 --2- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 0x7f73d40622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d9ffd700 1 --2- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4153ed0 0x7f73d40bf1d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f73c4009e60 tx=0x7f73c4009780).ready entity=mon.1 client_cookie=730788361ae27e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d8ffb700 1 -- 172.21.15.92:0/2883524534 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400e070 con 0x7f73d4153ed0 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d8ffb700 1 -- 172.21.15.92:0/2883524534 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c400bb60 con 0x7f73d4153ed0 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.077+0000 7f73d8ffb700 1 -- 172.21.15.92:0/2883524534 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4012410 con 0x7f73d4153ed0 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 -- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4153ed0 msgr2=0x7f73d40bf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 --2- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4153ed0 0x7f73d40bf1d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f73c4009e60 tx=0x7f73c4009780).stop 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 -- 172.21.15.92:0/2883524534 shutdown_connections 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 --2- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4153ed0 0x7f73d40bf1d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 --2- 172.21.15.92:0/2883524534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4067f60 0x7f73d40622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 -- 172.21.15.92:0/2883524534 >> 172.21.15.92:0/2883524534 conn(0x7f73d405dd50 msgr2=0x7f73d405e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 -- 172.21.15.92:0/2883524534 shutdown_connections 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.078+0000 7f73dc261700 1 -- 172.21.15.92:0/2883524534 wait complete. 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.079+0000 7f73dc261700 1 Processor -- start 2023-05-16T07:17:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.079+0000 7f73dc261700 1 -- start start 2023-05-16T07:17:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 0x7f73d4163cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4067f60 0x7f73d40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4153ed0 0x7f73d40ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:39.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40a8fa0 con 0x7f73d4062e80 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40ae820 con 0x7f73d4067f60 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40a8690 con 0x7f73d4153ed0 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 0x7f73d4163cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 0x7f73d4163cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46128/0 (socket says 172.21.15.92:46128) 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 -- 172.21.15.92:0/127007349 learned_addr learned my addr 172.21.15.92:0/127007349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d97fc700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4067f60 0x7f73d40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73da7fe700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4153ed0 0x7f73d40ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:39.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 -- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4153ed0 msgr2=0x7f73d40ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:39.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4153ed0 0x7f73d40ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 -- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4067f60 msgr2=0x7f73d40ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:39.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4067f60 0x7f73d40ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 -- 172.21.15.92:0/127007349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc009370 con 0x7f73d4062e80 2023-05-16T07:17:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73da7fe700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4153ed0 0x7f73d40ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d97fc700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4067f60 0x7f73d40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d9ffd700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 0x7f73d4163cd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f73c400b6c0 tx=0x7f73c4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c401d420 con 0x7f73d4062e80 2023-05-16T07:17:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c401d580 con 0x7f73d4062e80 2023-05-16T07:17:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c4009a20 con 0x7f73d4062e80 2023-05-16T07:17:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.080+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c401d940 con 0x7f73d4062e80 2023-05-16T07:17:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.081+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d40abd90 con 0x7f73d4062e80 2023-05-16T07:17:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.081+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d40a9210 con 0x7f73d4062e80 2023-05-16T07:17:39.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.082+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 30) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f73c401dd30 con 0x7f73d4062e80 2023-05-16T07:17:39.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.082+0000 7f73d1ffb700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73c005ecf0 0x7f73c00611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:39.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.082+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f73c407bd10 con 0x7f73d4062e80 2023-05-16T07:17:39.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.082+0000 7f73d97fc700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73c005ecf0 0x7f73c00611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:39.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.083+0000 7f73d97fc700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73c005ecf0 0x7f73c00611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f73d41575e0 tx=0x7f73c8007b10).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:39.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.086+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f73c40aa8f0 con 0x7f73d4062e80 2023-05-16T07:17:39.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.248+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73d406c350 con 0x7f73d4062e80 2023-05-16T07:17:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.946+0000 7f73d1ffb700 1 -- 172.21.15.92:0/127007349 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7f73c404cf30 con 0x7f73d4062e80 2023-05-16T07:17:39.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73c005ecf0 msgr2=0x7f73c00611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73c005ecf0 0x7f73c00611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f73d41575e0 tx=0x7f73c8007b10).stop 2023-05-16T07:17:39.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 msgr2=0x7f73d4163cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:39.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 0x7f73d4163cd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f73c400b6c0 tx=0x7f73c4009780).stop 2023-05-16T07:17:39.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 shutdown_connections 2023-05-16T07:17:39.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f73c005ecf0 0x7f73c00611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73d4153ed0 0x7f73d40ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d4062e80 0x7f73d4163cd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 --2- 172.21.15.92:0/127007349 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73d4067f60 0x7f73d40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.952+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 >> 172.21.15.92:0/127007349 conn(0x7f73d405dd50 msgr2=0x7f73d40c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.953+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 shutdown_connections 2023-05-16T07:17:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:39.954+0000 7f73dc261700 1 -- 172.21.15.92:0/127007349 wait complete. 2023-05-16T07:17:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:17:39.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:17:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.095+0000 7f59e756e700 1 Processor -- start 2023-05-16T07:17:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.095+0000 7f59e756e700 1 -- start start 2023-05-16T07:17:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e756e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e014f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e756e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f59e005e460 0x7f59e0064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e756e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00c3170 con 0x7f59e0152bc0 2023-05-16T07:17:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e756e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e005ec20 con 0x7f59e005e460 2023-05-16T07:17:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e756e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e005e940 con 0x7f59e014f3b0 2023-05-16T07:17:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e5d6b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f59e005e460 0x7f59e0064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e5d6b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f59e005e460 0x7f59e0064150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46140/0 (socket says 172.21.15.92:46140) 2023-05-16T07:17:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e5d6b700 1 -- 172.21.15.92:0/312217376 learned_addr learned my addr 172.21.15.92:0/312217376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e5d6b700 1 -- 172.21.15.92:0/312217376 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 msgr2=0x7f59e014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e656c700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.096+0000 7f59e5d6b700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e014f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e5d6b700 1 -- 172.21.15.92:0/312217376 >> v1:172.21.15.158:6789/0 conn(0x7f59e0152bc0 legacy=0x7f59e00a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e5d6b700 1 -- 172.21.15.92:0/312217376 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0063670 con 0x7f59e005e460 2023-05-16T07:17:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e5d6b700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e005e460 0x7f59e0064150 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f59d0006fd0 tx=0x7f59d00090f0).ready entity=mon.0 client_cookie=f1bb3589ac86a056 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e656c700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e556a700 1 -- 172.21.15.92:0/312217376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d000b600 con 0x7f59e005e460 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e556a700 1 -- 172.21.15.92:0/312217376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d000b760 con 0x7f59e005e460 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.097+0000 7f59e556a700 1 -- 172.21.15.92:0/312217376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d000bae0 con 0x7f59e005e460 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 -- 172.21.15.92:0/312217376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e005e460 msgr2=0x7f59e0064150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e005e460 0x7f59e0064150 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f59d0006fd0 tx=0x7f59d00090f0).stop 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 -- 172.21.15.92:0/312217376 shutdown_connections 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 --2- 172.21.15.92:0/312217376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e005e460 0x7f59e0064150 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 -- 172.21.15.92:0/312217376 >> 172.21.15.92:0/312217376 conn(0x7f59e0062d80 msgr2=0x7f59e005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 -- 172.21.15.92:0/312217376 shutdown_connections 2023-05-16T07:17:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.098+0000 7f59e756e700 1 -- 172.21.15.92:0/312217376 wait complete. 2023-05-16T07:17:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.099+0000 7f59e756e700 1 Processor -- start 2023-05-16T07:17:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.099+0000 7f59e756e700 1 -- start start 2023-05-16T07:17:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e756e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59e005e460 0x7f59e00ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:40.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e756e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e0162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:40.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e756e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 0x7f59e0163420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:40.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e756e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e014dd80 con 0x7f59e0152bc0 2023-05-16T07:17:40.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e756e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00ac450 con 0x7f59e005e460 2023-05-16T07:17:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e756e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00ab860 con 0x7f59e014f3b0 2023-05-16T07:17:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 0x7f59e0163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 0x7f59e0163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46144/0 (socket says 172.21.15.92:46144) 2023-05-16T07:17:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 -- 172.21.15.92:0/2645189864 learned_addr learned my addr 172.21.15.92:0/2645189864 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 -- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 msgr2=0x7f59e0162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e0162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 -- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59e005e460 msgr2=0x7f59e00ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59e005e460 0x7f59e00ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 -- 172.21.15.92:0/2645189864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0009370 con 0x7f59e0152bc0 2023-05-16T07:17:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59e6d6d700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 0x7f59e0163420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f59d800b810 tx=0x7f59d8009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.100+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800d600 con 0x7f59e0152bc0 2023-05-16T07:17:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.101+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800d780 con 0x7f59e0152bc0 2023-05-16T07:17:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.101+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800db40 con 0x7f59e0152bc0 2023-05-16T07:17:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.101+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e00abb20 con 0x7f59e0152bc0 2023-05-16T07:17:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.101+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e00ac0e0 con 0x7f59e0152bc0 2023-05-16T07:17:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.102+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e0047150 con 0x7f59e0152bc0 2023-05-16T07:17:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.102+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f59d8019460 con 0x7f59e0152bc0 2023-05-16T07:17:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.102+0000 7f59d67fc700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59c405ed90 0x7f59c4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.102+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6740+0+0 (secure 0 0 0) 0x7f59d807a010 con 0x7f59e0152bc0 2023-05-16T07:17:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.102+0000 7f59e656c700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59c405ed90 0x7f59c4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:40.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.103+0000 7f59e656c700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59c405ed90 0x7f59c4061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f59dc006fd0 tx=0x7f59dc008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.106+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f59d804b2e0 con 0x7f59e0152bc0 2023-05-16T07:17:40.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.270+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59e0063950 con 0x7f59e0152bc0 2023-05-16T07:17:40.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.952+0000 7f59d67fc700 1 -- 172.21.15.92:0/2645189864 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f59d804e900 con 0x7f59e0152bc0 2023-05-16T07:17:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59c405ed90 msgr2=0x7f59c4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59c405ed90 0x7f59c4061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f59dc006fd0 tx=0x7f59dc008040).stop 2023-05-16T07:17:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 msgr2=0x7f59e0163420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 0x7f59e0163420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f59d800b810 tx=0x7f59d8009290).stop 2023-05-16T07:17:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 shutdown_connections 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f59c405ed90 0x7f59c4061250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e0152bc0 0x7f59e0163420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59e005e460 0x7f59e00ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 --2- 172.21.15.92:0/2645189864 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59e014f3b0 0x7f59e0162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 >> 172.21.15.92:0/2645189864 conn(0x7f59e0062d80 msgr2=0x7f59e00c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.957+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 shutdown_connections 2023-05-16T07:17:40.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:40.958+0000 7f59e756e700 1 -- 172.21.15.92:0/2645189864 wait complete. 2023-05-16T07:17:40.964 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:17:40.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:17:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.088+0000 7f800e9a0700 1 Processor -- start 2023-05-16T07:17:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.088+0000 7f800e9a0700 1 -- start start 2023-05-16T07:17:41.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.089+0000 7f800e9a0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 0x7f80080aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.089+0000 7f800e9a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080c94d0 con 0x7f8008154470 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.089+0000 7f800e9a0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080cb7e0 con 0x7f8008154090 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.089+0000 7f800e9a0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c88c0 con 0x7f8008068870 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.089+0000 7f7ffffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8008154470 0x7f8008068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56902/0 (socket says 172.21.15.92:56902) 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.089+0000 7f7ffffff700 1 -- 172.21.15.92:0/3321718830 learned_addr learned my addr 172.21.15.92:0/3321718830 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 0x7f80080aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776355682 0 0) 0x7f80080c94d0 con 0x7f8008154470 2023-05-16T07:17:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff40033f0 con 0x7f8008154470 2023-05-16T07:17:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 533390169 0 0) 0x7f7ff40033f0 con 0x7f8008154470 2023-05-16T07:17:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 msgr2=0x7f80080aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 0x7f80080aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 >> v1:172.21.15.158:6789/0 conn(0x7f8008154090 legacy=0x7f80080a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080c8ba0 con 0x7f8008154470 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f7ff0001470 con 0x7f8008154470 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 >> v1:172.21.15.92:6789/0 conn(0x7f8008154470 legacy=0x7f8008068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 0x7f7ff4003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7ff4003e40 0x7f7ff4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4004b80 0x7f7ff4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c88c0 con 0x7f7ff4004b80 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff400b990 con 0x7f7ff4003e40 2023-05-16T07:17:41.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff400bb50 con 0x7f8008068870 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4004b80 0x7f7ff4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f8007fff700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 0x7f7ff4003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f7ffffff700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7ff4003e40 0x7f7ff4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 -- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 msgr2=0x7f7ff4003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8008068870 0x7f7ff4003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 -- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7ff4003e40 msgr2=0x7f7ff4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7ff4003e40 0x7f7ff4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 -- 172.21.15.92:0/3321718830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff40033f0 con 0x7f7ff4004b80 2023-05-16T07:17:41.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f7ffffff700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7ff4003e40 0x7f7ff4004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:41.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.090+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4004b80 0x7f7ff4005000 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f800000e1c0 tx=0x7f800000b040).ready entity=mon.0 client_cookie=169f7fc6287ddbbd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000009ac0 con 0x7f7ff4004b80 2023-05-16T07:17:41.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800000f040 con 0x7f7ff4004b80 2023-05-16T07:17:41.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f80077fe700 1 -- 172.21.15.92:0/3321718830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000eca0 con 0x7f7ff4004b80 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800e9a0700 1 -- 172.21.15.92:0/3321718830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4004b80 msgr2=0x7f7ff4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3321718830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4004b80 0x7f7ff4005000 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f800000e1c0 tx=0x7f800000b040).stop 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800cf3d700 1 -- 172.21.15.92:0/3321718830 reap_dead start 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800e9a0700 1 -- 172.21.15.92:0/3321718830 shutdown_connections 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800e9a0700 1 -- 172.21.15.92:0/3321718830 >> 172.21.15.92:0/3321718830 conn(0x7f8008067920 msgr2=0x7f80080cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800e9a0700 1 -- 172.21.15.92:0/3321718830 shutdown_connections 2023-05-16T07:17:41.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.091+0000 7f800e9a0700 1 -- 172.21.15.92:0/3321718830 wait complete. 2023-05-16T07:17:41.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.092+0000 7f800e9a0700 1 Processor -- start 2023-05-16T07:17:41.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.092+0000 7f800e9a0700 1 -- start start 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800e9a0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f800814ca60 0x7f8008156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800e9a0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f800814ce40 0x7f80080c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800e9a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 0x7f8008153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800e9a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c3280 con 0x7f8008152f00 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800e9a0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008153870 con 0x7f800814ce40 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800e9a0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8008157070 con 0x7f800814ca60 2023-05-16T07:17:41.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 0x7f8008153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 0x7f8008153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46166/0 (socket says 172.21.15.92:46166) 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 -- 172.21.15.92:0/3112517651 learned_addr learned my addr 172.21.15.92:0/3112517651 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f8007fff700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f800814ca60 0x7f8008156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f7ffffff700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f800814ce40 0x7f80080c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 -- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f800814ca60 msgr2=0x7f8008156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f800814ca60 0x7f8008156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 -- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f800814ce40 msgr2=0x7f80080c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f800814ce40 0x7f80080c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.093+0000 7f800cf3d700 1 -- 172.21.15.92:0/3112517651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008157350 con 0x7f8008152f00 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.094+0000 7f800cf3d700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 0x7f8008153380 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f8000014000 tx=0x7f800001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.094+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000ea70 con 0x7f8008152f00 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.094+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800000ebd0 con 0x7f8008152f00 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.094+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8008157630 con 0x7f8008152f00 2023-05-16T07:17:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.094+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800001c640 con 0x7f8008152f00 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.094+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080c2de0 con 0x7f8008152f00 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.095+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008066730 con 0x7f8008152f00 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.095+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f800002c070 con 0x7f8008152f00 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.096+0000 7f8004ff9700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7fe805ed90 0x7f7fe8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.096+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f800007b530 con 0x7f8008152f00 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.096+0000 7f8007fff700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7fe805ed90 0x7f7fe8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.096+0000 7f8007fff700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7fe805ed90 0x7f7fe8061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7ff8007cc0 tx=0x7f7ff8009040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.100+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f800004ca00 con 0x7f8008152f00 2023-05-16T07:17:41.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.271+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f800806c350 con 0x7f8008152f00 2023-05-16T07:17:41.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.275+0000 7f8004ff9700 1 -- 172.21.15.92:0/3112517651 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f800002a0a0 con 0x7f8008152f00 2023-05-16T07:17:41.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7fe805ed90 msgr2=0x7f7fe8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7fe805ed90 0x7f7fe8061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7ff8007cc0 tx=0x7f7ff8009040).stop 2023-05-16T07:17:41.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 msgr2=0x7f8008153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 0x7f8008153380 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f8000014000 tx=0x7f800001a000).stop 2023-05-16T07:17:41.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 shutdown_connections 2023-05-16T07:17:41.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7fe805ed90 0x7f7fe8061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f800814ca60 0x7f8008156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8008152f00 0x7f8008153380 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 --2- 172.21.15.92:0/3112517651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f800814ce40 0x7f80080c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 >> 172.21.15.92:0/3112517651 conn(0x7f8008067920 msgr2=0x7f800814d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 shutdown_connections 2023-05-16T07:17:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.277+0000 7f800e9a0700 1 -- 172.21.15.92:0/3112517651 wait complete. 2023-05-16T07:17:41.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:17:41.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.403+0000 7f4fcf22a700 1 Processor -- start 2023-05-16T07:17:41.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.403+0000 7f4fcf22a700 1 -- start start 2023-05-16T07:17:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fcf22a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4fc8154470 0x7f4fc80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fcf22a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 0x7f4fc80685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fcf22a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc80c94d0 con 0x7f4fc8154090 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fcf22a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc80cb7e0 con 0x7f4fc8154470 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fcf22a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8068aa0 con 0x7f4fc8068160 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fccfc6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4fc8154090 0x7f4fc80a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56918/0 (socket says 172.21.15.92:56918) 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fccfc6700 1 -- 172.21.15.92:0/1689226541 learned_addr learned my addr 172.21.15.92:0/1689226541 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fcd7c7700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 0x7f4fc80685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fc7fff700 1 --2- 172.21.15.92:0/1689226541 >> v2:172.21.15.158:3300/0 conn(0x7f4fc8154470 0x7f4fc80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982907191 0 0) 0x7f4fc80c94d0 con 0x7f4fc8154090 2023-05-16T07:17:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.404+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb00033f0 con 0x7f4fc8154090 2023-05-16T07:17:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2288006450 0 0) 0x7f4fb00033f0 con 0x7f4fc8154090 2023-05-16T07:17:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 msgr2=0x7f4fc80685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 0x7f4fc80685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 >> v2:172.21.15.158:3300/0 conn(0x7f4fc8154470 msgr2=0x7f4fc80aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 --2- 172.21.15.92:0/1689226541 >> v2:172.21.15.158:3300/0 conn(0x7f4fc8154470 0x7f4fc80aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80c88c0 con 0x7f4fc8154090 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4fb80017f0 con 0x7f4fc8154090 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 >> v1:172.21.15.92:6789/0 conn(0x7f4fc8154090 legacy=0x7f4fc80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fb00038e0 0x7f4fb0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 0x7f4fb0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb0004b80 0x7f4fb0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc80cb7e0 con 0x7f4fb0004b80 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8068aa0 con 0x7f4fb00038e0 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb000b8e0 con 0x7f4fc8068160 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb0004b80 0x7f4fb0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc7fff700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 0x7f4fb0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fccfc6700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fb00038e0 0x7f4fb0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 -- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 msgr2=0x7f4fb0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc8068160 0x7f4fb0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 -- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fb00038e0 msgr2=0x7f4fb0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fb00038e0 0x7f4fb0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 -- 172.21.15.92:0/1689226541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb00033f0 con 0x7f4fb0004b80 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fccfc6700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fb00038e0 0x7f4fb0003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fcd7c7700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb0004b80 0x7f4fb0005000 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f4fc0009a60 tx=0x7f4fc0009060).ready entity=mon.0 client_cookie=26887ceefb6314e9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0009ea0 con 0x7f4fb0004b80 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc0009430 con 0x7f4fb0004b80 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.405+0000 7f4fc77fe700 1 -- 172.21.15.92:0/1689226541 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000e870 con 0x7f4fb0004b80 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcf22a700 1 -- 172.21.15.92:0/1689226541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb0004b80 msgr2=0x7f4fb0005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/1689226541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb0004b80 0x7f4fb0005000 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f4fc0009a60 tx=0x7f4fc0009060).stop 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcd7c7700 1 -- 172.21.15.92:0/1689226541 reap_dead start 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcf22a700 1 -- 172.21.15.92:0/1689226541 shutdown_connections 2023-05-16T07:17:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcf22a700 1 -- 172.21.15.92:0/1689226541 >> 172.21.15.92:0/1689226541 conn(0x7f4fc8067920 msgr2=0x7f4fc80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcf22a700 1 -- 172.21.15.92:0/1689226541 shutdown_connections 2023-05-16T07:17:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.406+0000 7f4fcf22a700 1 -- 172.21.15.92:0/1689226541 wait complete. 2023-05-16T07:17:41.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.407+0000 7f4fcf22a700 1 Processor -- start 2023-05-16T07:17:41.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.407+0000 7f4fcf22a700 1 -- start start 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcf22a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 0x7f4fc81573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcf22a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fc8167ca0 0x7f4fc8168120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcf22a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc81527d0 0x7f4fc8152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcf22a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81578d0 con 0x7f4fc8157000 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcf22a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc80c2ca0 con 0x7f4fc8167ca0 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcf22a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8168610 con 0x7f4fc81527d0 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 0x7f4fc81573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 0x7f4fc81573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46184/0 (socket says 172.21.15.92:46184) 2023-05-16T07:17:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 -- 172.21.15.92:0/3320761192 learned_addr learned my addr 172.21.15.92:0/3320761192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fc7fff700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fc8167ca0 0x7f4fc8168120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fcd7c7700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc81527d0 0x7f4fc8152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 -- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc81527d0 msgr2=0x7f4fc8152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc81527d0 0x7f4fc8152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 -- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fc8167ca0 msgr2=0x7f4fc8168120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fc8167ca0 0x7f4fc8168120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 -- 172.21.15.92:0/3320761192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8153500 con 0x7f4fc8157000 2023-05-16T07:17:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fc7fff700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fc8167ca0 0x7f4fc8168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fccfc6700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 0x7f4fc81573e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f4fb8001350 tx=0x7f4fb8001490).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.408+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800e8a0 con 0x7f4fc8157000 2023-05-16T07:17:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.409+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb8010040 con 0x7f4fc8157000 2023-05-16T07:17:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.409+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc81537e0 con 0x7f4fc8157000 2023-05-16T07:17:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.409+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800ec30 con 0x7f4fc8157000 2023-05-16T07:17:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.409+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc80c3190 con 0x7f4fc8157000 2023-05-16T07:17:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.410+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8066730 con 0x7f4fc8157000 2023-05-16T07:17:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.410+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4fb8015400 con 0x7f4fc8157000 2023-05-16T07:17:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.410+0000 7f4fc4ff9700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fb405ed90 0x7f4fb4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.411+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f4fb8079fe0 con 0x7f4fc8157000 2023-05-16T07:17:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.411+0000 7f4fc7fff700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fb405ed90 0x7f4fb4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.411+0000 7f4fc7fff700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fb405ed90 0x7f4fb4061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f4fc80c49f0 tx=0x7f4fbc008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.415+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4fb804b450 con 0x7f4fc8157000 2023-05-16T07:17:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.577+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4fc806c350 con 0x7f4fc8157000 2023-05-16T07:17:41.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.580+0000 7f4fc4ff9700 1 -- 172.21.15.92:0/3320761192 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v13) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4fb804ea70 con 0x7f4fc8157000 2023-05-16T07:17:41.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fb405ed90 msgr2=0x7f4fb4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fb405ed90 0x7f4fb4061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f4fc80c49f0 tx=0x7f4fbc008040).stop 2023-05-16T07:17:41.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 msgr2=0x7f4fc81573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 0x7f4fc81573e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f4fb8001350 tx=0x7f4fb8001490).stop 2023-05-16T07:17:41.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 shutdown_connections 2023-05-16T07:17:41.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fb405ed90 0x7f4fb4061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4fc81527d0 0x7f4fc8152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fc8157000 0x7f4fc81573e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 --2- 172.21.15.92:0/3320761192 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fc8167ca0 0x7f4fc8168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 >> 172.21.15.92:0/3320761192 conn(0x7f4fc8067920 msgr2=0x7f4fc814f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:41.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 shutdown_connections 2023-05-16T07:17:41.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.582+0000 7f4fcf22a700 1 -- 172.21.15.92:0/3320761192 wait complete. 2023-05-16T07:17:41.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-05-16T07:17:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.705+0000 7f2dc2f11700 1 Processor -- start 2023-05-16T07:17:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.705+0000 7f2dc2f11700 1 -- start start 2023-05-16T07:17:41.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc2f11700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2dbc154170 0x7f2dbc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc2f11700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2dbc154550 0x7f2dbc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc2f11700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0c0660 con 0x7f2dbc0696a0 2023-05-16T07:17:41.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc2f11700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c1b00 con 0x7f2dbc154170 2023-05-16T07:17:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc2f11700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0bfd30 con 0x7f2dbc154550 2023-05-16T07:17:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2dbc154170 0x7f2dbc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2dbc154170 0x7f2dbc0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46190/0 (socket says 172.21.15.92:46190) 2023-05-16T07:17:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 -- 172.21.15.92:0/2851173645 learned_addr learned my addr 172.21.15.92:0/2851173645 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dbbfff700 1 --2- 172.21.15.92:0/2851173645 >> v2:172.21.15.158:3300/0 conn(0x7f2dbc154550 0x7f2dbc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 -- 172.21.15.92:0/2851173645 >> v2:172.21.15.158:3300/0 conn(0x7f2dbc154550 msgr2=0x7f2dbc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 --2- 172.21.15.92:0/2851173645 >> v2:172.21.15.158:3300/0 conn(0x7f2dbc154550 0x7f2dbc0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 -- 172.21.15.92:0/2851173645 >> v1:172.21.15.158:6789/0 conn(0x7f2dbc0696a0 legacy=0x7f2dbc157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 -- 172.21.15.92:0/2851173645 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc0c0010 con 0x7f2dbc154170 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dbbfff700 1 --2- 172.21.15.92:0/2851173645 >> v2:172.21.15.158:3300/0 conn(0x7f2dbc154550 0x7f2dbc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dc0cad700 1 --2- 172.21.15.92:0/2851173645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154170 0x7f2dbc0c5c70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f2dbc0844e0 tx=0x7f2db00090f0).ready entity=mon.0 client_cookie=95152f74ef6afb49 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dbb7fe700 1 -- 172.21.15.92:0/2851173645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000b5a0 con 0x7f2dbc154170 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dbb7fe700 1 -- 172.21.15.92:0/2851173645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db000b700 con 0x7f2dbc154170 2023-05-16T07:17:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.706+0000 7f2dbb7fe700 1 -- 172.21.15.92:0/2851173645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000baa0 con 0x7f2dbc154170 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2851173645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154170 msgr2=0x7f2dbc0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2851173645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154170 0x7f2dbc0c5c70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f2dbc0844e0 tx=0x7f2db00090f0).stop 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2851173645 shutdown_connections 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2851173645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154170 0x7f2dbc0c5c70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2851173645 >> v2:172.21.15.158:3300/0 conn(0x7f2dbc154550 0x7f2dbc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2851173645 >> 172.21.15.92:0/2851173645 conn(0x7f2dbc0cb700 msgr2=0x7f2dbc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2851173645 shutdown_connections 2023-05-16T07:17:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.707+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2851173645 wait complete. 2023-05-16T07:17:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.708+0000 7f2dc2f11700 1 Processor -- start 2023-05-16T07:17:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.708+0000 7f2dc2f11700 1 -- start start 2023-05-16T07:17:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2dbc0696a0 0x7f2dbc0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2dbc154170 0x7f2dbc0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 0x7f2dbc14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c3e00 con 0x7f2dbc154550 2023-05-16T07:17:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c31f0 con 0x7f2dbc154170 2023-05-16T07:17:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c34d0 con 0x7f2dbc0696a0 2023-05-16T07:17:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 0x7f2dbc14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 0x7f2dbc14d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46200/0 (socket says 172.21.15.92:46200) 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 -- 172.21.15.92:0/2357863904 learned_addr learned my addr 172.21.15.92:0/2357863904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dbbfff700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2dbc154170 0x7f2dbc0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc0cad700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2dbc0696a0 0x7f2dbc0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 -- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2dbc0696a0 msgr2=0x7f2dbc0c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2dbc0696a0 0x7f2dbc0c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 -- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2dbc154170 msgr2=0x7f2dbc0aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2dbc154170 0x7f2dbc0aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 -- 172.21.15.92:0/2357863904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0009370 con 0x7f2dbc154550 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc0cad700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2dbc0696a0 0x7f2dbc0c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc14ae700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 0x7f2dbc14d870 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f2db400c810 tx=0x7f2db4002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4010510 con 0x7f2dbc154550 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4010690 con 0x7f2dbc154550 2023-05-16T07:17:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc0c37b0 con 0x7f2dbc154550 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.709+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4010a50 con 0x7f2dbc154550 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.710+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc0aef10 con 0x7f2dbc154550 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.710+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc0666c0 con 0x7f2dbc154550 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.711+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2db4010070 con 0x7f2dbc154550 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.711+0000 7f2db8ff9700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2da005ed90 0x7f2da0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.711+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f2db40799d0 con 0x7f2dbc154550 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.711+0000 7f2dc0cad700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2da005ed90 0x7f2da0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.712+0000 7f2dc0cad700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2da005ed90 0x7f2da0061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2db0009130 tx=0x7f2db0015000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.715+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2db404aea0 con 0x7f2dbc154550 2023-05-16T07:17:41.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.941+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f2dbc06c350 con 0x7f2dbc154550 2023-05-16T07:17:41.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.954+0000 7f2db8ff9700 1 -- 172.21.15.92:0/2357863904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v550) v1 ==== 156+0+0 (secure 0 0 0) 0x7f2db404e4c0 con 0x7f2dbc154550 2023-05-16T07:17:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2da005ed90 msgr2=0x7f2da0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2da005ed90 0x7f2da0061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2db0009130 tx=0x7f2db0015000).stop 2023-05-16T07:17:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 msgr2=0x7f2dbc14d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:41.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 0x7f2dbc14d870 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f2db400c810 tx=0x7f2db4002560).stop 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 shutdown_connections 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2da005ed90 0x7f2da0061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2dbc0696a0 0x7f2dbc0c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dbc154550 0x7f2dbc14d870 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 --2- 172.21.15.92:0/2357863904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2dbc154170 0x7f2dbc0aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 >> 172.21.15.92:0/2357863904 conn(0x7f2dbc0cb700 msgr2=0x7f2dbc0c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:41.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 shutdown_connections 2023-05-16T07:17:41.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:41.956+0000 7f2dc2f11700 1 -- 172.21.15.92:0/2357863904 wait complete. 2023-05-16T07:17:41.971 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-05-16T07:17:41.972 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-05-16T07:17:41.973 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:17:41.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-05-16T07:17:42.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.080+0000 7f8d4bb63700 1 Processor -- start 2023-05-16T07:17:42.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.080+0000 7f8d4bb63700 1 -- start start 2023-05-16T07:17:42.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d4bb63700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d44153da0 0x7f8d44069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:42.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d4bb63700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d440bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:42.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d4bb63700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d44062b10 con 0x7f8d44154180 2023-05-16T07:17:42.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d4bb63700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d44068620 con 0x7f8d44153da0 2023-05-16T07:17:42.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d4bb63700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440622b0 con 0x7f8d44063e30 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d498ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d44153da0 0x7f8d44069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d490fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d44154180 0x7f8d44063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56952/0 (socket says 172.21.15.92:56952) 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d490fe700 1 -- 172.21.15.92:0/73926644 learned_addr learned my addr 172.21.15.92:0/73926644 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.081+0000 7f8d4a100700 1 --2- 172.21.15.92:0/73926644 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d440bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d498ff700 1 -- 172.21.15.92:0/73926644 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 msgr2=0x7f8d440bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d498ff700 1 --2- 172.21.15.92:0/73926644 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d440bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d498ff700 1 -- 172.21.15.92:0/73926644 >> v1:172.21.15.92:6789/0 conn(0x7f8d44154180 legacy=0x7f8d44063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d498ff700 1 -- 172.21.15.92:0/73926644 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d44068340 con 0x7f8d44153da0 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d488fd700 1 -- 172.21.15.92:0/73926644 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671470242 0 0) 0x7f8d44062b10 con 0x7f8d44154180 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d498ff700 1 --2- 172.21.15.92:0/73926644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44153da0 0x7f8d44069a10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f8d34009e60 tx=0x7f8d34009780).ready entity=mon.0 client_cookie=efbd75d6401459e8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d488fd700 1 -- 172.21.15.92:0/73926644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400e070 con 0x7f8d44153da0 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d488fd700 1 -- 172.21.15.92:0/73926644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3400bc30 con 0x7f8d44153da0 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d488fd700 1 -- 172.21.15.92:0/73926644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34012410 con 0x7f8d44153da0 2023-05-16T07:17:42.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 -- 172.21.15.92:0/73926644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44153da0 msgr2=0x7f8d44069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:42.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/73926644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44153da0 0x7f8d44069a10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f8d34009e60 tx=0x7f8d34009780).stop 2023-05-16T07:17:42.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 -- 172.21.15.92:0/73926644 shutdown_connections 2023-05-16T07:17:42.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/73926644 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d440bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/73926644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44153da0 0x7f8d44069a10 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 -- 172.21.15.92:0/73926644 >> 172.21.15.92:0/73926644 conn(0x7f8d440c8aa0 msgr2=0x7f8d440c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:42.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 -- 172.21.15.92:0/73926644 shutdown_connections 2023-05-16T07:17:42.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.082+0000 7f8d4bb63700 1 -- 172.21.15.92:0/73926644 wait complete. 2023-05-16T07:17:42.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.083+0000 7f8d4bb63700 1 Processor -- start 2023-05-16T07:17:42.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 -- start start 2023-05-16T07:17:42.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d44164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d44153da0 0x7f8d440a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 0x7f8d440a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440c0cb0 con 0x7f8d44154180 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440aa790 con 0x7f8d44153da0 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4bb63700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440a9ef0 con 0x7f8d44063e30 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4a100700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 0x7f8d440a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4a100700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 0x7f8d440a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46210/0 (socket says 172.21.15.92:46210) 2023-05-16T07:17:42.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d4a100700 1 -- 172.21.15.92:0/316124369 learned_addr learned my addr 172.21.15.92:0/316124369 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d498ff700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d44164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.084+0000 7f8d490fe700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d44153da0 0x7f8d440a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4a100700 1 -- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 msgr2=0x7f8d44164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4a100700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d44164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4a100700 1 -- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d44153da0 msgr2=0x7f8d440a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4a100700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d44153da0 0x7f8d440a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4a100700 1 -- 172.21.15.92:0/316124369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34009a20 con 0x7f8d44154180 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d490fe700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d44153da0 0x7f8d440a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4a100700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 0x7f8d440a6b10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f8d3c00d860 tx=0x7f8d3c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c011590 con 0x7f8d44154180 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c0116f0 con 0x7f8d44154180 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d440aa1d0 con 0x7f8d44154180 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c011a70 con 0x7f8d44154180 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.085+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d440a9b70 con 0x7f8d44154180 2023-05-16T07:17:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.086+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d440c0e60 con 0x7f8d44154180 2023-05-16T07:17:42.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.086+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8d3c01c460 con 0x7f8d44154180 2023-05-16T07:17:42.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.086+0000 7f8d41ffb700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d3005ed90 0x7f8d30061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:42.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.086+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f8d3c079b30 con 0x7f8d44154180 2023-05-16T07:17:42.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.087+0000 7f8d498ff700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d3005ed90 0x7f8d30061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:42.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.087+0000 7f8d498ff700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d3005ed90 0x7f8d30061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8d440c4cf0 tx=0x7f8d34009780).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:42.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.090+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8d440c0e60 con 0x7f8d44154180 2023-05-16T07:17:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.314+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f8d4406c350 con 0x7f8d44154180 2023-05-16T07:17:42.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.955+0000 7f8d41ffb700 1 -- 172.21.15.92:0/316124369 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v551) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8d3c04e9d0 con 0x7f8d44154180 2023-05-16T07:17:42.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.957+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d3005ed90 msgr2=0x7f8d30061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:42.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d3005ed90 0x7f8d30061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8d440c4cf0 tx=0x7f8d34009780).stop 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 msgr2=0x7f8d440a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 0x7f8d440a6b10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f8d3c00d860 tx=0x7f8d3c008040).stop 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 shutdown_connections 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8d3005ed90 0x7f8d30061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d44063e30 0x7f8d44164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d44154180 0x7f8d440a6b10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 --2- 172.21.15.92:0/316124369 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d44153da0 0x7f8d440a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:42.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 >> 172.21.15.92:0/316124369 conn(0x7f8d440c8aa0 msgr2=0x7f8d440ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:42.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 shutdown_connections 2023-05-16T07:17:42.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:42.958+0000 7f8d4bb63700 1 -- 172.21.15.92:0/316124369 wait complete. 2023-05-16T07:17:42.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:17:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.086+0000 7f183b59e700 1 Processor -- start 2023-05-16T07:17:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.086+0000 7f183b59e700 1 -- start start 2023-05-16T07:17:43.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.087+0000 7f183b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f183c153f90 0x7f183c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.087+0000 7f183b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.087+0000 7f183b59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f183c0c0150 0x7f183c0c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.087+0000 7f183b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c069f60 con 0x7f183c153f90 2023-05-16T07:17:43.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.087+0000 7f183b59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c06a9e0 con 0x7f183c0c0150 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.087+0000 7f183b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0c6670 con 0x7f183c0c5d00 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f183c153f90 0x7f183c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f183c153f90 0x7f183c0a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46216/0 (socket says 172.21.15.92:46216) 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 -- 172.21.15.92:0/1295574827 learned_addr learned my addr 172.21.15.92:0/1295574827 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183ad9d700 1 --2- 172.21.15.92:0/1295574827 >> v2:172.21.15.158:3300/0 conn(0x7f183c0c0150 0x7f183c0c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 -- 172.21.15.92:0/1295574827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 msgr2=0x7f183c0c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:43.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f1839d9b700 1 --2- 172.21.15.92:0/1295574827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 --2- 172.21.15.92:0/1295574827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 -- 172.21.15.92:0/1295574827 >> v2:172.21.15.158:3300/0 conn(0x7f183c0c0150 msgr2=0x7f183c0c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 --2- 172.21.15.92:0/1295574827 >> v2:172.21.15.158:3300/0 conn(0x7f183c0c0150 0x7f183c0c05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 -- 172.21.15.92:0/1295574827 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c06a140 con 0x7f183c153f90 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183ad9d700 1 --2- 172.21.15.92:0/1295574827 >> v2:172.21.15.158:3300/0 conn(0x7f183c0c0150 0x7f183c0c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183a59c700 1 --2- 172.21.15.92:0/1295574827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c0a9d20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f183c06bdd0 tx=0x7f182c009840).ready entity=mon.0 client_cookie=7071c222373b3f69 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183959a700 1 -- 172.21.15.92:0/1295574827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00e070 con 0x7f183c153f90 2023-05-16T07:17:43.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183959a700 1 -- 172.21.15.92:0/1295574827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00ba50 con 0x7f183c153f90 2023-05-16T07:17:43.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.088+0000 7f183959a700 1 -- 172.21.15.92:0/1295574827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00bdd0 con 0x7f183c153f90 2023-05-16T07:17:43.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 -- 172.21.15.92:0/1295574827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 msgr2=0x7f183c0a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 --2- 172.21.15.92:0/1295574827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c0a9d20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f183c06bdd0 tx=0x7f182c009840).stop 2023-05-16T07:17:43.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 -- 172.21.15.92:0/1295574827 shutdown_connections 2023-05-16T07:17:43.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 --2- 172.21.15.92:0/1295574827 >> v2:172.21.15.158:3300/0 conn(0x7f183c0c0150 0x7f183c0c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 --2- 172.21.15.92:0/1295574827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c0a9d20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 --2- 172.21.15.92:0/1295574827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 -- 172.21.15.92:0/1295574827 >> 172.21.15.92:0/1295574827 conn(0x7f183c0cb600 msgr2=0x7f183c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:43.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 -- 172.21.15.92:0/1295574827 shutdown_connections 2023-05-16T07:17:43.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.089+0000 7f183b59e700 1 -- 172.21.15.92:0/1295574827 wait complete. 2023-05-16T07:17:43.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.090+0000 7f183b59e700 1 Processor -- start 2023-05-16T07:17:43.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.090+0000 7f183b59e700 1 -- start start 2023-05-16T07:17:43.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f183c0c0150 0x7f183c0a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0af6f0 con 0x7f183c153f90 2023-05-16T07:17:43.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0aeb00 con 0x7f183c0c0150 2023-05-16T07:17:43.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0a1f60 con 0x7f183c0c5d00 2023-05-16T07:17:43.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c142f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46226/0 (socket says 172.21.15.92:46226) 2023-05-16T07:17:43.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 -- 172.21.15.92:0/3676853663 learned_addr learned my addr 172.21.15.92:0/3676853663 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:43.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183a59c700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f183c0c0150 0x7f183c0a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f1839d9b700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 -- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 msgr2=0x7f183c0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 -- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f183c0c0150 msgr2=0x7f183c0a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f183c0c0150 0x7f183c0a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 -- 172.21.15.92:0/3676853663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c009ae0 con 0x7f183c153f90 2023-05-16T07:17:43.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183ad9d700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c142f10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f183400c880 tx=0x7f1834008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834010580 con 0x7f183c153f90 2023-05-16T07:17:43.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18340106e0 con 0x7f183c153f90 2023-05-16T07:17:43.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c0a2240 con 0x7f183c153f90 2023-05-16T07:17:43.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834010a60 con 0x7f183c153f90 2023-05-16T07:17:43.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.091+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c0a2800 con 0x7f183c153f90 2023-05-16T07:17:43.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.092+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c0af8a0 con 0x7f183c153f90 2023-05-16T07:17:43.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.093+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f183401b460 con 0x7f183c153f90 2023-05-16T07:17:43.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.093+0000 7f18327fc700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f181c05ed90 0x7f181c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.093+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f1834079d30 con 0x7f183c153f90 2023-05-16T07:17:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.093+0000 7f183a59c700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f181c05ed90 0x7f181c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.094+0000 7f183a59c700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f181c05ed90 0x7f181c061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f182c00ba20 tx=0x7f182c009840).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.097+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f183c0af8a0 con 0x7f183c153f90 2023-05-16T07:17:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.263+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f183c0af8a0 con 0x7f183c153f90 2023-05-16T07:17:43.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.269+0000 7f18327fc700 1 -- 172.21.15.92:0/3676853663 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f183c0af8a0 con 0x7f183c153f90 2023-05-16T07:17:43.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f181c05ed90 msgr2=0x7f181c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f181c05ed90 0x7f181c061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f182c00ba20 tx=0x7f182c009840).stop 2023-05-16T07:17:43.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 msgr2=0x7f183c142f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c142f10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f183400c880 tx=0x7f1834008040).stop 2023-05-16T07:17:43.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 shutdown_connections 2023-05-16T07:17:43.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f181c05ed90 0x7f181c061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f183c153f90 0x7f183c142f10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f183c0c0150 0x7f183c0a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 --2- 172.21.15.92:0/3676853663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f183c0c5d00 0x7f183c0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 >> 172.21.15.92:0/3676853663 conn(0x7f183c0cb600 msgr2=0x7f183c0c05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:43.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 shutdown_connections 2023-05-16T07:17:43.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.271+0000 7f183b59e700 1 -- 172.21.15.92:0/3676853663 wait complete. 2023-05-16T07:17:43.286 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:17:43.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:43.313 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:17:43.345 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:17:43.345 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:17:43.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:43.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.461+0000 7f68d36af700 1 Processor -- start 2023-05-16T07:17:43.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.461+0000 7f68d36af700 1 -- start start 2023-05-16T07:17:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d36af700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68cc154170 0x7f68cc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d36af700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154550 0x7f68cc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d36af700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f68cc0696a0 0x7f68cc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d36af700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc0c0660 con 0x7f68cc154170 2023-05-16T07:17:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d36af700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc06a020 con 0x7f68cc0696a0 2023-05-16T07:17:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d36af700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0bfd30 con 0x7f68cc154550 2023-05-16T07:17:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68cc154170 0x7f68cc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68cc154170 0x7f68cc0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46230/0 (socket says 172.21.15.92:46230) 2023-05-16T07:17:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 -- 172.21.15.92:0/3992578315 learned_addr learned my addr 172.21.15.92:0/3992578315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 -- 172.21.15.92:0/3992578315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154550 msgr2=0x7f68cc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 --2- 172.21.15.92:0/3992578315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154550 0x7f68cc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 -- 172.21.15.92:0/3992578315 >> v2:172.21.15.158:3300/0 conn(0x7f68cc0696a0 msgr2=0x7f68cc069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d1c4c700 1 --2- 172.21.15.92:0/3992578315 >> v2:172.21.15.158:3300/0 conn(0x7f68cc0696a0 0x7f68cc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 --2- 172.21.15.92:0/3992578315 >> v2:172.21.15.158:3300/0 conn(0x7f68cc0696a0 0x7f68cc069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 -- 172.21.15.92:0/3992578315 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0c0010 con 0x7f68cc154170 2023-05-16T07:17:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d1c4c700 1 --2- 172.21.15.92:0/3992578315 >> v2:172.21.15.158:3300/0 conn(0x7f68cc0696a0 0x7f68cc069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:43.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.462+0000 7f68d144b700 1 --2- 172.21.15.92:0/3992578315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc154170 0x7f68cc0c5c70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f68cc158f90 tx=0x7f68bc009840).ready entity=mon.0 client_cookie=99fddc8b6b3f8c53 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68cbfff700 1 -- 172.21.15.92:0/3992578315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00e070 con 0x7f68cc154170 2023-05-16T07:17:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68cbfff700 1 -- 172.21.15.92:0/3992578315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc00ba50 con 0x7f68cc154170 2023-05-16T07:17:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68cbfff700 1 -- 172.21.15.92:0/3992578315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00bdd0 con 0x7f68cc154170 2023-05-16T07:17:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 -- 172.21.15.92:0/3992578315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc154170 msgr2=0x7f68cc0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 --2- 172.21.15.92:0/3992578315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc154170 0x7f68cc0c5c70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f68cc158f90 tx=0x7f68bc009840).stop 2023-05-16T07:17:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 -- 172.21.15.92:0/3992578315 shutdown_connections 2023-05-16T07:17:43.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 --2- 172.21.15.92:0/3992578315 >> v2:172.21.15.158:3300/0 conn(0x7f68cc0696a0 0x7f68cc069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 --2- 172.21.15.92:0/3992578315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc154170 0x7f68cc0c5c70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 --2- 172.21.15.92:0/3992578315 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154550 0x7f68cc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:43.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 -- 172.21.15.92:0/3992578315 >> 172.21.15.92:0/3992578315 conn(0x7f68cc0cb700 msgr2=0x7f68cc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:43.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 -- 172.21.15.92:0/3992578315 shutdown_connections 2023-05-16T07:17:43.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.463+0000 7f68d36af700 1 -- 172.21.15.92:0/3992578315 wait complete. 2023-05-16T07:17:43.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.464+0000 7f68d36af700 1 Processor -- start 2023-05-16T07:17:43.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.464+0000 7f68d36af700 1 -- start start 2023-05-16T07:17:43.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d36af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 0x7f68cc14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d36af700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154170 0x7f68cc150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d36af700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68cc154550 0x7f68cc164db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d36af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc151360 con 0x7f68cc0696a0 2023-05-16T07:17:43.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d36af700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc150ac0 con 0x7f68cc154550 2023-05-16T07:17:43.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d36af700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc150da0 con 0x7f68cc154170 2023-05-16T07:17:43.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d144b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 0x7f68cc14fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d144b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 0x7f68cc14fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46238/0 (socket says 172.21.15.92:46238) 2023-05-16T07:17:43.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d144b700 1 -- 172.21.15.92:0/846797575 learned_addr learned my addr 172.21.15.92:0/846797575 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:43.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d0c4a700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154170 0x7f68cc150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d1c4c700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68cc154550 0x7f68cc164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.465+0000 7f68d144b700 1 -- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154170 msgr2=0x7f68cc150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d144b700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154170 0x7f68cc150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d144b700 1 -- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68cc154550 msgr2=0x7f68cc164db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d144b700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68cc154550 0x7f68cc164db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d144b700 1 -- 172.21.15.92:0/846797575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc009ae0 con 0x7f68cc0696a0 2023-05-16T07:17:43.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d0c4a700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154170 0x7f68cc150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:43.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d1c4c700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68cc154550 0x7f68cc164db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:43.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d144b700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 0x7f68cc14fd90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f68bc009fd0 tx=0x7f68bc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00bbc0 con 0x7f68cc0696a0 2023-05-16T07:17:43.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc00bd20 con 0x7f68cc0696a0 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc151080 con 0x7f68cc0696a0 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc01e6c0 con 0x7f68cc0696a0 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.466+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc153000 con 0x7f68cc0696a0 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.467+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc0666c0 con 0x7f68cc0696a0 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.467+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f68bc025030 con 0x7f68cc0696a0 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.467+0000 7f68c97fa700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f68b805ed90 0x7f68b8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.468+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f68bc07b1e0 con 0x7f68cc0696a0 2023-05-16T07:17:43.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.468+0000 7f68d0c4a700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f68b805ed90 0x7f68b8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.468+0000 7f68d0c4a700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f68b805ed90 0x7f68b8061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f68c0006fd0 tx=0x7f68c0008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.472+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f68bc04c6b0 con 0x7f68cc0696a0 2023-05-16T07:17:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.675+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68cc06c350 con 0x7f68cc0696a0 2023-05-16T07:17:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.675+0000 7f68c97fa700 1 -- 172.21.15.92:0/846797575 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f68bc023070 con 0x7f68cc0696a0 2023-05-16T07:17:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:43.679 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/291769950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":291769950},{"type":"v1","addr":"172.21.15.158:6836","nonce":291769950}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1611701018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1611701018},{"type":"v1","addr":"172.21.15.158:6837","nonce":1611701018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4090767644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4090767644},{"type":"v1","addr":"172.21.15.92:6834","nonce":4090767644}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2240092587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2240092587},{"type":"v1","addr":"172.21.15.92:6835","nonce":2240092587}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-05-16T07:17:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f68b805ed90 msgr2=0x7f68b8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f68b805ed90 0x7f68b8061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f68c0006fd0 tx=0x7f68c0008040).stop 2023-05-16T07:17:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 msgr2=0x7f68cc14fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 0x7f68cc14fd90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f68bc009fd0 tx=0x7f68bc009840).stop 2023-05-16T07:17:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 shutdown_connections 2023-05-16T07:17:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f68b805ed90 0x7f68b8061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f68cc154550 0x7f68cc164db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68cc0696a0 0x7f68cc14fd90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 --2- 172.21.15.92:0/846797575 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68cc154170 0x7f68cc150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 >> 172.21.15.92:0/846797575 conn(0x7f68cc0cb700 msgr2=0x7f68cc0aa160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:43.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 shutdown_connections 2023-05-16T07:17:43.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.677+0000 7f68d36af700 1 -- 172.21.15.92:0/846797575 wait complete. 2023-05-16T07:17:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2023-05-16T07:17:43.693 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:17:43.693 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:17:43.693 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:17:43.694 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:43.694 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:17:43.694 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:17:43.694 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:17:43.695 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:43.695 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:17:43.695 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:17:43.695 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:17:43.695 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:43.696 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:17:43.696 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:17:43.696 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:17:43.696 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:17:43.702 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:17:43.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:17:43.703 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:17:43.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:17:43.706 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:17:43.706 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:17:43.707 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:17:43.707 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:17:43.745 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:17:43.746 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:43.747 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:17:43.747 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:43.748 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:17:43.749 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:43.749 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:17:43.750 DEBUG:teuthology.parallel:result is None 2023-05-16T07:17:43.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:17:43.817 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:17:43.817 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:17:43.829 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:17:43.829 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:17:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.871+0000 7fbb764ca700 1 Processor -- start 2023-05-16T07:17:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb764ca700 1 -- start start 2023-05-16T07:17:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb764ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb701570d0 0x7fbb701574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb764ca700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbb70062ff0 0x7fbb701585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb764ca700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb70158120 con 0x7fbb70069af0 2023-05-16T07:17:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb764ca700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb70069040 con 0x7fbb701570d0 2023-05-16T07:17:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb764ca700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb70157b40 con 0x7fbb70062ff0 2023-05-16T07:17:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.872+0000 7fbb6ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb701570d0 0x7fbb701574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb701570d0 0x7fbb701574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46268/0 (socket says 172.21.15.92:46268) 2023-05-16T07:17:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 -- 172.21.15.92:0/1263878201 learned_addr learned my addr 172.21.15.92:0/1263878201 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6f7fe700 1 --2- 172.21.15.92:0/1263878201 >> v2:172.21.15.158:3300/0 conn(0x7fbb70062ff0 0x7fbb701585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 -- 172.21.15.92:0/1263878201 >> v2:172.21.15.158:3300/0 conn(0x7fbb70062ff0 msgr2=0x7fbb701585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 --2- 172.21.15.92:0/1263878201 >> v2:172.21.15.158:3300/0 conn(0x7fbb70062ff0 0x7fbb701585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 -- 172.21.15.92:0/1263878201 >> v1:172.21.15.158:6789/0 conn(0x7fbb70069af0 legacy=0x7fbb700622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 -- 172.21.15.92:0/1263878201 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70069f60 con 0x7fbb701570d0 2023-05-16T07:17:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6f7fe700 1 --2- 172.21.15.92:0/1263878201 >> v2:172.21.15.158:3300/0 conn(0x7fbb70062ff0 0x7fbb701585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6ffff700 1 --2- 172.21.15.92:0/1263878201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb701570d0 0x7fbb701574b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fbb70065580 tx=0x7fbb600090f0).ready entity=mon.0 client_cookie=30903c5f58970738 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6effd700 1 -- 172.21.15.92:0/1263878201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000b600 con 0x7fbb701570d0 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6effd700 1 -- 172.21.15.92:0/1263878201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6000b760 con 0x7fbb701570d0 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb6effd700 1 -- 172.21.15.92:0/1263878201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000bae0 con 0x7fbb701570d0 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb764ca700 1 -- 172.21.15.92:0/1263878201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb701570d0 msgr2=0x7fbb701574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.873+0000 7fbb764ca700 1 --2- 172.21.15.92:0/1263878201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb701570d0 0x7fbb701574b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fbb70065580 tx=0x7fbb600090f0).stop 2023-05-16T07:17:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.874+0000 7fbb764ca700 1 -- 172.21.15.92:0/1263878201 shutdown_connections 2023-05-16T07:17:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.874+0000 7fbb764ca700 1 --2- 172.21.15.92:0/1263878201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb701570d0 0x7fbb701574b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.874+0000 7fbb764ca700 1 --2- 172.21.15.92:0/1263878201 >> v2:172.21.15.158:3300/0 conn(0x7fbb70062ff0 0x7fbb701585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.874+0000 7fbb764ca700 1 -- 172.21.15.92:0/1263878201 >> 172.21.15.92:0/1263878201 conn(0x7fbb700c8aa0 msgr2=0x7fbb700c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.874+0000 7fbb764ca700 1 -- 172.21.15.92:0/1263878201 shutdown_connections 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.874+0000 7fbb764ca700 1 -- 172.21.15.92:0/1263878201 wait complete. 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.875+0000 7fbb764ca700 1 Processor -- start 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.875+0000 7fbb764ca700 1 -- start start 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.875+0000 7fbb764ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 0x7fbb70145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.875+0000 7fbb764ca700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb70069af0 0x7fbb701553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.875+0000 7fbb764ca700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb701570d0 0x7fbb701529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb764ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70145b60 con 0x7fbb70062ff0 2023-05-16T07:17:43.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb764ca700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70164e90 con 0x7fbb70069af0 2023-05-16T07:17:43.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb764ca700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70164040 con 0x7fbb701570d0 2023-05-16T07:17:43.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 0x7fbb70145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 0x7fbb70145670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46274/0 (socket says 172.21.15.92:46274) 2023-05-16T07:17:43.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 -- 172.21.15.92:0/2179841941 learned_addr learned my addr 172.21.15.92:0/2179841941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 -- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb701570d0 msgr2=0x7fbb701529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6f7fe700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb70069af0 0x7fbb701553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb74a67700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb701570d0 0x7fbb701529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb701570d0 0x7fbb701529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 -- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb70069af0 msgr2=0x7fbb701553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb70069af0 0x7fbb701553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:43.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 -- 172.21.15.92:0/2179841941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60009370 con 0x7fbb70062ff0 2023-05-16T07:17:43.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6f7fe700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb70069af0 0x7fbb701553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:43.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb6ffff700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 0x7fbb70145670 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7fbb7006bfc0 tx=0x7fbb60017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000b8d0 con 0x7fbb70062ff0 2023-05-16T07:17:43.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6000ba30 con 0x7fbb70062ff0 2023-05-16T07:17:43.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb701558d0 con 0x7fbb70062ff0 2023-05-16T07:17:43.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6001b660 con 0x7fbb70062ff0 2023-05-16T07:17:43.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.876+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb70164280 con 0x7fbb70062ff0 2023-05-16T07:17:43.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.877+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70167640 con 0x7fbb70062ff0 2023-05-16T07:17:43.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.878+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbb6002b070 con 0x7fbb70062ff0 2023-05-16T07:17:43.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.878+0000 7fbb57fff700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb5c05ed90 0x7fbb5c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:43.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.878+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fbb6007ba40 con 0x7fbb70062ff0 2023-05-16T07:17:43.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.878+0000 7fbb6f7fe700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb5c05ed90 0x7fbb5c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:43.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.879+0000 7fbb6f7fe700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb5c05ed90 0x7fbb5c061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbb64006d80 tx=0x7fbb64008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:43.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:43.882+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbb6004cf10 con 0x7fbb70062ff0 2023-05-16T07:17:43.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-05-16T07:17:43.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-05-16T07:17:44.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.061+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbb7006c350 con 0x7fbb70062ff0 2023-05-16T07:17:44.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.961+0000 7fbb57fff700 1 -- 172.21.15.92:0/2179841941 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbb6001f020 con 0x7fbb70062ff0 2023-05-16T07:17:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.963+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb5c05ed90 msgr2=0x7fbb5c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.963+0000 7fbb764ca700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb5c05ed90 0x7fbb5c061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbb64006d80 tx=0x7fbb64008040).stop 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.963+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 msgr2=0x7fbb70145670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.963+0000 7fbb764ca700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 0x7fbb70145670 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7fbb7006bfc0 tx=0x7fbb60017000).stop 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 shutdown_connections 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb5c05ed90 0x7fbb5c061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb701570d0 0x7fbb701529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb70062ff0 0x7fbb70145670 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 --2- 172.21.15.92:0/2179841941 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb70069af0 0x7fbb701553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 >> 172.21.15.92:0/2179841941 conn(0x7fbb700c8aa0 msgr2=0x7fbb700c3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 shutdown_connections 2023-05-16T07:17:44.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:44.964+0000 7fbb764ca700 1 -- 172.21.15.92:0/2179841941 wait complete. 2023-05-16T07:17:44.969 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:17:44.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:17:45.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.088+0000 7f325c3b5700 1 Processor -- start 2023-05-16T07:17:45.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.088+0000 7f325c3b5700 1 -- start start 2023-05-16T07:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.089+0000 7f325c3b5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3254153eb0 0x7f3254154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.089+0000 7f325c3b5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3254068c70 0x7f3254066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.089+0000 7f325c3b5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32540bc9b0 con 0x7f3254065000 2023-05-16T07:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.089+0000 7f325c3b5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32540bcc90 con 0x7f3254153eb0 2023-05-16T07:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.089+0000 7f325c3b5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540bcf70 con 0x7f3254068c70 2023-05-16T07:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325a952700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3254065000 0x7f32540622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57036/0 (socket says 172.21.15.92:57036) 2023-05-16T07:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325a952700 1 -- 172.21.15.92:0/1709290411 learned_addr learned my addr 172.21.15.92:0/1709290411 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f3259950700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3254068c70 0x7f3254066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325a151700 1 --2- 172.21.15.92:0/1709290411 >> v2:172.21.15.158:3300/0 conn(0x7f3254153eb0 0x7f3254154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772768451 0 0) 0x7f32540bc9b0 con 0x7f3254065000 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f323c0033f0 con 0x7f3254065000 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2634568207 0 0) 0x7f323c0033f0 con 0x7f3254065000 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3254068c70 msgr2=0x7f3254066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3254068c70 0x7f3254066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 >> v2:172.21.15.158:3300/0 conn(0x7f3254153eb0 msgr2=0x7f3254154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 --2- 172.21.15.92:0/1709290411 >> v2:172.21.15.158:3300/0 conn(0x7f3254153eb0 0x7f3254154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540bd250 con 0x7f3254065000 2023-05-16T07:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f324c001470 con 0x7f3254065000 2023-05-16T07:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 >> v1:172.21.15.92:6789/0 conn(0x7f3254065000 legacy=0x7f32540622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f323c0038e0 0x7f323c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f323c004690 0x7f323c004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f323c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540bcc90 con 0x7f323c004fc0 2023-05-16T07:17:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540bcf70 con 0x7f323c0038e0 2023-05-16T07:17:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c004430 con 0x7f323c004690 2023-05-16T07:17:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325a952700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f323c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f325a151700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f323c0038e0 0x7f323c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.090+0000 7f3259950700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f323c004690 0x7f323c004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 -- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f323c004690 msgr2=0x7f323c004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f323c004690 0x7f323c004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 -- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f323c0038e0 msgr2=0x7f323c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f323c0038e0 0x7f323c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 -- 172.21.15.92:0/1709290411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0033f0 con 0x7f323c004fc0 2023-05-16T07:17:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f323c00b8e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f324c00c210 tx=0x7f324c001e90).ready entity=mon.0 client_cookie=c184c63b61d4d50b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c007a60 con 0x7f323c004fc0 2023-05-16T07:17:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c00cae0 con 0x7f323c004fc0 2023-05-16T07:17:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325914f700 1 -- 172.21.15.92:0/1709290411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c00ce60 con 0x7f323c004fc0 2023-05-16T07:17:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325c3b5700 1 -- 172.21.15.92:0/1709290411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 msgr2=0x7f323c00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325c3b5700 1 --2- 172.21.15.92:0/1709290411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f323c00b8e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f324c00c210 tx=0x7f324c001e90).stop 2023-05-16T07:17:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325a952700 1 -- 172.21.15.92:0/1709290411 reap_dead start 2023-05-16T07:17:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325c3b5700 1 -- 172.21.15.92:0/1709290411 shutdown_connections 2023-05-16T07:17:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325c3b5700 1 -- 172.21.15.92:0/1709290411 >> 172.21.15.92:0/1709290411 conn(0x7f32540c4430 msgr2=0x7f32540c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325c3b5700 1 -- 172.21.15.92:0/1709290411 shutdown_connections 2023-05-16T07:17:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.091+0000 7f325c3b5700 1 -- 172.21.15.92:0/1709290411 wait complete. 2023-05-16T07:17:45.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 Processor -- start 2023-05-16T07:17:45.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 -- start start 2023-05-16T07:17:45.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f32540a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3254163b70 0x7f32540a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f325414f690 0x7f3254155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540aab40 con 0x7f323c004fc0 2023-05-16T07:17:45.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540aa380 con 0x7f3254163b70 2023-05-16T07:17:45.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325c3b5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540aae00 con 0x7f325414f690 2023-05-16T07:17:45.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325a151700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f32540a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325a151700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f32540a77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46298/0 (socket says 172.21.15.92:46298) 2023-05-16T07:17:45.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325a151700 1 -- 172.21.15.92:0/322254095 learned_addr learned my addr 172.21.15.92:0/322254095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:45.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f325a952700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f325414f690 0x7f3254155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.093+0000 7f3259950700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3254163b70 0x7f32540a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325a151700 1 -- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f325414f690 msgr2=0x7f3254155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325a151700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f325414f690 0x7f3254155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325a151700 1 -- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3254163b70 msgr2=0x7f32540a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325a151700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3254163b70 0x7f32540a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325a151700 1 -- 172.21.15.92:0/322254095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540ab0e0 con 0x7f323c004fc0 2023-05-16T07:17:45.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325a151700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f32540a77b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f3244009fd0 tx=0x7f324400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400e040 con 0x7f323c004fc0 2023-05-16T07:17:45.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324400f040 con 0x7f323c004fc0 2023-05-16T07:17:45.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32540aa020 con 0x7f323c004fc0 2023-05-16T07:17:45.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400fda0 con 0x7f323c004fc0 2023-05-16T07:17:45.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.094+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325414fd30 con 0x7f323c004fc0 2023-05-16T07:17:45.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.095+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325406c350 con 0x7f323c004fc0 2023-05-16T07:17:45.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.095+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f3244015660 con 0x7f323c004fc0 2023-05-16T07:17:45.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.095+0000 7f32527fc700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324005ed90 0x7f3240061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.096+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f324407a9c0 con 0x7f323c004fc0 2023-05-16T07:17:45.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.096+0000 7f3259950700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324005ed90 0x7f3240061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.096+0000 7f3259950700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324005ed90 0x7f3240061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f32540a7ca0 tx=0x7f3248009060).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.100+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f324404c070 con 0x7f323c004fc0 2023-05-16T07:17:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.322+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f32540ab270 con 0x7f323c004fc0 2023-05-16T07:17:45.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.323+0000 7f32527fc700 1 -- 172.21.15.92:0/322254095 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f3244019070 con 0x7f323c004fc0 2023-05-16T07:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:17:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:17:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:17:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:17:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:17:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:17:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:17:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:17:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:17:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:17:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:17:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:17:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:17:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:17:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:17:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:17:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:17:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:17:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:17:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:17:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:17:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:17:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:17:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:17:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:17:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:17:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:17:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:17:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:17:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:17:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:17:45.363 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:17:45.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324005ed90 msgr2=0x7f3240061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324005ed90 0x7f3240061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f32540a7ca0 tx=0x7f3248009060).stop 2023-05-16T07:17:45.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 msgr2=0x7f32540a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f32540a77b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f3244009fd0 tx=0x7f324400b040).stop 2023-05-16T07:17:45.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 shutdown_connections 2023-05-16T07:17:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f324005ed90 0x7f3240061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f325414f690 0x7f3254155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c004fc0 0x7f32540a77b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 --2- 172.21.15.92:0/322254095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3254163b70 0x7f32540a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 >> 172.21.15.92:0/322254095 conn(0x7f32540c4430 msgr2=0x7f32540a55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 shutdown_connections 2023-05-16T07:17:45.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.325+0000 7f325c3b5700 1 -- 172.21.15.92:0/322254095 wait complete. 2023-05-16T07:17:45.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:45.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.475+0000 7f19a3e75700 1 Processor -- start 2023-05-16T07:17:45.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.475+0000 7f19a3e75700 1 -- start start 2023-05-16T07:17:45.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a3e75700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f199c069db0 0x7f199c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a3e75700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0c0660 con 0x7f199c154550 2023-05-16T07:17:45.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a3e75700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0c1b00 con 0x7f199c154170 2023-05-16T07:17:45.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a3e75700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199c0bfd30 con 0x7f199c069db0 2023-05-16T07:17:45.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a2412700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f199c069db0 0x7f199c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a1410700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f199c154550 0x7f199c0696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57054/0 (socket says 172.21.15.92:57054) 2023-05-16T07:17:45.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a1410700 1 -- 172.21.15.92:0/2826170441 learned_addr learned my addr 172.21.15.92:0/2826170441 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:45.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a2412700 1 -- 172.21.15.92:0/2826170441 >> v1:172.21.15.158:6789/0 conn(0x7f199c154170 legacy=0x7f199c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a2412700 1 -- 172.21.15.92:0/2826170441 >> v1:172.21.15.92:6789/0 conn(0x7f199c154550 legacy=0x7f199c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a2412700 1 -- 172.21.15.92:0/2826170441 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0c0010 con 0x7f199c069db0 2023-05-16T07:17:45.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a2412700 1 --2- 172.21.15.92:0/2826170441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c069db0 0x7f199c15f200 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f1994009e60 tx=0x7f1994009780).ready entity=mon.0 client_cookie=69592d711609a9f8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a0c0f700 1 -- 172.21.15.92:0/2826170441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199400e070 con 0x7f199c069db0 2023-05-16T07:17:45.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a0c0f700 1 -- 172.21.15.92:0/2826170441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199400bbf0 con 0x7f199c069db0 2023-05-16T07:17:45.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.476+0000 7f19a0c0f700 1 -- 172.21.15.92:0/2826170441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994012410 con 0x7f199c069db0 2023-05-16T07:17:45.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 -- 172.21.15.92:0/2826170441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c069db0 msgr2=0x7f199c15f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 --2- 172.21.15.92:0/2826170441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c069db0 0x7f199c15f200 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f1994009e60 tx=0x7f1994009780).stop 2023-05-16T07:17:45.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 -- 172.21.15.92:0/2826170441 shutdown_connections 2023-05-16T07:17:45.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 --2- 172.21.15.92:0/2826170441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c069db0 0x7f199c15f200 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 -- 172.21.15.92:0/2826170441 >> 172.21.15.92:0/2826170441 conn(0x7f199c0cb700 msgr2=0x7f199c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 -- 172.21.15.92:0/2826170441 shutdown_connections 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.477+0000 7f19a3e75700 1 -- 172.21.15.92:0/2826170441 wait complete. 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.478+0000 7f19a3e75700 1 Processor -- start 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.478+0000 7f19a3e75700 1 -- start start 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f199c069db0 0x7f199c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f199c154170 0x7f199c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 0x7f199c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c167e90 con 0x7f199c154550 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c164dc0 con 0x7f199c154170 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1650a0 con 0x7f199c069db0 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 0x7f199c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 0x7f199c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46316/0 (socket says 172.21.15.92:46316) 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 -- 172.21.15.92:0/201500379 learned_addr learned my addr 172.21.15.92:0/201500379 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a1410700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f199c154170 0x7f199c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a1c11700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f199c069db0 0x7f199c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 -- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f199c069db0 msgr2=0x7f199c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f199c069db0 0x7f199c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 -- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f199c154170 msgr2=0x7f199c1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f199c154170 0x7f199c1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 -- 172.21.15.92:0/201500379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1994009a20 con 0x7f199c154550 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a1410700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f199c154170 0x7f199c1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:45.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a2412700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 0x7f199c154960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f199400b6c0 tx=0x7f1994009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011750 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19940118b0 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c165380 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.479+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011c70 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.480+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c156880 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.480+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c0666c0 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.481+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f199401d630 con 0x7f199c154550 2023-05-16T07:17:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.481+0000 7f1991ffb700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f198005ed90 0x7f1980061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.481+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f199407bed0 con 0x7f199c154550 2023-05-16T07:17:45.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.481+0000 7f19a1c11700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f198005ed90 0x7f1980061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.482+0000 7f19a1c11700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f198005ed90 0x7f1980061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f1998003790 tx=0x7f1998009040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.485+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f199404d580 con 0x7f199c154550 2023-05-16T07:17:45.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.649+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f199c06c350 con 0x7f199c154550 2023-05-16T07:17:45.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.650+0000 7f1991ffb700 1 -- 172.21.15.92:0/201500379 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+14643 (secure 0 0 0) 0x7f1994050ba0 con 0x7f199c154550 2023-05-16T07:17:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:45.654 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":89,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:44.956694+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:45.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f198005ed90 msgr2=0x7f1980061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f198005ed90 0x7f1980061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f1998003790 tx=0x7f1998009040).stop 2023-05-16T07:17:45.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 msgr2=0x7f199c154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 0x7f199c154960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f199400b6c0 tx=0x7f1994009780).stop 2023-05-16T07:17:45.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 shutdown_connections 2023-05-16T07:17:45.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f198005ed90 0x7f1980061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f199c069db0 0x7f199c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f199c154550 0x7f199c154960 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 --2- 172.21.15.92:0/201500379 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f199c154170 0x7f199c1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 >> 172.21.15.92:0/201500379 conn(0x7f199c0cb700 msgr2=0x7f199c0ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:45.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 shutdown_connections 2023-05-16T07:17:45.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.652+0000 7f19a3e75700 1 -- 172.21.15.92:0/201500379 wait complete. 2023-05-16T07:17:45.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:17:45.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:17:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.779+0000 7f6e46a6f700 1 Processor -- start 2023-05-16T07:17:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.779+0000 7f6e46a6f700 1 -- start start 2023-05-16T07:17:45.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e46a6f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6e40154470 0x7f6e400aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e46a6f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 0x7f6e400685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e46a6f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400c94d0 con 0x7f6e40154090 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e46a6f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e400cb7e0 con 0x7f6e40154470 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e46a6f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40068aa0 con 0x7f6e40068160 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e4480b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6e40154090 0x7f6e400a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57068/0 (socket says 172.21.15.92:57068) 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.780+0000 7f6e4480b700 1 -- 172.21.15.92:0/3719210128 learned_addr learned my addr 172.21.15.92:0/3719210128 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3ffff700 1 --2- 172.21.15.92:0/3719210128 >> v2:172.21.15.158:3300/0 conn(0x7f6e40154470 0x7f6e400aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4500c700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 0x7f6e400685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020026339 0 0) 0x7f6e400c94d0 con 0x7f6e40154090 2023-05-16T07:17:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e280033f0 con 0x7f6e40154090 2023-05-16T07:17:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897256945 0 0) 0x7f6e280033f0 con 0x7f6e40154090 2023-05-16T07:17:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 msgr2=0x7f6e400685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 0x7f6e400685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 >> v2:172.21.15.158:3300/0 conn(0x7f6e40154470 msgr2=0x7f6e400aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 --2- 172.21.15.92:0/3719210128 >> v2:172.21.15.158:3300/0 conn(0x7f6e40154470 0x7f6e400aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400c88c0 con 0x7f6e40154090 2023-05-16T07:17:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6e30001470 con 0x7f6e40154090 2023-05-16T07:17:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3ffff700 1 --2- 172.21.15.92:0/3719210128 >> v2:172.21.15.158:3300/0 conn(0x7f6e40154470 0x7f6e400aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 >> v1:172.21.15.92:6789/0 conn(0x7f6e40154090 legacy=0x7f6e400a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e280038e0 0x7f6e28003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e28004690 0x7f6e28004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 0x7f6e28005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400cb7e0 con 0x7f6e280038e0 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40068aa0 con 0x7f6e28004690 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2800b8e0 con 0x7f6e40068160 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e280038e0 0x7f6e28003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e3ffff700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e28004690 0x7f6e28004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4500c700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 0x7f6e28005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 -- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 msgr2=0x7f6e28005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e40068160 0x7f6e28005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 -- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e28004690 msgr2=0x7f6e28004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e28004690 0x7f6e28004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 -- 172.21.15.92:0/3719210128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280033f0 con 0x7f6e280038e0 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.781+0000 7f6e4480b700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e280038e0 0x7f6e28003d00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f6e3000c210 tx=0x7f6e30001e90).ready entity=mon.0 client_cookie=4aa9b7a0ee07cab9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30007a60 con 0x7f6e280038e0 2023-05-16T07:17:45.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3000cae0 con 0x7f6e280038e0 2023-05-16T07:17:45.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e3f7fe700 1 -- 172.21.15.92:0/3719210128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3000ce60 con 0x7f6e280038e0 2023-05-16T07:17:45.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e46a6f700 1 -- 172.21.15.92:0/3719210128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e280038e0 msgr2=0x7f6e28003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/3719210128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e280038e0 0x7f6e28003d00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f6e3000c210 tx=0x7f6e30001e90).stop 2023-05-16T07:17:45.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e4500c700 1 -- 172.21.15.92:0/3719210128 reap_dead start 2023-05-16T07:17:45.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e46a6f700 1 -- 172.21.15.92:0/3719210128 shutdown_connections 2023-05-16T07:17:45.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e46a6f700 1 -- 172.21.15.92:0/3719210128 >> 172.21.15.92:0/3719210128 conn(0x7f6e40067920 msgr2=0x7f6e400cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:45.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e46a6f700 1 -- 172.21.15.92:0/3719210128 shutdown_connections 2023-05-16T07:17:45.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.782+0000 7f6e46a6f700 1 -- 172.21.15.92:0/3719210128 wait complete. 2023-05-16T07:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.783+0000 7f6e46a6f700 1 Processor -- start 2023-05-16T07:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.783+0000 7f6e46a6f700 1 -- start start 2023-05-16T07:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e46a6f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e280038e0 0x7f6e4014ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e46a6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 0x7f6e401682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e46a6f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e40143930 0x7f6e400c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e46a6f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4014f380 con 0x7f6e400aeb40 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e46a6f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400ae6b0 con 0x7f6e40143930 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e46a6f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e401687a0 con 0x7f6e280038e0 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e3ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 0x7f6e401682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e3ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 0x7f6e401682b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46334/0 (socket says 172.21.15.92:46334) 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e3ffff700 1 -- 172.21.15.92:0/554934942 learned_addr learned my addr 172.21.15.92:0/554934942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e4480b700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e280038e0 0x7f6e4014ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.784+0000 7f6e4500c700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e40143930 0x7f6e400c00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3ffff700 1 -- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e280038e0 msgr2=0x7f6e4014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3ffff700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e280038e0 0x7f6e4014ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3ffff700 1 -- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e40143930 msgr2=0x7f6e400c00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3ffff700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e40143930 0x7f6e400c00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3ffff700 1 -- 172.21.15.92:0/554934942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400add80 con 0x7f6e400aeb40 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e4500c700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e40143930 0x7f6e400c00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3ffff700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 0x7f6e401682b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f6e34011d40 tx=0x7f6e34009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400fac0 con 0x7f6e400aeb40 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3400fc20 con 0x7f6e400aeb40 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400ae060 con 0x7f6e400aeb40 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34014440 con 0x7f6e400aeb40 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.785+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e40143db0 con 0x7f6e400aeb40 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.786+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e40066730 con 0x7f6e400aeb40 2023-05-16T07:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.786+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6e34024070 con 0x7f6e400aeb40 2023-05-16T07:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.786+0000 7f6e3cff9700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e2c05ed60 0x7f6e2c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.786+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6e3407a850 con 0x7f6e400aeb40 2023-05-16T07:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.787+0000 7f6e4480b700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e2c05ed60 0x7f6e2c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.787+0000 7f6e4480b700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e2c05ed60 0x7f6e2c061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6e30001ed0 tx=0x7f6e30015000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.790+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6e3404bf00 con 0x7f6e400aeb40 2023-05-16T07:17:45.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.951+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6e400ae840 con 0x7f6e400aeb40 2023-05-16T07:17:45.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.951+0000 7f6e3cff9700 1 -- 172.21.15.92:0/554934942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v89) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6e34018070 con 0x7f6e400aeb40 2023-05-16T07:17:45.955 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:17:45.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e2c05ed60 msgr2=0x7f6e2c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e2c05ed60 0x7f6e2c061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6e30001ed0 tx=0x7f6e30015000).stop 2023-05-16T07:17:45.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 msgr2=0x7f6e401682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:45.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 0x7f6e401682b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f6e34011d40 tx=0x7f6e34009060).stop 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 shutdown_connections 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6e2c05ed60 0x7f6e2c061220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6e280038e0 0x7f6e4014ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e40143930 0x7f6e400c00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 --2- 172.21.15.92:0/554934942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e400aeb40 0x7f6e401682b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 >> 172.21.15.92:0/554934942 conn(0x7f6e40067920 msgr2=0x7f6e4014fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 shutdown_connections 2023-05-16T07:17:45.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:45.953+0000 7f6e46a6f700 1 -- 172.21.15.92:0/554934942 wait complete. 2023-05-16T07:17:45.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:17:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.078+0000 7f5b14df9700 1 Processor -- start 2023-05-16T07:17:46.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.078+0000 7f5b14df9700 1 -- start start 2023-05-16T07:17:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b14df9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5b10154090 0x7f5b100aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b14df9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5b10154470 0x7f5b100a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b14df9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100c94d0 con 0x7f5b100aa150 2023-05-16T07:17:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b14df9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b100aa5a0 con 0x7f5b10154470 2023-05-16T07:17:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b14df9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b100cb7e0 con 0x7f5b10154090 2023-05-16T07:17:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b0dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5b10154470 0x7f5b100a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b0ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b100aa150 0x7f5b10068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57090/0 (socket says 172.21.15.92:57090) 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.079+0000 7f5b0ed9d700 1 -- 172.21.15.92:0/3694564314 learned_addr learned my addr 172.21.15.92:0/3694564314 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0e59c700 1 --2- 172.21.15.92:0/3694564314 >> v2:172.21.15.158:3300/0 conn(0x7f5b10154090 0x7f5b100aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0dd9b700 1 -- 172.21.15.92:0/3694564314 >> v2:172.21.15.158:3300/0 conn(0x7f5b10154090 msgr2=0x7f5b100aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0dd9b700 1 --2- 172.21.15.92:0/3694564314 >> v2:172.21.15.158:3300/0 conn(0x7f5b10154090 0x7f5b100aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0dd9b700 1 -- 172.21.15.92:0/3694564314 >> v1:172.21.15.92:6789/0 conn(0x7f5b100aa150 legacy=0x7f5b10068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0d59a700 1 -- 172.21.15.92:0/3694564314 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945634873 0 0) 0x7f5b100c94d0 con 0x7f5b100aa150 2023-05-16T07:17:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0dd9b700 1 -- 172.21.15.92:0/3694564314 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100c88c0 con 0x7f5b10154470 2023-05-16T07:17:46.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0dd9b700 1 --2- 172.21.15.92:0/3694564314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b10154470 0x7f5b100a9c60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f5b1006bd10 tx=0x7f5af8009060).ready entity=mon.0 client_cookie=a9c07ff393215233 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0d59a700 1 -- 172.21.15.92:0/3694564314 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800b580 con 0x7f5b10154470 2023-05-16T07:17:46.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0d59a700 1 -- 172.21.15.92:0/3694564314 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af800b6e0 con 0x7f5b10154470 2023-05-16T07:17:46.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.080+0000 7f5b0d59a700 1 -- 172.21.15.92:0/3694564314 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800ba60 con 0x7f5b10154470 2023-05-16T07:17:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 -- 172.21.15.92:0/3694564314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b10154470 msgr2=0x7f5b100a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3694564314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b10154470 0x7f5b100a9c60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f5b1006bd10 tx=0x7f5af8009060).stop 2023-05-16T07:17:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 -- 172.21.15.92:0/3694564314 shutdown_connections 2023-05-16T07:17:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3694564314 >> v2:172.21.15.158:3300/0 conn(0x7f5b10154090 0x7f5b100aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3694564314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b10154470 0x7f5b100a9c60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 -- 172.21.15.92:0/3694564314 >> 172.21.15.92:0/3694564314 conn(0x7f5b10067920 msgr2=0x7f5b100cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 -- 172.21.15.92:0/3694564314 shutdown_connections 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.081+0000 7f5b14df9700 1 -- 172.21.15.92:0/3694564314 wait complete. 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 Processor -- start 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 -- start start 2023-05-16T07:17:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 0x7f5b10164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b10154090 0x7f5b10165380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b10154470 0x7f5b1014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101551e0 con 0x7f5b100aa150 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10155430 con 0x7f5b10154470 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.082+0000 7f5b14df9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10155710 con 0x7f5b10154090 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 0x7f5b10164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 0x7f5b10164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46348/0 (socket says 172.21.15.92:46348) 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 -- 172.21.15.92:0/3877065548 learned_addr learned my addr 172.21.15.92:0/3877065548 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0ed9d700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b10154470 0x7f5b1014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 -- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b10154090 msgr2=0x7f5b10165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b10154090 0x7f5b10165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 -- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b10154470 msgr2=0x7f5b1014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b10154470 0x7f5b1014f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 -- 172.21.15.92:0/3877065548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80092a0 con 0x7f5b100aa150 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b0e59c700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 0x7f5b10164bd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f5b0800d890 tx=0x7f5b08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b080125f0 con 0x7f5b100aa150 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08012750 con 0x7f5b100aa150 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08012ad0 con 0x7f5b100aa150 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b101559f0 con 0x7f5b100aa150 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.083+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b101650c0 con 0x7f5b100aa150 2023-05-16T07:17:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.084+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b10155f10 con 0x7f5b100aa150 2023-05-16T07:17:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.085+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5b08013480 con 0x7f5b100aa150 2023-05-16T07:17:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.085+0000 7f5b067fc700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5af005ed90 0x7f5af0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.085+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5b0807a0f0 con 0x7f5b100aa150 2023-05-16T07:17:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.085+0000 7f5b0dd9b700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5af005ed90 0x7f5af0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.086+0000 7f5b0dd9b700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5af005ed90 0x7f5af0061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5b1014e080 tx=0x7f5af8017000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.088+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5b0804b7c0 con 0x7f5b100aa150 2023-05-16T07:17:46.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.249+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b10155f10 con 0x7f5b100aa150 2023-05-16T07:17:46.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.250+0000 7f5b067fc700 1 -- 172.21.15.92:0/3877065548 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5b10155f10 con 0x7f5b100aa150 2023-05-16T07:17:46.254 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:17:46.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5af005ed90 msgr2=0x7f5af0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5af005ed90 0x7f5af0061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5b1014e080 tx=0x7f5af8017000).stop 2023-05-16T07:17:46.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 msgr2=0x7f5b10164bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 0x7f5b10164bd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f5b0800d890 tx=0x7f5b08008040).stop 2023-05-16T07:17:46.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 shutdown_connections 2023-05-16T07:17:46.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5af005ed90 0x7f5af0061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b10154470 0x7f5b1014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b100aa150 0x7f5b10164bd0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 --2- 172.21.15.92:0/3877065548 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5b10154090 0x7f5b10165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:46.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 >> 172.21.15.92:0/3877065548 conn(0x7f5b10067920 msgr2=0x7f5b10158560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:46.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 shutdown_connections 2023-05-16T07:17:46.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.252+0000 7f5b14df9700 1 -- 172.21.15.92:0/3877065548 wait complete. 2023-05-16T07:17:46.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:17:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.373+0000 7f217fbc7700 1 Processor -- start 2023-05-16T07:17:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.374+0000 7f217fbc7700 1 -- start start 2023-05-16T07:17:46.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.374+0000 7f217fbc7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2178154170 0x7f21780c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.374+0000 7f217fbc7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2178154550 0x7f21780c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.374+0000 7f217fbc7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780c0660 con 0x7f21780696a0 2023-05-16T07:17:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217fbc7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780c1b00 con 0x7f2178154170 2023-05-16T07:17:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217fbc7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780bfd30 con 0x7f2178154550 2023-05-16T07:17:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2178154170 0x7f21780c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2178154170 0x7f21780c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46354/0 (socket says 172.21.15.92:46354) 2023-05-16T07:17:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 -- 172.21.15.92:0/1150520682 learned_addr learned my addr 172.21.15.92:0/1150520682 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 -- 172.21.15.92:0/1150520682 >> v2:172.21.15.158:3300/0 conn(0x7f2178154550 msgr2=0x7f21780c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 --2- 172.21.15.92:0/1150520682 >> v2:172.21.15.158:3300/0 conn(0x7f2178154550 0x7f21780c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 -- 172.21.15.92:0/1150520682 >> v1:172.21.15.158:6789/0 conn(0x7f21780696a0 legacy=0x7f2178157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 -- 172.21.15.92:0/1150520682 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780c0010 con 0x7f2178154170 2023-05-16T07:17:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217d963700 1 --2- 172.21.15.92:0/1150520682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2178154170 0x7f21780c5c70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f21780844e0 tx=0x7f2168009840).ready entity=mon.0 client_cookie=78e7e2f99c2f0ce2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217c961700 1 -- 172.21.15.92:0/1150520682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216800e070 con 0x7f2178154170 2023-05-16T07:17:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217c961700 1 -- 172.21.15.92:0/1150520682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216800ba10 con 0x7f2178154170 2023-05-16T07:17:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.375+0000 7f217c961700 1 -- 172.21.15.92:0/1150520682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216800bd90 con 0x7f2178154170 2023-05-16T07:17:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 -- 172.21.15.92:0/1150520682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2178154170 msgr2=0x7f21780c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 --2- 172.21.15.92:0/1150520682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2178154170 0x7f21780c5c70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f21780844e0 tx=0x7f2168009840).stop 2023-05-16T07:17:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 -- 172.21.15.92:0/1150520682 shutdown_connections 2023-05-16T07:17:46.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 --2- 172.21.15.92:0/1150520682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2178154170 0x7f21780c5c70 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 --2- 172.21.15.92:0/1150520682 >> v2:172.21.15.158:3300/0 conn(0x7f2178154550 0x7f21780c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:46.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 -- 172.21.15.92:0/1150520682 >> 172.21.15.92:0/1150520682 conn(0x7f21780cb700 msgr2=0x7f21780c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:46.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 -- 172.21.15.92:0/1150520682 shutdown_connections 2023-05-16T07:17:46.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.376+0000 7f217fbc7700 1 -- 172.21.15.92:0/1150520682 wait complete. 2023-05-16T07:17:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.377+0000 7f217fbc7700 1 Processor -- start 2023-05-16T07:17:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.377+0000 7f217fbc7700 1 -- start start 2023-05-16T07:17:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217fbc7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 0x7f2178153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217fbc7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2178154170 0x7f2178154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217fbc7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2178154550 0x7f217814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217fbc7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2178155d50 con 0x7f21780696a0 2023-05-16T07:17:46.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217fbc7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2178154e50 con 0x7f2178154170 2023-05-16T07:17:46.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217fbc7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21781550d0 con 0x7f2178154550 2023-05-16T07:17:46.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 0x7f2178153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 0x7f2178153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46360/0 (socket says 172.21.15.92:46360) 2023-05-16T07:17:46.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 -- 172.21.15.92:0/3847840111 learned_addr learned my addr 172.21.15.92:0/3847840111 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:46.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 -- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2178154550 msgr2=0x7f217814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d162700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2178154170 0x7f2178154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217e164700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2178154550 0x7f217814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2178154550 0x7f217814df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 -- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2178154170 msgr2=0x7f2178154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2178154170 0x7f2178154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 -- 172.21.15.92:0/3847840111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009ae0 con 0x7f21780696a0 2023-05-16T07:17:46.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d162700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2178154170 0x7f2178154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:46.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217e164700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2178154550 0x7f217814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:46.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f217d963700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 0x7f2178153470 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f2168009930 tx=0x7f2168009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.378+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168013cc0 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.379+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168013e40 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.379+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21781553b0 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.379+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216801d660 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.379+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2178155970 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.379+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21780666c0 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.380+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f216801b040 con 0x7f21780696a0 2023-05-16T07:17:46.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.380+0000 7f2175ffb700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f216405ed90 0x7f2164061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.380+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f216807b380 con 0x7f21780696a0 2023-05-16T07:17:46.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.381+0000 7f217d162700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f216405ed90 0x7f2164061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.381+0000 7f217d162700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f216405ed90 0x7f2164061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f21780afdc0 tx=0x7f216c008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.384+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f216804ca30 con 0x7f21780696a0 2023-05-16T07:17:46.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.547+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f21780666c0 con 0x7f21780696a0 2023-05-16T07:17:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.548+0000 7f2175ffb700 1 -- 172.21.15.92:0/3847840111 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2168050050 con 0x7f21780696a0 2023-05-16T07:17:46.553 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:17:46.555 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:17:46.555 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:46.555 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:17:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.549+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f216405ed90 msgr2=0x7f2164061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.549+0000 7f217fbc7700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f216405ed90 0x7f2164061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f21780afdc0 tx=0x7f216c008040).stop 2023-05-16T07:17:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.549+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 msgr2=0x7f2178153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.549+0000 7f217fbc7700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 0x7f2178153470 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f2168009930 tx=0x7f2168009840).stop 2023-05-16T07:17:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 shutdown_connections 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f216405ed90 0x7f2164061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2178154550 0x7f217814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21780696a0 0x7f2178153470 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 --2- 172.21.15.92:0/3847840111 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2178154170 0x7f2178154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 >> 172.21.15.92:0/3847840111 conn(0x7f21780cb700 msgr2=0x7f21780c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 shutdown_connections 2023-05-16T07:17:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.550+0000 7f217fbc7700 1 -- 172.21.15.92:0/3847840111 wait complete. 2023-05-16T07:17:46.565 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:17:46.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:17:46.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.676+0000 7f54c98bf700 1 Processor -- start 2023-05-16T07:17:46.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.676+0000 7f54c98bf700 1 -- start start 2023-05-16T07:17:46.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c98bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54c41542c0 0x7f54c4062cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c98bf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c4155870 con 0x7f54c40be280 2023-05-16T07:17:46.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c98bf700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c41548b0 con 0x7f54c4153ee0 2023-05-16T07:17:46.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c98bf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c4155670 con 0x7f54c41542c0 2023-05-16T07:17:46.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54c41542c0 0x7f54c4062cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f54c40be280 0x7f54c4064e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57118/0 (socket says 172.21.15.92:57118) 2023-05-16T07:17:46.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c27fc700 1 -- 172.21.15.92:0/3621423963 learned_addr learned my addr 172.21.15.92:0/3621423963 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:46.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.677+0000 7f54c37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54c41542c0 0x7f54c4062cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46372/0 (socket says 172.21.15.92:46372) 2023-05-16T07:17:46.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c37fe700 1 -- 172.21.15.92:0/3621423963 >> v1:172.21.15.158:6789/0 conn(0x7f54c4153ee0 legacy=0x7f54c40bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:46.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c37fe700 1 -- 172.21.15.92:0/3621423963 >> v1:172.21.15.92:6789/0 conn(0x7f54c40be280 legacy=0x7f54c4064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c1ffb700 1 -- 172.21.15.92:0/3621423963 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452223104 0 0) 0x7f54c4155870 con 0x7f54c40be280 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c37fe700 1 -- 172.21.15.92:0/3621423963 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4154b90 con 0x7f54c41542c0 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c37fe700 1 --2- 172.21.15.92:0/3621423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c4062cc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f54bc009760 tx=0x7f54bc009060).ready entity=mon.0 client_cookie=38eaee68abe96309 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c1ffb700 1 -- 172.21.15.92:0/3621423963 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00b580 con 0x7f54c41542c0 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c1ffb700 1 -- 172.21.15.92:0/3621423963 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc00b6e0 con 0x7f54c41542c0 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c1ffb700 1 -- 172.21.15.92:0/3621423963 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00ba60 con 0x7f54c41542c0 2023-05-16T07:17:46.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 -- 172.21.15.92:0/3621423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 msgr2=0x7f54c4062cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 --2- 172.21.15.92:0/3621423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c4062cc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f54bc009760 tx=0x7f54bc009060).stop 2023-05-16T07:17:46.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 -- 172.21.15.92:0/3621423963 shutdown_connections 2023-05-16T07:17:46.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 --2- 172.21.15.92:0/3621423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c4062cc0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 -- 172.21.15.92:0/3621423963 >> 172.21.15.92:0/3621423963 conn(0x7f54c40c8420 msgr2=0x7f54c40c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:46.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 -- 172.21.15.92:0/3621423963 shutdown_connections 2023-05-16T07:17:46.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.678+0000 7f54c98bf700 1 -- 172.21.15.92:0/3621423963 wait complete. 2023-05-16T07:17:46.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 Processor -- start 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 -- start start 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54c40be280 0x7f54c41672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54c4153ee0 0x7f54c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c40c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4165410 con 0x7f54c41542c0 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c41677a0 con 0x7f54c40be280 2023-05-16T07:17:46.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c98bf700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4164800 con 0x7f54c4153ee0 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c40c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.680+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c40c3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46378/0 (socket says 172.21.15.92:46378) 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 -- 172.21.15.92:0/2736390586 learned_addr learned my addr 172.21.15.92:0/2736390586 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c2ffd700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54c40be280 0x7f54c41672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c27fc700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54c4153ee0 0x7f54c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 -- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54c4153ee0 msgr2=0x7f54c40a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54c4153ee0 0x7f54c40a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 -- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54c40be280 msgr2=0x7f54c41672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54c40be280 0x7f54c41672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 -- 172.21.15.92:0/2736390586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0092a0 con 0x7f54c41542c0 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c37fe700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c40c3350 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f54bc009eb0 tx=0x7f54bc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00ba60 con 0x7f54c41542c0 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc01b440 con 0x7f54c41542c0 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c4164ae0 con 0x7f54c41542c0 2023-05-16T07:17:46.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01b7e0 con 0x7f54c41542c0 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.681+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c41650a0 con 0x7f54c41542c0 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.682+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c40a7250 con 0x7f54c41542c0 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.682+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f54bc02a070 con 0x7f54c41542c0 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.683+0000 7f54baffd700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f54a405ed60 0x7f54a4061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.683+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f54bc07b360 con 0x7f54c41542c0 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.683+0000 7f54c2ffd700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f54a405ed60 0x7f54a4061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.683+0000 7f54c2ffd700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f54a405ed60 0x7f54a4061220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f54b4004290 tx=0x7f54b4001560).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:46.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.687+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f54c40a7250 con 0x7f54c41542c0 2023-05-16T07:17:46.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.850+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f54c40a7250 con 0x7f54c41542c0 2023-05-16T07:17:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.970+0000 7f54baffd700 1 -- 172.21.15.92:0/2736390586 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) v1 ==== 152+0+0 (secure 0 0 0) 0x7f54c40a7250 con 0x7f54c41542c0 2023-05-16T07:17:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.975+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f54a405ed60 msgr2=0x7f54a4061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.975+0000 7f54c98bf700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f54a405ed60 0x7f54a4061220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f54b4004290 tx=0x7f54b4001560).stop 2023-05-16T07:17:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.975+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 msgr2=0x7f54c40c3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.975+0000 7f54c98bf700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c40c3350 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f54bc009eb0 tx=0x7f54bc017000).stop 2023-05-16T07:17:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.976+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 shutdown_connections 2023-05-16T07:17:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.976+0000 7f54c98bf700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f54a405ed60 0x7f54a4061220 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.976+0000 7f54c98bf700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54c41542c0 0x7f54c40c3350 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.976+0000 7f54c98bf700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54c40be280 0x7f54c41672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.976+0000 7f54c98bf700 1 --2- 172.21.15.92:0/2736390586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54c4153ee0 0x7f54c40a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.976+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 >> 172.21.15.92:0/2736390586 conn(0x7f54c40c8420 msgr2=0x7f54c41525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.977+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 shutdown_connections 2023-05-16T07:17:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:46.977+0000 7f54c98bf700 1 -- 172.21.15.92:0/2736390586 wait complete. 2023-05-16T07:17:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:17:46.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:17:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.106+0000 7f0c9fcec700 1 Processor -- start 2023-05-16T07:17:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.107+0000 7f0c9fcec700 1 -- start start 2023-05-16T07:17:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.107+0000 7f0c9fcec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c980c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.107+0000 7f0c9fcec700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c980622b0 0x7f0c98062760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.107+0000 7f0c9fcec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c98062c50 con 0x7f0c98152e40 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.107+0000 7f0c9fcec700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c98063690 con 0x7f0c980622b0 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.107+0000 7f0c9fcec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9814fd70 con 0x7f0c9814f990 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9e289700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c980622b0 0x7f0c98062760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9d287700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0c98152e40 0x7f0c980c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57130/0 (socket says 172.21.15.92:57130) 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9d287700 1 -- 172.21.15.92:0/2764149908 learned_addr learned my addr 172.21.15.92:0/2764149908 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9da88700 1 --2- 172.21.15.92:0/2764149908 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c980c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9e289700 1 -- 172.21.15.92:0/2764149908 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 msgr2=0x7f0c980c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9e289700 1 --2- 172.21.15.92:0/2764149908 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c980c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9e289700 1 -- 172.21.15.92:0/2764149908 >> v1:172.21.15.92:6789/0 conn(0x7f0c98152e40 legacy=0x7f0c980c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9e289700 1 -- 172.21.15.92:0/2764149908 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980c52f0 con 0x7f0c980622b0 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9ca86700 1 -- 172.21.15.92:0/2764149908 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438489420 0 0) 0x7f0c8c001110 con 0x7f0c98152e40 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9e289700 1 --2- 172.21.15.92:0/2764149908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c980622b0 0x7f0c98062760 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f0c90009e60 tx=0x7f0c90009780).ready entity=mon.0 client_cookie=b25777c8cf973f3a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9ca86700 1 -- 172.21.15.92:0/2764149908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000e070 con 0x7f0c980622b0 2023-05-16T07:17:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9ca86700 1 -- 172.21.15.92:0/2764149908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000bc30 con 0x7f0c980622b0 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.108+0000 7f0c9ca86700 1 -- 172.21.15.92:0/2764149908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90012410 con 0x7f0c980622b0 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 -- 172.21.15.92:0/2764149908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c980622b0 msgr2=0x7f0c98062760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/2764149908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c980622b0 0x7f0c98062760 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f0c90009e60 tx=0x7f0c90009780).stop 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 -- 172.21.15.92:0/2764149908 shutdown_connections 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/2764149908 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c980c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/2764149908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c980622b0 0x7f0c98062760 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 -- 172.21.15.92:0/2764149908 >> 172.21.15.92:0/2764149908 conn(0x7f0c9805e820 msgr2=0x7f0c980c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 -- 172.21.15.92:0/2764149908 shutdown_connections 2023-05-16T07:17:47.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.109+0000 7f0c9fcec700 1 -- 172.21.15.92:0/2764149908 wait complete. 2023-05-16T07:17:47.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.110+0000 7f0c9fcec700 1 Processor -- start 2023-05-16T07:17:47.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.110+0000 7f0c9fcec700 1 -- start start 2023-05-16T07:17:47.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c980622b0 0x7f0c98168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:47.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c981500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:47.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 0x7f0c98155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98155220 con 0x7f0c98152e40 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98169170 con 0x7f0c980622b0 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98168c10 con 0x7f0c9814f990 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 0x7f0c98155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 0x7f0c98155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46390/0 (socket says 172.21.15.92:46390) 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 -- 172.21.15.92:0/3784827387 learned_addr learned my addr 172.21.15.92:0/3784827387 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9da88700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c980622b0 0x7f0c98168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9d287700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c981500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 -- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 msgr2=0x7f0c981500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c981500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 -- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c980622b0 msgr2=0x7f0c98168220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c980622b0 0x7f0c98168220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 -- 172.21.15.92:0/3784827387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009a20 con 0x7f0c98152e40 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9da88700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c980622b0 0x7f0c98168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9e289700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 0x7f0c98155c10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f0c9000b6c0 tx=0x7f0c90009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90011750 con 0x7f0c98152e40 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c900118b0 con 0x7f0c98152e40 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98155450 con 0x7f0c98152e40 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90011c70 con 0x7f0c98152e40 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.111+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c981556f0 con 0x7f0c98152e40 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.112+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9806c350 con 0x7f0c98152e40 2023-05-16T07:17:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.113+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0c9001d630 con 0x7f0c98152e40 2023-05-16T07:17:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.113+0000 7f0c95ffb700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8405ed90 0x7f0c84061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.113+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6223+0+0 (secure 0 0 0) 0x7f0c9007be60 con 0x7f0c98152e40 2023-05-16T07:17:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.113+0000 7f0c9da88700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8405ed90 0x7f0c84061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.114+0000 7f0c9da88700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8405ed90 0x7f0c84061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0c8800bb60 tx=0x7f0c88008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.117+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0c9004d290 con 0x7f0c98152e40 2023-05-16T07:17:47.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.279+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0c980c5ce0 con 0x7f0c98152e40 2023-05-16T07:17:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.977+0000 7f0c95ffb700 1 -- 172.21.15.92:0/3784827387 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0c900508b0 con 0x7f0c98152e40 2023-05-16T07:17:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.983+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8405ed90 msgr2=0x7f0c84061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.983+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8405ed90 0x7f0c84061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0c8800bb60 tx=0x7f0c88008040).stop 2023-05-16T07:17:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.983+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 msgr2=0x7f0c98155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.983+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 0x7f0c98155c10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f0c9000b6c0 tx=0x7f0c90009780).stop 2023-05-16T07:17:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.988+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 shutdown_connections 2023-05-16T07:17:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.988+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0c8405ed90 0x7f0c84061250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.988+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c98152e40 0x7f0c98155c10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.988+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0c980622b0 0x7f0c98168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.988+0000 7f0c9fcec700 1 --2- 172.21.15.92:0/3784827387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0c9814f990 0x7f0c981500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.988+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 >> 172.21.15.92:0/3784827387 conn(0x7f0c9805e820 msgr2=0x7f0c980c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.989+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 shutdown_connections 2023-05-16T07:17:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:47.989+0000 7f0c9fcec700 1 -- 172.21.15.92:0/3784827387 wait complete. 2023-05-16T07:17:47.997 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:17:48.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:17:48.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.123+0000 7fdc0b987700 1 Processor -- start 2023-05-16T07:17:48.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.123+0000 7fdc0b987700 1 -- start start 2023-05-16T07:17:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc0b987700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdc04154090 0x7fdc040aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc0b987700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc04154470 0x7fdc040a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc0b987700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc040c94d0 con 0x7fdc040aa150 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc0b987700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc040aa5a0 con 0x7fdc04154470 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc0b987700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc040cb7e0 con 0x7fdc04154090 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc04154470 0x7fdc040a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc09f24700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc040aa150 0x7fdc04068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57144/0 (socket says 172.21.15.92:57144) 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc04154470 0x7fdc040a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46398/0 (socket says 172.21.15.92:46398) 2023-05-16T07:17:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc09f24700 1 -- 172.21.15.92:0/3039072937 learned_addr learned my addr 172.21.15.92:0/3039072937 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc09723700 1 --2- 172.21.15.92:0/3039072937 >> v2:172.21.15.158:3300/0 conn(0x7fdc04154090 0x7fdc040aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 -- 172.21.15.92:0/3039072937 >> v2:172.21.15.158:3300/0 conn(0x7fdc04154090 msgr2=0x7fdc040aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 --2- 172.21.15.92:0/3039072937 >> v2:172.21.15.158:3300/0 conn(0x7fdc04154090 0x7fdc040aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 -- 172.21.15.92:0/3039072937 >> v1:172.21.15.92:6789/0 conn(0x7fdc040aa150 legacy=0x7fdc04068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc03fff700 1 -- 172.21.15.92:0/3039072937 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489012867 0 0) 0x7fdc040c94d0 con 0x7fdc040aa150 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 -- 172.21.15.92:0/3039072937 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc040c88c0 con 0x7fdc04154470 2023-05-16T07:17:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc09723700 1 --2- 172.21.15.92:0/3039072937 >> v2:172.21.15.158:3300/0 conn(0x7fdc04154090 0x7fdc040aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc08f22700 1 --2- 172.21.15.92:0/3039072937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154470 0x7fdc040a9c60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fdc0406bd10 tx=0x7fdbf8009060).ready entity=mon.0 client_cookie=fbcff039b20cf333 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc03fff700 1 -- 172.21.15.92:0/3039072937 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800b580 con 0x7fdc04154470 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc03fff700 1 -- 172.21.15.92:0/3039072937 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800b6e0 con 0x7fdc04154470 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.124+0000 7fdc03fff700 1 -- 172.21.15.92:0/3039072937 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800ba60 con 0x7fdc04154470 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 -- 172.21.15.92:0/3039072937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154470 msgr2=0x7fdc040a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3039072937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154470 0x7fdc040a9c60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fdc0406bd10 tx=0x7fdbf8009060).stop 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 -- 172.21.15.92:0/3039072937 shutdown_connections 2023-05-16T07:17:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3039072937 >> v2:172.21.15.158:3300/0 conn(0x7fdc04154090 0x7fdc040aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3039072937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154470 0x7fdc040a9c60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 -- 172.21.15.92:0/3039072937 >> 172.21.15.92:0/3039072937 conn(0x7fdc04067920 msgr2=0x7fdc040cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:48.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 -- 172.21.15.92:0/3039072937 shutdown_connections 2023-05-16T07:17:48.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.125+0000 7fdc0b987700 1 -- 172.21.15.92:0/3039072937 wait complete. 2023-05-16T07:17:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.126+0000 7fdc0b987700 1 Processor -- start 2023-05-16T07:17:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.126+0000 7fdc0b987700 1 -- start start 2023-05-16T07:17:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc0b987700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc040aa150 0x7fdc0414d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc0b987700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 0x7fdc040adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc0b987700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc04154470 0x7fdc040c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc0b987700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040ae5f0 con 0x7fdc04154090 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc0b987700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040ae8d0 con 0x7fdc04154470 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc0b987700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040aebb0 con 0x7fdc040aa150 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 0x7fdc040adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 0x7fdc040adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46400/0 (socket says 172.21.15.92:46400) 2023-05-16T07:17:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 -- 172.21.15.92:0/3631928163 learned_addr learned my addr 172.21.15.92:0/3631928163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 -- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc040aa150 msgr2=0x7fdc0414d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc040aa150 0x7fdc0414d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 -- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc04154470 msgr2=0x7fdc040c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc04154470 0x7fdc040c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 -- 172.21.15.92:0/3631928163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80092a0 con 0x7fdc04154090 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc08f22700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 0x7fdc040adbe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fdc0406bb30 tx=0x7fdbf8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.127+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800b850 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.128+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800b9b0 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.128+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf801b690 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.128+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc040aee90 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.128+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0414de60 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.128+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc04066730 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.129+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdbf800f070 con 0x7fdc04154090 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.129+0000 7fdc017fa700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbf005ed90 0x7fdbf0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.129+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fdbf807b580 con 0x7fdc04154090 2023-05-16T07:17:48.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.130+0000 7fdc09723700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbf005ed90 0x7fdbf0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.130+0000 7fdc09723700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbf005ed90 0x7fdbf0061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdbf400c3d0 tx=0x7fdbf40074d0).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.134+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdbf804c940 con 0x7fdc04154090 2023-05-16T07:17:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.335+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdc0406c350 con 0x7fdc04154090 2023-05-16T07:17:48.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.347+0000 7fdc017fa700 1 -- 172.21.15.92:0/3631928163 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v28) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdbf804ff60 con 0x7fdc04154090 2023-05-16T07:17:48.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.351+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbf005ed90 msgr2=0x7fdbf0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.351+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbf005ed90 0x7fdbf0061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdbf400c3d0 tx=0x7fdbf40074d0).stop 2023-05-16T07:17:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.351+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 msgr2=0x7fdc040adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.351+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 0x7fdc040adbe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fdc0406bb30 tx=0x7fdbf8017000).stop 2023-05-16T07:17:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.353+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 shutdown_connections 2023-05-16T07:17:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.353+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdbf005ed90 0x7fdbf0061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.353+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc040aa150 0x7fdc0414d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.353+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc04154470 0x7fdc040c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.353+0000 7fdc0b987700 1 --2- 172.21.15.92:0/3631928163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04154090 0x7fdc040adbe0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.353+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 >> 172.21.15.92:0/3631928163 conn(0x7fdc04067920 msgr2=0x7fdc0414e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.354+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 shutdown_connections 2023-05-16T07:17:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.354+0000 7fdc0b987700 1 -- 172.21.15.92:0/3631928163 wait complete. 2023-05-16T07:17:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 11 and data pool 12 2023-05-16T07:17:48.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:48.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 Processor -- start 2023-05-16T07:17:48.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 -- start start 2023-05-16T07:17:48.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb98069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb9806a2e0 0x7fcb98153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcb9806a740 0x7fcb98153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb980bfc80 con 0x7fcb9806a2e0 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb980c5d40 con 0x7fcb9806a740 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.485+0000 7fcb9e7c5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980c5fe0 con 0x7fcb98069750 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb9806a2e0 0x7fcb98153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb9806a2e0 0x7fcb98153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46424/0 (socket says 172.21.15.92:46424) 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 -- 172.21.15.92:0/866046504 learned_addr learned my addr 172.21.15.92:0/866046504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:48.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb9cd62700 1 --2- 172.21.15.92:0/866046504 >> v2:172.21.15.158:3300/0 conn(0x7fcb9806a740 0x7fcb98153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb97fff700 1 --2- 172.21.15.92:0/866046504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb98069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 -- 172.21.15.92:0/866046504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 msgr2=0x7fcb98069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 --2- 172.21.15.92:0/866046504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb98069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 -- 172.21.15.92:0/866046504 >> v2:172.21.15.158:3300/0 conn(0x7fcb9806a740 msgr2=0x7fcb98153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 --2- 172.21.15.92:0/866046504 >> v2:172.21.15.158:3300/0 conn(0x7fcb9806a740 0x7fcb98153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 -- 172.21.15.92:0/866046504 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb980c62c0 con 0x7fcb9806a2e0 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb9cd62700 1 --2- 172.21.15.92:0/866046504 >> v2:172.21.15.158:3300/0 conn(0x7fcb9806a740 0x7fcb98153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:48.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb977fe700 1 --2- 172.21.15.92:0/866046504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb98153ac0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fcb980c82c0 tx=0x7fcb8c0090f0).ready entity=mon.0 client_cookie=b07d14160571d295 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb96ffd700 1 -- 172.21.15.92:0/866046504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00b600 con 0x7fcb9806a2e0 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb96ffd700 1 -- 172.21.15.92:0/866046504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00b760 con 0x7fcb9806a2e0 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.486+0000 7fcb96ffd700 1 -- 172.21.15.92:0/866046504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00bae0 con 0x7fcb9806a2e0 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/866046504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 msgr2=0x7fcb98153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/866046504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb98153ac0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fcb980c82c0 tx=0x7fcb8c0090f0).stop 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/866046504 shutdown_connections 2023-05-16T07:17:48.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/866046504 >> v2:172.21.15.158:3300/0 conn(0x7fcb9806a740 0x7fcb98153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/866046504 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb98069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/866046504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb98153ac0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/866046504 >> 172.21.15.92:0/866046504 conn(0x7fcb980cb670 msgr2=0x7fcb980c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/866046504 shutdown_connections 2023-05-16T07:17:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.487+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/866046504 wait complete. 2023-05-16T07:17:48.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.488+0000 7fcb9e7c5700 1 Processor -- start 2023-05-16T07:17:48.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.488+0000 7fcb9e7c5700 1 -- start start 2023-05-16T07:17:48.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb980c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb980adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb980ae090 0x7fcb980afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98150ea0 con 0x7fcb9806a2e0 2023-05-16T07:17:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9814e220 con 0x7fcb980ae090 2023-05-16T07:17:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98167b40 con 0x7fcb98069750 2023-05-16T07:17:48.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb980adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb980adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46432/0 (socket says 172.21.15.92:46432) 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 -- 172.21.15.92:0/2961688310 learned_addr learned my addr 172.21.15.92:0/2961688310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9cd62700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb980ae090 0x7fcb980afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb97fff700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb980c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 -- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 msgr2=0x7fcb980c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb980c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 -- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb980ae090 msgr2=0x7fcb980afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb980ae090 0x7fcb980afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 -- 172.21.15.92:0/2961688310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c009370 con 0x7fcb9806a2e0 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb97fff700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb980c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb977fe700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb980adba0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fcb8c009130 tx=0x7fcb8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00b8d0 con 0x7fcb9806a2e0 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00ba30 con 0x7fcb9806a2e0 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb980ae820 con 0x7fcb9806a2e0 2023-05-16T07:17:48.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c01c670 con 0x7fcb9806a2e0 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.489+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb98168070 con 0x7fcb9806a2e0 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.490+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9806c350 con 0x7fcb9806a2e0 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.491+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcb8c00f070 con 0x7fcb9806a2e0 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.491+0000 7fcb7ffff700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb8405ed60 0x7fcb84061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.491+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fcb8c07b5a0 con 0x7fcb9806a2e0 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.491+0000 7fcb97fff700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb8405ed60 0x7fcb84061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.492+0000 7fcb97fff700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb8405ed60 0x7fcb84061220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fcb98167aa0 tx=0x7fcb88008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.495+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcb8c04ca20 con 0x7fcb9806a2e0 2023-05-16T07:17:48.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.662+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb980c4ed0 con 0x7fcb9806a2e0 2023-05-16T07:17:48.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.663+0000 7fcb7ffff700 1 -- 172.21.15.92:0/2961688310 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17874 (secure 0 0 0) 0x7fcb8c050040 con 0x7fcb9806a2e0 2023-05-16T07:17:48.667 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:48.667 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:48.337131+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:46.874223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:47.303318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:48.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb8405ed60 msgr2=0x7fcb84061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb8405ed60 0x7fcb84061220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fcb98167aa0 tx=0x7fcb88008040).stop 2023-05-16T07:17:48.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 msgr2=0x7fcb980adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb980adba0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fcb8c009130 tx=0x7fcb8c017000).stop 2023-05-16T07:17:48.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 shutdown_connections 2023-05-16T07:17:48.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb8405ed60 0x7fcb84061220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb98069750 0x7fcb980c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb980ae090 0x7fcb980afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 --2- 172.21.15.92:0/2961688310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb9806a2e0 0x7fcb980adba0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 >> 172.21.15.92:0/2961688310 conn(0x7fcb980cb670 msgr2=0x7fcb980ab6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 shutdown_connections 2023-05-16T07:17:48.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.665+0000 7fcb9e7c5700 1 -- 172.21.15.92:0/2961688310 wait complete. 2023-05-16T07:17:48.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.790+0000 7faa06752700 1 Processor -- start 2023-05-16T07:17:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.791+0000 7faa06752700 1 -- start start 2023-05-16T07:17:48.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.791+0000 7faa06752700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7faa0013a5a0 0x7faa00068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.791+0000 7faa06752700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa00154b80 con 0x7faa00068060 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.791+0000 7faa06752700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa00154e60 con 0x7faa000bd9a0 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.791+0000 7faa06752700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa00155140 con 0x7faa0013a5a0 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9f7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faa00068060 0x7faa000bd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57188/0 (socket says 172.21.15.92:57188) 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9f7fff700 1 -- 172.21.15.92:0/3529144255 learned_addr learned my addr 172.21.15.92:0/3529144255 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863969452 0 0) 0x7faa00154b80 con 0x7faa00068060 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9fffff700 1 --2- 172.21.15.92:0/3529144255 >> v2:172.21.15.158:3300/0 conn(0x7faa0013a5a0 0x7faa00068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec0033f0 con 0x7faa00068060 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667180145 0 0) 0x7fa9ec0033f0 con 0x7faa00068060 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 >> v2:172.21.15.158:3300/0 conn(0x7faa0013a5a0 msgr2=0x7faa00068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 --2- 172.21.15.92:0/3529144255 >> v2:172.21.15.158:3300/0 conn(0x7faa0013a5a0 0x7faa00068600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 >> v1:172.21.15.158:6789/0 conn(0x7faa000bd9a0 legacy=0x7faa000c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00155420 con 0x7faa00068060 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fa9e8001470 con 0x7faa00068060 2023-05-16T07:17:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 >> v1:172.21.15.92:6789/0 conn(0x7faa00068060 legacy=0x7faa000bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9ec0038e0 0x7fa9ec003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9ec004690 0x7fa9ec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec005000 0x7fa9ec00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00155140 con 0x7fa9ec004690 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00bf60 con 0x7fa9ec005000 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00c0f0 con 0x7fa9ec0038e0 2023-05-16T07:17:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9ec004690 0x7fa9ec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7fa9fffff700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9ec0038e0 0x7fa9ec003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.792+0000 7faa04cef700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec005000 0x7fa9ec00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9f7fff700 1 -- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9ec0038e0 msgr2=0x7fa9ec003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa9ec0038e0 0x7fa9ec003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9f7fff700 1 -- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec005000 msgr2=0x7fa9ec00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec005000 0x7fa9ec00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa04cef700 1 -- 172.21.15.92:0/3529144255 reap_dead start 2023-05-16T07:17:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9f7fff700 1 -- 172.21.15.92:0/3529144255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec0033f0 con 0x7fa9ec004690 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9ec004690 0x7fa9ec004b10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7faa00064fb0 tx=0x7fa9e8001e50).ready entity=mon.0 client_cookie=c33e990bf86cba3b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8007a40 con 0x7fa9ec004690 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800ca30 con 0x7fa9ec004690 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7fa9ff7fe700 1 -- 172.21.15.92:0/3529144255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800cdb0 con 0x7fa9ec004690 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 -- 172.21.15.92:0/3529144255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9ec004690 msgr2=0x7fa9ec004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9ec004690 0x7fa9ec004b10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7faa00064fb0 tx=0x7fa9e8001e50).stop 2023-05-16T07:17:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 -- 172.21.15.92:0/3529144255 shutdown_connections 2023-05-16T07:17:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 --2- 172.21.15.92:0/3529144255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9ec004690 0x7fa9ec004b10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 -- 172.21.15.92:0/3529144255 >> 172.21.15.92:0/3529144255 conn(0x7faa000c4430 msgr2=0x7faa000c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 -- 172.21.15.92:0/3529144255 shutdown_connections 2023-05-16T07:17:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.793+0000 7faa06752700 1 -- 172.21.15.92:0/3529144255 wait complete. 2023-05-16T07:17:48.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.794+0000 7faa06752700 1 Processor -- start 2023-05-16T07:17:48.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.794+0000 7faa06752700 1 -- start start 2023-05-16T07:17:48.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa06752700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec004690 0x7faa000c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa06752700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 0x7faa0014bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa06752700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faa00065000 0x7faa00065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa06752700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0014e910 con 0x7faa00136c70 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa06752700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0014acb0 con 0x7fa9ec004690 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa06752700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0014e480 con 0x7faa00065000 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7fa9f7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 0x7faa0014bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7fa9f7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 0x7faa0014bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46452/0 (socket says 172.21.15.92:46452) 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7fa9f7fff700 1 -- 172.21.15.92:0/1680063113 learned_addr learned my addr 172.21.15.92:0/1680063113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7faa04cef700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faa00065000 0x7faa00065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7fa9fffff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec004690 0x7faa000c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.795+0000 7fa9f7fff700 1 -- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faa00065000 msgr2=0x7faa00065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faa00065000 0x7faa00065480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9f7fff700 1 -- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec004690 msgr2=0x7faa000c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec004690 0x7faa000c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9f7fff700 1 -- 172.21.15.92:0/1680063113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00065970 con 0x7faa00136c70 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7faa04cef700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faa00065000 0x7faa00065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9fffff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec004690 0x7faa000c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:48.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9f7fff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 0x7faa0014bb00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fa9ec005640 tx=0x7fa9e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8007640 con 0x7faa00136c70 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8018070 con 0x7faa00136c70 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0014db50 con 0x7faa00136c70 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8004900 con 0x7faa00136c70 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.796+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0014e110 con 0x7faa00136c70 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.797+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0006c350 con 0x7faa00136c70 2023-05-16T07:17:48.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.797+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa9e801d070 con 0x7faa00136c70 2023-05-16T07:17:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.798+0000 7fa9fcff9700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9e005ed60 0x7fa9e0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.798+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fa9e807a5e0 con 0x7faa00136c70 2023-05-16T07:17:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.798+0000 7fa9fffff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9e005ed60 0x7fa9e0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.798+0000 7fa9fffff700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9e005ed60 0x7fa9e0061220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa9f0006d80 tx=0x7fa9f0008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.802+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa9e804b930 con 0x7faa00136c70 2023-05-16T07:17:48.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.964+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa0014e5c0 con 0x7faa00136c70 2023-05-16T07:17:48.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.965+0000 7fa9fcff9700 1 -- 172.21.15.92:0/1680063113 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17874 (secure 0 0 0) 0x7fa9e804ef50 con 0x7faa00136c70 2023-05-16T07:17:48.969 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:48.970 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:48.337131+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:46.874223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:47.303318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:48.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9e005ed60 msgr2=0x7fa9e0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9e005ed60 0x7fa9e0061220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa9f0006d80 tx=0x7fa9f0008040).stop 2023-05-16T07:17:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 msgr2=0x7faa0014bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 0x7faa0014bb00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fa9ec005640 tx=0x7fa9e8015000).stop 2023-05-16T07:17:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 shutdown_connections 2023-05-16T07:17:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa9e005ed60 0x7fa9e0061220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.967+0000 7faa06752700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faa00065000 0x7faa00065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.968+0000 7faa06752700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa9ec004690 0x7faa000c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.968+0000 7faa06752700 1 --2- 172.21.15.92:0/1680063113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa00136c70 0x7faa0014bb00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.968+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 >> 172.21.15.92:0/1680063113 conn(0x7faa000c4430 msgr2=0x7faa000622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.968+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 shutdown_connections 2023-05-16T07:17:48.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:48.968+0000 7faa06752700 1 -- 172.21.15.92:0/1680063113 wait complete. 2023-05-16T07:17:48.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:17:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.092+0000 7f8c5aede700 1 Processor -- start 2023-05-16T07:17:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5aede700 1 -- start start 2023-05-16T07:17:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5aede700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c540698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5aede700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8c54153f90 0x7f8c54067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5aede700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5405e760 con 0x7f8c540bd810 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5aede700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c54069ef0 con 0x7f8c54153f90 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5aede700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540bf540 con 0x7f8c54153bb0 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5a6dd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8c540bd810 0x7f8c540634d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57210/0 (socket says 172.21.15.92:57210) 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.093+0000 7f8c5a6dd700 1 -- 172.21.15.92:0/2587295447 learned_addr learned my addr 172.21.15.92:0/2587295447 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030043919 0 0) 0x7f8c5405e760 con 0x7f8c540bd810 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c3c0033f0 con 0x7f8c540bd810 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c596db700 1 --2- 172.21.15.92:0/2587295447 >> v2:172.21.15.158:3300/0 conn(0x7f8c54153f90 0x7f8c54067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c540698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358850563 0 0) 0x7f8c3c0033f0 con 0x7f8c540bd810 2023-05-16T07:17:49.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 msgr2=0x7f8c540698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:49.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c540698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 >> v2:172.21.15.158:3300/0 conn(0x7f8c54153f90 msgr2=0x7f8c54067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 --2- 172.21.15.92:0/2587295447 >> v2:172.21.15.158:3300/0 conn(0x7f8c54153f90 0x7f8c54067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c54068450 con 0x7f8c540bd810 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f8c4c001470 con 0x7f8c540bd810 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 >> v1:172.21.15.92:6789/0 conn(0x7f8c540bd810 legacy=0x7f8c540634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c3c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c004690 0x7f8c3c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c3c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54069ef0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540bf540 con 0x7f8c3c004690 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c00b8e0 con 0x7f8c54153bb0 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c3c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c596db700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c004690 0x7f8c3c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c5a6dd700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c3c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c59edc700 1 -- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 msgr2=0x7f8c3c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:49.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c3c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c59edc700 1 -- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c004690 msgr2=0x7f8c3c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.094+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c3c004690 0x7f8c3c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c59edc700 1 -- 172.21.15.92:0/2587295447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0033f0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5a6dd700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54153bb0 0x7f8c3c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c3c003d00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f8c4400e1c0 tx=0x7f8c4400b040).ready entity=mon.0 client_cookie=7a57b91b41165a30 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44009ac0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400f040 con 0x7f8c3c0038e0 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c58eda700 1 -- 172.21.15.92:0/2587295447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400eca0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5aede700 1 -- 172.21.15.92:0/2587295447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 msgr2=0x7f8c3c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2587295447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c3c003d00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f8c4400e1c0 tx=0x7f8c4400b040).stop 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5a6dd700 1 -- 172.21.15.92:0/2587295447 reap_dead start 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5aede700 1 -- 172.21.15.92:0/2587295447 shutdown_connections 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5aede700 1 -- 172.21.15.92:0/2587295447 >> 172.21.15.92:0/2587295447 conn(0x7f8c54062d80 msgr2=0x7f8c5405dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5aede700 1 -- 172.21.15.92:0/2587295447 shutdown_connections 2023-05-16T07:17:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.095+0000 7f8c5aede700 1 -- 172.21.15.92:0/2587295447 wait complete. 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.096+0000 7f8c5aede700 1 Processor -- start 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.096+0000 7f8c5aede700 1 -- start start 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5aede700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c541525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5aede700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c54165100 0x7f8c54063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5aede700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54165580 0x7f8c54063680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5aede700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540bf9e0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5aede700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54063b70 con 0x7f8c54165100 2023-05-16T07:17:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5aede700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54164f40 con 0x7f8c54165580 2023-05-16T07:17:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c541525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c541525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46470/0 (socket says 172.21.15.92:46470) 2023-05-16T07:17:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 -- 172.21.15.92:0/2718076509 learned_addr learned my addr 172.21.15.92:0/2718076509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c596db700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c54165100 0x7f8c54063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c5a6dd700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54165580 0x7f8c54063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 -- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54165580 msgr2=0x7f8c54063680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54165580 0x7f8c54063680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 -- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c54165100 msgr2=0x7f8c54063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c54165100 0x7f8c54063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c59edc700 1 -- 172.21.15.92:0/2718076509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540c0220 con 0x7f8c3c0038e0 2023-05-16T07:17:49.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.097+0000 7f8c596db700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c54165100 0x7f8c54063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.098+0000 7f8c59edc700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c541525f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f8c4400e7b0 tx=0x7f8c4401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.098+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44013e00 con 0x7f8c3c0038e0 2023-05-16T07:17:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.098+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4401e620 con 0x7f8c3c0038e0 2023-05-16T07:17:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.098+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c54152ae0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.098+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4401e7d0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.098+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c540bfbd0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.099+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c54047150 con 0x7f8c3c0038e0 2023-05-16T07:17:49.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.099+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8c4402e070 con 0x7f8c3c0038e0 2023-05-16T07:17:49.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.099+0000 7f8c51ffb700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c4005ed90 0x7f8c40061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:49.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.100+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f8c4407c080 con 0x7f8c3c0038e0 2023-05-16T07:17:49.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.100+0000 7f8c596db700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c4005ed90 0x7f8c40061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:49.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.100+0000 7f8c596db700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c4005ed90 0x7f8c40061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8c540ae1d0 tx=0x7f8c48008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:49.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.104+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8c4404d3d0 con 0x7f8c3c0038e0 2023-05-16T07:17:49.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.307+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8c54158460 con 0x7f8c3c0038e0 2023-05-16T07:17:49.651 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:17:49.648+0000 7fe5e1c0c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:17:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.983+0000 7f8c51ffb700 1 -- 172.21.15.92:0/2718076509 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c4402c070 con 0x7f8c3c0038e0 2023-05-16T07:17:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.985+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c4005ed90 msgr2=0x7f8c40061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.985+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c4005ed90 0x7f8c40061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8c540ae1d0 tx=0x7f8c48008040).stop 2023-05-16T07:17:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.985+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 msgr2=0x7f8c541525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.985+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c541525f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f8c4400e7b0 tx=0x7f8c4401a000).stop 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 shutdown_connections 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c4005ed90 0x7f8c40061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8c54165580 0x7f8c54063680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c3c0038e0 0x7f8c541525f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 --2- 172.21.15.92:0/2718076509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8c54165100 0x7f8c54063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:49.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 >> 172.21.15.92:0/2718076509 conn(0x7f8c54062d80 msgr2=0x7f8c540c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 shutdown_connections 2023-05-16T07:17:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:49.986+0000 7f8c5aede700 1 -- 172.21.15.92:0/2718076509 wait complete. 2023-05-16T07:17:50.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:17:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 Processor -- start 2023-05-16T07:17:50.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 -- start start 2023-05-16T07:17:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa2d8154090 0x7fa2d80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2d8154470 0x7fa2d80a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d80aa150 0x7fa2d8068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80aa5a0 con 0x7fa2d8154470 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80c94d0 con 0x7fa2d8154090 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.120+0000 7fa2ddcc5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80cb7e0 con 0x7fa2d80aa150 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2d8154470 0x7fa2d80a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2d8154470 0x7fa2d80a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46480/0 (socket says 172.21.15.92:46480) 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 -- 172.21.15.92:0/2963101516 learned_addr learned my addr 172.21.15.92:0/2963101516 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:50.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d77fe700 1 --2- 172.21.15.92:0/2963101516 >> v2:172.21.15.158:3300/0 conn(0x7fa2d8154090 0x7fa2d80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 -- 172.21.15.92:0/2963101516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d80aa150 msgr2=0x7fa2d8068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d7fff700 1 --2- 172.21.15.92:0/2963101516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d80aa150 0x7fa2d8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 --2- 172.21.15.92:0/2963101516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d80aa150 0x7fa2d8068160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 -- 172.21.15.92:0/2963101516 >> v2:172.21.15.158:3300/0 conn(0x7fa2d8154090 msgr2=0x7fa2d80aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 --2- 172.21.15.92:0/2963101516 >> v2:172.21.15.158:3300/0 conn(0x7fa2d8154090 0x7fa2d80aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 -- 172.21.15.92:0/2963101516 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80c88c0 con 0x7fa2d8154470 2023-05-16T07:17:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d6ffd700 1 --2- 172.21.15.92:0/2963101516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80a9c60 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fa2d806bd10 tx=0x7fa2cc0090f0).ready entity=mon.0 client_cookie=64ab9961e918ea30 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d67fc700 1 -- 172.21.15.92:0/2963101516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00b600 con 0x7fa2d8154470 2023-05-16T07:17:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d67fc700 1 -- 172.21.15.92:0/2963101516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00b760 con 0x7fa2d8154470 2023-05-16T07:17:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.121+0000 7fa2d67fc700 1 -- 172.21.15.92:0/2963101516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00bae0 con 0x7fa2d8154470 2023-05-16T07:17:50.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/2963101516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 msgr2=0x7fa2d80a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:50.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/2963101516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80a9c60 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fa2d806bd10 tx=0x7fa2cc0090f0).stop 2023-05-16T07:17:50.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/2963101516 shutdown_connections 2023-05-16T07:17:50.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/2963101516 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d80aa150 0x7fa2d8068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/2963101516 >> v2:172.21.15.158:3300/0 conn(0x7fa2d8154090 0x7fa2d80aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/2963101516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80a9c60 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/2963101516 >> 172.21.15.92:0/2963101516 conn(0x7fa2d8067920 msgr2=0x7fa2d80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:50.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/2963101516 shutdown_connections 2023-05-16T07:17:50.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.122+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/2963101516 wait complete. 2023-05-16T07:17:50.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.123+0000 7fa2ddcc5700 1 Processor -- start 2023-05-16T07:17:50.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.123+0000 7fa2ddcc5700 1 -- start start 2023-05-16T07:17:50.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d80aa150 0x7fa2d8157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d8154090 0x7fa2d813e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8157b60 con 0x7fa2d8154470 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c26a0 con 0x7fa2d80aa150 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c2b90 con 0x7fa2d8154090 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46488/0 (socket says 172.21.15.92:46488) 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 -- 172.21.15.92:0/868730477 learned_addr learned my addr 172.21.15.92:0/868730477 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d77fe700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d80aa150 0x7fa2d8157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 -- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d8154090 msgr2=0x7fa2d813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:50.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d6ffd700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d8154090 0x7fa2d813e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d8154090 0x7fa2d813e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 -- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d80aa150 msgr2=0x7fa2d8157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d80aa150 0x7fa2d8157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 -- 172.21.15.92:0/868730477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc009370 con 0x7fa2d8154470 2023-05-16T07:17:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d77fe700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d80aa150 0x7fa2d8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2d7fff700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80c9860 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fa2d000c550 tx=0x7fa2d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0010580 con 0x7fa2d8154470 2023-05-16T07:17:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d00106e0 con 0x7fa2d8154470 2023-05-16T07:17:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d81571f0 con 0x7fa2d8154470 2023-05-16T07:17:50.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0010a60 con 0x7fa2d8154470 2023-05-16T07:17:50.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.124+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8156ba0 con 0x7fa2d8154470 2023-05-16T07:17:50.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.125+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d8066730 con 0x7fa2d8154470 2023-05-16T07:17:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.126+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa2d001b460 con 0x7fa2d8154470 2023-05-16T07:17:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.126+0000 7fa2bf7fe700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c405ed60 0x7fa2c4061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.126+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fa2d0079dd0 con 0x7fa2d8154470 2023-05-16T07:17:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.126+0000 7fa2d77fe700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c405ed60 0x7fa2c4061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.127+0000 7fa2d77fe700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c405ed60 0x7fa2c4061220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa2c8007460 tx=0x7fa2c800b040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.130+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa2d004b0c0 con 0x7fa2d8154470 2023-05-16T07:17:50.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.335+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa2d806c350 con 0x7fa2d8154470 2023-05-16T07:17:50.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-05-16T07:17:50.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-05-16T07:17:50.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.990+0000 7fa2bf7fe700 1 -- 172.21.15.92:0/868730477 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2d004e6e0 con 0x7fa2d8154470 2023-05-16T07:17:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c405ed60 msgr2=0x7fa2c4061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c405ed60 0x7fa2c4061220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa2c8007460 tx=0x7fa2c800b040).stop 2023-05-16T07:17:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 msgr2=0x7fa2d80c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80c9860 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fa2d000c550 tx=0x7fa2d0008040).stop 2023-05-16T07:17:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 shutdown_connections 2023-05-16T07:17:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa2c405ed60 0x7fa2c4061220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8154470 0x7fa2d80c9860 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2d80aa150 0x7fa2d8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 --2- 172.21.15.92:0/868730477 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2d8154090 0x7fa2d813e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:50.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 >> 172.21.15.92:0/868730477 conn(0x7fa2d8067920 msgr2=0x7fa2d814da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:50.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 shutdown_connections 2023-05-16T07:17:50.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:50.993+0000 7fa2ddcc5700 1 -- 172.21.15.92:0/868730477 wait complete. 2023-05-16T07:17:51.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:51.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.120+0000 7f15da666700 1 Processor -- start 2023-05-16T07:17:51.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.120+0000 7f15da666700 1 -- start start 2023-05-16T07:17:51.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.121+0000 7f15da666700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15d4138840 0x7f15d4066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.121+0000 7f15da666700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f15d4065710 0x7f15d40a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.121+0000 7f15da666700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40bda10 con 0x7f15d40be4e0 2023-05-16T07:17:51.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.121+0000 7f15da666700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d4068be0 con 0x7f15d4138840 2023-05-16T07:17:51.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.121+0000 7f15da666700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d4068ec0 con 0x7f15d4065710 2023-05-16T07:17:51.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.121+0000 7f15d3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15d4138840 0x7f15d4066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15d4138840 0x7f15d4066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46492/0 (socket says 172.21.15.92:46492) 2023-05-16T07:17:51.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 -- 172.21.15.92:0/3700268278 learned_addr learned my addr 172.21.15.92:0/3700268278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:51.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 -- 172.21.15.92:0/3700268278 >> v2:172.21.15.158:3300/0 conn(0x7f15d4065710 msgr2=0x7f15d40a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:51.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 --2- 172.21.15.92:0/3700268278 >> v2:172.21.15.158:3300/0 conn(0x7f15d4065710 0x7f15d40a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 -- 172.21.15.92:0/3700268278 >> v1:172.21.15.158:6789/0 conn(0x7f15d40be4e0 legacy=0x7f15d4065000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 -- 172.21.15.92:0/3700268278 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40bd3c0 con 0x7f15d4138840 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d3fff700 1 --2- 172.21.15.92:0/3700268278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d4066e50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f15d405ecc0 tx=0x7f15c40090f0).ready entity=mon.0 client_cookie=6891c7c0d35cf2a6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d2ffd700 1 -- 172.21.15.92:0/3700268278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400b5a0 con 0x7f15d4138840 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d2ffd700 1 -- 172.21.15.92:0/3700268278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c400b700 con 0x7f15d4138840 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15d2ffd700 1 -- 172.21.15.92:0/3700268278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400ba80 con 0x7f15d4138840 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15da666700 1 -- 172.21.15.92:0/3700268278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 msgr2=0x7f15d4066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.122+0000 7f15da666700 1 --2- 172.21.15.92:0/3700268278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d4066e50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f15d405ecc0 tx=0x7f15c40090f0).stop 2023-05-16T07:17:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.123+0000 7f15da666700 1 -- 172.21.15.92:0/3700268278 shutdown_connections 2023-05-16T07:17:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.123+0000 7f15da666700 1 --2- 172.21.15.92:0/3700268278 >> v2:172.21.15.158:3300/0 conn(0x7f15d4065710 0x7f15d40a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.123+0000 7f15da666700 1 --2- 172.21.15.92:0/3700268278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d4066e50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.123+0000 7f15da666700 1 -- 172.21.15.92:0/3700268278 >> 172.21.15.92:0/3700268278 conn(0x7f15d40c4430 msgr2=0x7f15d40c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.123+0000 7f15da666700 1 -- 172.21.15.92:0/3700268278 shutdown_connections 2023-05-16T07:17:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.123+0000 7f15da666700 1 -- 172.21.15.92:0/3700268278 wait complete. 2023-05-16T07:17:51.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.124+0000 7f15da666700 1 Processor -- start 2023-05-16T07:17:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.124+0000 7f15da666700 1 -- start start 2023-05-16T07:17:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15d4065710 0x7f15d4164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15d40be4e0 0x7f15d40ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d414fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40ab890 con 0x7f15d4138840 2023-05-16T07:17:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4150090 con 0x7f15d4065710 2023-05-16T07:17:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40aba80 con 0x7f15d40be4e0 2023-05-16T07:17:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d414fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d414fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46500/0 (socket says 172.21.15.92:46500) 2023-05-16T07:17:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 -- 172.21.15.92:0/2827303163 learned_addr learned my addr 172.21.15.92:0/2827303163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d3fff700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15d4065710 0x7f15d4164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d37fe700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15d40be4e0 0x7f15d40ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 -- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15d40be4e0 msgr2=0x7f15d40ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15d40be4e0 0x7f15d40ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 -- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15d4065710 msgr2=0x7f15d4164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15d4065710 0x7f15d4164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 -- 172.21.15.92:0/2827303163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c4009370 con 0x7f15d4138840 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d37fe700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15d40be4e0 0x7f15d40ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15d8c03700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d414fba0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f15cc00b760 tx=0x7f15cc009310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00d580 con 0x7f15d4138840 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00d700 con 0x7f15d4138840 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00dac0 con 0x7f15d4138840 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d40abd60 con 0x7f15d4138840 2023-05-16T07:17:51.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.125+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d40ac600 con 0x7f15d4138840 2023-05-16T07:17:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.126+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d406c350 con 0x7f15d4138840 2023-05-16T07:17:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.127+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f15cc010440 con 0x7f15d4138840 2023-05-16T07:17:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.127+0000 7f15bbfff700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15c005ed90 0x7f15c0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.127+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f15cc079f10 con 0x7f15d4138840 2023-05-16T07:17:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.127+0000 7f15d3fff700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15c005ed90 0x7f15c0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.128+0000 7f15d3fff700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15c005ed90 0x7f15c0061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f15c4009130 tx=0x7f15c4015000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.131+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f15cc04b200 con 0x7f15d4138840 2023-05-16T07:17:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.333+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d40abef0 con 0x7f15d4138840 2023-05-16T07:17:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.333+0000 7f15bbfff700 1 -- 172.21.15.92:0/2827303163 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f15cc04e820 con 0x7f15d4138840 2023-05-16T07:17:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:51.338 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:48.337476+0000","modified":"2023-05-16T07:17:50.983467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1656722289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1656722289},{"type":"v1","addr":"172.21.15.92:6835","nonce":1656722289}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-05-16T07:17:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15c005ed90 msgr2=0x7f15c0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15c005ed90 0x7f15c0061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f15c4009130 tx=0x7f15c4015000).stop 2023-05-16T07:17:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 msgr2=0x7f15d414fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d414fba0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f15cc00b760 tx=0x7f15cc009310).stop 2023-05-16T07:17:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 shutdown_connections 2023-05-16T07:17:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15c005ed90 0x7f15c0061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d4138840 0x7f15d414fba0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15d4065710 0x7f15d4164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 --2- 172.21.15.92:0/2827303163 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15d40be4e0 0x7f15d40ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 >> 172.21.15.92:0/2827303163 conn(0x7f15d40c4430 msgr2=0x7f15d4136140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 shutdown_connections 2023-05-16T07:17:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.335+0000 7f15da666700 1 -- 172.21.15.92:0/2827303163 wait complete. 2023-05-16T07:17:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-05-16T07:17:51.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:17:51.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.459+0000 7f0bdd6b2700 1 Processor -- start 2023-05-16T07:17:51.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.459+0000 7f0bdd6b2700 1 -- start start 2023-05-16T07:17:51.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bdd6b2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 0x7f0bd80696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bdd6b2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd80c0660 con 0x7f0bd8154170 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bdd6b2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd806a020 con 0x7f0bd8069b90 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bdd6b2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd80bfd30 con 0x7f0bd8154550 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0bd8154170 0x7f0bd80c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57252/0 (socket says 172.21.15.92:57252) 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd6ffd700 1 -- 172.21.15.92:0/3296199659 learned_addr learned my addr 172.21.15.92:0/3296199659 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289045578 0 0) 0x7f0bd80c0660 con 0x7f0bd8154170 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc40033f0 con 0x7f0bd8154170 2023-05-16T07:17:51.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd67fc700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 0x7f0bd80696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894209096 0 0) 0x7f0bc40033f0 con 0x7f0bd8154170 2023-05-16T07:17:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 msgr2=0x7f0bd80696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 0x7f0bd80696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 >> v1:172.21.15.158:6789/0 conn(0x7f0bd8069b90 legacy=0x7f0bd80c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd80c0010 con 0x7f0bd8154170 2023-05-16T07:17:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f0bc80017f0 con 0x7f0bd8154170 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 >> v1:172.21.15.92:6789/0 conn(0x7f0bd8154170 legacy=0x7f0bd80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bc4003910 0x7f0bc4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.460+0000 7f0bd5ffb700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 0x7f0bc4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bc4004b80 0x7f0bc4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd80bfd30 con 0x7f0bc4004b80 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400bac0 con 0x7f0bc4003910 2023-05-16T07:17:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400bc80 con 0x7f0bd8154550 2023-05-16T07:17:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bc4004b80 0x7f0bc4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd6ffd700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bc4003910 0x7f0bc4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd67fc700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 0x7f0bc4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 -- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 msgr2=0x7f0bc4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8154550 0x7f0bc4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 -- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bc4003910 msgr2=0x7f0bc4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:51.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bc4003910 0x7f0bc4003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 -- 172.21.15.92:0/3296199659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40033f0 con 0x7f0bc4004b80 2023-05-16T07:17:51.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd77fe700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bc4004b80 0x7f0bc4005000 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f0bd0001760 tx=0x7f0bd0007bb0).ready entity=mon.0 client_cookie=4e825302e227deff server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0007020 con 0x7f0bc4004b80 2023-05-16T07:17:51.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0007180 con 0x7f0bc4004b80 2023-05-16T07:17:51.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bd5ffb700 1 -- 172.21.15.92:0/3296199659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0007500 con 0x7f0bc4004b80 2023-05-16T07:17:51.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.461+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/3296199659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bc4004b80 msgr2=0x7f0bc4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.462+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/3296199659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bc4004b80 0x7f0bc4005000 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f0bd0001760 tx=0x7f0bd0007bb0).stop 2023-05-16T07:17:51.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.462+0000 7f0bd77fe700 1 -- 172.21.15.92:0/3296199659 reap_dead start 2023-05-16T07:17:51.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.462+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/3296199659 shutdown_connections 2023-05-16T07:17:51.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.462+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/3296199659 >> 172.21.15.92:0/3296199659 conn(0x7f0bd80cb700 msgr2=0x7f0bd80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.462+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/3296199659 shutdown_connections 2023-05-16T07:17:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.462+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/3296199659 wait complete. 2023-05-16T07:17:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.463+0000 7f0bdd6b2700 1 Processor -- start 2023-05-16T07:17:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.463+0000 7f0bdd6b2700 1 -- start start 2023-05-16T07:17:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 0x7f0bd80c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8146b80 0x7f0bd80c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bd80ab9f0 0x7f0bd80abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8159b90 con 0x7f0bd81467a0 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd80c2f30 con 0x7f0bd80ab9f0 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd80ac360 con 0x7f0bd8146b80 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 0x7f0bd80c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 0x7f0bd80c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46516/0 (socket says 172.21.15.92:46516) 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 -- 172.21.15.92:0/2771520481 learned_addr learned my addr 172.21.15.92:0/2771520481 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:51.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd77fe700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bd80ab9f0 0x7f0bd80abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd67fc700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8146b80 0x7f0bd80c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 -- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8146b80 msgr2=0x7f0bd80c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8146b80 0x7f0bd80c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 -- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bd80ab9f0 msgr2=0x7f0bd80abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bd80ab9f0 0x7f0bd80abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 -- 172.21.15.92:0/2771520481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8164710 con 0x7f0bd81467a0 2023-05-16T07:17:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd77fe700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bd80ab9f0 0x7f0bd80abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bd6ffd700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 0x7f0bd80c1d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f0bc8007740 tx=0x7f0bc80018a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800f970 con 0x7f0bd81467a0 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc800fad0 con 0x7f0bd81467a0 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd80c2320 con 0x7f0bd81467a0 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800fe90 con 0x7f0bd81467a0 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.464+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd80c28e0 con 0x7f0bd81467a0 2023-05-16T07:17:51.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.465+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd80c3210 con 0x7f0bd81467a0 2023-05-16T07:17:51.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.466+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0bc801b030 con 0x7f0bd81467a0 2023-05-16T07:17:51.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.466+0000 7f0bceffd700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bb805ed90 0x7f0bb8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.466+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f0bc8079840 con 0x7f0bd81467a0 2023-05-16T07:17:51.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.466+0000 7f0bd67fc700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bb805ed90 0x7f0bb8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.467+0000 7f0bd67fc700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bb805ed90 0x7f0bb8061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0bc000b160 tx=0x7f0bc0008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.470+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0bc804ac40 con 0x7f0bd81467a0 2023-05-16T07:17:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.631+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd80666c0 con 0x7f0bd81467a0 2023-05-16T07:17:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.632+0000 7f0bceffd700 1 -- 172.21.15.92:0/2771520481 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17861 (secure 0 0 0) 0x7f0bc8019020 con 0x7f0bd81467a0 2023-05-16T07:17:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:51.636 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":93,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:49.338873+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:46.874223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:47.303318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:17:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bb805ed90 msgr2=0x7f0bb8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bb805ed90 0x7f0bb8061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0bc000b160 tx=0x7f0bc0008040).stop 2023-05-16T07:17:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 msgr2=0x7f0bd80c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 0x7f0bd80c1d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f0bc8007740 tx=0x7f0bc80018a0).stop 2023-05-16T07:17:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 shutdown_connections 2023-05-16T07:17:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0bb805ed90 0x7f0bb8061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0bd80ab9f0 0x7f0bd80abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd81467a0 0x7f0bd80c1d40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 --2- 172.21.15.92:0/2771520481 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0bd8146b80 0x7f0bd80c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 >> 172.21.15.92:0/2771520481 conn(0x7f0bd80cb700 msgr2=0x7f0bd80aae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 shutdown_connections 2023-05-16T07:17:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.634+0000 7f0bdd6b2700 1 -- 172.21.15.92:0/2771520481 wait complete. 2023-05-16T07:17:51.650 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:17:51.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:17:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.758+0000 7fc1314c6700 1 Processor -- start 2023-05-16T07:17:51.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.758+0000 7fc1314c6700 1 -- start start 2023-05-16T07:17:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.758+0000 7fc1314c6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc12c154550 0x7fc12c0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.758+0000 7fc1314c6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c0c0660 con 0x7fc12c069b90 2023-05-16T07:17:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.758+0000 7fc1314c6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c069fc0 con 0x7fc12c154170 2023-05-16T07:17:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.758+0000 7fc1314c6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c0bfd30 con 0x7fc12c154550 2023-05-16T07:17:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc12c154550 0x7fc12c0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc12c069b90 0x7fc12c0c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57274/0 (socket says 172.21.15.92:57274) 2023-05-16T07:17:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12b7fe700 1 -- 172.21.15.92:0/309812787 learned_addr learned my addr 172.21.15.92:0/309812787 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12a7fc700 1 -- 172.21.15.92:0/309812787 >> v1:172.21.15.158:6789/0 conn(0x7fc12c154170 legacy=0x7fc12c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12a7fc700 1 -- 172.21.15.92:0/309812787 >> v1:172.21.15.92:6789/0 conn(0x7fc12c069b90 legacy=0x7fc12c0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12a7fc700 1 -- 172.21.15.92:0/309812787 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0c0010 con 0x7fc12c154550 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc129ffb700 1 -- 172.21.15.92:0/309812787 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604315202 0 0) 0x7fc12c0c0660 con 0x7fc12c069b90 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc12a7fc700 1 --2- 172.21.15.92:0/309812787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154550 0x7fc12c0696a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fc114009760 tx=0x7fc114009060).ready entity=mon.0 client_cookie=8b8e6234336b59b9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc129ffb700 1 -- 172.21.15.92:0/309812787 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114009120 con 0x7fc12c154550 2023-05-16T07:17:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc129ffb700 1 -- 172.21.15.92:0/309812787 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400b580 con 0x7fc12c154550 2023-05-16T07:17:51.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.759+0000 7fc129ffb700 1 -- 172.21.15.92:0/309812787 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400b900 con 0x7fc12c154550 2023-05-16T07:17:51.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 -- 172.21.15.92:0/309812787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154550 msgr2=0x7fc12c0696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 --2- 172.21.15.92:0/309812787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154550 0x7fc12c0696a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fc114009760 tx=0x7fc114009060).stop 2023-05-16T07:17:51.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 -- 172.21.15.92:0/309812787 shutdown_connections 2023-05-16T07:17:51.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 --2- 172.21.15.92:0/309812787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154550 0x7fc12c0696a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 -- 172.21.15.92:0/309812787 >> 172.21.15.92:0/309812787 conn(0x7fc12c0cb700 msgr2=0x7fc12c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:51.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 -- 172.21.15.92:0/309812787 shutdown_connections 2023-05-16T07:17:51.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.760+0000 7fc1314c6700 1 -- 172.21.15.92:0/309812787 wait complete. 2023-05-16T07:17:51.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.761+0000 7fc1314c6700 1 Processor -- start 2023-05-16T07:17:51.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.761+0000 7fc1314c6700 1 -- start start 2023-05-16T07:17:51.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc12c069b90 0x7fc12c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 0x7fc12c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc12c154550 0x7fc12c0c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c1564c0 con 0x7fc12c154170 2023-05-16T07:17:51.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c155340 con 0x7fc12c069b90 2023-05-16T07:17:51.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c155620 con 0x7fc12c154550 2023-05-16T07:17:51.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 0x7fc12c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 0x7fc12c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46532/0 (socket says 172.21.15.92:46532) 2023-05-16T07:17:51.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 -- 172.21.15.92:0/4264195749 learned_addr learned my addr 172.21.15.92:0/4264195749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:51.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12affd700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc12c069b90 0x7fc12c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 -- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc12c154550 msgr2=0x7fc12c0c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc12c154550 0x7fc12c0c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:51.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 -- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc12c069b90 msgr2=0x7fc12c14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc12c069b90 0x7fc12c14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 -- 172.21.15.92:0/4264195749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140092a0 con 0x7fc12c154170 2023-05-16T07:17:51.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12affd700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc12c069b90 0x7fc12c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:51.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc12a7fc700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 0x7fc12c154960 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fc12c083060 tx=0x7fc114017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114006970 con 0x7fc12c154170 2023-05-16T07:17:51.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400bcc0 con 0x7fc12c154170 2023-05-16T07:17:51.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c155900 con 0x7fc12c154170 2023-05-16T07:17:51.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11401b440 con 0x7fc12c154170 2023-05-16T07:17:51.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.762+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c155be0 con 0x7fc12c154170 2023-05-16T07:17:51.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.763+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c083f70 con 0x7fc12c154170 2023-05-16T07:17:51.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.763+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc11400f070 con 0x7fc12c154170 2023-05-16T07:17:51.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.764+0000 7fc122ffd700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc10c05ed90 0x7fc10c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:51.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.764+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fc11407b460 con 0x7fc12c154170 2023-05-16T07:17:51.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.764+0000 7fc12affd700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc10c05ed90 0x7fc10c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:51.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.765+0000 7fc12affd700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc10c05ed90 0x7fc10c061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc11c007fd0 tx=0x7fc11c009040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:51.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.768+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc11404c7d0 con 0x7fc12c154170 2023-05-16T07:17:51.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.927+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc12c0666c0 con 0x7fc10c05ed90 2023-05-16T07:17:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.938+0000 7fc122ffd700 1 -- 172.21.15.92:0/4264195749 <== mgr.6019 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361285 (secure 0 0 0) 0x7fc12c0666c0 con 0x7fc10c05ed90 2023-05-16T07:17:51.947 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:51.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc10c05ed90 msgr2=0x7fc10c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc10c05ed90 0x7fc10c061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc11c007fd0 tx=0x7fc11c009040).stop 2023-05-16T07:17:51.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 msgr2=0x7fc12c154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:51.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 0x7fc12c154960 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fc12c083060 tx=0x7fc114017000).stop 2023-05-16T07:17:51.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 shutdown_connections 2023-05-16T07:17:51.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc10c05ed90 0x7fc10c061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc12c154550 0x7fc12c0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:51.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc12c069b90 0x7fc12c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 --2- 172.21.15.92:0/4264195749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c154170 0x7fc12c154960 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:51.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.941+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 >> 172.21.15.92:0/4264195749 conn(0x7fc12c0cb700 msgr2=0x7fc12c14da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:51.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.942+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 shutdown_connections 2023-05-16T07:17:51.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:51.942+0000 7fc1314c6700 1 -- 172.21.15.92:0/4264195749 wait complete. 2023-05-16T07:17:51.951 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:17:51.957 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-05-16T07:17:51.783931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":148084,"kb_used_data":8692,"kb_used_omap":0,"kb_used_meta":139328,"kb_avail":754826636,"statfs":{"total":773094113280,"available":772942475264,"internally_reserved":0,"allocated":8900608,"data_stored":4557980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":142671872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10436,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-61,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.601392"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369831+0000","last_change":"2023-05-16T07:17:48.369831+0000","last_active":"2023-05-16T07:17:48.369831+0000","last_peered":"2023-05-16T07:17:48.369831+0000","last_clean":"2023-05-16T07:17:48.369831+0000","last_became_active":"2023-05-16T07:17:48.369377+0000","last_became_peered":"2023-05-16T07:17:48.369377+0000","last_unstale":"2023-05-16T07:17:48.369831+0000","last_undegraded":"2023-05-16T07:17:48.369831+0000","last_fullsized":"2023-05-16T07:17:48.369831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"92'3","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362034+0000","last_change":"2023-05-16T07:17:47.989734+0000","last_active":"2023-05-16T07:17:48.362034+0000","last_peered":"2023-05-16T07:17:48.362034+0000","last_clean":"2023-05-16T07:17:48.362034+0000","last_became_active":"2023-05-16T07:17:47.988957+0000","last_became_peered":"2023-05-16T07:17:47.988957+0000","last_unstale":"2023-05-16T07:17:48.362034+0000","last_undegraded":"2023-05-16T07:17:48.362034+0000","last_fullsized":"2023-05-16T07:17:48.362034+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356259+0000","last_change":"2023-05-16T07:17:48.356259+0000","last_active":"2023-05-16T07:17:48.356259+0000","last_peered":"2023-05-16T07:17:48.356259+0000","last_clean":"2023-05-16T07:17:48.356259+0000","last_became_active":"2023-05-16T07:17:48.355996+0000","last_became_peered":"2023-05-16T07:17:48.355996+0000","last_unstale":"2023-05-16T07:17:48.356259+0000","last_undegraded":"2023-05-16T07:17:48.356259+0000","last_fullsized":"2023-05-16T07:17:48.356259+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999272+0000","last_change":"2023-05-16T07:17:47.995555+0000","last_active":"2023-05-16T07:17:47.999272+0000","last_peered":"2023-05-16T07:17:47.999272+0000","last_clean":"2023-05-16T07:17:47.999272+0000","last_became_active":"2023-05-16T07:17:47.994097+0000","last_became_peered":"2023-05-16T07:17:47.994097+0000","last_unstale":"2023-05-16T07:17:47.999272+0000","last_undegraded":"2023-05-16T07:17:47.999272+0000","last_fullsized":"2023-05-16T07:17:47.999272+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356980+0000","last_change":"2023-05-16T07:17:48.356980+0000","last_active":"2023-05-16T07:17:48.356980+0000","last_peered":"2023-05-16T07:17:48.356980+0000","last_clean":"2023-05-16T07:17:48.356980+0000","last_became_active":"2023-05-16T07:17:48.356667+0000","last_became_peered":"2023-05-16T07:17:48.356667+0000","last_unstale":"2023-05-16T07:17:48.356980+0000","last_undegraded":"2023-05-16T07:17:48.356980+0000","last_fullsized":"2023-05-16T07:17:48.356980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999080+0000","last_change":"2023-05-16T07:17:47.994556+0000","last_active":"2023-05-16T07:17:47.999080+0000","last_peered":"2023-05-16T07:17:47.999080+0000","last_clean":"2023-05-16T07:17:47.999080+0000","last_became_active":"2023-05-16T07:17:47.994290+0000","last_became_peered":"2023-05-16T07:17:47.994290+0000","last_unstale":"2023-05-16T07:17:47.999080+0000","last_undegraded":"2023-05-16T07:17:47.999080+0000","last_fullsized":"2023-05-16T07:17:47.999080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369137+0000","last_change":"2023-05-16T07:17:48.369137+0000","last_active":"2023-05-16T07:17:48.369137+0000","last_peered":"2023-05-16T07:17:48.369137+0000","last_clean":"2023-05-16T07:17:48.369137+0000","last_became_active":"2023-05-16T07:17:48.365347+0000","last_became_peered":"2023-05-16T07:17:48.365347+0000","last_unstale":"2023-05-16T07:17:48.369137+0000","last_undegraded":"2023-05-16T07:17:48.369137+0000","last_fullsized":"2023-05-16T07:17:48.369137+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363169+0000","last_change":"2023-05-16T07:17:47.997705+0000","last_active":"2023-05-16T07:17:48.363169+0000","last_peered":"2023-05-16T07:17:48.363169+0000","last_clean":"2023-05-16T07:17:48.363169+0000","last_became_active":"2023-05-16T07:17:47.997370+0000","last_became_peered":"2023-05-16T07:17:47.997370+0000","last_unstale":"2023-05-16T07:17:48.363169+0000","last_undegraded":"2023-05-16T07:17:48.363169+0000","last_fullsized":"2023-05-16T07:17:48.363169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361521+0000","last_change":"2023-05-16T07:17:48.361521+0000","last_active":"2023-05-16T07:17:48.361521+0000","last_peered":"2023-05-16T07:17:48.361521+0000","last_clean":"2023-05-16T07:17:48.361521+0000","last_became_active":"2023-05-16T07:17:48.361246+0000","last_became_peered":"2023-05-16T07:17:48.361246+0000","last_unstale":"2023-05-16T07:17:48.361521+0000","last_undegraded":"2023-05-16T07:17:48.361521+0000","last_fullsized":"2023-05-16T07:17:48.361521+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3b","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.354191+0000","last_change":"2023-05-16T07:17:47.991798+0000","last_active":"2023-05-16T07:17:48.354191+0000","last_peered":"2023-05-16T07:17:48.354191+0000","last_clean":"2023-05-16T07:17:48.354191+0000","last_became_active":"2023-05-16T07:17:47.990666+0000","last_became_peered":"2023-05-16T07:17:47.990666+0000","last_unstale":"2023-05-16T07:17:48.354191+0000","last_undegraded":"2023-05-16T07:17:48.354191+0000","last_fullsized":"2023-05-16T07:17:48.354191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.374222+0000","last_change":"2023-05-16T07:17:48.374222+0000","last_active":"2023-05-16T07:17:48.374222+0000","last_peered":"2023-05-16T07:17:48.374222+0000","last_clean":"2023-05-16T07:17:48.374222+0000","last_became_active":"2023-05-16T07:17:48.373923+0000","last_became_peered":"2023-05-16T07:17:48.373923+0000","last_unstale":"2023-05-16T07:17:48.374222+0000","last_undegraded":"2023-05-16T07:17:48.374222+0000","last_fullsized":"2023-05-16T07:17:48.374222+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.342394+0000","last_change":"2023-05-16T07:17:47.996336+0000","last_active":"2023-05-16T07:17:48.342394+0000","last_peered":"2023-05-16T07:17:48.342394+0000","last_clean":"2023-05-16T07:17:48.342394+0000","last_became_active":"2023-05-16T07:17:47.994144+0000","last_became_peered":"2023-05-16T07:17:47.994144+0000","last_unstale":"2023-05-16T07:17:48.342394+0000","last_undegraded":"2023-05-16T07:17:48.342394+0000","last_fullsized":"2023-05-16T07:17:48.342394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366242+0000","last_change":"2023-05-16T07:17:48.366242+0000","last_active":"2023-05-16T07:17:48.366242+0000","last_peered":"2023-05-16T07:17:48.366242+0000","last_clean":"2023-05-16T07:17:48.366242+0000","last_became_active":"2023-05-16T07:17:48.365793+0000","last_became_peered":"2023-05-16T07:17:48.365793+0000","last_unstale":"2023-05-16T07:17:48.366242+0000","last_undegraded":"2023-05-16T07:17:48.366242+0000","last_fullsized":"2023-05-16T07:17:48.366242+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004103+0000","last_change":"2023-05-16T07:17:48.002774+0000","last_active":"2023-05-16T07:17:48.004103+0000","last_peered":"2023-05-16T07:17:48.004103+0000","last_clean":"2023-05-16T07:17:48.004103+0000","last_became_active":"2023-05-16T07:17:48.002495+0000","last_became_peered":"2023-05-16T07:17:48.002495+0000","last_unstale":"2023-05-16T07:17:48.004103+0000","last_undegraded":"2023-05-16T07:17:48.004103+0000","last_fullsized":"2023-05-16T07:17:48.004103+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.353979+0000","last_change":"2023-05-16T07:17:48.353979+0000","last_active":"2023-05-16T07:17:48.353979+0000","last_peered":"2023-05-16T07:17:48.353979+0000","last_clean":"2023-05-16T07:17:48.353979+0000","last_became_active":"2023-05-16T07:17:48.353702+0000","last_became_peered":"2023-05-16T07:17:48.353702+0000","last_unstale":"2023-05-16T07:17:48.353979+0000","last_undegraded":"2023-05-16T07:17:48.353979+0000","last_fullsized":"2023-05-16T07:17:48.353979+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.001977+0000","last_change":"2023-05-16T07:17:47.999945+0000","last_active":"2023-05-16T07:17:48.001977+0000","last_peered":"2023-05-16T07:17:48.001977+0000","last_clean":"2023-05-16T07:17:48.001977+0000","last_became_active":"2023-05-16T07:17:47.999519+0000","last_became_peered":"2023-05-16T07:17:47.999519+0000","last_unstale":"2023-05-16T07:17:48.001977+0000","last_undegraded":"2023-05-16T07:17:48.001977+0000","last_fullsized":"2023-05-16T07:17:48.001977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360887+0000","last_change":"2023-05-16T07:17:48.360887+0000","last_active":"2023-05-16T07:17:48.360887+0000","last_peered":"2023-05-16T07:17:48.360887+0000","last_clean":"2023-05-16T07:17:48.360887+0000","last_became_active":"2023-05-16T07:17:48.360602+0000","last_became_peered":"2023-05-16T07:17:48.360602+0000","last_unstale":"2023-05-16T07:17:48.360887+0000","last_undegraded":"2023-05-16T07:17:48.360887+0000","last_fullsized":"2023-05-16T07:17:48.360887+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992700+0000","last_change":"2023-05-16T07:17:47.986607+0000","last_active":"2023-05-16T07:17:47.992700+0000","last_peered":"2023-05-16T07:17:47.992700+0000","last_clean":"2023-05-16T07:17:47.992700+0000","last_became_active":"2023-05-16T07:17:47.986302+0000","last_became_peered":"2023-05-16T07:17:47.986302+0000","last_unstale":"2023-05-16T07:17:47.992700+0000","last_undegraded":"2023-05-16T07:17:47.992700+0000","last_fullsized":"2023-05-16T07:17:47.992700+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370115+0000","last_change":"2023-05-16T07:17:48.370115+0000","last_active":"2023-05-16T07:17:48.370115+0000","last_peered":"2023-05-16T07:17:48.370115+0000","last_clean":"2023-05-16T07:17:48.370115+0000","last_became_active":"2023-05-16T07:17:48.366731+0000","last_became_peered":"2023-05-16T07:17:48.366731+0000","last_unstale":"2023-05-16T07:17:48.370115+0000","last_undegraded":"2023-05-16T07:17:48.370115+0000","last_fullsized":"2023-05-16T07:17:48.370115+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999757+0000","last_change":"2023-05-16T07:17:47.991747+0000","last_active":"2023-05-16T07:17:47.999757+0000","last_peered":"2023-05-16T07:17:47.999757+0000","last_clean":"2023-05-16T07:17:47.999757+0000","last_became_active":"2023-05-16T07:17:47.991461+0000","last_became_peered":"2023-05-16T07:17:47.991461+0000","last_unstale":"2023-05-16T07:17:47.999757+0000","last_undegraded":"2023-05-16T07:17:47.999757+0000","last_fullsized":"2023-05-16T07:17:47.999757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.357059+0000","last_change":"2023-05-16T07:17:48.357059+0000","last_active":"2023-05-16T07:17:48.357059+0000","last_peered":"2023-05-16T07:17:48.357059+0000","last_clean":"2023-05-16T07:17:48.357059+0000","last_became_active":"2023-05-16T07:17:48.356598+0000","last_became_peered":"2023-05-16T07:17:48.356598+0000","last_unstale":"2023-05-16T07:17:48.357059+0000","last_undegraded":"2023-05-16T07:17:48.357059+0000","last_fullsized":"2023-05-16T07:17:48.357059+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000493+0000","last_change":"2023-05-16T07:17:47.994026+0000","last_active":"2023-05-16T07:17:48.000493+0000","last_peered":"2023-05-16T07:17:48.000493+0000","last_clean":"2023-05-16T07:17:48.000493+0000","last_became_active":"2023-05-16T07:17:47.993403+0000","last_became_peered":"2023-05-16T07:17:47.993403+0000","last_unstale":"2023-05-16T07:17:48.000493+0000","last_undegraded":"2023-05-16T07:17:48.000493+0000","last_fullsized":"2023-05-16T07:17:48.000493+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369132+0000","last_change":"2023-05-16T07:17:48.369132+0000","last_active":"2023-05-16T07:17:48.369132+0000","last_peered":"2023-05-16T07:17:48.369132+0000","last_clean":"2023-05-16T07:17:48.369132+0000","last_became_active":"2023-05-16T07:17:48.365980+0000","last_became_peered":"2023-05-16T07:17:48.365980+0000","last_unstale":"2023-05-16T07:17:48.369132+0000","last_undegraded":"2023-05-16T07:17:48.369132+0000","last_fullsized":"2023-05-16T07:17:48.369132+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.34","version":"92'4","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.374458+0000","last_change":"2023-05-16T07:17:48.000143+0000","last_active":"2023-05-16T07:17:48.374458+0000","last_peered":"2023-05-16T07:17:48.374458+0000","last_clean":"2023-05-16T07:17:48.374458+0000","last_became_active":"2023-05-16T07:17:47.999123+0000","last_became_peered":"2023-05-16T07:17:47.999123+0000","last_unstale":"2023-05-16T07:17:48.374458+0000","last_undegraded":"2023-05-16T07:17:48.374458+0000","last_fullsized":"2023-05-16T07:17:48.374458+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.373339+0000","last_change":"2023-05-16T07:17:48.373339+0000","last_active":"2023-05-16T07:17:48.373339+0000","last_peered":"2023-05-16T07:17:48.373339+0000","last_clean":"2023-05-16T07:17:48.373339+0000","last_became_active":"2023-05-16T07:17:48.372991+0000","last_became_peered":"2023-05-16T07:17:48.372991+0000","last_unstale":"2023-05-16T07:17:48.373339+0000","last_undegraded":"2023-05-16T07:17:48.373339+0000","last_fullsized":"2023-05-16T07:17:48.373339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.349344+0000","last_change":"2023-05-16T07:17:47.990064+0000","last_active":"2023-05-16T07:17:48.349344+0000","last_peered":"2023-05-16T07:17:48.349344+0000","last_clean":"2023-05-16T07:17:48.349344+0000","last_became_active":"2023-05-16T07:17:47.989173+0000","last_became_peered":"2023-05-16T07:17:47.989173+0000","last_unstale":"2023-05-16T07:17:48.349344+0000","last_undegraded":"2023-05-16T07:17:48.349344+0000","last_fullsized":"2023-05-16T07:17:48.349344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.374587+0000","last_change":"2023-05-16T07:17:48.374587+0000","last_active":"2023-05-16T07:17:48.374587+0000","last_peered":"2023-05-16T07:17:48.374587+0000","last_clean":"2023-05-16T07:17:48.374587+0000","last_became_active":"2023-05-16T07:17:48.374320+0000","last_became_peered":"2023-05-16T07:17:48.374320+0000","last_unstale":"2023-05-16T07:17:48.374587+0000","last_undegraded":"2023-05-16T07:17:48.374587+0000","last_fullsized":"2023-05-16T07:17:48.374587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004935+0000","last_change":"2023-05-16T07:17:48.001873+0000","last_active":"2023-05-16T07:17:48.004935+0000","last_peered":"2023-05-16T07:17:48.004935+0000","last_clean":"2023-05-16T07:17:48.004935+0000","last_became_active":"2023-05-16T07:17:48.001576+0000","last_became_peered":"2023-05-16T07:17:48.001576+0000","last_unstale":"2023-05-16T07:17:48.004935+0000","last_undegraded":"2023-05-16T07:17:48.004935+0000","last_fullsized":"2023-05-16T07:17:48.004935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359447+0000","last_change":"2023-05-16T07:17:48.359447+0000","last_active":"2023-05-16T07:17:48.359447+0000","last_peered":"2023-05-16T07:17:48.359447+0000","last_clean":"2023-05-16T07:17:48.359447+0000","last_became_active":"2023-05-16T07:17:48.359170+0000","last_became_peered":"2023-05-16T07:17:48.359170+0000","last_unstale":"2023-05-16T07:17:48.359447+0000","last_undegraded":"2023-05-16T07:17:48.359447+0000","last_fullsized":"2023-05-16T07:17:48.359447+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.997113+0000","last_change":"2023-05-16T07:17:47.995682+0000","last_active":"2023-05-16T07:17:47.997113+0000","last_peered":"2023-05-16T07:17:47.997113+0000","last_clean":"2023-05-16T07:17:47.997113+0000","last_became_active":"2023-05-16T07:17:47.995394+0000","last_became_peered":"2023-05-16T07:17:47.995394+0000","last_unstale":"2023-05-16T07:17:47.997113+0000","last_undegraded":"2023-05-16T07:17:47.997113+0000","last_fullsized":"2023-05-16T07:17:47.997113+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.357536+0000","last_change":"2023-05-16T07:17:48.357536+0000","last_active":"2023-05-16T07:17:48.357536+0000","last_peered":"2023-05-16T07:17:48.357536+0000","last_clean":"2023-05-16T07:17:48.357536+0000","last_became_active":"2023-05-16T07:17:48.357300+0000","last_became_peered":"2023-05-16T07:17:48.357300+0000","last_unstale":"2023-05-16T07:17:48.357536+0000","last_undegraded":"2023-05-16T07:17:48.357536+0000","last_fullsized":"2023-05-16T07:17:48.357536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"92'4","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360547+0000","last_change":"2023-05-16T07:17:47.994055+0000","last_active":"2023-05-16T07:17:48.360547+0000","last_peered":"2023-05-16T07:17:48.360547+0000","last_clean":"2023-05-16T07:17:48.360547+0000","last_became_active":"2023-05-16T07:17:47.993775+0000","last_became_peered":"2023-05-16T07:17:47.993775+0000","last_unstale":"2023-05-16T07:17:48.360547+0000","last_undegraded":"2023-05-16T07:17:48.360547+0000","last_fullsized":"2023-05-16T07:17:48.360547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370894+0000","last_change":"2023-05-16T07:17:48.370894+0000","last_active":"2023-05-16T07:17:48.370894+0000","last_peered":"2023-05-16T07:17:48.370894+0000","last_clean":"2023-05-16T07:17:48.370894+0000","last_became_active":"2023-05-16T07:17:48.370659+0000","last_became_peered":"2023-05-16T07:17:48.370659+0000","last_unstale":"2023-05-16T07:17:48.370894+0000","last_undegraded":"2023-05-16T07:17:48.370894+0000","last_fullsized":"2023-05-16T07:17:48.370894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992659+0000","last_change":"2023-05-16T07:17:47.986437+0000","last_active":"2023-05-16T07:17:47.992659+0000","last_peered":"2023-05-16T07:17:47.992659+0000","last_clean":"2023-05-16T07:17:47.992659+0000","last_became_active":"2023-05-16T07:17:47.985976+0000","last_became_peered":"2023-05-16T07:17:47.985976+0000","last_unstale":"2023-05-16T07:17:47.992659+0000","last_undegraded":"2023-05-16T07:17:47.992659+0000","last_fullsized":"2023-05-16T07:17:47.992659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361142+0000","last_change":"2023-05-16T07:17:48.361142+0000","last_active":"2023-05-16T07:17:48.361142+0000","last_peered":"2023-05-16T07:17:48.361142+0000","last_clean":"2023-05-16T07:17:48.361142+0000","last_became_active":"2023-05-16T07:17:48.360856+0000","last_became_peered":"2023-05-16T07:17:48.360856+0000","last_unstale":"2023-05-16T07:17:48.361142+0000","last_undegraded":"2023-05-16T07:17:48.361142+0000","last_fullsized":"2023-05-16T07:17:48.361142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003301+0000","last_change":"2023-05-16T07:17:47.998342+0000","last_active":"2023-05-16T07:17:48.003301+0000","last_peered":"2023-05-16T07:17:48.003301+0000","last_clean":"2023-05-16T07:17:48.003301+0000","last_became_active":"2023-05-16T07:17:47.997982+0000","last_became_peered":"2023-05-16T07:17:47.997982+0000","last_unstale":"2023-05-16T07:17:48.003301+0000","last_undegraded":"2023-05-16T07:17:48.003301+0000","last_fullsized":"2023-05-16T07:17:48.003301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365772+0000","last_change":"2023-05-16T07:17:48.365772+0000","last_active":"2023-05-16T07:17:48.365772+0000","last_peered":"2023-05-16T07:17:48.365772+0000","last_clean":"2023-05-16T07:17:48.365772+0000","last_became_active":"2023-05-16T07:17:48.365529+0000","last_became_peered":"2023-05-16T07:17:48.365529+0000","last_unstale":"2023-05-16T07:17:48.365772+0000","last_undegraded":"2023-05-16T07:17:48.365772+0000","last_fullsized":"2023-05-16T07:17:48.365772+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2d","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.342923+0000","last_change":"2023-05-16T07:17:47.999853+0000","last_active":"2023-05-16T07:17:48.342923+0000","last_peered":"2023-05-16T07:17:48.342923+0000","last_clean":"2023-05-16T07:17:48.342923+0000","last_became_active":"2023-05-16T07:17:47.999547+0000","last_became_peered":"2023-05-16T07:17:47.999547+0000","last_unstale":"2023-05-16T07:17:48.342923+0000","last_undegraded":"2023-05-16T07:17:48.342923+0000","last_fullsized":"2023-05-16T07:17:48.342923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364708+0000","last_change":"2023-05-16T07:17:48.364708+0000","last_active":"2023-05-16T07:17:48.364708+0000","last_peered":"2023-05-16T07:17:48.364708+0000","last_clean":"2023-05-16T07:17:48.364708+0000","last_became_active":"2023-05-16T07:17:48.364421+0000","last_became_peered":"2023-05-16T07:17:48.364421+0000","last_unstale":"2023-05-16T07:17:48.364708+0000","last_undegraded":"2023-05-16T07:17:48.364708+0000","last_fullsized":"2023-05-16T07:17:48.364708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004983+0000","last_change":"2023-05-16T07:17:48.000706+0000","last_active":"2023-05-16T07:17:48.004983+0000","last_peered":"2023-05-16T07:17:48.004983+0000","last_clean":"2023-05-16T07:17:48.004983+0000","last_became_active":"2023-05-16T07:17:48.000496+0000","last_became_peered":"2023-05-16T07:17:48.000496+0000","last_unstale":"2023-05-16T07:17:48.004983+0000","last_undegraded":"2023-05-16T07:17:48.004983+0000","last_fullsized":"2023-05-16T07:17:48.004983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365532+0000","last_change":"2023-05-16T07:17:48.365532+0000","last_active":"2023-05-16T07:17:48.365532+0000","last_peered":"2023-05-16T07:17:48.365532+0000","last_clean":"2023-05-16T07:17:48.365532+0000","last_became_active":"2023-05-16T07:17:48.365217+0000","last_became_peered":"2023-05-16T07:17:48.365217+0000","last_unstale":"2023-05-16T07:17:48.365532+0000","last_undegraded":"2023-05-16T07:17:48.365532+0000","last_fullsized":"2023-05-16T07:17:48.365532+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.350514+0000","last_change":"2023-05-16T07:17:47.995065+0000","last_active":"2023-05-16T07:17:48.350514+0000","last_peered":"2023-05-16T07:17:48.350514+0000","last_clean":"2023-05-16T07:17:48.350514+0000","last_became_active":"2023-05-16T07:17:47.994756+0000","last_became_peered":"2023-05-16T07:17:47.994756+0000","last_unstale":"2023-05-16T07:17:48.350514+0000","last_undegraded":"2023-05-16T07:17:48.350514+0000","last_fullsized":"2023-05-16T07:17:48.350514+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360987+0000","last_change":"2023-05-16T07:17:48.360987+0000","last_active":"2023-05-16T07:17:48.360987+0000","last_peered":"2023-05-16T07:17:48.360987+0000","last_clean":"2023-05-16T07:17:48.360987+0000","last_became_active":"2023-05-16T07:17:48.360819+0000","last_became_peered":"2023-05-16T07:17:48.360819+0000","last_unstale":"2023-05-16T07:17:48.360987+0000","last_undegraded":"2023-05-16T07:17:48.360987+0000","last_fullsized":"2023-05-16T07:17:48.360987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000719+0000","last_change":"2023-05-16T07:17:47.995552+0000","last_active":"2023-05-16T07:17:48.000719+0000","last_peered":"2023-05-16T07:17:48.000719+0000","last_clean":"2023-05-16T07:17:48.000719+0000","last_became_active":"2023-05-16T07:17:47.995251+0000","last_became_peered":"2023-05-16T07:17:47.995251+0000","last_unstale":"2023-05-16T07:17:48.000719+0000","last_undegraded":"2023-05-16T07:17:48.000719+0000","last_fullsized":"2023-05-16T07:17:48.000719+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367914+0000","last_change":"2023-05-16T07:17:48.367914+0000","last_active":"2023-05-16T07:17:48.367914+0000","last_peered":"2023-05-16T07:17:48.367914+0000","last_clean":"2023-05-16T07:17:48.367914+0000","last_became_active":"2023-05-16T07:17:48.367714+0000","last_became_peered":"2023-05-16T07:17:48.367714+0000","last_unstale":"2023-05-16T07:17:48.367914+0000","last_undegraded":"2023-05-16T07:17:48.367914+0000","last_fullsized":"2023-05-16T07:17:48.367914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003169+0000","last_change":"2023-05-16T07:17:47.998143+0000","last_active":"2023-05-16T07:17:48.003169+0000","last_peered":"2023-05-16T07:17:48.003169+0000","last_clean":"2023-05-16T07:17:48.003169+0000","last_became_active":"2023-05-16T07:17:47.997849+0000","last_became_peered":"2023-05-16T07:17:47.997849+0000","last_unstale":"2023-05-16T07:17:48.003169+0000","last_undegraded":"2023-05-16T07:17:48.003169+0000","last_fullsized":"2023-05-16T07:17:48.003169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365356+0000","last_change":"2023-05-16T07:17:48.365356+0000","last_active":"2023-05-16T07:17:48.365356+0000","last_peered":"2023-05-16T07:17:48.365356+0000","last_clean":"2023-05-16T07:17:48.365356+0000","last_became_active":"2023-05-16T07:17:48.365075+0000","last_became_peered":"2023-05-16T07:17:48.365075+0000","last_unstale":"2023-05-16T07:17:48.365356+0000","last_undegraded":"2023-05-16T07:17:48.365356+0000","last_fullsized":"2023-05-16T07:17:48.365356+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002985+0000","last_change":"2023-05-16T07:17:47.997193+0000","last_active":"2023-05-16T07:17:48.002985+0000","last_peered":"2023-05-16T07:17:48.002985+0000","last_clean":"2023-05-16T07:17:48.002985+0000","last_became_active":"2023-05-16T07:17:47.996908+0000","last_became_peered":"2023-05-16T07:17:47.996908+0000","last_unstale":"2023-05-16T07:17:48.002985+0000","last_undegraded":"2023-05-16T07:17:48.002985+0000","last_fullsized":"2023-05-16T07:17:48.002985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.355450+0000","last_change":"2023-05-16T07:17:48.355450+0000","last_active":"2023-05-16T07:17:48.355450+0000","last_peered":"2023-05-16T07:17:48.355450+0000","last_clean":"2023-05-16T07:17:48.355450+0000","last_became_active":"2023-05-16T07:17:48.355254+0000","last_became_peered":"2023-05-16T07:17:48.355254+0000","last_unstale":"2023-05-16T07:17:48.355450+0000","last_undegraded":"2023-05-16T07:17:48.355450+0000","last_fullsized":"2023-05-16T07:17:48.355450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000797+0000","last_change":"2023-05-16T07:17:47.994674+0000","last_active":"2023-05-16T07:17:48.000797+0000","last_peered":"2023-05-16T07:17:48.000797+0000","last_clean":"2023-05-16T07:17:48.000797+0000","last_became_active":"2023-05-16T07:17:47.992629+0000","last_became_peered":"2023-05-16T07:17:47.992629+0000","last_unstale":"2023-05-16T07:17:48.000797+0000","last_undegraded":"2023-05-16T07:17:48.000797+0000","last_fullsized":"2023-05-16T07:17:48.000797+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.358186+0000","last_change":"2023-05-16T07:17:48.358186+0000","last_active":"2023-05-16T07:17:48.358186+0000","last_peered":"2023-05-16T07:17:48.358186+0000","last_clean":"2023-05-16T07:17:48.358186+0000","last_became_active":"2023-05-16T07:17:48.357966+0000","last_became_peered":"2023-05-16T07:17:48.357966+0000","last_unstale":"2023-05-16T07:17:48.358186+0000","last_undegraded":"2023-05-16T07:17:48.358186+0000","last_fullsized":"2023-05-16T07:17:48.358186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003416+0000","last_change":"2023-05-16T07:17:48.002061+0000","last_active":"2023-05-16T07:17:48.003416+0000","last_peered":"2023-05-16T07:17:48.003416+0000","last_clean":"2023-05-16T07:17:48.003416+0000","last_became_active":"2023-05-16T07:17:48.001737+0000","last_became_peered":"2023-05-16T07:17:48.001737+0000","last_unstale":"2023-05-16T07:17:48.003416+0000","last_undegraded":"2023-05-16T07:17:48.003416+0000","last_fullsized":"2023-05-16T07:17:48.003416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366958+0000","last_change":"2023-05-16T07:17:48.366958+0000","last_active":"2023-05-16T07:17:48.366958+0000","last_peered":"2023-05-16T07:17:48.366958+0000","last_clean":"2023-05-16T07:17:48.366958+0000","last_became_active":"2023-05-16T07:17:48.366688+0000","last_became_peered":"2023-05-16T07:17:48.366688+0000","last_unstale":"2023-05-16T07:17:48.366958+0000","last_undegraded":"2023-05-16T07:17:48.366958+0000","last_fullsized":"2023-05-16T07:17:48.366958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.996841+0000","last_change":"2023-05-16T07:17:47.993000+0000","last_active":"2023-05-16T07:17:47.996841+0000","last_peered":"2023-05-16T07:17:47.996841+0000","last_clean":"2023-05-16T07:17:47.996841+0000","last_became_active":"2023-05-16T07:17:47.992750+0000","last_became_peered":"2023-05-16T07:17:47.992750+0000","last_unstale":"2023-05-16T07:17:47.996841+0000","last_undegraded":"2023-05-16T07:17:47.996841+0000","last_fullsized":"2023-05-16T07:17:47.996841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366152+0000","last_change":"2023-05-16T07:17:48.366152+0000","last_active":"2023-05-16T07:17:48.366152+0000","last_peered":"2023-05-16T07:17:48.366152+0000","last_clean":"2023-05-16T07:17:48.366152+0000","last_became_active":"2023-05-16T07:17:48.365878+0000","last_became_peered":"2023-05-16T07:17:48.365878+0000","last_unstale":"2023-05-16T07:17:48.366152+0000","last_undegraded":"2023-05-16T07:17:48.366152+0000","last_fullsized":"2023-05-16T07:17:48.366152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992800+0000","last_change":"2023-05-16T07:17:47.990784+0000","last_active":"2023-05-16T07:17:47.992800+0000","last_peered":"2023-05-16T07:17:47.992800+0000","last_clean":"2023-05-16T07:17:47.992800+0000","last_became_active":"2023-05-16T07:17:47.990448+0000","last_became_peered":"2023-05-16T07:17:47.990448+0000","last_unstale":"2023-05-16T07:17:47.992800+0000","last_undegraded":"2023-05-16T07:17:47.992800+0000","last_fullsized":"2023-05-16T07:17:47.992800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364916+0000","last_change":"2023-05-16T07:17:48.364916+0000","last_active":"2023-05-16T07:17:48.364916+0000","last_peered":"2023-05-16T07:17:48.364916+0000","last_clean":"2023-05-16T07:17:48.364916+0000","last_became_active":"2023-05-16T07:17:48.359947+0000","last_became_peered":"2023-05-16T07:17:48.359947+0000","last_unstale":"2023-05-16T07:17:48.364916+0000","last_undegraded":"2023-05-16T07:17:48.364916+0000","last_fullsized":"2023-05-16T07:17:48.364916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999155+0000","last_change":"2023-05-16T07:17:47.996275+0000","last_active":"2023-05-16T07:17:47.999155+0000","last_peered":"2023-05-16T07:17:47.999155+0000","last_clean":"2023-05-16T07:17:47.999155+0000","last_became_active":"2023-05-16T07:17:47.996043+0000","last_became_peered":"2023-05-16T07:17:47.996043+0000","last_unstale":"2023-05-16T07:17:47.999155+0000","last_undegraded":"2023-05-16T07:17:47.999155+0000","last_fullsized":"2023-05-16T07:17:47.999155+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359288+0000","last_change":"2023-05-16T07:17:48.359288+0000","last_active":"2023-05-16T07:17:48.359288+0000","last_peered":"2023-05-16T07:17:48.359288+0000","last_clean":"2023-05-16T07:17:48.359288+0000","last_became_active":"2023-05-16T07:17:48.359014+0000","last_became_peered":"2023-05-16T07:17:48.359014+0000","last_unstale":"2023-05-16T07:17:48.359288+0000","last_undegraded":"2023-05-16T07:17:48.359288+0000","last_fullsized":"2023-05-16T07:17:48.359288+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002785+0000","last_change":"2023-05-16T07:17:48.001724+0000","last_active":"2023-05-16T07:17:48.002785+0000","last_peered":"2023-05-16T07:17:48.002785+0000","last_clean":"2023-05-16T07:17:48.002785+0000","last_became_active":"2023-05-16T07:17:48.001379+0000","last_became_peered":"2023-05-16T07:17:48.001379+0000","last_unstale":"2023-05-16T07:17:48.002785+0000","last_undegraded":"2023-05-16T07:17:48.002785+0000","last_fullsized":"2023-05-16T07:17:48.002785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.373320+0000","last_change":"2023-05-16T07:17:48.373320+0000","last_active":"2023-05-16T07:17:48.373320+0000","last_peered":"2023-05-16T07:17:48.373320+0000","last_clean":"2023-05-16T07:17:48.373320+0000","last_became_active":"2023-05-16T07:17:48.373007+0000","last_became_peered":"2023-05-16T07:17:48.373007+0000","last_unstale":"2023-05-16T07:17:48.373320+0000","last_undegraded":"2023-05-16T07:17:48.373320+0000","last_fullsized":"2023-05-16T07:17:48.373320+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004084+0000","last_change":"2023-05-16T07:17:48.002739+0000","last_active":"2023-05-16T07:17:48.004084+0000","last_peered":"2023-05-16T07:17:48.004084+0000","last_clean":"2023-05-16T07:17:48.004084+0000","last_became_active":"2023-05-16T07:17:48.002458+0000","last_became_peered":"2023-05-16T07:17:48.002458+0000","last_unstale":"2023-05-16T07:17:48.004084+0000","last_undegraded":"2023-05-16T07:17:48.004084+0000","last_fullsized":"2023-05-16T07:17:48.004084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370260+0000","last_change":"2023-05-16T07:17:48.370260+0000","last_active":"2023-05-16T07:17:48.370260+0000","last_peered":"2023-05-16T07:17:48.370260+0000","last_clean":"2023-05-16T07:17:48.370260+0000","last_became_active":"2023-05-16T07:17:48.369957+0000","last_became_peered":"2023-05-16T07:17:48.369957+0000","last_unstale":"2023-05-16T07:17:48.370260+0000","last_undegraded":"2023-05-16T07:17:48.370260+0000","last_fullsized":"2023-05-16T07:17:48.370260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992409+0000","last_change":"2023-05-16T07:17:47.986886+0000","last_active":"2023-05-16T07:17:47.992409+0000","last_peered":"2023-05-16T07:17:47.992409+0000","last_clean":"2023-05-16T07:17:47.992409+0000","last_became_active":"2023-05-16T07:17:47.986620+0000","last_became_peered":"2023-05-16T07:17:47.986620+0000","last_unstale":"2023-05-16T07:17:47.992409+0000","last_undegraded":"2023-05-16T07:17:47.992409+0000","last_fullsized":"2023-05-16T07:17:47.992409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367601+0000","last_change":"2023-05-16T07:17:48.367601+0000","last_active":"2023-05-16T07:17:48.367601+0000","last_peered":"2023-05-16T07:17:48.367601+0000","last_clean":"2023-05-16T07:17:48.367601+0000","last_became_active":"2023-05-16T07:17:48.367376+0000","last_became_peered":"2023-05-16T07:17:48.367376+0000","last_unstale":"2023-05-16T07:17:48.367601+0000","last_undegraded":"2023-05-16T07:17:48.367601+0000","last_fullsized":"2023-05-16T07:17:48.367601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356220+0000","last_change":"2023-05-16T07:17:47.986498+0000","last_active":"2023-05-16T07:17:48.356220+0000","last_peered":"2023-05-16T07:17:48.356220+0000","last_clean":"2023-05-16T07:17:48.356220+0000","last_became_active":"2023-05-16T07:17:47.986245+0000","last_became_peered":"2023-05-16T07:17:47.986245+0000","last_unstale":"2023-05-16T07:17:48.356220+0000","last_undegraded":"2023-05-16T07:17:48.356220+0000","last_fullsized":"2023-05-16T07:17:48.356220+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369102+0000","last_change":"2023-05-16T07:17:48.369102+0000","last_active":"2023-05-16T07:17:48.369102+0000","last_peered":"2023-05-16T07:17:48.369102+0000","last_clean":"2023-05-16T07:17:48.369102+0000","last_became_active":"2023-05-16T07:17:48.366539+0000","last_became_peered":"2023-05-16T07:17:48.366539+0000","last_unstale":"2023-05-16T07:17:48.369102+0000","last_undegraded":"2023-05-16T07:17:48.369102+0000","last_fullsized":"2023-05-16T07:17:48.369102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.344179+0000","last_change":"2023-05-16T07:17:47.992338+0000","last_active":"2023-05-16T07:17:48.344179+0000","last_peered":"2023-05-16T07:17:48.344179+0000","last_clean":"2023-05-16T07:17:48.344179+0000","last_became_active":"2023-05-16T07:17:47.992074+0000","last_became_peered":"2023-05-16T07:17:47.992074+0000","last_unstale":"2023-05-16T07:17:48.344179+0000","last_undegraded":"2023-05-16T07:17:48.344179+0000","last_fullsized":"2023-05-16T07:17:48.344179+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363249+0000","last_change":"2023-05-16T07:17:48.363249+0000","last_active":"2023-05-16T07:17:48.363249+0000","last_peered":"2023-05-16T07:17:48.363249+0000","last_clean":"2023-05-16T07:17:48.363249+0000","last_became_active":"2023-05-16T07:17:48.362952+0000","last_became_peered":"2023-05-16T07:17:48.362952+0000","last_unstale":"2023-05-16T07:17:48.363249+0000","last_undegraded":"2023-05-16T07:17:48.363249+0000","last_fullsized":"2023-05-16T07:17:48.363249+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.372658+0000","last_change":"2023-05-16T07:17:47.994226+0000","last_active":"2023-05-16T07:17:48.372658+0000","last_peered":"2023-05-16T07:17:48.372658+0000","last_clean":"2023-05-16T07:17:48.372658+0000","last_became_active":"2023-05-16T07:17:47.993765+0000","last_became_peered":"2023-05-16T07:17:47.993765+0000","last_unstale":"2023-05-16T07:17:48.372658+0000","last_undegraded":"2023-05-16T07:17:48.372658+0000","last_fullsized":"2023-05-16T07:17:48.372658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364624+0000","last_change":"2023-05-16T07:17:48.364624+0000","last_active":"2023-05-16T07:17:48.364624+0000","last_peered":"2023-05-16T07:17:48.364624+0000","last_clean":"2023-05-16T07:17:48.364624+0000","last_became_active":"2023-05-16T07:17:48.364343+0000","last_became_peered":"2023-05-16T07:17:48.364343+0000","last_unstale":"2023-05-16T07:17:48.364624+0000","last_undegraded":"2023-05-16T07:17:48.364624+0000","last_fullsized":"2023-05-16T07:17:48.364624+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000133+0000","last_change":"2023-05-16T07:17:47.997474+0000","last_active":"2023-05-16T07:17:48.000133+0000","last_peered":"2023-05-16T07:17:48.000133+0000","last_clean":"2023-05-16T07:17:48.000133+0000","last_became_active":"2023-05-16T07:17:47.997194+0000","last_became_peered":"2023-05-16T07:17:47.997194+0000","last_unstale":"2023-05-16T07:17:48.000133+0000","last_undegraded":"2023-05-16T07:17:48.000133+0000","last_fullsized":"2023-05-16T07:17:48.000133+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364592+0000","last_change":"2023-05-16T07:17:48.364592+0000","last_active":"2023-05-16T07:17:48.364592+0000","last_peered":"2023-05-16T07:17:48.364592+0000","last_clean":"2023-05-16T07:17:48.364592+0000","last_became_active":"2023-05-16T07:17:48.364360+0000","last_became_peered":"2023-05-16T07:17:48.364360+0000","last_unstale":"2023-05-16T07:17:48.364592+0000","last_undegraded":"2023-05-16T07:17:48.364592+0000","last_fullsized":"2023-05-16T07:17:48.364592+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000752+0000","last_change":"2023-05-16T07:17:47.994013+0000","last_active":"2023-05-16T07:17:48.000752+0000","last_peered":"2023-05-16T07:17:48.000752+0000","last_clean":"2023-05-16T07:17:48.000752+0000","last_became_active":"2023-05-16T07:17:47.993444+0000","last_became_peered":"2023-05-16T07:17:47.993444+0000","last_unstale":"2023-05-16T07:17:48.000752+0000","last_undegraded":"2023-05-16T07:17:48.000752+0000","last_fullsized":"2023-05-16T07:17:48.000752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370940+0000","last_change":"2023-05-16T07:17:48.370940+0000","last_active":"2023-05-16T07:17:48.370940+0000","last_peered":"2023-05-16T07:17:48.370940+0000","last_clean":"2023-05-16T07:17:48.370940+0000","last_became_active":"2023-05-16T07:17:48.370708+0000","last_became_peered":"2023-05-16T07:17:48.370708+0000","last_unstale":"2023-05-16T07:17:48.370940+0000","last_undegraded":"2023-05-16T07:17:48.370940+0000","last_fullsized":"2023-05-16T07:17:48.370940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002039+0000","last_change":"2023-05-16T07:17:48.000829+0000","last_active":"2023-05-16T07:17:48.002039+0000","last_peered":"2023-05-16T07:17:48.002039+0000","last_clean":"2023-05-16T07:17:48.002039+0000","last_became_active":"2023-05-16T07:17:48.000513+0000","last_became_peered":"2023-05-16T07:17:48.000513+0000","last_unstale":"2023-05-16T07:17:48.002039+0000","last_undegraded":"2023-05-16T07:17:48.002039+0000","last_fullsized":"2023-05-16T07:17:48.002039+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362178+0000","last_change":"2023-05-16T07:17:48.362178+0000","last_active":"2023-05-16T07:17:48.362178+0000","last_peered":"2023-05-16T07:17:48.362178+0000","last_clean":"2023-05-16T07:17:48.362178+0000","last_became_active":"2023-05-16T07:17:48.361907+0000","last_became_peered":"2023-05-16T07:17:48.361907+0000","last_unstale":"2023-05-16T07:17:48.362178+0000","last_undegraded":"2023-05-16T07:17:48.362178+0000","last_fullsized":"2023-05-16T07:17:48.362178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002118+0000","last_change":"2023-05-16T07:17:47.998161+0000","last_active":"2023-05-16T07:17:48.002118+0000","last_peered":"2023-05-16T07:17:48.002118+0000","last_clean":"2023-05-16T07:17:48.002118+0000","last_became_active":"2023-05-16T07:17:47.997888+0000","last_became_peered":"2023-05-16T07:17:47.997888+0000","last_unstale":"2023-05-16T07:17:48.002118+0000","last_undegraded":"2023-05-16T07:17:48.002118+0000","last_fullsized":"2023-05-16T07:17:48.002118+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.355734+0000","last_change":"2023-05-16T07:17:48.355734+0000","last_active":"2023-05-16T07:17:48.355734+0000","last_peered":"2023-05-16T07:17:48.355734+0000","last_clean":"2023-05-16T07:17:48.355734+0000","last_became_active":"2023-05-16T07:17:48.355470+0000","last_became_peered":"2023-05-16T07:17:48.355470+0000","last_unstale":"2023-05-16T07:17:48.355734+0000","last_undegraded":"2023-05-16T07:17:48.355734+0000","last_fullsized":"2023-05-16T07:17:48.355734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.998926+0000","last_change":"2023-05-16T07:17:47.993782+0000","last_active":"2023-05-16T07:17:47.998926+0000","last_peered":"2023-05-16T07:17:47.998926+0000","last_clean":"2023-05-16T07:17:47.998926+0000","last_became_active":"2023-05-16T07:17:47.993482+0000","last_became_peered":"2023-05-16T07:17:47.993482+0000","last_unstale":"2023-05-16T07:17:47.998926+0000","last_undegraded":"2023-05-16T07:17:47.998926+0000","last_fullsized":"2023-05-16T07:17:47.998926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367888+0000","last_change":"2023-05-16T07:17:48.367888+0000","last_active":"2023-05-16T07:17:48.367888+0000","last_peered":"2023-05-16T07:17:48.367888+0000","last_clean":"2023-05-16T07:17:48.367888+0000","last_became_active":"2023-05-16T07:17:48.367585+0000","last_became_peered":"2023-05-16T07:17:48.367585+0000","last_unstale":"2023-05-16T07:17:48.367888+0000","last_undegraded":"2023-05-16T07:17:48.367888+0000","last_fullsized":"2023-05-16T07:17:48.367888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003086+0000","last_change":"2023-05-16T07:17:47.997668+0000","last_active":"2023-05-16T07:17:48.003086+0000","last_peered":"2023-05-16T07:17:48.003086+0000","last_clean":"2023-05-16T07:17:48.003086+0000","last_became_active":"2023-05-16T07:17:47.997389+0000","last_became_peered":"2023-05-16T07:17:47.997389+0000","last_unstale":"2023-05-16T07:17:48.003086+0000","last_undegraded":"2023-05-16T07:17:48.003086+0000","last_fullsized":"2023-05-16T07:17:48.003086+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361350+0000","last_change":"2023-05-16T07:17:48.361350+0000","last_active":"2023-05-16T07:17:48.361350+0000","last_peered":"2023-05-16T07:17:48.361350+0000","last_clean":"2023-05-16T07:17:48.361350+0000","last_became_active":"2023-05-16T07:17:48.361051+0000","last_became_peered":"2023-05-16T07:17:48.361051+0000","last_unstale":"2023-05-16T07:17:48.361350+0000","last_undegraded":"2023-05-16T07:17:48.361350+0000","last_fullsized":"2023-05-16T07:17:48.361350+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.001575+0000","last_change":"2023-05-16T07:17:47.999934+0000","last_active":"2023-05-16T07:17:48.001575+0000","last_peered":"2023-05-16T07:17:48.001575+0000","last_clean":"2023-05-16T07:17:48.001575+0000","last_became_active":"2023-05-16T07:17:47.999659+0000","last_became_peered":"2023-05-16T07:17:47.999659+0000","last_unstale":"2023-05-16T07:17:48.001575+0000","last_undegraded":"2023-05-16T07:17:48.001575+0000","last_fullsized":"2023-05-16T07:17:48.001575+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.354944+0000","last_change":"2023-05-16T07:17:48.354944+0000","last_active":"2023-05-16T07:17:48.354944+0000","last_peered":"2023-05-16T07:17:48.354944+0000","last_clean":"2023-05-16T07:17:48.354944+0000","last_became_active":"2023-05-16T07:17:48.354592+0000","last_became_peered":"2023-05-16T07:17:48.354592+0000","last_unstale":"2023-05-16T07:17:48.354944+0000","last_undegraded":"2023-05-16T07:17:48.354944+0000","last_fullsized":"2023-05-16T07:17:48.354944+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"92'3","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.371635+0000","last_change":"2023-05-16T07:17:47.999263+0000","last_active":"2023-05-16T07:17:48.371635+0000","last_peered":"2023-05-16T07:17:48.371635+0000","last_clean":"2023-05-16T07:17:48.371635+0000","last_became_active":"2023-05-16T07:17:47.998864+0000","last_became_peered":"2023-05-16T07:17:47.998864+0000","last_unstale":"2023-05-16T07:17:48.371635+0000","last_undegraded":"2023-05-16T07:17:48.371635+0000","last_fullsized":"2023-05-16T07:17:48.371635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367339+0000","last_change":"2023-05-16T07:17:48.367339+0000","last_active":"2023-05-16T07:17:48.367339+0000","last_peered":"2023-05-16T07:17:48.367339+0000","last_clean":"2023-05-16T07:17:48.367339+0000","last_became_active":"2023-05-16T07:17:48.367030+0000","last_became_peered":"2023-05-16T07:17:48.367030+0000","last_unstale":"2023-05-16T07:17:48.367339+0000","last_undegraded":"2023-05-16T07:17:48.367339+0000","last_fullsized":"2023-05-16T07:17:48.367339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.358531+0000","last_change":"2023-05-16T07:17:47.997418+0000","last_active":"2023-05-16T07:17:48.358531+0000","last_peered":"2023-05-16T07:17:48.358531+0000","last_clean":"2023-05-16T07:17:48.358531+0000","last_became_active":"2023-05-16T07:17:47.996792+0000","last_became_peered":"2023-05-16T07:17:47.996792+0000","last_unstale":"2023-05-16T07:17:48.358531+0000","last_undegraded":"2023-05-16T07:17:48.358531+0000","last_fullsized":"2023-05-16T07:17:48.358531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362857+0000","last_change":"2023-05-16T07:17:48.362857+0000","last_active":"2023-05-16T07:17:48.362857+0000","last_peered":"2023-05-16T07:17:48.362857+0000","last_clean":"2023-05-16T07:17:48.362857+0000","last_became_active":"2023-05-16T07:17:48.362511+0000","last_became_peered":"2023-05-16T07:17:48.362511+0000","last_unstale":"2023-05-16T07:17:48.362857+0000","last_undegraded":"2023-05-16T07:17:48.362857+0000","last_fullsized":"2023-05-16T07:17:48.362857+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.352942+0000","last_change":"2023-05-16T07:17:47.996369+0000","last_active":"2023-05-16T07:17:48.352942+0000","last_peered":"2023-05-16T07:17:48.352942+0000","last_clean":"2023-05-16T07:17:48.352942+0000","last_became_active":"2023-05-16T07:17:47.996116+0000","last_became_peered":"2023-05-16T07:17:47.996116+0000","last_unstale":"2023-05-16T07:17:48.352942+0000","last_undegraded":"2023-05-16T07:17:48.352942+0000","last_fullsized":"2023-05-16T07:17:48.352942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362701+0000","last_change":"2023-05-16T07:17:48.362701+0000","last_active":"2023-05-16T07:17:48.362701+0000","last_peered":"2023-05-16T07:17:48.362701+0000","last_clean":"2023-05-16T07:17:48.362701+0000","last_became_active":"2023-05-16T07:17:48.362100+0000","last_became_peered":"2023-05-16T07:17:48.362100+0000","last_unstale":"2023-05-16T07:17:48.362701+0000","last_undegraded":"2023-05-16T07:17:48.362701+0000","last_fullsized":"2023-05-16T07:17:48.362701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.994190+0000","last_change":"2023-05-16T07:17:47.989228+0000","last_active":"2023-05-16T07:17:47.994190+0000","last_peered":"2023-05-16T07:17:47.994190+0000","last_clean":"2023-05-16T07:17:47.994190+0000","last_became_active":"2023-05-16T07:17:47.988986+0000","last_became_peered":"2023-05-16T07:17:47.988986+0000","last_unstale":"2023-05-16T07:17:47.994190+0000","last_undegraded":"2023-05-16T07:17:47.994190+0000","last_fullsized":"2023-05-16T07:17:47.994190+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356914+0000","last_change":"2023-05-16T07:17:48.356914+0000","last_active":"2023-05-16T07:17:48.356914+0000","last_peered":"2023-05-16T07:17:48.356914+0000","last_clean":"2023-05-16T07:17:48.356914+0000","last_became_active":"2023-05-16T07:17:48.356624+0000","last_became_peered":"2023-05-16T07:17:48.356624+0000","last_unstale":"2023-05-16T07:17:48.356914+0000","last_undegraded":"2023-05-16T07:17:48.356914+0000","last_fullsized":"2023-05-16T07:17:48.356914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992974+0000","last_change":"2023-05-16T07:17:47.986788+0000","last_active":"2023-05-16T07:17:47.992974+0000","last_peered":"2023-05-16T07:17:47.992974+0000","last_clean":"2023-05-16T07:17:47.992974+0000","last_became_active":"2023-05-16T07:17:47.986069+0000","last_became_peered":"2023-05-16T07:17:47.986069+0000","last_unstale":"2023-05-16T07:17:47.992974+0000","last_undegraded":"2023-05-16T07:17:47.992974+0000","last_fullsized":"2023-05-16T07:17:47.992974+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.348736+0000","last_change":"2023-05-16T07:17:48.348736+0000","last_active":"2023-05-16T07:17:48.348736+0000","last_peered":"2023-05-16T07:17:48.348736+0000","last_clean":"2023-05-16T07:17:48.348736+0000","last_became_active":"2023-05-16T07:17:48.348445+0000","last_became_peered":"2023-05-16T07:17:48.348445+0000","last_unstale":"2023-05-16T07:17:48.348736+0000","last_undegraded":"2023-05-16T07:17:48.348736+0000","last_fullsized":"2023-05-16T07:17:48.348736+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"92'4","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.372428+0000","last_change":"2023-05-16T07:17:47.994173+0000","last_active":"2023-05-16T07:17:48.372428+0000","last_peered":"2023-05-16T07:17:48.372428+0000","last_clean":"2023-05-16T07:17:48.372428+0000","last_became_active":"2023-05-16T07:17:47.993972+0000","last_became_peered":"2023-05-16T07:17:47.993972+0000","last_unstale":"2023-05-16T07:17:48.372428+0000","last_undegraded":"2023-05-16T07:17:48.372428+0000","last_fullsized":"2023-05-16T07:17:48.372428+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359764+0000","last_change":"2023-05-16T07:17:48.359764+0000","last_active":"2023-05-16T07:17:48.359764+0000","last_peered":"2023-05-16T07:17:48.359764+0000","last_clean":"2023-05-16T07:17:48.359764+0000","last_became_active":"2023-05-16T07:17:48.359461+0000","last_became_peered":"2023-05-16T07:17:48.359461+0000","last_unstale":"2023-05-16T07:17:48.359764+0000","last_undegraded":"2023-05-16T07:17:48.359764+0000","last_fullsized":"2023-05-16T07:17:48.359764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363322+0000","last_change":"2023-05-16T07:17:48.003802+0000","last_active":"2023-05-16T07:17:48.363322+0000","last_peered":"2023-05-16T07:17:48.363322+0000","last_clean":"2023-05-16T07:17:48.363322+0000","last_became_active":"2023-05-16T07:17:48.003484+0000","last_became_peered":"2023-05-16T07:17:48.003484+0000","last_unstale":"2023-05-16T07:17:48.363322+0000","last_undegraded":"2023-05-16T07:17:48.363322+0000","last_fullsized":"2023-05-16T07:17:48.363322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.346775+0000","last_change":"2023-05-16T07:17:48.346775+0000","last_active":"2023-05-16T07:17:48.346775+0000","last_peered":"2023-05-16T07:17:48.346775+0000","last_clean":"2023-05-16T07:17:48.346775+0000","last_became_active":"2023-05-16T07:17:48.346437+0000","last_became_peered":"2023-05-16T07:17:48.346437+0000","last_unstale":"2023-05-16T07:17:48.346775+0000","last_undegraded":"2023-05-16T07:17:48.346775+0000","last_fullsized":"2023-05-16T07:17:48.346775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.998900+0000","last_change":"2023-05-16T07:17:47.993667+0000","last_active":"2023-05-16T07:17:47.998900+0000","last_peered":"2023-05-16T07:17:47.998900+0000","last_clean":"2023-05-16T07:17:47.998900+0000","last_became_active":"2023-05-16T07:17:47.993457+0000","last_became_peered":"2023-05-16T07:17:47.993457+0000","last_unstale":"2023-05-16T07:17:47.998900+0000","last_undegraded":"2023-05-16T07:17:47.998900+0000","last_fullsized":"2023-05-16T07:17:47.998900+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363037+0000","last_change":"2023-05-16T07:17:48.363037+0000","last_active":"2023-05-16T07:17:48.363037+0000","last_peered":"2023-05-16T07:17:48.363037+0000","last_clean":"2023-05-16T07:17:48.363037+0000","last_became_active":"2023-05-16T07:17:48.362749+0000","last_became_peered":"2023-05-16T07:17:48.362749+0000","last_unstale":"2023-05-16T07:17:48.363037+0000","last_undegraded":"2023-05-16T07:17:48.363037+0000","last_fullsized":"2023-05-16T07:17:48.363037+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003276+0000","last_change":"2023-05-16T07:17:48.000633+0000","last_active":"2023-05-16T07:17:48.003276+0000","last_peered":"2023-05-16T07:17:48.003276+0000","last_clean":"2023-05-16T07:17:48.003276+0000","last_became_active":"2023-05-16T07:17:48.000299+0000","last_became_peered":"2023-05-16T07:17:48.000299+0000","last_unstale":"2023-05-16T07:17:48.003276+0000","last_undegraded":"2023-05-16T07:17:48.003276+0000","last_fullsized":"2023-05-16T07:17:48.003276+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369795+0000","last_change":"2023-05-16T07:17:48.369795+0000","last_active":"2023-05-16T07:17:48.369795+0000","last_peered":"2023-05-16T07:17:48.369795+0000","last_clean":"2023-05-16T07:17:48.369795+0000","last_became_active":"2023-05-16T07:17:48.369454+0000","last_became_peered":"2023-05-16T07:17:48.369454+0000","last_unstale":"2023-05-16T07:17:48.369795+0000","last_undegraded":"2023-05-16T07:17:48.369795+0000","last_fullsized":"2023-05-16T07:17:48.369795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003191+0000","last_change":"2023-05-16T07:17:47.997268+0000","last_active":"2023-05-16T07:17:48.003191+0000","last_peered":"2023-05-16T07:17:48.003191+0000","last_clean":"2023-05-16T07:17:48.003191+0000","last_became_active":"2023-05-16T07:17:47.996980+0000","last_became_peered":"2023-05-16T07:17:47.996980+0000","last_unstale":"2023-05-16T07:17:48.003191+0000","last_undegraded":"2023-05-16T07:17:48.003191+0000","last_fullsized":"2023-05-16T07:17:48.003191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.349014+0000","last_change":"2023-05-16T07:17:48.349014+0000","last_active":"2023-05-16T07:17:48.349014+0000","last_peered":"2023-05-16T07:17:48.349014+0000","last_clean":"2023-05-16T07:17:48.349014+0000","last_became_active":"2023-05-16T07:17:48.348473+0000","last_became_peered":"2023-05-16T07:17:48.348473+0000","last_unstale":"2023-05-16T07:17:48.349014+0000","last_undegraded":"2023-05-16T07:17:48.349014+0000","last_fullsized":"2023-05-16T07:17:48.349014+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.994092+0000","last_change":"2023-05-16T07:17:47.990507+0000","last_active":"2023-05-16T07:17:47.994092+0000","last_peered":"2023-05-16T07:17:47.994092+0000","last_clean":"2023-05-16T07:17:47.994092+0000","last_became_active":"2023-05-16T07:17:47.990284+0000","last_became_peered":"2023-05-16T07:17:47.990284+0000","last_unstale":"2023-05-16T07:17:47.994092+0000","last_undegraded":"2023-05-16T07:17:47.994092+0000","last_fullsized":"2023-05-16T07:17:47.994092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360885+0000","last_change":"2023-05-16T07:17:48.360885+0000","last_active":"2023-05-16T07:17:48.360885+0000","last_peered":"2023-05-16T07:17:48.360885+0000","last_clean":"2023-05-16T07:17:48.360885+0000","last_became_active":"2023-05-16T07:17:48.360567+0000","last_became_peered":"2023-05-16T07:17:48.360567+0000","last_unstale":"2023-05-16T07:17:48.360885+0000","last_undegraded":"2023-05-16T07:17:48.360885+0000","last_fullsized":"2023-05-16T07:17:48.360885+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.989650+0000","last_change":"2023-05-16T07:17:47.984015+0000","last_active":"2023-05-16T07:17:47.989650+0000","last_peered":"2023-05-16T07:17:47.989650+0000","last_clean":"2023-05-16T07:17:47.989650+0000","last_became_active":"2023-05-16T07:17:47.983727+0000","last_became_peered":"2023-05-16T07:17:47.983727+0000","last_unstale":"2023-05-16T07:17:47.989650+0000","last_undegraded":"2023-05-16T07:17:47.989650+0000","last_fullsized":"2023-05-16T07:17:47.989650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.373211+0000","last_change":"2023-05-16T07:17:48.373211+0000","last_active":"2023-05-16T07:17:48.373211+0000","last_peered":"2023-05-16T07:17:48.373211+0000","last_clean":"2023-05-16T07:17:48.373211+0000","last_became_active":"2023-05-16T07:17:48.372954+0000","last_became_peered":"2023-05-16T07:17:48.372954+0000","last_unstale":"2023-05-16T07:17:48.373211+0000","last_undegraded":"2023-05-16T07:17:48.373211+0000","last_fullsized":"2023-05-16T07:17:48.373211+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999851+0000","last_change":"2023-05-16T07:17:47.996848+0000","last_active":"2023-05-16T07:17:47.999851+0000","last_peered":"2023-05-16T07:17:47.999851+0000","last_clean":"2023-05-16T07:17:47.999851+0000","last_became_active":"2023-05-16T07:17:47.996523+0000","last_became_peered":"2023-05-16T07:17:47.996523+0000","last_unstale":"2023-05-16T07:17:47.999851+0000","last_undegraded":"2023-05-16T07:17:47.999851+0000","last_fullsized":"2023-05-16T07:17:47.999851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367667+0000","last_change":"2023-05-16T07:17:48.367667+0000","last_active":"2023-05-16T07:17:48.367667+0000","last_peered":"2023-05-16T07:17:48.367667+0000","last_clean":"2023-05-16T07:17:48.367667+0000","last_became_active":"2023-05-16T07:17:48.367451+0000","last_became_peered":"2023-05-16T07:17:48.367451+0000","last_unstale":"2023-05-16T07:17:48.367667+0000","last_undegraded":"2023-05-16T07:17:48.367667+0000","last_fullsized":"2023-05-16T07:17:48.367667+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003234+0000","last_change":"2023-05-16T07:17:47.996821+0000","last_active":"2023-05-16T07:17:48.003234+0000","last_peered":"2023-05-16T07:17:48.003234+0000","last_clean":"2023-05-16T07:17:48.003234+0000","last_became_active":"2023-05-16T07:17:47.996524+0000","last_became_peered":"2023-05-16T07:17:47.996524+0000","last_unstale":"2023-05-16T07:17:48.003234+0000","last_undegraded":"2023-05-16T07:17:48.003234+0000","last_fullsized":"2023-05-16T07:17:48.003234+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369060+0000","last_change":"2023-05-16T07:17:48.369060+0000","last_active":"2023-05-16T07:17:48.369060+0000","last_peered":"2023-05-16T07:17:48.369060+0000","last_clean":"2023-05-16T07:17:48.369060+0000","last_became_active":"2023-05-16T07:17:48.366229+0000","last_became_peered":"2023-05-16T07:17:48.366229+0000","last_unstale":"2023-05-16T07:17:48.369060+0000","last_undegraded":"2023-05-16T07:17:48.369060+0000","last_fullsized":"2023-05-16T07:17:48.369060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002889+0000","last_change":"2023-05-16T07:17:47.993822+0000","last_active":"2023-05-16T07:17:48.002889+0000","last_peered":"2023-05-16T07:17:48.002889+0000","last_clean":"2023-05-16T07:17:48.002889+0000","last_became_active":"2023-05-16T07:17:47.993553+0000","last_became_peered":"2023-05-16T07:17:47.993553+0000","last_unstale":"2023-05-16T07:17:48.002889+0000","last_undegraded":"2023-05-16T07:17:48.002889+0000","last_fullsized":"2023-05-16T07:17:48.002889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.991073+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:17:47.991073+0000","last_peered":"2023-05-16T07:17:47.991073+0000","last_clean":"2023-05-16T07:17:47.991073+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:17:47.991073+0000","last_undegraded":"2023-05-16T07:17:47.991073+0000","last_fullsized":"2023-05-16T07:17:47.991073+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365098+0000","last_change":"2023-05-16T07:17:48.365098+0000","last_active":"2023-05-16T07:17:48.365098+0000","last_peered":"2023-05-16T07:17:48.365098+0000","last_clean":"2023-05-16T07:17:48.365098+0000","last_became_active":"2023-05-16T07:17:48.364842+0000","last_became_peered":"2023-05-16T07:17:48.364842+0000","last_unstale":"2023-05-16T07:17:48.365098+0000","last_undegraded":"2023-05-16T07:17:48.365098+0000","last_fullsized":"2023-05-16T07:17:48.365098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.344591+0000","last_change":"2023-05-16T07:17:47.999936+0000","last_active":"2023-05-16T07:17:48.344591+0000","last_peered":"2023-05-16T07:17:48.344591+0000","last_clean":"2023-05-16T07:17:48.344591+0000","last_became_active":"2023-05-16T07:17:47.999667+0000","last_became_peered":"2023-05-16T07:17:47.999667+0000","last_unstale":"2023-05-16T07:17:48.344591+0000","last_undegraded":"2023-05-16T07:17:48.344591+0000","last_fullsized":"2023-05-16T07:17:48.344591+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992638+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:17:47.992638+0000","last_peered":"2023-05-16T07:17:47.992638+0000","last_clean":"2023-05-16T07:17:47.992638+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:17:47.992638+0000","last_undegraded":"2023-05-16T07:17:47.992638+0000","last_fullsized":"2023-05-16T07:17:47.992638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359291+0000","last_change":"2023-05-16T07:17:48.359291+0000","last_active":"2023-05-16T07:17:48.359291+0000","last_peered":"2023-05-16T07:17:48.359291+0000","last_clean":"2023-05-16T07:17:48.359291+0000","last_became_active":"2023-05-16T07:17:48.358992+0000","last_became_peered":"2023-05-16T07:17:48.358992+0000","last_unstale":"2023-05-16T07:17:48.359291+0000","last_undegraded":"2023-05-16T07:17:48.359291+0000","last_fullsized":"2023-05-16T07:17:48.359291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000165+0000","last_change":"2023-05-16T07:17:47.997323+0000","last_active":"2023-05-16T07:17:48.000165+0000","last_peered":"2023-05-16T07:17:48.000165+0000","last_clean":"2023-05-16T07:17:48.000165+0000","last_became_active":"2023-05-16T07:17:47.997062+0000","last_became_peered":"2023-05-16T07:17:47.997062+0000","last_unstale":"2023-05-16T07:17:48.000165+0000","last_undegraded":"2023-05-16T07:17:48.000165+0000","last_fullsized":"2023-05-16T07:17:48.000165+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.976935+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:17:47.976935+0000","last_peered":"2023-05-16T07:17:47.976935+0000","last_clean":"2023-05-16T07:17:47.976935+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:17:47.976935+0000","last_undegraded":"2023-05-16T07:17:47.976935+0000","last_fullsized":"2023-05-16T07:17:47.976935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367598+0000","last_change":"2023-05-16T07:17:48.367598+0000","last_active":"2023-05-16T07:17:48.367598+0000","last_peered":"2023-05-16T07:17:48.367598+0000","last_clean":"2023-05-16T07:17:48.367598+0000","last_became_active":"2023-05-16T07:17:48.367333+0000","last_became_peered":"2023-05-16T07:17:48.367333+0000","last_unstale":"2023-05-16T07:17:48.367598+0000","last_undegraded":"2023-05-16T07:17:48.367598+0000","last_fullsized":"2023-05-16T07:17:48.367598+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.993947+0000","last_change":"2023-05-16T07:17:47.989476+0000","last_active":"2023-05-16T07:17:47.993947+0000","last_peered":"2023-05-16T07:17:47.993947+0000","last_clean":"2023-05-16T07:17:47.993947+0000","last_became_active":"2023-05-16T07:17:47.989033+0000","last_became_peered":"2023-05-16T07:17:47.989033+0000","last_unstale":"2023-05-16T07:17:47.993947+0000","last_undegraded":"2023-05-16T07:17:47.993947+0000","last_fullsized":"2023-05-16T07:17:47.993947+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.986875+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:17:47.986875+0000","last_peered":"2023-05-16T07:17:47.986875+0000","last_clean":"2023-05-16T07:17:47.986875+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:17:47.986875+0000","last_undegraded":"2023-05-16T07:17:47.986875+0000","last_fullsized":"2023-05-16T07:17:47.986875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.984937+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:17:47.984937+0000","last_peered":"2023-05-16T07:17:47.984937+0000","last_clean":"2023-05-16T07:17:47.984937+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:17:47.984937+0000","last_undegraded":"2023-05-16T07:17:47.984937+0000","last_fullsized":"2023-05-16T07:17:47.984937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366068+0000","last_change":"2023-05-16T07:17:48.366068+0000","last_active":"2023-05-16T07:17:48.366068+0000","last_peered":"2023-05-16T07:17:48.366068+0000","last_clean":"2023-05-16T07:17:48.366068+0000","last_became_active":"2023-05-16T07:17:48.365421+0000","last_became_peered":"2023-05-16T07:17:48.365421+0000","last_unstale":"2023-05-16T07:17:48.366068+0000","last_undegraded":"2023-05-16T07:17:48.366068+0000","last_fullsized":"2023-05-16T07:17:48.366068+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.997143+0000","last_change":"2023-05-16T07:17:47.992963+0000","last_active":"2023-05-16T07:17:47.997143+0000","last_peered":"2023-05-16T07:17:47.997143+0000","last_clean":"2023-05-16T07:17:47.997143+0000","last_became_active":"2023-05-16T07:17:47.992701+0000","last_became_peered":"2023-05-16T07:17:47.992701+0000","last_unstale":"2023-05-16T07:17:47.997143+0000","last_undegraded":"2023-05-16T07:17:47.997143+0000","last_fullsized":"2023-05-16T07:17:47.997143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.984959+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:17:47.984959+0000","last_peered":"2023-05-16T07:17:47.984959+0000","last_clean":"2023-05-16T07:17:47.984959+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:17:47.984959+0000","last_undegraded":"2023-05-16T07:17:47.984959+0000","last_fullsized":"2023-05-16T07:17:47.984959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.352790+0000","last_change":"2023-05-16T07:17:48.352790+0000","last_active":"2023-05-16T07:17:48.352790+0000","last_peered":"2023-05-16T07:17:48.352790+0000","last_clean":"2023-05-16T07:17:48.352790+0000","last_became_active":"2023-05-16T07:17:48.352531+0000","last_became_peered":"2023-05-16T07:17:48.352531+0000","last_unstale":"2023-05-16T07:17:48.352790+0000","last_undegraded":"2023-05-16T07:17:48.352790+0000","last_fullsized":"2023-05-16T07:17:48.352790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003320+0000","last_change":"2023-05-16T07:17:47.997542+0000","last_active":"2023-05-16T07:17:48.003320+0000","last_peered":"2023-05-16T07:17:48.003320+0000","last_clean":"2023-05-16T07:17:48.003320+0000","last_became_active":"2023-05-16T07:17:47.997099+0000","last_became_peered":"2023-05-16T07:17:47.997099+0000","last_unstale":"2023-05-16T07:17:48.003320+0000","last_undegraded":"2023-05-16T07:17:48.003320+0000","last_fullsized":"2023-05-16T07:17:48.003320+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.983661+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:17:47.983661+0000","last_peered":"2023-05-16T07:17:47.983661+0000","last_clean":"2023-05-16T07:17:47.983661+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:17:47.983661+0000","last_undegraded":"2023-05-16T07:17:47.983661+0000","last_fullsized":"2023-05-16T07:17:47.983661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361582+0000","last_change":"2023-05-16T07:17:48.361582+0000","last_active":"2023-05-16T07:17:48.361582+0000","last_peered":"2023-05-16T07:17:48.361582+0000","last_clean":"2023-05-16T07:17:48.361582+0000","last_became_active":"2023-05-16T07:17:48.361365+0000","last_became_peered":"2023-05-16T07:17:48.361365+0000","last_unstale":"2023-05-16T07:17:48.361582+0000","last_undegraded":"2023-05-16T07:17:48.361582+0000","last_fullsized":"2023-05-16T07:17:48.361582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.344381+0000","last_change":"2023-05-16T07:17:47.998272+0000","last_active":"2023-05-16T07:17:48.344381+0000","last_peered":"2023-05-16T07:17:48.344381+0000","last_clean":"2023-05-16T07:17:48.344381+0000","last_became_active":"2023-05-16T07:17:47.998004+0000","last_became_peered":"2023-05-16T07:17:47.998004+0000","last_unstale":"2023-05-16T07:17:48.344381+0000","last_undegraded":"2023-05-16T07:17:48.344381+0000","last_fullsized":"2023-05-16T07:17:48.344381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.990320+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:17:47.990320+0000","last_peered":"2023-05-16T07:17:47.990320+0000","last_clean":"2023-05-16T07:17:47.990320+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:17:47.990320+0000","last_undegraded":"2023-05-16T07:17:47.990320+0000","last_fullsized":"2023-05-16T07:17:47.990320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359459+0000","last_change":"2023-05-16T07:17:48.359459+0000","last_active":"2023-05-16T07:17:48.359459+0000","last_peered":"2023-05-16T07:17:48.359459+0000","last_clean":"2023-05-16T07:17:48.359459+0000","last_became_active":"2023-05-16T07:17:48.359183+0000","last_became_peered":"2023-05-16T07:17:48.359183+0000","last_unstale":"2023-05-16T07:17:48.359459+0000","last_undegraded":"2023-05-16T07:17:48.359459+0000","last_fullsized":"2023-05-16T07:17:48.359459+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363348+0000","last_change":"2023-05-16T07:17:47.999113+0000","last_active":"2023-05-16T07:17:48.363348+0000","last_peered":"2023-05-16T07:17:48.363348+0000","last_clean":"2023-05-16T07:17:48.363348+0000","last_became_active":"2023-05-16T07:17:47.998844+0000","last_became_peered":"2023-05-16T07:17:47.998844+0000","last_unstale":"2023-05-16T07:17:48.363348+0000","last_undegraded":"2023-05-16T07:17:48.363348+0000","last_fullsized":"2023-05-16T07:17:48.363348+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.980790+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:17:47.980790+0000","last_peered":"2023-05-16T07:17:47.980790+0000","last_clean":"2023-05-16T07:17:47.980790+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:17:47.980790+0000","last_undegraded":"2023-05-16T07:17:47.980790+0000","last_fullsized":"2023-05-16T07:17:47.980790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575033,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17624,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":16512,"kb_avail":94354216,"statfs":{"total":96636764160,"available":96618717184,"internally_reserved":0,"allocated":1130496,"data_stored":577635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16908288},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.439,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.751,"5min":0.81399999999999995,"15min":0.84199999999999997},"last":0.371},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.47399999999999998,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":0.873},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.752,"5min":0.876,"15min":0.876},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.48599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.438}]},{"osd":3,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66600000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.308},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.752,"15min":0.81999999999999995},"last":0.33000000000000002}]},{"osd":5,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68500000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.40899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.999},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.95099999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.46200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21060,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94350780,"statfs":{"total":96636764160,"available":96615198720,"internally_reserved":0,"allocated":1110016,"data_stored":569296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.317,"15min":0.26400000000000001},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66500000000000004,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76000000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.311,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.89400000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.503,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":4.048,"15min":4.048},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.749,"5min":0.749,"15min":0.92300000000000004},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.62}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17028,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":15936,"kb_avail":94354812,"statfs":{"total":96636764160,"available":96619327488,"internally_reserved":0,"allocated":1110016,"data_stored":569296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16318464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.69899999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.48199999999999998,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.313}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36499999999999999,"15min":0.378},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26000000000000001},"max":{"1min":0.49399999999999999,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.52700000000000002,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38300000000000001,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.501,"5min":0.622,"15min":0.64000000000000001},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71199999999999997,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.66900000000000004}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76100000000000001,"5min":0.94699999999999995,"15min":1.087},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.93700000000000006,"15min":1.204},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.627,"15min":0.64100000000000001},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41299999999999998},"max":{"1min":0.76900000000000002,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16056,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":14976,"kb_avail":94355784,"statfs":{"total":96636764160,"available":96620322816,"internally_reserved":0,"allocated":1097728,"data_stored":562033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15335424},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33900000000000002,"15min":0.35699999999999998},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47099999999999997,"5min":0.53700000000000003,"15min":0.60199999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.38600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.35299999999999998,"15min":0.373},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.40300000000000002},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.50600000000000001,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.42099999999999999}]},{"osd":3,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38600000000000001,"15min":0.40699999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.24199999999999999},"max":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.62}]},{"osd":5,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60599999999999998,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":1.157},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.505}]},{"osd":6,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":1.0209999999999999}]},{"osd":7,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.65000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85499999999999998,"5min":0.93700000000000006,"15min":1.196},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.64500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18012,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":16896,"kb_avail":94353828,"statfs":{"total":96636764160,"available":96618319872,"internally_reserved":0,"allocated":1134592,"data_stored":578595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17301504},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.5,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.312},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.375,"15min":0.376},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.434,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.32200000000000001}]},{"osd":3,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.64700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.432,"15min":0.439},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60899999999999999,"5min":0.71399999999999997,"15min":0.72899999999999998},"last":0.27600000000000002}]},{"osd":4,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.274},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82599999999999996,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.50900000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.37},"max":{"1min":0.872,"5min":0.88100000000000001,"15min":1.5609999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.47},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88200000000000001,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.54500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575033,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20944,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94350896,"statfs":{"total":96636764160,"available":96615317504,"internally_reserved":0,"allocated":1122304,"data_stored":577059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.29799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.40899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42199999999999999,"15min":0.41499999999999998},"min":{"1min":0.33100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.25},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.33800000000000002,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.626,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.374}]},{"osd":4,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77500000000000002,"5min":0.93100000000000005,"15min":1.0489999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83799999999999997,"5min":0.93799999999999994,"15min":1.0409999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.504,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.97399999999999998,"15min":1.0780000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":1.0049999999999999},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.44,"15min":0.40100000000000002},"max":{"1min":0.75800000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":1.02},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17016,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":15936,"kb_avail":94354824,"statfs":{"total":96636764160,"available":96619339776,"internally_reserved":0,"allocated":1097728,"data_stored":562033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16318464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.82099999999999995,"15min":0.83199999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.81200000000000006},"last":0.439}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.627,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.67500000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.495},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.90500000000000003,"15min":0.999},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.97999999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.629,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84099999999999997,"5min":0.875,"15min":0.93100000000000005},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":1.0640000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.31900000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70699999999999996,"5min":0.79200000000000004,"15min":0.94599999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.86199999999999999,"15min":1.2689999999999999},"last":0.33800000000000002}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":1.48,"15min":1.48},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":1.046},"last":0.44700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20344,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":19264,"kb_avail":94351496,"statfs":{"total":96636764160,"available":96615931904,"internally_reserved":0,"allocated":1097728,"data_stored":562033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19726336},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.36599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81599999999999995,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.81000000000000005,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":0.96099999999999997,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86399999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.54300000000000004}]},{"osd":2,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71399999999999997,"5min":2.278,"15min":2.278},"last":0.503},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.45100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.748,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.33900000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":1.333,"15min":1.333},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.309},"max":{"1min":0.84899999999999998,"5min":2.081,"15min":2.081},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":1.024,"15min":1.024},"last":0.44900000000000001}]},{"osd":7,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.878,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.432},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.86599999999999999,"15min":0.878},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:17:51.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:17:52.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.103+0000 7f0f99ded700 1 Processor -- start 2023-05-16T07:17:52.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.103+0000 7f0f99ded700 1 -- start start 2023-05-16T07:17:52.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f99ded700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f94154550 0x7f0f940696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f99ded700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f940c0660 con 0x7f0f94154170 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f99ded700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f94069fc0 con 0x7f0f94069b90 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f99ded700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940bfd30 con 0x7f0f94154550 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f92ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f94154550 0x7f0f940696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f92ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f94154550 0x7f0f940696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46540/0 (socket says 172.21.15.92:46540) 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f937fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0f94154170 0x7f0f940c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57286/0 (socket says 172.21.15.92:57286) 2023-05-16T07:17:52.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f92ffd700 1 -- 172.21.15.92:0/3589094461 learned_addr learned my addr 172.21.15.92:0/3589094461 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f92ffd700 1 -- 172.21.15.92:0/3589094461 >> v1:172.21.15.158:6789/0 conn(0x7f0f94069b90 legacy=0x7f0f940c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f92ffd700 1 -- 172.21.15.92:0/3589094461 >> v1:172.21.15.92:6789/0 conn(0x7f0f94154170 legacy=0x7f0f940c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f92ffd700 1 -- 172.21.15.92:0/3589094461 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940c0010 con 0x7f0f94154550 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.104+0000 7f0f927fc700 1 -- 172.21.15.92:0/3589094461 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311383219 0 0) 0x7f0f940c0660 con 0x7f0f94154170 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f92ffd700 1 --2- 172.21.15.92:0/3589094461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f940696a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f0f88006fd0 tx=0x7f0f880090f0).ready entity=mon.0 client_cookie=329485191f0afb68 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f927fc700 1 -- 172.21.15.92:0/3589094461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800b600 con 0x7f0f94154550 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f927fc700 1 -- 172.21.15.92:0/3589094461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8800b760 con 0x7f0f94154550 2023-05-16T07:17:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f927fc700 1 -- 172.21.15.92:0/3589094461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800bb00 con 0x7f0f94154550 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 -- 172.21.15.92:0/3589094461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 msgr2=0x7f0f940696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3589094461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f940696a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f0f88006fd0 tx=0x7f0f880090f0).stop 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 -- 172.21.15.92:0/3589094461 shutdown_connections 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3589094461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f940696a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 -- 172.21.15.92:0/3589094461 >> 172.21.15.92:0/3589094461 conn(0x7f0f940cb700 msgr2=0x7f0f940c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 -- 172.21.15.92:0/3589094461 shutdown_connections 2023-05-16T07:17:52.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.105+0000 7f0f99ded700 1 -- 172.21.15.92:0/3589094461 wait complete. 2023-05-16T07:17:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.106+0000 7f0f99ded700 1 Processor -- start 2023-05-16T07:17:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.106+0000 7f0f99ded700 1 -- start start 2023-05-16T07:17:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f99ded700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f94069b90 0x7f0f940c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f99ded700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f94154170 0x7f0f94155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f99ded700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f94164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f99ded700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94165560 con 0x7f0f94154550 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f99ded700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940c44b0 con 0x7f0f94154170 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f99ded700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94164fa0 con 0x7f0f94069b90 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f94164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f94164530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46548/0 (socket says 172.21.15.92:46548) 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f93fff700 1 -- 172.21.15.92:0/3371319193 learned_addr learned my addr 172.21.15.92:0/3371319193 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:52.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.107+0000 7f0f92ffd700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f94154170 0x7f0f94155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f937fe700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f94069b90 0x7f0f940c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f93fff700 1 -- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f94069b90 msgr2=0x7f0f940c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f93fff700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f94069b90 0x7f0f940c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f93fff700 1 -- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f94154170 msgr2=0x7f0f94155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f93fff700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f94154170 0x7f0f94155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f93fff700 1 -- 172.21.15.92:0/3371319193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009370 con 0x7f0f94154550 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f93fff700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f94164530 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f0f8c00c440 tx=0x7f0f8c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00f470 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c001e90 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f94165280 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00f7e0 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.108+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f940c41f0 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.109+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f940c2c80 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.109+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0f8c010070 con 0x7f0f94154550 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.109+0000 7f0f7b7fe700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f8005ed90 0x7f0f80061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.110+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f0f8c0790d0 con 0x7f0f94154550 2023-05-16T07:17:52.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.110+0000 7f0f937fe700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f8005ed90 0x7f0f80061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.110+0000 7f0f937fe700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f8005ed90 0x7f0f80061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0f84007fd0 tx=0x7f0f84007dd0).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.113+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0f8c0478f0 con 0x7f0f94154550 2023-05-16T07:17:52.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.271+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f940666c0 con 0x7f0f8005ed90 2023-05-16T07:17:52.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.282+0000 7f0f7b7fe700 1 -- 172.21.15.92:0/3371319193 <== mgr.6019 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361285 (secure 0 0 0) 0x7f0f940666c0 con 0x7f0f8005ed90 2023-05-16T07:17:52.290 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:52.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f8005ed90 msgr2=0x7f0f80061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f8005ed90 0x7f0f80061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0f84007fd0 tx=0x7f0f84007dd0).stop 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 msgr2=0x7f0f94164530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f94164530 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f0f8c00c440 tx=0x7f0f8c007d70).stop 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 shutdown_connections 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0f8005ed90 0x7f0f80061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0f94069b90 0x7f0f940c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f94154550 0x7f0f94164530 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 --2- 172.21.15.92:0/3371319193 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0f94154170 0x7f0f94155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 >> 172.21.15.92:0/3371319193 conn(0x7f0f940cb700 msgr2=0x7f0f9414d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:52.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 shutdown_connections 2023-05-16T07:17:52.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.285+0000 7f0f99ded700 1 -- 172.21.15.92:0/3371319193 wait complete. 2023-05-16T07:17:52.295 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:17:52.301 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-05-16T07:17:51.783931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":148084,"kb_used_data":8692,"kb_used_omap":0,"kb_used_meta":139328,"kb_avail":754826636,"statfs":{"total":773094113280,"available":772942475264,"internally_reserved":0,"allocated":8900608,"data_stored":4557980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":142671872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10436,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-61,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.601392"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369831+0000","last_change":"2023-05-16T07:17:48.369831+0000","last_active":"2023-05-16T07:17:48.369831+0000","last_peered":"2023-05-16T07:17:48.369831+0000","last_clean":"2023-05-16T07:17:48.369831+0000","last_became_active":"2023-05-16T07:17:48.369377+0000","last_became_peered":"2023-05-16T07:17:48.369377+0000","last_unstale":"2023-05-16T07:17:48.369831+0000","last_undegraded":"2023-05-16T07:17:48.369831+0000","last_fullsized":"2023-05-16T07:17:48.369831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"92'3","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362034+0000","last_change":"2023-05-16T07:17:47.989734+0000","last_active":"2023-05-16T07:17:48.362034+0000","last_peered":"2023-05-16T07:17:48.362034+0000","last_clean":"2023-05-16T07:17:48.362034+0000","last_became_active":"2023-05-16T07:17:47.988957+0000","last_became_peered":"2023-05-16T07:17:47.988957+0000","last_unstale":"2023-05-16T07:17:48.362034+0000","last_undegraded":"2023-05-16T07:17:48.362034+0000","last_fullsized":"2023-05-16T07:17:48.362034+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356259+0000","last_change":"2023-05-16T07:17:48.356259+0000","last_active":"2023-05-16T07:17:48.356259+0000","last_peered":"2023-05-16T07:17:48.356259+0000","last_clean":"2023-05-16T07:17:48.356259+0000","last_became_active":"2023-05-16T07:17:48.355996+0000","last_became_peered":"2023-05-16T07:17:48.355996+0000","last_unstale":"2023-05-16T07:17:48.356259+0000","last_undegraded":"2023-05-16T07:17:48.356259+0000","last_fullsized":"2023-05-16T07:17:48.356259+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999272+0000","last_change":"2023-05-16T07:17:47.995555+0000","last_active":"2023-05-16T07:17:47.999272+0000","last_peered":"2023-05-16T07:17:47.999272+0000","last_clean":"2023-05-16T07:17:47.999272+0000","last_became_active":"2023-05-16T07:17:47.994097+0000","last_became_peered":"2023-05-16T07:17:47.994097+0000","last_unstale":"2023-05-16T07:17:47.999272+0000","last_undegraded":"2023-05-16T07:17:47.999272+0000","last_fullsized":"2023-05-16T07:17:47.999272+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356980+0000","last_change":"2023-05-16T07:17:48.356980+0000","last_active":"2023-05-16T07:17:48.356980+0000","last_peered":"2023-05-16T07:17:48.356980+0000","last_clean":"2023-05-16T07:17:48.356980+0000","last_became_active":"2023-05-16T07:17:48.356667+0000","last_became_peered":"2023-05-16T07:17:48.356667+0000","last_unstale":"2023-05-16T07:17:48.356980+0000","last_undegraded":"2023-05-16T07:17:48.356980+0000","last_fullsized":"2023-05-16T07:17:48.356980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999080+0000","last_change":"2023-05-16T07:17:47.994556+0000","last_active":"2023-05-16T07:17:47.999080+0000","last_peered":"2023-05-16T07:17:47.999080+0000","last_clean":"2023-05-16T07:17:47.999080+0000","last_became_active":"2023-05-16T07:17:47.994290+0000","last_became_peered":"2023-05-16T07:17:47.994290+0000","last_unstale":"2023-05-16T07:17:47.999080+0000","last_undegraded":"2023-05-16T07:17:47.999080+0000","last_fullsized":"2023-05-16T07:17:47.999080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369137+0000","last_change":"2023-05-16T07:17:48.369137+0000","last_active":"2023-05-16T07:17:48.369137+0000","last_peered":"2023-05-16T07:17:48.369137+0000","last_clean":"2023-05-16T07:17:48.369137+0000","last_became_active":"2023-05-16T07:17:48.365347+0000","last_became_peered":"2023-05-16T07:17:48.365347+0000","last_unstale":"2023-05-16T07:17:48.369137+0000","last_undegraded":"2023-05-16T07:17:48.369137+0000","last_fullsized":"2023-05-16T07:17:48.369137+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363169+0000","last_change":"2023-05-16T07:17:47.997705+0000","last_active":"2023-05-16T07:17:48.363169+0000","last_peered":"2023-05-16T07:17:48.363169+0000","last_clean":"2023-05-16T07:17:48.363169+0000","last_became_active":"2023-05-16T07:17:47.997370+0000","last_became_peered":"2023-05-16T07:17:47.997370+0000","last_unstale":"2023-05-16T07:17:48.363169+0000","last_undegraded":"2023-05-16T07:17:48.363169+0000","last_fullsized":"2023-05-16T07:17:48.363169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361521+0000","last_change":"2023-05-16T07:17:48.361521+0000","last_active":"2023-05-16T07:17:48.361521+0000","last_peered":"2023-05-16T07:17:48.361521+0000","last_clean":"2023-05-16T07:17:48.361521+0000","last_became_active":"2023-05-16T07:17:48.361246+0000","last_became_peered":"2023-05-16T07:17:48.361246+0000","last_unstale":"2023-05-16T07:17:48.361521+0000","last_undegraded":"2023-05-16T07:17:48.361521+0000","last_fullsized":"2023-05-16T07:17:48.361521+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3b","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.354191+0000","last_change":"2023-05-16T07:17:47.991798+0000","last_active":"2023-05-16T07:17:48.354191+0000","last_peered":"2023-05-16T07:17:48.354191+0000","last_clean":"2023-05-16T07:17:48.354191+0000","last_became_active":"2023-05-16T07:17:47.990666+0000","last_became_peered":"2023-05-16T07:17:47.990666+0000","last_unstale":"2023-05-16T07:17:48.354191+0000","last_undegraded":"2023-05-16T07:17:48.354191+0000","last_fullsized":"2023-05-16T07:17:48.354191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.374222+0000","last_change":"2023-05-16T07:17:48.374222+0000","last_active":"2023-05-16T07:17:48.374222+0000","last_peered":"2023-05-16T07:17:48.374222+0000","last_clean":"2023-05-16T07:17:48.374222+0000","last_became_active":"2023-05-16T07:17:48.373923+0000","last_became_peered":"2023-05-16T07:17:48.373923+0000","last_unstale":"2023-05-16T07:17:48.374222+0000","last_undegraded":"2023-05-16T07:17:48.374222+0000","last_fullsized":"2023-05-16T07:17:48.374222+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.342394+0000","last_change":"2023-05-16T07:17:47.996336+0000","last_active":"2023-05-16T07:17:48.342394+0000","last_peered":"2023-05-16T07:17:48.342394+0000","last_clean":"2023-05-16T07:17:48.342394+0000","last_became_active":"2023-05-16T07:17:47.994144+0000","last_became_peered":"2023-05-16T07:17:47.994144+0000","last_unstale":"2023-05-16T07:17:48.342394+0000","last_undegraded":"2023-05-16T07:17:48.342394+0000","last_fullsized":"2023-05-16T07:17:48.342394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366242+0000","last_change":"2023-05-16T07:17:48.366242+0000","last_active":"2023-05-16T07:17:48.366242+0000","last_peered":"2023-05-16T07:17:48.366242+0000","last_clean":"2023-05-16T07:17:48.366242+0000","last_became_active":"2023-05-16T07:17:48.365793+0000","last_became_peered":"2023-05-16T07:17:48.365793+0000","last_unstale":"2023-05-16T07:17:48.366242+0000","last_undegraded":"2023-05-16T07:17:48.366242+0000","last_fullsized":"2023-05-16T07:17:48.366242+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004103+0000","last_change":"2023-05-16T07:17:48.002774+0000","last_active":"2023-05-16T07:17:48.004103+0000","last_peered":"2023-05-16T07:17:48.004103+0000","last_clean":"2023-05-16T07:17:48.004103+0000","last_became_active":"2023-05-16T07:17:48.002495+0000","last_became_peered":"2023-05-16T07:17:48.002495+0000","last_unstale":"2023-05-16T07:17:48.004103+0000","last_undegraded":"2023-05-16T07:17:48.004103+0000","last_fullsized":"2023-05-16T07:17:48.004103+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.353979+0000","last_change":"2023-05-16T07:17:48.353979+0000","last_active":"2023-05-16T07:17:48.353979+0000","last_peered":"2023-05-16T07:17:48.353979+0000","last_clean":"2023-05-16T07:17:48.353979+0000","last_became_active":"2023-05-16T07:17:48.353702+0000","last_became_peered":"2023-05-16T07:17:48.353702+0000","last_unstale":"2023-05-16T07:17:48.353979+0000","last_undegraded":"2023-05-16T07:17:48.353979+0000","last_fullsized":"2023-05-16T07:17:48.353979+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.001977+0000","last_change":"2023-05-16T07:17:47.999945+0000","last_active":"2023-05-16T07:17:48.001977+0000","last_peered":"2023-05-16T07:17:48.001977+0000","last_clean":"2023-05-16T07:17:48.001977+0000","last_became_active":"2023-05-16T07:17:47.999519+0000","last_became_peered":"2023-05-16T07:17:47.999519+0000","last_unstale":"2023-05-16T07:17:48.001977+0000","last_undegraded":"2023-05-16T07:17:48.001977+0000","last_fullsized":"2023-05-16T07:17:48.001977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360887+0000","last_change":"2023-05-16T07:17:48.360887+0000","last_active":"2023-05-16T07:17:48.360887+0000","last_peered":"2023-05-16T07:17:48.360887+0000","last_clean":"2023-05-16T07:17:48.360887+0000","last_became_active":"2023-05-16T07:17:48.360602+0000","last_became_peered":"2023-05-16T07:17:48.360602+0000","last_unstale":"2023-05-16T07:17:48.360887+0000","last_undegraded":"2023-05-16T07:17:48.360887+0000","last_fullsized":"2023-05-16T07:17:48.360887+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992700+0000","last_change":"2023-05-16T07:17:47.986607+0000","last_active":"2023-05-16T07:17:47.992700+0000","last_peered":"2023-05-16T07:17:47.992700+0000","last_clean":"2023-05-16T07:17:47.992700+0000","last_became_active":"2023-05-16T07:17:47.986302+0000","last_became_peered":"2023-05-16T07:17:47.986302+0000","last_unstale":"2023-05-16T07:17:47.992700+0000","last_undegraded":"2023-05-16T07:17:47.992700+0000","last_fullsized":"2023-05-16T07:17:47.992700+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370115+0000","last_change":"2023-05-16T07:17:48.370115+0000","last_active":"2023-05-16T07:17:48.370115+0000","last_peered":"2023-05-16T07:17:48.370115+0000","last_clean":"2023-05-16T07:17:48.370115+0000","last_became_active":"2023-05-16T07:17:48.366731+0000","last_became_peered":"2023-05-16T07:17:48.366731+0000","last_unstale":"2023-05-16T07:17:48.370115+0000","last_undegraded":"2023-05-16T07:17:48.370115+0000","last_fullsized":"2023-05-16T07:17:48.370115+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999757+0000","last_change":"2023-05-16T07:17:47.991747+0000","last_active":"2023-05-16T07:17:47.999757+0000","last_peered":"2023-05-16T07:17:47.999757+0000","last_clean":"2023-05-16T07:17:47.999757+0000","last_became_active":"2023-05-16T07:17:47.991461+0000","last_became_peered":"2023-05-16T07:17:47.991461+0000","last_unstale":"2023-05-16T07:17:47.999757+0000","last_undegraded":"2023-05-16T07:17:47.999757+0000","last_fullsized":"2023-05-16T07:17:47.999757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.357059+0000","last_change":"2023-05-16T07:17:48.357059+0000","last_active":"2023-05-16T07:17:48.357059+0000","last_peered":"2023-05-16T07:17:48.357059+0000","last_clean":"2023-05-16T07:17:48.357059+0000","last_became_active":"2023-05-16T07:17:48.356598+0000","last_became_peered":"2023-05-16T07:17:48.356598+0000","last_unstale":"2023-05-16T07:17:48.357059+0000","last_undegraded":"2023-05-16T07:17:48.357059+0000","last_fullsized":"2023-05-16T07:17:48.357059+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000493+0000","last_change":"2023-05-16T07:17:47.994026+0000","last_active":"2023-05-16T07:17:48.000493+0000","last_peered":"2023-05-16T07:17:48.000493+0000","last_clean":"2023-05-16T07:17:48.000493+0000","last_became_active":"2023-05-16T07:17:47.993403+0000","last_became_peered":"2023-05-16T07:17:47.993403+0000","last_unstale":"2023-05-16T07:17:48.000493+0000","last_undegraded":"2023-05-16T07:17:48.000493+0000","last_fullsized":"2023-05-16T07:17:48.000493+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369132+0000","last_change":"2023-05-16T07:17:48.369132+0000","last_active":"2023-05-16T07:17:48.369132+0000","last_peered":"2023-05-16T07:17:48.369132+0000","last_clean":"2023-05-16T07:17:48.369132+0000","last_became_active":"2023-05-16T07:17:48.365980+0000","last_became_peered":"2023-05-16T07:17:48.365980+0000","last_unstale":"2023-05-16T07:17:48.369132+0000","last_undegraded":"2023-05-16T07:17:48.369132+0000","last_fullsized":"2023-05-16T07:17:48.369132+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.34","version":"92'4","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.374458+0000","last_change":"2023-05-16T07:17:48.000143+0000","last_active":"2023-05-16T07:17:48.374458+0000","last_peered":"2023-05-16T07:17:48.374458+0000","last_clean":"2023-05-16T07:17:48.374458+0000","last_became_active":"2023-05-16T07:17:47.999123+0000","last_became_peered":"2023-05-16T07:17:47.999123+0000","last_unstale":"2023-05-16T07:17:48.374458+0000","last_undegraded":"2023-05-16T07:17:48.374458+0000","last_fullsized":"2023-05-16T07:17:48.374458+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.373339+0000","last_change":"2023-05-16T07:17:48.373339+0000","last_active":"2023-05-16T07:17:48.373339+0000","last_peered":"2023-05-16T07:17:48.373339+0000","last_clean":"2023-05-16T07:17:48.373339+0000","last_became_active":"2023-05-16T07:17:48.372991+0000","last_became_peered":"2023-05-16T07:17:48.372991+0000","last_unstale":"2023-05-16T07:17:48.373339+0000","last_undegraded":"2023-05-16T07:17:48.373339+0000","last_fullsized":"2023-05-16T07:17:48.373339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.349344+0000","last_change":"2023-05-16T07:17:47.990064+0000","last_active":"2023-05-16T07:17:48.349344+0000","last_peered":"2023-05-16T07:17:48.349344+0000","last_clean":"2023-05-16T07:17:48.349344+0000","last_became_active":"2023-05-16T07:17:47.989173+0000","last_became_peered":"2023-05-16T07:17:47.989173+0000","last_unstale":"2023-05-16T07:17:48.349344+0000","last_undegraded":"2023-05-16T07:17:48.349344+0000","last_fullsized":"2023-05-16T07:17:48.349344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.374587+0000","last_change":"2023-05-16T07:17:48.374587+0000","last_active":"2023-05-16T07:17:48.374587+0000","last_peered":"2023-05-16T07:17:48.374587+0000","last_clean":"2023-05-16T07:17:48.374587+0000","last_became_active":"2023-05-16T07:17:48.374320+0000","last_became_peered":"2023-05-16T07:17:48.374320+0000","last_unstale":"2023-05-16T07:17:48.374587+0000","last_undegraded":"2023-05-16T07:17:48.374587+0000","last_fullsized":"2023-05-16T07:17:48.374587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004935+0000","last_change":"2023-05-16T07:17:48.001873+0000","last_active":"2023-05-16T07:17:48.004935+0000","last_peered":"2023-05-16T07:17:48.004935+0000","last_clean":"2023-05-16T07:17:48.004935+0000","last_became_active":"2023-05-16T07:17:48.001576+0000","last_became_peered":"2023-05-16T07:17:48.001576+0000","last_unstale":"2023-05-16T07:17:48.004935+0000","last_undegraded":"2023-05-16T07:17:48.004935+0000","last_fullsized":"2023-05-16T07:17:48.004935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359447+0000","last_change":"2023-05-16T07:17:48.359447+0000","last_active":"2023-05-16T07:17:48.359447+0000","last_peered":"2023-05-16T07:17:48.359447+0000","last_clean":"2023-05-16T07:17:48.359447+0000","last_became_active":"2023-05-16T07:17:48.359170+0000","last_became_peered":"2023-05-16T07:17:48.359170+0000","last_unstale":"2023-05-16T07:17:48.359447+0000","last_undegraded":"2023-05-16T07:17:48.359447+0000","last_fullsized":"2023-05-16T07:17:48.359447+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.997113+0000","last_change":"2023-05-16T07:17:47.995682+0000","last_active":"2023-05-16T07:17:47.997113+0000","last_peered":"2023-05-16T07:17:47.997113+0000","last_clean":"2023-05-16T07:17:47.997113+0000","last_became_active":"2023-05-16T07:17:47.995394+0000","last_became_peered":"2023-05-16T07:17:47.995394+0000","last_unstale":"2023-05-16T07:17:47.997113+0000","last_undegraded":"2023-05-16T07:17:47.997113+0000","last_fullsized":"2023-05-16T07:17:47.997113+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.357536+0000","last_change":"2023-05-16T07:17:48.357536+0000","last_active":"2023-05-16T07:17:48.357536+0000","last_peered":"2023-05-16T07:17:48.357536+0000","last_clean":"2023-05-16T07:17:48.357536+0000","last_became_active":"2023-05-16T07:17:48.357300+0000","last_became_peered":"2023-05-16T07:17:48.357300+0000","last_unstale":"2023-05-16T07:17:48.357536+0000","last_undegraded":"2023-05-16T07:17:48.357536+0000","last_fullsized":"2023-05-16T07:17:48.357536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"92'4","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360547+0000","last_change":"2023-05-16T07:17:47.994055+0000","last_active":"2023-05-16T07:17:48.360547+0000","last_peered":"2023-05-16T07:17:48.360547+0000","last_clean":"2023-05-16T07:17:48.360547+0000","last_became_active":"2023-05-16T07:17:47.993775+0000","last_became_peered":"2023-05-16T07:17:47.993775+0000","last_unstale":"2023-05-16T07:17:48.360547+0000","last_undegraded":"2023-05-16T07:17:48.360547+0000","last_fullsized":"2023-05-16T07:17:48.360547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370894+0000","last_change":"2023-05-16T07:17:48.370894+0000","last_active":"2023-05-16T07:17:48.370894+0000","last_peered":"2023-05-16T07:17:48.370894+0000","last_clean":"2023-05-16T07:17:48.370894+0000","last_became_active":"2023-05-16T07:17:48.370659+0000","last_became_peered":"2023-05-16T07:17:48.370659+0000","last_unstale":"2023-05-16T07:17:48.370894+0000","last_undegraded":"2023-05-16T07:17:48.370894+0000","last_fullsized":"2023-05-16T07:17:48.370894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992659+0000","last_change":"2023-05-16T07:17:47.986437+0000","last_active":"2023-05-16T07:17:47.992659+0000","last_peered":"2023-05-16T07:17:47.992659+0000","last_clean":"2023-05-16T07:17:47.992659+0000","last_became_active":"2023-05-16T07:17:47.985976+0000","last_became_peered":"2023-05-16T07:17:47.985976+0000","last_unstale":"2023-05-16T07:17:47.992659+0000","last_undegraded":"2023-05-16T07:17:47.992659+0000","last_fullsized":"2023-05-16T07:17:47.992659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361142+0000","last_change":"2023-05-16T07:17:48.361142+0000","last_active":"2023-05-16T07:17:48.361142+0000","last_peered":"2023-05-16T07:17:48.361142+0000","last_clean":"2023-05-16T07:17:48.361142+0000","last_became_active":"2023-05-16T07:17:48.360856+0000","last_became_peered":"2023-05-16T07:17:48.360856+0000","last_unstale":"2023-05-16T07:17:48.361142+0000","last_undegraded":"2023-05-16T07:17:48.361142+0000","last_fullsized":"2023-05-16T07:17:48.361142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003301+0000","last_change":"2023-05-16T07:17:47.998342+0000","last_active":"2023-05-16T07:17:48.003301+0000","last_peered":"2023-05-16T07:17:48.003301+0000","last_clean":"2023-05-16T07:17:48.003301+0000","last_became_active":"2023-05-16T07:17:47.997982+0000","last_became_peered":"2023-05-16T07:17:47.997982+0000","last_unstale":"2023-05-16T07:17:48.003301+0000","last_undegraded":"2023-05-16T07:17:48.003301+0000","last_fullsized":"2023-05-16T07:17:48.003301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365772+0000","last_change":"2023-05-16T07:17:48.365772+0000","last_active":"2023-05-16T07:17:48.365772+0000","last_peered":"2023-05-16T07:17:48.365772+0000","last_clean":"2023-05-16T07:17:48.365772+0000","last_became_active":"2023-05-16T07:17:48.365529+0000","last_became_peered":"2023-05-16T07:17:48.365529+0000","last_unstale":"2023-05-16T07:17:48.365772+0000","last_undegraded":"2023-05-16T07:17:48.365772+0000","last_fullsized":"2023-05-16T07:17:48.365772+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2d","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.342923+0000","last_change":"2023-05-16T07:17:47.999853+0000","last_active":"2023-05-16T07:17:48.342923+0000","last_peered":"2023-05-16T07:17:48.342923+0000","last_clean":"2023-05-16T07:17:48.342923+0000","last_became_active":"2023-05-16T07:17:47.999547+0000","last_became_peered":"2023-05-16T07:17:47.999547+0000","last_unstale":"2023-05-16T07:17:48.342923+0000","last_undegraded":"2023-05-16T07:17:48.342923+0000","last_fullsized":"2023-05-16T07:17:48.342923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364708+0000","last_change":"2023-05-16T07:17:48.364708+0000","last_active":"2023-05-16T07:17:48.364708+0000","last_peered":"2023-05-16T07:17:48.364708+0000","last_clean":"2023-05-16T07:17:48.364708+0000","last_became_active":"2023-05-16T07:17:48.364421+0000","last_became_peered":"2023-05-16T07:17:48.364421+0000","last_unstale":"2023-05-16T07:17:48.364708+0000","last_undegraded":"2023-05-16T07:17:48.364708+0000","last_fullsized":"2023-05-16T07:17:48.364708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004983+0000","last_change":"2023-05-16T07:17:48.000706+0000","last_active":"2023-05-16T07:17:48.004983+0000","last_peered":"2023-05-16T07:17:48.004983+0000","last_clean":"2023-05-16T07:17:48.004983+0000","last_became_active":"2023-05-16T07:17:48.000496+0000","last_became_peered":"2023-05-16T07:17:48.000496+0000","last_unstale":"2023-05-16T07:17:48.004983+0000","last_undegraded":"2023-05-16T07:17:48.004983+0000","last_fullsized":"2023-05-16T07:17:48.004983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365532+0000","last_change":"2023-05-16T07:17:48.365532+0000","last_active":"2023-05-16T07:17:48.365532+0000","last_peered":"2023-05-16T07:17:48.365532+0000","last_clean":"2023-05-16T07:17:48.365532+0000","last_became_active":"2023-05-16T07:17:48.365217+0000","last_became_peered":"2023-05-16T07:17:48.365217+0000","last_unstale":"2023-05-16T07:17:48.365532+0000","last_undegraded":"2023-05-16T07:17:48.365532+0000","last_fullsized":"2023-05-16T07:17:48.365532+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.350514+0000","last_change":"2023-05-16T07:17:47.995065+0000","last_active":"2023-05-16T07:17:48.350514+0000","last_peered":"2023-05-16T07:17:48.350514+0000","last_clean":"2023-05-16T07:17:48.350514+0000","last_became_active":"2023-05-16T07:17:47.994756+0000","last_became_peered":"2023-05-16T07:17:47.994756+0000","last_unstale":"2023-05-16T07:17:48.350514+0000","last_undegraded":"2023-05-16T07:17:48.350514+0000","last_fullsized":"2023-05-16T07:17:48.350514+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360987+0000","last_change":"2023-05-16T07:17:48.360987+0000","last_active":"2023-05-16T07:17:48.360987+0000","last_peered":"2023-05-16T07:17:48.360987+0000","last_clean":"2023-05-16T07:17:48.360987+0000","last_became_active":"2023-05-16T07:17:48.360819+0000","last_became_peered":"2023-05-16T07:17:48.360819+0000","last_unstale":"2023-05-16T07:17:48.360987+0000","last_undegraded":"2023-05-16T07:17:48.360987+0000","last_fullsized":"2023-05-16T07:17:48.360987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000719+0000","last_change":"2023-05-16T07:17:47.995552+0000","last_active":"2023-05-16T07:17:48.000719+0000","last_peered":"2023-05-16T07:17:48.000719+0000","last_clean":"2023-05-16T07:17:48.000719+0000","last_became_active":"2023-05-16T07:17:47.995251+0000","last_became_peered":"2023-05-16T07:17:47.995251+0000","last_unstale":"2023-05-16T07:17:48.000719+0000","last_undegraded":"2023-05-16T07:17:48.000719+0000","last_fullsized":"2023-05-16T07:17:48.000719+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367914+0000","last_change":"2023-05-16T07:17:48.367914+0000","last_active":"2023-05-16T07:17:48.367914+0000","last_peered":"2023-05-16T07:17:48.367914+0000","last_clean":"2023-05-16T07:17:48.367914+0000","last_became_active":"2023-05-16T07:17:48.367714+0000","last_became_peered":"2023-05-16T07:17:48.367714+0000","last_unstale":"2023-05-16T07:17:48.367914+0000","last_undegraded":"2023-05-16T07:17:48.367914+0000","last_fullsized":"2023-05-16T07:17:48.367914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003169+0000","last_change":"2023-05-16T07:17:47.998143+0000","last_active":"2023-05-16T07:17:48.003169+0000","last_peered":"2023-05-16T07:17:48.003169+0000","last_clean":"2023-05-16T07:17:48.003169+0000","last_became_active":"2023-05-16T07:17:47.997849+0000","last_became_peered":"2023-05-16T07:17:47.997849+0000","last_unstale":"2023-05-16T07:17:48.003169+0000","last_undegraded":"2023-05-16T07:17:48.003169+0000","last_fullsized":"2023-05-16T07:17:48.003169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365356+0000","last_change":"2023-05-16T07:17:48.365356+0000","last_active":"2023-05-16T07:17:48.365356+0000","last_peered":"2023-05-16T07:17:48.365356+0000","last_clean":"2023-05-16T07:17:48.365356+0000","last_became_active":"2023-05-16T07:17:48.365075+0000","last_became_peered":"2023-05-16T07:17:48.365075+0000","last_unstale":"2023-05-16T07:17:48.365356+0000","last_undegraded":"2023-05-16T07:17:48.365356+0000","last_fullsized":"2023-05-16T07:17:48.365356+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002985+0000","last_change":"2023-05-16T07:17:47.997193+0000","last_active":"2023-05-16T07:17:48.002985+0000","last_peered":"2023-05-16T07:17:48.002985+0000","last_clean":"2023-05-16T07:17:48.002985+0000","last_became_active":"2023-05-16T07:17:47.996908+0000","last_became_peered":"2023-05-16T07:17:47.996908+0000","last_unstale":"2023-05-16T07:17:48.002985+0000","last_undegraded":"2023-05-16T07:17:48.002985+0000","last_fullsized":"2023-05-16T07:17:48.002985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.355450+0000","last_change":"2023-05-16T07:17:48.355450+0000","last_active":"2023-05-16T07:17:48.355450+0000","last_peered":"2023-05-16T07:17:48.355450+0000","last_clean":"2023-05-16T07:17:48.355450+0000","last_became_active":"2023-05-16T07:17:48.355254+0000","last_became_peered":"2023-05-16T07:17:48.355254+0000","last_unstale":"2023-05-16T07:17:48.355450+0000","last_undegraded":"2023-05-16T07:17:48.355450+0000","last_fullsized":"2023-05-16T07:17:48.355450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000797+0000","last_change":"2023-05-16T07:17:47.994674+0000","last_active":"2023-05-16T07:17:48.000797+0000","last_peered":"2023-05-16T07:17:48.000797+0000","last_clean":"2023-05-16T07:17:48.000797+0000","last_became_active":"2023-05-16T07:17:47.992629+0000","last_became_peered":"2023-05-16T07:17:47.992629+0000","last_unstale":"2023-05-16T07:17:48.000797+0000","last_undegraded":"2023-05-16T07:17:48.000797+0000","last_fullsized":"2023-05-16T07:17:48.000797+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.358186+0000","last_change":"2023-05-16T07:17:48.358186+0000","last_active":"2023-05-16T07:17:48.358186+0000","last_peered":"2023-05-16T07:17:48.358186+0000","last_clean":"2023-05-16T07:17:48.358186+0000","last_became_active":"2023-05-16T07:17:48.357966+0000","last_became_peered":"2023-05-16T07:17:48.357966+0000","last_unstale":"2023-05-16T07:17:48.358186+0000","last_undegraded":"2023-05-16T07:17:48.358186+0000","last_fullsized":"2023-05-16T07:17:48.358186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003416+0000","last_change":"2023-05-16T07:17:48.002061+0000","last_active":"2023-05-16T07:17:48.003416+0000","last_peered":"2023-05-16T07:17:48.003416+0000","last_clean":"2023-05-16T07:17:48.003416+0000","last_became_active":"2023-05-16T07:17:48.001737+0000","last_became_peered":"2023-05-16T07:17:48.001737+0000","last_unstale":"2023-05-16T07:17:48.003416+0000","last_undegraded":"2023-05-16T07:17:48.003416+0000","last_fullsized":"2023-05-16T07:17:48.003416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366958+0000","last_change":"2023-05-16T07:17:48.366958+0000","last_active":"2023-05-16T07:17:48.366958+0000","last_peered":"2023-05-16T07:17:48.366958+0000","last_clean":"2023-05-16T07:17:48.366958+0000","last_became_active":"2023-05-16T07:17:48.366688+0000","last_became_peered":"2023-05-16T07:17:48.366688+0000","last_unstale":"2023-05-16T07:17:48.366958+0000","last_undegraded":"2023-05-16T07:17:48.366958+0000","last_fullsized":"2023-05-16T07:17:48.366958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.996841+0000","last_change":"2023-05-16T07:17:47.993000+0000","last_active":"2023-05-16T07:17:47.996841+0000","last_peered":"2023-05-16T07:17:47.996841+0000","last_clean":"2023-05-16T07:17:47.996841+0000","last_became_active":"2023-05-16T07:17:47.992750+0000","last_became_peered":"2023-05-16T07:17:47.992750+0000","last_unstale":"2023-05-16T07:17:47.996841+0000","last_undegraded":"2023-05-16T07:17:47.996841+0000","last_fullsized":"2023-05-16T07:17:47.996841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366152+0000","last_change":"2023-05-16T07:17:48.366152+0000","last_active":"2023-05-16T07:17:48.366152+0000","last_peered":"2023-05-16T07:17:48.366152+0000","last_clean":"2023-05-16T07:17:48.366152+0000","last_became_active":"2023-05-16T07:17:48.365878+0000","last_became_peered":"2023-05-16T07:17:48.365878+0000","last_unstale":"2023-05-16T07:17:48.366152+0000","last_undegraded":"2023-05-16T07:17:48.366152+0000","last_fullsized":"2023-05-16T07:17:48.366152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992800+0000","last_change":"2023-05-16T07:17:47.990784+0000","last_active":"2023-05-16T07:17:47.992800+0000","last_peered":"2023-05-16T07:17:47.992800+0000","last_clean":"2023-05-16T07:17:47.992800+0000","last_became_active":"2023-05-16T07:17:47.990448+0000","last_became_peered":"2023-05-16T07:17:47.990448+0000","last_unstale":"2023-05-16T07:17:47.992800+0000","last_undegraded":"2023-05-16T07:17:47.992800+0000","last_fullsized":"2023-05-16T07:17:47.992800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364916+0000","last_change":"2023-05-16T07:17:48.364916+0000","last_active":"2023-05-16T07:17:48.364916+0000","last_peered":"2023-05-16T07:17:48.364916+0000","last_clean":"2023-05-16T07:17:48.364916+0000","last_became_active":"2023-05-16T07:17:48.359947+0000","last_became_peered":"2023-05-16T07:17:48.359947+0000","last_unstale":"2023-05-16T07:17:48.364916+0000","last_undegraded":"2023-05-16T07:17:48.364916+0000","last_fullsized":"2023-05-16T07:17:48.364916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999155+0000","last_change":"2023-05-16T07:17:47.996275+0000","last_active":"2023-05-16T07:17:47.999155+0000","last_peered":"2023-05-16T07:17:47.999155+0000","last_clean":"2023-05-16T07:17:47.999155+0000","last_became_active":"2023-05-16T07:17:47.996043+0000","last_became_peered":"2023-05-16T07:17:47.996043+0000","last_unstale":"2023-05-16T07:17:47.999155+0000","last_undegraded":"2023-05-16T07:17:47.999155+0000","last_fullsized":"2023-05-16T07:17:47.999155+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359288+0000","last_change":"2023-05-16T07:17:48.359288+0000","last_active":"2023-05-16T07:17:48.359288+0000","last_peered":"2023-05-16T07:17:48.359288+0000","last_clean":"2023-05-16T07:17:48.359288+0000","last_became_active":"2023-05-16T07:17:48.359014+0000","last_became_peered":"2023-05-16T07:17:48.359014+0000","last_unstale":"2023-05-16T07:17:48.359288+0000","last_undegraded":"2023-05-16T07:17:48.359288+0000","last_fullsized":"2023-05-16T07:17:48.359288+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002785+0000","last_change":"2023-05-16T07:17:48.001724+0000","last_active":"2023-05-16T07:17:48.002785+0000","last_peered":"2023-05-16T07:17:48.002785+0000","last_clean":"2023-05-16T07:17:48.002785+0000","last_became_active":"2023-05-16T07:17:48.001379+0000","last_became_peered":"2023-05-16T07:17:48.001379+0000","last_unstale":"2023-05-16T07:17:48.002785+0000","last_undegraded":"2023-05-16T07:17:48.002785+0000","last_fullsized":"2023-05-16T07:17:48.002785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.373320+0000","last_change":"2023-05-16T07:17:48.373320+0000","last_active":"2023-05-16T07:17:48.373320+0000","last_peered":"2023-05-16T07:17:48.373320+0000","last_clean":"2023-05-16T07:17:48.373320+0000","last_became_active":"2023-05-16T07:17:48.373007+0000","last_became_peered":"2023-05-16T07:17:48.373007+0000","last_unstale":"2023-05-16T07:17:48.373320+0000","last_undegraded":"2023-05-16T07:17:48.373320+0000","last_fullsized":"2023-05-16T07:17:48.373320+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.004084+0000","last_change":"2023-05-16T07:17:48.002739+0000","last_active":"2023-05-16T07:17:48.004084+0000","last_peered":"2023-05-16T07:17:48.004084+0000","last_clean":"2023-05-16T07:17:48.004084+0000","last_became_active":"2023-05-16T07:17:48.002458+0000","last_became_peered":"2023-05-16T07:17:48.002458+0000","last_unstale":"2023-05-16T07:17:48.004084+0000","last_undegraded":"2023-05-16T07:17:48.004084+0000","last_fullsized":"2023-05-16T07:17:48.004084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370260+0000","last_change":"2023-05-16T07:17:48.370260+0000","last_active":"2023-05-16T07:17:48.370260+0000","last_peered":"2023-05-16T07:17:48.370260+0000","last_clean":"2023-05-16T07:17:48.370260+0000","last_became_active":"2023-05-16T07:17:48.369957+0000","last_became_peered":"2023-05-16T07:17:48.369957+0000","last_unstale":"2023-05-16T07:17:48.370260+0000","last_undegraded":"2023-05-16T07:17:48.370260+0000","last_fullsized":"2023-05-16T07:17:48.370260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992409+0000","last_change":"2023-05-16T07:17:47.986886+0000","last_active":"2023-05-16T07:17:47.992409+0000","last_peered":"2023-05-16T07:17:47.992409+0000","last_clean":"2023-05-16T07:17:47.992409+0000","last_became_active":"2023-05-16T07:17:47.986620+0000","last_became_peered":"2023-05-16T07:17:47.986620+0000","last_unstale":"2023-05-16T07:17:47.992409+0000","last_undegraded":"2023-05-16T07:17:47.992409+0000","last_fullsized":"2023-05-16T07:17:47.992409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367601+0000","last_change":"2023-05-16T07:17:48.367601+0000","last_active":"2023-05-16T07:17:48.367601+0000","last_peered":"2023-05-16T07:17:48.367601+0000","last_clean":"2023-05-16T07:17:48.367601+0000","last_became_active":"2023-05-16T07:17:48.367376+0000","last_became_peered":"2023-05-16T07:17:48.367376+0000","last_unstale":"2023-05-16T07:17:48.367601+0000","last_undegraded":"2023-05-16T07:17:48.367601+0000","last_fullsized":"2023-05-16T07:17:48.367601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356220+0000","last_change":"2023-05-16T07:17:47.986498+0000","last_active":"2023-05-16T07:17:48.356220+0000","last_peered":"2023-05-16T07:17:48.356220+0000","last_clean":"2023-05-16T07:17:48.356220+0000","last_became_active":"2023-05-16T07:17:47.986245+0000","last_became_peered":"2023-05-16T07:17:47.986245+0000","last_unstale":"2023-05-16T07:17:48.356220+0000","last_undegraded":"2023-05-16T07:17:48.356220+0000","last_fullsized":"2023-05-16T07:17:48.356220+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369102+0000","last_change":"2023-05-16T07:17:48.369102+0000","last_active":"2023-05-16T07:17:48.369102+0000","last_peered":"2023-05-16T07:17:48.369102+0000","last_clean":"2023-05-16T07:17:48.369102+0000","last_became_active":"2023-05-16T07:17:48.366539+0000","last_became_peered":"2023-05-16T07:17:48.366539+0000","last_unstale":"2023-05-16T07:17:48.369102+0000","last_undegraded":"2023-05-16T07:17:48.369102+0000","last_fullsized":"2023-05-16T07:17:48.369102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.344179+0000","last_change":"2023-05-16T07:17:47.992338+0000","last_active":"2023-05-16T07:17:48.344179+0000","last_peered":"2023-05-16T07:17:48.344179+0000","last_clean":"2023-05-16T07:17:48.344179+0000","last_became_active":"2023-05-16T07:17:47.992074+0000","last_became_peered":"2023-05-16T07:17:47.992074+0000","last_unstale":"2023-05-16T07:17:48.344179+0000","last_undegraded":"2023-05-16T07:17:48.344179+0000","last_fullsized":"2023-05-16T07:17:48.344179+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363249+0000","last_change":"2023-05-16T07:17:48.363249+0000","last_active":"2023-05-16T07:17:48.363249+0000","last_peered":"2023-05-16T07:17:48.363249+0000","last_clean":"2023-05-16T07:17:48.363249+0000","last_became_active":"2023-05-16T07:17:48.362952+0000","last_became_peered":"2023-05-16T07:17:48.362952+0000","last_unstale":"2023-05-16T07:17:48.363249+0000","last_undegraded":"2023-05-16T07:17:48.363249+0000","last_fullsized":"2023-05-16T07:17:48.363249+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.372658+0000","last_change":"2023-05-16T07:17:47.994226+0000","last_active":"2023-05-16T07:17:48.372658+0000","last_peered":"2023-05-16T07:17:48.372658+0000","last_clean":"2023-05-16T07:17:48.372658+0000","last_became_active":"2023-05-16T07:17:47.993765+0000","last_became_peered":"2023-05-16T07:17:47.993765+0000","last_unstale":"2023-05-16T07:17:48.372658+0000","last_undegraded":"2023-05-16T07:17:48.372658+0000","last_fullsized":"2023-05-16T07:17:48.372658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364624+0000","last_change":"2023-05-16T07:17:48.364624+0000","last_active":"2023-05-16T07:17:48.364624+0000","last_peered":"2023-05-16T07:17:48.364624+0000","last_clean":"2023-05-16T07:17:48.364624+0000","last_became_active":"2023-05-16T07:17:48.364343+0000","last_became_peered":"2023-05-16T07:17:48.364343+0000","last_unstale":"2023-05-16T07:17:48.364624+0000","last_undegraded":"2023-05-16T07:17:48.364624+0000","last_fullsized":"2023-05-16T07:17:48.364624+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000133+0000","last_change":"2023-05-16T07:17:47.997474+0000","last_active":"2023-05-16T07:17:48.000133+0000","last_peered":"2023-05-16T07:17:48.000133+0000","last_clean":"2023-05-16T07:17:48.000133+0000","last_became_active":"2023-05-16T07:17:47.997194+0000","last_became_peered":"2023-05-16T07:17:47.997194+0000","last_unstale":"2023-05-16T07:17:48.000133+0000","last_undegraded":"2023-05-16T07:17:48.000133+0000","last_fullsized":"2023-05-16T07:17:48.000133+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.364592+0000","last_change":"2023-05-16T07:17:48.364592+0000","last_active":"2023-05-16T07:17:48.364592+0000","last_peered":"2023-05-16T07:17:48.364592+0000","last_clean":"2023-05-16T07:17:48.364592+0000","last_became_active":"2023-05-16T07:17:48.364360+0000","last_became_peered":"2023-05-16T07:17:48.364360+0000","last_unstale":"2023-05-16T07:17:48.364592+0000","last_undegraded":"2023-05-16T07:17:48.364592+0000","last_fullsized":"2023-05-16T07:17:48.364592+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000752+0000","last_change":"2023-05-16T07:17:47.994013+0000","last_active":"2023-05-16T07:17:48.000752+0000","last_peered":"2023-05-16T07:17:48.000752+0000","last_clean":"2023-05-16T07:17:48.000752+0000","last_became_active":"2023-05-16T07:17:47.993444+0000","last_became_peered":"2023-05-16T07:17:47.993444+0000","last_unstale":"2023-05-16T07:17:48.000752+0000","last_undegraded":"2023-05-16T07:17:48.000752+0000","last_fullsized":"2023-05-16T07:17:48.000752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.370940+0000","last_change":"2023-05-16T07:17:48.370940+0000","last_active":"2023-05-16T07:17:48.370940+0000","last_peered":"2023-05-16T07:17:48.370940+0000","last_clean":"2023-05-16T07:17:48.370940+0000","last_became_active":"2023-05-16T07:17:48.370708+0000","last_became_peered":"2023-05-16T07:17:48.370708+0000","last_unstale":"2023-05-16T07:17:48.370940+0000","last_undegraded":"2023-05-16T07:17:48.370940+0000","last_fullsized":"2023-05-16T07:17:48.370940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002039+0000","last_change":"2023-05-16T07:17:48.000829+0000","last_active":"2023-05-16T07:17:48.002039+0000","last_peered":"2023-05-16T07:17:48.002039+0000","last_clean":"2023-05-16T07:17:48.002039+0000","last_became_active":"2023-05-16T07:17:48.000513+0000","last_became_peered":"2023-05-16T07:17:48.000513+0000","last_unstale":"2023-05-16T07:17:48.002039+0000","last_undegraded":"2023-05-16T07:17:48.002039+0000","last_fullsized":"2023-05-16T07:17:48.002039+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362178+0000","last_change":"2023-05-16T07:17:48.362178+0000","last_active":"2023-05-16T07:17:48.362178+0000","last_peered":"2023-05-16T07:17:48.362178+0000","last_clean":"2023-05-16T07:17:48.362178+0000","last_became_active":"2023-05-16T07:17:48.361907+0000","last_became_peered":"2023-05-16T07:17:48.361907+0000","last_unstale":"2023-05-16T07:17:48.362178+0000","last_undegraded":"2023-05-16T07:17:48.362178+0000","last_fullsized":"2023-05-16T07:17:48.362178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002118+0000","last_change":"2023-05-16T07:17:47.998161+0000","last_active":"2023-05-16T07:17:48.002118+0000","last_peered":"2023-05-16T07:17:48.002118+0000","last_clean":"2023-05-16T07:17:48.002118+0000","last_became_active":"2023-05-16T07:17:47.997888+0000","last_became_peered":"2023-05-16T07:17:47.997888+0000","last_unstale":"2023-05-16T07:17:48.002118+0000","last_undegraded":"2023-05-16T07:17:48.002118+0000","last_fullsized":"2023-05-16T07:17:48.002118+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.355734+0000","last_change":"2023-05-16T07:17:48.355734+0000","last_active":"2023-05-16T07:17:48.355734+0000","last_peered":"2023-05-16T07:17:48.355734+0000","last_clean":"2023-05-16T07:17:48.355734+0000","last_became_active":"2023-05-16T07:17:48.355470+0000","last_became_peered":"2023-05-16T07:17:48.355470+0000","last_unstale":"2023-05-16T07:17:48.355734+0000","last_undegraded":"2023-05-16T07:17:48.355734+0000","last_fullsized":"2023-05-16T07:17:48.355734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.998926+0000","last_change":"2023-05-16T07:17:47.993782+0000","last_active":"2023-05-16T07:17:47.998926+0000","last_peered":"2023-05-16T07:17:47.998926+0000","last_clean":"2023-05-16T07:17:47.998926+0000","last_became_active":"2023-05-16T07:17:47.993482+0000","last_became_peered":"2023-05-16T07:17:47.993482+0000","last_unstale":"2023-05-16T07:17:47.998926+0000","last_undegraded":"2023-05-16T07:17:47.998926+0000","last_fullsized":"2023-05-16T07:17:47.998926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367888+0000","last_change":"2023-05-16T07:17:48.367888+0000","last_active":"2023-05-16T07:17:48.367888+0000","last_peered":"2023-05-16T07:17:48.367888+0000","last_clean":"2023-05-16T07:17:48.367888+0000","last_became_active":"2023-05-16T07:17:48.367585+0000","last_became_peered":"2023-05-16T07:17:48.367585+0000","last_unstale":"2023-05-16T07:17:48.367888+0000","last_undegraded":"2023-05-16T07:17:48.367888+0000","last_fullsized":"2023-05-16T07:17:48.367888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003086+0000","last_change":"2023-05-16T07:17:47.997668+0000","last_active":"2023-05-16T07:17:48.003086+0000","last_peered":"2023-05-16T07:17:48.003086+0000","last_clean":"2023-05-16T07:17:48.003086+0000","last_became_active":"2023-05-16T07:17:47.997389+0000","last_became_peered":"2023-05-16T07:17:47.997389+0000","last_unstale":"2023-05-16T07:17:48.003086+0000","last_undegraded":"2023-05-16T07:17:48.003086+0000","last_fullsized":"2023-05-16T07:17:48.003086+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361350+0000","last_change":"2023-05-16T07:17:48.361350+0000","last_active":"2023-05-16T07:17:48.361350+0000","last_peered":"2023-05-16T07:17:48.361350+0000","last_clean":"2023-05-16T07:17:48.361350+0000","last_became_active":"2023-05-16T07:17:48.361051+0000","last_became_peered":"2023-05-16T07:17:48.361051+0000","last_unstale":"2023-05-16T07:17:48.361350+0000","last_undegraded":"2023-05-16T07:17:48.361350+0000","last_fullsized":"2023-05-16T07:17:48.361350+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.001575+0000","last_change":"2023-05-16T07:17:47.999934+0000","last_active":"2023-05-16T07:17:48.001575+0000","last_peered":"2023-05-16T07:17:48.001575+0000","last_clean":"2023-05-16T07:17:48.001575+0000","last_became_active":"2023-05-16T07:17:47.999659+0000","last_became_peered":"2023-05-16T07:17:47.999659+0000","last_unstale":"2023-05-16T07:17:48.001575+0000","last_undegraded":"2023-05-16T07:17:48.001575+0000","last_fullsized":"2023-05-16T07:17:48.001575+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.354944+0000","last_change":"2023-05-16T07:17:48.354944+0000","last_active":"2023-05-16T07:17:48.354944+0000","last_peered":"2023-05-16T07:17:48.354944+0000","last_clean":"2023-05-16T07:17:48.354944+0000","last_became_active":"2023-05-16T07:17:48.354592+0000","last_became_peered":"2023-05-16T07:17:48.354592+0000","last_unstale":"2023-05-16T07:17:48.354944+0000","last_undegraded":"2023-05-16T07:17:48.354944+0000","last_fullsized":"2023-05-16T07:17:48.354944+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"92'3","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.371635+0000","last_change":"2023-05-16T07:17:47.999263+0000","last_active":"2023-05-16T07:17:48.371635+0000","last_peered":"2023-05-16T07:17:48.371635+0000","last_clean":"2023-05-16T07:17:48.371635+0000","last_became_active":"2023-05-16T07:17:47.998864+0000","last_became_peered":"2023-05-16T07:17:47.998864+0000","last_unstale":"2023-05-16T07:17:48.371635+0000","last_undegraded":"2023-05-16T07:17:48.371635+0000","last_fullsized":"2023-05-16T07:17:48.371635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367339+0000","last_change":"2023-05-16T07:17:48.367339+0000","last_active":"2023-05-16T07:17:48.367339+0000","last_peered":"2023-05-16T07:17:48.367339+0000","last_clean":"2023-05-16T07:17:48.367339+0000","last_became_active":"2023-05-16T07:17:48.367030+0000","last_became_peered":"2023-05-16T07:17:48.367030+0000","last_unstale":"2023-05-16T07:17:48.367339+0000","last_undegraded":"2023-05-16T07:17:48.367339+0000","last_fullsized":"2023-05-16T07:17:48.367339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.358531+0000","last_change":"2023-05-16T07:17:47.997418+0000","last_active":"2023-05-16T07:17:48.358531+0000","last_peered":"2023-05-16T07:17:48.358531+0000","last_clean":"2023-05-16T07:17:48.358531+0000","last_became_active":"2023-05-16T07:17:47.996792+0000","last_became_peered":"2023-05-16T07:17:47.996792+0000","last_unstale":"2023-05-16T07:17:48.358531+0000","last_undegraded":"2023-05-16T07:17:48.358531+0000","last_fullsized":"2023-05-16T07:17:48.358531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362857+0000","last_change":"2023-05-16T07:17:48.362857+0000","last_active":"2023-05-16T07:17:48.362857+0000","last_peered":"2023-05-16T07:17:48.362857+0000","last_clean":"2023-05-16T07:17:48.362857+0000","last_became_active":"2023-05-16T07:17:48.362511+0000","last_became_peered":"2023-05-16T07:17:48.362511+0000","last_unstale":"2023-05-16T07:17:48.362857+0000","last_undegraded":"2023-05-16T07:17:48.362857+0000","last_fullsized":"2023-05-16T07:17:48.362857+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.352942+0000","last_change":"2023-05-16T07:17:47.996369+0000","last_active":"2023-05-16T07:17:48.352942+0000","last_peered":"2023-05-16T07:17:48.352942+0000","last_clean":"2023-05-16T07:17:48.352942+0000","last_became_active":"2023-05-16T07:17:47.996116+0000","last_became_peered":"2023-05-16T07:17:47.996116+0000","last_unstale":"2023-05-16T07:17:48.352942+0000","last_undegraded":"2023-05-16T07:17:48.352942+0000","last_fullsized":"2023-05-16T07:17:48.352942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.362701+0000","last_change":"2023-05-16T07:17:48.362701+0000","last_active":"2023-05-16T07:17:48.362701+0000","last_peered":"2023-05-16T07:17:48.362701+0000","last_clean":"2023-05-16T07:17:48.362701+0000","last_became_active":"2023-05-16T07:17:48.362100+0000","last_became_peered":"2023-05-16T07:17:48.362100+0000","last_unstale":"2023-05-16T07:17:48.362701+0000","last_undegraded":"2023-05-16T07:17:48.362701+0000","last_fullsized":"2023-05-16T07:17:48.362701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.994190+0000","last_change":"2023-05-16T07:17:47.989228+0000","last_active":"2023-05-16T07:17:47.994190+0000","last_peered":"2023-05-16T07:17:47.994190+0000","last_clean":"2023-05-16T07:17:47.994190+0000","last_became_active":"2023-05-16T07:17:47.988986+0000","last_became_peered":"2023-05-16T07:17:47.988986+0000","last_unstale":"2023-05-16T07:17:47.994190+0000","last_undegraded":"2023-05-16T07:17:47.994190+0000","last_fullsized":"2023-05-16T07:17:47.994190+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.356914+0000","last_change":"2023-05-16T07:17:48.356914+0000","last_active":"2023-05-16T07:17:48.356914+0000","last_peered":"2023-05-16T07:17:48.356914+0000","last_clean":"2023-05-16T07:17:48.356914+0000","last_became_active":"2023-05-16T07:17:48.356624+0000","last_became_peered":"2023-05-16T07:17:48.356624+0000","last_unstale":"2023-05-16T07:17:48.356914+0000","last_undegraded":"2023-05-16T07:17:48.356914+0000","last_fullsized":"2023-05-16T07:17:48.356914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992974+0000","last_change":"2023-05-16T07:17:47.986788+0000","last_active":"2023-05-16T07:17:47.992974+0000","last_peered":"2023-05-16T07:17:47.992974+0000","last_clean":"2023-05-16T07:17:47.992974+0000","last_became_active":"2023-05-16T07:17:47.986069+0000","last_became_peered":"2023-05-16T07:17:47.986069+0000","last_unstale":"2023-05-16T07:17:47.992974+0000","last_undegraded":"2023-05-16T07:17:47.992974+0000","last_fullsized":"2023-05-16T07:17:47.992974+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.348736+0000","last_change":"2023-05-16T07:17:48.348736+0000","last_active":"2023-05-16T07:17:48.348736+0000","last_peered":"2023-05-16T07:17:48.348736+0000","last_clean":"2023-05-16T07:17:48.348736+0000","last_became_active":"2023-05-16T07:17:48.348445+0000","last_became_peered":"2023-05-16T07:17:48.348445+0000","last_unstale":"2023-05-16T07:17:48.348736+0000","last_undegraded":"2023-05-16T07:17:48.348736+0000","last_fullsized":"2023-05-16T07:17:48.348736+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"92'4","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.372428+0000","last_change":"2023-05-16T07:17:47.994173+0000","last_active":"2023-05-16T07:17:48.372428+0000","last_peered":"2023-05-16T07:17:48.372428+0000","last_clean":"2023-05-16T07:17:48.372428+0000","last_became_active":"2023-05-16T07:17:47.993972+0000","last_became_peered":"2023-05-16T07:17:47.993972+0000","last_unstale":"2023-05-16T07:17:48.372428+0000","last_undegraded":"2023-05-16T07:17:48.372428+0000","last_fullsized":"2023-05-16T07:17:48.372428+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359764+0000","last_change":"2023-05-16T07:17:48.359764+0000","last_active":"2023-05-16T07:17:48.359764+0000","last_peered":"2023-05-16T07:17:48.359764+0000","last_clean":"2023-05-16T07:17:48.359764+0000","last_became_active":"2023-05-16T07:17:48.359461+0000","last_became_peered":"2023-05-16T07:17:48.359461+0000","last_unstale":"2023-05-16T07:17:48.359764+0000","last_undegraded":"2023-05-16T07:17:48.359764+0000","last_fullsized":"2023-05-16T07:17:48.359764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363322+0000","last_change":"2023-05-16T07:17:48.003802+0000","last_active":"2023-05-16T07:17:48.363322+0000","last_peered":"2023-05-16T07:17:48.363322+0000","last_clean":"2023-05-16T07:17:48.363322+0000","last_became_active":"2023-05-16T07:17:48.003484+0000","last_became_peered":"2023-05-16T07:17:48.003484+0000","last_unstale":"2023-05-16T07:17:48.363322+0000","last_undegraded":"2023-05-16T07:17:48.363322+0000","last_fullsized":"2023-05-16T07:17:48.363322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.346775+0000","last_change":"2023-05-16T07:17:48.346775+0000","last_active":"2023-05-16T07:17:48.346775+0000","last_peered":"2023-05-16T07:17:48.346775+0000","last_clean":"2023-05-16T07:17:48.346775+0000","last_became_active":"2023-05-16T07:17:48.346437+0000","last_became_peered":"2023-05-16T07:17:48.346437+0000","last_unstale":"2023-05-16T07:17:48.346775+0000","last_undegraded":"2023-05-16T07:17:48.346775+0000","last_fullsized":"2023-05-16T07:17:48.346775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.998900+0000","last_change":"2023-05-16T07:17:47.993667+0000","last_active":"2023-05-16T07:17:47.998900+0000","last_peered":"2023-05-16T07:17:47.998900+0000","last_clean":"2023-05-16T07:17:47.998900+0000","last_became_active":"2023-05-16T07:17:47.993457+0000","last_became_peered":"2023-05-16T07:17:47.993457+0000","last_unstale":"2023-05-16T07:17:47.998900+0000","last_undegraded":"2023-05-16T07:17:47.998900+0000","last_fullsized":"2023-05-16T07:17:47.998900+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363037+0000","last_change":"2023-05-16T07:17:48.363037+0000","last_active":"2023-05-16T07:17:48.363037+0000","last_peered":"2023-05-16T07:17:48.363037+0000","last_clean":"2023-05-16T07:17:48.363037+0000","last_became_active":"2023-05-16T07:17:48.362749+0000","last_became_peered":"2023-05-16T07:17:48.362749+0000","last_unstale":"2023-05-16T07:17:48.363037+0000","last_undegraded":"2023-05-16T07:17:48.363037+0000","last_fullsized":"2023-05-16T07:17:48.363037+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003276+0000","last_change":"2023-05-16T07:17:48.000633+0000","last_active":"2023-05-16T07:17:48.003276+0000","last_peered":"2023-05-16T07:17:48.003276+0000","last_clean":"2023-05-16T07:17:48.003276+0000","last_became_active":"2023-05-16T07:17:48.000299+0000","last_became_peered":"2023-05-16T07:17:48.000299+0000","last_unstale":"2023-05-16T07:17:48.003276+0000","last_undegraded":"2023-05-16T07:17:48.003276+0000","last_fullsized":"2023-05-16T07:17:48.003276+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369795+0000","last_change":"2023-05-16T07:17:48.369795+0000","last_active":"2023-05-16T07:17:48.369795+0000","last_peered":"2023-05-16T07:17:48.369795+0000","last_clean":"2023-05-16T07:17:48.369795+0000","last_became_active":"2023-05-16T07:17:48.369454+0000","last_became_peered":"2023-05-16T07:17:48.369454+0000","last_unstale":"2023-05-16T07:17:48.369795+0000","last_undegraded":"2023-05-16T07:17:48.369795+0000","last_fullsized":"2023-05-16T07:17:48.369795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003191+0000","last_change":"2023-05-16T07:17:47.997268+0000","last_active":"2023-05-16T07:17:48.003191+0000","last_peered":"2023-05-16T07:17:48.003191+0000","last_clean":"2023-05-16T07:17:48.003191+0000","last_became_active":"2023-05-16T07:17:47.996980+0000","last_became_peered":"2023-05-16T07:17:47.996980+0000","last_unstale":"2023-05-16T07:17:48.003191+0000","last_undegraded":"2023-05-16T07:17:48.003191+0000","last_fullsized":"2023-05-16T07:17:48.003191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.349014+0000","last_change":"2023-05-16T07:17:48.349014+0000","last_active":"2023-05-16T07:17:48.349014+0000","last_peered":"2023-05-16T07:17:48.349014+0000","last_clean":"2023-05-16T07:17:48.349014+0000","last_became_active":"2023-05-16T07:17:48.348473+0000","last_became_peered":"2023-05-16T07:17:48.348473+0000","last_unstale":"2023-05-16T07:17:48.349014+0000","last_undegraded":"2023-05-16T07:17:48.349014+0000","last_fullsized":"2023-05-16T07:17:48.349014+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.994092+0000","last_change":"2023-05-16T07:17:47.990507+0000","last_active":"2023-05-16T07:17:47.994092+0000","last_peered":"2023-05-16T07:17:47.994092+0000","last_clean":"2023-05-16T07:17:47.994092+0000","last_became_active":"2023-05-16T07:17:47.990284+0000","last_became_peered":"2023-05-16T07:17:47.990284+0000","last_unstale":"2023-05-16T07:17:47.994092+0000","last_undegraded":"2023-05-16T07:17:47.994092+0000","last_fullsized":"2023-05-16T07:17:47.994092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.360885+0000","last_change":"2023-05-16T07:17:48.360885+0000","last_active":"2023-05-16T07:17:48.360885+0000","last_peered":"2023-05-16T07:17:48.360885+0000","last_clean":"2023-05-16T07:17:48.360885+0000","last_became_active":"2023-05-16T07:17:48.360567+0000","last_became_peered":"2023-05-16T07:17:48.360567+0000","last_unstale":"2023-05-16T07:17:48.360885+0000","last_undegraded":"2023-05-16T07:17:48.360885+0000","last_fullsized":"2023-05-16T07:17:48.360885+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.989650+0000","last_change":"2023-05-16T07:17:47.984015+0000","last_active":"2023-05-16T07:17:47.989650+0000","last_peered":"2023-05-16T07:17:47.989650+0000","last_clean":"2023-05-16T07:17:47.989650+0000","last_became_active":"2023-05-16T07:17:47.983727+0000","last_became_peered":"2023-05-16T07:17:47.983727+0000","last_unstale":"2023-05-16T07:17:47.989650+0000","last_undegraded":"2023-05-16T07:17:47.989650+0000","last_fullsized":"2023-05-16T07:17:47.989650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.373211+0000","last_change":"2023-05-16T07:17:48.373211+0000","last_active":"2023-05-16T07:17:48.373211+0000","last_peered":"2023-05-16T07:17:48.373211+0000","last_clean":"2023-05-16T07:17:48.373211+0000","last_became_active":"2023-05-16T07:17:48.372954+0000","last_became_peered":"2023-05-16T07:17:48.372954+0000","last_unstale":"2023-05-16T07:17:48.373211+0000","last_undegraded":"2023-05-16T07:17:48.373211+0000","last_fullsized":"2023-05-16T07:17:48.373211+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.999851+0000","last_change":"2023-05-16T07:17:47.996848+0000","last_active":"2023-05-16T07:17:47.999851+0000","last_peered":"2023-05-16T07:17:47.999851+0000","last_clean":"2023-05-16T07:17:47.999851+0000","last_became_active":"2023-05-16T07:17:47.996523+0000","last_became_peered":"2023-05-16T07:17:47.996523+0000","last_unstale":"2023-05-16T07:17:47.999851+0000","last_undegraded":"2023-05-16T07:17:47.999851+0000","last_fullsized":"2023-05-16T07:17:47.999851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367667+0000","last_change":"2023-05-16T07:17:48.367667+0000","last_active":"2023-05-16T07:17:48.367667+0000","last_peered":"2023-05-16T07:17:48.367667+0000","last_clean":"2023-05-16T07:17:48.367667+0000","last_became_active":"2023-05-16T07:17:48.367451+0000","last_became_peered":"2023-05-16T07:17:48.367451+0000","last_unstale":"2023-05-16T07:17:48.367667+0000","last_undegraded":"2023-05-16T07:17:48.367667+0000","last_fullsized":"2023-05-16T07:17:48.367667+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003234+0000","last_change":"2023-05-16T07:17:47.996821+0000","last_active":"2023-05-16T07:17:48.003234+0000","last_peered":"2023-05-16T07:17:48.003234+0000","last_clean":"2023-05-16T07:17:48.003234+0000","last_became_active":"2023-05-16T07:17:47.996524+0000","last_became_peered":"2023-05-16T07:17:47.996524+0000","last_unstale":"2023-05-16T07:17:48.003234+0000","last_undegraded":"2023-05-16T07:17:48.003234+0000","last_fullsized":"2023-05-16T07:17:48.003234+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.369060+0000","last_change":"2023-05-16T07:17:48.369060+0000","last_active":"2023-05-16T07:17:48.369060+0000","last_peered":"2023-05-16T07:17:48.369060+0000","last_clean":"2023-05-16T07:17:48.369060+0000","last_became_active":"2023-05-16T07:17:48.366229+0000","last_became_peered":"2023-05-16T07:17:48.366229+0000","last_unstale":"2023-05-16T07:17:48.369060+0000","last_undegraded":"2023-05-16T07:17:48.369060+0000","last_fullsized":"2023-05-16T07:17:48.369060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.002889+0000","last_change":"2023-05-16T07:17:47.993822+0000","last_active":"2023-05-16T07:17:48.002889+0000","last_peered":"2023-05-16T07:17:48.002889+0000","last_clean":"2023-05-16T07:17:48.002889+0000","last_became_active":"2023-05-16T07:17:47.993553+0000","last_became_peered":"2023-05-16T07:17:47.993553+0000","last_unstale":"2023-05-16T07:17:48.002889+0000","last_undegraded":"2023-05-16T07:17:48.002889+0000","last_fullsized":"2023-05-16T07:17:48.002889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.991073+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:17:47.991073+0000","last_peered":"2023-05-16T07:17:47.991073+0000","last_clean":"2023-05-16T07:17:47.991073+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:17:47.991073+0000","last_undegraded":"2023-05-16T07:17:47.991073+0000","last_fullsized":"2023-05-16T07:17:47.991073+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.365098+0000","last_change":"2023-05-16T07:17:48.365098+0000","last_active":"2023-05-16T07:17:48.365098+0000","last_peered":"2023-05-16T07:17:48.365098+0000","last_clean":"2023-05-16T07:17:48.365098+0000","last_became_active":"2023-05-16T07:17:48.364842+0000","last_became_peered":"2023-05-16T07:17:48.364842+0000","last_unstale":"2023-05-16T07:17:48.365098+0000","last_undegraded":"2023-05-16T07:17:48.365098+0000","last_fullsized":"2023-05-16T07:17:48.365098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.344591+0000","last_change":"2023-05-16T07:17:47.999936+0000","last_active":"2023-05-16T07:17:48.344591+0000","last_peered":"2023-05-16T07:17:48.344591+0000","last_clean":"2023-05-16T07:17:48.344591+0000","last_became_active":"2023-05-16T07:17:47.999667+0000","last_became_peered":"2023-05-16T07:17:47.999667+0000","last_unstale":"2023-05-16T07:17:48.344591+0000","last_undegraded":"2023-05-16T07:17:48.344591+0000","last_fullsized":"2023-05-16T07:17:48.344591+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.992638+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:17:47.992638+0000","last_peered":"2023-05-16T07:17:47.992638+0000","last_clean":"2023-05-16T07:17:47.992638+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:17:47.992638+0000","last_undegraded":"2023-05-16T07:17:47.992638+0000","last_fullsized":"2023-05-16T07:17:47.992638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359291+0000","last_change":"2023-05-16T07:17:48.359291+0000","last_active":"2023-05-16T07:17:48.359291+0000","last_peered":"2023-05-16T07:17:48.359291+0000","last_clean":"2023-05-16T07:17:48.359291+0000","last_became_active":"2023-05-16T07:17:48.358992+0000","last_became_peered":"2023-05-16T07:17:48.358992+0000","last_unstale":"2023-05-16T07:17:48.359291+0000","last_undegraded":"2023-05-16T07:17:48.359291+0000","last_fullsized":"2023-05-16T07:17:48.359291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.000165+0000","last_change":"2023-05-16T07:17:47.997323+0000","last_active":"2023-05-16T07:17:48.000165+0000","last_peered":"2023-05-16T07:17:48.000165+0000","last_clean":"2023-05-16T07:17:48.000165+0000","last_became_active":"2023-05-16T07:17:47.997062+0000","last_became_peered":"2023-05-16T07:17:47.997062+0000","last_unstale":"2023-05-16T07:17:48.000165+0000","last_undegraded":"2023-05-16T07:17:48.000165+0000","last_fullsized":"2023-05-16T07:17:48.000165+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.976935+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:17:47.976935+0000","last_peered":"2023-05-16T07:17:47.976935+0000","last_clean":"2023-05-16T07:17:47.976935+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:17:47.976935+0000","last_undegraded":"2023-05-16T07:17:47.976935+0000","last_fullsized":"2023-05-16T07:17:47.976935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.367598+0000","last_change":"2023-05-16T07:17:48.367598+0000","last_active":"2023-05-16T07:17:48.367598+0000","last_peered":"2023-05-16T07:17:48.367598+0000","last_clean":"2023-05-16T07:17:48.367598+0000","last_became_active":"2023-05-16T07:17:48.367333+0000","last_became_peered":"2023-05-16T07:17:48.367333+0000","last_unstale":"2023-05-16T07:17:48.367598+0000","last_undegraded":"2023-05-16T07:17:48.367598+0000","last_fullsized":"2023-05-16T07:17:48.367598+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.993947+0000","last_change":"2023-05-16T07:17:47.989476+0000","last_active":"2023-05-16T07:17:47.993947+0000","last_peered":"2023-05-16T07:17:47.993947+0000","last_clean":"2023-05-16T07:17:47.993947+0000","last_became_active":"2023-05-16T07:17:47.989033+0000","last_became_peered":"2023-05-16T07:17:47.989033+0000","last_unstale":"2023-05-16T07:17:47.993947+0000","last_undegraded":"2023-05-16T07:17:47.993947+0000","last_fullsized":"2023-05-16T07:17:47.993947+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.986875+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:17:47.986875+0000","last_peered":"2023-05-16T07:17:47.986875+0000","last_clean":"2023-05-16T07:17:47.986875+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:17:47.986875+0000","last_undegraded":"2023-05-16T07:17:47.986875+0000","last_fullsized":"2023-05-16T07:17:47.986875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.984937+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:17:47.984937+0000","last_peered":"2023-05-16T07:17:47.984937+0000","last_clean":"2023-05-16T07:17:47.984937+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:17:47.984937+0000","last_undegraded":"2023-05-16T07:17:47.984937+0000","last_fullsized":"2023-05-16T07:17:47.984937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.366068+0000","last_change":"2023-05-16T07:17:48.366068+0000","last_active":"2023-05-16T07:17:48.366068+0000","last_peered":"2023-05-16T07:17:48.366068+0000","last_clean":"2023-05-16T07:17:48.366068+0000","last_became_active":"2023-05-16T07:17:48.365421+0000","last_became_peered":"2023-05-16T07:17:48.365421+0000","last_unstale":"2023-05-16T07:17:48.366068+0000","last_undegraded":"2023-05-16T07:17:48.366068+0000","last_fullsized":"2023-05-16T07:17:48.366068+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.997143+0000","last_change":"2023-05-16T07:17:47.992963+0000","last_active":"2023-05-16T07:17:47.997143+0000","last_peered":"2023-05-16T07:17:47.997143+0000","last_clean":"2023-05-16T07:17:47.997143+0000","last_became_active":"2023-05-16T07:17:47.992701+0000","last_became_peered":"2023-05-16T07:17:47.992701+0000","last_unstale":"2023-05-16T07:17:47.997143+0000","last_undegraded":"2023-05-16T07:17:47.997143+0000","last_fullsized":"2023-05-16T07:17:47.997143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.984959+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:17:47.984959+0000","last_peered":"2023-05-16T07:17:47.984959+0000","last_clean":"2023-05-16T07:17:47.984959+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:17:47.984959+0000","last_undegraded":"2023-05-16T07:17:47.984959+0000","last_fullsized":"2023-05-16T07:17:47.984959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.352790+0000","last_change":"2023-05-16T07:17:48.352790+0000","last_active":"2023-05-16T07:17:48.352790+0000","last_peered":"2023-05-16T07:17:48.352790+0000","last_clean":"2023-05-16T07:17:48.352790+0000","last_became_active":"2023-05-16T07:17:48.352531+0000","last_became_peered":"2023-05-16T07:17:48.352531+0000","last_unstale":"2023-05-16T07:17:48.352790+0000","last_undegraded":"2023-05-16T07:17:48.352790+0000","last_fullsized":"2023-05-16T07:17:48.352790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.003320+0000","last_change":"2023-05-16T07:17:47.997542+0000","last_active":"2023-05-16T07:17:48.003320+0000","last_peered":"2023-05-16T07:17:48.003320+0000","last_clean":"2023-05-16T07:17:48.003320+0000","last_became_active":"2023-05-16T07:17:47.997099+0000","last_became_peered":"2023-05-16T07:17:47.997099+0000","last_unstale":"2023-05-16T07:17:48.003320+0000","last_undegraded":"2023-05-16T07:17:48.003320+0000","last_fullsized":"2023-05-16T07:17:48.003320+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.983661+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:17:47.983661+0000","last_peered":"2023-05-16T07:17:47.983661+0000","last_clean":"2023-05-16T07:17:47.983661+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:17:47.983661+0000","last_undegraded":"2023-05-16T07:17:47.983661+0000","last_fullsized":"2023-05-16T07:17:47.983661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.361582+0000","last_change":"2023-05-16T07:17:48.361582+0000","last_active":"2023-05-16T07:17:48.361582+0000","last_peered":"2023-05-16T07:17:48.361582+0000","last_clean":"2023-05-16T07:17:48.361582+0000","last_became_active":"2023-05-16T07:17:48.361365+0000","last_became_peered":"2023-05-16T07:17:48.361365+0000","last_unstale":"2023-05-16T07:17:48.361582+0000","last_undegraded":"2023-05-16T07:17:48.361582+0000","last_fullsized":"2023-05-16T07:17:48.361582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.344381+0000","last_change":"2023-05-16T07:17:47.998272+0000","last_active":"2023-05-16T07:17:48.344381+0000","last_peered":"2023-05-16T07:17:48.344381+0000","last_clean":"2023-05-16T07:17:48.344381+0000","last_became_active":"2023-05-16T07:17:47.998004+0000","last_became_peered":"2023-05-16T07:17:47.998004+0000","last_unstale":"2023-05-16T07:17:48.344381+0000","last_undegraded":"2023-05-16T07:17:48.344381+0000","last_fullsized":"2023-05-16T07:17:48.344381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.990320+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:17:47.990320+0000","last_peered":"2023-05-16T07:17:47.990320+0000","last_clean":"2023-05-16T07:17:47.990320+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:17:47.990320+0000","last_undegraded":"2023-05-16T07:17:47.990320+0000","last_fullsized":"2023-05-16T07:17:47.990320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.359459+0000","last_change":"2023-05-16T07:17:48.359459+0000","last_active":"2023-05-16T07:17:48.359459+0000","last_peered":"2023-05-16T07:17:48.359459+0000","last_clean":"2023-05-16T07:17:48.359459+0000","last_became_active":"2023-05-16T07:17:48.359183+0000","last_became_peered":"2023-05-16T07:17:48.359183+0000","last_unstale":"2023-05-16T07:17:48.359459+0000","last_undegraded":"2023-05-16T07:17:48.359459+0000","last_fullsized":"2023-05-16T07:17:48.359459+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:47.970364+0000","last_clean_scrub_stamp":"2023-05-16T07:17:47.970364+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-05-16T07:17:48.363348+0000","last_change":"2023-05-16T07:17:47.999113+0000","last_active":"2023-05-16T07:17:48.363348+0000","last_peered":"2023-05-16T07:17:48.363348+0000","last_clean":"2023-05-16T07:17:48.363348+0000","last_became_active":"2023-05-16T07:17:47.998844+0000","last_became_peered":"2023-05-16T07:17:47.998844+0000","last_unstale":"2023-05-16T07:17:48.363348+0000","last_undegraded":"2023-05-16T07:17:48.363348+0000","last_fullsized":"2023-05-16T07:17:48.363348+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:17:46.965431+0000","last_clean_scrub_stamp":"2023-05-16T07:17:46.965431+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-05-16T07:17:47.980790+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:17:47.980790+0000","last_peered":"2023-05-16T07:17:47.980790+0000","last_clean":"2023-05-16T07:17:47.980790+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:17:47.980790+0000","last_undegraded":"2023-05-16T07:17:47.980790+0000","last_fullsized":"2023-05-16T07:17:47.980790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575033,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17624,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":16512,"kb_avail":94354216,"statfs":{"total":96636764160,"available":96618717184,"internally_reserved":0,"allocated":1130496,"data_stored":577635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16908288},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.439,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.751,"5min":0.81399999999999995,"15min":0.84199999999999997},"last":0.371},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.47399999999999998,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":0.873},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.752,"5min":0.876,"15min":0.876},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.48599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.438}]},{"osd":3,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66600000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.308},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.752,"15min":0.81999999999999995},"last":0.33000000000000002}]},{"osd":5,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68500000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.40899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.999},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue May 16 07:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.95099999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.46200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21060,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94350780,"statfs":{"total":96636764160,"available":96615198720,"internally_reserved":0,"allocated":1110016,"data_stored":569296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.317,"15min":0.26400000000000001},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66500000000000004,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76000000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.311,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.89400000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.503,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":4.048,"15min":4.048},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.749,"5min":0.749,"15min":0.92300000000000004},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.62}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17028,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":15936,"kb_avail":94354812,"statfs":{"total":96636764160,"available":96619327488,"internally_reserved":0,"allocated":1110016,"data_stored":569296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16318464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.69899999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.48199999999999998,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.313}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36499999999999999,"15min":0.378},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26000000000000001},"max":{"1min":0.49399999999999999,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.52700000000000002,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38300000000000001,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.501,"5min":0.622,"15min":0.64000000000000001},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71199999999999997,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.66900000000000004}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76100000000000001,"5min":0.94699999999999995,"15min":1.087},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.93700000000000006,"15min":1.204},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.627,"15min":0.64100000000000001},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41299999999999998},"max":{"1min":0.76900000000000002,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16056,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":14976,"kb_avail":94355784,"statfs":{"total":96636764160,"available":96620322816,"internally_reserved":0,"allocated":1097728,"data_stored":562033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15335424},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33900000000000002,"15min":0.35699999999999998},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47099999999999997,"5min":0.53700000000000003,"15min":0.60199999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.38600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.35299999999999998,"15min":0.373},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.40300000000000002},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.50600000000000001,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.42099999999999999}]},{"osd":3,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38600000000000001,"15min":0.40699999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.24199999999999999},"max":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.62}]},{"osd":5,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60599999999999998,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":1.157},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.505}]},{"osd":6,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":1.0209999999999999}]},{"osd":7,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.65000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85499999999999998,"5min":0.93700000000000006,"15min":1.196},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.64500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18012,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":16896,"kb_avail":94353828,"statfs":{"total":96636764160,"available":96618319872,"internally_reserved":0,"allocated":1134592,"data_stored":578595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17301504},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.5,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.312},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.375,"15min":0.376},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.434,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.32200000000000001}]},{"osd":3,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.64700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.432,"15min":0.439},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60899999999999999,"5min":0.71399999999999997,"15min":0.72899999999999998},"last":0.27600000000000002}]},{"osd":4,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.274},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82599999999999996,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.50900000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.37},"max":{"1min":0.872,"5min":0.88100000000000001,"15min":1.5609999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.47},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88200000000000001,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.54500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575033,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20944,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94350896,"statfs":{"total":96636764160,"available":96615317504,"internally_reserved":0,"allocated":1122304,"data_stored":577059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.29799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.40899999999999997},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42199999999999999,"15min":0.41499999999999998},"min":{"1min":0.33100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.25},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.33800000000000002,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.626,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.374}]},{"osd":4,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77500000000000002,"5min":0.93100000000000005,"15min":1.0489999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83799999999999997,"5min":0.93799999999999994,"15min":1.0409999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.504,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":0.97399999999999998,"15min":1.0780000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":1.0049999999999999},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue May 16 07:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.44,"15min":0.40100000000000002},"max":{"1min":0.75800000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":1.02},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17016,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":15936,"kb_avail":94354824,"statfs":{"total":96636764160,"available":96619339776,"internally_reserved":0,"allocated":1097728,"data_stored":562033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16318464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.82099999999999995,"15min":0.83199999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.81200000000000006},"last":0.439}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.627,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.67500000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.495},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.90500000000000003,"15min":0.999},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.97999999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.629,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84099999999999997,"5min":0.875,"15min":0.93100000000000005},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":1.0640000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.31900000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70699999999999996,"5min":0.79200000000000004,"15min":0.94599999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.86199999999999999,"15min":1.2689999999999999},"last":0.33800000000000002}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":1.48,"15min":1.48},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":1.046},"last":0.44700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20344,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":19264,"kb_avail":94351496,"statfs":{"total":96636764160,"available":96615931904,"internally_reserved":0,"allocated":1097728,"data_stored":562033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19726336},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.36599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81599999999999995,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.81000000000000005,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":0.96099999999999997,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86399999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.54300000000000004}]},{"osd":2,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71399999999999997,"5min":2.278,"15min":2.278},"last":0.503},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.45100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.748,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.33900000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":1.333,"15min":1.333},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.309},"max":{"1min":0.84899999999999998,"5min":2.081,"15min":2.081},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":1.024,"15min":1.024},"last":0.44900000000000001}]},{"osd":7,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.878,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.432},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.86599999999999999,"15min":0.878},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:17:52.310 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:17:52.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:17:52.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.454+0000 7f271500f700 1 Processor -- start 2023-05-16T07:17:52.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.454+0000 7f271500f700 1 -- start start 2023-05-16T07:17:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.454+0000 7f271500f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2710154470 0x7f27100aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.454+0000 7f271500f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100c94d0 con 0x7f2710154090 2023-05-16T07:17:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.454+0000 7f271500f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100cb7e0 con 0x7f2710068160 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.454+0000 7f271500f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100c88c0 con 0x7f2710154470 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2710154090 0x7f27100a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57300/0 (socket says 172.21.15.92:57300) 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270ed9d700 1 -- 172.21.15.92:0/3960217459 learned_addr learned my addr 172.21.15.92:0/3960217459 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270659c700 1 --2- 172.21.15.92:0/3960217459 >> v2:172.21.15.158:3300/0 conn(0x7f2710154470 0x7f27100aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886536416 0 0) 0x7f27100c94d0 con 0x7f2710154090 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26fc0033f0 con 0x7f2710154090 2023-05-16T07:17:52.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 703836077 0 0) 0x7f26fc0033f0 con 0x7f2710154090 2023-05-16T07:17:52.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 >> v2:172.21.15.158:3300/0 conn(0x7f2710154470 msgr2=0x7f27100aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:52.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 --2- 172.21.15.92:0/3960217459 >> v2:172.21.15.158:3300/0 conn(0x7f2710154470 0x7f27100aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 >> v1:172.21.15.158:6789/0 conn(0x7f2710068160 legacy=0x7f27100c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100c8ba0 con 0x7f2710154090 2023-05-16T07:17:52.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f2700001470 con 0x7f2710154090 2023-05-16T07:17:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 >> v1:172.21.15.92:6789/0 conn(0x7f2710154090 legacy=0x7f27100a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f26fc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc005000 0x7f26fc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100c88c0 con 0x7f26fc004690 2023-05-16T07:17:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc00bf60 con 0x7f26fc005000 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc00c0f0 con 0x7f26fc0038e0 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270659c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270ed9d700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f26fc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.455+0000 7f270f59e700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc005000 0x7f26fc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270659c700 1 -- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 msgr2=0x7f26fc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270659c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26fc0038e0 0x7f26fc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270659c700 1 -- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc005000 msgr2=0x7f26fc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270659c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc005000 0x7f26fc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270659c700 1 -- 172.21.15.92:0/3960217459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0033f0 con 0x7f26fc004690 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270f59e700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc005000 0x7f26fc00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270f59e700 1 -- 172.21.15.92:0/3960217459 reap_dead start 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270659c700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f26f8009a60 tx=0x7f26f8009060).ready entity=mon.0 client_cookie=b4d924d664a13033 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8009ea0 con 0x7f26fc004690 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f8009430 con 0x7f26fc004690 2023-05-16T07:17:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f270e59c700 1 -- 172.21.15.92:0/3960217459 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f800e870 con 0x7f26fc004690 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 -- 172.21.15.92:0/3960217459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc004690 msgr2=0x7f26fc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f26f8009a60 tx=0x7f26f8009060).stop 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 -- 172.21.15.92:0/3960217459 shutdown_connections 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 --2- 172.21.15.92:0/3960217459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc004690 0x7f26fc004b10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 -- 172.21.15.92:0/3960217459 >> 172.21.15.92:0/3960217459 conn(0x7f2710067920 msgr2=0x7f27100cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 -- 172.21.15.92:0/3960217459 shutdown_connections 2023-05-16T07:17:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.456+0000 7f271500f700 1 -- 172.21.15.92:0/3960217459 wait complete. 2023-05-16T07:17:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 Processor -- start 2023-05-16T07:17:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 -- start start 2023-05-16T07:17:52.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f2710152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710150a00 0x7f27100b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 0x7f2710165f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710150e80 con 0x7f2710165ad0 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710151070 con 0x7f26fc004690 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f271500f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2710151350 con 0x7f2710150a00 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.458+0000 7f270f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 0x7f2710165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 0x7f2710165f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46566/0 (socket says 172.21.15.92:46566) 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 -- 172.21.15.92:0/1672660420 learned_addr learned my addr 172.21.15.92:0/1672660420 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270ed9d700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f2710152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270659c700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710150a00 0x7f27100b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 -- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710150a00 msgr2=0x7f27100b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710150a00 0x7f27100b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 -- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 msgr2=0x7f2710152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f2710152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 -- 172.21.15.92:0/1672660420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2710151630 con 0x7f2710165ad0 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f270f59e700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 0x7f2710165f50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f2710069010 tx=0x7f2708009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270800ee90 con 0x7f2710165ad0 2023-05-16T07:17:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708002490 con 0x7f2710165ad0 2023-05-16T07:17:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710151910 con 0x7f2710165ad0 2023-05-16T07:17:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708002810 con 0x7f2710165ad0 2023-05-16T07:17:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.459+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100b0950 con 0x7f2710165ad0 2023-05-16T07:17:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.460+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710151e30 con 0x7f2710165ad0 2023-05-16T07:17:52.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.460+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2708002b80 con 0x7f2710165ad0 2023-05-16T07:17:52.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.461+0000 7f27077fe700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed90 0x7f26f0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.461+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f2708079930 con 0x7f2710165ad0 2023-05-16T07:17:52.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.461+0000 7f270ed9d700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed90 0x7f26f0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.461+0000 7f270ed9d700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed90 0x7f26f0061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2700000e50 tx=0x7f2700001e50).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.465+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f27080498f0 con 0x7f2710165ad0 2023-05-16T07:17:52.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.686+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f2710166860 con 0x7f2710165ad0 2023-05-16T07:17:52.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.690+0000 7f27077fe700 1 -- 172.21.15.92:0/1672660420 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2710151e30 con 0x7f2710165ad0 2023-05-16T07:17:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed90 msgr2=0x7f26f0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed90 0x7f26f0061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2700000e50 tx=0x7f2700001e50).stop 2023-05-16T07:17:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 msgr2=0x7f2710165f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 0x7f2710165f50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f2710069010 tx=0x7f2708009040).stop 2023-05-16T07:17:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 shutdown_connections 2023-05-16T07:17:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f26f005ed90 0x7f26f0061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2710165ad0 0x7f2710165f50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26fc004690 0x7f2710152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 --2- 172.21.15.92:0/1672660420 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2710150a00 0x7f27100b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 >> 172.21.15.92:0/1672660420 conn(0x7f2710067920 msgr2=0x7f27100c11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 shutdown_connections 2023-05-16T07:17:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.692+0000 7f271500f700 1 -- 172.21.15.92:0/1672660420 wait complete. 2023-05-16T07:17:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:17:52.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:17:52.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.814+0000 7fa23d952700 1 Processor -- start 2023-05-16T07:17:52.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.814+0000 7fa23d952700 1 -- start start 2023-05-16T07:17:52.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23d952700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa238151bd0 0x7fa238156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23d952700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2380a76f0 0x7fa2380a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23d952700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380c8750 con 0x7fa2381517f0 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23d952700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa238155d70 con 0x7fa238151bd0 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23d952700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380a8030 con 0x7fa2380a76f0 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23c950700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa2381517f0 0x7fa238152a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57320/0 (socket says 172.21.15.92:57320) 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa23c950700 1 -- 172.21.15.92:0/1710850440 learned_addr learned my addr 172.21.15.92:0/1710850440 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa237fff700 1 --2- 172.21.15.92:0/1710850440 >> v2:172.21.15.92:3300/0 conn(0x7fa238151bd0 0x7fa238156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa2377fe700 1 -- 172.21.15.92:0/1710850440 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631016770 0 0) 0x7fa2380c8750 con 0x7fa2381517f0 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa2377fe700 1 -- 172.21.15.92:0/1710850440 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2240033f0 con 0x7fa2381517f0 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa237fff700 1 -- 172.21.15.92:0/1710850440 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2380a76f0 msgr2=0x7fa2380a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa237fff700 1 --2- 172.21.15.92:0/1710850440 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2380a76f0 0x7fa2380a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa237fff700 1 -- 172.21.15.92:0/1710850440 >> v1:172.21.15.92:6789/0 conn(0x7fa2381517f0 legacy=0x7fa238152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa237fff700 1 -- 172.21.15.92:0/1710850440 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380c8470 con 0x7fa238151bd0 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa237fff700 1 --2- 172.21.15.92:0/1710850440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa238151bd0 0x7fa238156200 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fa228009e60 tx=0x7fa228009780).ready entity=mon.0 client_cookie=9f08b59d6d7686b3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa2377fe700 1 -- 172.21.15.92:0/1710850440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800e070 con 0x7fa238151bd0 2023-05-16T07:17:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa2377fe700 1 -- 172.21.15.92:0/1710850440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22800bb60 con 0x7fa238151bd0 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.815+0000 7fa2377fe700 1 -- 172.21.15.92:0/1710850440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228012410 con 0x7fa238151bd0 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 -- 172.21.15.92:0/1710850440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa238151bd0 msgr2=0x7fa238156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 --2- 172.21.15.92:0/1710850440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa238151bd0 0x7fa238156200 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fa228009e60 tx=0x7fa228009780).stop 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 -- 172.21.15.92:0/1710850440 shutdown_connections 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 --2- 172.21.15.92:0/1710850440 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa2380a76f0 0x7fa2380a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 --2- 172.21.15.92:0/1710850440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa238151bd0 0x7fa238156200 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 -- 172.21.15.92:0/1710850440 >> 172.21.15.92:0/1710850440 conn(0x7fa23805e820 msgr2=0x7fa2380622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 -- 172.21.15.92:0/1710850440 shutdown_connections 2023-05-16T07:17:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.816+0000 7fa23d952700 1 -- 172.21.15.92:0/1710850440 wait complete. 2023-05-16T07:17:52.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.817+0000 7fa23d952700 1 Processor -- start 2023-05-16T07:17:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.817+0000 7fa23d952700 1 -- start start 2023-05-16T07:17:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 0x7fa2380c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2381517f0 0x7fa2380c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa238151bd0 0x7fa23814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c5420 con 0x7fa2380a76f0 2023-05-16T07:17:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380a8e60 con 0x7fa2381517f0 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380a8830 con 0x7fa238151bd0 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 0x7fa2380c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 0x7fa2380c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46578/0 (socket says 172.21.15.92:46578) 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 -- 172.21.15.92:0/2931830270 learned_addr learned my addr 172.21.15.92:0/2931830270 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 -- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa238151bd0 msgr2=0x7fa23814f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa238151bd0 0x7fa23814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 -- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2381517f0 msgr2=0x7fa2380c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2381517f0 0x7fa2380c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 -- 172.21.15.92:0/2931830270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa228009a20 con 0x7fa2380a76f0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23c950700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 0x7fa2380c4a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fa22000c4b0 tx=0x7fa220007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200072e0 con 0x7fa2380a76f0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa220007440 con 0x7fa2380a76f0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa238150560 con 0x7fa2380a76f0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.818+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220005880 con 0x7fa2380a76f0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.819+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa238161930 con 0x7fa2380a76f0 2023-05-16T07:17:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.819+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa238047150 con 0x7fa2380a76f0 2023-05-16T07:17:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.820+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa22001a060 con 0x7fa2380a76f0 2023-05-16T07:17:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.820+0000 7fa234ff9700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa21805ed90 0x7fa218061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.820+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fa2200795a0 con 0x7fa2380a76f0 2023-05-16T07:17:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.820+0000 7fa237fff700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa21805ed90 0x7fa218061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.821+0000 7fa237fff700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa21805ed90 0x7fa218061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa238154320 tx=0x7fa228009780).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:52.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:52.824+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa22004a8f0 con 0x7fa2380a76f0 2023-05-16T07:17:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.026+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa238155720 con 0x7fa2380a76f0 2023-05-16T07:17:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.026+0000 7fa234ff9700 1 -- 172.21.15.92:0/2931830270 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fa220018020 con 0x7fa2380a76f0 2023-05-16T07:17:53.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:17:53.031 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:48.337476+0000","modified":"2023-05-16T07:17:50.983467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1656722289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1656722289},{"type":"v1","addr":"172.21.15.92:6835","nonce":1656722289}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-05-16T07:17:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa21805ed90 msgr2=0x7fa218061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa21805ed90 0x7fa218061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa238154320 tx=0x7fa228009780).stop 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 msgr2=0x7fa2380c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 0x7fa2380c4a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fa22000c4b0 tx=0x7fa220007b00).stop 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 shutdown_connections 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa21805ed90 0x7fa218061250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa238151bd0 0x7fa23814f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2380a76f0 0x7fa2380c4a40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 --2- 172.21.15.92:0/2931830270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa2381517f0 0x7fa2380c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.028+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 >> 172.21.15.92:0/2931830270 conn(0x7fa23805e820 msgr2=0x7fa23814bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.029+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 shutdown_connections 2023-05-16T07:17:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.029+0000 7fa23d952700 1 -- 172.21.15.92:0/2931830270 wait complete. 2023-05-16T07:17:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-05-16T07:17:53.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:17:48.337476+0000', 'modified': '2023-05-16T07:17:50.983467+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6594}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6594': {'gid': 6594, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/1656722289', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6833', 'nonce': 1656722289}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1656722289}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:17:53.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:17:53.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-16T07:17:53.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.557+0000 7f79c61cf700 1 Processor -- start 2023-05-16T07:17:53.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79c61cf700 1 -- start start 2023-05-16T07:17:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79c61cf700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79c0063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79c61cf700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f79c0062fa0 0x7f79c0063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79c61cf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c0062b10 con 0x7f79c0153da0 2023-05-16T07:17:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79c61cf700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c0064100 con 0x7f79c0062fa0 2023-05-16T07:17:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79c61cf700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0068620 con 0x7f79c0154180 2023-05-16T07:17:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79bf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f79c0153da0 0x7f79c0069a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57334/0 (socket says 172.21.15.92:57334) 2023-05-16T07:17:53.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.558+0000 7f79bf7fe700 1 -- 172.21.15.92:0/1727672924 learned_addr learned my addr 172.21.15.92:0/1727672924 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:53.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045214693 0 0) 0x7f79c0062b10 con 0x7f79c0153da0 2023-05-16T07:17:53.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a80033f0 con 0x7f79c0153da0 2023-05-16T07:17:53.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79bffff700 1 --2- 172.21.15.92:0/1727672924 >> v2:172.21.15.158:3300/0 conn(0x7f79c0062fa0 0x7f79c0063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79beffd700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79c0063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2439203955 0 0) 0x7f79a80033f0 con 0x7f79c0153da0 2023-05-16T07:17:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 msgr2=0x7f79c0063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79c0063720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 >> v2:172.21.15.158:3300/0 conn(0x7f79c0062fa0 msgr2=0x7f79c0063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 --2- 172.21.15.92:0/1727672924 >> v2:172.21.15.158:3300/0 conn(0x7f79c0062fa0 0x7f79c0063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0068340 con 0x7f79c0153da0 2023-05-16T07:17:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f79b00017f0 con 0x7f79c0153da0 2023-05-16T07:17:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79beffd700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79c0063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:17:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 >> v1:172.21.15.92:6789/0 conn(0x7f79c0153da0 legacy=0x7f79c0069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79a80038e0 0x7f79a8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79a8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79a8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0064100 con 0x7f79a8004b80 2023-05-16T07:17:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0068620 con 0x7f79a80038e0 2023-05-16T07:17:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a800b8e0 con 0x7f79c0154180 2023-05-16T07:17:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79bffff700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79a8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79beffd700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79a8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79a80038e0 0x7f79a8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79bffff700 1 -- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 msgr2=0x7f79a8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.559+0000 7f79bffff700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c0154180 0x7f79a8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79bffff700 1 -- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79a80038e0 msgr2=0x7f79a8003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79bffff700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79a80038e0 0x7f79a8003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79bffff700 1 -- 172.21.15.92:0/1727672924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80033f0 con 0x7f79a8004b80 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79bffff700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79a8005000 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f79b8009ab0 tx=0x7f79b8009060).ready entity=mon.0 client_cookie=554ae5177ae29d61 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8009430 con 0x7f79a8004b80 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b800e640 con 0x7f79a8004b80 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79be7fc700 1 -- 172.21.15.92:0/1727672924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b800e9c0 con 0x7f79a8004b80 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79c61cf700 1 -- 172.21.15.92:0/1727672924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 msgr2=0x7f79a8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1727672924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79a8005000 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f79b8009ab0 tx=0x7f79b8009060).stop 2023-05-16T07:17:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79bffff700 1 -- 172.21.15.92:0/1727672924 reap_dead start 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79c61cf700 1 -- 172.21.15.92:0/1727672924 shutdown_connections 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79c61cf700 1 -- 172.21.15.92:0/1727672924 >> 172.21.15.92:0/1727672924 conn(0x7f79c00c8aa0 msgr2=0x7f79c00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79c61cf700 1 -- 172.21.15.92:0/1727672924 shutdown_connections 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.560+0000 7f79c61cf700 1 -- 172.21.15.92:0/1727672924 wait complete. 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 Processor -- start 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 -- start start 2023-05-16T07:17:53.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79c014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79c0150c10 0x7f79c00c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c00c1f20 0x7f79c0159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00c23a0 con 0x7f79a8004b80 2023-05-16T07:17:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0156d40 con 0x7f79c0150c10 2023-05-16T07:17:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79c61cf700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c01578c0 con 0x7f79c00c1f20 2023-05-16T07:17:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79c014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79c014f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46598/0 (socket says 172.21.15.92:46598) 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79bf7fe700 1 -- 172.21.15.92:0/1258870493 learned_addr learned my addr 172.21.15.92:0/1258870493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.562+0000 7f79beffd700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79c0150c10 0x7f79c00c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bffff700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c00c1f20 0x7f79c0159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bf7fe700 1 -- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c00c1f20 msgr2=0x7f79c0159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c00c1f20 0x7f79c0159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bf7fe700 1 -- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79c0150c10 msgr2=0x7f79c00c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79c0150c10 0x7f79c00c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bf7fe700 1 -- 172.21.15.92:0/1258870493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00c0e20 con 0x7f79a8004b80 2023-05-16T07:17:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79c014f9c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f79b0007710 tx=0x7f79b0002210).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79a77fe700 1 -- 172.21.15.92:0/1258870493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000f9a0 con 0x7f79a8004b80 2023-05-16T07:17:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79a77fe700 1 -- 172.21.15.92:0/1258870493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b000fb00 con 0x7f79a8004b80 2023-05-16T07:17:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c0156f90 con 0x7f79a8004b80 2023-05-16T07:17:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79a77fe700 1 -- 172.21.15.92:0/1258870493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000fe80 con 0x7f79a8004b80 2023-05-16T07:17:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.563+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c0157550 con 0x7f79a8004b80 2023-05-16T07:17:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.564+0000 7f79a77fe700 1 -- 172.21.15.92:0/1258870493 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f79b0015620 con 0x7f79a8004b80 2023-05-16T07:17:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.565+0000 7f79a77fe700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f79ac05ed90 0x7f79ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.565+0000 7f79a77fe700 1 -- 172.21.15.92:0/1258870493 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f79b007a4a0 con 0x7f79a8004b80 2023-05-16T07:17:53.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.565+0000 7f79beffd700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f79ac05ed90 0x7f79ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.565+0000 7f79beffd700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f79ac05ed90 0x7f79ac061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f79c015ae50 tx=0x7f79b4008040).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.566+0000 7f79d7b61b80 1 Processor -- start 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 -- start start 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd54560 0x558a0fd37c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 0x558a0fd54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fcd55e0 con 0x558a0fd59480 2023-05-16T07:17:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fc4c8a0 con 0x558a0fd54560 2023-05-16T07:17:53.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.567+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a0fc61900 con 0x558a0fd10300 2023-05-16T07:17:53.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 0x558a0fd54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 0x558a0fd54940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46608/0 (socket says 172.21.15.92:46608) 2023-05-16T07:17:53.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 -- 172.21.15.92:0/1593945854 learned_addr learned my addr 172.21.15.92:0/1593945854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd54560 0x558a0fd37c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79bffff700 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 -- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 msgr2=0x558a0fd0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd0a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 -- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd54560 msgr2=0x558a0fd37c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd54560 0x558a0fd37c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 -- 172.21.15.92:0/1593945854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0fcd5450 con 0x558a0fd59480 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79beffd700 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 0x558a0fd54940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f79b4013ae0 tx=0x7f79b4018000).ready entity=mon.0 client_cookie=22076d33a3079197 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79a4ff9700 1 -- 172.21.15.92:0/1593945854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4024070 con 0x558a0fd59480 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79a4ff9700 1 -- 172.21.15.92:0/1593945854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4013ec0 con 0x558a0fd59480 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.568+0000 7f79a4ff9700 1 -- 172.21.15.92:0/1593945854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4015cd0 con 0x558a0fd59480 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1593945854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 msgr2=0x558a0fd54940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 0x558a0fd54940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f79b4013ae0 tx=0x7f79b4018000).stop 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1593945854 shutdown_connections 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd54560 0x558a0fd37c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1593945854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd59480 0x558a0fd54940 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1593945854 >> 172.21.15.92:0/1593945854 conn(0x558a0facce40 msgr2=0x558a0fd589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1593945854 shutdown_connections 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1593945854 wait complete. 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 -- start start 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.569+0000 7f79d7b61b80 1 Processor -- start 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd5e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd64d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd59480 0x558a0fd51dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fc28ed0 con 0x558a0fd54560 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd5ecb0 con 0x558a0fd59480 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd0b410 con 0x558a0fd10300 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79d7b61b80 10 client.0 ll_register_callbacks cb 0x558a0fccf1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd64d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd64d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46614/0 (socket says 172.21.15.92:46614) 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79beffd700 1 -- 172.21.15.92:0/3806757286 learned_addr learned my addr 172.21.15.92:0/3806757286 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd5e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.570+0000 7f79bffff700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd59480 0x558a0fd51dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79beffd700 1 -- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 msgr2=0x558a0fd5e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79beffd700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd5e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79beffd700 1 -- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd59480 msgr2=0x558a0fd51dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79beffd700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd59480 0x558a0fd51dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79beffd700 1 -- 172.21.15.92:0/3806757286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b400e040 con 0x558a0fd54560 2023-05-16T07:17:53.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79beffd700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd64d10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x558a0fc4d8c0 tx=0x7f79b4024000).ready entity=mon.0 client_cookie=3d7a7868f1054bb6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f799dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:17:53.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f799dffb700 1 -- 172.21.15.92:0/3806757286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4031030 con 0x558a0fd54560 2023-05-16T07:17:53.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f799dffb700 1 -- 172.21.15.92:0/3806757286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f79b4032070 con 0x558a0fd54560 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a0fac8ca0 con 0x558a0fd54560 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f799dffb700 1 -- 172.21.15.92:0/3806757286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b4006440 con 0x558a0fd54560 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 10 client.6756 fetch_fsmap learned FSMap version 31 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x558a0fb04310 con 0x558a0fd54560 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f799dffb700 1 -- 172.21.15.92:0/3806757286 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0fac8ca0 con 0x558a0fd54560 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f799dffb700 1 -- 172.21.15.92:0/3806757286 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f79b401d030 con 0x558a0fd54560 2023-05-16T07:17:53.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 10 client.6756 fetch_fsmap finished waiting for FSMap version 31 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 10 client.6756 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 20 client.6756 populate_metadata read hostname 'smithi092' 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd32ba0 0x558a0fd32f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 4 client.6756 mds_command: new command op to 6594 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 --> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a0faffca0 con 0x558a0fd32ba0 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.571+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd32ba0 0x558a0fd32f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f799dffb700 10 client.6756 ms_handle_connect on v2:172.21.15.92:6833/1656722289 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd32ba0 0x558a0fd32f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2615fd2fc2eb2898 server_cookie=73c0b5ee12cf1fc6 in_seq=0 out_seq=0 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f799dffb700 1 -- 172.21.15.92:0/3806757286 <== mds.0 v2:172.21.15.92:6833/1656722289 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x558a0faffca0 con 0x558a0fd32ba0 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f799dffb700 10 client.6756 handle_command_reply: tid=0 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 client.6756 shutdown 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 msgr2=0x558a0fd64d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd64d10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x558a0fc4d8c0 tx=0x7f79b4024000).stop 2023-05-16T07:17:53.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 shutdown_connections 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd32ba0 0x558a0fd32f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd10300 0x558a0fd5e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f799dffb700 0 client.6756 ms_handle_reset on v2:172.21.15.92:6833/1656722289 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd59480 0x558a0fd51dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.572+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/3806757286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd64d10 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.573+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 >> 172.21.15.92:0/3806757286 conn(0x558a0facce40 msgr2=0x558a0fd0f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.573+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 shutdown_connections 2023-05-16T07:17:53.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.573+0000 7f79d7b61b80 1 -- 172.21.15.92:0/3806757286 wait complete. 2023-05-16T07:17:53.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.573+0000 7f79d7b61b80 20 client.6756 trim_cache size 0 max 16384 2023-05-16T07:17:53.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.589+0000 7f79d7b61b80 1 Processor -- start 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.589+0000 7f79d7b61b80 1 -- start start 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd0f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd51fe0 con 0x558a0fd54560 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd559c0 con 0x558a0fd10300 2023-05-16T07:17:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd0c250 con 0x558a0fd59480 2023-05-16T07:17:53.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd0b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46622/0 (socket says 172.21.15.92:46622) 2023-05-16T07:17:53.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 -- 172.21.15.92:0/2732115503 learned_addr learned my addr 172.21.15.92:0/2732115503 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79bffff700 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd0f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 -- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 msgr2=0x558a0fd0f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd0f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 -- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 msgr2=0x558a0fd53d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd53d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 -- 172.21.15.92:0/2732115503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0fc5a880 con 0x558a0fd54560 2023-05-16T07:17:53.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79beffd700 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd0b550 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f79b400ba40 tx=0x7f79b402f070).ready entity=mon.0 client_cookie=4a2a6bf72583ed12 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79a57fa700 1 -- 172.21.15.92:0/2732115503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4031070 con 0x558a0fd54560 2023-05-16T07:17:53.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79a57fa700 1 -- 172.21.15.92:0/2732115503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4006c90 con 0x558a0fd54560 2023-05-16T07:17:53.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.590+0000 7f79a57fa700 1 -- 172.21.15.92:0/2732115503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4024040 con 0x558a0fd54560 2023-05-16T07:17:53.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 -- 172.21.15.92:0/2732115503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 msgr2=0x558a0fd0b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd0b550 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f79b400ba40 tx=0x7f79b402f070).stop 2023-05-16T07:17:53.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 -- 172.21.15.92:0/2732115503 shutdown_connections 2023-05-16T07:17:53.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd0f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd53d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/2732115503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd0b550 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 -- 172.21.15.92:0/2732115503 >> 172.21.15.92:0/2732115503 conn(0x558a0facce40 msgr2=0x558a0fd552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 -- 172.21.15.92:0/2732115503 shutdown_connections 2023-05-16T07:17:53.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 -- 172.21.15.92:0/2732115503 wait complete. 2023-05-16T07:17:53.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 -- start start 2023-05-16T07:17:53.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.591+0000 7f79d7b61b80 1 Processor -- start 2023-05-16T07:17:53.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd50800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd5d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd537f0 con 0x558a0fd54560 2023-05-16T07:17:53.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd653e0 con 0x558a0fd10300 2023-05-16T07:17:53.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a0fd51030 con 0x558a0fd59480 2023-05-16T07:17:53.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79d7b61b80 10 client.0 ll_register_callbacks cb 0x558a0fccf1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:17:53.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd52d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46626/0 (socket says 172.21.15.92:46626) 2023-05-16T07:17:53.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 -- 172.21.15.92:0/1302882045 learned_addr learned my addr 172.21.15.92:0/1302882045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd50800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79bffff700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd5d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 -- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 msgr2=0x558a0fd5d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd5d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 -- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 msgr2=0x558a0fd50800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd50800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.592+0000 7f79beffd700 1 -- 172.21.15.92:0/1302882045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a0fb04310 con 0x558a0fd54560 2023-05-16T07:17:53.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79beffd700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd52d70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f79b4040040 tx=0x7f79b4031000).ready entity=mon.0 client_cookie=e529a4e08d6c4798 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:17:53.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 1 -- 172.21.15.92:0/1302882045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4024040 con 0x558a0fd54560 2023-05-16T07:17:53.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 1 -- 172.21.15.92:0/1302882045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f79b4015800 con 0x558a0fd54560 2023-05-16T07:17:53.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a0fd03460 con 0x558a0fd54560 2023-05-16T07:17:53.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 1 -- 172.21.15.92:0/1302882045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b4006af0 con 0x558a0fd54560 2023-05-16T07:17:53.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 10 client.6762 fetch_fsmap learned FSMap version 31 2023-05-16T07:17:53.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x558a0fc56070 con 0x558a0fd54560 2023-05-16T07:17:53.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 1 -- 172.21.15.92:0/1302882045 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0fd03460 con 0x558a0fd54560 2023-05-16T07:17:53.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 1 -- 172.21.15.92:0/1302882045 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f79b401d070 con 0x558a0fd54560 2023-05-16T07:17:53.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 10 client.6762 fetch_fsmap finished waiting for FSMap version 31 2023-05-16T07:17:53.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 10 client.6762 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:17:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 20 client.6762 populate_metadata read hostname 'smithi092' 2023-05-16T07:17:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd362c0 0x558a0fd0f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 4 client.6762 mds_command: new command op to 6594 tid=0[{"prefix": "status"}] 2023-05-16T07:17:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 --> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a0fb2e540 con 0x558a0fd362c0 2023-05-16T07:17:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd362c0 0x558a0fd0f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f799dffb700 10 client.6762 ms_handle_connect on v2:172.21.15.92:6833/1656722289 2023-05-16T07:17:53.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.593+0000 7f79bf7fe700 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd362c0 0x558a0fd0f3d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1932d8de4dace751 server_cookie=6ef9f1217aa59243 in_seq=0 out_seq=0 2023-05-16T07:17:53.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f799dffb700 1 -- 172.21.15.92:0/1302882045 <== mds.0 v2:172.21.15.92:6833/1656722289 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558a0fb2e540 con 0x558a0fd362c0 2023-05-16T07:17:53.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f799dffb700 10 client.6762 handle_command_reply: tid=0 2023-05-16T07:17:53.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 client.6762 shutdown 2023-05-16T07:17:53.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 msgr2=0x558a0fd52d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd52d70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f79b4040040 tx=0x7f79b4031000).stop 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 shutdown_connections 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:6833/1656722289,v1:172.21.15.92:6835/1656722289] conn(0x558a0fd362c0 0x558a0fd0f3d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x558a0fd59480 0x558a0fd5d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x558a0fd10300 0x558a0fd50800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 --2- 172.21.15.92:0/1302882045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a0fd54560 0x558a0fd52d70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 >> 172.21.15.92:0/1302882045 conn(0x558a0facce40 msgr2=0x558a0fd0b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f799dffb700 0 client.6762 ms_handle_reset on v2:172.21.15.92:6833/1656722289 2023-05-16T07:17:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 shutdown_connections 2023-05-16T07:17:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 1 -- 172.21.15.92:0/1302882045 wait complete. 2023-05-16T07:17:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79d7b61b80 20 client.6762 trim_cache size 0 max 16384 2023-05-16T07:17:53.626 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:17:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:17:53.626 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:17:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6594, 2023-05-16T07:17:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:17:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:17:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:17:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.2557049689999999, 2023-05-16T07:17:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 31, 2023-05-16T07:17:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 93, 2023-05-16T07:17:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 93, 2023-05-16T07:17:53.628 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.7794418000000007 2023-05-16T07:17:53.628 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:17:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f79ac05ed90 msgr2=0x7f79ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f79ac05ed90 0x7f79ac061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f79c015ae50 tx=0x7f79b4008040).stop 2023-05-16T07:17:53.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 msgr2=0x7f79c014f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79c014f9c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f79b0007710 tx=0x7f79b0002210).stop 2023-05-16T07:17:53.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 shutdown_connections 2023-05-16T07:17:53.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f79ac05ed90 0x7f79ac061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f79c00c1f20 0x7f79c0159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79a8004b80 0x7f79c014f9c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 --2- 172.21.15.92:0/1258870493 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f79c0150c10 0x7f79c00c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 >> 172.21.15.92:0/1258870493 conn(0x7f79c00c8aa0 msgr2=0x7f79c00c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 shutdown_connections 2023-05-16T07:17:53.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.594+0000 7f79c61cf700 1 -- 172.21.15.92:0/1258870493 wait complete. 2023-05-16T07:17:53.631 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:17:53.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.727+0000 7fc349131700 1 Processor -- start 2023-05-16T07:17:53.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.727+0000 7fc349131700 1 -- start start 2023-05-16T07:17:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc349131700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc344151bf0 0x7fc3441546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc349131700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440c3800 0x7fc3440beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc349131700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3440bdf50 con 0x7fc3440bda50 2023-05-16T07:17:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc349131700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440be230 con 0x7fc344151bf0 2023-05-16T07:17:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc349131700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440be510 con 0x7fc3440c3800 2023-05-16T07:17:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc344151bf0 0x7fc3441546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc344151bf0 0x7fc3441546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46634/0 (socket says 172.21.15.92:46634) 2023-05-16T07:17:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 -- 172.21.15.92:0/3847021676 learned_addr learned my addr 172.21.15.92:0/3847021676 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 -- 172.21.15.92:0/3847021676 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440c3800 msgr2=0x7fc3440beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 --2- 172.21.15.92:0/3847021676 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440c3800 0x7fc3440beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 -- 172.21.15.92:0/3847021676 >> v1:172.21.15.158:6789/0 conn(0x7fc3440bda50 legacy=0x7fc3440c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 -- 172.21.15.92:0/3847021676 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440be7f0 con 0x7fc344151bf0 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc342d9d700 1 --2- 172.21.15.92:0/3847021676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3441546e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7fc334006fd0 tx=0x7fc3340090f0).ready entity=mon.0 client_cookie=291de55e2f0414a5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc34259c700 1 -- 172.21.15.92:0/3847021676 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33400b600 con 0x7fc344151bf0 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc34259c700 1 -- 172.21.15.92:0/3847021676 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33400b760 con 0x7fc344151bf0 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.728+0000 7fc34259c700 1 -- 172.21.15.92:0/3847021676 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33400bae0 con 0x7fc344151bf0 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 -- 172.21.15.92:0/3847021676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 msgr2=0x7fc3441546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 --2- 172.21.15.92:0/3847021676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3441546e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7fc334006fd0 tx=0x7fc3340090f0).stop 2023-05-16T07:17:53.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 -- 172.21.15.92:0/3847021676 shutdown_connections 2023-05-16T07:17:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 --2- 172.21.15.92:0/3847021676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3441546e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 --2- 172.21.15.92:0/3847021676 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440c3800 0x7fc3440beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 -- 172.21.15.92:0/3847021676 >> 172.21.15.92:0/3847021676 conn(0x7fc344062db0 msgr2=0x7fc3440c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 -- 172.21.15.92:0/3847021676 shutdown_connections 2023-05-16T07:17:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.729+0000 7fc349131700 1 -- 172.21.15.92:0/3847021676 wait complete. 2023-05-16T07:17:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.730+0000 7fc349131700 1 Processor -- start 2023-05-16T07:17:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.730+0000 7fc349131700 1 -- start start 2023-05-16T07:17:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440bda50 0x7fc34413ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc3440c3800 0x7fc344139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3440ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ae080 con 0x7fc344151bf0 2023-05-16T07:17:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ad470 con 0x7fc3440c3800 2023-05-16T07:17:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ad750 con 0x7fc3440bda50 2023-05-16T07:17:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3440ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3440ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46642/0 (socket says 172.21.15.92:46642) 2023-05-16T07:17:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 -- 172.21.15.92:0/1542839790 learned_addr learned my addr 172.21.15.92:0/1542839790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc33bfff700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc3440c3800 0x7fc344139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc342d9d700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440bda50 0x7fc34413ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 -- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440bda50 msgr2=0x7fc34413ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440bda50 0x7fc34413ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 -- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc3440c3800 msgr2=0x7fc344139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc3440c3800 0x7fc344139880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 -- 172.21.15.92:0/1542839790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc334009370 con 0x7fc344151bf0 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc33bfff700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc3440c3800 0x7fc344139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc342d9d700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440bda50 0x7fc34413ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc34359e700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3440ae510 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fc33c00bc30 tx=0x7fc33c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00c7a0 con 0x7fc344151bf0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c010070 con 0x7fc344151bf0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00c960 con 0x7fc344151bf0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3440ada30 con 0x7fc344151bf0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.731+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34413b360 con 0x7fc344151bf0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.732+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34405efa0 con 0x7fc344151bf0 2023-05-16T07:17:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.733+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc33c01c070 con 0x7fc344151bf0 2023-05-16T07:17:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.733+0000 7fc33affd700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc32405ed90 0x7fc324061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.733+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fc33c0794e0 con 0x7fc344151bf0 2023-05-16T07:17:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.733+0000 7fc342d9d700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc32405ed90 0x7fc324061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.734+0000 7fc342d9d700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc32405ed90 0x7fc324061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc334006fd0 tx=0x7fc334017000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.737+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc33c04a830 con 0x7fc344151bf0 2023-05-16T07:17:53.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.946+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc3440bead0 con 0x7fc344151bf0 2023-05-16T07:17:53.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.947+0000 7fc33affd700 1 -- 172.21.15.92:0/1542839790 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc33c01a070 con 0x7fc344151bf0 2023-05-16T07:17:53.951 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:17:53.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.948+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc32405ed90 msgr2=0x7fc324061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.948+0000 7fc349131700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc32405ed90 0x7fc324061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc334006fd0 tx=0x7fc334017000).stop 2023-05-16T07:17:53.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.948+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 msgr2=0x7fc3440ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:53.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.948+0000 7fc349131700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3440ae510 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fc33c00bc30 tx=0x7fc33c009840).stop 2023-05-16T07:17:53.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 shutdown_connections 2023-05-16T07:17:53.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc32405ed90 0x7fc324061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440bda50 0x7fc34413ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bf0 0x7fc3440ae510 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 --2- 172.21.15.92:0/1542839790 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc3440c3800 0x7fc344139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:53.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 >> 172.21.15.92:0/1542839790 conn(0x7fc344062db0 msgr2=0x7fc344068190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:53.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 shutdown_connections 2023-05-16T07:17:53.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:53.949+0000 7fc349131700 1 -- 172.21.15.92:0/1542839790 wait complete. 2023-05-16T07:17:53.963 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:17:53.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:17:53.964 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:17:53.964 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:17:53.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:53.965 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:17:53.965 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:17:53.965 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:17:53.965 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:17:53.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:53.966 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:17:53.981 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:17:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:17:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41081sec preferred_lft 41081sec 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:17:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:17:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:17:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:17:53.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:53.986 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:17:54.044 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:17:54.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:54.046 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:17:54.101 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:17:54.101 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:17:54.102 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:17:54.102 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:17:54.102 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:17:54.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:54.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:17:54.103 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:17:54.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:17:54.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:17:54.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:17:54.104 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:17:54.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:54.200 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:17:54.214 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:17:54.214 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:17:54.214 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:17:54.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:54.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:17:54.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:54.278 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:54.331 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:17:54.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:17:54.409 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:17:54.409 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:17:54.452 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:17:54.452 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:17:54.454 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:17:54.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:17:54.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:17:54.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:17:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.667+0000 7f6acb59e700 1 Processor -- start 2023-05-16T07:17:54.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.668+0000 7f6acb59e700 1 -- start start 2023-05-16T07:17:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.668+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 0x7f6acc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6acb59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc06a920 con 0x7f6acc0aab40 2023-05-16T07:17:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6acb59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0bfa30 con 0x7f6acc153b00 2023-05-16T07:17:54.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6acb59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0c65f0 con 0x7f6acc069750 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6acc0aab40 0x7f6acc0c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57404/0 (socket says 172.21.15.92:57404) 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac9d9b700 1 -- 172.21.15.92:0/2526447831 learned_addr learned my addr 172.21.15.92:0/2526447831 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667871291 0 0) 0x7f6acc06a920 con 0x7f6acc0aab40 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab80033f0 con 0x7f6acc0aab40 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6aca59c700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 0x7f6acc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012232053 0 0) 0x7f6ab80033f0 con 0x7f6acc0aab40 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 msgr2=0x7f6acc069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:54.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 0x7f6acc069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 >> v1:172.21.15.158:6789/0 conn(0x7f6acc153b00 legacy=0x7f6acc153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0bfc80 con 0x7f6acc0aab40 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6ab4001470 con 0x7f6acc0aab40 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 >> v1:172.21.15.92:6789/0 conn(0x7f6acc0aab40 legacy=0x7f6acc0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab8003910 0x7f6ab8003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ab8004690 0x7f6ab8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 0x7f6ab8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0c65f0 con 0x7f6ab8003910 2023-05-16T07:17:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab800b8e0 con 0x7f6ab8004690 2023-05-16T07:17:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab800ba50 con 0x7f6acc069750 2023-05-16T07:17:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.669+0000 7f6aca59c700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab8003910 0x7f6ab8003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6ac9d9b700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ab8004690 0x7f6ab8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6acad9d700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 0x7f6ab8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6aca59c700 1 -- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 msgr2=0x7f6ab8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6aca59c700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6acc069750 0x7f6ab8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6aca59c700 1 -- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ab8004690 msgr2=0x7f6ab8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6aca59c700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ab8004690 0x7f6ab8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6aca59c700 1 -- 172.21.15.92:0/2526447831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80033f0 con 0x7f6ab8003910 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6aca59c700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab8003910 0x7f6ab8003cf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f6abc00e1c0 tx=0x7f6abc00b040).ready entity=mon.0 client_cookie=17d265af8cda8d51 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:54.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc009ac0 con 0x7f6ab8003910 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc00f040 con 0x7f6ab8003910 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6ac959a700 1 -- 172.21.15.92:0/2526447831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00ec40 con 0x7f6ab8003910 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6acb59e700 1 -- 172.21.15.92:0/2526447831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab8003910 msgr2=0x7f6ab8003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6acb59e700 1 --2- 172.21.15.92:0/2526447831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab8003910 0x7f6ab8003cf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f6abc00e1c0 tx=0x7f6abc00b040).stop 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6acad9d700 1 -- 172.21.15.92:0/2526447831 reap_dead start 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6acb59e700 1 -- 172.21.15.92:0/2526447831 shutdown_connections 2023-05-16T07:17:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.670+0000 7f6acb59e700 1 -- 172.21.15.92:0/2526447831 >> 172.21.15.92:0/2526447831 conn(0x7f6acc0cb550 msgr2=0x7f6acc0c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.671+0000 7f6acb59e700 1 -- 172.21.15.92:0/2526447831 shutdown_connections 2023-05-16T07:17:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.671+0000 7f6acb59e700 1 -- 172.21.15.92:0/2526447831 wait complete. 2023-05-16T07:17:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 Processor -- start 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 -- start start 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ab8003910 0x7f6acc14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6acc0ae920 0x7f6acc167960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 0x7f6acc0ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc157ab0 con 0x7f6acc0c3820 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc168220 con 0x7f6acc0ae920 2023-05-16T07:17:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acb59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc168500 con 0x7f6ab8003910 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 0x7f6acc0ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 0x7f6acc0ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46668/0 (socket says 172.21.15.92:46668) 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6acad9d700 1 -- 172.21.15.92:0/1132577234 learned_addr learned my addr 172.21.15.92:0/1132577234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6ac9d9b700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6acc0ae920 0x7f6acc167960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.672+0000 7f6aca59c700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ab8003910 0x7f6acc14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acad9d700 1 -- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ab8003910 msgr2=0x7f6acc14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acad9d700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ab8003910 0x7f6acc14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acad9d700 1 -- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6acc0ae920 msgr2=0x7f6acc167960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acad9d700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6acc0ae920 0x7f6acc167960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acad9d700 1 -- 172.21.15.92:0/1132577234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc157d50 con 0x7f6acc0c3820 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acad9d700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 0x7f6acc0ac460 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f6acc153ee0 tx=0x7f6ac4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400ee40 con 0x7f6acc0c3820 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac4014430 con 0x7f6acc0c3820 2023-05-16T07:17:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc1687e0 con 0x7f6acc0c3820 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40147b0 con 0x7f6acc0c3820 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.673+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc167e50 con 0x7f6acc0c3820 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.674+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc083040 con 0x7f6acc0c3820 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.674+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6ac4014b20 con 0x7f6acc0c3820 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.675+0000 7f6ac27fc700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aac05ed90 0x7f6aac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.675+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f6ac4079780 con 0x7f6acc0c3820 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.675+0000 7f6aca59c700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aac05ed90 0x7f6aac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.675+0000 7f6aca59c700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aac05ed90 0x7f6aac061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6abc00f010 tx=0x7f6abc01a000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.679+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6ac40498f0 con 0x7f6acc0c3820 2023-05-16T07:17:54.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.873+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6acc157f40 con 0x7f6aac05ed90 2023-05-16T07:17:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.874+0000 7f6ac27fc700 1 -- 172.21.15.92:0/1132577234 <== mgr.6019 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6acc157f40 con 0x7f6aac05ed90 2023-05-16T07:17:54.879 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:17:54.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:17:54.879 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:17:54.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:17:54.880 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:17:54.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aac05ed90 msgr2=0x7f6aac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aac05ed90 0x7f6aac061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6abc00f010 tx=0x7f6abc01a000).stop 2023-05-16T07:17:54.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 msgr2=0x7f6acc0ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:54.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 0x7f6acc0ac460 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f6acc153ee0 tx=0x7f6ac4009040).stop 2023-05-16T07:17:54.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 shutdown_connections 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aac05ed90 0x7f6aac061250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ab8003910 0x7f6acc14e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6acc0c3820 0x7f6acc0ac460 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 --2- 172.21.15.92:0/1132577234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6acc0ae920 0x7f6acc167960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 >> 172.21.15.92:0/1132577234 conn(0x7f6acc0cb550 msgr2=0x7f6acc0aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 shutdown_connections 2023-05-16T07:17:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:54.876+0000 7f6acb59e700 1 -- 172.21.15.92:0/1132577234 wait complete. 2023-05-16T07:17:54.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:17:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.001+0000 7f6ecc438700 1 Processor -- start 2023-05-16T07:17:55.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.001+0000 7f6ecc438700 1 -- start start 2023-05-16T07:17:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ecc438700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ec4154470 0x7f6ec40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ecc438700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec40c94d0 con 0x7f6ec4068160 2023-05-16T07:17:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ecc438700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec40cb7e0 con 0x7f6ec4154090 2023-05-16T07:17:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ecc438700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40c88c0 con 0x7f6ec4154470 2023-05-16T07:17:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec99d3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ec4154470 0x7f6ec40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6eca9d5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ec4068160 0x7f6ec40c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57422/0 (socket says 172.21.15.92:57422) 2023-05-16T07:17:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec99d3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ec4154470 0x7f6ec40aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46676/0 (socket says 172.21.15.92:46676) 2023-05-16T07:17:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6eca9d5700 1 -- 172.21.15.92:0/1711524022 learned_addr learned my addr 172.21.15.92:0/1711524022 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec99d3700 1 -- 172.21.15.92:0/1711524022 >> v1:172.21.15.158:6789/0 conn(0x7f6ec4154090 legacy=0x7f6ec40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec99d3700 1 -- 172.21.15.92:0/1711524022 >> v1:172.21.15.92:6789/0 conn(0x7f6ec4068160 legacy=0x7f6ec40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec91d2700 1 -- 172.21.15.92:0/1711524022 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861374910 0 0) 0x7f6ec40c94d0 con 0x7f6ec4068160 2023-05-16T07:17:55.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec99d3700 1 -- 172.21.15.92:0/1711524022 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40c8ba0 con 0x7f6ec4154470 2023-05-16T07:17:55.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec99d3700 1 --2- 172.21.15.92:0/1711524022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4154470 0x7f6ec40aaca0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f6eb8009760 tx=0x7f6eb8009060).ready entity=mon.0 client_cookie=a38598bfa1cadac7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec91d2700 1 -- 172.21.15.92:0/1711524022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800b580 con 0x7f6ec4154470 2023-05-16T07:17:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec91d2700 1 -- 172.21.15.92:0/1711524022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb800b6e0 con 0x7f6ec4154470 2023-05-16T07:17:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.002+0000 7f6ec91d2700 1 -- 172.21.15.92:0/1711524022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800ba60 con 0x7f6ec4154470 2023-05-16T07:17:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 -- 172.21.15.92:0/1711524022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4154470 msgr2=0x7f6ec40aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1711524022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4154470 0x7f6ec40aaca0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f6eb8009760 tx=0x7f6eb8009060).stop 2023-05-16T07:17:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 -- 172.21.15.92:0/1711524022 shutdown_connections 2023-05-16T07:17:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1711524022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4154470 0x7f6ec40aaca0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 -- 172.21.15.92:0/1711524022 >> 172.21.15.92:0/1711524022 conn(0x7f6ec4067920 msgr2=0x7f6ec40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 -- 172.21.15.92:0/1711524022 shutdown_connections 2023-05-16T07:17:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.003+0000 7f6ecc438700 1 -- 172.21.15.92:0/1711524022 wait complete. 2023-05-16T07:17:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.004+0000 7f6ecc438700 1 Processor -- start 2023-05-16T07:17:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.004+0000 7f6ecc438700 1 -- start start 2023-05-16T07:17:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 0x7f6ec4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ec4154090 0x7f6ec40aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ec4154470 0x7f6ec413f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4155cb0 con 0x7f6ec4068160 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4155f90 con 0x7f6ec4154470 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41550a0 con 0x7f6ec4154090 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 0x7f6ec4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 0x7f6ec4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46678/0 (socket says 172.21.15.92:46678) 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 -- 172.21.15.92:0/1932671422 learned_addr learned my addr 172.21.15.92:0/1932671422 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 -- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ec4154090 msgr2=0x7f6ec40aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ec4154090 0x7f6ec40aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 -- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ec4154470 msgr2=0x7f6ec413f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ec4154470 0x7f6ec413f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 -- 172.21.15.92:0/1932671422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80092a0 con 0x7f6ec4068160 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6eca1d4700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 0x7f6ec4154850 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f6eb400c870 tx=0x7f6eb4001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4006a20 con 0x7f6ec4068160 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4006b80 con 0x7f6ec4068160 2023-05-16T07:17:55.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec4155380 con 0x7f6ec4068160 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.005+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4007020 con 0x7f6ec4068160 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.006+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec4154d40 con 0x7f6ec4068160 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.006+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec4066730 con 0x7f6ec4068160 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.007+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 31) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6eb4018020 con 0x7f6ec4068160 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.007+0000 7f6ec27fc700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6eb005ed90 0x7f6eb0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.007+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f6eb40793e0 con 0x7f6ec4068160 2023-05-16T07:17:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.007+0000 7f6ec99d3700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6eb005ed90 0x7f6eb0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.008+0000 7f6ec99d3700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6eb005ed90 0x7f6eb0061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6ec40c0c50 tx=0x7f6eb8017000).ready entity=mgr.6019 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:55.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.011+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6eb40488f0 con 0x7f6ec4068160 2023-05-16T07:17:55.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.216+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6ec406c350 con 0x7f6ec4068160 2023-05-16T07:17:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.650+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 32) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f6eb4010070 con 0x7f6ec4068160 2023-05-16T07:17:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.651+0000 7f6ec27fc700 1 -- 172.21.15.92:0/1932671422 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v32) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6eb4073830 con 0x7f6ec4068160 2023-05-16T07:17:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6eb005ed90 msgr2=0x7f6eb0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6eb005ed90 0x7f6eb0061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6ec40c0c50 tx=0x7f6eb8017000).stop 2023-05-16T07:17:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 msgr2=0x7f6ec4154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 0x7f6ec4154850 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f6eb400c870 tx=0x7f6eb4001660).stop 2023-05-16T07:17:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 shutdown_connections 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6eb005ed90 0x7f6eb0061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ec4154470 0x7f6ec413f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec4068160 0x7f6ec4154850 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 --2- 172.21.15.92:0/1932671422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ec4154090 0x7f6ec40aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 >> 172.21.15.92:0/1932671422 conn(0x7f6ec4067920 msgr2=0x7f6ec40c11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 shutdown_connections 2023-05-16T07:17:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.653+0000 7f6ecc438700 1 -- 172.21.15.92:0/1932671422 wait complete. 2023-05-16T07:17:55.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:17:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.784+0000 7f13cf258700 1 Processor -- start 2023-05-16T07:17:55.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.784+0000 7f13cf258700 1 -- start start 2023-05-16T07:17:55.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13cf258700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154090 0x7f13c80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13cf258700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80c94d0 con 0x7f13c8154470 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13cf258700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80cb7e0 con 0x7f13c8068160 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13cf258700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80c88c0 con 0x7f13c8154090 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f13c8154470 0x7f13c80a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57460/0 (socket says 172.21.15.92:57460) 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c7fff700 1 -- 172.21.15.92:0/2359144655 learned_addr learned my addr 172.21.15.92:0/2359144655 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13ccff4700 1 --2- 172.21.15.92:0/2359144655 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154090 0x7f13c80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437142653 0 0) 0x7f13c80c94d0 con 0x7f13c8154470 2023-05-16T07:17:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b80033f0 con 0x7f13c8154470 2023-05-16T07:17:55.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887874937 0 0) 0x7f13c80cb7e0 con 0x7f13c8068160 2023-05-16T07:17:55.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c80c94d0 con 0x7f13c8068160 2023-05-16T07:17:55.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3918205484 0 0) 0x7f13b80033f0 con 0x7f13c8154470 2023-05-16T07:17:55.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13ccff4700 1 -- 172.21.15.92:0/2359144655 >> v1:172.21.15.158:6789/0 conn(0x7f13c8068160 legacy=0x7f13c8068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13ccff4700 1 -- 172.21.15.92:0/2359144655 >> v1:172.21.15.92:6789/0 conn(0x7f13c8154470 legacy=0x7f13c80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13ccff4700 1 -- 172.21.15.92:0/2359144655 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80c8ba0 con 0x7f13c8154090 2023-05-16T07:17:55.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.786+0000 7f13ccff4700 1 --2- 172.21.15.92:0/2359144655 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154090 0x7f13c80aaca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f13bc009790 tx=0x7f13bc009060).ready entity=mon.2 client_cookie=5de243e7ec28bff6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:55.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.787+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4001470 con 0x7f13c8154090 2023-05-16T07:17:55.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.787+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc011400 con 0x7f13c8154090 2023-05-16T07:17:55.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.787+0000 7f13c77fe700 1 -- 172.21.15.92:0/2359144655 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc011780 con 0x7f13c8154090 2023-05-16T07:17:55.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.787+0000 7f13cf258700 1 -- 172.21.15.92:0/2359144655 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154090 msgr2=0x7f13c80aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.787+0000 7f13cf258700 1 --2- 172.21.15.92:0/2359144655 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154090 0x7f13c80aaca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f13bc009790 tx=0x7f13bc009060).stop 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.788+0000 7f13cf258700 1 -- 172.21.15.92:0/2359144655 shutdown_connections 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.788+0000 7f13cf258700 1 --2- 172.21.15.92:0/2359144655 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154090 0x7f13c80aaca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.788+0000 7f13cf258700 1 -- 172.21.15.92:0/2359144655 >> 172.21.15.92:0/2359144655 conn(0x7f13c8067920 msgr2=0x7f13c80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.788+0000 7f13cf258700 1 -- 172.21.15.92:0/2359144655 shutdown_connections 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.788+0000 7f13cf258700 1 -- 172.21.15.92:0/2359144655 wait complete. 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.789+0000 7f13cf258700 1 Processor -- start 2023-05-16T07:17:55.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.790+0000 7f13cf258700 1 -- start start 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cf258700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 0x7f13c8168a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cf258700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f13c8154090 0x7f13c80a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cf258700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154470 0x7f13c80c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cf258700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81685f0 con 0x7f13c8068160 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cf258700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8167fc0 con 0x7f13c8154090 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cf258700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80b0850 con 0x7f13c8154470 2023-05-16T07:17:55.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 0x7f13c8168a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 0x7f13c8168a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46718/0 (socket says 172.21.15.92:46718) 2023-05-16T07:17:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 -- 172.21.15.92:0/3462459982 learned_addr learned my addr 172.21.15.92:0/3462459982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13cd7f5700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154470 0x7f13c80c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13c7fff700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f13c8154090 0x7f13c80a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 -- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154470 msgr2=0x7f13c80c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154470 0x7f13c80c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 -- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f13c8154090 msgr2=0x7f13c80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f13c8154090 0x7f13c80a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 -- 172.21.15.92:0/3462459982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc0092a0 con 0x7f13c8068160 2023-05-16T07:17:55.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.791+0000 7f13ccff4700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 0x7f13c8168a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f13c806bb30 tx=0x7f13bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:55.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.792+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0106c0 con 0x7f13c8068160 2023-05-16T07:17:55.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.792+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc010840 con 0x7f13c8068160 2023-05-16T07:17:55.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.792+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c80b0b30 con 0x7f13c8068160 2023-05-16T07:17:55.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.792+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc010c00 con 0x7f13c8068160 2023-05-16T07:17:55.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.792+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c80b0470 con 0x7f13c8068160 2023-05-16T07:17:55.801 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:55.791+0000 7fe1ae7d9680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:55.801 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:55.792+0000 7f0b9e9ee680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:17:55.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.793+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c81688d0 con 0x7f13c8068160 2023-05-16T07:17:55.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.793+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 32) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f13bc00f070 con 0x7f13c8068160 2023-05-16T07:17:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.793+0000 7f13c4ff9700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f13ac05ede0 0x7f13ac0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.793+0000 7f13c7fff700 1 -- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f13ac05ede0 msgr2=0x7f13ac0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.793+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f13bc07b0b0 con 0x7f13c8068160 2023-05-16T07:17:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.793+0000 7f13c7fff700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f13ac05ede0 0x7f13ac0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.797+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f13bc04c400 con 0x7f13c8068160 2023-05-16T07:17:55.858 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:55.854+0000 7fe1ae7d9680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:55.861 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:55.857+0000 7f0b9e9ee680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:17:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.967+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f13c814ea70 con 0x7f13c8068160 2023-05-16T07:17:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.971+0000 7f13c4ff9700 1 -- 172.21.15.92:0/3462459982 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v15)=0 v15) v1 ==== 138+0+0 (secure 0 0 0) 0x7f13bc04fa20 con 0x7f13c8068160 2023-05-16T07:17:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f13ac05ede0 msgr2=0x7f13ac0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f13ac05ede0 0x7f13ac0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 msgr2=0x7f13c8168a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 0x7f13c8168a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f13c806bb30 tx=0x7f13bc016000).stop 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 shutdown_connections 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f13ac05ede0 0x7f13ac0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f13c8154470 0x7f13c80c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c8068160 0x7f13c8168a80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 --2- 172.21.15.92:0/3462459982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f13c8154090 0x7f13c80a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 >> 172.21.15.92:0/3462459982 conn(0x7f13c8067920 msgr2=0x7f13c814e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 shutdown_connections 2023-05-16T07:17:55.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:55.973+0000 7f13cf258700 1 -- 172.21.15.92:0/3462459982 wait complete. 2023-05-16T07:17:55.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-05-16T07:17:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.104+0000 7fa7e6481700 1 Processor -- start 2023-05-16T07:17:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.105+0000 7fa7e6481700 1 -- start start 2023-05-16T07:17:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e6481700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa7e0153f90 0x7fa7e00a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e6481700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c5d00 0x7fa7e00c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e6481700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00c6670 con 0x7fa7e00c0150 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e6481700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e0069f60 con 0x7fa7e0153f90 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e6481700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e006a9e0 con 0x7fa7e00c5d00 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e547f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa7e0153f90 0x7fa7e00a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e4c7e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c5d00 0x7fa7e00c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e5c80700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fa7e00c0150 0x7fa7e00c05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:32886/0 (socket says 172.21.15.92:32886) 2023-05-16T07:17:56.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e5c80700 1 -- 172.21.15.92:0/11078490 learned_addr learned my addr 172.21.15.92:0/11078490 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:17:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7d3fff700 1 -- 172.21.15.92:0/11078490 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815844912 0 0) 0x7fa7e00c6670 con 0x7fa7e00c0150 2023-05-16T07:17:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7d3fff700 1 -- 172.21.15.92:0/11078490 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7cc0033f0 con 0x7fa7e00c0150 2023-05-16T07:17:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e547f700 1 -- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c5d00 msgr2=0x7fa7e00c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:17:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e547f700 1 --2- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c5d00 0x7fa7e00c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e547f700 1 -- 172.21.15.92:0/11078490 >> v1:172.21.15.158:6789/0 conn(0x7fa7e00c0150 legacy=0x7fa7e00c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e547f700 1 -- 172.21.15.92:0/11078490 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e006a140 con 0x7fa7e0153f90 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c5d00 0x7fa7e00c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.106+0000 7fa7e547f700 1 --2- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e0153f90 0x7fa7e00a9d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa7d4009760 tx=0x7fa7d4009060).ready entity=mon.1 client_cookie=71dc7e963e85580b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7d3fff700 1 -- 172.21.15.92:0/11078490 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400d4a0 con 0x7fa7e0153f90 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7d3fff700 1 -- 172.21.15.92:0/11078490 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d400d600 con 0x7fa7e0153f90 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7d3fff700 1 -- 172.21.15.92:0/11078490 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400d980 con 0x7fa7e0153f90 2023-05-16T07:17:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 -- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e0153f90 msgr2=0x7fa7e00a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 --2- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e0153f90 0x7fa7e00a9d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa7d4009760 tx=0x7fa7d4009060).stop 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 -- 172.21.15.92:0/11078490 shutdown_connections 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 --2- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e0153f90 0x7fa7e00a9d20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 --2- 172.21.15.92:0/11078490 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c5d00 0x7fa7e00c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 -- 172.21.15.92:0/11078490 >> 172.21.15.92:0/11078490 conn(0x7fa7e00cb600 msgr2=0x7fa7e00c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 -- 172.21.15.92:0/11078490 shutdown_connections 2023-05-16T07:17:56.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.107+0000 7fa7e6481700 1 -- 172.21.15.92:0/11078490 wait complete. 2023-05-16T07:17:56.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.108+0000 7fa7e6481700 1 Processor -- start 2023-05-16T07:17:56.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.108+0000 7fa7e6481700 1 -- start start 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e6481700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c0150 0x7fa7e0154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e6481700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e00c5d00 0x7fa7e0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e6481700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e0153f90 0x7fa7e00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e6481700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0165e60 con 0x7fa7e0153f90 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e6481700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0165250 con 0x7fa7e00c5d00 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e6481700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0165530 con 0x7fa7e00c0150 2023-05-16T07:17:56.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e5c80700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e0153f90 0x7fa7e00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e547f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c0150 0x7fa7e0154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e547f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c0150 0x7fa7e0154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:39348/0 (socket says 172.21.15.92:39348) 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e5c80700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e0153f90 0x7fa7e00b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46780/0 (socket says 172.21.15.92:46780) 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e547f700 1 -- 172.21.15.92:0/2684954003 learned_addr learned my addr 172.21.15.92:0/2684954003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.109+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e00c5d00 0x7fa7e0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e547f700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e00c5d00 msgr2=0x7fa7e0150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:56.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e547f700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e00c5d00 0x7fa7e0150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e547f700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e0153f90 msgr2=0x7fa7e00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e547f700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e0153f90 0x7fa7e00b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e547f700 1 -- 172.21.15.92:0/2684954003 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0092a0 con 0x7fa7e00c0150 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e5c80700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e0153f90 0x7fa7e00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e547f700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa7e00c0150 0x7fa7e0154510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fa7d4009d50 tx=0x7fa7d4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4000c90 con 0x7fa7e00c0150 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d400d770 con 0x7fa7e00c0150 2023-05-16T07:17:56.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d401d430 con 0x7fa7e00c0150 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e6481700 1 -- 172.21.15.92:0/2684954003 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d40092a0 con 0x7fa7e00c0150 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.110+0000 7fa7e6481700 1 -- 172.21.15.92:0/2684954003 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e0165af0 con 0x7fa7e00c0150 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.111+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 32) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fa7d400f070 con 0x7fa7e00c0150 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.111+0000 7fa7e6481700 1 -- 172.21.15.92:0/2684954003 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e0083120 con 0x7fa7e00c0150 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.112+0000 7fa7d17fa700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.112+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fa7d407b5f0 con 0x7fa7e00c0150 2023-05-16T07:17:56.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.112+0000 7fa7e4c7e700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 msgr2=0x7fa7c40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:56.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.112+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:17:56.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.116+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa7d404c940 con 0x7fa7e00c0150 2023-05-16T07:17:56.131 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:56.127+0000 7fe1ae7d9680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:56.158 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:56.154+0000 7f0b9e9ee680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:17:56.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.313+0000 7fa7e4c7e700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 msgr2=0x7fa7c40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:56.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.313+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:17:56.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.321+0000 7fa7e6481700 1 -- 172.21.15.92:0/2684954003 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e0066770 con 0x7fa7c405ede0 2023-05-16T07:17:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.729+0000 7fa7e4c7e700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 msgr2=0x7fa7c40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:56.729+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:17:56.736 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:56.732+0000 7fe1ae7d9680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:56.766 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:56.762+0000 7f0b9e9ee680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:17:56.807 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:56.803+0000 7fe1ae7d9680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:56.836 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:56.833+0000 7f0b9e9ee680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:17:56.991 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:56.987+0000 7fe1ae7d9680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:57.020 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:57.016+0000 7f0b9e9ee680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:17:57.402 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:57.398+0000 7fe1ae7d9680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:57.429 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:57.426+0000 7f0b9e9ee680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:17:57.504 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:57.499+0000 7fe1ae7d9680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:57.530 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:57.526+0000 7f0b9e9ee680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:57.530+0000 7fa7e4c7e700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 msgr2=0x7fa7c40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:57.530+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:17:57.655 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:57.650+0000 7fe1ae7d9680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:57.680 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:57.676+0000 7f0b9e9ee680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:57.728 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:57.724+0000 7fe1ae7d9680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:57.753 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:57.749+0000 7f0b9e9ee680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:17:57.793 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:57.789+0000 7fe1ae7d9680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:57.819 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:57.815+0000 7f0b9e9ee680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:17:58.046 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:58.041+0000 7fe1ae7d9680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:58.070 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:58.067+0000 7f0b9e9ee680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:17:58.226 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:58.221+0000 7fe1ae7d9680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:58.249 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:58.245+0000 7f0b9e9ee680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:17:58.866 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:58.861+0000 7fe1ae7d9680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:58.885 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:58.881+0000 7f0b9e9ee680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:17:58.933 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:58.928+0000 7fe1ae7d9680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:58.951 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:58.948+0000 7f0b9e9ee680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:17:59.005 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.001+0000 7fe1ae7d9680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:59.023 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.020+0000 7f0b9e9ee680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:17:59.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:59.131+0000 7fa7e4c7e700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 msgr2=0x7fa7c40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:17:59.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:59.132+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:17:59.147 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.143+0000 7fe1ae7d9680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:59.166 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.162+0000 7f0b9e9ee680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:17:59.216 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.211+0000 7fe1ae7d9680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:59.234 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.231+0000 7f0b9e9ee680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:17:59.328 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.324+0000 7fe1ae7d9680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:59.346 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.343+0000 7f0b9e9ee680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:17:59.433 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.428+0000 7fe1ae7d9680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:59.452 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.448+0000 7f0b9e9ee680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:17:59.720 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.715+0000 7fe1ae7d9680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:59.738 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.735+0000 7f0b9e9ee680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:17:59.789 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.785+0000 7fe1ae7d9680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:59.808 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:17:59.804+0000 7f0b9e9ee680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:17:59.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:59.858+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mgrmap(e 33) v1 ==== 75109+0+0 (secure 0 0 0) 0x7fa7d40792a0 con 0x7fa7e00c0150 2023-05-16T07:17:59.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:59.858+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 msgr2=0x7fa7c40612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:17:59.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:17:59.858+0000 7fa7d17fa700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:17:59.922 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe12cfbb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.923 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe12cfbb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.923 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe12cfbb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.923 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe12cfbb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.923 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe12cfbb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.923 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe129fb5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.924 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe129fb5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.924 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe129fb5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.924 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe129fb5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:17:59.924 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:17:59.918+0000 7fe129fb5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:00.871+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mon.2 v2:172.21.15.158:3301/0 8 ==== mgrmap(e 34) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fa7d4079620 con 0x7fa7e00c0150 2023-05-16T07:18:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:00.871+0000 7fa7d17fa700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c402fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:00.871+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7c4065370 con 0x7fa7c405ede0 2023-05-16T07:18:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:00.873+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c402fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:00.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:00.890+0000 7fa7e4c7e700 1 --2- 172.21.15.92:0/2684954003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa7c405ede0 0x7fa7c402fd40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fa7e00af350 tx=0x7fa7dc009060).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:00.895 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:00.891+0000 7fe15c264700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-05-16T07:18:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:00.892+0000 7fa7d17fa700 1 -- 172.21.15.92:0/2684954003 <== mgr.5864 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fa7c4065370 con 0x7fa7c405ede0 2023-05-16T07:18:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-05-16T07:18:00.900 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-05-16T07:18:00.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:18:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 Processor -- start 2023-05-16T07:18:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 -- start start 2023-05-16T07:18:01.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac153f90 0x7fc7ac0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc7ac0c5d00 0x7fc7ac0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7ac0c0150 0x7fc7ac0c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac069f60 con 0x7fc7ac0c0150 2023-05-16T07:18:01.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac06a9e0 con 0x7fc7ac0c5d00 2023-05-16T07:18:01.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.013+0000 7fc7b0e8d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0c6670 con 0x7fc7ac153f90 2023-05-16T07:18:01.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7ac0c0150 0x7fc7ac0c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7ac0c0150 0x7fc7ac0c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46928/0 (socket says 172.21.15.92:46928) 2023-05-16T07:18:01.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 -- 172.21.15.92:0/2994885707 learned_addr learned my addr 172.21.15.92:0/2994885707 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7ab7fe700 1 --2- 172.21.15.92:0/2994885707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac153f90 0x7fc7ac0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7aaffd700 1 --2- 172.21.15.92:0/2994885707 >> v2:172.21.15.158:3300/0 conn(0x7fc7ac0c5d00 0x7fc7ac0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 -- 172.21.15.92:0/2994885707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac153f90 msgr2=0x7fc7ac0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 --2- 172.21.15.92:0/2994885707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac153f90 0x7fc7ac0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 -- 172.21.15.92:0/2994885707 >> v2:172.21.15.158:3300/0 conn(0x7fc7ac0c5d00 msgr2=0x7fc7ac0c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 --2- 172.21.15.92:0/2994885707 >> v2:172.21.15.158:3300/0 conn(0x7fc7ac0c5d00 0x7fc7ac0c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 -- 172.21.15.92:0/2994885707 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac06a140 con 0x7fc7ac0c0150 2023-05-16T07:18:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7abfff700 1 --2- 172.21.15.92:0/2994885707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac0c05c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7fc7ac083000 tx=0x7fc7a4009840).ready entity=mon.0 client_cookie=16e795ab20881d93 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7aa7fc700 1 -- 172.21.15.92:0/2994885707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400e070 con 0x7fc7ac0c0150 2023-05-16T07:18:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7aa7fc700 1 -- 172.21.15.92:0/2994885707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a400ba50 con 0x7fc7ac0c0150 2023-05-16T07:18:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.014+0000 7fc7aa7fc700 1 -- 172.21.15.92:0/2994885707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400bdd0 con 0x7fc7ac0c0150 2023-05-16T07:18:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/2994885707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 msgr2=0x7fc7ac0c05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/2994885707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac0c05c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7fc7ac083000 tx=0x7fc7a4009840).stop 2023-05-16T07:18:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/2994885707 shutdown_connections 2023-05-16T07:18:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/2994885707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac0c05c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/2994885707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac153f90 0x7fc7ac0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/2994885707 >> v2:172.21.15.158:3300/0 conn(0x7fc7ac0c5d00 0x7fc7ac0c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/2994885707 >> 172.21.15.92:0/2994885707 conn(0x7fc7ac0cb600 msgr2=0x7fc7ac0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/2994885707 shutdown_connections 2023-05-16T07:18:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.015+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/2994885707 wait complete. 2023-05-16T07:18:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.016+0000 7fc7b0e8d700 1 Processor -- start 2023-05-16T07:18:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.016+0000 7fc7b0e8d700 1 -- start start 2023-05-16T07:18:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac0c5d00 0x7fc7ac151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7ac153f90 0x7fc7ac0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac152210 con 0x7fc7ac0c0150 2023-05-16T07:18:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0aea40 con 0x7fc7ac153f90 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0ade50 con 0x7fc7ac0c5d00 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46930/0 (socket says 172.21.15.92:46930) 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 -- 172.21.15.92:0/1056718094 learned_addr learned my addr 172.21.15.92:0/1056718094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7aaffd700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac0c5d00 0x7fc7ac151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7abfff700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7ac153f90 0x7fc7ac0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 -- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac0c5d00 msgr2=0x7fc7ac151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac0c5d00 0x7fc7ac151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 -- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7ac153f90 msgr2=0x7fc7ac0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7ac153f90 0x7fc7ac0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 -- 172.21.15.92:0/1056718094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009ae0 con 0x7fc7ac0c0150 2023-05-16T07:18:01.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7aaffd700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac0c5d00 0x7fc7ac151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:01.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7abfff700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7ac153f90 0x7fc7ac0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:01.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7ab7fe700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac151190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fc79400d860 tx=0x7fc794008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794011590 con 0x7fc7ac0c0150 2023-05-16T07:18:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7940116f0 con 0x7fc7ac0c0150 2023-05-16T07:18:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794011a70 con 0x7fc7ac0c0150 2023-05-16T07:18:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac151c50 con 0x7fc7ac0c0150 2023-05-16T07:18:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.017+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac0ae6d0 con 0x7fc7ac0c0150 2023-05-16T07:18:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.018+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac0ae530 con 0x7fc7ac0c0150 2023-05-16T07:18:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.019+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 34) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fc794012420 con 0x7fc7ac0c0150 2023-05-16T07:18:01.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.019+0000 7fc7a37fe700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc78c05ed40 0x7fc78c061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:01.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.019+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fc794079940 con 0x7fc7ac0c0150 2023-05-16T07:18:01.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.019+0000 7fc7aaffd700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc78c05ed40 0x7fc78c061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.020+0000 7fc7aaffd700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc78c05ed40 0x7fc78c061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc7ac0bffe0 tx=0x7fc79c008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.023+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc7ac0ae530 con 0x7fc7ac0c0150 2023-05-16T07:18:01.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.228+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc7ac0af7e0 con 0x7fc7ac0c0150 2023-05-16T07:18:01.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.873+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fc7940775f0 con 0x7fc7ac0c0150 2023-05-16T07:18:01.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.874+0000 7fc7a37fe700 1 -- 172.21.15.92:0/1056718094 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc794072d90 con 0x7fc7ac0c0150 2023-05-16T07:18:01.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc78c05ed40 msgr2=0x7fc78c061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:01.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc78c05ed40 0x7fc78c061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc7ac0bffe0 tx=0x7fc79c008040).stop 2023-05-16T07:18:01.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 msgr2=0x7fc7ac151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:01.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac151190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fc79400d860 tx=0x7fc794008040).stop 2023-05-16T07:18:01.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 shutdown_connections 2023-05-16T07:18:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc78c05ed40 0x7fc78c061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc7ac153f90 0x7fc7ac0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ac0c0150 0x7fc7ac151190 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 --2- 172.21.15.92:0/1056718094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc7ac0c5d00 0x7fc7ac151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 >> 172.21.15.92:0/1056718094 conn(0x7fc7ac0cb600 msgr2=0x7fc7ac0c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 shutdown_connections 2023-05-16T07:18:01.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:01.876+0000 7fc7b0e8d700 1 -- 172.21.15.92:0/1056718094 wait complete. 2023-05-16T07:18:01.893 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:18:01.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:18:01.920 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:18:01.949 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:18:01.949 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:18:01.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:01.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:18:01.994 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:18:01.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:01.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:18:02.015 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:02.011+0000 7f3a4d667680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:18:02.021 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:02.017+0000 7f9dcf306680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:18:02.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:02.079 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:02.075+0000 7f3a4d667680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:18:02.082 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:02.077+0000 7f9dcf306680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:18:02.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.181+0000 7fe7be3b8700 1 Processor -- start 2023-05-16T07:18:02.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.181+0000 7fe7be3b8700 1 -- start start 2023-05-16T07:18:02.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7be3b8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe7b8154d50 0x7fe7b80a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7be3b8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8067560 0x7fe7b80a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7be3b8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b8068750 con 0x7fe7b8155130 2023-05-16T07:18:02.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7be3b8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b80a5260 con 0x7fe7b8154d50 2023-05-16T07:18:02.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7be3b8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8067b40 con 0x7fe7b8067560 2023-05-16T07:18:02.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7bcbb5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7b8155130 0x7fe7b8066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57750/0 (socket says 172.21.15.92:57750) 2023-05-16T07:18:02.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7bcbb5700 1 -- 172.21.15.92:0/4260191314 learned_addr learned my addr 172.21.15.92:0/4260191314 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:02.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7bd3b6700 1 --2- 172.21.15.92:0/4260191314 >> v2:172.21.15.158:3300/0 conn(0x7fe7b8154d50 0x7fe7b80a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7affff700 1 -- 172.21.15.92:0/4260191314 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179992423 0 0) 0x7fe7b8068750 con 0x7fe7b8155130 2023-05-16T07:18:02.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.182+0000 7fe7affff700 1 -- 172.21.15.92:0/4260191314 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a40033f0 con 0x7fe7b8155130 2023-05-16T07:18:02.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7bdbb7700 1 --2- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8067560 0x7fe7b80a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8067560 msgr2=0x7fe7b80a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:02.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7bd3b6700 1 --2- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8067560 0x7fe7b80a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/4260191314 >> v1:172.21.15.92:6789/0 conn(0x7fe7b8155130 legacy=0x7fe7b8066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/4260191314 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b8067e20 con 0x7fe7b8154d50 2023-05-16T07:18:02.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7bd3b6700 1 --2- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80a4880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fe7a8009760 tx=0x7fe7a8009060).ready entity=mon.1 client_cookie=32e7fdb46de36b31 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:02.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7affff700 1 -- 172.21.15.92:0/4260191314 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80090d0 con 0x7fe7b8154d50 2023-05-16T07:18:02.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7affff700 1 -- 172.21.15.92:0/4260191314 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800f510 con 0x7fe7b8154d50 2023-05-16T07:18:02.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7affff700 1 -- 172.21.15.92:0/4260191314 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800f890 con 0x7fe7b8154d50 2023-05-16T07:18:02.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7be3b8700 1 -- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 msgr2=0x7fe7b80a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.183+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80a4880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fe7a8009760 tx=0x7fe7a8009060).stop 2023-05-16T07:18:02.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.184+0000 7fe7be3b8700 1 -- 172.21.15.92:0/4260191314 shutdown_connections 2023-05-16T07:18:02.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.184+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8067560 0x7fe7b80a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.184+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/4260191314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80a4880 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.184+0000 7fe7be3b8700 1 -- 172.21.15.92:0/4260191314 >> 172.21.15.92:0/4260191314 conn(0x7fe7b8069940 msgr2=0x7fe7b80c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.184+0000 7fe7be3b8700 1 -- 172.21.15.92:0/4260191314 shutdown_connections 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.184+0000 7fe7be3b8700 1 -- 172.21.15.92:0/4260191314 wait complete. 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.185+0000 7fe7be3b8700 1 Processor -- start 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.185+0000 7fe7be3b8700 1 -- start start 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 0x7fe7b814a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8155130 0x7fe7b80ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80a7e80 con 0x7fe7b8067560 2023-05-16T07:18:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80acf00 con 0x7fe7b8154d50 2023-05-16T07:18:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80ac940 con 0x7fe7b8155130 2023-05-16T07:18:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 0x7fe7b814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 0x7fe7b814a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47006/0 (socket says 172.21.15.92:47006) 2023-05-16T07:18:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/2229313615 learned_addr learned my addr 172.21.15.92:0/2229313615 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bcbb5700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bdbb7700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8155130 0x7fe7b80ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8155130 msgr2=0x7fe7b80ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8155130 0x7fe7b80ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 msgr2=0x7fe7b80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 -- 172.21.15.92:0/2229313615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80092a0 con 0x7fe7b8067560 2023-05-16T07:18:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7bd3b6700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 0x7fe7b814a630 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fe7a8009e00 tx=0x7fe7a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800f890 con 0x7fe7b8067560 2023-05-16T07:18:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800fc50 con 0x7fe7b8067560 2023-05-16T07:18:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b80acc20 con 0x7fe7b8067560 2023-05-16T07:18:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.186+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a801b410 con 0x7fe7b8067560 2023-05-16T07:18:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.187+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b814ab20 con 0x7fe7b8067560 2023-05-16T07:18:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.188+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fe7a800f070 con 0x7fe7b8067560 2023-05-16T07:18:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.188+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b8047150 con 0x7fe7b8067560 2023-05-16T07:18:02.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.188+0000 7fe7ad7fa700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 0x7fe79c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.188+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fe7a807b7d0 con 0x7fe7b8067560 2023-05-16T07:18:02.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.188+0000 7fe7bcbb5700 1 -- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 msgr2=0x7fe79c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:02.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.188+0000 7fe7bcbb5700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 0x7fe79c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:02.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.196+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe7a804c9b0 con 0x7fe7b8067560 2023-05-16T07:18:02.349 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:02.344+0000 7f9dcf306680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:18:02.362 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:02.359+0000 7f3a4d667680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:18:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.389+0000 7fe7bcbb5700 1 -- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 msgr2=0x7fe79c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:02.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.389+0000 7fe7bcbb5700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 0x7fe79c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:02.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.407+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7b8068100 con 0x7fe7b8067560 2023-05-16T07:18:02.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.408+0000 7fe7ad7fa700 1 -- 172.21.15.92:0/2229313615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fe7a8028020 con 0x7fe7b8067560 2023-05-16T07:18:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:02.412 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:48.337476+0000","modified":"2023-05-16T07:17:50.983467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1656722289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1656722289},{"type":"v1","addr":"172.21.15.92:6835","nonce":1656722289}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-05-16T07:18:02.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 msgr2=0x7fe79c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 0x7fe79c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 msgr2=0x7fe7b814a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 0x7fe7b814a630 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fe7a8009e00 tx=0x7fe7a8017000).stop 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 shutdown_connections 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79c05ede0 0x7fe79c0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7b8155130 0x7fe7b80ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7b8067560 0x7fe7b814a630 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 --2- 172.21.15.92:0/2229313615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7b8154d50 0x7fe7b80abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 >> 172.21.15.92:0/2229313615 conn(0x7fe7b8069940 msgr2=0x7fe7b80c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:02.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 shutdown_connections 2023-05-16T07:18:02.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.410+0000 7fe7be3b8700 1 -- 172.21.15.92:0/2229313615 wait complete. 2023-05-16T07:18:02.415 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-05-16T07:18:02.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.575+0000 7fbf79913700 1 Processor -- start 2023-05-16T07:18:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.575+0000 7fbf79913700 1 -- start start 2023-05-16T07:18:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf79913700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbf7414e5e0 0x7fbf74157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf79913700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf7415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf79913700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf740683d0 con 0x7fbf7414e200 2023-05-16T07:18:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf79913700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf74137470 con 0x7fbf7414e5e0 2023-05-16T07:18:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf79913700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740c3520 con 0x7fbf74157f40 2023-05-16T07:18:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf727fc700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbf7414e5e0 0x7fbf74157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf737fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf7415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.576+0000 7fbf72ffd700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fbf7414e200 0x7fbf74066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:33176/0 (socket says 172.21.15.92:33176) 2023-05-16T07:18:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf72ffd700 1 -- 172.21.15.92:0/513961304 learned_addr learned my addr 172.21.15.92:0/513961304 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf727fc700 1 -- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 msgr2=0x7fbf7415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf727fc700 1 --2- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf7415ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf727fc700 1 -- 172.21.15.92:0/513961304 >> v1:172.21.15.158:6789/0 conn(0x7fbf7414e200 legacy=0x7fbf74066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf727fc700 1 -- 172.21.15.92:0/513961304 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740be060 con 0x7fbf7414e5e0 2023-05-16T07:18:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf71ffb700 1 -- 172.21.15.92:0/513961304 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910581552 0 0) 0x7fbf740683d0 con 0x7fbf7414e200 2023-05-16T07:18:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf737fe700 1 --2- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf7415ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:18:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf727fc700 1 --2- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e5e0 0x7fbf74157a50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fbf64009760 tx=0x7fbf64009060).ready entity=mon.1 client_cookie=dc6b8192ba65202 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf71ffb700 1 -- 172.21.15.92:0/513961304 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400f510 con 0x7fbf7414e5e0 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf71ffb700 1 -- 172.21.15.92:0/513961304 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6400f670 con 0x7fbf7414e5e0 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.577+0000 7fbf71ffb700 1 -- 172.21.15.92:0/513961304 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400f9f0 con 0x7fbf7414e5e0 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 -- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e5e0 msgr2=0x7fbf74157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 --2- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e5e0 0x7fbf74157a50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fbf64009760 tx=0x7fbf64009060).stop 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 -- 172.21.15.92:0/513961304 shutdown_connections 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 --2- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf7415ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 --2- 172.21.15.92:0/513961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e5e0 0x7fbf74157a50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 -- 172.21.15.92:0/513961304 >> 172.21.15.92:0/513961304 conn(0x7fbf740c7bc0 msgr2=0x7fbf740c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:02.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 -- 172.21.15.92:0/513961304 shutdown_connections 2023-05-16T07:18:02.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.578+0000 7fbf79913700 1 -- 172.21.15.92:0/513961304 wait complete. 2023-05-16T07:18:02.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.579+0000 7fbf79913700 1 Processor -- start 2023-05-16T07:18:02.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.579+0000 7fbf79913700 1 -- start start 2023-05-16T07:18:02.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf79913700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e200 0x7fbf740abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf79913700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 0x7fbf740ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf79913700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf740acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf79913700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740ad0c0 con 0x7fbf7414e5e0 2023-05-16T07:18:02.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf79913700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740ac2b0 con 0x7fbf7414e200 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf79913700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740a8d50 con 0x7fbf74157f40 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 0x7fbf740ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 0x7fbf740ac6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47068/0 (socket says 172.21.15.92:47068) 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 -- 172.21.15.92:0/1368273949 learned_addr learned my addr 172.21.15.92:0/1368273949 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf72ffd700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e200 0x7fbf740abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf737fe700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf740acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 -- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 msgr2=0x7fbf740acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf740acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 -- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e200 msgr2=0x7fbf740abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e200 0x7fbf740abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 -- 172.21.15.92:0/1368273949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c009a20 con 0x7fbf7414e5e0 2023-05-16T07:18:02.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.580+0000 7fbf727fc700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 0x7fbf740ac6e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fbf64009e00 tx=0x7fbf64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:02.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.581+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400f7e0 con 0x7fbf7414e5e0 2023-05-16T07:18:02.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.581+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6400f940 con 0x7fbf7414e5e0 2023-05-16T07:18:02.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.581+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf640092a0 con 0x7fbf7414e5e0 2023-05-16T07:18:02.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.581+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6401b660 con 0x7fbf7414e5e0 2023-05-16T07:18:02.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.581+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf740a92d0 con 0x7fbf7414e5e0 2023-05-16T07:18:02.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.581+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf74155590 con 0x7fbf7414e5e0 2023-05-16T07:18:02.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.582+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fbf6402b070 con 0x7fbf7414e5e0 2023-05-16T07:18:02.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.582+0000 7fbf5affd700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf6005edb0 0x7fbf60061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.582+0000 7fbf72ffd700 1 -- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf6005edb0 msgr2=0x7fbf60061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:02.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.582+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fbf6407b660 con 0x7fbf7414e5e0 2023-05-16T07:18:02.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.582+0000 7fbf72ffd700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf6005edb0 0x7fbf60061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:02.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.587+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbf6404c840 con 0x7fbf7414e5e0 2023-05-16T07:18:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.756+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf7406c350 con 0x7fbf7414e5e0 2023-05-16T07:18:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.757+0000 7fbf5affd700 1 -- 172.21.15.92:0/1368273949 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18108 (secure 0 0 0) 0x7fbf6401f090 con 0x7fbf7414e5e0 2023-05-16T07:18:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:02.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:59.800152+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:46.874223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:47.303318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/2540760732":"2023-05-17T07:17:59.800121+0000","172.21.15.158:6833/86301":"2023-05-17T07:17:59.800121+0000","172.21.15.158:6832/86301":"2023-05-17T07:17:59.800121+0000","172.21.15.158:0/3814686992":"2023-05-17T07:17:59.800121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf6005edb0 msgr2=0x7fbf60061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf6005edb0 0x7fbf60061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 msgr2=0x7fbf740ac6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 0x7fbf740ac6e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fbf64009e00 tx=0x7fbf64017000).stop 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 shutdown_connections 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbf6005edb0 0x7fbf60061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf74157f40 0x7fbf740acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf7414e200 0x7fbf740abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 --2- 172.21.15.92:0/1368273949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7414e5e0 0x7fbf740ac6e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 >> 172.21.15.92:0/1368273949 conn(0x7fbf740c7bc0 msgr2=0x7fbf741369d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:02.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 shutdown_connections 2023-05-16T07:18:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.759+0000 7fbf79913700 1 -- 172.21.15.92:0/1368273949 wait complete. 2023-05-16T07:18:02.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:17:46.874223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:17:47.303318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:18:02.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:18:02.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.893+0000 7f628af1e700 1 Processor -- start 2023-05-16T07:18:02.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.894+0000 7f628af1e700 1 -- start start 2023-05-16T07:18:02.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.894+0000 7f628af1e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f628413b420 0x7f62840be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.894+0000 7f628af1e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.894+0000 7f628af1e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840622b0 con 0x7f6284067340 2023-05-16T07:18:02.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.894+0000 7f628af1e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6284063110 con 0x7f628413b420 2023-05-16T07:18:02.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.894+0000 7f628af1e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62840677d0 con 0x7f62840be590 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6288cba700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f628413b420 0x7f62840be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f62894bb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6284067340 0x7f62840692b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57822/0 (socket says 172.21.15.92:57822) 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f62894bb700 1 -- 172.21.15.92:0/3760316858 learned_addr learned my addr 172.21.15.92:0/3760316858 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6283fff700 1 --2- 172.21.15.92:0/3760316858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6288cba700 1 -- 172.21.15.92:0/3760316858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 msgr2=0x7f6284066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6288cba700 1 --2- 172.21.15.92:0/3760316858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6288cba700 1 -- 172.21.15.92:0/3760316858 >> v1:172.21.15.92:6789/0 conn(0x7f6284067340 legacy=0x7f62840692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f62837fe700 1 -- 172.21.15.92:0/3760316858 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743414820 0 0) 0x7f62840622b0 con 0x7f6284067340 2023-05-16T07:18:02.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6288cba700 1 -- 172.21.15.92:0/3760316858 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284063600 con 0x7f628413b420 2023-05-16T07:18:02.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f6288cba700 1 --2- 172.21.15.92:0/3760316858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840be0a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f6278009760 tx=0x7f6278009060).ready entity=mon.0 client_cookie=587dd9717aa750ad server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:02.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f62837fe700 1 -- 172.21.15.92:0/3760316858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627800b580 con 0x7f628413b420 2023-05-16T07:18:02.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f62837fe700 1 -- 172.21.15.92:0/3760316858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627800b6e0 con 0x7f628413b420 2023-05-16T07:18:02.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.895+0000 7f62837fe700 1 -- 172.21.15.92:0/3760316858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627800ba60 con 0x7f628413b420 2023-05-16T07:18:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 -- 172.21.15.92:0/3760316858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 msgr2=0x7f62840be0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 --2- 172.21.15.92:0/3760316858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840be0a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f6278009760 tx=0x7f6278009060).stop 2023-05-16T07:18:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 -- 172.21.15.92:0/3760316858 shutdown_connections 2023-05-16T07:18:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 --2- 172.21.15.92:0/3760316858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840be0a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 --2- 172.21.15.92:0/3760316858 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 -- 172.21.15.92:0/3760316858 >> 172.21.15.92:0/3760316858 conn(0x7f62840c8470 msgr2=0x7f62840c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 -- 172.21.15.92:0/3760316858 shutdown_connections 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.896+0000 7f628af1e700 1 -- 172.21.15.92:0/3760316858 wait complete. 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.897+0000 7f628af1e700 1 Processor -- start 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.897+0000 7f628af1e700 1 -- start start 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f628af1e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6284067340 0x7f62840ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f628af1e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f628af1e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840691e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f628af1e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840c2ca0 con 0x7f628413b420 2023-05-16T07:18:02.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f628af1e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840c2f80 con 0x7f6284067340 2023-05-16T07:18:02.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f628af1e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62840a4a20 con 0x7f62840be590 2023-05-16T07:18:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47082/0 (socket says 172.21.15.92:47082) 2023-05-16T07:18:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 -- 172.21.15.92:0/1573775803 learned_addr learned my addr 172.21.15.92:0/1573775803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f6288cba700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6284067340 0x7f62840ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 -- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 msgr2=0x7f6284138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 -- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6284067340 msgr2=0x7f62840ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6284067340 0x7f62840ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 -- 172.21.15.92:0/1573775803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780092a0 con 0x7f628413b420 2023-05-16T07:18:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f62894bb700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840691e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f627c00c7e0 tx=0x7f627c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.898+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c005d30 con 0x7f628413b420 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.899+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c005e90 con 0x7f628413b420 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.899+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62840a4d00 con 0x7f628413b420 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.899+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c007320 con 0x7f628413b420 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.899+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62840ac770 con 0x7f628413b420 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.899+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62840a4880 con 0x7f628413b420 2023-05-16T07:18:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.900+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f627c010030 con 0x7f628413b420 2023-05-16T07:18:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.900+0000 7f6280ff9700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 0x7f62680612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.900+0000 7f6288cba700 1 -- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 msgr2=0x7f62680612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.900+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f627c079dd0 con 0x7f628413b420 2023-05-16T07:18:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.900+0000 7f6288cba700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 0x7f62680612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:02.904+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f62840a4880 con 0x7f628413b420 2023-05-16T07:18:02.953 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:02.945+0000 7f9dcf306680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:18:03.009 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:03.006+0000 7f3a4d667680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:18:03.020 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:03.015+0000 7f9dcf306680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:18:03.075 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:03.071+0000 7f3a4d667680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:18:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.101+0000 7f6288cba700 1 -- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 msgr2=0x7f62680612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.101+0000 7f6288cba700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 0x7f62680612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.106+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f62840a4880 con 0x7f628413b420 2023-05-16T07:18:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.106+0000 7f6280ff9700 1 -- 172.21.15.92:0/1573775803 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7f62840a4880 con 0x7f628413b420 2023-05-16T07:18:03.110 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:18:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 msgr2=0x7f62680612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 0x7f62680612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 msgr2=0x7f62840691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840691e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f627c00c7e0 tx=0x7f627c007f90).stop 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 shutdown_connections 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f626805ede0 0x7f62680612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f628413b420 0x7f62840691e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6284067340 0x7f62840ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 --2- 172.21.15.92:0/1573775803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f62840be590 0x7f6284138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.108+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 >> 172.21.15.92:0/1573775803 conn(0x7f62840c8470 msgr2=0x7f62840a7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:03.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.109+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 shutdown_connections 2023-05-16T07:18:03.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.109+0000 7f628af1e700 1 -- 172.21.15.92:0/1573775803 wait complete. 2023-05-16T07:18:03.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:18:03.202 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:03.197+0000 7f9dcf306680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:18:03.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.240+0000 7facb871a700 1 Processor -- start 2023-05-16T07:18:03.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.240+0000 7facb871a700 1 -- start start 2023-05-16T07:18:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.241+0000 7facb871a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7facb0155c70 0x7facb0156050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.241+0000 7facb871a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb0069e60 0x7facb006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.241+0000 7facb871a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb0064260 con 0x7facb014f460 2023-05-16T07:18:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.241+0000 7facb871a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00689a0 con 0x7facb0155c70 2023-05-16T07:18:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.241+0000 7facb871a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb00683e0 con 0x7facb0069e60 2023-05-16T07:18:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb64b6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7facb0155c70 0x7facb0156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb5cb5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb0069e60 0x7facb006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb6cb7700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7facb014f460 0x7facb014f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:33206/0 (socket says 172.21.15.92:33206) 2023-05-16T07:18:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb6cb7700 1 -- 172.21.15.92:0/236541444 learned_addr learned my addr 172.21.15.92:0/236541444 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb64b6700 1 -- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb0069e60 msgr2=0x7facb006a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:03.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb64b6700 1 --2- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb0069e60 0x7facb006a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb64b6700 1 -- 172.21.15.92:0/236541444 >> v1:172.21.15.158:6789/0 conn(0x7facb014f460 legacy=0x7facb014f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb64b6700 1 -- 172.21.15.92:0/236541444 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb00686c0 con 0x7facb0155c70 2023-05-16T07:18:03.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb54b4700 1 -- 172.21.15.92:0/236541444 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154950005 0 0) 0x7facb0064260 con 0x7facb014f460 2023-05-16T07:18:03.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb5cb5700 1 --2- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb0069e60 0x7facb006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:18:03.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb64b6700 1 --2- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0156050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7facac009760 tx=0x7facac009060).ready entity=mon.1 client_cookie=29aba83ae2b58019 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:03.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb54b4700 1 -- 172.21.15.92:0/236541444 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00f510 con 0x7facb0155c70 2023-05-16T07:18:03.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb54b4700 1 -- 172.21.15.92:0/236541444 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac00f670 con 0x7facb0155c70 2023-05-16T07:18:03.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.242+0000 7facb54b4700 1 -- 172.21.15.92:0/236541444 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00f9f0 con 0x7facb0155c70 2023-05-16T07:18:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 -- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 msgr2=0x7facb0156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 --2- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0156050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7facac009760 tx=0x7facac009060).stop 2023-05-16T07:18:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 -- 172.21.15.92:0/236541444 shutdown_connections 2023-05-16T07:18:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 --2- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0156050 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 --2- 172.21.15.92:0/236541444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb0069e60 0x7facb006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 -- 172.21.15.92:0/236541444 >> 172.21.15.92:0/236541444 conn(0x7facb00c3d90 msgr2=0x7facb00c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 -- 172.21.15.92:0/236541444 shutdown_connections 2023-05-16T07:18:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.243+0000 7facb871a700 1 -- 172.21.15.92:0/236541444 wait complete. 2023-05-16T07:18:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.244+0000 7facb871a700 1 Processor -- start 2023-05-16T07:18:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.244+0000 7facb871a700 1 -- start start 2023-05-16T07:18:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 0x7facb00abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb014f460 0x7facb0163c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0164140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb0167220 con 0x7facb0069e60 2023-05-16T07:18:03.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb0164760 con 0x7facb0155c70 2023-05-16T07:18:03.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb0164a40 con 0x7facb014f460 2023-05-16T07:18:03.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 0x7facb00abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 0x7facb00abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47094/0 (socket says 172.21.15.92:47094) 2023-05-16T07:18:03.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 -- 172.21.15.92:0/3162097684 learned_addr learned my addr 172.21.15.92:0/3162097684 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb6cb7700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0164140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb5cb5700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb014f460 0x7facb0163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 -- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb014f460 msgr2=0x7facb0163c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb014f460 0x7facb0163c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 -- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 msgr2=0x7facb0164140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0164140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 -- 172.21.15.92:0/3162097684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca00092a0 con 0x7facb0069e60 2023-05-16T07:18:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb64b6700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 0x7facb00abcc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7facac009e00 tx=0x7facac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00f7e0 con 0x7facb0069e60 2023-05-16T07:18:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac00f940 con 0x7facb0069e60 2023-05-16T07:18:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac0092a0 con 0x7facb0069e60 2023-05-16T07:18:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac01b660 con 0x7facb0069e60 2023-05-16T07:18:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.245+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb01677a0 con 0x7facb0069e60 2023-05-16T07:18:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.247+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb006c350 con 0x7facb0069e60 2023-05-16T07:18:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.247+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7facac00f070 con 0x7facb0069e60 2023-05-16T07:18:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.247+0000 7faca67fc700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fac9405ede0 0x7fac940612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.247+0000 7facb5cb5700 1 -- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fac9405ede0 msgr2=0x7fac940612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:03.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.247+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7facac07b580 con 0x7facb0069e60 2023-05-16T07:18:03.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.247+0000 7facb5cb5700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fac9405ede0 0x7fac940612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:03.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.251+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7facac04c760 con 0x7facb0069e60 2023-05-16T07:18:03.268 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:03.265+0000 7f3a4d667680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:18:03.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.420+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7facb00c7240 con 0x7facb0069e60 2023-05-16T07:18:03.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.420+0000 7faca67fc700 1 -- 172.21.15.92:0/3162097684 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7facac04fd80 con 0x7facb0069e60 2023-05-16T07:18:03.424 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:18:03.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fac9405ede0 msgr2=0x7fac940612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:03.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fac9405ede0 0x7fac940612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 msgr2=0x7facb00abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 0x7facb00abcc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7facac009e00 tx=0x7facac017000).stop 2023-05-16T07:18:03.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 shutdown_connections 2023-05-16T07:18:03.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fac9405ede0 0x7fac940612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7facb0155c70 0x7facb0164140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facb0069e60 0x7facb00abcc0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 --2- 172.21.15.92:0/3162097684 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7facb014f460 0x7facb0163c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 >> 172.21.15.92:0/3162097684 conn(0x7facb00c3d90 msgr2=0x7facb0153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 shutdown_connections 2023-05-16T07:18:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.422+0000 7facb871a700 1 -- 172.21.15.92:0/3162097684 wait complete. 2023-05-16T07:18:03.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:18:03.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.553+0000 7fb88ee6b700 1 Processor -- start 2023-05-16T07:18:03.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.553+0000 7fb88ee6b700 1 -- start start 2023-05-16T07:18:03.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88ee6b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb88814ffa0 0x7fb888150380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88ee6b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb8880bd810 0x7fb888153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88ee6b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8880bc770 con 0x7fb8880bc290 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88ee6b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88806a510 con 0x7fb8880bd810 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88ee6b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880bca50 con 0x7fb88814ffa0 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88d408700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb8880bd810 0x7fb888153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb887fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb8880bc290 0x7fb888069e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57894/0 (socket says 172.21.15.92:57894) 2023-05-16T07:18:03.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb887fff700 1 -- 172.21.15.92:0/1941295606 learned_addr learned my addr 172.21.15.92:0/1941295606 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.554+0000 7fb88cc07700 1 --2- 172.21.15.92:0/1941295606 >> v2:172.21.15.158:3300/0 conn(0x7fb88814ffa0 0x7fb888150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88d408700 1 -- 172.21.15.92:0/1941295606 >> v2:172.21.15.158:3300/0 conn(0x7fb88814ffa0 msgr2=0x7fb888150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88d408700 1 --2- 172.21.15.92:0/1941295606 >> v2:172.21.15.158:3300/0 conn(0x7fb88814ffa0 0x7fb888150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88d408700 1 -- 172.21.15.92:0/1941295606 >> v1:172.21.15.92:6789/0 conn(0x7fb8880bc290 legacy=0x7fb888069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb8877fe700 1 -- 172.21.15.92:0/1941295606 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862274269 0 0) 0x7fb8880bc770 con 0x7fb8880bc290 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88d408700 1 -- 172.21.15.92:0/1941295606 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880bcd30 con 0x7fb8880bd810 2023-05-16T07:18:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88d408700 1 --2- 172.21.15.92:0/1941295606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8880bd810 0x7fb888153eb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fb88000af70 tx=0x7fb88000a890).ready entity=mon.0 client_cookie=ad1687c08ffd7de1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:03.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb8877fe700 1 -- 172.21.15.92:0/1941295606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000f070 con 0x7fb8880bd810 2023-05-16T07:18:03.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb8877fe700 1 -- 172.21.15.92:0/1941295606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000a2e0 con 0x7fb8880bd810 2023-05-16T07:18:03.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb8877fe700 1 -- 172.21.15.92:0/1941295606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000cbf0 con 0x7fb8880bd810 2023-05-16T07:18:03.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1941295606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8880bd810 msgr2=0x7fb888153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1941295606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8880bd810 0x7fb888153eb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fb88000af70 tx=0x7fb88000a890).stop 2023-05-16T07:18:03.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1941295606 shutdown_connections 2023-05-16T07:18:03.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1941295606 >> v2:172.21.15.158:3300/0 conn(0x7fb88814ffa0 0x7fb888150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1941295606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8880bd810 0x7fb888153eb0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.555+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1941295606 >> 172.21.15.92:0/1941295606 conn(0x7fb8880c4620 msgr2=0x7fb888066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.556+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1941295606 shutdown_connections 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.556+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1941295606 wait complete. 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.557+0000 7fb88ee6b700 1 Processor -- start 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.557+0000 7fb88ee6b700 1 -- start start 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8880bc290 0x7fb8880beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb8880bd810 0x7fb888067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 0x7fb888068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880bf000 con 0x7fb88814ffa0 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb888068870 con 0x7fb8880bc290 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88815fa70 con 0x7fb8880bd810 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 0x7fb888068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 0x7fb888068380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47156/0 (socket says 172.21.15.92:47156) 2023-05-16T07:18:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 -- 172.21.15.92:0/1150565344 learned_addr learned my addr 172.21.15.92:0/1150565344 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88cc07700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8880bc290 0x7fb8880beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb887fff700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb8880bd810 0x7fb888067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 -- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb8880bd810 msgr2=0x7fb888067e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb8880bd810 0x7fb888067e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 -- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8880bc290 msgr2=0x7fb8880beb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8880bc290 0x7fb8880beb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 -- 172.21.15.92:0/1150565344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88000ab30 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88d408700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 0x7fb888068380 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fb8880637d0 tx=0x7fb88000a890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880008ca0 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880008e00 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8881605f0 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880014670 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.558+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88815ffa0 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.559+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb888160780 con 0x7fb88814ffa0 2023-05-16T07:18:03.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.560+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fb88001c040 con 0x7fb88814ffa0 2023-05-16T07:18:03.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.560+0000 7fb884ff9700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb86c05edb0 0x7fb86c061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.560+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fb88007ca30 con 0x7fb88814ffa0 2023-05-16T07:18:03.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.560+0000 7fb88cc07700 1 -- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb86c05edb0 msgr2=0x7fb86c061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:03.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.560+0000 7fb88cc07700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb86c05edb0 0x7fb86c061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:03.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.564+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb88004dc10 con 0x7fb88814ffa0 2023-05-16T07:18:03.614 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:03.610+0000 7f9dcf306680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:18:03.698 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:03.695+0000 7f3a4d667680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:18:03.716 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:03.711+0000 7f9dcf306680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.733+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8880bd010 con 0x7fb88814ffa0 2023-05-16T07:18:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.734+0000 7fb884ff9700 1 -- 172.21.15.92:0/1150565344 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb880051230 con 0x7fb88814ffa0 2023-05-16T07:18:03.738 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:18:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb86c05edb0 msgr2=0x7fb86c061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb86c05edb0 0x7fb86c061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 msgr2=0x7fb888068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 0x7fb888068380 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fb8880637d0 tx=0x7fb88000a890).stop 2023-05-16T07:18:03.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 shutdown_connections 2023-05-16T07:18:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb86c05edb0 0x7fb86c061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88814ffa0 0x7fb888068380 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8880bc290 0x7fb8880beb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 --2- 172.21.15.92:0/1150565344 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb8880bd810 0x7fb888067e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 >> 172.21.15.92:0/1150565344 conn(0x7fb8880c4620 msgr2=0x7fb888064970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:03.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 shutdown_connections 2023-05-16T07:18:03.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.736+0000 7fb88ee6b700 1 -- 172.21.15.92:0/1150565344 wait complete. 2023-05-16T07:18:03.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:18:03.805 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:03.802+0000 7f3a4d667680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:03.867 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:03.862+0000 7f9dcf306680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.866+0000 7f147d4c6700 1 Processor -- start 2023-05-16T07:18:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.866+0000 7f147d4c6700 1 -- start start 2023-05-16T07:18:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f147d4c6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1478154000 0x7f1478068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f147d4c6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780c9120 con 0x7f14780c62f0 2023-05-16T07:18:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f147d4c6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780ca5c0 con 0x7f1478068650 2023-05-16T07:18:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f147d4c6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c8830 con 0x7f1478154000 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f14777fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f14780c62f0 0x7f14781543e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57908/0 (socket says 172.21.15.92:57908) 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f14777fe700 1 -- 172.21.15.92:0/1571605112 learned_addr learned my addr 172.21.15.92:0/1571605112 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1476ffd700 1 --2- 172.21.15.92:0/1571605112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1478154000 0x7f1478068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12596140 0 0) 0x7f14780c9120 con 0x7f14780c62f0 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14640033f0 con 0x7f14780c62f0 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347131890 0 0) 0x7f14780ca5c0 con 0x7f1478068650 2023-05-16T07:18:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14780c9120 con 0x7f1478068650 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1476ffd700 1 -- 172.21.15.92:0/1571605112 >> v1:172.21.15.158:6789/0 conn(0x7f1478068650 legacy=0x7f14780c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1476ffd700 1 -- 172.21.15.92:0/1571605112 >> v1:172.21.15.92:6789/0 conn(0x7f14780c62f0 legacy=0x7f14781543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1476ffd700 1 -- 172.21.15.92:0/1571605112 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780c8ad0 con 0x7f1478154000 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1476ffd700 1 --2- 172.21.15.92:0/1571605112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1478154000 0x7f1478068160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1460009760 tx=0x7f1460006f40).ready entity=mon.2 client_cookie=a913398536006a10 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.867+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460005880 con 0x7f1478154000 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14600059e0 con 0x7f1478154000 2023-05-16T07:18:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f1475ffb700 1 -- 172.21.15.92:0/1571605112 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460005d60 con 0x7f1478154000 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 -- 172.21.15.92:0/1571605112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1478154000 msgr2=0x7f1478068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1571605112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1478154000 0x7f1478068160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1460009760 tx=0x7f1460006f40).stop 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 -- 172.21.15.92:0/1571605112 shutdown_connections 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1571605112 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1478154000 0x7f1478068160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 -- 172.21.15.92:0/1571605112 >> 172.21.15.92:0/1571605112 conn(0x7f14780678e0 msgr2=0x7f14780cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 -- 172.21.15.92:0/1571605112 shutdown_connections 2023-05-16T07:18:03.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.868+0000 7f147d4c6700 1 -- 172.21.15.92:0/1571605112 wait complete. 2023-05-16T07:18:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.869+0000 7f147d4c6700 1 Processor -- start 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 -- start start 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 0x7f14780b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14780c62f0 0x7f14780ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1478154000 0x7f1478065990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780af7c0 con 0x7f1478068650 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780afaa0 con 0x7f1478154000 2023-05-16T07:18:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f147d4c6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780afd80 con 0x7f14780c62f0 2023-05-16T07:18:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 0x7f14780b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 0x7f14780b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47164/0 (socket says 172.21.15.92:47164) 2023-05-16T07:18:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f1476ffd700 1 -- 172.21.15.92:0/1464672866 learned_addr learned my addr 172.21.15.92:0/1464672866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f14767fc700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14780c62f0 0x7f14780ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.870+0000 7f14777fe700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1478154000 0x7f1478065990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f1476ffd700 1 -- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14780c62f0 msgr2=0x7f14780ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f1476ffd700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14780c62f0 0x7f14780ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f1476ffd700 1 -- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1478154000 msgr2=0x7f1478065990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f1476ffd700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1478154000 0x7f1478065990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f1476ffd700 1 -- 172.21.15.92:0/1464672866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1460009250 con 0x7f1478068650 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f1476ffd700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 0x7f14780b0860 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f1460007fd0 tx=0x7f1460015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460006bb0 con 0x7f1478068650 2023-05-16T07:18:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146000d6b0 con 0x7f1478068650 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14780b0060 con 0x7f1478068650 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000da30 con 0x7f1478068650 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.871+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14780aea80 con 0x7f1478068650 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.872+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478066730 con 0x7f1478068650 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.872+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f1460024030 con 0x7f1478068650 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.873+0000 7f146effd700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f145805ede0 0x7f14580612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.873+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f146007b060 con 0x7f1478068650 2023-05-16T07:18:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.873+0000 7f14767fc700 1 -- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f145805ede0 msgr2=0x7f14580612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.873+0000 7f14767fc700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f145805ede0 0x7f14580612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:03.877+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f146004c2f0 con 0x7f1478068650 2023-05-16T07:18:03.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:03.937+0000 7f9dcf306680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:18:03.963 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:03.960+0000 7f3a4d667680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:04.008 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:04.004+0000 7f9dcf306680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:18:04.041 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:04.038+0000 7f3a4d667680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:18:04.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.051+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14780aec70 con 0x7f1478068650 2023-05-16T07:18:04.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.051+0000 7f146effd700 1 -- 172.21.15.92:0/1464672866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1460022070 con 0x7f1478068650 2023-05-16T07:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:18:04.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f145805ede0 msgr2=0x7f14580612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:04.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f145805ede0 0x7f14580612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:04.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 msgr2=0x7f14780b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 0x7f14780b0860 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f1460007fd0 tx=0x7f1460015000).stop 2023-05-16T07:18:04.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 shutdown_connections 2023-05-16T07:18:04.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f145805ede0 0x7f14580612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:04.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1478154000 0x7f1478065990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1478068650 0x7f14780b0860 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 --2- 172.21.15.92:0/1464672866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f14780c62f0 0x7f14780ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 >> 172.21.15.92:0/1464672866 conn(0x7f14780678e0 msgr2=0x7f14781543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:04.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 shutdown_connections 2023-05-16T07:18:04.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.053+0000 7f147d4c6700 1 -- 172.21.15.92:0/1464672866 wait complete. 2023-05-16T07:18:04.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:04.112 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:04.109+0000 7f3a4d667680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:18:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.179+0000 7ff8e0945700 1 Processor -- start 2023-05-16T07:18:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.179+0000 7ff8e0945700 1 -- start start 2023-05-16T07:18:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8e0945700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8dc154550 0x7ff8dc0aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8e0945700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc0c08b0 con 0x7ff8dc0c5cb0 2023-05-16T07:18:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8e0945700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc0bfca0 con 0x7ff8dc154170 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8e0945700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8dc0ab200 con 0x7ff8dc154550 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8d9d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8dc154550 0x7ff8dc0aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8dad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff8dc0c5cb0 0x7ff8dc0c0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57924/0 (socket says 172.21.15.92:57924) 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8dad9d700 1 -- 172.21.15.92:0/2602669081 learned_addr learned my addr 172.21.15.92:0/2602669081 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2602669081 >> v1:172.21.15.158:6789/0 conn(0x7ff8dc154170 legacy=0x7ff8dc0a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2602669081 >> v1:172.21.15.92:6789/0 conn(0x7ff8dc0c5cb0 legacy=0x7ff8dc0c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2602669081 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0bff80 con 0x7ff8dc154550 2023-05-16T07:18:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8d959a700 1 -- 172.21.15.92:0/2602669081 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444528826 0 0) 0x7ff8dc0c08b0 con 0x7ff8dc0c5cb0 2023-05-16T07:18:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.180+0000 7ff8d9d9b700 1 --2- 172.21.15.92:0/2602669081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154550 0x7ff8dc0aad10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7ff8d0009760 tx=0x7ff8d0009060).ready entity=mon.0 client_cookie=930d88c56ee8fbf9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8d959a700 1 -- 172.21.15.92:0/2602669081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000b580 con 0x7ff8dc154550 2023-05-16T07:18:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8d959a700 1 -- 172.21.15.92:0/2602669081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d000b6e0 con 0x7ff8dc154550 2023-05-16T07:18:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8d959a700 1 -- 172.21.15.92:0/2602669081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000ba60 con 0x7ff8dc154550 2023-05-16T07:18:04.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 -- 172.21.15.92:0/2602669081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154550 msgr2=0x7ff8dc0aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2602669081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154550 0x7ff8dc0aad10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7ff8d0009760 tx=0x7ff8d0009060).stop 2023-05-16T07:18:04.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 -- 172.21.15.92:0/2602669081 shutdown_connections 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2602669081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154550 0x7ff8dc0aad10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 -- 172.21.15.92:0/2602669081 >> 172.21.15.92:0/2602669081 conn(0x7ff8dc0cb6d0 msgr2=0x7ff8dc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 -- 172.21.15.92:0/2602669081 shutdown_connections 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.181+0000 7ff8e0945700 1 -- 172.21.15.92:0/2602669081 wait complete. 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.182+0000 7ff8e0945700 1 Processor -- start 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.182+0000 7ff8e0945700 1 -- start start 2023-05-16T07:18:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8e0945700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8dc0c5cb0 0x7ff8dc14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8e0945700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 0x7ff8dc0adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8e0945700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8dc154550 0x7ff8dc0c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8e0945700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0c36a0 con 0x7ff8dc154170 2023-05-16T07:18:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8e0945700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0c2b00 con 0x7ff8dc0c5cb0 2023-05-16T07:18:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8e0945700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0c2d70 con 0x7ff8dc154550 2023-05-16T07:18:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8d9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 0x7ff8dc0adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8d9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 0x7ff8dc0adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47180/0 (socket says 172.21.15.92:47180) 2023-05-16T07:18:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2157889049 learned_addr learned my addr 172.21.15.92:0/2157889049 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8da59c700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8dc0c5cb0 0x7ff8dc14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.183+0000 7ff8dad9d700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8dc154550 0x7ff8dc0c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8dc154550 msgr2=0x7ff8dc0c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8d9d9b700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8dc154550 0x7ff8dc0c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8dc0c5cb0 msgr2=0x7ff8dc14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8d9d9b700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8dc0c5cb0 0x7ff8dc14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8d9d9b700 1 -- 172.21.15.92:0/2157889049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d00092a0 con 0x7ff8dc154170 2023-05-16T07:18:04.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8d9d9b700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 0x7ff8dc0adc50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7ff8d0009eb0 tx=0x7ff8d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:04.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000b850 con 0x7ff8dc154170 2023-05-16T07:18:04.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d000b9b0 con 0x7ff8dc154170 2023-05-16T07:18:04.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc0c3050 con 0x7ff8dc154170 2023-05-16T07:18:04.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d001b620 con 0x7ff8dc154170 2023-05-16T07:18:04.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.184+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc0c2230 con 0x7ff8dc154170 2023-05-16T07:18:04.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.185+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc0666c0 con 0x7ff8dc154170 2023-05-16T07:18:04.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.186+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7ff8d002b070 con 0x7ff8dc154170 2023-05-16T07:18:04.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.186+0000 7ff8ca7fc700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 0x7ff8bc0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.186+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7ff8d007b780 con 0x7ff8dc154170 2023-05-16T07:18:04.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.186+0000 7ff8da59c700 1 -- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 msgr2=0x7ff8bc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:04.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.186+0000 7ff8da59c700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 0x7ff8bc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.190+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff8d004c960 con 0x7ff8dc154170 2023-05-16T07:18:04.259 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:04.255+0000 7f9dcf306680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:18:04.378 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:04.375+0000 7f3a4d667680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:18:04.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.387+0000 7ff8da59c700 1 -- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 msgr2=0x7ff8bc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:04.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.387+0000 7ff8da59c700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 0x7ff8bc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.406+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8dc0c37e0 con 0x7ff8dc154170 2023-05-16T07:18:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.407+0000 7ff8ca7fc700 1 -- 172.21.15.92:0/2157889049 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4494 (secure 0 0 0) 0x7ff8d001f090 con 0x7ff8dc154170 2023-05-16T07:18:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:04.411 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:48.337476+0000","modified":"2023-05-16T07:17:50.983467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1656722289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1656722289},{"type":"v1","addr":"172.21.15.92:6835","nonce":1656722289}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-05-16T07:18:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 msgr2=0x7ff8bc0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 0x7ff8bc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 msgr2=0x7ff8dc0adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 0x7ff8dc0adc50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7ff8d0009eb0 tx=0x7ff8d0017000).stop 2023-05-16T07:18:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 shutdown_connections 2023-05-16T07:18:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff8bc05ede0 0x7ff8bc0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff8dc154550 0x7ff8dc0c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff8dc0c5cb0 0x7ff8dc14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 --2- 172.21.15.92:0/2157889049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8dc154170 0x7ff8dc0adc50 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 >> 172.21.15.92:0/2157889049 conn(0x7ff8dc0cb6d0 msgr2=0x7ff8dc0acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:04.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 shutdown_connections 2023-05-16T07:18:04.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.409+0000 7ff8e0945700 1 -- 172.21.15.92:0/2157889049 wait complete. 2023-05-16T07:18:04.415 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-05-16T07:18:04.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:04.440 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:04.436+0000 7f9dcf306680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:04.568 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:04.564+0000 7f3a4d667680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.580+0000 7f4ac1aa0700 1 Processor -- start 2023-05-16T07:18:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.580+0000 7f4ac1aa0700 1 -- start start 2023-05-16T07:18:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.580+0000 7f4ac1aa0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.580+0000 7f4ac1aa0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4abc063e30 0x7f4abc062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4ac1aa0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc062960 con 0x7f4abc0bf4f0 2023-05-16T07:18:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4ac1aa0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc154420 con 0x7f4abc063e30 2023-05-16T07:18:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4ac1aa0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc069fb0 con 0x7f4abc154040 2023-05-16T07:18:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4abc063e30 0x7f4abc062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4abc063e30 0x7f4abc062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47192/0 (socket says 172.21.15.92:47192) 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 -- 172.21.15.92:0/1270542939 learned_addr learned my addr 172.21.15.92:0/1270542939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abb7fe700 1 --2- 172.21.15.92:0/1270542939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 -- 172.21.15.92:0/1270542939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 msgr2=0x7f4abc069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 --2- 172.21.15.92:0/1270542939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 -- 172.21.15.92:0/1270542939 >> v1:172.21.15.158:6789/0 conn(0x7f4abc0bf4f0 legacy=0x7f4abc063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4aba7fc700 1 -- 172.21.15.92:0/1270542939 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539047309 0 0) 0x7f4abc062960 con 0x7f4abc0bf4f0 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 -- 172.21.15.92:0/1270542939 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc062350 con 0x7f4abc063e30 2023-05-16T07:18:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4abbfff700 1 --2- 172.21.15.92:0/1270542939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc062df0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f4aac006fd0 tx=0x7f4aac0090f0).ready entity=mon.0 client_cookie=ce18649b8e8c0b57 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4aba7fc700 1 -- 172.21.15.92:0/1270542939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00b580 con 0x7f4abc063e30 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4aba7fc700 1 -- 172.21.15.92:0/1270542939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac00b6e0 con 0x7f4abc063e30 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.581+0000 7f4aba7fc700 1 -- 172.21.15.92:0/1270542939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00ba60 con 0x7f4abc063e30 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/1270542939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 msgr2=0x7f4abc062df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/1270542939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc062df0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f4aac006fd0 tx=0x7f4aac0090f0).stop 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/1270542939 shutdown_connections 2023-05-16T07:18:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/1270542939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc062df0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/1270542939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/1270542939 >> 172.21.15.92:0/1270542939 conn(0x7f4abc0c8aa0 msgr2=0x7f4abc0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/1270542939 shutdown_connections 2023-05-16T07:18:04.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.582+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/1270542939 wait complete. 2023-05-16T07:18:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.583+0000 7f4ac1aa0700 1 Processor -- start 2023-05-16T07:18:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.583+0000 7f4ac1aa0700 1 -- start start 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4ac1aa0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc0ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4ac1aa0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4abc0bf4f0 0x7f4abc0aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4ac1aa0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4ac1aa0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc152f70 con 0x7f4abc063e30 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4ac1aa0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc152ae0 con 0x7f4abc0bf4f0 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4ac1aa0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc151ed0 con 0x7f4abc154040 2023-05-16T07:18:04.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc0ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc0ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47194/0 (socket says 172.21.15.92:47194) 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 -- 172.21.15.92:0/589503171 learned_addr learned my addr 172.21.15.92:0/589503171 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abbfff700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abaffd700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4abc0bf4f0 0x7f4abc0aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 -- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 msgr2=0x7f4abc0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc0adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 -- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4abc0bf4f0 msgr2=0x7f4abc0aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4abc0bf4f0 0x7f4abc0aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abb7fe700 1 -- 172.21.15.92:0/589503171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac009370 con 0x7f4abc063e30 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.584+0000 7f4abaffd700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4abc0bf4f0 0x7f4abc0aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.585+0000 7f4abb7fe700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc0ac430 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f4ab4009f10 tx=0x7f4ab400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.585+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400cc20 con 0x7f4abc063e30 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.585+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab400f040 con 0x7f4abc063e30 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.585+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc1521b0 con 0x7f4abc063e30 2023-05-16T07:18:04.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.585+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4013400 con 0x7f4abc063e30 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.585+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc152770 con 0x7f4abc063e30 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.586+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc152dc0 con 0x7f4abc063e30 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.586+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f4ab4013770 con 0x7f4abc063e30 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.587+0000 7f4ab37fe700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4a9c05ede0 0x7f4a9c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.587+0000 7f4abaffd700 1 -- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4a9c05ede0 msgr2=0x7f4a9c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.587+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f4ab407b3e0 con 0x7f4abc063e30 2023-05-16T07:18:04.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.587+0000 7f4abaffd700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4a9c05ede0 0x7f4a9c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:04.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.591+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4ab404c640 con 0x7f4abc063e30 2023-05-16T07:18:04.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.761+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4abc06c350 con 0x7f4abc063e30 2023-05-16T07:18:04.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.762+0000 7f4ab37fe700 1 -- 172.21.15.92:0/589503171 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18108 (secure 0 0 0) 0x7f4ab404fc60 con 0x7f4abc063e30 2023-05-16T07:18:04.766 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:04.766 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:59.800152+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:46.874223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:47.303318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/2540760732":"2023-05-17T07:17:59.800121+0000","172.21.15.158:6833/86301":"2023-05-17T07:17:59.800121+0000","172.21.15.158:6832/86301":"2023-05-17T07:17:59.800121+0000","172.21.15.158:0/3814686992":"2023-05-17T07:17:59.800121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4a9c05ede0 msgr2=0x7f4a9c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4a9c05ede0 0x7f4a9c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 msgr2=0x7f4abc0ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc0ac430 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f4ab4009f10 tx=0x7f4ab400b040).stop 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 shutdown_connections 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4a9c05ede0 0x7f4a9c0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:04.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4abc154040 0x7f4abc0adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4abc063e30 0x7f4abc0ac430 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 --2- 172.21.15.92:0/589503171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4abc0bf4f0 0x7f4abc0aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 >> 172.21.15.92:0/589503171 conn(0x7f4abc0c8aa0 msgr2=0x7f4abc156580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:04.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 shutdown_connections 2023-05-16T07:18:04.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.764+0000 7f4ac1aa0700 1 -- 172.21.15.92:0/589503171 wait complete. 2023-05-16T07:18:04.780 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:18:04.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:04.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.892+0000 7fbdab3c2700 1 Processor -- start 2023-05-16T07:18:04.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.892+0000 7fbdab3c2700 1 -- start start 2023-05-16T07:18:04.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbdab3c2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbda4154040 0x7fbda4069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbdab3c2700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbda4063e30 0x7fbda4062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbdab3c2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda4062960 con 0x7fbda40bf4f0 2023-05-16T07:18:04.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbdab3c2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda4154420 con 0x7fbda4154040 2023-05-16T07:18:04.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbdab3c2700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda4069fb0 con 0x7fbda4063e30 2023-05-16T07:18:04.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbda4154040 0x7fbda4069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbda4154040 0x7fbda4069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47202/0 (socket says 172.21.15.92:47202) 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 -- 172.21.15.92:0/361437128 learned_addr learned my addr 172.21.15.92:0/361437128 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda995f700 1 --2- 172.21.15.92:0/361437128 >> v2:172.21.15.158:3300/0 conn(0x7fbda4063e30 0x7fbda4062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 -- 172.21.15.92:0/361437128 >> v2:172.21.15.158:3300/0 conn(0x7fbda4063e30 msgr2=0x7fbda4062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 --2- 172.21.15.92:0/361437128 >> v2:172.21.15.158:3300/0 conn(0x7fbda4063e30 0x7fbda4062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 -- 172.21.15.92:0/361437128 >> v1:172.21.15.158:6789/0 conn(0x7fbda40bf4f0 legacy=0x7fbda4063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda915e700 1 -- 172.21.15.92:0/361437128 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4062350 con 0x7fbda4154040 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.893+0000 7fbda995f700 1 --2- 172.21.15.92:0/361437128 >> v2:172.21.15.158:3300/0 conn(0x7fbda4063e30 0x7fbda4062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbda915e700 1 --2- 172.21.15.92:0/361437128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda4069ac0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7fbda4064210 tx=0x7fbd940090f0).ready entity=mon.0 client_cookie=ad5e6c436eb49509 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbd9bfff700 1 -- 172.21.15.92:0/361437128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400b600 con 0x7fbda4154040 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbd9bfff700 1 -- 172.21.15.92:0/361437128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9400b760 con 0x7fbda4154040 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbd9bfff700 1 -- 172.21.15.92:0/361437128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400bae0 con 0x7fbda4154040 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 -- 172.21.15.92:0/361437128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 msgr2=0x7fbda4069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/361437128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda4069ac0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7fbda4064210 tx=0x7fbd940090f0).stop 2023-05-16T07:18:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 -- 172.21.15.92:0/361437128 shutdown_connections 2023-05-16T07:18:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/361437128 >> v2:172.21.15.158:3300/0 conn(0x7fbda4063e30 0x7fbda4062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/361437128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda4069ac0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 -- 172.21.15.92:0/361437128 >> 172.21.15.92:0/361437128 conn(0x7fbda40c8aa0 msgr2=0x7fbda40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 -- 172.21.15.92:0/361437128 shutdown_connections 2023-05-16T07:18:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.894+0000 7fbdab3c2700 1 -- 172.21.15.92:0/361437128 wait complete. 2023-05-16T07:18:04.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.896+0000 7fbdab3c2700 1 Processor -- start 2023-05-16T07:18:04.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.896+0000 7fbdab3c2700 1 -- start start 2023-05-16T07:18:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda4063e30 0x7fbda40a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda40bf4f0 0x7fbda414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda40adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40a9b20 con 0x7fbda4154040 2023-05-16T07:18:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40a8f10 con 0x7fbda4063e30 2023-05-16T07:18:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40a91f0 con 0x7fbda40bf4f0 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda40adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda40adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47234/0 (socket says 172.21.15.92:47234) 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 -- 172.21.15.92:0/4246032495 learned_addr learned my addr 172.21.15.92:0/4246032495 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda915e700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda4063e30 0x7fbda40a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda895d700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda40bf4f0 0x7fbda414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 -- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda40bf4f0 msgr2=0x7fbda414e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda40bf4f0 0x7fbda414e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 -- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda4063e30 msgr2=0x7fbda40a85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda4063e30 0x7fbda40a85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 -- 172.21.15.92:0/4246032495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94009370 con 0x7fbda4154040 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbda995f700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda40adfa0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7fbd9c009f70 tx=0x7fbd9c0099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00db80 con 0x7fbda4154040 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00f040 con 0x7fbda4154040 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda40a94d0 con 0x7fbda4154040 2023-05-16T07:18:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0133f0 con 0x7fbda4154040 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.897+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda40a8a90 con 0x7fbda4154040 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.898+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda40a9cd0 con 0x7fbda4154040 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.899+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fbd9c0137e0 con 0x7fbda4154040 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.900+0000 7fbd997fa700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 0x7fbd880612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.900+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fbd9c07b240 con 0x7fbda4154040 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.900+0000 7fbda915e700 1 -- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 msgr2=0x7fbd880612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.900+0000 7fbda915e700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 0x7fbd880612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:04.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:04.903+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbd9c04c440 con 0x7fbda4154040 2023-05-16T07:18:05.076 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.072+0000 7f9dcf306680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:18:05.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.101+0000 7fbda915e700 1 -- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 msgr2=0x7fbd880612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:05.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.101+0000 7fbda915e700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 0x7fbd880612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:05.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.114+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbda406c350 con 0x7fbda4154040 2023-05-16T07:18:05.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.115+0000 7fbd997fa700 1 -- 172.21.15.92:0/4246032495 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fbd9c04fa60 con 0x7fbda4154040 2023-05-16T07:18:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:05.120 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:48.337476+0000","modified":"2023-05-16T07:17:50.983467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1656722289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1656722289},{"type":"v1","addr":"172.21.15.92:6835","nonce":1656722289}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-05-16T07:18:05.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.116+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 msgr2=0x7fbd880612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:05.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.116+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 0x7fbd880612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:05.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 msgr2=0x7fbda40adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda40adfa0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7fbd9c009f70 tx=0x7fbd9c0099b0).stop 2023-05-16T07:18:05.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 shutdown_connections 2023-05-16T07:18:05.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbd8805ede0 0x7fbd880612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:05.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbda4154040 0x7fbda40adfa0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbda4063e30 0x7fbda40a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 --2- 172.21.15.92:0/4246032495 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbda40bf4f0 0x7fbda414e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 >> 172.21.15.92:0/4246032495 conn(0x7fbda40c8aa0 msgr2=0x7fbda4154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 shutdown_connections 2023-05-16T07:18:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.117+0000 7fbdab3c2700 1 -- 172.21.15.92:0/4246032495 wait complete. 2023-05-16T07:18:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-05-16T07:18:05.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:05.143 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.138+0000 7f9dcf306680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:18:05.215 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.210+0000 7f9dcf306680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:18:05.240 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.237+0000 7f3a4d667680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:18:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.246+0000 7f90c8ba5700 1 Processor -- start 2023-05-16T07:18:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.246+0000 7f90c8ba5700 1 -- start start 2023-05-16T07:18:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.247+0000 7f90c8ba5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f90c4153eb0 0x7f90c4154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.247+0000 7f90c8ba5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90c4065000 0x7f90c4065480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.247+0000 7f90c8ba5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c4067560 con 0x7f90c4068c70 2023-05-16T07:18:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.247+0000 7f90c8ba5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c4065970 con 0x7f90c4065000 2023-05-16T07:18:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.247+0000 7f90c8ba5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40bc9b0 con 0x7f90c4153eb0 2023-05-16T07:18:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90c4065000 0x7f90c4065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c1d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f90c4068c70 0x7f90c4066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57988/0 (socket says 172.21.15.92:57988) 2023-05-16T07:18:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c1d9b700 1 -- 172.21.15.92:0/2718538471 learned_addr learned my addr 172.21.15.92:0/2718538471 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c259c700 1 --2- 172.21.15.92:0/2718538471 >> v2:172.21.15.158:3300/0 conn(0x7f90c4153eb0 0x7f90c4154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c159a700 1 -- 172.21.15.92:0/2718538471 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545288655 0 0) 0x7f90b4001110 con 0x7f90c4068c70 2023-05-16T07:18:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c159a700 1 -- 172.21.15.92:0/2718538471 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ac0033f0 con 0x7f90c4068c70 2023-05-16T07:18:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c2d9d700 1 -- 172.21.15.92:0/2718538471 >> v2:172.21.15.158:3300/0 conn(0x7f90c4153eb0 msgr2=0x7f90c4154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c2d9d700 1 --2- 172.21.15.92:0/2718538471 >> v2:172.21.15.158:3300/0 conn(0x7f90c4153eb0 0x7f90c4154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c2d9d700 1 -- 172.21.15.92:0/2718538471 >> v1:172.21.15.92:6789/0 conn(0x7f90c4068c70 legacy=0x7f90c4066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c2d9d700 1 -- 172.21.15.92:0/2718538471 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40bcc90 con 0x7f90c4065000 2023-05-16T07:18:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c2d9d700 1 --2- 172.21.15.92:0/2718538471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4065480 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f90c4064430 tx=0x7f90bc009780).ready entity=mon.0 client_cookie=f46e79f8b05d79d7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c159a700 1 -- 172.21.15.92:0/2718538471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00e070 con 0x7f90c4065000 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c159a700 1 -- 172.21.15.92:0/2718538471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00bab0 con 0x7f90c4065000 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.248+0000 7f90c159a700 1 -- 172.21.15.92:0/2718538471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00be30 con 0x7f90c4065000 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2718538471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 msgr2=0x7f90c4065480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2718538471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4065480 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f90c4064430 tx=0x7f90bc009780).stop 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2718538471 shutdown_connections 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2718538471 >> v2:172.21.15.158:3300/0 conn(0x7f90c4153eb0 0x7f90c4154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2718538471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4065480 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2718538471 >> 172.21.15.92:0/2718538471 conn(0x7f90c40c4430 msgr2=0x7f90c40c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2718538471 shutdown_connections 2023-05-16T07:18:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.249+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2718538471 wait complete. 2023-05-16T07:18:05.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.251+0000 7f90c8ba5700 1 Processor -- start 2023-05-16T07:18:05.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.251+0000 7f90c8ba5700 1 -- start start 2023-05-16T07:18:05.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c8ba5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c8ba5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c4153eb0 0x7f90c40c1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c8ba5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c4164970 0x7f90c40bf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c8ba5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41662d0 con 0x7f90c4065000 2023-05-16T07:18:05.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c8ba5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41656c0 con 0x7f90c4153eb0 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c8ba5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41659a0 con 0x7f90c4164970 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4165100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47244/0 (socket says 172.21.15.92:47244) 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 -- 172.21.15.92:0/2603360345 learned_addr learned my addr 172.21.15.92:0/2603360345 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c1d9b700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c4153eb0 0x7f90c40c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c2d9d700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c4164970 0x7f90c40bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 -- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c4164970 msgr2=0x7f90c40bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c4164970 0x7f90c40bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 -- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c4153eb0 msgr2=0x7f90c40c1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.252+0000 7f90c259c700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c4153eb0 0x7f90c40c1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90c259c700 1 -- 172.21.15.92:0/2603360345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009a20 con 0x7f90c4065000 2023-05-16T07:18:05.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90c259c700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4165100 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f90b800e900 tx=0x7f90b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:05.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8012600 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b8012760 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4165c80 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8012ae0 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.253+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c4164df0 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.254+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c41665b0 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.254+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f90b8022070 con 0x7f90c4065000 2023-05-16T07:18:05.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.254+0000 7f90b27fc700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 0x7f90a40612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.254+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f90b807a1a0 con 0x7f90c4065000 2023-05-16T07:18:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.254+0000 7f90c1d9b700 1 -- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 msgr2=0x7f90a40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.254+0000 7f90c1d9b700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 0x7f90a40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.258+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f90b804b380 con 0x7f90c4065000 2023-05-16T07:18:05.313 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.309+0000 7f3a4d667680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:18:05.357 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.353+0000 7f9dcf306680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:18:05.388 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.385+0000 7f3a4d667680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:18:05.425 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.421+0000 7f9dcf306680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:18:05.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.455+0000 7f90c1d9b700 1 -- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 msgr2=0x7f90a40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:05.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.455+0000 7f90c1d9b700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 0x7f90a40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:05.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.471+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90c4166120 con 0x7f90c4065000 2023-05-16T07:18:05.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.471+0000 7f90b27fc700 1 -- 172.21.15.92:0/2603360345 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f90b804e9a0 con 0x7f90c4065000 2023-05-16T07:18:05.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:05.477 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:17:48.337476+0000","modified":"2023-05-16T07:17:50.983467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1656722289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1656722289},{"type":"v1","addr":"172.21.15.92:6835","nonce":1656722289}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-05-16T07:18:05.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 msgr2=0x7f90a40612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:05.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 0x7f90a40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:05.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 msgr2=0x7f90c4165100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4165100 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f90b800e900 tx=0x7f90b8009060).stop 2023-05-16T07:18:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 shutdown_connections 2023-05-16T07:18:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f90a405ede0 0x7f90a40612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90c4164970 0x7f90c40bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c4065000 0x7f90c4165100 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 --2- 172.21.15.92:0/2603360345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90c4153eb0 0x7f90c40c1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.473+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 >> 172.21.15.92:0/2603360345 conn(0x7f90c40c4430 msgr2=0x7f90c40622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:05.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.474+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 shutdown_connections 2023-05-16T07:18:05.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.474+0000 7f90c8ba5700 1 -- 172.21.15.92:0/2603360345 wait complete. 2023-05-16T07:18:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-05-16T07:18:05.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:05.537 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.532+0000 7f9dcf306680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:18:05.538 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.535+0000 7f3a4d667680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:18:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.600+0000 7f3d0593f700 1 Processor -- start 2023-05-16T07:18:05.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.600+0000 7f3d0593f700 1 -- start start 2023-05-16T07:18:05.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3d0593f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d001519f0 0x7f3d000beef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3d0593f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3d000a7f30 0x7f3d000c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3d0593f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d000bfa10 con 0x7f3d000c2a30 2023-05-16T07:18:05.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3d0593f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d00064220 con 0x7f3d001519f0 2023-05-16T07:18:05.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3d0593f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d000bfcb0 con 0x7f3d000a7f30 2023-05-16T07:18:05.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d001519f0 0x7f3d000beef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d001519f0 0x7f3d000beef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47258/0 (socket says 172.21.15.92:47258) 2023-05-16T07:18:05.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 -- 172.21.15.92:0/1746695361 learned_addr learned my addr 172.21.15.92:0/1746695361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:05.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfe7fc700 1 --1- 172.21.15.92:0/1746695361 >> v1:172.21.15.92:6789/0 conn(0x7f3d000c2a30 0x7f3d000a7820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58002/0 (socket says 172.21.15.92:58002) 2023-05-16T07:18:05.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cff7fe700 1 --2- 172.21.15.92:0/1746695361 >> v2:172.21.15.158:3300/0 conn(0x7f3d000a7f30 0x7f3d000c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 -- 172.21.15.92:0/1746695361 >> v2:172.21.15.158:3300/0 conn(0x7f3d000a7f30 msgr2=0x7f3d000c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:05.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 --2- 172.21.15.92:0/1746695361 >> v2:172.21.15.158:3300/0 conn(0x7f3d000a7f30 0x7f3d000c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 -- 172.21.15.92:0/1746695361 >> v1:172.21.15.92:6789/0 conn(0x7f3d000c2a30 legacy=0x7f3d000a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfeffd700 1 -- 172.21.15.92:0/1746695361 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000bff90 con 0x7f3d001519f0 2023-05-16T07:18:05.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.601+0000 7f3cfdffb700 1 -- 172.21.15.92:0/1746695361 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106512223 0 0) 0x7f3d000bfa10 con 0x7f3d000c2a30 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3cfeffd700 1 --2- 172.21.15.92:0/1746695361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d000beef0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f3d000c29c0 tx=0x7f3cf40090f0).ready entity=mon.0 client_cookie=feab05eca17bd4df server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3cfdffb700 1 -- 172.21.15.92:0/1746695361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400b520 con 0x7f3d001519f0 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3cfdffb700 1 -- 172.21.15.92:0/1746695361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf400b680 con 0x7f3d001519f0 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3cfdffb700 1 -- 172.21.15.92:0/1746695361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400ba00 con 0x7f3d001519f0 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 -- 172.21.15.92:0/1746695361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 msgr2=0x7f3d000beef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 --2- 172.21.15.92:0/1746695361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d000beef0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f3d000c29c0 tx=0x7f3cf40090f0).stop 2023-05-16T07:18:05.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 -- 172.21.15.92:0/1746695361 shutdown_connections 2023-05-16T07:18:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 --2- 172.21.15.92:0/1746695361 >> v2:172.21.15.158:3300/0 conn(0x7f3d000a7f30 0x7f3d000c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 --2- 172.21.15.92:0/1746695361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d000beef0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 -- 172.21.15.92:0/1746695361 >> 172.21.15.92:0/1746695361 conn(0x7f3d000622b0 msgr2=0x7f3d000626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 -- 172.21.15.92:0/1746695361 shutdown_connections 2023-05-16T07:18:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.602+0000 7f3d0593f700 1 -- 172.21.15.92:0/1746695361 wait complete. 2023-05-16T07:18:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.603+0000 7f3d0593f700 1 Processor -- start 2023-05-16T07:18:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.603+0000 7f3d0593f700 1 -- start start 2023-05-16T07:18:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3d0593f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d000a7f30 0x7f3d0014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3d0593f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d000c2a30 0x7f3d00157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3d0593f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d00141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3d0593f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00141950 con 0x7f3d001519f0 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3d0593f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00067f60 con 0x7f3d000c2a30 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3d0593f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00068520 con 0x7f3d000a7f30 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3cff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d00141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3cff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d00141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47264/0 (socket says 172.21.15.92:47264) 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3cff7fe700 1 -- 172.21.15.92:0/4150804319 learned_addr learned my addr 172.21.15.92:0/4150804319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3cfe7fc700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d000c2a30 0x7f3d00157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.604+0000 7f3cfeffd700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d000a7f30 0x7f3d0014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3cff7fe700 1 -- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d000a7f30 msgr2=0x7f3d0014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3cff7fe700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d000a7f30 0x7f3d0014c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3cff7fe700 1 -- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d000c2a30 msgr2=0x7f3d00157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3cff7fe700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d000c2a30 0x7f3d00157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3cff7fe700 1 -- 172.21.15.92:0/4150804319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4009370 con 0x7f3d001519f0 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3cff7fe700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d00141460 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f3cf8009e90 tx=0x7f3cf800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800caa0 con 0x7f3d001519f0 2023-05-16T07:18:05.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf800f040 con 0x7f3d001519f0 2023-05-16T07:18:05.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800ce10 con 0x7f3d001519f0 2023-05-16T07:18:05.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d00157fc0 con 0x7f3d001519f0 2023-05-16T07:18:05.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.605+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d000681b0 con 0x7f3d001519f0 2023-05-16T07:18:05.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.606+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d00047150 con 0x7f3d001519f0 2023-05-16T07:18:05.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.606+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f3cf8013610 con 0x7f3d001519f0 2023-05-16T07:18:05.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.607+0000 7f3ce6ffd700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cec05ede0 0x7f3cec0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.607+0000 7f3cfeffd700 1 -- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cec05ede0 msgr2=0x7f3cec0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:05.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.607+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f3cf807a970 con 0x7f3d001519f0 2023-05-16T07:18:05.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.607+0000 7f3cfeffd700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cec05ede0 0x7f3cec0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:05.617 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.609+0000 7f3a4d667680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:18:05.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.611+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3cf804bad0 con 0x7f3d001519f0 2023-05-16T07:18:05.642 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.638+0000 7f9dcf306680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:05.730 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.726+0000 7f3a4d667680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:18:05.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.781+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d000c0270 con 0x7f3d001519f0 2023-05-16T07:18:05.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.782+0000 7f3ce6ffd700 1 -- 172.21.15.92:0/4150804319 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18108 (secure 0 0 0) 0x7f3cf804f0f0 con 0x7f3d001519f0 2023-05-16T07:18:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:05.787 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:17:59.800152+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:17:46.874223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-05-16T07:17:47.303318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/2540760732":"2023-05-17T07:17:59.800121+0000","172.21.15.158:6833/86301":"2023-05-17T07:17:59.800121+0000","172.21.15.158:6832/86301":"2023-05-17T07:17:59.800121+0000","172.21.15.158:0/3814686992":"2023-05-17T07:17:59.800121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:05.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cec05ede0 msgr2=0x7f3cec0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:05.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cec05ede0 0x7f3cec0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:05.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 msgr2=0x7f3d00141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d00141460 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f3cf8009e90 tx=0x7f3cf800b040).stop 2023-05-16T07:18:05.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 shutdown_connections 2023-05-16T07:18:05.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3cec05ede0 0x7f3cec0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:05.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d000a7f30 0x7f3d0014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d001519f0 0x7f3d00141460 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 --2- 172.21.15.92:0/4150804319 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d000c2a30 0x7f3d00157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 >> 172.21.15.92:0/4150804319 conn(0x7f3d000622b0 msgr2=0x7f3d000c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:05.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 shutdown_connections 2023-05-16T07:18:05.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.784+0000 7f3d0593f700 1 -- 172.21.15.92:0/4150804319 wait complete. 2023-05-16T07:18:05.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:18:05.841 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:05.838+0000 7f3a4d667680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.914+0000 7fefa514e700 1 Processor -- start 2023-05-16T07:18:05.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.914+0000 7fefa514e700 1 -- start start 2023-05-16T07:18:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fefa514e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefa0154040 0x7fefa0069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fefa514e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa00bf4f0 0x7fefa0062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fefa514e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefa0063720 0x7fefa0063bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fefa514e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa0154420 con 0x7fefa0063720 2023-05-16T07:18:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fefa514e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa0062960 con 0x7fefa0154040 2023-05-16T07:18:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fefa514e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0069fb0 con 0x7fefa00bf4f0 2023-05-16T07:18:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefa0063720 0x7fefa0063bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefa0063720 0x7fefa0063bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47272/0 (socket says 172.21.15.92:47272) 2023-05-16T07:18:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 -- 172.21.15.92:0/2820806916 learned_addr learned my addr 172.21.15.92:0/2820806916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9ed9d700 1 --2- 172.21.15.92:0/2820806916 >> v2:172.21.15.158:3300/0 conn(0x7fefa0154040 0x7fefa0069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9e59c700 1 --2- 172.21.15.92:0/2820806916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa00bf4f0 0x7fefa0062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 -- 172.21.15.92:0/2820806916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa00bf4f0 msgr2=0x7fefa0062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 --2- 172.21.15.92:0/2820806916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa00bf4f0 0x7fefa0062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 -- 172.21.15.92:0/2820806916 >> v2:172.21.15.158:3300/0 conn(0x7fefa0154040 msgr2=0x7fefa0069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 --2- 172.21.15.92:0/2820806916 >> v2:172.21.15.158:3300/0 conn(0x7fefa0154040 0x7fefa0069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 -- 172.21.15.92:0/2820806916 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa00640a0 con 0x7fefa0063720 2023-05-16T07:18:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9f59e700 1 --2- 172.21.15.92:0/2820806916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0063720 0x7fefa0063bb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fefa00c89b0 tx=0x7fef98009840).ready entity=mon.0 client_cookie=3866797ebdcbd8dd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.915+0000 7fef9dd9b700 1 -- 172.21.15.92:0/2820806916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800e070 con 0x7fefa0063720 2023-05-16T07:18:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fef9dd9b700 1 -- 172.21.15.92:0/2820806916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9800b970 con 0x7fefa0063720 2023-05-16T07:18:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fef9dd9b700 1 -- 172.21.15.92:0/2820806916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800bcf0 con 0x7fefa0063720 2023-05-16T07:18:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 -- 172.21.15.92:0/2820806916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0063720 msgr2=0x7fefa0063bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 --2- 172.21.15.92:0/2820806916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0063720 0x7fefa0063bb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fefa00c89b0 tx=0x7fef98009840).stop 2023-05-16T07:18:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 -- 172.21.15.92:0/2820806916 shutdown_connections 2023-05-16T07:18:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 --2- 172.21.15.92:0/2820806916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0063720 0x7fefa0063bb0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 --2- 172.21.15.92:0/2820806916 >> v2:172.21.15.158:3300/0 conn(0x7fefa0154040 0x7fefa0069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 --2- 172.21.15.92:0/2820806916 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa00bf4f0 0x7fefa0062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 -- 172.21.15.92:0/2820806916 >> 172.21.15.92:0/2820806916 conn(0x7fefa00c8aa0 msgr2=0x7fefa00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 -- 172.21.15.92:0/2820806916 shutdown_connections 2023-05-16T07:18:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.916+0000 7fefa514e700 1 -- 172.21.15.92:0/2820806916 wait complete. 2023-05-16T07:18:05.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.917+0000 7fefa514e700 1 Processor -- start 2023-05-16T07:18:05.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.917+0000 7fefa514e700 1 -- start start 2023-05-16T07:18:05.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fefa514e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa0063720 0x7fefa0156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fefa514e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa00bf4f0 0x7fefa00c1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fefa514e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 0x7fefa00ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fefa514e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c1600 con 0x7fefa0154040 2023-05-16T07:18:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fefa514e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c0050 con 0x7fefa00bf4f0 2023-05-16T07:18:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fefa514e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c1db0 con 0x7fefa0063720 2023-05-16T07:18:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fef9f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 0x7fefa00ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fef9f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 0x7fefa00ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47278/0 (socket says 172.21.15.92:47278) 2023-05-16T07:18:05.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fef9f59e700 1 -- 172.21.15.92:0/1222595627 learned_addr learned my addr 172.21.15.92:0/1222595627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:05.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fef9e59c700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa00bf4f0 0x7fefa00c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.918+0000 7fef9ed9d700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa0063720 0x7fefa0156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:05.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9f59e700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa0063720 msgr2=0x7fefa0156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9f59e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa0063720 0x7fefa0156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9f59e700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa00bf4f0 msgr2=0x7fefa00c1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:05.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9f59e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa00bf4f0 0x7fefa00c1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:05.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9f59e700 1 -- 172.21.15.92:0/1222595627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef98009ae0 con 0x7fefa0154040 2023-05-16T07:18:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9e59c700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa00bf4f0 0x7fefa00c1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:18:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef9f59e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 0x7fefa00ae2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fef98007080 tx=0x7fef98009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800bcf0 con 0x7fefa0154040 2023-05-16T07:18:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9800b640 con 0x7fefa0154040 2023-05-16T07:18:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa00c17f0 con 0x7fefa0154040 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9801d6b0 con 0x7fefa0154040 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.919+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa00bfce0 con 0x7fefa0154040 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.920+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa00bfb40 con 0x7fefa0154040 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.920+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 35) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fef98024030 con 0x7fefa0154040 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.921+0000 7fef86ffd700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 0x7fef8c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.921+0000 7fef9ed9d700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 msgr2=0x7fef8c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.921+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fef9807b2c0 con 0x7fefa0154040 2023-05-16T07:18:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.921+0000 7fef9ed9d700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 0x7fef8c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:05.935 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.927+0000 7f9dcf306680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:18:05.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:05.927+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fefa00bfb40 con 0x7fefa0154040 2023-05-16T07:18:06.001 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:05.996+0000 7f9dcf306680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:18:06.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.122+0000 7fef9ed9d700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 msgr2=0x7fef8c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.122+0000 7fef9ed9d700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 0x7fef8c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:06.139 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:06.136+0000 7f3a4d667680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:18:06.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.140+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fefa00bfb40 con 0x7fefa0154040 2023-05-16T07:18:06.210 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:06.206+0000 7f3a4d667680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:18:06.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.523+0000 7fef9ed9d700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 msgr2=0x7fef8c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:06.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.523+0000 7fef9ed9d700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 0x7fef8c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:18:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.888+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 36) v1 ==== 75092+0+0 (secure 0 0 0) 0x7fef98078e00 con 0x7fefa0154040 2023-05-16T07:18:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.889+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 msgr2=0x7fef8c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:06.889+0000 7fef86ffd700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 0x7fef8c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:06.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d522f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d522f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d522f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d522f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d522f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d4f2eb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.942 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d4f2eb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.942 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d4f2eb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.942 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d4f2eb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.942 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:06.936+0000 7f9d4f2eb700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:06.944 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:18:06.940+0000 7fe5de405700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:18:06.944 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:18:06.940+0000 7fe5de405700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:18:06.944 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:18:06.940+0000 7fe5de405700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:18:06.946 INFO:tasks.ceph.mds.a.smithi092.stderr: -1175> 2023-05-16T07:17:49.648+0000 7fe5e1c0c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:18:06.948 INFO:tasks.ceph.mds.a.smithi092.stderr: -4> 2023-05-16T07:18:06.940+0000 7fe5de405700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:18:06.948 INFO:tasks.ceph.mds.a.smithi092.stderr: -3> 2023-05-16T07:18:06.940+0000 7fe5de405700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:18:06.949 INFO:tasks.ceph.mds.a.smithi092.stderr: -1> 2023-05-16T07:18:06.940+0000 7fe5de405700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:18:06.966 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:18:07.016 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:18:07.012+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:18:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.014+0000 7fef86ffd700 1 -- 172.21.15.92:0/1222595627 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7fef980553e0 con 0x7fefa0154040 2023-05-16T07:18:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 msgr2=0x7fefa00ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 0x7fefa00ae2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fef98007080 tx=0x7fef98009840).stop 2023-05-16T07:18:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 shutdown_connections 2023-05-16T07:18:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fef8c05ede0 0x7fef8c0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefa0063720 0x7fefa0156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa0154040 0x7fefa00ae2e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:07.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 --2- 172.21.15.92:0/1222595627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefa00bf4f0 0x7fefa00c1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:07.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 >> 172.21.15.92:0/1222595627 conn(0x7fefa00c8aa0 msgr2=0x7fefa00c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:07.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 shutdown_connections 2023-05-16T07:18:07.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.016+0000 7fefa514e700 1 -- 172.21.15.92:0/1222595627 wait complete. 2023-05-16T07:18:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:18:07.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:18:07.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:18:07.118 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:18:07.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.185+0000 7f9d38b45700 1 Processor -- start 2023-05-16T07:18:07.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.185+0000 7f9d38b45700 1 -- start start 2023-05-16T07:18:07.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d38b45700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9d3413b420 0x7f9d340be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d38b45700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d340622b0 con 0x7f9d340be590 2023-05-16T07:18:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d38b45700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d34063110 con 0x7f9d34067560 2023-05-16T07:18:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d38b45700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d34063600 con 0x7f9d3413b420 2023-05-16T07:18:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d31d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9d340be590 0x7f9d34066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58062/0 (socket says 172.21.15.92:58062) 2023-05-16T07:18:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d31d9b700 1 -- 172.21.15.92:0/4159475533 learned_addr learned my addr 172.21.15.92:0/4159475533 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d3259c700 1 --2- 172.21.15.92:0/4159475533 >> v2:172.21.15.158:3300/0 conn(0x7f9d3413b420 0x7f9d340be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d3159a700 1 -- 172.21.15.92:0/4159475533 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568188070 0 0) 0x7f9d340622b0 con 0x7f9d340be590 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.186+0000 7f9d3159a700 1 -- 172.21.15.92:0/4159475533 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d200033f0 con 0x7f9d340be590 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3159a700 1 -- 172.21.15.92:0/4159475533 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2259642833 0 0) 0x7f9d200033f0 con 0x7f9d340be590 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3259c700 1 -- 172.21.15.92:0/4159475533 >> v1:172.21.15.158:6789/0 conn(0x7f9d34067560 legacy=0x7f9d340a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3259c700 1 -- 172.21.15.92:0/4159475533 >> v1:172.21.15.92:6789/0 conn(0x7f9d340be590 legacy=0x7f9d34066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3259c700 1 -- 172.21.15.92:0/4159475533 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d34062500 con 0x7f9d3413b420 2023-05-16T07:18:07.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3259c700 1 --2- 172.21.15.92:0/4159475533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d3413b420 0x7f9d340be0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f9d24009e90 tx=0x7f9d24009780).ready entity=mon.1 client_cookie=cee2824db62ab79a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3159a700 1 -- 172.21.15.92:0/4159475533 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400e070 con 0x7f9d3413b420 2023-05-16T07:18:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3159a700 1 -- 172.21.15.92:0/4159475533 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2400b850 con 0x7f9d3413b420 2023-05-16T07:18:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d3159a700 1 -- 172.21.15.92:0/4159475533 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c001470 con 0x7f9d3413b420 2023-05-16T07:18:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 -- 172.21.15.92:0/4159475533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d3413b420 msgr2=0x7f9d340be0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 --2- 172.21.15.92:0/4159475533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d3413b420 0x7f9d340be0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f9d24009e90 tx=0x7f9d24009780).stop 2023-05-16T07:18:07.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 -- 172.21.15.92:0/4159475533 shutdown_connections 2023-05-16T07:18:07.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 --2- 172.21.15.92:0/4159475533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d3413b420 0x7f9d340be0a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 -- 172.21.15.92:0/4159475533 >> 172.21.15.92:0/4159475533 conn(0x7f9d340c8470 msgr2=0x7f9d340c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 -- 172.21.15.92:0/4159475533 shutdown_connections 2023-05-16T07:18:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.187+0000 7f9d38b45700 1 -- 172.21.15.92:0/4159475533 wait complete. 2023-05-16T07:18:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 Processor -- start 2023-05-16T07:18:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 -- start start 2023-05-16T07:18:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d34067560 0x7f9d340c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:07.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d340be590 0x7f9d34151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:07.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 0x7f9d340a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:07.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340bf830 con 0x7f9d3413b420 2023-05-16T07:18:07.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340bec20 con 0x7f9d340be590 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d38b45700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340bef00 con 0x7f9d34067560 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d32d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 0x7f9d340a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d32d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 0x7f9d340a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47320/0 (socket says 172.21.15.92:47320) 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.189+0000 7f9d32d9d700 1 -- 172.21.15.92:0/397579560 learned_addr learned my addr 172.21.15.92:0/397579560 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d31d9b700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d340be590 0x7f9d34151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d3259c700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d34067560 0x7f9d340c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d32d9d700 1 -- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d34067560 msgr2=0x7f9d340c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d32d9d700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d34067560 0x7f9d340c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d32d9d700 1 -- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d340be590 msgr2=0x7f9d34151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d32d9d700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d340be590 0x7f9d34151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d32d9d700 1 -- 172.21.15.92:0/397579560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24009a20 con 0x7f9d3413b420 2023-05-16T07:18:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d32d9d700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 0x7f9d340a9480 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f9d2c00c410 tx=0x7f9d2c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00f470 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c00f5f0 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d340bf1e0 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00f9b0 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.190+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d340c17c0 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.191+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d340c1cc0 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.191+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 36) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f9d2c019030 con 0x7f9d3413b420 2023-05-16T07:18:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.192+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f9d2c079280 con 0x7f9d3413b420 2023-05-16T07:18:07.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.196+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9d2c04a4a0 con 0x7f9d3413b420 2023-05-16T07:18:07.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.401+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9d3406c350 con 0x7f9d3413b420 2023-05-16T07:18:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.891+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 37) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f9d2c010070 con 0x7f9d3413b420 2023-05-16T07:18:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.892+0000 7f9d2a7fc700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9d14030180 0x7f9d14067cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:07.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.893+0000 7f9d3259c700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9d14030180 0x7f9d14067cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:07.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:07.895+0000 7f9d3259c700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9d14030180 0x7f9d14067cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f9d24009e60 tx=0x7f9d24009780).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:08.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.015+0000 7f9d2a7fc700 1 -- 172.21.15.92:0/397579560 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9d2c06e720 con 0x7f9d3413b420 2023-05-16T07:18:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9d14030180 msgr2=0x7f9d14067cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9d14030180 0x7f9d14067cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f9d24009e60 tx=0x7f9d24009780).stop 2023-05-16T07:18:08.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 msgr2=0x7f9d340a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 0x7f9d340a9480 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f9d2c00c410 tx=0x7f9d2c007c40).stop 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 shutdown_connections 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9d14030180 0x7f9d14067cb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d34067560 0x7f9d340c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3413b420 0x7f9d340a9480 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 --2- 172.21.15.92:0/397579560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d340be590 0x7f9d34151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 >> 172.21.15.92:0/397579560 conn(0x7f9d340c8470 msgr2=0x7f9d341396d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:08.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 shutdown_connections 2023-05-16T07:18:08.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.017+0000 7f9d38b45700 1 -- 172.21.15.92:0/397579560 wait complete. 2023-05-16T07:18:08.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:18:08.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.144+0000 7f87e92af700 1 Processor -- start 2023-05-16T07:18:08.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.144+0000 7f87e92af700 1 -- start start 2023-05-16T07:18:08.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.145+0000 7f87e92af700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 0x7f87e40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.145+0000 7f87e92af700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e40c94d0 con 0x7f87e4154090 2023-05-16T07:18:08.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.145+0000 7f87e92af700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e40cb7e0 con 0x7f87e4068160 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.145+0000 7f87e92af700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c88c0 con 0x7f87e4154470 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.145+0000 7f87e2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f87e4154090 0x7f87e40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58088/0 (socket says 172.21.15.92:58088) 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.145+0000 7f87e2ffd700 1 -- 172.21.15.92:0/933164723 learned_addr learned my addr 172.21.15.92:0/933164723 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 0x7f87e40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793215568 0 0) 0x7f87e40c94d0 con 0x7f87e4154090 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d00033f0 con 0x7f87e4154090 2023-05-16T07:18:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3245118370 0 0) 0x7f87d00033f0 con 0x7f87e4154090 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 msgr2=0x7f87e40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 0x7f87e40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 >> v1:172.21.15.158:6789/0 conn(0x7f87e4068160 legacy=0x7f87e40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40c8ba0 con 0x7f87e4154090 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f87d40017f0 con 0x7f87e4154090 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 >> v1:172.21.15.92:6789/0 conn(0x7f87e4154090 legacy=0x7f87e40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 0x7f87d0003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87d0003e40 0x7f87d0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87d0004b80 0x7f87d0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c88c0 con 0x7f87d0003e40 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d000b990 con 0x7f87d0004b80 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d000bb50 con 0x7f87e4154470 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87d0003e40 0x7f87d0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e2ffd700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 0x7f87d0003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e37fe700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87d0004b80 0x7f87d0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 -- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 msgr2=0x7f87d0003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87e4154470 0x7f87d0003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 -- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87d0004b80 msgr2=0x7f87d0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87d0004b80 0x7f87d0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 -- 172.21.15.92:0/933164723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00033f0 con 0x7f87d0003e40 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e37fe700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87d0004b80 0x7f87d0005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:08.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.146+0000 7f87e27fc700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87d0003e40 0x7f87d0004690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f87cc009a80 tx=0x7f87cc009060).ready entity=mon.0 client_cookie=16439237835b0308 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc009430 con 0x7f87d0003e40 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc00e640 con 0x7f87d0003e40 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e1ffb700 1 -- 172.21.15.92:0/933164723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00e9c0 con 0x7f87d0003e40 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e92af700 1 -- 172.21.15.92:0/933164723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87d0003e40 msgr2=0x7f87d0004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e92af700 1 --2- 172.21.15.92:0/933164723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87d0003e40 0x7f87d0004690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f87cc009a80 tx=0x7f87cc009060).stop 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e37fe700 1 -- 172.21.15.92:0/933164723 reap_dead start 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e92af700 1 -- 172.21.15.92:0/933164723 shutdown_connections 2023-05-16T07:18:08.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e92af700 1 -- 172.21.15.92:0/933164723 >> 172.21.15.92:0/933164723 conn(0x7f87e4067920 msgr2=0x7f87e40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:08.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e92af700 1 -- 172.21.15.92:0/933164723 shutdown_connections 2023-05-16T07:18:08.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.147+0000 7f87e92af700 1 -- 172.21.15.92:0/933164723 wait complete. 2023-05-16T07:18:08.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.148+0000 7f87e92af700 1 Processor -- start 2023-05-16T07:18:08.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.148+0000 7f87e92af700 1 -- start start 2023-05-16T07:18:08.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e92af700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87d0003e40 0x7f87e40aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e92af700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87e4156f60 0x7f87e414c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e92af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 0x7f87e40acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e92af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40af1f0 con 0x7f87e40ac930 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e92af700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40ad280 con 0x7f87e4156f60 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e92af700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4157a90 con 0x7f87d0003e40 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 0x7f87e40acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 0x7f87e40acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47356/0 (socket says 172.21.15.92:47356) 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 -- 172.21.15.92:0/138428523 learned_addr learned my addr 172.21.15.92:0/138428523 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e27fc700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87e4156f60 0x7f87e414c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e2ffd700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87d0003e40 0x7f87e40aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 -- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87d0003e40 msgr2=0x7f87e40aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87d0003e40 0x7f87e40aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 -- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87e4156f60 msgr2=0x7f87e414c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87e4156f60 0x7f87e414c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.149+0000 7f87e37fe700 1 -- 172.21.15.92:0/138428523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e41577b0 con 0x7f87e40ac930 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.150+0000 7f87e37fe700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 0x7f87e40acd90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f87dc000c00 tx=0x7f87dc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:08.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.150+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00ee40 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.150+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc014430 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.150+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e40aeba0 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.150+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0147b0 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.150+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41573e0 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.151+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e4066730 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.151+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 37) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f87dc014be0 con 0x7f87e40ac930 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.151+0000 7f87daffd700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87c405ecf0 0x7f87c40611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.151+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f87dc078d40 con 0x7f87e40ac930 2023-05-16T07:18:08.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.151+0000 7f87e2ffd700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87c405ecf0 0x7f87c40611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:08.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.152+0000 7f87e2ffd700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87c405ecf0 0x7f87c40611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f87d40014a0 tx=0x7f87d4000fb0).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:08.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.155+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f87dc049f60 con 0x7f87e40ac930 2023-05-16T07:18:08.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:08.319+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87e406c350 con 0x7f87e40ac930 2023-05-16T07:18:09.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.020+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f87dc0768c0 con 0x7f87e40ac930 2023-05-16T07:18:09.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.027+0000 7f87daffd700 1 -- 172.21.15.92:0/138428523 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f87dc072180 con 0x7f87e40ac930 2023-05-16T07:18:09.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87c405ecf0 msgr2=0x7f87c40611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87c405ecf0 0x7f87c40611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f87d40014a0 tx=0x7f87d4000fb0).stop 2023-05-16T07:18:09.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 msgr2=0x7f87e40acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 0x7f87e40acd90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f87dc000c00 tx=0x7f87dc009040).stop 2023-05-16T07:18:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 shutdown_connections 2023-05-16T07:18:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87c405ecf0 0x7f87c40611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f87d0003e40 0x7f87e40aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e40ac930 0x7f87e40acd90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 --2- 172.21.15.92:0/138428523 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87e4156f60 0x7f87e414c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 >> 172.21.15.92:0/138428523 conn(0x7f87e4067920 msgr2=0x7f87e414d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:09.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.031+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 shutdown_connections 2023-05-16T07:18:09.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.032+0000 7f87e92af700 1 -- 172.21.15.92:0/138428523 wait complete. 2023-05-16T07:18:09.038 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:18:09.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:18:09.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.168+0000 7fbbc5138700 1 Processor -- start 2023-05-16T07:18:09.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.168+0000 7fbbc5138700 1 -- start start 2023-05-16T07:18:09.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.168+0000 7fbbc5138700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 0x7fbbc00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.168+0000 7fbbc5138700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00c94d0 con 0x7fbbc0154090 2023-05-16T07:18:09.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.168+0000 7fbbc5138700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00cb7e0 con 0x7fbbc0154470 2023-05-16T07:18:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.168+0000 7fbbc5138700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00c88c0 con 0x7fbbc0068870 2023-05-16T07:18:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbbc0154090 0x7fbbc00a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58108/0 (socket says 172.21.15.92:58108) 2023-05-16T07:18:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbed9d700 1 -- 172.21.15.92:0/3428241141 learned_addr learned my addr 172.21.15.92:0/3428241141 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbf59e700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 0x7fbbc00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410705 0 0) 0x7fbbc00c94d0 con 0x7fbbc0154090 2023-05-16T07:18:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba80033f0 con 0x7fbbc0154090 2023-05-16T07:18:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2116289699 0 0) 0x7fbba80033f0 con 0x7fbbc0154090 2023-05-16T07:18:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 msgr2=0x7fbbc00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 0x7fbbc00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 >> v1:172.21.15.158:6789/0 conn(0x7fbbc0154470 legacy=0x7fbbc0068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00c8ba0 con 0x7fbbc0154090 2023-05-16T07:18:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fbbb80017f0 con 0x7fbbc0154090 2023-05-16T07:18:09.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 >> v1:172.21.15.92:6789/0 conn(0x7fbbc0154090 legacy=0x7fbbc00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbba8003910 0x7fbba8003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbba8004690 0x7fbba8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 0x7fbba8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00c88c0 con 0x7fbba8004690 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800b910 con 0x7fbba8003910 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800bad0 con 0x7fbbc0068870 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbe59c700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbba8004690 0x7fbba8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbf59e700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 0x7fbba8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.169+0000 7fbbbed9d700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbba8003910 0x7fbba8003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbe59c700 1 -- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 msgr2=0x7fbba8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:09.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbe59c700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbbc0068870 0x7fbba8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbe59c700 1 -- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbba8003910 msgr2=0x7fbba8003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:09.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbe59c700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbba8003910 0x7fbba8003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbe59c700 1 -- 172.21.15.92:0/3428241141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80033f0 con 0x7fbba8004690 2023-05-16T07:18:09.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbe59c700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbba8004690 0x7fbba8004b10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fbbb4001730 tx=0x7fbbb4007bb0).ready entity=mon.0 client_cookie=640dce027c6a8319 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:09.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4007020 con 0x7fbba8004690 2023-05-16T07:18:09.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4007180 con 0x7fbba8004690 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbdd9b700 1 -- 172.21.15.92:0/3428241141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4007500 con 0x7fbba8004690 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbc5138700 1 -- 172.21.15.92:0/3428241141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbba8004690 msgr2=0x7fbba8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbc5138700 1 --2- 172.21.15.92:0/3428241141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbba8004690 0x7fbba8004b10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fbbb4001730 tx=0x7fbbb4007bb0).stop 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbbf59e700 1 -- 172.21.15.92:0/3428241141 reap_dead start 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbc5138700 1 -- 172.21.15.92:0/3428241141 shutdown_connections 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbc5138700 1 -- 172.21.15.92:0/3428241141 >> 172.21.15.92:0/3428241141 conn(0x7fbbc0067920 msgr2=0x7fbbc00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:09.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbc5138700 1 -- 172.21.15.92:0/3428241141 shutdown_connections 2023-05-16T07:18:09.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.170+0000 7fbbc5138700 1 -- 172.21.15.92:0/3428241141 wait complete. 2023-05-16T07:18:09.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 Processor -- start 2023-05-16T07:18:09.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 -- start start 2023-05-16T07:18:09.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbba8004690 0x7fbbc0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbc00c4610 0x7fbbc0167d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 0x7fbbc0156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00af1a0 con 0x7fbbc01569e0 2023-05-16T07:18:09.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0157350 con 0x7fbbc00c4610 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbc5138700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0168560 con 0x7fbba8004690 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbbf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 0x7fbbc0156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbbf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 0x7fbbc0156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47376/0 (socket says 172.21.15.92:47376) 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.172+0000 7fbbbf59e700 1 -- 172.21.15.92:0/4000183848 learned_addr learned my addr 172.21.15.92:0/4000183848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbe59c700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbc00c4610 0x7fbbc0167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbed9d700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbba8004690 0x7fbbc0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbf59e700 1 -- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbba8004690 msgr2=0x7fbbc0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbf59e700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbba8004690 0x7fbbc0156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbf59e700 1 -- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbc00c4610 msgr2=0x7fbbc0167d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:09.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbf59e700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbc00c4610 0x7fbbc0167d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:09.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbf59e700 1 -- 172.21.15.92:0/4000183848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0168840 con 0x7fbbc01569e0 2023-05-16T07:18:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbbf59e700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 0x7fbbc0156e60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fbbb0006d80 tx=0x7fbbb0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0013680 con 0x7fbbc01569e0 2023-05-16T07:18:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb00137e0 con 0x7fbbc01569e0 2023-05-16T07:18:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc00aee30 con 0x7fbbc01569e0 2023-05-16T07:18:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0013b80 con 0x7fbbc01569e0 2023-05-16T07:18:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.173+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc00c1cd0 con 0x7fbbc01569e0 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.174+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc0066730 con 0x7fbbc01569e0 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.174+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbbb0014440 con 0x7fbbc01569e0 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.175+0000 7fbba6ffd700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbac05ed90 0x7fbbac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.175+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6740+0+0 (secure 0 0 0) 0x7fbbb007a440 con 0x7fbbc01569e0 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.175+0000 7fbbbed9d700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbac05ed90 0x7fbbac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.175+0000 7fbbbed9d700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbac05ed90 0x7fbbac061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbbb8004be0 tx=0x7fbbb8001060).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.179+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbbb004b710 con 0x7fbbc01569e0 2023-05-16T07:18:09.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:09.342+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbc00aefc0 con 0x7fbbc01569e0 2023-05-16T07:18:10.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.030+0000 7fbba6ffd700 1 -- 172.21.15.92:0/4000183848 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbbb004ed30 con 0x7fbbc01569e0 2023-05-16T07:18:10.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbac05ed90 msgr2=0x7fbbac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbac05ed90 0x7fbbac061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbbb8004be0 tx=0x7fbbb8001060).stop 2023-05-16T07:18:10.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 msgr2=0x7fbbc0156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 0x7fbbc0156e60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fbbb0006d80 tx=0x7fbbb0009060).stop 2023-05-16T07:18:10.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 shutdown_connections 2023-05-16T07:18:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbbac05ed90 0x7fbbac061250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbba8004690 0x7fbbc0156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc01569e0 0x7fbbc0156e60 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 --2- 172.21.15.92:0/4000183848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbbc00c4610 0x7fbbc0167d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 >> 172.21.15.92:0/4000183848 conn(0x7fbbc0067920 msgr2=0x7fbbc014e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 shutdown_connections 2023-05-16T07:18:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.034+0000 7fbbc5138700 1 -- 172.21.15.92:0/4000183848 wait complete. 2023-05-16T07:18:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:18:10.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:18:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.166+0000 7fafec215700 1 Processor -- start 2023-05-16T07:18:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.166+0000 7fafec215700 1 -- start start 2023-05-16T07:18:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafec215700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fafe414e440 0x7fafe4157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafec215700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fafe4154dc0 0x7fafe4066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafec215700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe40653b0 con 0x7fafe4067340 2023-05-16T07:18:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafec215700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe40677b0 con 0x7fafe414e440 2023-05-16T07:18:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafec215700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe4158000 con 0x7fafe4154dc0 2023-05-16T07:18:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafe9fb1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fafe414e440 0x7fafe4157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafe9fb1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fafe414e440 0x7fafe4157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47380/0 (socket says 172.21.15.92:47380) 2023-05-16T07:18:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.167+0000 7fafe9fb1700 1 -- 172.21.15.92:0/526653918 learned_addr learned my addr 172.21.15.92:0/526653918 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe97b0700 1 --2- 172.21.15.92:0/526653918 >> v2:172.21.15.158:3300/0 conn(0x7fafe4154dc0 0x7fafe4066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe9fb1700 1 -- 172.21.15.92:0/526653918 >> v2:172.21.15.158:3300/0 conn(0x7fafe4154dc0 msgr2=0x7fafe4066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe9fb1700 1 --2- 172.21.15.92:0/526653918 >> v2:172.21.15.158:3300/0 conn(0x7fafe4154dc0 0x7fafe4066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe9fb1700 1 -- 172.21.15.92:0/526653918 >> v1:172.21.15.158:6789/0 conn(0x7fafe4067340 legacy=0x7fafe4065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe9fb1700 1 -- 172.21.15.92:0/526653918 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe41582e0 con 0x7fafe414e440 2023-05-16T07:18:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe9fb1700 1 --2- 172.21.15.92:0/526653918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe414e440 0x7fafe4157a20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fafe40bea50 tx=0x7fafd4009840).ready entity=mon.0 client_cookie=a72dac8b3281b9da server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe8faf700 1 -- 172.21.15.92:0/526653918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400e070 con 0x7fafe414e440 2023-05-16T07:18:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe8faf700 1 -- 172.21.15.92:0/526653918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400ba10 con 0x7fafe414e440 2023-05-16T07:18:10.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafe8faf700 1 -- 172.21.15.92:0/526653918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400bd90 con 0x7fafe414e440 2023-05-16T07:18:10.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafec215700 1 -- 172.21.15.92:0/526653918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe414e440 msgr2=0x7fafe4157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafec215700 1 --2- 172.21.15.92:0/526653918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe414e440 0x7fafe4157a20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fafe40bea50 tx=0x7fafd4009840).stop 2023-05-16T07:18:10.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafec215700 1 -- 172.21.15.92:0/526653918 shutdown_connections 2023-05-16T07:18:10.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafec215700 1 --2- 172.21.15.92:0/526653918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe414e440 0x7fafe4157a20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafec215700 1 --2- 172.21.15.92:0/526653918 >> v2:172.21.15.158:3300/0 conn(0x7fafe4154dc0 0x7fafe4066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.168+0000 7fafec215700 1 -- 172.21.15.92:0/526653918 >> 172.21.15.92:0/526653918 conn(0x7fafe40c8470 msgr2=0x7fafe40c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:10.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.169+0000 7fafec215700 1 -- 172.21.15.92:0/526653918 shutdown_connections 2023-05-16T07:18:10.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.169+0000 7fafec215700 1 -- 172.21.15.92:0/526653918 wait complete. 2023-05-16T07:18:10.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 Processor -- start 2023-05-16T07:18:10.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 -- start start 2023-05-16T07:18:10.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe4067340 0x7fafe40a8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe414e440 0x7fafe40a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 0x7fafe40a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40a54f0 con 0x7fafe4154dc0 2023-05-16T07:18:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40a6360 con 0x7fafe414e440 2023-05-16T07:18:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafec215700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40a4920 con 0x7fafe4067340 2023-05-16T07:18:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafea7b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 0x7fafe40a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafea7b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 0x7fafe40a5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47388/0 (socket says 172.21.15.92:47388) 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.170+0000 7fafea7b2700 1 -- 172.21.15.92:0/1374521152 learned_addr learned my addr 172.21.15.92:0/1374521152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafe97b0700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe414e440 0x7fafe40a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafe9fb1700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe4067340 0x7fafe40a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafea7b2700 1 -- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe4067340 msgr2=0x7fafe40a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafea7b2700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe4067340 0x7fafe40a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafea7b2700 1 -- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe414e440 msgr2=0x7fafe40a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafea7b2700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe414e440 0x7fafe40a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafea7b2700 1 -- 172.21.15.92:0/1374521152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009ae0 con 0x7fafe4154dc0 2023-05-16T07:18:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafe9fb1700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe4067340 0x7fafe40a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafea7b2700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 0x7fafe40a5e70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fafdc00c490 tx=0x7fafdc007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc002ea0 con 0x7fafe4154dc0 2023-05-16T07:18:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc00ce90 con 0x7fafe4154dc0 2023-05-16T07:18:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe40a4bc0 con 0x7fafe4154dc0 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc00f6a0 con 0x7fafe4154dc0 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.171+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe40a5180 con 0x7fafe4154dc0 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.172+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe406c350 con 0x7fafe4154dc0 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.172+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fafdc010070 con 0x7fafe4154dc0 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.173+0000 7fafe1ffb700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafd005ed90 0x7fafd0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.173+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fafdc078e60 con 0x7fafe4154dc0 2023-05-16T07:18:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.173+0000 7fafe9fb1700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafd005ed90 0x7fafd0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.173+0000 7fafe9fb1700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafd005ed90 0x7fafd0061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fafd4009540 tx=0x7fafd4009840).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.177+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fafdc0478f0 con 0x7fafe4154dc0 2023-05-16T07:18:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.337+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fafe41585c0 con 0x7fafe4154dc0 2023-05-16T07:18:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.341+0000 7fafe1ffb700 1 -- 172.21.15.92:0/1374521152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7fafdc04a330 con 0x7fafe4154dc0 2023-05-16T07:18:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafd005ed90 msgr2=0x7fafd0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafd005ed90 0x7fafd0061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fafd4009540 tx=0x7fafd4009840).stop 2023-05-16T07:18:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 msgr2=0x7fafe40a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 0x7fafe40a5e70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fafdc00c490 tx=0x7fafdc007de0).stop 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 shutdown_connections 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafd005ed90 0x7fafd0061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe4067340 0x7fafe40a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe4154dc0 0x7fafe40a5e70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 --2- 172.21.15.92:0/1374521152 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe414e440 0x7fafe40a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 >> 172.21.15.92:0/1374521152 conn(0x7fafe40c8470 msgr2=0x7fafe4152630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 shutdown_connections 2023-05-16T07:18:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.343+0000 7fafec215700 1 -- 172.21.15.92:0/1374521152 wait complete. 2023-05-16T07:18:10.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:18:10.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.466+0000 7ffaef787700 1 Processor -- start 2023-05-16T07:18:10.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.466+0000 7ffaef787700 1 -- start start 2023-05-16T07:18:10.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.467+0000 7ffaef787700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ffae8154470 0x7ffae80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.467+0000 7ffaef787700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80c94d0 con 0x7ffae8068160 2023-05-16T07:18:10.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.467+0000 7ffaef787700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80cb7e0 con 0x7ffae8154090 2023-05-16T07:18:10.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.467+0000 7ffaef787700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae80c88c0 con 0x7ffae8154470 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.467+0000 7ffaedd24700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffae8068160 0x7ffae80c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58142/0 (socket says 172.21.15.92:58142) 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.467+0000 7ffaedd24700 1 -- 172.21.15.92:0/2482034662 learned_addr learned my addr 172.21.15.92:0/2482034662 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaecd22700 1 --2- 172.21.15.92:0/2482034662 >> v2:172.21.15.158:3300/0 conn(0x7ffae8154470 0x7ffae80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214283655 0 0) 0x7ffae80c94d0 con 0x7ffae8068160 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad00033f0 con 0x7ffae8068160 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213392741 0 0) 0x7ffad00033f0 con 0x7ffae8068160 2023-05-16T07:18:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 >> v2:172.21.15.158:3300/0 conn(0x7ffae8154470 msgr2=0x7ffae80aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 --2- 172.21.15.92:0/2482034662 >> v2:172.21.15.158:3300/0 conn(0x7ffae8154470 0x7ffae80aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 >> v1:172.21.15.158:6789/0 conn(0x7ffae8154090 legacy=0x7ffae80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80c8ba0 con 0x7ffae8068160 2023-05-16T07:18:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7ffae0001470 con 0x7ffae8068160 2023-05-16T07:18:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 >> v1:172.21.15.92:6789/0 conn(0x7ffae8068160 legacy=0x7ffae80c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffad00038e0 0x7ffad0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0004690 0x7ffad0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffad0005000 0x7ffad000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80c88c0 con 0x7ffad0005000 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad000bf60 con 0x7ffad00038e0 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad000c0f0 con 0x7ffad0004690 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffad0005000 0x7ffad000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaed523700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffad00038e0 0x7ffad0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaecd22700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0004690 0x7ffad0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 -- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0004690 msgr2=0x7ffad0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0004690 0x7ffad0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 -- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffad00038e0 msgr2=0x7ffad0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffad00038e0 0x7ffad0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 -- 172.21.15.92:0/2482034662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00033f0 con 0x7ffad0005000 2023-05-16T07:18:10.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaecd22700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0004690 0x7ffad0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 -- 172.21.15.92:0/2482034662 reap_dead start 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.468+0000 7ffaedd24700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffad0005000 0x7ffad000b8e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ffae000c190 tx=0x7ffae0001e50).ready entity=mon.0 client_cookie=9fa745f2bf28008d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0007a20 con 0x7ffad0005000 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000cac0 con 0x7ffad0005000 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffae7fff700 1 -- 172.21.15.92:0/2482034662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000ce40 con 0x7ffad0005000 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 -- 172.21.15.92:0/2482034662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffad0005000 msgr2=0x7ffad000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffad0005000 0x7ffad000b8e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ffae000c190 tx=0x7ffae0001e50).stop 2023-05-16T07:18:10.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 -- 172.21.15.92:0/2482034662 shutdown_connections 2023-05-16T07:18:10.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 --2- 172.21.15.92:0/2482034662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffad0005000 0x7ffad000b8e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 -- 172.21.15.92:0/2482034662 >> 172.21.15.92:0/2482034662 conn(0x7ffae8067920 msgr2=0x7ffae80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:10.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 -- 172.21.15.92:0/2482034662 shutdown_connections 2023-05-16T07:18:10.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.469+0000 7ffaef787700 1 -- 172.21.15.92:0/2482034662 wait complete. 2023-05-16T07:18:10.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.470+0000 7ffaef787700 1 Processor -- start 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.470+0000 7ffaef787700 1 -- start start 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaef787700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0005000 0x7ffae81552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaef787700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 0x7ffae80c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaef787700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffae80c4a60 0x7ffae814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaef787700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80c26a0 con 0x7ffae81557c0 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaef787700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80b0270 con 0x7ffae80c4a60 2023-05-16T07:18:10.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaef787700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80c1d70 con 0x7ffad0005000 2023-05-16T07:18:10.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaecd22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 0x7ffae80c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaecd22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 0x7ffae80c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47402/0 (socket says 172.21.15.92:47402) 2023-05-16T07:18:10.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaecd22700 1 -- 172.21.15.92:0/457273214 learned_addr learned my addr 172.21.15.92:0/457273214 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:10.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaedd24700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffae80c4a60 0x7ffae814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaed523700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0005000 0x7ffae81552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.471+0000 7ffaecd22700 1 -- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0005000 msgr2=0x7ffae81552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaecd22700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0005000 0x7ffae81552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaecd22700 1 -- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffae80c4a60 msgr2=0x7ffae814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaecd22700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffae80c4a60 0x7ffae814d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaecd22700 1 -- 172.21.15.92:0/457273214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80c2050 con 0x7ffae81557c0 2023-05-16T07:18:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaedd24700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffae80c4a60 0x7ffae814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaecd22700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 0x7ffae80c4570 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7ffadc006f70 tx=0x7ffadc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc013620 con 0x7ffae81557c0 2023-05-16T07:18:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc013780 con 0x7ffae81557c0 2023-05-16T07:18:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae80c2330 con 0x7ffae81557c0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc013b00 con 0x7ffae81557c0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.472+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae814df60 con 0x7ffae81557c0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.473+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae80c2850 con 0x7ffae81557c0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.473+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ffadc012040 con 0x7ffae81557c0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.473+0000 7ffae57fa700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffad405ed90 0x7ffad4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.473+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7ffadc07a240 con 0x7ffae81557c0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.473+0000 7ffaed523700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffad405ed90 0x7ffad4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.474+0000 7ffaed523700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffad405ed90 0x7ffad4061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7ffad8001600 tx=0x7ffad8007f30).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.477+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ffae80c2850 con 0x7ffae81557c0 2023-05-16T07:18:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.639+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ffae814e240 con 0x7ffae81557c0 2023-05-16T07:18:10.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.643+0000 7ffae57fa700 1 -- 172.21.15.92:0/457273214 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v17) v1 ==== 120+0+0 (secure 0 0 0) 0x7ffadc04b710 con 0x7ffae81557c0 2023-05-16T07:18:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffad405ed90 msgr2=0x7ffad4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffad405ed90 0x7ffad4061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7ffad8001600 tx=0x7ffad8007f30).stop 2023-05-16T07:18:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 msgr2=0x7ffae80c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 0x7ffae80c4570 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7ffadc006f70 tx=0x7ffadc008040).stop 2023-05-16T07:18:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 shutdown_connections 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ffad405ed90 0x7ffad4061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffad0005000 0x7ffae81552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffae80c4a60 0x7ffae814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 --2- 172.21.15.92:0/457273214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae81557c0 0x7ffae80c4570 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 >> 172.21.15.92:0/457273214 conn(0x7ffae8067920 msgr2=0x7ffae80c5140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 shutdown_connections 2023-05-16T07:18:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.645+0000 7ffaef787700 1 -- 172.21.15.92:0/457273214 wait complete. 2023-05-16T07:18:10.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-05-16T07:18:10.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.768+0000 7f25d0758700 1 Processor -- start 2023-05-16T07:18:10.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.768+0000 7f25d0758700 1 -- start start 2023-05-16T07:18:10.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.769+0000 7f25d0758700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f25c8154170 0x7f25c80c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.769+0000 7f25d0758700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.769+0000 7f25d0758700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c80c0660 con 0x7f25c8154550 2023-05-16T07:18:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.769+0000 7f25d0758700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c80c1b00 con 0x7f25c8154170 2023-05-16T07:18:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.769+0000 7f25d0758700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80bfd30 con 0x7f25c8069db0 2023-05-16T07:18:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25ce4f4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f25c8154170 0x7f25c80c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:39984/0 (socket says 172.21.15.92:39984) 2023-05-16T07:18:10.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cdcf3700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f25c8154550 0x7f25c80696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:33526/0 (socket says 172.21.15.92:33526) 2023-05-16T07:18:10.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 -- 172.21.15.92:0/2226659978 learned_addr learned my addr 172.21.15.92:0/2226659978 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 -- 172.21.15.92:0/2226659978 >> v2:172.21.15.158:3300/0 conn(0x7f25c8154170 msgr2=0x7f25c80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 --2- 172.21.15.92:0/2226659978 >> v2:172.21.15.158:3300/0 conn(0x7f25c8154170 0x7f25c80c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 -- 172.21.15.92:0/2226659978 >> v1:172.21.15.158:6789/0 conn(0x7f25c8154550 legacy=0x7f25c80696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 -- 172.21.15.92:0/2226659978 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80c0010 con 0x7f25c8069db0 2023-05-16T07:18:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cecf5700 1 --2- 172.21.15.92:0/2226659978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c6160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f25c0009f20 tx=0x7f25c0009840).ready entity=mon.2 client_cookie=4cd44243d3454910 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cd4f2700 1 -- 172.21.15.92:0/2226659978 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c000e070 con 0x7f25c8069db0 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cd4f2700 1 -- 172.21.15.92:0/2226659978 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c000b9d0 con 0x7f25c8069db0 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.770+0000 7f25cd4f2700 1 -- 172.21.15.92:0/2226659978 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c000bd50 con 0x7f25c8069db0 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 -- 172.21.15.92:0/2226659978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 msgr2=0x7f25c80c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 --2- 172.21.15.92:0/2226659978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c6160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f25c0009f20 tx=0x7f25c0009840).stop 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 -- 172.21.15.92:0/2226659978 shutdown_connections 2023-05-16T07:18:10.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 --2- 172.21.15.92:0/2226659978 >> v2:172.21.15.158:3300/0 conn(0x7f25c8154170 0x7f25c80c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 --2- 172.21.15.92:0/2226659978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c6160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 -- 172.21.15.92:0/2226659978 >> 172.21.15.92:0/2226659978 conn(0x7f25c80cb700 msgr2=0x7f25c80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 -- 172.21.15.92:0/2226659978 shutdown_connections 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.771+0000 7f25d0758700 1 -- 172.21.15.92:0/2226659978 wait complete. 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.772+0000 7f25d0758700 1 Processor -- start 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.772+0000 7f25d0758700 1 -- start start 2023-05-16T07:18:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25d0758700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25d0758700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25c8154170 0x7f25c80ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25d0758700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 0x7f25c814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25d0758700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8155f00 con 0x7f25c8154550 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25d0758700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80af6c0 con 0x7f25c8154170 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25d0758700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81561c0 con 0x7f25c8069db0 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 0x7f25c814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 0x7f25c814d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47418/0 (socket says 172.21.15.92:47418) 2023-05-16T07:18:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 -- 172.21.15.92:0/1779533007 learned_addr learned my addr 172.21.15.92:0/1779533007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cdcf3700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25c8154170 0x7f25c80ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25ce4f4700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 -- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 msgr2=0x7f25c80c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 -- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25c8154170 msgr2=0x7f25c80ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25c8154170 0x7f25c80ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cecf5700 1 -- 172.21.15.92:0/1779533007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009ae0 con 0x7f25c8154550 2023-05-16T07:18:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.773+0000 7f25cdcf3700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25c8154170 0x7f25c80ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.774+0000 7f25cecf5700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 0x7f25c814d970 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f25c000b610 tx=0x7f25c0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.774+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0007080 con 0x7f25c8154550 2023-05-16T07:18:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.774+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c00092e0 con 0x7f25c8154550 2023-05-16T07:18:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.774+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c80aee20 con 0x7f25c8154550 2023-05-16T07:18:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.774+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c001d640 con 0x7f25c8154550 2023-05-16T07:18:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.774+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c80af3e0 con 0x7f25c8154550 2023-05-16T07:18:10.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.775+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c80666c0 con 0x7f25c8154550 2023-05-16T07:18:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.775+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f25c001b040 con 0x7f25c8154550 2023-05-16T07:18:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.775+0000 7f25be7fc700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25ac05ed90 0x7f25ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.775+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f25c007b700 con 0x7f25c8154550 2023-05-16T07:18:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.776+0000 7f25ce4f4700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25ac05ed90 0x7f25ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.776+0000 7f25ce4f4700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25ac05ed90 0x7f25ac061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f25c4009270 tx=0x7f25c4008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:10.780+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f25c004cac0 con 0x7f25c8154550 2023-05-16T07:18:11.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.004+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f25c806c350 con 0x7f25c8154550 2023-05-16T07:18:11.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.029+0000 7f25be7fc700 1 -- 172.21.15.92:0/1779533007 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v576) v1 ==== 166+0+0 (secure 0 0 0) 0x7f25c00500e0 con 0x7f25c8154550 2023-05-16T07:18:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25ac05ed90 msgr2=0x7f25ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25ac05ed90 0x7f25ac061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f25c4009270 tx=0x7f25c4008040).stop 2023-05-16T07:18:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 msgr2=0x7f25c814d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 0x7f25c814d970 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f25c000b610 tx=0x7f25c0009840).stop 2023-05-16T07:18:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 shutdown_connections 2023-05-16T07:18:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f25ac05ed90 0x7f25ac061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f25c8069db0 0x7f25c80c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25c8154550 0x7f25c814d970 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 --2- 172.21.15.92:0/1779533007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25c8154170 0x7f25c80ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 >> 172.21.15.92:0/1779533007 conn(0x7f25c80cb700 msgr2=0x7f25c80acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 shutdown_connections 2023-05-16T07:18:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.031+0000 7f25d0758700 1 -- 172.21.15.92:0/1779533007 wait complete. 2023-05-16T07:18:11.047 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-05-16T07:18:11.047 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-05-16T07:18:11.048 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:18:11.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-05-16T07:18:11.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.155+0000 7fe676fb3700 1 Processor -- start 2023-05-16T07:18:11.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.155+0000 7fe676fb3700 1 -- start start 2023-05-16T07:18:11.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe676fb3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 0x7fe6700696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe676fb3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700c0660 con 0x7fe670069b90 2023-05-16T07:18:11.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe676fb3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67006a020 con 0x7fe670154170 2023-05-16T07:18:11.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe676fb3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700bfd30 con 0x7fe670154550 2023-05-16T07:18:11.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe675550700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe670069b90 0x7fe6700c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58174/0 (socket says 172.21.15.92:58174) 2023-05-16T07:18:11.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe675550700 1 -- 172.21.15.92:0/310466862 learned_addr learned my addr 172.21.15.92:0/310466862 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:11.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66ffff700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 0x7fe6700696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541997173 0 0) 0x7fe6700c0660 con 0x7fe670069b90 2023-05-16T07:18:11.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6600033f0 con 0x7fe670069b90 2023-05-16T07:18:11.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989474476 0 0) 0x7fe6600033f0 con 0x7fe670069b90 2023-05-16T07:18:11.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 msgr2=0x7fe6700696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 0x7fe6700696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 >> v1:172.21.15.158:6789/0 conn(0x7fe670154170 legacy=0x7fe6700c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700c0010 con 0x7fe670069b90 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fe6680017f0 con 0x7fe670069b90 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.156+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 >> v1:172.21.15.92:6789/0 conn(0x7fe670069b90 legacy=0x7fe6700c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe660003910 0x7fe660003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660004690 0x7fe660004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 0x7fe660005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700bfd30 con 0x7fe660004690 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66000b930 con 0x7fe660003910 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66000baf0 con 0x7fe670154550 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660004690 0x7fe660004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe675550700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 0x7fe660005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe674d4f700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe660003910 0x7fe660003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 -- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 msgr2=0x7fe660005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe670154550 0x7fe660005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 -- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe660003910 msgr2=0x7fe660003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe660003910 0x7fe660003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 -- 172.21.15.92:0/310466862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600033f0 con 0x7fe660004690 2023-05-16T07:18:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66ffff700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660004690 0x7fe660004b10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fe670154930 tx=0x7fe65c009060).ready entity=mon.0 client_cookie=d36aef90d583dc44 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c009e90 con 0x7fe660004690 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c009430 con 0x7fe660004690 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe66f7fe700 1 -- 172.21.15.92:0/310466862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00e850 con 0x7fe660004690 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe676fb3700 1 -- 172.21.15.92:0/310466862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660004690 msgr2=0x7fe660004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.157+0000 7fe676fb3700 1 --2- 172.21.15.92:0/310466862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe660004690 0x7fe660004b10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fe670154930 tx=0x7fe65c009060).stop 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.158+0000 7fe675550700 1 -- 172.21.15.92:0/310466862 reap_dead start 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.158+0000 7fe676fb3700 1 -- 172.21.15.92:0/310466862 shutdown_connections 2023-05-16T07:18:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.158+0000 7fe676fb3700 1 -- 172.21.15.92:0/310466862 >> 172.21.15.92:0/310466862 conn(0x7fe6700cb700 msgr2=0x7fe6700c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:11.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.158+0000 7fe676fb3700 1 -- 172.21.15.92:0/310466862 shutdown_connections 2023-05-16T07:18:11.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.158+0000 7fe676fb3700 1 -- 172.21.15.92:0/310466862 wait complete. 2023-05-16T07:18:11.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.159+0000 7fe676fb3700 1 Processor -- start 2023-05-16T07:18:11.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.159+0000 7fe676fb3700 1 -- start start 2023-05-16T07:18:11.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.159+0000 7fe676fb3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe660004690 0x7fe670167600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6701658b0 0x7fe670164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 0x7fe6700acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670167af0 con 0x7fe6700c44a0 2023-05-16T07:18:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701509f0 con 0x7fe6701658b0 2023-05-16T07:18:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe670164fe0 con 0x7fe660004690 2023-05-16T07:18:11.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 0x7fe6700acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 0x7fe6700acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47438/0 (socket says 172.21.15.92:47438) 2023-05-16T07:18:11.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 -- 172.21.15.92:0/4205441034 learned_addr learned my addr 172.21.15.92:0/4205441034 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:11.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe674d4f700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe660004690 0x7fe670167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe66ffff700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6701658b0 0x7fe670164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 -- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe660004690 msgr2=0x7fe670167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe660004690 0x7fe670167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 -- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6701658b0 msgr2=0x7fe670164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6701658b0 0x7fe670164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:11.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 -- 172.21.15.92:0/4205441034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6701652c0 con 0x7fe6700c44a0 2023-05-16T07:18:11.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe66ffff700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6701658b0 0x7fe670164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:11.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe674d4f700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe660004690 0x7fe670167600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:11.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe675550700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 0x7fe6700acca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fe660004610 tx=0x7fe668001850).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:11.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800e620 con 0x7fe6700c44a0 2023-05-16T07:18:11.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66800e780 con 0x7fe6700c44a0 2023-05-16T07:18:11.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6701655a0 con 0x7fe6700c44a0 2023-05-16T07:18:11.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800eb00 con 0x7fe6700c44a0 2023-05-16T07:18:11.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.160+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe670164b80 con 0x7fe6700c44a0 2023-05-16T07:18:11.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.161+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6700666c0 con 0x7fe6700c44a0 2023-05-16T07:18:11.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.161+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe66801a020 con 0x7fe6700c44a0 2023-05-16T07:18:11.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.162+0000 7fe66cff9700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe65405ed90 0x7fe654061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:11.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.162+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fe668079740 con 0x7fe6700c44a0 2023-05-16T07:18:11.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.162+0000 7fe674d4f700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe65405ed90 0x7fe654061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.162+0000 7fe674d4f700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe65405ed90 0x7fe654061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe6640078e0 tx=0x7fe664009040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.166+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe66804ac10 con 0x7fe6700c44a0 2023-05-16T07:18:11.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:11.390+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fe670150b80 con 0x7fe6700c44a0 2023-05-16T07:18:12.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.030+0000 7fe66cff9700 1 -- 172.21.15.92:0/4205441034 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v577) v1 ==== 151+0+0 (secure 0 0 0) 0x7fe66804e230 con 0x7fe6700c44a0 2023-05-16T07:18:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe65405ed90 msgr2=0x7fe654061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe65405ed90 0x7fe654061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe6640078e0 tx=0x7fe664009040).stop 2023-05-16T07:18:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 msgr2=0x7fe6700acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 0x7fe6700acca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fe660004610 tx=0x7fe668001850).stop 2023-05-16T07:18:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 shutdown_connections 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe65405ed90 0x7fe654061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe660004690 0x7fe670167600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6700c44a0 0x7fe6700acca0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 --2- 172.21.15.92:0/4205441034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6701658b0 0x7fe670164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 >> 172.21.15.92:0/4205441034 conn(0x7fe6700cb700 msgr2=0x7fe6700afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 shutdown_connections 2023-05-16T07:18:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.032+0000 7fe676fb3700 1 -- 172.21.15.92:0/4205441034 wait complete. 2023-05-16T07:18:12.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:18:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.162+0000 7f18da576700 1 Processor -- start 2023-05-16T07:18:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.162+0000 7f18da576700 1 -- start start 2023-05-16T07:18:12.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.163+0000 7f18da576700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18d4068870 0x7f18d40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.163+0000 7f18da576700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d40c94d0 con 0x7f18d4154470 2023-05-16T07:18:12.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.163+0000 7f18da576700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d40cb7e0 con 0x7f18d4154090 2023-05-16T07:18:12.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.163+0000 7f18da576700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40c88c0 con 0x7f18d4068870 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.163+0000 7f18cbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f18d4154470 0x7f18d4068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58194/0 (socket says 172.21.15.92:58194) 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.163+0000 7f18cbfff700 1 -- 172.21.15.92:0/3110302190 learned_addr learned my addr 172.21.15.92:0/3110302190 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356274755 0 0) 0x7f18d40c94d0 con 0x7f18d4154470 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c00033f0 con 0x7f18d4154470 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d8b13700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18d4068870 0x7f18d40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1166953119 0 0) 0x7f18c00033f0 con 0x7f18d4154470 2023-05-16T07:18:12.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18d4068870 msgr2=0x7f18d40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18d4068870 0x7f18d40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 >> v1:172.21.15.158:6789/0 conn(0x7f18d4154090 legacy=0x7f18d40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40c8ba0 con 0x7f18d4154470 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f18bc001470 con 0x7f18d4154470 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 >> v1:172.21.15.92:6789/0 conn(0x7f18d4154470 legacy=0x7f18d4068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0003910 0x7f18c0003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18c0004690 0x7f18c0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18c0005000 0x7f18c000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40c88c0 con 0x7f18c0004690 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c000bf30 con 0x7f18c0005000 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c000c0f0 con 0x7f18c0003910 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d8b13700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18d4068870 0x7f18d40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18c0004690 0x7f18c0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d3fff700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0003910 0x7f18c0003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d8b13700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18c0005000 0x7f18c000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 -- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0003910 msgr2=0x7f18c0003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0003910 0x7f18c0003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 -- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18c0005000 msgr2=0x7f18c000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18c0005000 0x7f18c000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 -- 172.21.15.92:0/3110302190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00033f0 con 0x7f18c0004690 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18d3fff700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0003910 0x7f18c0003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.164+0000 7f18cbfff700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18c0004690 0x7f18c0004b10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f18bc00c190 tx=0x7f18bc001e50).ready entity=mon.0 client_cookie=3b2c7a96dd9774a5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc007a20 con 0x7f18c0004690 2023-05-16T07:18:12.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc00cac0 con 0x7f18c0004690 2023-05-16T07:18:12.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18d37fe700 1 -- 172.21.15.92:0/3110302190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00ce40 con 0x7f18c0004690 2023-05-16T07:18:12.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18da576700 1 -- 172.21.15.92:0/3110302190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18c0004690 msgr2=0x7f18c0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18da576700 1 --2- 172.21.15.92:0/3110302190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18c0004690 0x7f18c0004b10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f18bc00c190 tx=0x7f18bc001e50).stop 2023-05-16T07:18:12.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18d8b13700 1 -- 172.21.15.92:0/3110302190 reap_dead start 2023-05-16T07:18:12.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18da576700 1 -- 172.21.15.92:0/3110302190 shutdown_connections 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18da576700 1 -- 172.21.15.92:0/3110302190 >> 172.21.15.92:0/3110302190 conn(0x7f18d4067920 msgr2=0x7f18d40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18da576700 1 -- 172.21.15.92:0/3110302190 shutdown_connections 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.165+0000 7f18da576700 1 -- 172.21.15.92:0/3110302190 wait complete. 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.166+0000 7f18da576700 1 Processor -- start 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.166+0000 7f18da576700 1 -- start start 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18da576700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0004690 0x7f18d40c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18da576700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18d414ea40 0x7f18d414e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18da576700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 0x7f18d40a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18da576700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4157810 con 0x7f18d414eea0 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18da576700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40c4180 con 0x7f18d414ea40 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18da576700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40c3570 con 0x7f18c0004690 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 0x7f18d40a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 0x7f18d40a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47458/0 (socket says 172.21.15.92:47458) 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 -- 172.21.15.92:0/3918252584 learned_addr learned my addr 172.21.15.92:0/3918252584 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d3fff700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0004690 0x7f18d40c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18cbfff700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18d414ea40 0x7f18d414e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 -- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0004690 msgr2=0x7f18d40c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0004690 0x7f18d40c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 -- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18d414ea40 msgr2=0x7f18d414e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18d414ea40 0x7f18d414e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 -- 172.21.15.92:0/3918252584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40c3850 con 0x7f18d414eea0 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d3fff700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0004690 0x7f18d40c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18cbfff700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18d414ea40 0x7f18d414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.167+0000 7f18d8b13700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 0x7f18d40a9bf0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f18d405eeb0 tx=0x7f18cc0098f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.168+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc002ce0 con 0x7f18d414eea0 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.168+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc00f030 con 0x7f18d414eea0 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.168+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc015630 con 0x7f18d414eea0 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.168+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d40c3b30 con 0x7f18d414eea0 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.168+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d40c3220 con 0x7f18d414eea0 2023-05-16T07:18:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.169+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d4066730 con 0x7f18d414eea0 2023-05-16T07:18:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.169+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f18cc0159e0 con 0x7f18d414eea0 2023-05-16T07:18:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.169+0000 7f18d0ff9700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f18b405ed90 0x7f18b4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.169+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f18cc07a590 con 0x7f18d414eea0 2023-05-16T07:18:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.170+0000 7f18d3fff700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f18b405ed90 0x7f18b4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.170+0000 7f18d3fff700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f18b405ed90 0x7f18b4061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f18c40038e0 tx=0x7f18c4009040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.173+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f18cc04ba60 con 0x7f18d414eea0 2023-05-16T07:18:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.336+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f18d406c350 con 0x7f18d414eea0 2023-05-16T07:18:12.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.340+0000 7f18d0ff9700 1 -- 172.21.15.92:0/3918252584 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f18cc04f080 con 0x7f18d414eea0 2023-05-16T07:18:12.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.341+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f18b405ed90 msgr2=0x7f18b4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.341+0000 7f18da576700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f18b405ed90 0x7f18b4061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f18c40038e0 tx=0x7f18c4009040).stop 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.341+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 msgr2=0x7f18d40a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.341+0000 7f18da576700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 0x7f18d40a9bf0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f18d405eeb0 tx=0x7f18cc0098f0).stop 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 shutdown_connections 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f18b405ed90 0x7f18b4061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18c0004690 0x7f18d40c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d414eea0 0x7f18d40a9bf0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 --2- 172.21.15.92:0/3918252584 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18d414ea40 0x7f18d414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 >> 172.21.15.92:0/3918252584 conn(0x7f18d4067920 msgr2=0x7f18d40c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:12.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 shutdown_connections 2023-05-16T07:18:12.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.342+0000 7f18da576700 1 -- 172.21.15.92:0/3918252584 wait complete. 2023-05-16T07:18:12.358 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:18:12.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:18:12.385 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:18:12.418 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:18:12.419 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:18:12.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:12.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.531+0000 7fd7b4e8c700 1 Processor -- start 2023-05-16T07:18:12.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.531+0000 7fd7b4e8c700 1 -- start start 2023-05-16T07:18:12.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7b4e8c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd7b01555c0 0x7fd7b0068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7b4e8c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd7b0068650 0x7fd7b00bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7b4e8c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b0155a40 con 0x7fd7b0151ff0 2023-05-16T07:18:12.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7b4e8c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b00c55d0 con 0x7fd7b01555c0 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7b4e8c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b0155130 con 0x7fd7b0068650 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd7b01555c0 0x7fd7b0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd7b01555c0 0x7fd7b0068160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47466/0 (socket says 172.21.15.92:47466) 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 -- 172.21.15.92:0/4082781391 learned_addr learned my addr 172.21.15.92:0/4082781391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7aed9d700 1 --2- 172.21.15.92:0/4082781391 >> v2:172.21.15.158:3300/0 conn(0x7fd7b0068650 0x7fd7b00bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 -- 172.21.15.92:0/4082781391 >> v2:172.21.15.158:3300/0 conn(0x7fd7b0068650 msgr2=0x7fd7b00bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 --2- 172.21.15.92:0/4082781391 >> v2:172.21.15.158:3300/0 conn(0x7fd7b0068650 0x7fd7b00bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 -- 172.21.15.92:0/4082781391 >> v1:172.21.15.158:6789/0 conn(0x7fd7b0151ff0 legacy=0x7fd7b00a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 -- 172.21.15.92:0/4082781391 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00aa5f0 con 0x7fd7b01555c0 2023-05-16T07:18:12.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.532+0000 7fd7add9b700 1 --2- 172.21.15.92:0/4082781391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b01555c0 0x7fd7b0068160 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7fd7b0071480 tx=0x7fd7980090f0).ready entity=mon.0 client_cookie=620f1bc21393f8e8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7ad59a700 1 -- 172.21.15.92:0/4082781391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800b600 con 0x7fd7b01555c0 2023-05-16T07:18:12.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7ad59a700 1 -- 172.21.15.92:0/4082781391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79800b760 con 0x7fd7b01555c0 2023-05-16T07:18:12.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7ad59a700 1 -- 172.21.15.92:0/4082781391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800bae0 con 0x7fd7b01555c0 2023-05-16T07:18:12.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4082781391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b01555c0 msgr2=0x7fd7b0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4082781391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b01555c0 0x7fd7b0068160 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7fd7b0071480 tx=0x7fd7980090f0).stop 2023-05-16T07:18:12.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4082781391 shutdown_connections 2023-05-16T07:18:12.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4082781391 >> v2:172.21.15.158:3300/0 conn(0x7fd7b0068650 0x7fd7b00bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4082781391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b01555c0 0x7fd7b0068160 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4082781391 >> 172.21.15.92:0/4082781391 conn(0x7fd7b0067920 msgr2=0x7fd7b00cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4082781391 shutdown_connections 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.533+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4082781391 wait complete. 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.534+0000 7fd7b4e8c700 1 Processor -- start 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.534+0000 7fd7b4e8c700 1 -- start start 2023-05-16T07:18:12.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7b4e8c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7b0068650 0x7fd7b00aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7b4e8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 0x7fd7b01571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7b4e8c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7b01555c0 0x7fd7b014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7b4e8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00af700 con 0x7fd7b0151ff0 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7b4e8c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00af9e0 con 0x7fd7b0068650 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7b4e8c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00afcc0 con 0x7fd7b01555c0 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 0x7fd7b01571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 0x7fd7b01571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47468/0 (socket says 172.21.15.92:47468) 2023-05-16T07:18:12.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 -- 172.21.15.92:0/4104942545 learned_addr learned my addr 172.21.15.92:0/4104942545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7aed9d700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7b01555c0 0x7fd7b014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7ae59c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7b0068650 0x7fd7b00aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 -- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7b01555c0 msgr2=0x7fd7b014c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7b01555c0 0x7fd7b014c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 -- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7b0068650 msgr2=0x7fd7b00aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7b0068650 0x7fd7b00aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 -- 172.21.15.92:0/4104942545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798009370 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.535+0000 7fd7add9b700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 0x7fd7b01571b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7fd798009130 tx=0x7fd798017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.536+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800b8d0 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.536+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79800ba30 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.536+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79801c670 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.536+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b00affa0 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.536+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b00af230 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.537+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b00666c0 con 0x7fd7b0151ff0 2023-05-16T07:18:12.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.537+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd79802c070 con 0x7fd7b0151ff0 2023-05-16T07:18:12.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.537+0000 7fd7a67fc700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd79005ed90 0x7fd790061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.537+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fd79807b890 con 0x7fd7b0151ff0 2023-05-16T07:18:12.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.537+0000 7fd7ae59c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd79005ed90 0x7fd790061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.538+0000 7fd7ae59c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd79005ed90 0x7fd790061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd7a8003e70 tx=0x7fd7a8009040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.541+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd79804cd60 con 0x7fd7b0151ff0 2023-05-16T07:18:12.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.745+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b006c350 con 0x7fd7b0151ff0 2023-05-16T07:18:12.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.746+0000 7fd7a67fc700 1 -- 172.21.15.92:0/4104942545 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fd79802a090 con 0x7fd7b0151ff0 2023-05-16T07:18:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:12.751 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/523819676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":523819676},{"type":"v1","addr":"172.21.15.158:6835","nonce":523819676}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/2890536268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":2890536268},{"type":"v1","addr":"172.21.15.158:6837","nonce":2890536268}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6591,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/1663011469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1663011469},{"type":"v1","addr":"172.21.15.92:6834","nonce":1663011469}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/661643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":661643993},{"type":"v1","addr":"172.21.15.92:6835","nonce":661643993}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[]} 2023-05-16T07:18:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd79005ed90 msgr2=0x7fd790061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd79005ed90 0x7fd790061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd7a8003e70 tx=0x7fd7a8009040).stop 2023-05-16T07:18:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 msgr2=0x7fd7b01571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 0x7fd7b01571b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7fd798009130 tx=0x7fd798017000).stop 2023-05-16T07:18:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 shutdown_connections 2023-05-16T07:18:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd79005ed90 0x7fd790061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd7b01555c0 0x7fd7b014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd7b0068650 0x7fd7b00aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 --2- 172.21.15.92:0/4104942545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7b0151ff0 0x7fd7b01571b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 >> 172.21.15.92:0/4104942545 conn(0x7fd7b0067920 msgr2=0x7fd7b00c0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 shutdown_connections 2023-05-16T07:18:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.748+0000 7fd7b4e8c700 1 -- 172.21.15.92:0/4104942545 wait complete. 2023-05-16T07:18:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2023-05-16T07:18:12.765 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:18:12.765 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:18:12.765 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:18:12.766 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:18:12.766 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:18:12.766 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:18:12.767 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:18:12.767 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:18:12.767 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:18:12.767 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:18:12.768 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:18:12.768 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:18:12.768 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:18:12.769 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:18:12.769 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:18:12.769 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:18:12.774 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:18:12.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:18:12.775 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:18:12.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:18:12.778 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:18:12.779 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:18:12.780 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:18:12.780 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:18:12.817 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:18:12.818 DEBUG:teuthology.parallel:result is None 2023-05-16T07:18:12.818 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:18:12.819 DEBUG:teuthology.parallel:result is None 2023-05-16T07:18:12.822 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:18:12.822 DEBUG:teuthology.parallel:result is None 2023-05-16T07:18:12.823 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:18:12.823 DEBUG:teuthology.parallel:result is None 2023-05-16T07:18:12.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:18:12.887 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:18:12.888 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:18:12.904 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:18:12.905 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:18:12.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.938+0000 7fcb5634e700 1 Processor -- start 2023-05-16T07:18:12.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.938+0000 7fcb5634e700 1 -- start start 2023-05-16T07:18:12.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb5634e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb50154170 0x7fcb500c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb5634e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154550 0x7fcb500c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb5634e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb500c0660 con 0x7fcb500696a0 2023-05-16T07:18:12.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb5634e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb500c1b00 con 0x7fcb50154170 2023-05-16T07:18:12.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb5634e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500bfd30 con 0x7fcb50154550 2023-05-16T07:18:12.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb4ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb50154170 0x7fcb500c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb4ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb50154170 0x7fcb500c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47500/0 (socket says 172.21.15.92:47500) 2023-05-16T07:18:12.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb4ffff700 1 -- 172.21.15.92:0/44931127 learned_addr learned my addr 172.21.15.92:0/44931127 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:12.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.939+0000 7fcb4f7fe700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154550 0x7fcb500c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4ffff700 1 -- 172.21.15.92:0/44931127 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154550 msgr2=0x7fcb500c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4ffff700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154550 0x7fcb500c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4ffff700 1 -- 172.21.15.92:0/44931127 >> v1:172.21.15.158:6789/0 conn(0x7fcb500696a0 legacy=0x7fcb50157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4ffff700 1 -- 172.21.15.92:0/44931127 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500c0010 con 0x7fcb50154170 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4f7fe700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154550 0x7fcb500c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4ffff700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb50154170 0x7fcb500c5c70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7fcb40006fd0 tx=0x7fcb400090f0).ready entity=mon.0 client_cookie=835ca78a62865e2d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4effd700 1 -- 172.21.15.92:0/44931127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000b600 con 0x7fcb50154170 2023-05-16T07:18:12.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4effd700 1 -- 172.21.15.92:0/44931127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4000b760 con 0x7fcb50154170 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb4effd700 1 -- 172.21.15.92:0/44931127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000bb00 con 0x7fcb50154170 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 -- 172.21.15.92:0/44931127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb50154170 msgr2=0x7fcb500c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb50154170 0x7fcb500c5c70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7fcb40006fd0 tx=0x7fcb400090f0).stop 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 -- 172.21.15.92:0/44931127 shutdown_connections 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb50154170 0x7fcb500c5c70 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 --2- 172.21.15.92:0/44931127 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154550 0x7fcb500c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 -- 172.21.15.92:0/44931127 >> 172.21.15.92:0/44931127 conn(0x7fcb500cb700 msgr2=0x7fcb500c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:12.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 -- 172.21.15.92:0/44931127 shutdown_connections 2023-05-16T07:18:12.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.940+0000 7fcb5634e700 1 -- 172.21.15.92:0/44931127 wait complete. 2023-05-16T07:18:12.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.941+0000 7fcb5634e700 1 Processor -- start 2023-05-16T07:18:12.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 -- start start 2023-05-16T07:18:12.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 0x7fcb501447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154170 0x7fcb50167c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb50154550 0x7fcb50168180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50147040 con 0x7fcb500696a0 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50168750 con 0x7fcb50154550 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb5634e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50168a30 con 0x7fcb50154170 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb4ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 0x7fcb501447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb4ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 0x7fcb501447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47506/0 (socket says 172.21.15.92:47506) 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.942+0000 7fcb4ffff700 1 -- 172.21.15.92:0/2033374973 learned_addr learned my addr 172.21.15.92:0/2033374973 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4f7fe700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154170 0x7fcb50167c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb548eb700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb50154550 0x7fcb50168180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4ffff700 1 -- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154170 msgr2=0x7fcb50167c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4ffff700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154170 0x7fcb50167c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4ffff700 1 -- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb50154550 msgr2=0x7fcb50168180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4ffff700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb50154550 0x7fcb50168180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4ffff700 1 -- 172.21.15.92:0/2033374973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40009370 con 0x7fcb500696a0 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb4ffff700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 0x7fcb501447c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fcb40009f50 tx=0x7fcb40017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40002f70 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4000b8d0 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb50146cd0 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4001c420 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.943+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb50146980 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.944+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb500666c0 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.944+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcb4002c070 con 0x7fcb500696a0 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.945+0000 7fcb37fff700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb3c05ed90 0x7fcb3c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:12.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.945+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fcb4007b7e0 con 0x7fcb500696a0 2023-05-16T07:18:12.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.945+0000 7fcb4f7fe700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb3c05ed90 0x7fcb3c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:12.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.945+0000 7fcb4f7fe700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb3c05ed90 0x7fcb3c061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fcb44006d80 tx=0x7fcb44008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:12.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:12.949+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcb4004cba0 con 0x7fcb500696a0 2023-05-16T07:18:13.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:13.127+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcb5006c350 con 0x7fcb500696a0 2023-05-16T07:18:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.041+0000 7fcb37fff700 1 -- 172.21.15.92:0/2033374973 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcb4002a020 con 0x7fcb500696a0 2023-05-16T07:18:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb3c05ed90 msgr2=0x7fcb3c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb3c05ed90 0x7fcb3c061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fcb44006d80 tx=0x7fcb44008040).stop 2023-05-16T07:18:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 msgr2=0x7fcb501447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 0x7fcb501447c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fcb40009f50 tx=0x7fcb40017000).stop 2023-05-16T07:18:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 shutdown_connections 2023-05-16T07:18:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcb3c05ed90 0x7fcb3c061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcb50154550 0x7fcb50168180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb500696a0 0x7fcb501447c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 --2- 172.21.15.92:0/2033374973 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcb50154170 0x7fcb50167c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 >> 172.21.15.92:0/2033374973 conn(0x7fcb500cb700 msgr2=0x7fcb500ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 shutdown_connections 2023-05-16T07:18:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.044+0000 7fcb5634e700 1 -- 172.21.15.92:0/2033374973 wait complete. 2023-05-16T07:18:14.052 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:18:14.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:18:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.171+0000 7f8cc0c8c700 1 Processor -- start 2023-05-16T07:18:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.171+0000 7f8cc0c8c700 1 -- start start 2023-05-16T07:18:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.172+0000 7f8cc0c8c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 0x7f8cbc0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.172+0000 7f8cc0c8c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc0c0660 con 0x7f8cbc069b90 2023-05-16T07:18:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.172+0000 7f8cc0c8c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc06a020 con 0x7f8cbc154170 2023-05-16T07:18:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.172+0000 7f8cc0c8c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0bfd30 con 0x7f8cbc154550 2023-05-16T07:18:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cbad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8cbc069b90 0x7f8cbc0c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58266/0 (socket says 172.21.15.92:58266) 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cbad9d700 1 -- 172.21.15.92:0/3606867072 learned_addr learned my addr 172.21.15.92:0/3606867072 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 0x7f8cbc0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990783721 0 0) 0x7f8cbc0c0660 con 0x7f8cbc069b90 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca80033f0 con 0x7f8cbc069b90 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 407103300 0 0) 0x7f8ca80033f0 con 0x7f8cbc069b90 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 msgr2=0x7f8cbc0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 0x7f8cbc0696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 >> v1:172.21.15.158:6789/0 conn(0x7f8cbc154170 legacy=0x7f8cbc0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0c0010 con 0x7f8cbc069b90 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f8cb4001470 con 0x7f8cbc069b90 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 >> v1:172.21.15.92:6789/0 conn(0x7f8cbc069b90 legacy=0x7f8cbc0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ca8003910 0x7f8ca8003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8ca8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 0x7f8ca80050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0bfd30 con 0x7f8ca8004690 2023-05-16T07:18:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca800b9c0 con 0x7f8ca8003910 2023-05-16T07:18:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca800bb80 con 0x7f8cbc154550 2023-05-16T07:18:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8ca8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cbad9d700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 0x7f8ca80050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.173+0000 7f8cba59c700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ca8003910 0x7f8ca8003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb9d9b700 1 -- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 msgr2=0x7f8ca80050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc154550 0x7f8ca80050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb9d9b700 1 -- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ca8003910 msgr2=0x7f8ca8003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ca8003910 0x7f8ca8003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb9d9b700 1 -- 172.21.15.92:0/3606867072 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80033f0 con 0x7f8ca8004690 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cba59c700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8ca8003910 0x7f8ca8003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8ca8004b10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f8cbc154930 tx=0x7f8ca4009060).ready entity=mon.0 client_cookie=183cedddaf55b6b0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4009430 con 0x7f8ca8004690 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4009e70 con 0x7f8ca8004690 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cb959a700 1 -- 172.21.15.92:0/3606867072 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400e850 con 0x7f8ca8004690 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/3606867072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 msgr2=0x7f8ca8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/3606867072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8ca8004b10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f8cbc154930 tx=0x7f8ca4009060).stop 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cbad9d700 1 -- 172.21.15.92:0/3606867072 reap_dead start 2023-05-16T07:18:14.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/3606867072 shutdown_connections 2023-05-16T07:18:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/3606867072 >> 172.21.15.92:0/3606867072 conn(0x7f8cbc0cb700 msgr2=0x7f8cbc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/3606867072 shutdown_connections 2023-05-16T07:18:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.174+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/3606867072 wait complete. 2023-05-16T07:18:14.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.175+0000 7f8cc0c8c700 1 Processor -- start 2023-05-16T07:18:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 -- start start 2023-05-16T07:18:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8cbc0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc0c4cd0 0x7f8cbc148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8cbc149110 0x7f8cbc14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0c2fa0 con 0x7f8ca8004690 2023-05-16T07:18:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0c3280 con 0x7f8cbc149110 2023-05-16T07:18:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cc0c8c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0c3560 con 0x7f8cbc0c4cd0 2023-05-16T07:18:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8cbc0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8cbc0c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47528/0 (socket says 172.21.15.92:47528) 2023-05-16T07:18:14.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cba59c700 1 -- 172.21.15.92:0/1019499820 learned_addr learned my addr 172.21.15.92:0/1019499820 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc0c4cd0 0x7f8cbc148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.176+0000 7f8cbad9d700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8cbc149110 0x7f8cbc14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cba59c700 1 -- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc0c4cd0 msgr2=0x7f8cbc148c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cba59c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc0c4cd0 0x7f8cbc148c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cba59c700 1 -- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8cbc149110 msgr2=0x7f8cbc14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cba59c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8cbc149110 0x7f8cbc14ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cba59c700 1 -- 172.21.15.92:0/1019499820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc14d210 con 0x7f8ca8004690 2023-05-16T07:18:14.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cbad9d700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8cbc149110 0x7f8cbc14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:14.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cba59c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8cbc0c2710 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f8cac00d950 tx=0x7f8cac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00b630 con 0x7f8ca8004690 2023-05-16T07:18:14.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac00b790 con 0x7f8ca8004690 2023-05-16T07:18:14.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00bb10 con 0x7f8ca8004690 2023-05-16T07:18:14.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc14cec0 con 0x7f8ca8004690 2023-05-16T07:18:14.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.177+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc14b320 con 0x7f8ca8004690 2023-05-16T07:18:14.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.178+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc14d580 con 0x7f8ca8004690 2023-05-16T07:18:14.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.178+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8cac01c460 con 0x7f8ca8004690 2023-05-16T07:18:14.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.179+0000 7f8cb27fc700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c9c05ed90 0x7f8c9c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.179+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8cac079e90 con 0x7f8ca8004690 2023-05-16T07:18:14.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.179+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c9c05ed90 0x7f8c9c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.179+0000 7f8cb9d9b700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c9c05ed90 0x7f8c9c061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8ca4006d50 tx=0x7f8ca401a000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.183+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8cac04b3d0 con 0x7f8ca8004690 2023-05-16T07:18:14.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.407+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8cbc0666c0 con 0x7f8ca8004690 2023-05-16T07:18:14.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.407+0000 7f8cb27fc700 1 -- 172.21.15.92:0/1019499820 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f8cac04e9f0 con 0x7f8ca8004690 2023-05-16T07:18:14.412 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:14.412 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:18:14.412 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:18:14.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.412 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:18:14.412 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:18:14.414 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.415 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.419 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:18:14.421 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.422 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.425 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.426 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:18:14.427 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.428 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:18:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:18:14.430 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:18:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:18:14.433 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:18:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:18:14.435 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:18:14.436 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:18:14.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:18:14.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:14.436 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:18:14.436 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:18:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.409+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c9c05ed90 msgr2=0x7f8c9c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.409+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c9c05ed90 0x7f8c9c061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8ca4006d50 tx=0x7f8ca401a000).stop 2023-05-16T07:18:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.409+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 msgr2=0x7f8cbc0c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.409+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8cbc0c2710 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f8cac00d950 tx=0x7f8cac009040).stop 2023-05-16T07:18:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 shutdown_connections 2023-05-16T07:18:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8c9c05ed90 0x7f8c9c061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8cbc149110 0x7f8cbc14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca8004690 0x7f8cbc0c2710 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 --2- 172.21.15.92:0/1019499820 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8cbc0c4cd0 0x7f8cbc148c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 >> 172.21.15.92:0/1019499820 conn(0x7f8cbc0cb700 msgr2=0x7f8cbc0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 shutdown_connections 2023-05-16T07:18:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.410+0000 7f8cc0c8c700 1 -- 172.21.15.92:0/1019499820 wait complete. 2023-05-16T07:18:14.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.547+0000 7ff96e31e700 1 Processor -- start 2023-05-16T07:18:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.547+0000 7ff96e31e700 1 -- start start 2023-05-16T07:18:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96e31e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff9680c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96e31e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff968069db0 0x7ff9680c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96e31e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9680c0660 con 0x7ff968154550 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96e31e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9680c1b00 con 0x7ff968069db0 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96e31e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680bfd30 con 0x7ff968154170 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff968069db0 0x7ff9680c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff968069db0 0x7ff9680c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47540/0 (socket says 172.21.15.92:47540) 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 -- 172.21.15.92:0/710969964 learned_addr learned my addr 172.21.15.92:0/710969964 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff967fff700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff9680c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 -- 172.21.15.92:0/710969964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 msgr2=0x7ff9680c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff9680c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 -- 172.21.15.92:0/710969964 >> v1:172.21.15.158:6789/0 conn(0x7ff968154550 legacy=0x7ff9680696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 -- 172.21.15.92:0/710969964 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9680c0010 con 0x7ff968069db0 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff967fff700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff9680c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff96c8bb700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968069db0 0x7ff9680c6160 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7ff960009f20 tx=0x7ff960009840).ready entity=mon.0 client_cookie=9fb2c4cca464399d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff966ffd700 1 -- 172.21.15.92:0/710969964 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000e070 con 0x7ff968069db0 2023-05-16T07:18:14.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff966ffd700 1 -- 172.21.15.92:0/710969964 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000ba40 con 0x7ff968069db0 2023-05-16T07:18:14.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.548+0000 7ff966ffd700 1 -- 172.21.15.92:0/710969964 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000bdc0 con 0x7ff968069db0 2023-05-16T07:18:14.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 -- 172.21.15.92:0/710969964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968069db0 msgr2=0x7ff9680c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968069db0 0x7ff9680c6160 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7ff960009f20 tx=0x7ff960009840).stop 2023-05-16T07:18:14.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 -- 172.21.15.92:0/710969964 shutdown_connections 2023-05-16T07:18:14.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968069db0 0x7ff9680c6160 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 --2- 172.21.15.92:0/710969964 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff9680c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 -- 172.21.15.92:0/710969964 >> 172.21.15.92:0/710969964 conn(0x7ff9680cb700 msgr2=0x7ff9680c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:14.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 -- 172.21.15.92:0/710969964 shutdown_connections 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.549+0000 7ff96e31e700 1 -- 172.21.15.92:0/710969964 wait complete. 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.550+0000 7ff96e31e700 1 Processor -- start 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.550+0000 7ff96e31e700 1 -- start start 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96e31e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff968069db0 0x7ff96814a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96e31e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff968154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96e31e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 0x7ff9680c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96e31e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681564c0 con 0x7ff968154550 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96e31e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968155340 con 0x7ff968069db0 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96e31e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff968155620 con 0x7ff968154170 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 0x7ff9680c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 0x7ff9680c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47544/0 (socket says 172.21.15.92:47544) 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 -- 172.21.15.92:0/398528399 learned_addr learned my addr 172.21.15.92:0/398528399 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 -- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 msgr2=0x7ff968154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff9677fe700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff968154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff968154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 -- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff968069db0 msgr2=0x7ff96814a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff968069db0 0x7ff96814a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 -- 172.21.15.92:0/398528399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960009ae0 con 0x7ff968154550 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff96c8bb700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 0x7ff9680c9490 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7ff960010000 tx=0x7ff960009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.551+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600092e0 con 0x7ff968154550 2023-05-16T07:18:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.552+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96001e470 con 0x7ff968154550 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.552+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96001e7f0 con 0x7ff968154550 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.552+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968155900 con 0x7ff968154550 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.552+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9680c4d10 con 0x7ff968154550 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.553+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9680c4f00 con 0x7ff968154550 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.553+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff960025030 con 0x7ff968154550 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.553+0000 7ff94ffff700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff95405ed90 0x7ff954061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.553+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff96007b280 con 0x7ff968154550 2023-05-16T07:18:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.553+0000 7ff967fff700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff95405ed90 0x7ff954061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.554+0000 7ff967fff700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff95405ed90 0x7ff954061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff95c006fd0 tx=0x7ff95c008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.557+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff96004c9e0 con 0x7ff968154550 2023-05-16T07:18:14.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.720+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9680666c0 con 0x7ff968154550 2023-05-16T07:18:14.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.721+0000 7ff94ffff700 1 -- 172.21.15.92:0/398528399 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14644 (secure 0 0 0) 0x7ff960023070 con 0x7ff968154550 2023-05-16T07:18:14.725 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:14.725 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":103,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:14.032398+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:14.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.722+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff95405ed90 msgr2=0x7ff954061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.722+0000 7ff96e31e700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff95405ed90 0x7ff954061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff95c006fd0 tx=0x7ff95c008040).stop 2023-05-16T07:18:14.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.722+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 msgr2=0x7ff9680c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.722+0000 7ff96e31e700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 0x7ff9680c9490 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7ff960010000 tx=0x7ff960009840).stop 2023-05-16T07:18:14.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 shutdown_connections 2023-05-16T07:18:14.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff95405ed90 0x7ff954061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff968154550 0x7ff9680c9490 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff968069db0 0x7ff96814a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:14.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 --2- 172.21.15.92:0/398528399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff968154170 0x7ff968154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 >> 172.21.15.92:0/398528399 conn(0x7ff9680cb700 msgr2=0x7ff96814d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:14.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 shutdown_connections 2023-05-16T07:18:14.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.723+0000 7ff96e31e700 1 -- 172.21.15.92:0/398528399 wait complete. 2023-05-16T07:18:14.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:18:14.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:18:14.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.849+0000 7fa4cf09a700 1 Processor -- start 2023-05-16T07:18:14.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.849+0000 7fa4cf09a700 1 -- start start 2023-05-16T07:18:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cf09a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4c8153f90 0x7fa4c80a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cf09a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa4c8155380 0x7fa4c80c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cf09a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80c66a0 con 0x7fa4c80c5d00 2023-05-16T07:18:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cf09a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c8069f60 con 0x7fa4c8153f90 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cf09a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c806a9e0 con 0x7fa4c8155380 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4c8153f90 0x7fa4c80a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4c8153f90 0x7fa4c80a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47550/0 (socket says 172.21.15.92:47550) 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 -- 172.21.15.92:0/4035703877 learned_addr learned my addr 172.21.15.92:0/4035703877 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce899700 1 --2- 172.21.15.92:0/4035703877 >> v2:172.21.15.158:3300/0 conn(0x7fa4c8155380 0x7fa4c80c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 -- 172.21.15.92:0/4035703877 >> v2:172.21.15.158:3300/0 conn(0x7fa4c8155380 msgr2=0x7fa4c80c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 --2- 172.21.15.92:0/4035703877 >> v2:172.21.15.158:3300/0 conn(0x7fa4c8155380 0x7fa4c80c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 -- 172.21.15.92:0/4035703877 >> v1:172.21.15.158:6789/0 conn(0x7fa4c80c5d00 legacy=0x7fa4c80c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 -- 172.21.15.92:0/4035703877 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c8155780 con 0x7fa4c8153f90 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4ce098700 1 --2- 172.21.15.92:0/4035703877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8153f90 0x7fa4c80a9d20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fa4c80c1b50 tx=0x7fa4c4009840).ready entity=mon.0 client_cookie=eba44a86afa42239 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cd096700 1 -- 172.21.15.92:0/4035703877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400e070 con 0x7fa4c8153f90 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cd096700 1 -- 172.21.15.92:0/4035703877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c400ba10 con 0x7fa4c8153f90 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.850+0000 7fa4cd096700 1 -- 172.21.15.92:0/4035703877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400bd90 con 0x7fa4c8153f90 2023-05-16T07:18:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 -- 172.21.15.92:0/4035703877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8153f90 msgr2=0x7fa4c80a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/4035703877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8153f90 0x7fa4c80a9d20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fa4c80c1b50 tx=0x7fa4c4009840).stop 2023-05-16T07:18:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 -- 172.21.15.92:0/4035703877 shutdown_connections 2023-05-16T07:18:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/4035703877 >> v2:172.21.15.158:3300/0 conn(0x7fa4c8155380 0x7fa4c80c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/4035703877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8153f90 0x7fa4c80a9d20 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 -- 172.21.15.92:0/4035703877 >> 172.21.15.92:0/4035703877 conn(0x7fa4c80cb600 msgr2=0x7fa4c80c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 -- 172.21.15.92:0/4035703877 shutdown_connections 2023-05-16T07:18:14.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.851+0000 7fa4cf09a700 1 -- 172.21.15.92:0/4035703877 wait complete. 2023-05-16T07:18:14.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.852+0000 7fa4cf09a700 1 Processor -- start 2023-05-16T07:18:14.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.852+0000 7fa4cf09a700 1 -- start start 2023-05-16T07:18:14.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c80c5d00 0x7fa4c8165150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c8153f90 0x7fa4c814db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 0x7fa4c80c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c3720 con 0x7fa4c8155380 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c814dff0 con 0x7fa4c80c5d00 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c2b10 con 0x7fa4c8153f90 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 0x7fa4c80c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 0x7fa4c80c22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47558/0 (socket says 172.21.15.92:47558) 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 -- 172.21.15.92:0/2142144534 learned_addr learned my addr 172.21.15.92:0/2142144534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce098700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c80c5d00 0x7fa4c8165150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cd897700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c8153f90 0x7fa4c814db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 -- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c8153f90 msgr2=0x7fa4c814db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c8153f90 0x7fa4c814db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 -- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c80c5d00 msgr2=0x7fa4c8165150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c80c5d00 0x7fa4c8165150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 -- 172.21.15.92:0/2142144534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c4009ae0 con 0x7fa4c8155380 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4ce899700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 0x7fa4c80c22c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fa4c000c680 tx=0x7fa4c0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0010580 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c00106e0 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.853+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c80c2df0 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.854+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0010a60 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.854+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c80c33b0 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.854+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c80c3ab0 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.855+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa4c001b460 con 0x7fa4c8155380 2023-05-16T07:18:14.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.855+0000 7fa4be7fc700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4ac05ed90 0x7fa4ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.855+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa4c0079ba0 con 0x7fa4c8155380 2023-05-16T07:18:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.855+0000 7fa4ce098700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4ac05ed90 0x7fa4ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.856+0000 7fa4ce098700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4ac05ed90 0x7fa4ac061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa4c4009540 tx=0x7fa4c4009840).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:14.859+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa4c80c3ab0 con 0x7fa4c8155380 2023-05-16T07:18:15.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.020+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa4c80c3ab0 con 0x7fa4c8155380 2023-05-16T07:18:15.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.020+0000 7fa4be7fc700 1 -- 172.21.15.92:0/2142144534 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa4c80c3ab0 con 0x7fa4c8155380 2023-05-16T07:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:18:15.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4ac05ed90 msgr2=0x7fa4ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4ac05ed90 0x7fa4ac061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa4c4009540 tx=0x7fa4c4009840).stop 2023-05-16T07:18:15.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 msgr2=0x7fa4c80c22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 0x7fa4c80c22c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fa4c000c680 tx=0x7fa4c0008040).stop 2023-05-16T07:18:15.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 shutdown_connections 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fa4ac05ed90 0x7fa4ac061250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c8155380 0x7fa4c80c22c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa4c80c5d00 0x7fa4c8165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 --2- 172.21.15.92:0/2142144534 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa4c8153f90 0x7fa4c814db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 >> 172.21.15.92:0/2142144534 conn(0x7fa4c80cb600 msgr2=0x7fa4c80682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 shutdown_connections 2023-05-16T07:18:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.022+0000 7fa4cf09a700 1 -- 172.21.15.92:0/2142144534 wait complete. 2023-05-16T07:18:15.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:18:15.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 Processor -- start 2023-05-16T07:18:15.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 -- start start 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8070064920 0x7f8070153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8070065aa0 con 0x7f8070150250 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80700c2720 con 0x7f8070064920 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.148+0000 7f8074e52700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8070154800 con 0x7f8070154390 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8070064920 0x7f8070153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8070150250 0x7f80700c2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58312/0 (socket says 172.21.15.92:58312) 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806f7fe700 1 -- 172.21.15.92:0/2754018388 learned_addr learned my addr 172.21.15.92:0/2754018388 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806ffff700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806effd700 1 -- 172.21.15.92:0/2754018388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 msgr2=0x7f8070066ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806effd700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070066ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806e7fc700 1 -- 172.21.15.92:0/2754018388 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080045095 0 0) 0x7f8070065aa0 con 0x7f8070150250 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806effd700 1 -- 172.21.15.92:0/2754018388 >> v1:172.21.15.92:6789/0 conn(0x7f8070150250 legacy=0x7f80700c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806effd700 1 -- 172.21.15.92:0/2754018388 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8070064e90 con 0x7f8070064920 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806ffff700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070066ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806effd700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f8070153ea0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8064009760 tx=0x7f8064009060).ready entity=mon.0 client_cookie=93cdf2da1cd0242b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806e7fc700 1 -- 172.21.15.92:0/2754018388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806400b580 con 0x7f8070064920 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806e7fc700 1 -- 172.21.15.92:0/2754018388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806400b6e0 con 0x7f8070064920 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.149+0000 7f806e7fc700 1 -- 172.21.15.92:0/2754018388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806400ba60 con 0x7f8070064920 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 -- 172.21.15.92:0/2754018388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 msgr2=0x7f8070153ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f8070153ea0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8064009760 tx=0x7f8064009060).stop 2023-05-16T07:18:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 -- 172.21.15.92:0/2754018388 shutdown_connections 2023-05-16T07:18:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070066ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 --2- 172.21.15.92:0/2754018388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f8070153ea0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 -- 172.21.15.92:0/2754018388 >> 172.21.15.92:0/2754018388 conn(0x7f80700c7a80 msgr2=0x7f80700c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 -- 172.21.15.92:0/2754018388 shutdown_connections 2023-05-16T07:18:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.150+0000 7f8074e52700 1 -- 172.21.15.92:0/2754018388 wait complete. 2023-05-16T07:18:15.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.151+0000 7f8074e52700 1 Processor -- start 2023-05-16T07:18:15.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.151+0000 7f8074e52700 1 -- start start 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f80700a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8070150250 0x7f807013e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8070151d20 con 0x7f8070064920 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8070152000 con 0x7f8070150250 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80701522e0 con 0x7f8070154390 2023-05-16T07:18:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f80700a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f80700a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47570/0 (socket says 172.21.15.92:47570) 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 -- 172.21.15.92:0/705495984 learned_addr learned my addr 172.21.15.92:0/705495984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 -- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 msgr2=0x7f8070152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 -- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8070150250 msgr2=0x7f807013e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8070150250 0x7f807013e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 -- 172.21.15.92:0/705495984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80640092a0 con 0x7f8070064920 2023-05-16T07:18:15.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f806f7fe700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f80700a9f40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f806000c960 tx=0x7f8060001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060005d30 con 0x7f8070064920 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060005e90 con 0x7f8070064920 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80701525c0 con 0x7f8070064920 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060007320 con 0x7f8070064920 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.152+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701481f0 con 0x7f8070064920 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.153+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8070152ae0 con 0x7f8070064920 2023-05-16T07:18:15.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.154+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8060010030 con 0x7f8070064920 2023-05-16T07:18:15.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.154+0000 7f80577fe700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f805c05ed90 0x7f805c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.154+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8060079a50 con 0x7f8070064920 2023-05-16T07:18:15.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.154+0000 7f806effd700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f805c05ed90 0x7f805c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.155+0000 7f806effd700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f805c05ed90 0x7f805c061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8070165760 tx=0x7f8064017000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.158+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8070152ae0 con 0x7f8070064920 2023-05-16T07:18:15.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.319+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8070152ae0 con 0x7f8070064920 2023-05-16T07:18:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.319+0000 7f80577fe700 1 -- 172.21.15.92:0/705495984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8070152ae0 con 0x7f8070064920 2023-05-16T07:18:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:18:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f805c05ed90 msgr2=0x7f805c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f805c05ed90 0x7f805c061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8070165760 tx=0x7f8064017000).stop 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 msgr2=0x7f80700a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f80700a9f40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f806000c960 tx=0x7f8060001630).stop 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 shutdown_connections 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f805c05ed90 0x7f805c061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8070154390 0x7f8070152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8070064920 0x7f80700a9f40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 --2- 172.21.15.92:0/705495984 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8070150250 0x7f807013e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 >> 172.21.15.92:0/705495984 conn(0x7f80700c7a80 msgr2=0x7f8070066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 shutdown_connections 2023-05-16T07:18:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.321+0000 7f8074e52700 1 -- 172.21.15.92:0/705495984 wait complete. 2023-05-16T07:18:15.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:18:15.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.445+0000 7f0e8b59e700 1 Processor -- start 2023-05-16T07:18:15.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.445+0000 7f0e8b59e700 1 -- start start 2023-05-16T07:18:15.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e8b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e8c154e50 0x7f0e8c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e8b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c158970 con 0x7f0e8c0ad040 2023-05-16T07:18:15.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e8b59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c155300 con 0x7f0e8c0c7380 2023-05-16T07:18:15.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e8b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c157d60 con 0x7f0e8c154e50 2023-05-16T07:18:15.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e83fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e8c154e50 0x7f0e8c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e8ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0e8c0ad040 0x7f0e8c14dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58330/0 (socket says 172.21.15.92:58330) 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e8ad9d700 1 -- 172.21.15.92:0/1587739248 learned_addr learned my addr 172.21.15.92:0/1587739248 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e83fff700 1 -- 172.21.15.92:0/1587739248 >> v1:172.21.15.158:6789/0 conn(0x7f0e8c0c7380 legacy=0x7f0e8c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e83fff700 1 -- 172.21.15.92:0/1587739248 >> v1:172.21.15.92:6789/0 conn(0x7f0e8c0ad040 legacy=0x7f0e8c14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e83fff700 1 -- 172.21.15.92:0/1587739248 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c158040 con 0x7f0e8c154e50 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e89d9b700 1 -- 172.21.15.92:0/1587739248 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711494312 0 0) 0x7f0e8c158970 con 0x7f0e8c0ad040 2023-05-16T07:18:15.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e83fff700 1 --2- 172.21.15.92:0/1587739248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c154e50 0x7f0e8c0acb50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f0e74009760 tx=0x7f0e74009060).ready entity=mon.0 client_cookie=4ddca40ae9ba62a5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e89d9b700 1 -- 172.21.15.92:0/1587739248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400b520 con 0x7f0e8c154e50 2023-05-16T07:18:15.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e89d9b700 1 -- 172.21.15.92:0/1587739248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7400b680 con 0x7f0e8c154e50 2023-05-16T07:18:15.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.446+0000 7f0e89d9b700 1 -- 172.21.15.92:0/1587739248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400ba00 con 0x7f0e8c154e50 2023-05-16T07:18:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 -- 172.21.15.92:0/1587739248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c154e50 msgr2=0x7f0e8c0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/1587739248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c154e50 0x7f0e8c0acb50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f0e74009760 tx=0x7f0e74009060).stop 2023-05-16T07:18:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 -- 172.21.15.92:0/1587739248 shutdown_connections 2023-05-16T07:18:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/1587739248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c154e50 0x7f0e8c0acb50 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 -- 172.21.15.92:0/1587739248 >> 172.21.15.92:0/1587739248 conn(0x7f0e8c068330 msgr2=0x7f0e8c068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 -- 172.21.15.92:0/1587739248 shutdown_connections 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.447+0000 7f0e8b59e700 1 -- 172.21.15.92:0/1587739248 wait complete. 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.448+0000 7f0e8b59e700 1 Processor -- start 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.448+0000 7f0e8b59e700 1 -- start start 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 0x7f0e8c0c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e8c0c7380 0x7f0e8c0c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e8c154e50 0x7f0e8c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0c39f0 con 0x7f0e8c0ad040 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0c8980 con 0x7f0e8c154e50 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0c8c20 con 0x7f0e8c0c7380 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 0x7f0e8c0c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 0x7f0e8c0c2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47584/0 (socket says 172.21.15.92:47584) 2023-05-16T07:18:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 -- 172.21.15.92:0/2909985923 learned_addr learned my addr 172.21.15.92:0/2909985923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e83fff700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e8c0c7380 0x7f0e8c0c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8ad9d700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e8c154e50 0x7f0e8c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 -- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e8c0c7380 msgr2=0x7f0e8c0c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e8c0c7380 0x7f0e8c0c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 -- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e8c154e50 msgr2=0x7f0e8c14e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e8c154e50 0x7f0e8c14e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 -- 172.21.15.92:0/2909985923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740092a0 con 0x7f0e8c0ad040 2023-05-16T07:18:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8ad9d700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e8c154e50 0x7f0e8c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8a59c700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 0x7f0e8c0c2700 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f0e7c00b490 tx=0x7f0e7c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00d4a0 con 0x7f0e8c0ad040 2023-05-16T07:18:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c00d640 con 0x7f0e8c0ad040 2023-05-16T07:18:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c0c8f00 con 0x7f0e8c0ad040 2023-05-16T07:18:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00da00 con 0x7f0e8c0ad040 2023-05-16T07:18:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.449+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c0c3c70 con 0x7f0e8c0ad040 2023-05-16T07:18:15.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.450+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c0c9420 con 0x7f0e8c0ad040 2023-05-16T07:18:15.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.451+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0e7c00f030 con 0x7f0e8c0ad040 2023-05-16T07:18:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.451+0000 7f0e827fc700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e6c05ed90 0x7f0e6c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.451+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0e7c0797e0 con 0x7f0e8c0ad040 2023-05-16T07:18:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.451+0000 7f0e83fff700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e6c05ed90 0x7f0e6c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.452+0000 7f0e83fff700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e6c05ed90 0x7f0e6c061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0e8c0c4390 tx=0x7f0e74017000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.455+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0e7c04ae90 con 0x7f0e8c0ad040 2023-05-16T07:18:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.620+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0e8c0bfb90 con 0x7f0e8c0ad040 2023-05-16T07:18:15.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.620+0000 7f0e827fc700 1 -- 172.21.15.92:0/2909985923 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0e7c00ddb0 con 0x7f0e8c0ad040 2023-05-16T07:18:15.624 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:18:15.626 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:18:15.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:15.626 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:18:15.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e6c05ed90 msgr2=0x7f0e6c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e6c05ed90 0x7f0e6c061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0e8c0c4390 tx=0x7f0e74017000).stop 2023-05-16T07:18:15.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 msgr2=0x7f0e8c0c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 0x7f0e8c0c2700 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f0e7c00b490 tx=0x7f0e7c009060).stop 2023-05-16T07:18:15.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 shutdown_connections 2023-05-16T07:18:15.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e6c05ed90 0x7f0e6c061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e8c154e50 0x7f0e8c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e8c0ad040 0x7f0e8c0c2700 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 --2- 172.21.15.92:0/2909985923 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e8c0c7380 0x7f0e8c0c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 >> 172.21.15.92:0/2909985923 conn(0x7f0e8c068330 msgr2=0x7f0e8c150100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:15.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 shutdown_connections 2023-05-16T07:18:15.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.622+0000 7f0e8b59e700 1 -- 172.21.15.92:0/2909985923 wait complete. 2023-05-16T07:18:15.637 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:18:15.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:18:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.749+0000 7f1615b1a700 1 Processor -- start 2023-05-16T07:18:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.749+0000 7f1615b1a700 1 -- start start 2023-05-16T07:18:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.750+0000 7f1615b1a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1610154100 0x7f161006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.750+0000 7f1615b1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161006a720 con 0x7f16101544e0 2023-05-16T07:18:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.750+0000 7f1615b1a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16100bfbd0 con 0x7f1610069590 2023-05-16T07:18:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.750+0000 7f1615b1a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16100aad80 con 0x7f1610154100 2023-05-16T07:18:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.750+0000 7f160ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f16101544e0 0x7f16100a9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58344/0 (socket says 172.21.15.92:58344) 2023-05-16T07:18:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.750+0000 7f160ffff700 1 -- 172.21.15.92:0/2694194542 learned_addr learned my addr 172.21.15.92:0/2694194542 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 --2- 172.21.15.92:0/2694194542 >> v2:172.21.15.158:3300/0 conn(0x7f1610154100 0x7f161006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807660123 0 0) 0x7f161006a720 con 0x7f16101544e0 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f80033f0 con 0x7f16101544e0 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756378276 0 0) 0x7f15f80033f0 con 0x7f16101544e0 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 >> v2:172.21.15.158:3300/0 conn(0x7f1610154100 msgr2=0x7f161006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 --2- 172.21.15.92:0/2694194542 >> v2:172.21.15.158:3300/0 conn(0x7f1610154100 0x7f161006a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 >> v1:172.21.15.158:6789/0 conn(0x7f1610069590 legacy=0x7f16100c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100bfeb0 con 0x7f16101544e0 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f1604001470 con 0x7f16101544e0 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 >> v1:172.21.15.92:6789/0 conn(0x7f16101544e0 legacy=0x7f16100a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15f80038e0 0x7f15f8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15f8004690 0x7f15f8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f8005000 0x7f15f800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100aad80 con 0x7f15f80038e0 2023-05-16T07:18:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f800bf60 con 0x7f15f8004690 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f800c0f0 con 0x7f15f8005000 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15f80038e0 0x7f15f8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160ffff700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15f8004690 0x7f15f8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1615319700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f8005000 0x7f15f800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 -- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f8005000 msgr2=0x7f15f800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f8005000 0x7f15f800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 -- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15f8004690 msgr2=0x7f15f8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15f8004690 0x7f15f8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 -- 172.21.15.92:0/2694194542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80033f0 con 0x7f15f80038e0 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1615319700 1 -- 172.21.15.92:0/2694194542 reap_dead start 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f160ffff700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15f8004690 0x7f15f8004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.751+0000 7f1614b18700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15f80038e0 0x7f15f8003d00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f1600009a60 tx=0x7f1600009060).ready entity=mon.0 client_cookie=1e0e2b99f068c8ff server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600009ea0 con 0x7f15f80038e0 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600009430 con 0x7f15f80038e0 2023-05-16T07:18:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f160f7fe700 1 -- 172.21.15.92:0/2694194542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000e870 con 0x7f15f80038e0 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 -- 172.21.15.92:0/2694194542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15f80038e0 msgr2=0x7f15f8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15f80038e0 0x7f15f8003d00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f1600009a60 tx=0x7f1600009060).stop 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 -- 172.21.15.92:0/2694194542 shutdown_connections 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 --2- 172.21.15.92:0/2694194542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15f80038e0 0x7f15f8003d00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 -- 172.21.15.92:0/2694194542 >> 172.21.15.92:0/2694194542 conn(0x7f16100cb5a0 msgr2=0x7f16100c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 -- 172.21.15.92:0/2694194542 shutdown_connections 2023-05-16T07:18:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.752+0000 7f1615b1a700 1 -- 172.21.15.92:0/2694194542 wait complete. 2023-05-16T07:18:15.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.753+0000 7f1615b1a700 1 Processor -- start 2023-05-16T07:18:15.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.753+0000 7f1615b1a700 1 -- start start 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615b1a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f80038e0 0x7f1610150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615b1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 0x7f161014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615b1a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16100c2580 0x7f16100c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615b1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1610151ed0 con 0x7f16100c4830 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615b1a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100c2ef0 con 0x7f16100c2580 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615b1a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16100af0f0 con 0x7f15f80038e0 2023-05-16T07:18:15.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 0x7f161014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 0x7f161014e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47606/0 (socket says 172.21.15.92:47606) 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 -- 172.21.15.92:0/1352764693 learned_addr learned my addr 172.21.15.92:0/1352764693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1614b18700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f80038e0 0x7f1610150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f1615319700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16100c2580 0x7f16100c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 -- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f80038e0 msgr2=0x7f1610150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f80038e0 0x7f1610150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 -- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16100c2580 msgr2=0x7f16100c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16100c2580 0x7f16100c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.754+0000 7f160ffff700 1 -- 172.21.15.92:0/1352764693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100af3d0 con 0x7f16100c4830 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.755+0000 7f160ffff700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 0x7f161014e800 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f1604000c00 tx=0x7f1604001430).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.755+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604005240 con 0x7f16100c4830 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.755+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16040053a0 con 0x7f16100c4830 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.755+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1610152140 con 0x7f16100c4830 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.755+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16040046b0 con 0x7f16100c4830 2023-05-16T07:18:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.755+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161014ecf0 con 0x7f16100c4830 2023-05-16T07:18:15.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.756+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1610151d30 con 0x7f16100c4830 2023-05-16T07:18:15.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.756+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1604004a20 con 0x7f16100c4830 2023-05-16T07:18:15.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.756+0000 7f160cff9700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15fc05edc0 0x7f15fc061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:15.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.756+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1604079c00 con 0x7f16100c4830 2023-05-16T07:18:15.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.757+0000 7f1614b18700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15fc05edc0 0x7f15fc061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:15.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.757+0000 7f1614b18700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15fc05edc0 0x7f15fc061280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1600006d50 tx=0x7f160001a000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:15.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.760+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1610151d30 con 0x7f16100c4830 2023-05-16T07:18:15.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:15.922+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f1610151d30 con 0x7f16100c4830 2023-05-16T07:18:16.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.049+0000 7f160cff9700 1 -- 172.21.15.92:0/1352764693 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 152+0+0 (secure 0 0 0) 0x7f1610151d30 con 0x7f16100c4830 2023-05-16T07:18:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15fc05edc0 msgr2=0x7f15fc061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15fc05edc0 0x7f15fc061280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1600006d50 tx=0x7f160001a000).stop 2023-05-16T07:18:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 msgr2=0x7f161014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 0x7f161014e800 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f1604000c00 tx=0x7f1604001430).stop 2023-05-16T07:18:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 shutdown_connections 2023-05-16T07:18:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f15fc05edc0 0x7f15fc061280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15f80038e0 0x7f1610150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16100c2580 0x7f16100c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 --2- 172.21.15.92:0/1352764693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16100c4830 0x7f161014e800 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.052+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 >> 172.21.15.92:0/1352764693 conn(0x7f16100cb5a0 msgr2=0x7f16100ab380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.054+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 shutdown_connections 2023-05-16T07:18:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.054+0000 7f1615b1a700 1 -- 172.21.15.92:0/1352764693 wait complete. 2023-05-16T07:18:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:18:16.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:18:16.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.185+0000 7f2e6b7a0700 1 Processor -- start 2023-05-16T07:18:16.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.185+0000 7f2e6b7a0700 1 -- start start 2023-05-16T07:18:16.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e6b7a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2e64154470 0x7f2e640aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:16.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e6b7a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640c94d0 con 0x7f2e64154090 2023-05-16T07:18:16.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e6b7a0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640cb7e0 con 0x7f2e64068160 2023-05-16T07:18:16.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e6b7a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e640c88c0 con 0x7f2e64154470 2023-05-16T07:18:16.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e68d3b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2e64154470 0x7f2e640aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:16.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e6953c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2e64154090 0x7f2e640a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58360/0 (socket says 172.21.15.92:58360) 2023-05-16T07:18:16.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e68d3b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2e64154470 0x7f2e640aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47614/0 (socket says 172.21.15.92:47614) 2023-05-16T07:18:16.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e6953c700 1 -- 172.21.15.92:0/3943665490 learned_addr learned my addr 172.21.15.92:0/3943665490 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:16.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e68d3b700 1 -- 172.21.15.92:0/3943665490 >> v1:172.21.15.158:6789/0 conn(0x7f2e64068160 legacy=0x7f2e640c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e68d3b700 1 -- 172.21.15.92:0/3943665490 >> v1:172.21.15.92:6789/0 conn(0x7f2e64154090 legacy=0x7f2e640a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e63fff700 1 -- 172.21.15.92:0/3943665490 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747039823 0 0) 0x7f2e640c94d0 con 0x7f2e64154090 2023-05-16T07:18:16.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e68d3b700 1 -- 172.21.15.92:0/3943665490 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640c8ba0 con 0x7f2e64154470 2023-05-16T07:18:16.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e68d3b700 1 --2- 172.21.15.92:0/3943665490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64154470 0x7f2e640aaca0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f2e58009760 tx=0x7f2e58009060).ready entity=mon.0 client_cookie=36a80e5e4bef3e2e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:16.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e63fff700 1 -- 172.21.15.92:0/3943665490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800b580 con 0x7f2e64154470 2023-05-16T07:18:16.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e63fff700 1 -- 172.21.15.92:0/3943665490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5800b6e0 con 0x7f2e64154470 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.186+0000 7f2e63fff700 1 -- 172.21.15.92:0/3943665490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800ba60 con 0x7f2e64154470 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/3943665490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64154470 msgr2=0x7f2e640aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/3943665490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64154470 0x7f2e640aaca0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f2e58009760 tx=0x7f2e58009060).stop 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/3943665490 shutdown_connections 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/3943665490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64154470 0x7f2e640aaca0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/3943665490 >> 172.21.15.92:0/3943665490 conn(0x7f2e64067920 msgr2=0x7f2e640cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:16.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/3943665490 shutdown_connections 2023-05-16T07:18:16.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.187+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/3943665490 wait complete. 2023-05-16T07:18:16.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.188+0000 7f2e6b7a0700 1 Processor -- start 2023-05-16T07:18:16.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.188+0000 7f2e6b7a0700 1 -- start start 2023-05-16T07:18:16.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6b7a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 0x7f2e641646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:16.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6b7a0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e64154090 0x7f2e6414e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6b7a0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2e64154470 0x7f2e64158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6b7a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64164bc0 con 0x7f2e64068160 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6b7a0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64156540 con 0x7f2e64154090 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6b7a0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64155930 con 0x7f2e64154470 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 0x7f2e641646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 0x7f2e641646d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47622/0 (socket says 172.21.15.92:47622) 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 -- 172.21.15.92:0/4105229249 learned_addr learned my addr 172.21.15.92:0/4105229249 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:16.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e69d3d700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2e64154470 0x7f2e64158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e68d3b700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e64154090 0x7f2e6414e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 -- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2e64154470 msgr2=0x7f2e64158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2e64154470 0x7f2e64158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 -- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e64154090 msgr2=0x7f2e6414e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.189+0000 7f2e6953c700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e64154090 0x7f2e6414e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e6953c700 1 -- 172.21.15.92:0/4105229249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580092a0 con 0x7f2e64068160 2023-05-16T07:18:16.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e68d3b700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e64154090 0x7f2e6414e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e6953c700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 0x7f2e641646d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f2e5400c840 tx=0x7f2e54007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54005d30 con 0x7f2e64068160 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e54005e90 con 0x7f2e64068160 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e64155c10 con 0x7f2e64068160 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54007340 con 0x7f2e64068160 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.190+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e641561d0 con 0x7f2e64068160 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.191+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e64066730 con 0x7f2e64068160 2023-05-16T07:18:16.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.191+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2e54010030 con 0x7f2e64068160 2023-05-16T07:18:16.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.192+0000 7f2e617fa700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2e5005ed90 0x7f2e50061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:16.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.192+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6223+0+0 (secure 0 0 0) 0x7f2e540799f0 con 0x7f2e64068160 2023-05-16T07:18:16.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.192+0000 7f2e68d3b700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2e5005ed90 0x7f2e50061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:16.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.192+0000 7f2e68d3b700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2e5005ed90 0x7f2e50061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2e64159ad0 tx=0x7f2e58017000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:16.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.196+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2e5407be60 con 0x7f2e64068160 2023-05-16T07:18:16.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:16.361+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2e640c33b0 con 0x7f2e64068160 2023-05-16T07:18:17.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.057+0000 7f2e617fa700 1 -- 172.21.15.92:0/4105229249 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2e5404af30 con 0x7f2e64068160 2023-05-16T07:18:17.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.074+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2e5005ed90 msgr2=0x7f2e50061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.074+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2e5005ed90 0x7f2e50061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2e64159ad0 tx=0x7f2e58017000).stop 2023-05-16T07:18:17.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.074+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 msgr2=0x7f2e641646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.074+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 0x7f2e641646d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f2e5400c840 tx=0x7f2e54007f90).stop 2023-05-16T07:18:17.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.078+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 shutdown_connections 2023-05-16T07:18:17.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.078+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2e5005ed90 0x7f2e50061250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.078+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2e64154470 0x7f2e64158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.078+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e64068160 0x7f2e641646d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.078+0000 7f2e6b7a0700 1 --2- 172.21.15.92:0/4105229249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2e64154090 0x7f2e6414e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.078+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 >> 172.21.15.92:0/4105229249 conn(0x7f2e64067920 msgr2=0x7f2e640bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.079+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 shutdown_connections 2023-05-16T07:18:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.079+0000 7f2e6b7a0700 1 -- 172.21.15.92:0/4105229249 wait complete. 2023-05-16T07:18:17.086 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:18:17.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:18:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.207+0000 7fb6a0068700 1 Processor -- start 2023-05-16T07:18:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.208+0000 7fb6a0068700 1 -- start start 2023-05-16T07:18:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.208+0000 7fb6a0068700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb698154090 0x7fb6980aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.208+0000 7fb6a0068700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb698068160 0x7fb6980685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.208+0000 7fb6a0068700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6980c94d0 con 0x7fb698154470 2023-05-16T07:18:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.208+0000 7fb6a0068700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6980cb7e0 con 0x7fb698154090 2023-05-16T07:18:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.208+0000 7fb6a0068700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb698068aa0 con 0x7fb698068160 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69de04700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb698154090 0x7fb6980aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69d603700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb698154470 0x7fb6980a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58376/0 (socket says 172.21.15.92:58376) 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69d603700 1 -- 172.21.15.92:0/3592711807 learned_addr learned my addr 172.21.15.92:0/3592711807 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69e605700 1 --2- 172.21.15.92:0/3592711807 >> v2:172.21.15.158:3300/0 conn(0x7fb698068160 0x7fb6980685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69de04700 1 -- 172.21.15.92:0/3592711807 >> v2:172.21.15.158:3300/0 conn(0x7fb698068160 msgr2=0x7fb6980685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69de04700 1 --2- 172.21.15.92:0/3592711807 >> v2:172.21.15.158:3300/0 conn(0x7fb698068160 0x7fb6980685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69de04700 1 -- 172.21.15.92:0/3592711807 >> v1:172.21.15.92:6789/0 conn(0x7fb698154470 legacy=0x7fb6980a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69de04700 1 -- 172.21.15.92:0/3592711807 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6980c88c0 con 0x7fb698154090 2023-05-16T07:18:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69ce02700 1 -- 172.21.15.92:0/3592711807 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149227554 0 0) 0x7fb6980c94d0 con 0x7fb698154470 2023-05-16T07:18:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69e605700 1 --2- 172.21.15.92:0/3592711807 >> v2:172.21.15.158:3300/0 conn(0x7fb698068160 0x7fb6980685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69de04700 1 --2- 172.21.15.92:0/3592711807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aaca0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fb69805f9e0 tx=0x7fb688009780).ready entity=mon.0 client_cookie=4eace07eb4daf75d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69ce02700 1 -- 172.21.15.92:0/3592711807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e070 con 0x7fb698154090 2023-05-16T07:18:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69ce02700 1 -- 172.21.15.92:0/3592711807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800bc30 con 0x7fb698154090 2023-05-16T07:18:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.209+0000 7fb69ce02700 1 -- 172.21.15.92:0/3592711807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688012410 con 0x7fb698154090 2023-05-16T07:18:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 -- 172.21.15.92:0/3592711807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 msgr2=0x7fb6980aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 --2- 172.21.15.92:0/3592711807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aaca0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fb69805f9e0 tx=0x7fb688009780).stop 2023-05-16T07:18:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 -- 172.21.15.92:0/3592711807 shutdown_connections 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 --2- 172.21.15.92:0/3592711807 >> v2:172.21.15.158:3300/0 conn(0x7fb698068160 0x7fb6980685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 --2- 172.21.15.92:0/3592711807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aaca0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 -- 172.21.15.92:0/3592711807 >> 172.21.15.92:0/3592711807 conn(0x7fb698067920 msgr2=0x7fb6980cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 -- 172.21.15.92:0/3592711807 shutdown_connections 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.210+0000 7fb6a0068700 1 -- 172.21.15.92:0/3592711807 wait complete. 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.211+0000 7fb6a0068700 1 Processor -- start 2023-05-16T07:18:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.211+0000 7fb6a0068700 1 -- start start 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb6a0068700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb698068160 0x7fb698145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb6a0068700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb6a0068700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb698154470 0x7fb6980abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb6a0068700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698165800 con 0x7fb698154090 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb6a0068700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980ac450 con 0x7fb698068160 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb6a0068700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb698166440 con 0x7fb698154470 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47634/0 (socket says 172.21.15.92:47634) 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 -- 172.21.15.92:0/801082008 learned_addr learned my addr 172.21.15.92:0/801082008 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 -- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb698154470 msgr2=0x7fb6980abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb698154470 0x7fb6980abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 -- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb698068160 msgr2=0x7fb698145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb698068160 0x7fb698145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 -- 172.21.15.92:0/801082008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688009a20 con 0x7fb698154090 2023-05-16T07:18:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.212+0000 7fb69d603700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aba70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fb68c001eb0 tx=0x7fb68c007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.213+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0072e0 con 0x7fb698154090 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.213+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c007440 con 0x7fb698154090 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.213+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0057a0 con 0x7fb698154090 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.213+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb698165b10 con 0x7fb698154090 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.213+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6981660d0 con 0x7fb698154090 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.214+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb698066730 con 0x7fb698154090 2023-05-16T07:18:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.214+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb68c010070 con 0x7fb698154090 2023-05-16T07:18:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.214+0000 7fb695ffb700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb68405ed90 0x7fb684061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.214+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fb68c079450 con 0x7fb698154090 2023-05-16T07:18:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.215+0000 7fb69de04700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb68405ed90 0x7fb684061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.215+0000 7fb69de04700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb68405ed90 0x7fb684061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb688009f90 tx=0x7fb688009780).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.219+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb68c04a700 con 0x7fb698154090 2023-05-16T07:18:17.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.422+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb6980c8ba0 con 0x7fb698154090 2023-05-16T07:18:17.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.441+0000 7fb695ffb700 1 -- 172.21.15.92:0/801082008 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb68c04dd20 con 0x7fb698154090 2023-05-16T07:18:17.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb68405ed90 msgr2=0x7fb684061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb68405ed90 0x7fb684061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb688009f90 tx=0x7fb688009780).stop 2023-05-16T07:18:17.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 msgr2=0x7fb6980aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aba70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fb68c001eb0 tx=0x7fb68c007e90).stop 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 shutdown_connections 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb68405ed90 0x7fb684061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb698154470 0x7fb6980abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb698068160 0x7fb698145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 --2- 172.21.15.92:0/801082008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb698154090 0x7fb6980aba70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 >> 172.21.15.92:0/801082008 conn(0x7fb698067920 msgr2=0x7fb698153240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:17.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 shutdown_connections 2023-05-16T07:18:17.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.450+0000 7fb6a0068700 1 -- 172.21.15.92:0/801082008 wait complete. 2023-05-16T07:18:17.456 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 13 and data pool 14 2023-05-16T07:18:17.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:17.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.574+0000 7f20553ff700 1 Processor -- start 2023-05-16T07:18:17.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.574+0000 7f20553ff700 1 -- start start 2023-05-16T07:18:17.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f20553ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20501509c0 0x7f2050064e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f20553ff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2050150da0 0x7f20500c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f20553ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2050066050 con 0x7f20501547c0 2023-05-16T07:18:17.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f20553ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2050065440 con 0x7f20501509c0 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f20553ff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2050065720 con 0x7f2050150da0 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f204effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20501509c0 0x7f2050064e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f204effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20501509c0 0x7f2050064e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47654/0 (socket says 172.21.15.92:47654) 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f204effd700 1 -- 172.21.15.92:0/1791740753 learned_addr learned my addr 172.21.15.92:0/1791740753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f204f7fe700 1 --1- 172.21.15.92:0/1791740753 >> v1:172.21.15.92:6789/0 conn(0x7f20501547c0 0x7f2050066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58406/0 (socket says 172.21.15.92:58406) 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.575+0000 7f204e7fc700 1 --2- 172.21.15.92:0/1791740753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2050150da0 0x7f20500c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204effd700 1 -- 172.21.15.92:0/1791740753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2050150da0 msgr2=0x7f20500c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204effd700 1 --2- 172.21.15.92:0/1791740753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2050150da0 0x7f20500c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204effd700 1 -- 172.21.15.92:0/1791740753 >> v1:172.21.15.92:6789/0 conn(0x7f20501547c0 legacy=0x7f2050066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204effd700 1 -- 172.21.15.92:0/1791740753 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050065a00 con 0x7f20501509c0 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204effd700 1 --2- 172.21.15.92:0/1791740753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501509c0 0x7f2050064e40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f2040009f20 tx=0x7f2040009840).ready entity=mon.0 client_cookie=51a895785f3938b1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204dffb700 1 -- 172.21.15.92:0/1791740753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000e070 con 0x7f20501509c0 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204dffb700 1 -- 172.21.15.92:0/1791740753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204000ba80 con 0x7f20501509c0 2023-05-16T07:18:17.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f204dffb700 1 -- 172.21.15.92:0/1791740753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000be00 con 0x7f20501509c0 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 -- 172.21.15.92:0/1791740753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501509c0 msgr2=0x7f2050064e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 --2- 172.21.15.92:0/1791740753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501509c0 0x7f2050064e40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f2040009f20 tx=0x7f2040009840).stop 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 -- 172.21.15.92:0/1791740753 shutdown_connections 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 --2- 172.21.15.92:0/1791740753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501509c0 0x7f2050064e40 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 --2- 172.21.15.92:0/1791740753 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2050150da0 0x7f20500c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 -- 172.21.15.92:0/1791740753 >> 172.21.15.92:0/1791740753 conn(0x7f20500c6f40 msgr2=0x7f20500c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:17.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 -- 172.21.15.92:0/1791740753 shutdown_connections 2023-05-16T07:18:17.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.576+0000 7f20553ff700 1 -- 172.21.15.92:0/1791740753 wait complete. 2023-05-16T07:18:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.577+0000 7f20553ff700 1 Processor -- start 2023-05-16T07:18:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 -- start start 2023-05-16T07:18:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20501509c0 0x7f2050142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2050150da0 0x7f20500ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 0x7f20500acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501623d0 con 0x7f20501547c0 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501626b0 con 0x7f2050150da0 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f20553ff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2050162990 con 0x7f20501509c0 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 0x7f20500acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 0x7f20500acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47664/0 (socket says 172.21.15.92:47664) 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f204f7fe700 1 -- 172.21.15.92:0/3064254122 learned_addr learned my addr 172.21.15.92:0/3064254122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.578+0000 7f204effd700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20501509c0 0x7f2050142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204e7fc700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2050150da0 0x7f20500ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204f7fe700 1 -- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20501509c0 msgr2=0x7f2050142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204f7fe700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20501509c0 0x7f2050142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204f7fe700 1 -- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2050150da0 msgr2=0x7f20500ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204f7fe700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2050150da0 0x7f20500ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204f7fe700 1 -- 172.21.15.92:0/3064254122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2040009ae0 con 0x7f20501547c0 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204e7fc700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2050150da0 0x7f20500ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f204f7fe700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 0x7f20500acb00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f204800c440 tx=0x7f2048007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800ce40 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20480072e0 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2050162c70 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20480056e0 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.579+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501620c0 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.580+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050047150 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.580+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f204801a060 con 0x7f20501547c0 2023-05-16T07:18:17.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.580+0000 7f2046ffd700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f203005ed90 0x7f2030061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.581+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f2048079400 con 0x7f20501547c0 2023-05-16T07:18:17.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.581+0000 7f204effd700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f203005ed90 0x7f2030061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.581+0000 7f204effd700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f203005ed90 0x7f2030061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f204000ba50 tx=0x7f2040009840).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.585+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f204804a640 con 0x7f20501547c0 2023-05-16T07:18:17.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.748+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20500bc4b0 con 0x7f20501547c0 2023-05-16T07:18:17.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.749+0000 7f2046ffd700 1 -- 172.21.15.92:0/3064254122 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17885 (secure 0 0 0) 0x7f2048018020 con 0x7f20501547c0 2023-05-16T07:18:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:17.753 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":106,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:17.423241+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:18:15.946822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-05-16T07:18:16.384841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:17.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.750+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f203005ed90 msgr2=0x7f2030061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.750+0000 7f20553ff700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f203005ed90 0x7f2030061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f204000ba50 tx=0x7f2040009840).stop 2023-05-16T07:18:17.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.750+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 msgr2=0x7f20500acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.750+0000 7f20553ff700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 0x7f20500acb00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f204800c440 tx=0x7f2048007dd0).stop 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 shutdown_connections 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f203005ed90 0x7f2030061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f20501509c0 0x7f2050142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20501547c0 0x7f20500acb00 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 --2- 172.21.15.92:0/3064254122 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2050150da0 0x7f20500ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 >> 172.21.15.92:0/3064254122 conn(0x7f20500c6f40 msgr2=0x7f205014dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:17.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 shutdown_connections 2023-05-16T07:18:17.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.751+0000 7f20553ff700 1 -- 172.21.15.92:0/3064254122 wait complete. 2023-05-16T07:18:17.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:17.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.875+0000 7f2bbf4db700 1 Processor -- start 2023-05-16T07:18:17.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 -- start start 2023-05-16T07:18:17.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2bb813b550 0x7f2bb80c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb813b930 0x7f2bb80abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2bb8062c80 0x7f2bb8063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb80c5400 con 0x7f2bb813b550 2023-05-16T07:18:17.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb80bdc30 con 0x7f2bb8062c80 2023-05-16T07:18:17.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.876+0000 7f2bbf4db700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8063130 con 0x7f2bb813b930 2023-05-16T07:18:17.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2bb813b550 0x7f2bb80c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2bb813b550 0x7f2bb80c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47668/0 (socket says 172.21.15.92:47668) 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 -- 172.21.15.92:0/961512088 learned_addr learned my addr 172.21.15.92:0/961512088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbda78700 1 --2- 172.21.15.92:0/961512088 >> v2:172.21.15.158:3300/0 conn(0x7f2bb8062c80 0x7f2bb8063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbca76700 1 --2- 172.21.15.92:0/961512088 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb813b930 0x7f2bb80abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 -- 172.21.15.92:0/961512088 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb813b930 msgr2=0x7f2bb80abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 --2- 172.21.15.92:0/961512088 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb813b930 0x7f2bb80abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 -- 172.21.15.92:0/961512088 >> v2:172.21.15.158:3300/0 conn(0x7f2bb8062c80 msgr2=0x7f2bb8063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:17.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 --2- 172.21.15.92:0/961512088 >> v2:172.21.15.158:3300/0 conn(0x7f2bb8062c80 0x7f2bb8063720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 -- 172.21.15.92:0/961512088 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80c4e40 con 0x7f2bb813b550 2023-05-16T07:18:17.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2bbd277700 1 --2- 172.21.15.92:0/961512088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80c31f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f2bb8065250 tx=0x7f2bb40090f0).ready entity=mon.0 client_cookie=732d6fb00b4b1743 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2baffff700 1 -- 172.21.15.92:0/961512088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f510 con 0x7f2bb813b550 2023-05-16T07:18:17.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2baffff700 1 -- 172.21.15.92:0/961512088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb400f670 con 0x7f2bb813b550 2023-05-16T07:18:17.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.877+0000 7f2baffff700 1 -- 172.21.15.92:0/961512088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f9f0 con 0x7f2bb813b550 2023-05-16T07:18:17.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 -- 172.21.15.92:0/961512088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 msgr2=0x7f2bb80c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/961512088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80c31f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f2bb8065250 tx=0x7f2bb40090f0).stop 2023-05-16T07:18:17.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 -- 172.21.15.92:0/961512088 shutdown_connections 2023-05-16T07:18:17.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/961512088 >> v2:172.21.15.158:3300/0 conn(0x7f2bb8062c80 0x7f2bb8063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/961512088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80c31f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/961512088 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb813b930 0x7f2bb80abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 -- 172.21.15.92:0/961512088 >> 172.21.15.92:0/961512088 conn(0x7f2bb80c8aa0 msgr2=0x7f2bb80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 -- 172.21.15.92:0/961512088 shutdown_connections 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.878+0000 7f2bbf4db700 1 -- 172.21.15.92:0/961512088 wait complete. 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.879+0000 7f2bbf4db700 1 Processor -- start 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.879+0000 7f2bbf4db700 1 -- start start 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbf4db700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb8062c80 0x7f2bb812c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbf4db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbf4db700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2bb813b930 0x7f2bb80c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbf4db700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c12c0 con 0x7f2bb813b550 2023-05-16T07:18:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbf4db700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c06b0 con 0x7f2bb813b930 2023-05-16T07:18:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbf4db700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c0990 con 0x7f2bb8062c80 2023-05-16T07:18:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47674/0 (socket says 172.21.15.92:47674) 2023-05-16T07:18:17.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 -- 172.21.15.92:0/1152014389 learned_addr learned my addr 172.21.15.92:0/1152014389 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:17.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbd277700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb8062c80 0x7f2bb812c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbda78700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2bb813b930 0x7f2bb80c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 -- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb8062c80 msgr2=0x7f2bb812c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb8062c80 0x7f2bb812c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 -- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2bb813b930 msgr2=0x7f2bb80c00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2bb813b930 0x7f2bb80c00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 -- 172.21.15.92:0/1152014389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4009370 con 0x7f2bb813b550 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.880+0000 7f2bbca76700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80bfbf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f2ba800e950 tx=0x7f2ba8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.881+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8012600 con 0x7f2bb813b550 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.881+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8012760 con 0x7f2bb813b550 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.881+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb80c0c70 con 0x7f2bb813b550 2023-05-16T07:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.881+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8012ae0 con 0x7f2bb813b550 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.881+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8137ca0 con 0x7f2bb813b550 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.881+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8065930 con 0x7f2bb813b550 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.882+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2ba8022070 con 0x7f2bb813b550 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.882+0000 7f2bad7fa700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2b9c05ed90 0x7f2b9c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.882+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f2ba807a0e0 con 0x7f2bb813b550 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.882+0000 7f2bbd277700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2b9c05ed90 0x7f2b9c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.883+0000 7f2bbd277700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2b9c05ed90 0x7f2b9c061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2bb4009130 tx=0x7f2bb4017000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:17.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:17.886+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2ba804b430 con 0x7f2bb813b550 2023-05-16T07:18:18.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.049+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bb806c350 con 0x7f2bb813b550 2023-05-16T07:18:18.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.050+0000 7f2bad7fa700 1 -- 172.21.15.92:0/1152014389 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17885 (secure 0 0 0) 0x7f2ba8016090 con 0x7f2bb813b550 2023-05-16T07:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":106,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:17.423241+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:18:15.946822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-05-16T07:18:16.384841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:18.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2b9c05ed90 msgr2=0x7f2b9c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2b9c05ed90 0x7f2b9c061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2bb4009130 tx=0x7f2bb4017000).stop 2023-05-16T07:18:18.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 msgr2=0x7f2bb80bfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80bfbf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f2ba800e950 tx=0x7f2ba8009060).stop 2023-05-16T07:18:18.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 shutdown_connections 2023-05-16T07:18:18.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2b9c05ed90 0x7f2b9c061250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2bb8062c80 0x7f2bb812c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2bb813b930 0x7f2bb80c00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 --2- 172.21.15.92:0/1152014389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb813b550 0x7f2bb80bfbf0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 >> 172.21.15.92:0/1152014389 conn(0x7f2bb80c8aa0 msgr2=0x7f2bb80ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:18.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 shutdown_connections 2023-05-16T07:18:18.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.052+0000 7f2bbf4db700 1 -- 172.21.15.92:0/1152014389 wait complete. 2023-05-16T07:18:18.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:18:18.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.179+0000 7f63d28aa700 1 Processor -- start 2023-05-16T07:18:18.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.179+0000 7f63d28aa700 1 -- start start 2023-05-16T07:18:18.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63d28aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63cc156050 0x7f63cc14f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63d28aa700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f63cc14f950 0x7f63cc0646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63d28aa700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc064260 con 0x7f63cc155c70 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63d28aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0689a0 con 0x7f63cc156050 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63d28aa700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc1564b0 con 0x7f63cc14f950 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63cc156050 0x7f63cc14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63cc156050 0x7f63cc14f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47682/0 (socket says 172.21.15.92:47682) 2023-05-16T07:18:18.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1948043526 learned_addr learned my addr 172.21.15.92:0/1948043526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63d0e47700 1 --2- 172.21.15.92:0/1948043526 >> v2:172.21.15.158:3300/0 conn(0x7f63cc14f950 0x7f63cc0646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1948043526 >> v2:172.21.15.158:3300/0 conn(0x7f63cc14f950 msgr2=0x7f63cc0646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1948043526 >> v2:172.21.15.158:3300/0 conn(0x7f63cc14f950 0x7f63cc0646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1948043526 >> v1:172.21.15.158:6789/0 conn(0x7f63cc155c70 legacy=0x7f63cc069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 -- 172.21.15.92:0/1948043526 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0683e0 con 0x7f63cc156050 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1948043526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc156050 0x7f63cc14f460 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f63cc0641d0 tx=0x7f63bc0090f0).ready entity=mon.0 client_cookie=30ea0655e4e50fb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63caffd700 1 -- 172.21.15.92:0/1948043526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00b5a0 con 0x7f63cc156050 2023-05-16T07:18:18.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63caffd700 1 -- 172.21.15.92:0/1948043526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00b700 con 0x7f63cc156050 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.180+0000 7f63caffd700 1 -- 172.21.15.92:0/1948043526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00ba80 con 0x7f63cc156050 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 -- 172.21.15.92:0/1948043526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc156050 msgr2=0x7f63cc14f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1948043526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc156050 0x7f63cc14f460 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f63cc0641d0 tx=0x7f63bc0090f0).stop 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 -- 172.21.15.92:0/1948043526 shutdown_connections 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1948043526 >> v2:172.21.15.158:3300/0 conn(0x7f63cc14f950 0x7f63cc0646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1948043526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc156050 0x7f63cc14f460 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 -- 172.21.15.92:0/1948043526 >> 172.21.15.92:0/1948043526 conn(0x7f63cc0c3d90 msgr2=0x7f63cc0c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:18.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 -- 172.21.15.92:0/1948043526 shutdown_connections 2023-05-16T07:18:18.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.181+0000 7f63d28aa700 1 -- 172.21.15.92:0/1948043526 wait complete. 2023-05-16T07:18:18.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.182+0000 7f63d28aa700 1 Processor -- start 2023-05-16T07:18:18.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.182+0000 7f63d28aa700 1 -- start start 2023-05-16T07:18:18.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d28aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 0x7f63cc0c0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d28aa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc155c70 0x7f63cc0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d28aa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc156050 0x7f63cc0a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d28aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc14a080 con 0x7f63cc14f950 2023-05-16T07:18:18.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d28aa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1508e0 con 0x7f63cc156050 2023-05-16T07:18:18.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d28aa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0a6bd0 con 0x7f63cc155c70 2023-05-16T07:18:18.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 0x7f63cc0c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 0x7f63cc0c0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47688/0 (socket says 172.21.15.92:47688) 2023-05-16T07:18:18.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 -- 172.21.15.92:0/1673101838 learned_addr learned my addr 172.21.15.92:0/1673101838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:18.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63d0e47700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc156050 0x7f63cc0a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc155c70 0x7f63cc0a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 -- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc155c70 msgr2=0x7f63cc0a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc155c70 0x7f63cc0a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 -- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc156050 msgr2=0x7f63cc0a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc156050 0x7f63cc0a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 -- 172.21.15.92:0/1673101838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc009370 con 0x7f63cc14f950 2023-05-16T07:18:18.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc155c70 0x7f63cc0a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:18.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.183+0000 7f63cbfff700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 0x7f63cc0c0300 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f63c000c440 tx=0x7f63c0007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:18.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.184+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000cea0 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.184+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c000f470 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.184+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc0a6eb0 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.184+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000f810 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.184+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc0a7470 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.184+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc06c350 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.186+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f63c0019030 con 0x7f63cc14f950 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.186+0000 7f63b3fff700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63b805ed90 0x7f63b8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.186+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f63c0079900 con 0x7f63cc14f950 2023-05-16T07:18:18.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.186+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63b805ed90 0x7f63b8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.187+0000 7f63cb7fe700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63b805ed90 0x7f63b8061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f63bc009130 tx=0x7f63bc015000).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:18.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.189+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f63c004ad20 con 0x7f63cc14f950 2023-05-16T07:18:18.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.398+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f63cc0a7a80 con 0x7f63cc14f950 2023-05-16T07:18:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.427+0000 7f63b3fff700 1 -- 172.21.15.92:0/1673101838 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7f63c0021070 con 0x7f63cc14f950 2023-05-16T07:18:18.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63b805ed90 msgr2=0x7f63b8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63b805ed90 0x7f63b8061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f63bc009130 tx=0x7f63bc015000).stop 2023-05-16T07:18:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 msgr2=0x7f63cc0c0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 0x7f63cc0c0300 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f63c000c440 tx=0x7f63c0007de0).stop 2023-05-16T07:18:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 shutdown_connections 2023-05-16T07:18:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f63b805ed90 0x7f63b8061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f63cc156050 0x7f63cc0a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63cc14f950 0x7f63cc0c0300 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 --2- 172.21.15.92:0/1673101838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f63cc155c70 0x7f63cc0a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 >> 172.21.15.92:0/1673101838 conn(0x7f63cc0c3d90 msgr2=0x7f63cc0c53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 shutdown_connections 2023-05-16T07:18:18.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.429+0000 7f63d28aa700 1 -- 172.21.15.92:0/1673101838 wait complete. 2023-05-16T07:18:18.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:18:18.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.554+0000 7fd53759e700 1 Processor -- start 2023-05-16T07:18:18.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.554+0000 7fd53759e700 1 -- start start 2023-05-16T07:18:18.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.555+0000 7fd53759e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd5380bdf20 0x7fd5380698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.555+0000 7fd53759e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 0x7fd538067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.555+0000 7fd53759e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53805e760 con 0x7fd538153bb0 2023-05-16T07:18:18.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.555+0000 7fd53759e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd538069ef0 con 0x7fd5380bdf20 2023-05-16T07:18:18.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.555+0000 7fd53759e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380bf540 con 0x7fd538153f90 2023-05-16T07:18:18.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 0x7fd538067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 0x7fd538067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:40270/0 (socket says 172.21.15.92:40270) 2023-05-16T07:18:18.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 -- 172.21.15.92:0/2262270029 learned_addr learned my addr 172.21.15.92:0/2262270029 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:18.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd535d9b700 1 --2- 172.21.15.92:0/2262270029 >> v2:172.21.15.158:3300/0 conn(0x7fd5380bdf20 0x7fd5380698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 -- 172.21.15.92:0/2262270029 >> v2:172.21.15.158:3300/0 conn(0x7fd5380bdf20 msgr2=0x7fd5380698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 --2- 172.21.15.92:0/2262270029 >> v2:172.21.15.158:3300/0 conn(0x7fd5380bdf20 0x7fd5380698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 -- 172.21.15.92:0/2262270029 >> v1:172.21.15.158:6789/0 conn(0x7fd538153bb0 legacy=0x7fd5380bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd53559a700 1 -- 172.21.15.92:0/2262270029 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692399801 0 0) 0x7fd53805e760 con 0x7fd538153bb0 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 -- 172.21.15.92:0/2262270029 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd538068450 con 0x7fd538153f90 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd535d9b700 1 --2- 172.21.15.92:0/2262270029 >> v2:172.21.15.158:3300/0 conn(0x7fd5380bdf20 0x7fd5380698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:18:18.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd536d9d700 1 --2- 172.21.15.92:0/2262270029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 0x7fd538067f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd530006fd0 tx=0x7fd5300090f0).ready entity=mon.2 client_cookie=d8528cd4c5f122f4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:18.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd53559a700 1 -- 172.21.15.92:0/2262270029 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000b580 con 0x7fd538153f90 2023-05-16T07:18:18.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd53559a700 1 -- 172.21.15.92:0/2262270029 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000b6e0 con 0x7fd538153f90 2023-05-16T07:18:18.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.556+0000 7fd53559a700 1 -- 172.21.15.92:0/2262270029 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000ba60 con 0x7fd538153f90 2023-05-16T07:18:18.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 -- 172.21.15.92:0/2262270029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 msgr2=0x7fd538067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 --2- 172.21.15.92:0/2262270029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 0x7fd538067f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd530006fd0 tx=0x7fd5300090f0).stop 2023-05-16T07:18:18.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 -- 172.21.15.92:0/2262270029 shutdown_connections 2023-05-16T07:18:18.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 --2- 172.21.15.92:0/2262270029 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd538153f90 0x7fd538067f60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 --2- 172.21.15.92:0/2262270029 >> v2:172.21.15.158:3300/0 conn(0x7fd5380bdf20 0x7fd5380698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 -- 172.21.15.92:0/2262270029 >> 172.21.15.92:0/2262270029 conn(0x7fd538062d80 msgr2=0x7fd53805dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:18.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 -- 172.21.15.92:0/2262270029 shutdown_connections 2023-05-16T07:18:18.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.557+0000 7fd53759e700 1 -- 172.21.15.92:0/2262270029 wait complete. 2023-05-16T07:18:18.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.558+0000 7fd53759e700 1 Processor -- start 2023-05-16T07:18:18.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.558+0000 7fd53759e700 1 -- start start 2023-05-16T07:18:18.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53759e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5380bdf20 0x7fd538163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 0x7fd5381533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53759e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd538153f90 0x7fd5380ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381643c0 con 0x7fd538153bb0 2023-05-16T07:18:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53759e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380bfc50 con 0x7fd538153f90 2023-05-16T07:18:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53759e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381556b0 con 0x7fd5380bdf20 2023-05-16T07:18:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 0x7fd5381533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 0x7fd5381533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47704/0 (socket says 172.21.15.92:47704) 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 -- 172.21.15.92:0/1237460945 learned_addr learned my addr 172.21.15.92:0/1237460945 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd536d9d700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd538153f90 0x7fd5380ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd53659c700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5380bdf20 0x7fd538163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 -- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5380bdf20 msgr2=0x7fd538163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5380bdf20 0x7fd538163ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 -- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd538153f90 msgr2=0x7fd5380ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd538153f90 0x7fd5380ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 -- 172.21.15.92:0/1237460945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200092a0 con 0x7fd538153bb0 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd536d9d700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd538153f90 0x7fd5380ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.559+0000 7fd535d9b700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 0x7fd5381533f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fd52000e980 tx=0x7fd520009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.560+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000c7f0 con 0x7fd538153bb0 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.560+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52000c950 con 0x7fd538153bb0 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.560+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd530009370 con 0x7fd538153bb0 2023-05-16T07:18:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.560+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000ccd0 con 0x7fd538153bb0 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.560+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd538155c50 con 0x7fd538153bb0 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.561+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd538047150 con 0x7fd538153bb0 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.561+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd520022070 con 0x7fd538153bb0 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.561+0000 7fd52e7fc700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd51805ed90 0x7fd518061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.561+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fd52007a220 con 0x7fd538153bb0 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.562+0000 7fd53659c700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd51805ed90 0x7fd518061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.562+0000 7fd53659c700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd51805ed90 0x7fd518061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd528001560 tx=0x7fd528009040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:18.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.565+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd52004b570 con 0x7fd538153bb0 2023-05-16T07:18:18.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:18.770+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd538158460 con 0x7fd538153bb0 2023-05-16T07:18:19.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.649+0000 7fd52e7fc700 1 -- 172.21.15.92:0/1237460945 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd52004eb90 con 0x7fd538153bb0 2023-05-16T07:18:19.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd51805ed90 msgr2=0x7fd518061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:19.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd51805ed90 0x7fd518061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd528001560 tx=0x7fd528009040).stop 2023-05-16T07:18:19.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 msgr2=0x7fd5381533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:19.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 0x7fd5381533f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fd52000e980 tx=0x7fd520009060).stop 2023-05-16T07:18:19.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 shutdown_connections 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd51805ed90 0x7fd518061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5380bdf20 0x7fd538163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd538153f90 0x7fd5380ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 --2- 172.21.15.92:0/1237460945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd538153bb0 0x7fd5381533f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.651+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 >> 172.21.15.92:0/1237460945 conn(0x7fd538062d80 msgr2=0x7fd5380c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.652+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 shutdown_connections 2023-05-16T07:18:19.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.652+0000 7fd53759e700 1 -- 172.21.15.92:0/1237460945 wait complete. 2023-05-16T07:18:19.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.781+0000 7f02fadab700 1 Processor -- start 2023-05-16T07:18:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.781+0000 7f02fadab700 1 -- start start 2023-05-16T07:18:19.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02fadab700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:19.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02fadab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02f4069b90 0x7f02f40c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:19.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02fadab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f40c0660 con 0x7f02f4154170 2023-05-16T07:18:19.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02fadab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f406a020 con 0x7f02f4069b90 2023-05-16T07:18:19.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02fadab700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f40bfd30 con 0x7f02f4154550 2023-05-16T07:18:19.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02f9348700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02f4069b90 0x7f02f40c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:19.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02f8b47700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f02f4154170 0x7f02f40c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58456/0 (socket says 172.21.15.92:58456) 2023-05-16T07:18:19.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02f8b47700 1 -- 172.21.15.92:0/1349794160 learned_addr learned my addr 172.21.15.92:0/1349794160 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:19.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.782+0000 7f02f3fff700 1 --2- 172.21.15.92:0/1349794160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:19.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f9348700 1 -- 172.21.15.92:0/1349794160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 msgr2=0x7f02f40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:19.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f9348700 1 --2- 172.21.15.92:0/1349794160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f9348700 1 -- 172.21.15.92:0/1349794160 >> v1:172.21.15.92:6789/0 conn(0x7f02f4154170 legacy=0x7f02f40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:19.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f37fe700 1 -- 172.21.15.92:0/1349794160 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256430551 0 0) 0x7f02f40c0660 con 0x7f02f4154170 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f9348700 1 -- 172.21.15.92:0/1349794160 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f40c0010 con 0x7f02f4069b90 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f9348700 1 --2- 172.21.15.92:0/1349794160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f40c0af0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f02ec009760 tx=0x7f02ec009060).ready entity=mon.0 client_cookie=7f2cb7c8a4fd14b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f37fe700 1 -- 172.21.15.92:0/1349794160 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec0090d0 con 0x7f02f4069b90 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f37fe700 1 -- 172.21.15.92:0/1349794160 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec00b580 con 0x7f02f4069b90 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02f37fe700 1 -- 172.21.15.92:0/1349794160 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00b900 con 0x7f02f4069b90 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 -- 172.21.15.92:0/1349794160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 msgr2=0x7f02f40c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 --2- 172.21.15.92:0/1349794160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f40c0af0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f02ec009760 tx=0x7f02ec009060).stop 2023-05-16T07:18:19.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 -- 172.21.15.92:0/1349794160 shutdown_connections 2023-05-16T07:18:19.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 --2- 172.21.15.92:0/1349794160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f40c0af0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 --2- 172.21.15.92:0/1349794160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 -- 172.21.15.92:0/1349794160 >> 172.21.15.92:0/1349794160 conn(0x7f02f40cb700 msgr2=0x7f02f40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:19.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 -- 172.21.15.92:0/1349794160 shutdown_connections 2023-05-16T07:18:19.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.783+0000 7f02fadab700 1 -- 172.21.15.92:0/1349794160 wait complete. 2023-05-16T07:18:19.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.784+0000 7f02fadab700 1 Processor -- start 2023-05-16T07:18:19.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 -- start start 2023-05-16T07:18:19.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f4164200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:19.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02f4154170 0x7f02f41681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:19.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:19.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41651f0 con 0x7f02f4069b90 2023-05-16T07:18:19.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41646f0 con 0x7f02f4154170 2023-05-16T07:18:19.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02fadab700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f4168690 con 0x7f02f4154550 2023-05-16T07:18:19.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02f8b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f4164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:19.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02f8b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f4164200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47712/0 (socket says 172.21.15.92:47712) 2023-05-16T07:18:19.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.785+0000 7f02f8b47700 1 -- 172.21.15.92:0/3548853280 learned_addr learned my addr 172.21.15.92:0/3548853280 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f9348700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f3fff700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02f4154170 0x7f02f41681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f8b47700 1 -- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 msgr2=0x7f02f40ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f8b47700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f8b47700 1 -- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02f4154170 msgr2=0x7f02f41681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f8b47700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02f4154170 0x7f02f41681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:19.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f8b47700 1 -- 172.21.15.92:0/3548853280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0092a0 con 0x7f02f4069b90 2023-05-16T07:18:19.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f8b47700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f4164200 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f02e400c960 tx=0x7f02e4001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:19.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4005d30 con 0x7f02f4069b90 2023-05-16T07:18:19.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e4005e90 con 0x7f02f4069b90 2023-05-16T07:18:19.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f40c3c60 con 0x7f02f4069b90 2023-05-16T07:18:19.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4007320 con 0x7f02f4069b90 2023-05-16T07:18:19.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.786+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f4164c30 con 0x7f02f4069b90 2023-05-16T07:18:19.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.787+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f40666c0 con 0x7f02f4069b90 2023-05-16T07:18:19.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.787+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f02e4010030 con 0x7f02f4069b90 2023-05-16T07:18:19.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.788+0000 7f02f0ff9700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f02e005ed60 0x7f02e0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:19.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.788+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f02e4079c60 con 0x7f02f4069b90 2023-05-16T07:18:19.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.788+0000 7f02f3fff700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f02e005ed60 0x7f02e0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:19.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.788+0000 7f02f3fff700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f02e005ed60 0x7f02e0061220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f02f40ac860 tx=0x7f02e8008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:19.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.792+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f02e404af50 con 0x7f02f4069b90 2023-05-16T07:18:19.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.994+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02f406c350 con 0x7f02f4069b90 2023-05-16T07:18:19.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.995+0000 7f02f0ff9700 1 -- 172.21.15.92:0/3548853280 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f02e4001770 con 0x7f02f4069b90 2023-05-16T07:18:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:19.999 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:18:17.423562+0000","modified":"2023-05-16T07:18:19.645369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7017":{"gid":7017,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1226709881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1226709881},{"type":"v1","addr":"172.21.15.92:6835","nonce":1226709881}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-05-16T07:18:20.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f02e005ed60 msgr2=0x7f02e0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f02e005ed60 0x7f02e0061220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f02f40ac860 tx=0x7f02e8008040).stop 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 msgr2=0x7f02f4164200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f4164200 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f02e400c960 tx=0x7f02e4001630).stop 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 shutdown_connections 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f02e005ed60 0x7f02e0061220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f02f4154550 0x7f02f40ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f4069b90 0x7f02f4164200 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 --2- 172.21.15.92:0/3548853280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02f4154170 0x7f02f41681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 >> 172.21.15.92:0/3548853280 conn(0x7f02f40cb700 msgr2=0x7f02f414fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 shutdown_connections 2023-05-16T07:18:20.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:19.997+0000 7f02fadab700 1 -- 172.21.15.92:0/3548853280 wait complete. 2023-05-16T07:18:20.002 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-05-16T07:18:20.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:18:20.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.119+0000 7fc986078700 1 Processor -- start 2023-05-16T07:18:20.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.119+0000 7fc986078700 1 -- start start 2023-05-16T07:18:20.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc986078700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 0x7fc9800c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc986078700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 0x7fc9800c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc986078700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9800c0660 con 0x7fc980154550 2023-05-16T07:18:20.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc986078700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9800c1b00 con 0x7fc980154170 2023-05-16T07:18:20.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc986078700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800bfd30 con 0x7fc980069db0 2023-05-16T07:18:20.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 0x7fc9800c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc97f7fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 0x7fc9800c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 0x7fc9800c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:40296/0 (socket says 172.21.15.92:40296) 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc97f7fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 0x7fc9800c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:51356/0 (socket says 172.21.15.92:51356) 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.120+0000 7fc97ffff700 1 -- 172.21.15.92:0/973061087 learned_addr learned my addr 172.21.15.92:0/973061087 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97ffff700 1 -- 172.21.15.92:0/973061087 >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 msgr2=0x7fc9800c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97ffff700 1 --2- 172.21.15.92:0/973061087 >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 0x7fc9800c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97ffff700 1 -- 172.21.15.92:0/973061087 >> v1:172.21.15.158:6789/0 conn(0x7fc980154550 legacy=0x7fc9800696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97ffff700 1 -- 172.21.15.92:0/973061087 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800c0010 con 0x7fc980069db0 2023-05-16T07:18:20.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97f7fe700 1 --2- 172.21.15.92:0/973061087 >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 0x7fc9800c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:18:20.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97ffff700 1 --2- 172.21.15.92:0/973061087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 0x7fc9800c6160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fc978006fd0 tx=0x7fc9780090f0).ready entity=mon.2 client_cookie=7ebd06e9dd2f87e3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97e7fc700 1 -- 172.21.15.92:0/973061087 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800b580 con 0x7fc980069db0 2023-05-16T07:18:20.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97e7fc700 1 -- 172.21.15.92:0/973061087 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97800b6e0 con 0x7fc980069db0 2023-05-16T07:18:20.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.121+0000 7fc97e7fc700 1 -- 172.21.15.92:0/973061087 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800ba80 con 0x7fc980069db0 2023-05-16T07:18:20.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 -- 172.21.15.92:0/973061087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 msgr2=0x7fc9800c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 --2- 172.21.15.92:0/973061087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 0x7fc9800c6160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fc978006fd0 tx=0x7fc9780090f0).stop 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 -- 172.21.15.92:0/973061087 shutdown_connections 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 --2- 172.21.15.92:0/973061087 >> v2:172.21.15.158:3300/0 conn(0x7fc980154170 0x7fc9800c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 --2- 172.21.15.92:0/973061087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980069db0 0x7fc9800c6160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 -- 172.21.15.92:0/973061087 >> 172.21.15.92:0/973061087 conn(0x7fc9800cb700 msgr2=0x7fc9800c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 -- 172.21.15.92:0/973061087 shutdown_connections 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.122+0000 7fc986078700 1 -- 172.21.15.92:0/973061087 wait complete. 2023-05-16T07:18:20.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.123+0000 7fc986078700 1 Processor -- start 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.123+0000 7fc986078700 1 -- start start 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 0x7fc9800c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980154170 0x7fc9800ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc980154550 0x7fc98014d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980155f00 con 0x7fc980069db0 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800af6c0 con 0x7fc980154550 2023-05-16T07:18:20.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801561c0 con 0x7fc980154170 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 0x7fc9800c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 0x7fc9800c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47726/0 (socket says 172.21.15.92:47726) 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 -- 172.21.15.92:0/2867352530 learned_addr learned my addr 172.21.15.92:0/2867352530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97effd700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980154170 0x7fc9800ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97ffff700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc980154550 0x7fc98014d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 -- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980154170 msgr2=0x7fc9800ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980154170 0x7fc9800ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 -- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc980154550 msgr2=0x7fc98014d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc980154550 0x7fc98014d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 -- 172.21.15.92:0/2867352530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970009a20 con 0x7fc980069db0 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97ffff700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc980154550 0x7fc98014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc97f7fe700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 0x7fc9800c4ab0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7fc97000c1a0 tx=0x7fc97000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000cd90 con 0x7fc980069db0 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97000f040 con 0x7fc980069db0 2023-05-16T07:18:20.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc978009370 con 0x7fc980069db0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970013610 con 0x7fc980069db0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.124+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9800aeed0 con 0x7fc980069db0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.125+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9800666c0 con 0x7fc980069db0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.125+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc9700139c0 con 0x7fc980069db0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.126+0000 7fc9677fe700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc96c05ed90 0x7fc96c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.126+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fc97007a8e0 con 0x7fc980069db0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.126+0000 7fc97effd700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc96c05ed90 0x7fc96c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.126+0000 7fc97effd700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc96c05ed90 0x7fc96c061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc980167a30 tx=0x7fc974007d40).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.130+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc97004bc30 con 0x7fc980069db0 2023-05-16T07:18:20.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.293+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc98006c350 con 0x7fc980069db0 2023-05-16T07:18:20.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.294+0000 7fc9677fe700 1 -- 172.21.15.92:0/2867352530 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17872 (secure 0 0 0) 0x7fc97004f250 con 0x7fc980069db0 2023-05-16T07:18:20.297 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:20.298 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":107,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:18.425215+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:18:15.946822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-05-16T07:18:16.384841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:18:20.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc96c05ed90 msgr2=0x7fc96c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc96c05ed90 0x7fc96c061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc980167a30 tx=0x7fc974007d40).stop 2023-05-16T07:18:20.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 msgr2=0x7fc9800c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 0x7fc9800c4ab0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7fc97000c1a0 tx=0x7fc97000b040).stop 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 shutdown_connections 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc96c05ed90 0x7fc96c061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc980154550 0x7fc98014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc980069db0 0x7fc9800c4ab0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 --2- 172.21.15.92:0/2867352530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc980154170 0x7fc9800ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 >> 172.21.15.92:0/2867352530 conn(0x7fc9800cb700 msgr2=0x7fc9800acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 shutdown_connections 2023-05-16T07:18:20.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.295+0000 7fc986078700 1 -- 172.21.15.92:0/2867352530 wait complete. 2023-05-16T07:18:20.311 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:18:20.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:18:20.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.418+0000 7f348dd89700 1 Processor -- start 2023-05-16T07:18:20.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.418+0000 7f348dd89700 1 -- start start 2023-05-16T07:18:20.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f348dd89700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f348813b550 0x7f34880c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f348dd89700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f348813b930 0x7f34880abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f348dd89700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34880bdc30 con 0x7f3488062c80 2023-05-16T07:18:20.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f348dd89700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34880c5400 con 0x7f348813b550 2023-05-16T07:18:20.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f348dd89700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3488063110 con 0x7f348813b930 2023-05-16T07:18:20.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f34877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f348813b550 0x7f34880c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f3487fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3488062c80 0x7f3488063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58482/0 (socket says 172.21.15.92:58482) 2023-05-16T07:18:20.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f3487fff700 1 -- 172.21.15.92:0/2455845928 learned_addr learned my addr 172.21.15.92:0/2455845928 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:20.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.419+0000 7f3486ffd700 1 --2- 172.21.15.92:0/2455845928 >> v2:172.21.15.158:3300/0 conn(0x7f348813b930 0x7f34880abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34877fe700 1 -- 172.21.15.92:0/2455845928 >> v2:172.21.15.158:3300/0 conn(0x7f348813b930 msgr2=0x7f34880abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34877fe700 1 --2- 172.21.15.92:0/2455845928 >> v2:172.21.15.158:3300/0 conn(0x7f348813b930 0x7f34880abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34877fe700 1 -- 172.21.15.92:0/2455845928 >> v1:172.21.15.92:6789/0 conn(0x7f3488062c80 legacy=0x7f3488063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34867fc700 1 -- 172.21.15.92:0/2455845928 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243691265 0 0) 0x7f34880bdc30 con 0x7f3488062c80 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34877fe700 1 -- 172.21.15.92:0/2455845928 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880c4e40 con 0x7f348813b550 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34877fe700 1 --2- 172.21.15.92:0/2455845928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348813b550 0x7f34880c31f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f3488064700 tx=0x7f3478009060).ready entity=mon.0 client_cookie=c09f814acd47d51b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34867fc700 1 -- 172.21.15.92:0/2455845928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800b580 con 0x7f348813b550 2023-05-16T07:18:20.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34867fc700 1 -- 172.21.15.92:0/2455845928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800b6e0 con 0x7f348813b550 2023-05-16T07:18:20.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f34867fc700 1 -- 172.21.15.92:0/2455845928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800ba60 con 0x7f348813b550 2023-05-16T07:18:20.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 -- 172.21.15.92:0/2455845928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348813b550 msgr2=0x7f34880c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 --2- 172.21.15.92:0/2455845928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348813b550 0x7f34880c31f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f3488064700 tx=0x7f3478009060).stop 2023-05-16T07:18:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 -- 172.21.15.92:0/2455845928 shutdown_connections 2023-05-16T07:18:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 --2- 172.21.15.92:0/2455845928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348813b550 0x7f34880c31f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 --2- 172.21.15.92:0/2455845928 >> v2:172.21.15.158:3300/0 conn(0x7f348813b930 0x7f34880abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 -- 172.21.15.92:0/2455845928 >> 172.21.15.92:0/2455845928 conn(0x7f34880c8aa0 msgr2=0x7f34880c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 -- 172.21.15.92:0/2455845928 shutdown_connections 2023-05-16T07:18:20.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.420+0000 7f348dd89700 1 -- 172.21.15.92:0/2455845928 wait complete. 2023-05-16T07:18:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 Processor -- start 2023-05-16T07:18:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 -- start start 2023-05-16T07:18:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 0x7f34880ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f348813b550 0x7f3488137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f348813b930 0x7f34881512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880ae300 con 0x7f3488062c80 2023-05-16T07:18:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488165b70 con 0x7f348813b550 2023-05-16T07:18:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f348dd89700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3488165e50 con 0x7f348813b930 2023-05-16T07:18:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f34877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 0x7f34880ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f34877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 0x7f34880ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47740/0 (socket says 172.21.15.92:47740) 2023-05-16T07:18:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.422+0000 7f34877fe700 1 -- 172.21.15.92:0/3501430547 learned_addr learned my addr 172.21.15.92:0/3501430547 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f348813b550 0x7f3488137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f3487fff700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f348813b930 0x7f34881512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f34877fe700 1 -- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f348813b930 msgr2=0x7f34881512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f34877fe700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f348813b930 0x7f34881512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f34877fe700 1 -- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f348813b550 msgr2=0x7f3488137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f34877fe700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f348813b550 0x7f3488137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f34877fe700 1 -- 172.21.15.92:0/3501430547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780092a0 con 0x7f3488062c80 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f348813b550 0x7f3488137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f3487fff700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f348813b930 0x7f34881512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f34877fe700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 0x7f34880ade10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f3478009e80 tx=0x7f3478017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800b850 con 0x7f3488062c80 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800b9b0 con 0x7f3488062c80 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3488166130 con 0x7f3488062c80 2023-05-16T07:18:20.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347801b640 con 0x7f3488062c80 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.423+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34881666f0 con 0x7f3488062c80 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.424+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3488166910 con 0x7f3488062c80 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.424+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f347800f070 con 0x7f3488062c80 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.425+0000 7f346f7fe700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f347405ed60 0x7f3474061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.425+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f347807b550 con 0x7f3488062c80 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.425+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f347405ed60 0x7f3474061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.425+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f347405ed60 0x7f3474061220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f347c006fd0 tx=0x7f347c008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.429+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f347804c8a0 con 0x7f3488062c80 2023-05-16T07:18:20.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.587+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f348806c350 con 0x7f347405ed60 2023-05-16T07:18:20.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.598+0000 7f346f7fe700 1 -- 172.21.15.92:0/3501430547 <== mgr.6373 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361751 (secure 0 0 0) 0x7f348806c350 con 0x7f347405ed60 2023-05-16T07:18:20.606 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f347405ed60 msgr2=0x7f3474061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f347405ed60 0x7f3474061220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f347c006fd0 tx=0x7f347c008040).stop 2023-05-16T07:18:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 msgr2=0x7f34880ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 0x7f34880ade10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f3478009e80 tx=0x7f3478017000).stop 2023-05-16T07:18:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 shutdown_connections 2023-05-16T07:18:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f347405ed60 0x7f3474061220 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f348813b930 0x7f34881512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488062c80 0x7f34880ade10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 --2- 172.21.15.92:0/3501430547 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f348813b550 0x7f3488137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 >> 172.21.15.92:0/3501430547 conn(0x7f34880c8aa0 msgr2=0x7f34880c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 shutdown_connections 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.601+0000 7f348dd89700 1 -- 172.21.15.92:0/3501430547 wait complete. 2023-05-16T07:18:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:18:20.617 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:18:18.899332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":155992,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":145856,"kb_avail":754818728,"statfs":{"total":773094113280,"available":772934377472,"internally_reserved":0,"allocated":10313728,"data_stored":5330692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":149356544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4751,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-53,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.597100"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441008+0000","last_change":"2023-05-16T07:18:17.441008+0000","last_active":"2023-05-16T07:18:17.441008+0000","last_peered":"2023-05-16T07:18:17.441008+0000","last_clean":"2023-05-16T07:18:17.441008+0000","last_became_active":"2023-05-16T07:18:17.440798+0000","last_became_peered":"2023-05-16T07:18:17.440798+0000","last_unstale":"2023-05-16T07:18:17.441008+0000","last_undegraded":"2023-05-16T07:18:17.441008+0000","last_fullsized":"2023-05-16T07:18:17.441008+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"106'3","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449285+0000","last_change":"2023-05-16T07:18:17.064002+0000","last_active":"2023-05-16T07:18:17.449285+0000","last_peered":"2023-05-16T07:18:17.449285+0000","last_clean":"2023-05-16T07:18:17.449285+0000","last_became_active":"2023-05-16T07:18:17.063705+0000","last_became_peered":"2023-05-16T07:18:17.063705+0000","last_unstale":"2023-05-16T07:18:17.449285+0000","last_undegraded":"2023-05-16T07:18:17.449285+0000","last_fullsized":"2023-05-16T07:18:17.449285+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.433114+0000","last_change":"2023-05-16T07:18:17.433114+0000","last_active":"2023-05-16T07:18:17.433114+0000","last_peered":"2023-05-16T07:18:17.433114+0000","last_clean":"2023-05-16T07:18:17.433114+0000","last_became_active":"2023-05-16T07:18:17.432833+0000","last_became_peered":"2023-05-16T07:18:17.432833+0000","last_unstale":"2023-05-16T07:18:17.433114+0000","last_undegraded":"2023-05-16T07:18:17.433114+0000","last_fullsized":"2023-05-16T07:18:17.433114+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082128+0000","last_change":"2023-05-16T07:18:17.076423+0000","last_active":"2023-05-16T07:18:17.082128+0000","last_peered":"2023-05-16T07:18:17.082128+0000","last_clean":"2023-05-16T07:18:17.082128+0000","last_became_active":"2023-05-16T07:18:17.075266+0000","last_became_peered":"2023-05-16T07:18:17.075266+0000","last_unstale":"2023-05-16T07:18:17.082128+0000","last_undegraded":"2023-05-16T07:18:17.082128+0000","last_fullsized":"2023-05-16T07:18:17.082128+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445829+0000","last_change":"2023-05-16T07:18:17.445829+0000","last_active":"2023-05-16T07:18:17.445829+0000","last_peered":"2023-05-16T07:18:17.445829+0000","last_clean":"2023-05-16T07:18:17.445829+0000","last_became_active":"2023-05-16T07:18:17.445573+0000","last_became_peered":"2023-05-16T07:18:17.445573+0000","last_unstale":"2023-05-16T07:18:17.445829+0000","last_undegraded":"2023-05-16T07:18:17.445829+0000","last_fullsized":"2023-05-16T07:18:17.445829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.078239+0000","last_change":"2023-05-16T07:18:17.076072+0000","last_active":"2023-05-16T07:18:17.078239+0000","last_peered":"2023-05-16T07:18:17.078239+0000","last_clean":"2023-05-16T07:18:17.078239+0000","last_became_active":"2023-05-16T07:18:17.075802+0000","last_became_peered":"2023-05-16T07:18:17.075802+0000","last_unstale":"2023-05-16T07:18:17.078239+0000","last_undegraded":"2023-05-16T07:18:17.078239+0000","last_fullsized":"2023-05-16T07:18:17.078239+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448115+0000","last_change":"2023-05-16T07:18:17.448115+0000","last_active":"2023-05-16T07:18:17.448115+0000","last_peered":"2023-05-16T07:18:17.448115+0000","last_clean":"2023-05-16T07:18:17.448115+0000","last_became_active":"2023-05-16T07:18:17.447681+0000","last_became_peered":"2023-05-16T07:18:17.447681+0000","last_unstale":"2023-05-16T07:18:17.448115+0000","last_undegraded":"2023-05-16T07:18:17.448115+0000","last_fullsized":"2023-05-16T07:18:17.448115+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3c","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.450659+0000","last_change":"2023-05-16T07:18:17.082046+0000","last_active":"2023-05-16T07:18:17.450659+0000","last_peered":"2023-05-16T07:18:17.450659+0000","last_clean":"2023-05-16T07:18:17.450659+0000","last_became_active":"2023-05-16T07:18:17.081825+0000","last_became_peered":"2023-05-16T07:18:17.081825+0000","last_unstale":"2023-05-16T07:18:17.450659+0000","last_undegraded":"2023-05-16T07:18:17.450659+0000","last_fullsized":"2023-05-16T07:18:17.450659+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449706+0000","last_change":"2023-05-16T07:18:17.449706+0000","last_active":"2023-05-16T07:18:17.449706+0000","last_peered":"2023-05-16T07:18:17.449706+0000","last_clean":"2023-05-16T07:18:17.449706+0000","last_became_active":"2023-05-16T07:18:17.449409+0000","last_became_peered":"2023-05-16T07:18:17.449409+0000","last_unstale":"2023-05-16T07:18:17.449706+0000","last_undegraded":"2023-05-16T07:18:17.449706+0000","last_fullsized":"2023-05-16T07:18:17.449706+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3b","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.431110+0000","last_change":"2023-05-16T07:18:17.079457+0000","last_active":"2023-05-16T07:18:17.431110+0000","last_peered":"2023-05-16T07:18:17.431110+0000","last_clean":"2023-05-16T07:18:17.431110+0000","last_became_active":"2023-05-16T07:18:17.079187+0000","last_became_peered":"2023-05-16T07:18:17.079187+0000","last_unstale":"2023-05-16T07:18:17.431110+0000","last_undegraded":"2023-05-16T07:18:17.431110+0000","last_fullsized":"2023-05-16T07:18:17.431110+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.442170+0000","last_change":"2023-05-16T07:18:17.442170+0000","last_active":"2023-05-16T07:18:17.442170+0000","last_peered":"2023-05-16T07:18:17.442170+0000","last_clean":"2023-05-16T07:18:17.442170+0000","last_became_active":"2023-05-16T07:18:17.441943+0000","last_became_peered":"2023-05-16T07:18:17.441943+0000","last_unstale":"2023-05-16T07:18:17.442170+0000","last_undegraded":"2023-05-16T07:18:17.442170+0000","last_fullsized":"2023-05-16T07:18:17.442170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3a","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.439179+0000","last_change":"2023-05-16T07:18:17.072055+0000","last_active":"2023-05-16T07:18:17.439179+0000","last_peered":"2023-05-16T07:18:17.439179+0000","last_clean":"2023-05-16T07:18:17.439179+0000","last_became_active":"2023-05-16T07:18:17.070976+0000","last_became_peered":"2023-05-16T07:18:17.070976+0000","last_unstale":"2023-05-16T07:18:17.439179+0000","last_undegraded":"2023-05-16T07:18:17.439179+0000","last_fullsized":"2023-05-16T07:18:17.439179+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449864+0000","last_change":"2023-05-16T07:18:17.449864+0000","last_active":"2023-05-16T07:18:17.449864+0000","last_peered":"2023-05-16T07:18:17.449864+0000","last_clean":"2023-05-16T07:18:17.449864+0000","last_became_active":"2023-05-16T07:18:17.449465+0000","last_became_peered":"2023-05-16T07:18:17.449465+0000","last_unstale":"2023-05-16T07:18:17.449864+0000","last_undegraded":"2023-05-16T07:18:17.449864+0000","last_fullsized":"2023-05-16T07:18:17.449864+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083278+0000","last_change":"2023-05-16T07:18:17.075912+0000","last_active":"2023-05-16T07:18:17.083278+0000","last_peered":"2023-05-16T07:18:17.083278+0000","last_clean":"2023-05-16T07:18:17.083278+0000","last_became_active":"2023-05-16T07:18:17.075626+0000","last_became_peered":"2023-05-16T07:18:17.075626+0000","last_unstale":"2023-05-16T07:18:17.083278+0000","last_undegraded":"2023-05-16T07:18:17.083278+0000","last_fullsized":"2023-05-16T07:18:17.083278+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.434268+0000","last_change":"2023-05-16T07:18:17.434268+0000","last_active":"2023-05-16T07:18:17.434268+0000","last_peered":"2023-05-16T07:18:17.434268+0000","last_clean":"2023-05-16T07:18:17.434268+0000","last_became_active":"2023-05-16T07:18:17.433897+0000","last_became_peered":"2023-05-16T07:18:17.433897+0000","last_unstale":"2023-05-16T07:18:17.434268+0000","last_undegraded":"2023-05-16T07:18:17.434268+0000","last_fullsized":"2023-05-16T07:18:17.434268+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082364+0000","last_change":"2023-05-16T07:18:17.081786+0000","last_active":"2023-05-16T07:18:17.082364+0000","last_peered":"2023-05-16T07:18:17.082364+0000","last_clean":"2023-05-16T07:18:17.082364+0000","last_became_active":"2023-05-16T07:18:17.081481+0000","last_became_peered":"2023-05-16T07:18:17.081481+0000","last_unstale":"2023-05-16T07:18:17.082364+0000","last_undegraded":"2023-05-16T07:18:17.082364+0000","last_fullsized":"2023-05-16T07:18:17.082364+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.437966+0000","last_change":"2023-05-16T07:18:17.437966+0000","last_active":"2023-05-16T07:18:17.437966+0000","last_peered":"2023-05-16T07:18:17.437966+0000","last_clean":"2023-05-16T07:18:17.437966+0000","last_became_active":"2023-05-16T07:18:17.437656+0000","last_became_peered":"2023-05-16T07:18:17.437656+0000","last_unstale":"2023-05-16T07:18:17.437966+0000","last_undegraded":"2023-05-16T07:18:17.437966+0000","last_fullsized":"2023-05-16T07:18:17.437966+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073450+0000","last_change":"2023-05-16T07:18:17.070729+0000","last_active":"2023-05-16T07:18:17.073450+0000","last_peered":"2023-05-16T07:18:17.073450+0000","last_clean":"2023-05-16T07:18:17.073450+0000","last_became_active":"2023-05-16T07:18:17.070299+0000","last_became_peered":"2023-05-16T07:18:17.070299+0000","last_unstale":"2023-05-16T07:18:17.073450+0000","last_undegraded":"2023-05-16T07:18:17.073450+0000","last_fullsized":"2023-05-16T07:18:17.073450+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.451546+0000","last_change":"2023-05-16T07:18:17.451546+0000","last_active":"2023-05-16T07:18:17.451546+0000","last_peered":"2023-05-16T07:18:17.451546+0000","last_clean":"2023-05-16T07:18:17.451546+0000","last_became_active":"2023-05-16T07:18:17.451294+0000","last_became_peered":"2023-05-16T07:18:17.451294+0000","last_unstale":"2023-05-16T07:18:17.451546+0000","last_undegraded":"2023-05-16T07:18:17.451546+0000","last_fullsized":"2023-05-16T07:18:17.451546+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076953+0000","last_change":"2023-05-16T07:18:17.071331+0000","last_active":"2023-05-16T07:18:17.076953+0000","last_peered":"2023-05-16T07:18:17.076953+0000","last_clean":"2023-05-16T07:18:17.076953+0000","last_became_active":"2023-05-16T07:18:17.070948+0000","last_became_peered":"2023-05-16T07:18:17.070948+0000","last_unstale":"2023-05-16T07:18:17.076953+0000","last_undegraded":"2023-05-16T07:18:17.076953+0000","last_fullsized":"2023-05-16T07:18:17.076953+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445514+0000","last_change":"2023-05-16T07:18:17.445514+0000","last_active":"2023-05-16T07:18:17.445514+0000","last_peered":"2023-05-16T07:18:17.445514+0000","last_clean":"2023-05-16T07:18:17.445514+0000","last_became_active":"2023-05-16T07:18:17.441495+0000","last_became_peered":"2023-05-16T07:18:17.441495+0000","last_unstale":"2023-05-16T07:18:17.445514+0000","last_undegraded":"2023-05-16T07:18:17.445514+0000","last_fullsized":"2023-05-16T07:18:17.445514+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083045+0000","last_change":"2023-05-16T07:18:17.079104+0000","last_active":"2023-05-16T07:18:17.083045+0000","last_peered":"2023-05-16T07:18:17.083045+0000","last_clean":"2023-05-16T07:18:17.083045+0000","last_became_active":"2023-05-16T07:18:17.078833+0000","last_became_peered":"2023-05-16T07:18:17.078833+0000","last_unstale":"2023-05-16T07:18:17.083045+0000","last_undegraded":"2023-05-16T07:18:17.083045+0000","last_fullsized":"2023-05-16T07:18:17.083045+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444167+0000","last_change":"2023-05-16T07:18:17.444167+0000","last_active":"2023-05-16T07:18:17.444167+0000","last_peered":"2023-05-16T07:18:17.444167+0000","last_clean":"2023-05-16T07:18:17.444167+0000","last_became_active":"2023-05-16T07:18:17.443907+0000","last_became_peered":"2023-05-16T07:18:17.443907+0000","last_unstale":"2023-05-16T07:18:17.444167+0000","last_undegraded":"2023-05-16T07:18:17.444167+0000","last_fullsized":"2023-05-16T07:18:17.444167+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.34","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.450671+0000","last_change":"2023-05-16T07:18:17.074971+0000","last_active":"2023-05-16T07:18:17.450671+0000","last_peered":"2023-05-16T07:18:17.450671+0000","last_clean":"2023-05-16T07:18:17.450671+0000","last_became_active":"2023-05-16T07:18:17.074091+0000","last_became_peered":"2023-05-16T07:18:17.074091+0000","last_unstale":"2023-05-16T07:18:17.450671+0000","last_undegraded":"2023-05-16T07:18:17.450671+0000","last_fullsized":"2023-05-16T07:18:17.450671+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440344+0000","last_change":"2023-05-16T07:18:17.440344+0000","last_active":"2023-05-16T07:18:17.440344+0000","last_peered":"2023-05-16T07:18:17.440344+0000","last_clean":"2023-05-16T07:18:17.440344+0000","last_became_active":"2023-05-16T07:18:17.439931+0000","last_became_peered":"2023-05-16T07:18:17.439931+0000","last_unstale":"2023-05-16T07:18:17.440344+0000","last_undegraded":"2023-05-16T07:18:17.440344+0000","last_fullsized":"2023-05-16T07:18:17.440344+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.431266+0000","last_change":"2023-05-16T07:18:17.075712+0000","last_active":"2023-05-16T07:18:17.431266+0000","last_peered":"2023-05-16T07:18:17.431266+0000","last_clean":"2023-05-16T07:18:17.431266+0000","last_became_active":"2023-05-16T07:18:17.074991+0000","last_became_peered":"2023-05-16T07:18:17.074991+0000","last_unstale":"2023-05-16T07:18:17.431266+0000","last_undegraded":"2023-05-16T07:18:17.431266+0000","last_fullsized":"2023-05-16T07:18:17.431266+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441254+0000","last_change":"2023-05-16T07:18:17.441254+0000","last_active":"2023-05-16T07:18:17.441254+0000","last_peered":"2023-05-16T07:18:17.441254+0000","last_clean":"2023-05-16T07:18:17.441254+0000","last_became_active":"2023-05-16T07:18:17.440951+0000","last_became_peered":"2023-05-16T07:18:17.440951+0000","last_unstale":"2023-05-16T07:18:17.441254+0000","last_undegraded":"2023-05-16T07:18:17.441254+0000","last_fullsized":"2023-05-16T07:18:17.441254+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.085887+0000","last_change":"2023-05-16T07:18:17.082754+0000","last_active":"2023-05-16T07:18:17.085887+0000","last_peered":"2023-05-16T07:18:17.085887+0000","last_clean":"2023-05-16T07:18:17.085887+0000","last_became_active":"2023-05-16T07:18:17.082463+0000","last_became_peered":"2023-05-16T07:18:17.082463+0000","last_unstale":"2023-05-16T07:18:17.085887+0000","last_undegraded":"2023-05-16T07:18:17.085887+0000","last_fullsized":"2023-05-16T07:18:17.085887+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446101+0000","last_change":"2023-05-16T07:18:17.446101+0000","last_active":"2023-05-16T07:18:17.446101+0000","last_peered":"2023-05-16T07:18:17.446101+0000","last_clean":"2023-05-16T07:18:17.446101+0000","last_became_active":"2023-05-16T07:18:17.445777+0000","last_became_peered":"2023-05-16T07:18:17.445777+0000","last_unstale":"2023-05-16T07:18:17.446101+0000","last_undegraded":"2023-05-16T07:18:17.446101+0000","last_fullsized":"2023-05-16T07:18:17.446101+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077966+0000","last_change":"2023-05-16T07:18:17.074634+0000","last_active":"2023-05-16T07:18:17.077966+0000","last_peered":"2023-05-16T07:18:17.077966+0000","last_clean":"2023-05-16T07:18:17.077966+0000","last_became_active":"2023-05-16T07:18:17.074417+0000","last_became_peered":"2023-05-16T07:18:17.074417+0000","last_unstale":"2023-05-16T07:18:17.077966+0000","last_undegraded":"2023-05-16T07:18:17.077966+0000","last_fullsized":"2023-05-16T07:18:17.077966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449312+0000","last_change":"2023-05-16T07:18:17.449312+0000","last_active":"2023-05-16T07:18:17.449312+0000","last_peered":"2023-05-16T07:18:17.449312+0000","last_clean":"2023-05-16T07:18:17.449312+0000","last_became_active":"2023-05-16T07:18:17.449021+0000","last_became_peered":"2023-05-16T07:18:17.449021+0000","last_unstale":"2023-05-16T07:18:17.449312+0000","last_undegraded":"2023-05-16T07:18:17.449312+0000","last_fullsized":"2023-05-16T07:18:17.449312+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449262+0000","last_change":"2023-05-16T07:18:17.065614+0000","last_active":"2023-05-16T07:18:17.449262+0000","last_peered":"2023-05-16T07:18:17.449262+0000","last_clean":"2023-05-16T07:18:17.449262+0000","last_became_active":"2023-05-16T07:18:17.065308+0000","last_became_peered":"2023-05-16T07:18:17.065308+0000","last_unstale":"2023-05-16T07:18:17.449262+0000","last_undegraded":"2023-05-16T07:18:17.449262+0000","last_fullsized":"2023-05-16T07:18:17.449262+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440043+0000","last_change":"2023-05-16T07:18:17.440043+0000","last_active":"2023-05-16T07:18:17.440043+0000","last_peered":"2023-05-16T07:18:17.440043+0000","last_clean":"2023-05-16T07:18:17.440043+0000","last_became_active":"2023-05-16T07:18:17.439759+0000","last_became_peered":"2023-05-16T07:18:17.439759+0000","last_unstale":"2023-05-16T07:18:17.440043+0000","last_undegraded":"2023-05-16T07:18:17.440043+0000","last_fullsized":"2023-05-16T07:18:17.440043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.072974+0000","last_change":"2023-05-16T07:18:17.069028+0000","last_active":"2023-05-16T07:18:17.072974+0000","last_peered":"2023-05-16T07:18:17.072974+0000","last_clean":"2023-05-16T07:18:17.072974+0000","last_became_active":"2023-05-16T07:18:17.068704+0000","last_became_peered":"2023-05-16T07:18:17.068704+0000","last_unstale":"2023-05-16T07:18:17.072974+0000","last_undegraded":"2023-05-16T07:18:17.072974+0000","last_fullsized":"2023-05-16T07:18:17.072974+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.443753+0000","last_change":"2023-05-16T07:18:17.443753+0000","last_active":"2023-05-16T07:18:17.443753+0000","last_peered":"2023-05-16T07:18:17.443753+0000","last_clean":"2023-05-16T07:18:17.443753+0000","last_became_active":"2023-05-16T07:18:17.443368+0000","last_became_peered":"2023-05-16T07:18:17.443368+0000","last_unstale":"2023-05-16T07:18:17.443753+0000","last_undegraded":"2023-05-16T07:18:17.443753+0000","last_fullsized":"2023-05-16T07:18:17.443753+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.074803+0000","last_change":"2023-05-16T07:18:17.070554+0000","last_active":"2023-05-16T07:18:17.074803+0000","last_peered":"2023-05-16T07:18:17.074803+0000","last_clean":"2023-05-16T07:18:17.074803+0000","last_became_active":"2023-05-16T07:18:17.070216+0000","last_became_peered":"2023-05-16T07:18:17.070216+0000","last_unstale":"2023-05-16T07:18:17.074803+0000","last_undegraded":"2023-05-16T07:18:17.074803+0000","last_fullsized":"2023-05-16T07:18:17.074803+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.439496+0000","last_change":"2023-05-16T07:18:17.439496+0000","last_active":"2023-05-16T07:18:17.439496+0000","last_peered":"2023-05-16T07:18:17.439496+0000","last_clean":"2023-05-16T07:18:17.439496+0000","last_became_active":"2023-05-16T07:18:17.439319+0000","last_became_peered":"2023-05-16T07:18:17.439319+0000","last_unstale":"2023-05-16T07:18:17.439496+0000","last_undegraded":"2023-05-16T07:18:17.439496+0000","last_fullsized":"2023-05-16T07:18:17.439496+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.436779+0000","last_change":"2023-05-16T07:18:17.070160+0000","last_active":"2023-05-16T07:18:17.436779+0000","last_peered":"2023-05-16T07:18:17.436779+0000","last_clean":"2023-05-16T07:18:17.436779+0000","last_became_active":"2023-05-16T07:18:17.069862+0000","last_became_peered":"2023-05-16T07:18:17.069862+0000","last_unstale":"2023-05-16T07:18:17.436779+0000","last_undegraded":"2023-05-16T07:18:17.436779+0000","last_fullsized":"2023-05-16T07:18:17.436779+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447639+0000","last_change":"2023-05-16T07:18:17.447639+0000","last_active":"2023-05-16T07:18:17.447639+0000","last_peered":"2023-05-16T07:18:17.447639+0000","last_clean":"2023-05-16T07:18:17.447639+0000","last_became_active":"2023-05-16T07:18:17.447383+0000","last_became_peered":"2023-05-16T07:18:17.447383+0000","last_unstale":"2023-05-16T07:18:17.447639+0000","last_undegraded":"2023-05-16T07:18:17.447639+0000","last_fullsized":"2023-05-16T07:18:17.447639+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.070466+0000","last_change":"2023-05-16T07:18:17.068150+0000","last_active":"2023-05-16T07:18:17.070466+0000","last_peered":"2023-05-16T07:18:17.070466+0000","last_clean":"2023-05-16T07:18:17.070466+0000","last_became_active":"2023-05-16T07:18:17.067885+0000","last_became_peered":"2023-05-16T07:18:17.067885+0000","last_unstale":"2023-05-16T07:18:17.070466+0000","last_undegraded":"2023-05-16T07:18:17.070466+0000","last_fullsized":"2023-05-16T07:18:17.070466+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.433159+0000","last_change":"2023-05-16T07:18:17.433159+0000","last_active":"2023-05-16T07:18:17.433159+0000","last_peered":"2023-05-16T07:18:17.433159+0000","last_clean":"2023-05-16T07:18:17.433159+0000","last_became_active":"2023-05-16T07:18:17.432906+0000","last_became_peered":"2023-05-16T07:18:17.432906+0000","last_unstale":"2023-05-16T07:18:17.433159+0000","last_undegraded":"2023-05-16T07:18:17.433159+0000","last_fullsized":"2023-05-16T07:18:17.433159+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2b","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.433443+0000","last_change":"2023-05-16T07:18:17.073894+0000","last_active":"2023-05-16T07:18:17.433443+0000","last_peered":"2023-05-16T07:18:17.433443+0000","last_clean":"2023-05-16T07:18:17.433443+0000","last_became_active":"2023-05-16T07:18:17.073587+0000","last_became_peered":"2023-05-16T07:18:17.073587+0000","last_unstale":"2023-05-16T07:18:17.433443+0000","last_undegraded":"2023-05-16T07:18:17.433443+0000","last_fullsized":"2023-05-16T07:18:17.433443+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440803+0000","last_change":"2023-05-16T07:18:17.440803+0000","last_active":"2023-05-16T07:18:17.440803+0000","last_peered":"2023-05-16T07:18:17.440803+0000","last_clean":"2023-05-16T07:18:17.440803+0000","last_became_active":"2023-05-16T07:18:17.440339+0000","last_became_peered":"2023-05-16T07:18:17.440339+0000","last_unstale":"2023-05-16T07:18:17.440803+0000","last_undegraded":"2023-05-16T07:18:17.440803+0000","last_fullsized":"2023-05-16T07:18:17.440803+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083302+0000","last_change":"2023-05-16T07:18:17.079203+0000","last_active":"2023-05-16T07:18:17.083302+0000","last_peered":"2023-05-16T07:18:17.083302+0000","last_clean":"2023-05-16T07:18:17.083302+0000","last_became_active":"2023-05-16T07:18:17.078786+0000","last_became_peered":"2023-05-16T07:18:17.078786+0000","last_unstale":"2023-05-16T07:18:17.083302+0000","last_undegraded":"2023-05-16T07:18:17.083302+0000","last_fullsized":"2023-05-16T07:18:17.083302+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445963+0000","last_change":"2023-05-16T07:18:17.445963+0000","last_active":"2023-05-16T07:18:17.445963+0000","last_peered":"2023-05-16T07:18:17.445963+0000","last_clean":"2023-05-16T07:18:17.445963+0000","last_became_active":"2023-05-16T07:18:17.445678+0000","last_became_peered":"2023-05-16T07:18:17.445678+0000","last_unstale":"2023-05-16T07:18:17.445963+0000","last_undegraded":"2023-05-16T07:18:17.445963+0000","last_fullsized":"2023-05-16T07:18:17.445963+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.074938+0000","last_change":"2023-05-16T07:18:17.069343+0000","last_active":"2023-05-16T07:18:17.074938+0000","last_peered":"2023-05-16T07:18:17.074938+0000","last_clean":"2023-05-16T07:18:17.074938+0000","last_became_active":"2023-05-16T07:18:17.068386+0000","last_became_peered":"2023-05-16T07:18:17.068386+0000","last_unstale":"2023-05-16T07:18:17.074938+0000","last_undegraded":"2023-05-16T07:18:17.074938+0000","last_fullsized":"2023-05-16T07:18:17.074938+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.438135+0000","last_change":"2023-05-16T07:18:17.438135+0000","last_active":"2023-05-16T07:18:17.438135+0000","last_peered":"2023-05-16T07:18:17.438135+0000","last_clean":"2023-05-16T07:18:17.438135+0000","last_became_active":"2023-05-16T07:18:17.437725+0000","last_became_peered":"2023-05-16T07:18:17.437725+0000","last_unstale":"2023-05-16T07:18:17.438135+0000","last_undegraded":"2023-05-16T07:18:17.438135+0000","last_fullsized":"2023-05-16T07:18:17.438135+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.072948+0000","last_change":"2023-05-16T07:18:17.068861+0000","last_active":"2023-05-16T07:18:17.072948+0000","last_peered":"2023-05-16T07:18:17.072948+0000","last_clean":"2023-05-16T07:18:17.072948+0000","last_became_active":"2023-05-16T07:18:17.068442+0000","last_became_peered":"2023-05-16T07:18:17.068442+0000","last_unstale":"2023-05-16T07:18:17.072948+0000","last_undegraded":"2023-05-16T07:18:17.072948+0000","last_fullsized":"2023-05-16T07:18:17.072948+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445332+0000","last_change":"2023-05-16T07:18:17.445332+0000","last_active":"2023-05-16T07:18:17.445332+0000","last_peered":"2023-05-16T07:18:17.445332+0000","last_clean":"2023-05-16T07:18:17.445332+0000","last_became_active":"2023-05-16T07:18:17.445106+0000","last_became_peered":"2023-05-16T07:18:17.445106+0000","last_unstale":"2023-05-16T07:18:17.445332+0000","last_undegraded":"2023-05-16T07:18:17.445332+0000","last_fullsized":"2023-05-16T07:18:17.445332+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083039+0000","last_change":"2023-05-16T07:18:17.079895+0000","last_active":"2023-05-16T07:18:17.083039+0000","last_peered":"2023-05-16T07:18:17.083039+0000","last_clean":"2023-05-16T07:18:17.083039+0000","last_became_active":"2023-05-16T07:18:17.079651+0000","last_became_peered":"2023-05-16T07:18:17.079651+0000","last_unstale":"2023-05-16T07:18:17.083039+0000","last_undegraded":"2023-05-16T07:18:17.083039+0000","last_fullsized":"2023-05-16T07:18:17.083039+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.436610+0000","last_change":"2023-05-16T07:18:17.436610+0000","last_active":"2023-05-16T07:18:17.436610+0000","last_peered":"2023-05-16T07:18:17.436610+0000","last_clean":"2023-05-16T07:18:17.436610+0000","last_became_active":"2023-05-16T07:18:17.436332+0000","last_became_peered":"2023-05-16T07:18:17.436332+0000","last_unstale":"2023-05-16T07:18:17.436610+0000","last_undegraded":"2023-05-16T07:18:17.436610+0000","last_fullsized":"2023-05-16T07:18:17.436610+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080490+0000","last_change":"2023-05-16T07:18:17.079450+0000","last_active":"2023-05-16T07:18:17.080490+0000","last_peered":"2023-05-16T07:18:17.080490+0000","last_clean":"2023-05-16T07:18:17.080490+0000","last_became_active":"2023-05-16T07:18:17.079149+0000","last_became_peered":"2023-05-16T07:18:17.079149+0000","last_unstale":"2023-05-16T07:18:17.080490+0000","last_undegraded":"2023-05-16T07:18:17.080490+0000","last_fullsized":"2023-05-16T07:18:17.080490+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.443664+0000","last_change":"2023-05-16T07:18:17.443664+0000","last_active":"2023-05-16T07:18:17.443664+0000","last_peered":"2023-05-16T07:18:17.443664+0000","last_clean":"2023-05-16T07:18:17.443664+0000","last_became_active":"2023-05-16T07:18:17.443389+0000","last_became_peered":"2023-05-16T07:18:17.443389+0000","last_unstale":"2023-05-16T07:18:17.443664+0000","last_undegraded":"2023-05-16T07:18:17.443664+0000","last_fullsized":"2023-05-16T07:18:17.443664+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073645+0000","last_change":"2023-05-16T07:18:17.071912+0000","last_active":"2023-05-16T07:18:17.073645+0000","last_peered":"2023-05-16T07:18:17.073645+0000","last_clean":"2023-05-16T07:18:17.073645+0000","last_became_active":"2023-05-16T07:18:17.071611+0000","last_became_peered":"2023-05-16T07:18:17.071611+0000","last_unstale":"2023-05-16T07:18:17.073645+0000","last_undegraded":"2023-05-16T07:18:17.073645+0000","last_fullsized":"2023-05-16T07:18:17.073645+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446935+0000","last_change":"2023-05-16T07:18:17.446935+0000","last_active":"2023-05-16T07:18:17.446935+0000","last_peered":"2023-05-16T07:18:17.446935+0000","last_clean":"2023-05-16T07:18:17.446935+0000","last_became_active":"2023-05-16T07:18:17.446070+0000","last_became_peered":"2023-05-16T07:18:17.446070+0000","last_unstale":"2023-05-16T07:18:17.446935+0000","last_undegraded":"2023-05-16T07:18:17.446935+0000","last_fullsized":"2023-05-16T07:18:17.446935+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077133+0000","last_change":"2023-05-16T07:18:17.073490+0000","last_active":"2023-05-16T07:18:17.077133+0000","last_peered":"2023-05-16T07:18:17.077133+0000","last_clean":"2023-05-16T07:18:17.077133+0000","last_became_active":"2023-05-16T07:18:17.072900+0000","last_became_peered":"2023-05-16T07:18:17.072900+0000","last_unstale":"2023-05-16T07:18:17.077133+0000","last_undegraded":"2023-05-16T07:18:17.077133+0000","last_fullsized":"2023-05-16T07:18:17.077133+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440186+0000","last_change":"2023-05-16T07:18:17.440186+0000","last_active":"2023-05-16T07:18:17.440186+0000","last_peered":"2023-05-16T07:18:17.440186+0000","last_clean":"2023-05-16T07:18:17.440186+0000","last_became_active":"2023-05-16T07:18:17.439583+0000","last_became_peered":"2023-05-16T07:18:17.439583+0000","last_unstale":"2023-05-16T07:18:17.440186+0000","last_undegraded":"2023-05-16T07:18:17.440186+0000","last_fullsized":"2023-05-16T07:18:17.440186+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083444+0000","last_change":"2023-05-16T07:18:17.080943+0000","last_active":"2023-05-16T07:18:17.083444+0000","last_peered":"2023-05-16T07:18:17.083444+0000","last_clean":"2023-05-16T07:18:17.083444+0000","last_became_active":"2023-05-16T07:18:17.080691+0000","last_became_peered":"2023-05-16T07:18:17.080691+0000","last_unstale":"2023-05-16T07:18:17.083444+0000","last_undegraded":"2023-05-16T07:18:17.083444+0000","last_fullsized":"2023-05-16T07:18:17.083444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445398+0000","last_change":"2023-05-16T07:18:17.445398+0000","last_active":"2023-05-16T07:18:17.445398+0000","last_peered":"2023-05-16T07:18:17.445398+0000","last_clean":"2023-05-16T07:18:17.445398+0000","last_became_active":"2023-05-16T07:18:17.445117+0000","last_became_peered":"2023-05-16T07:18:17.445117+0000","last_unstale":"2023-05-16T07:18:17.445398+0000","last_undegraded":"2023-05-16T07:18:17.445398+0000","last_fullsized":"2023-05-16T07:18:17.445398+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083319+0000","last_change":"2023-05-16T07:18:17.079286+0000","last_active":"2023-05-16T07:18:17.083319+0000","last_peered":"2023-05-16T07:18:17.083319+0000","last_clean":"2023-05-16T07:18:17.083319+0000","last_became_active":"2023-05-16T07:18:17.079026+0000","last_became_peered":"2023-05-16T07:18:17.079026+0000","last_unstale":"2023-05-16T07:18:17.083319+0000","last_undegraded":"2023-05-16T07:18:17.083319+0000","last_fullsized":"2023-05-16T07:18:17.083319+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447130+0000","last_change":"2023-05-16T07:18:17.447130+0000","last_active":"2023-05-16T07:18:17.447130+0000","last_peered":"2023-05-16T07:18:17.447130+0000","last_clean":"2023-05-16T07:18:17.447130+0000","last_became_active":"2023-05-16T07:18:17.446863+0000","last_became_peered":"2023-05-16T07:18:17.446863+0000","last_unstale":"2023-05-16T07:18:17.447130+0000","last_undegraded":"2023-05-16T07:18:17.447130+0000","last_fullsized":"2023-05-16T07:18:17.447130+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073021+0000","last_change":"2023-05-16T07:18:17.069695+0000","last_active":"2023-05-16T07:18:17.073021+0000","last_peered":"2023-05-16T07:18:17.073021+0000","last_clean":"2023-05-16T07:18:17.073021+0000","last_became_active":"2023-05-16T07:18:17.069412+0000","last_became_peered":"2023-05-16T07:18:17.069412+0000","last_unstale":"2023-05-16T07:18:17.073021+0000","last_undegraded":"2023-05-16T07:18:17.073021+0000","last_fullsized":"2023-05-16T07:18:17.073021+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.437779+0000","last_change":"2023-05-16T07:18:17.437779+0000","last_active":"2023-05-16T07:18:17.437779+0000","last_peered":"2023-05-16T07:18:17.437779+0000","last_clean":"2023-05-16T07:18:17.437779+0000","last_became_active":"2023-05-16T07:18:17.437483+0000","last_became_peered":"2023-05-16T07:18:17.437483+0000","last_unstale":"2023-05-16T07:18:17.437779+0000","last_undegraded":"2023-05-16T07:18:17.437779+0000","last_fullsized":"2023-05-16T07:18:17.437779+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080411+0000","last_change":"2023-05-16T07:18:17.079735+0000","last_active":"2023-05-16T07:18:17.080411+0000","last_peered":"2023-05-16T07:18:17.080411+0000","last_clean":"2023-05-16T07:18:17.080411+0000","last_became_active":"2023-05-16T07:18:17.079437+0000","last_became_peered":"2023-05-16T07:18:17.079437+0000","last_unstale":"2023-05-16T07:18:17.080411+0000","last_undegraded":"2023-05-16T07:18:17.080411+0000","last_fullsized":"2023-05-16T07:18:17.080411+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448903+0000","last_change":"2023-05-16T07:18:17.448903+0000","last_active":"2023-05-16T07:18:17.448903+0000","last_peered":"2023-05-16T07:18:17.448903+0000","last_clean":"2023-05-16T07:18:17.448903+0000","last_became_active":"2023-05-16T07:18:17.448649+0000","last_became_peered":"2023-05-16T07:18:17.448649+0000","last_unstale":"2023-05-16T07:18:17.448903+0000","last_undegraded":"2023-05-16T07:18:17.448903+0000","last_fullsized":"2023-05-16T07:18:17.448903+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.428980+0000","last_change":"2023-05-16T07:18:17.069782+0000","last_active":"2023-05-16T07:18:17.428980+0000","last_peered":"2023-05-16T07:18:17.428980+0000","last_clean":"2023-05-16T07:18:17.428980+0000","last_became_active":"2023-05-16T07:18:17.069495+0000","last_became_peered":"2023-05-16T07:18:17.069495+0000","last_unstale":"2023-05-16T07:18:17.428980+0000","last_undegraded":"2023-05-16T07:18:17.428980+0000","last_fullsized":"2023-05-16T07:18:17.428980+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446676+0000","last_change":"2023-05-16T07:18:17.446676+0000","last_active":"2023-05-16T07:18:17.446676+0000","last_peered":"2023-05-16T07:18:17.446676+0000","last_clean":"2023-05-16T07:18:17.446676+0000","last_became_active":"2023-05-16T07:18:17.446399+0000","last_became_peered":"2023-05-16T07:18:17.446399+0000","last_unstale":"2023-05-16T07:18:17.446676+0000","last_undegraded":"2023-05-16T07:18:17.446676+0000","last_fullsized":"2023-05-16T07:18:17.446676+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.429012+0000","last_change":"2023-05-16T07:18:17.075494+0000","last_active":"2023-05-16T07:18:17.429012+0000","last_peered":"2023-05-16T07:18:17.429012+0000","last_clean":"2023-05-16T07:18:17.429012+0000","last_became_active":"2023-05-16T07:18:17.075186+0000","last_became_peered":"2023-05-16T07:18:17.075186+0000","last_unstale":"2023-05-16T07:18:17.429012+0000","last_undegraded":"2023-05-16T07:18:17.429012+0000","last_fullsized":"2023-05-16T07:18:17.429012+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441741+0000","last_change":"2023-05-16T07:18:17.441741+0000","last_active":"2023-05-16T07:18:17.441741+0000","last_peered":"2023-05-16T07:18:17.441741+0000","last_clean":"2023-05-16T07:18:17.441741+0000","last_became_active":"2023-05-16T07:18:17.441499+0000","last_became_peered":"2023-05-16T07:18:17.441499+0000","last_unstale":"2023-05-16T07:18:17.441741+0000","last_undegraded":"2023-05-16T07:18:17.441741+0000","last_fullsized":"2023-05-16T07:18:17.441741+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.450222+0000","last_change":"2023-05-16T07:18:17.074477+0000","last_active":"2023-05-16T07:18:17.450222+0000","last_peered":"2023-05-16T07:18:17.450222+0000","last_clean":"2023-05-16T07:18:17.450222+0000","last_became_active":"2023-05-16T07:18:17.074197+0000","last_became_peered":"2023-05-16T07:18:17.074197+0000","last_unstale":"2023-05-16T07:18:17.450222+0000","last_undegraded":"2023-05-16T07:18:17.450222+0000","last_fullsized":"2023-05-16T07:18:17.450222+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447125+0000","last_change":"2023-05-16T07:18:17.447125+0000","last_active":"2023-05-16T07:18:17.447125+0000","last_peered":"2023-05-16T07:18:17.447125+0000","last_clean":"2023-05-16T07:18:17.447125+0000","last_became_active":"2023-05-16T07:18:17.446348+0000","last_became_peered":"2023-05-16T07:18:17.446348+0000","last_unstale":"2023-05-16T07:18:17.447125+0000","last_undegraded":"2023-05-16T07:18:17.447125+0000","last_fullsized":"2023-05-16T07:18:17.447125+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.079927+0000","last_change":"2023-05-16T07:18:17.077674+0000","last_active":"2023-05-16T07:18:17.079927+0000","last_peered":"2023-05-16T07:18:17.079927+0000","last_clean":"2023-05-16T07:18:17.079927+0000","last_became_active":"2023-05-16T07:18:17.077366+0000","last_became_peered":"2023-05-16T07:18:17.077366+0000","last_unstale":"2023-05-16T07:18:17.079927+0000","last_undegraded":"2023-05-16T07:18:17.079927+0000","last_fullsized":"2023-05-16T07:18:17.079927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440306+0000","last_change":"2023-05-16T07:18:17.440306+0000","last_active":"2023-05-16T07:18:17.440306+0000","last_peered":"2023-05-16T07:18:17.440306+0000","last_clean":"2023-05-16T07:18:17.440306+0000","last_became_active":"2023-05-16T07:18:17.440054+0000","last_became_peered":"2023-05-16T07:18:17.440054+0000","last_unstale":"2023-05-16T07:18:17.440306+0000","last_undegraded":"2023-05-16T07:18:17.440306+0000","last_fullsized":"2023-05-16T07:18:17.440306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073649+0000","last_change":"2023-05-16T07:18:17.071694+0000","last_active":"2023-05-16T07:18:17.073649+0000","last_peered":"2023-05-16T07:18:17.073649+0000","last_clean":"2023-05-16T07:18:17.073649+0000","last_became_active":"2023-05-16T07:18:17.071461+0000","last_became_peered":"2023-05-16T07:18:17.071461+0000","last_unstale":"2023-05-16T07:18:17.073649+0000","last_undegraded":"2023-05-16T07:18:17.073649+0000","last_fullsized":"2023-05-16T07:18:17.073649+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440621+0000","last_change":"2023-05-16T07:18:17.440621+0000","last_active":"2023-05-16T07:18:17.440621+0000","last_peered":"2023-05-16T07:18:17.440621+0000","last_clean":"2023-05-16T07:18:17.440621+0000","last_became_active":"2023-05-16T07:18:17.439959+0000","last_became_peered":"2023-05-16T07:18:17.439959+0000","last_unstale":"2023-05-16T07:18:17.440621+0000","last_undegraded":"2023-05-16T07:18:17.440621+0000","last_fullsized":"2023-05-16T07:18:17.440621+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083282+0000","last_change":"2023-05-16T07:18:17.079082+0000","last_active":"2023-05-16T07:18:17.083282+0000","last_peered":"2023-05-16T07:18:17.083282+0000","last_clean":"2023-05-16T07:18:17.083282+0000","last_became_active":"2023-05-16T07:18:17.078439+0000","last_became_peered":"2023-05-16T07:18:17.078439+0000","last_unstale":"2023-05-16T07:18:17.083282+0000","last_undegraded":"2023-05-16T07:18:17.083282+0000","last_fullsized":"2023-05-16T07:18:17.083282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449702+0000","last_change":"2023-05-16T07:18:17.449702+0000","last_active":"2023-05-16T07:18:17.449702+0000","last_peered":"2023-05-16T07:18:17.449702+0000","last_clean":"2023-05-16T07:18:17.449702+0000","last_became_active":"2023-05-16T07:18:17.449141+0000","last_became_peered":"2023-05-16T07:18:17.449141+0000","last_unstale":"2023-05-16T07:18:17.449702+0000","last_undegraded":"2023-05-16T07:18:17.449702+0000","last_fullsized":"2023-05-16T07:18:17.449702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077985+0000","last_change":"2023-05-16T07:18:17.075129+0000","last_active":"2023-05-16T07:18:17.077985+0000","last_peered":"2023-05-16T07:18:17.077985+0000","last_clean":"2023-05-16T07:18:17.077985+0000","last_became_active":"2023-05-16T07:18:17.074646+0000","last_became_peered":"2023-05-16T07:18:17.074646+0000","last_unstale":"2023-05-16T07:18:17.077985+0000","last_undegraded":"2023-05-16T07:18:17.077985+0000","last_fullsized":"2023-05-16T07:18:17.077985+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440935+0000","last_change":"2023-05-16T07:18:17.440935+0000","last_active":"2023-05-16T07:18:17.440935+0000","last_peered":"2023-05-16T07:18:17.440935+0000","last_clean":"2023-05-16T07:18:17.440935+0000","last_became_active":"2023-05-16T07:18:17.440668+0000","last_became_peered":"2023-05-16T07:18:17.440668+0000","last_unstale":"2023-05-16T07:18:17.440935+0000","last_undegraded":"2023-05-16T07:18:17.440935+0000","last_fullsized":"2023-05-16T07:18:17.440935+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080445+0000","last_change":"2023-05-16T07:18:17.078439+0000","last_active":"2023-05-16T07:18:17.080445+0000","last_peered":"2023-05-16T07:18:17.080445+0000","last_clean":"2023-05-16T07:18:17.080445+0000","last_became_active":"2023-05-16T07:18:17.078154+0000","last_became_peered":"2023-05-16T07:18:17.078154+0000","last_unstale":"2023-05-16T07:18:17.080445+0000","last_undegraded":"2023-05-16T07:18:17.080445+0000","last_fullsized":"2023-05-16T07:18:17.080445+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441562+0000","last_change":"2023-05-16T07:18:17.441562+0000","last_active":"2023-05-16T07:18:17.441562+0000","last_peered":"2023-05-16T07:18:17.441562+0000","last_clean":"2023-05-16T07:18:17.441562+0000","last_became_active":"2023-05-16T07:18:17.441280+0000","last_became_peered":"2023-05-16T07:18:17.441280+0000","last_unstale":"2023-05-16T07:18:17.441562+0000","last_undegraded":"2023-05-16T07:18:17.441562+0000","last_fullsized":"2023-05-16T07:18:17.441562+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073469+0000","last_change":"2023-05-16T07:18:17.071277+0000","last_active":"2023-05-16T07:18:17.073469+0000","last_peered":"2023-05-16T07:18:17.073469+0000","last_clean":"2023-05-16T07:18:17.073469+0000","last_became_active":"2023-05-16T07:18:17.070993+0000","last_became_peered":"2023-05-16T07:18:17.070993+0000","last_unstale":"2023-05-16T07:18:17.073469+0000","last_undegraded":"2023-05-16T07:18:17.073469+0000","last_fullsized":"2023-05-16T07:18:17.073469+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.437669+0000","last_change":"2023-05-16T07:18:17.437669+0000","last_active":"2023-05-16T07:18:17.437669+0000","last_peered":"2023-05-16T07:18:17.437669+0000","last_clean":"2023-05-16T07:18:17.437669+0000","last_became_active":"2023-05-16T07:18:17.437387+0000","last_became_peered":"2023-05-16T07:18:17.437387+0000","last_unstale":"2023-05-16T07:18:17.437669+0000","last_undegraded":"2023-05-16T07:18:17.437669+0000","last_fullsized":"2023-05-16T07:18:17.437669+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080390+0000","last_change":"2023-05-16T07:18:17.078210+0000","last_active":"2023-05-16T07:18:17.080390+0000","last_peered":"2023-05-16T07:18:17.080390+0000","last_clean":"2023-05-16T07:18:17.080390+0000","last_became_active":"2023-05-16T07:18:17.077324+0000","last_became_peered":"2023-05-16T07:18:17.077324+0000","last_unstale":"2023-05-16T07:18:17.080390+0000","last_undegraded":"2023-05-16T07:18:17.080390+0000","last_fullsized":"2023-05-16T07:18:17.080390+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447448+0000","last_change":"2023-05-16T07:18:17.447448+0000","last_active":"2023-05-16T07:18:17.447448+0000","last_peered":"2023-05-16T07:18:17.447448+0000","last_clean":"2023-05-16T07:18:17.447448+0000","last_became_active":"2023-05-16T07:18:17.447120+0000","last_became_peered":"2023-05-16T07:18:17.447120+0000","last_unstale":"2023-05-16T07:18:17.447448+0000","last_undegraded":"2023-05-16T07:18:17.447448+0000","last_fullsized":"2023-05-16T07:18:17.447448+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"106'3","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.451251+0000","last_change":"2023-05-16T07:18:17.078005+0000","last_active":"2023-05-16T07:18:17.451251+0000","last_peered":"2023-05-16T07:18:17.451251+0000","last_clean":"2023-05-16T07:18:17.451251+0000","last_became_active":"2023-05-16T07:18:17.077298+0000","last_became_peered":"2023-05-16T07:18:17.077298+0000","last_unstale":"2023-05-16T07:18:17.451251+0000","last_undegraded":"2023-05-16T07:18:17.451251+0000","last_fullsized":"2023-05-16T07:18:17.451251+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448543+0000","last_change":"2023-05-16T07:18:17.448543+0000","last_active":"2023-05-16T07:18:17.448543+0000","last_peered":"2023-05-16T07:18:17.448543+0000","last_clean":"2023-05-16T07:18:17.448543+0000","last_became_active":"2023-05-16T07:18:17.448270+0000","last_became_peered":"2023-05-16T07:18:17.448270+0000","last_unstale":"2023-05-16T07:18:17.448543+0000","last_undegraded":"2023-05-16T07:18:17.448543+0000","last_fullsized":"2023-05-16T07:18:17.448543+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446783+0000","last_change":"2023-05-16T07:18:17.070416+0000","last_active":"2023-05-16T07:18:17.446783+0000","last_peered":"2023-05-16T07:18:17.446783+0000","last_clean":"2023-05-16T07:18:17.446783+0000","last_became_active":"2023-05-16T07:18:17.069703+0000","last_became_peered":"2023-05-16T07:18:17.069703+0000","last_unstale":"2023-05-16T07:18:17.446783+0000","last_undegraded":"2023-05-16T07:18:17.446783+0000","last_fullsized":"2023-05-16T07:18:17.446783+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449318+0000","last_change":"2023-05-16T07:18:17.449318+0000","last_active":"2023-05-16T07:18:17.449318+0000","last_peered":"2023-05-16T07:18:17.449318+0000","last_clean":"2023-05-16T07:18:17.449318+0000","last_became_active":"2023-05-16T07:18:17.448949+0000","last_became_peered":"2023-05-16T07:18:17.448949+0000","last_unstale":"2023-05-16T07:18:17.449318+0000","last_undegraded":"2023-05-16T07:18:17.449318+0000","last_fullsized":"2023-05-16T07:18:17.449318+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.431385+0000","last_change":"2023-05-16T07:18:17.077857+0000","last_active":"2023-05-16T07:18:17.431385+0000","last_peered":"2023-05-16T07:18:17.431385+0000","last_clean":"2023-05-16T07:18:17.431385+0000","last_became_active":"2023-05-16T07:18:17.077540+0000","last_became_peered":"2023-05-16T07:18:17.077540+0000","last_unstale":"2023-05-16T07:18:17.431385+0000","last_undegraded":"2023-05-16T07:18:17.431385+0000","last_fullsized":"2023-05-16T07:18:17.431385+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.434362+0000","last_change":"2023-05-16T07:18:17.434362+0000","last_active":"2023-05-16T07:18:17.434362+0000","last_peered":"2023-05-16T07:18:17.434362+0000","last_clean":"2023-05-16T07:18:17.434362+0000","last_became_active":"2023-05-16T07:18:17.433890+0000","last_became_peered":"2023-05-16T07:18:17.433890+0000","last_unstale":"2023-05-16T07:18:17.434362+0000","last_undegraded":"2023-05-16T07:18:17.434362+0000","last_fullsized":"2023-05-16T07:18:17.434362+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076801+0000","last_change":"2023-05-16T07:18:17.071296+0000","last_active":"2023-05-16T07:18:17.076801+0000","last_peered":"2023-05-16T07:18:17.076801+0000","last_clean":"2023-05-16T07:18:17.076801+0000","last_became_active":"2023-05-16T07:18:17.071047+0000","last_became_peered":"2023-05-16T07:18:17.071047+0000","last_unstale":"2023-05-16T07:18:17.076801+0000","last_undegraded":"2023-05-16T07:18:17.076801+0000","last_fullsized":"2023-05-16T07:18:17.076801+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446038+0000","last_change":"2023-05-16T07:18:17.446038+0000","last_active":"2023-05-16T07:18:17.446038+0000","last_peered":"2023-05-16T07:18:17.446038+0000","last_clean":"2023-05-16T07:18:17.446038+0000","last_became_active":"2023-05-16T07:18:17.445783+0000","last_became_peered":"2023-05-16T07:18:17.445783+0000","last_unstale":"2023-05-16T07:18:17.446038+0000","last_undegraded":"2023-05-16T07:18:17.446038+0000","last_fullsized":"2023-05-16T07:18:17.446038+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082875+0000","last_change":"2023-05-16T07:18:17.077641+0000","last_active":"2023-05-16T07:18:17.082875+0000","last_peered":"2023-05-16T07:18:17.082875+0000","last_clean":"2023-05-16T07:18:17.082875+0000","last_became_active":"2023-05-16T07:18:17.077383+0000","last_became_peered":"2023-05-16T07:18:17.077383+0000","last_unstale":"2023-05-16T07:18:17.082875+0000","last_undegraded":"2023-05-16T07:18:17.082875+0000","last_fullsized":"2023-05-16T07:18:17.082875+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445432+0000","last_change":"2023-05-16T07:18:17.445432+0000","last_active":"2023-05-16T07:18:17.445432+0000","last_peered":"2023-05-16T07:18:17.445432+0000","last_clean":"2023-05-16T07:18:17.445432+0000","last_became_active":"2023-05-16T07:18:17.445208+0000","last_became_peered":"2023-05-16T07:18:17.445208+0000","last_unstale":"2023-05-16T07:18:17.445432+0000","last_undegraded":"2023-05-16T07:18:17.445432+0000","last_fullsized":"2023-05-16T07:18:17.445432+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.451663+0000","last_change":"2023-05-16T07:18:17.077534+0000","last_active":"2023-05-16T07:18:17.451663+0000","last_peered":"2023-05-16T07:18:17.451663+0000","last_clean":"2023-05-16T07:18:17.451663+0000","last_became_active":"2023-05-16T07:18:17.077247+0000","last_became_peered":"2023-05-16T07:18:17.077247+0000","last_unstale":"2023-05-16T07:18:17.451663+0000","last_undegraded":"2023-05-16T07:18:17.451663+0000","last_fullsized":"2023-05-16T07:18:17.451663+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.434819+0000","last_change":"2023-05-16T07:18:17.434819+0000","last_active":"2023-05-16T07:18:17.434819+0000","last_peered":"2023-05-16T07:18:17.434819+0000","last_clean":"2023-05-16T07:18:17.434819+0000","last_became_active":"2023-05-16T07:18:17.434534+0000","last_became_peered":"2023-05-16T07:18:17.434534+0000","last_unstale":"2023-05-16T07:18:17.434819+0000","last_undegraded":"2023-05-16T07:18:17.434819+0000","last_fullsized":"2023-05-16T07:18:17.434819+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448921+0000","last_change":"2023-05-16T07:18:17.074502+0000","last_active":"2023-05-16T07:18:17.448921+0000","last_peered":"2023-05-16T07:18:17.448921+0000","last_clean":"2023-05-16T07:18:17.448921+0000","last_became_active":"2023-05-16T07:18:17.074176+0000","last_became_peered":"2023-05-16T07:18:17.074176+0000","last_unstale":"2023-05-16T07:18:17.448921+0000","last_undegraded":"2023-05-16T07:18:17.448921+0000","last_fullsized":"2023-05-16T07:18:17.448921+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448702+0000","last_change":"2023-05-16T07:18:17.448702+0000","last_active":"2023-05-16T07:18:17.448702+0000","last_peered":"2023-05-16T07:18:17.448702+0000","last_clean":"2023-05-16T07:18:17.448702+0000","last_became_active":"2023-05-16T07:18:17.448435+0000","last_became_peered":"2023-05-16T07:18:17.448435+0000","last_unstale":"2023-05-16T07:18:17.448702+0000","last_undegraded":"2023-05-16T07:18:17.448702+0000","last_fullsized":"2023-05-16T07:18:17.448702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.067294+0000","last_change":"2023-05-16T07:18:17.065247+0000","last_active":"2023-05-16T07:18:17.067294+0000","last_peered":"2023-05-16T07:18:17.067294+0000","last_clean":"2023-05-16T07:18:17.067294+0000","last_became_active":"2023-05-16T07:18:17.064966+0000","last_became_peered":"2023-05-16T07:18:17.064966+0000","last_unstale":"2023-05-16T07:18:17.067294+0000","last_undegraded":"2023-05-16T07:18:17.067294+0000","last_fullsized":"2023-05-16T07:18:17.067294+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444722+0000","last_change":"2023-05-16T07:18:17.444722+0000","last_active":"2023-05-16T07:18:17.444722+0000","last_peered":"2023-05-16T07:18:17.444722+0000","last_clean":"2023-05-16T07:18:17.444722+0000","last_became_active":"2023-05-16T07:18:17.444477+0000","last_became_peered":"2023-05-16T07:18:17.444477+0000","last_unstale":"2023-05-16T07:18:17.444722+0000","last_undegraded":"2023-05-16T07:18:17.444722+0000","last_fullsized":"2023-05-16T07:18:17.444722+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.074975+0000","last_change":"2023-05-16T07:18:17.072555+0000","last_active":"2023-05-16T07:18:17.074975+0000","last_peered":"2023-05-16T07:18:17.074975+0000","last_clean":"2023-05-16T07:18:17.074975+0000","last_became_active":"2023-05-16T07:18:17.072276+0000","last_became_peered":"2023-05-16T07:18:17.072276+0000","last_unstale":"2023-05-16T07:18:17.074975+0000","last_undegraded":"2023-05-16T07:18:17.074975+0000","last_fullsized":"2023-05-16T07:18:17.074975+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448772+0000","last_change":"2023-05-16T07:18:17.448772+0000","last_active":"2023-05-16T07:18:17.448772+0000","last_peered":"2023-05-16T07:18:17.448772+0000","last_clean":"2023-05-16T07:18:17.448772+0000","last_became_active":"2023-05-16T07:18:17.446518+0000","last_became_peered":"2023-05-16T07:18:17.446518+0000","last_unstale":"2023-05-16T07:18:17.448772+0000","last_undegraded":"2023-05-16T07:18:17.448772+0000","last_fullsized":"2023-05-16T07:18:17.448772+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077152+0000","last_change":"2023-05-16T07:18:17.073523+0000","last_active":"2023-05-16T07:18:17.077152+0000","last_peered":"2023-05-16T07:18:17.077152+0000","last_clean":"2023-05-16T07:18:17.077152+0000","last_became_active":"2023-05-16T07:18:17.073223+0000","last_became_peered":"2023-05-16T07:18:17.073223+0000","last_unstale":"2023-05-16T07:18:17.077152+0000","last_undegraded":"2023-05-16T07:18:17.077152+0000","last_fullsized":"2023-05-16T07:18:17.077152+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440599+0000","last_change":"2023-05-16T07:18:17.440599+0000","last_active":"2023-05-16T07:18:17.440599+0000","last_peered":"2023-05-16T07:18:17.440599+0000","last_clean":"2023-05-16T07:18:17.440599+0000","last_became_active":"2023-05-16T07:18:17.437506+0000","last_became_peered":"2023-05-16T07:18:17.437506+0000","last_unstale":"2023-05-16T07:18:17.440599+0000","last_undegraded":"2023-05-16T07:18:17.440599+0000","last_fullsized":"2023-05-16T07:18:17.440599+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076845+0000","last_change":"2023-05-16T07:18:17.071698+0000","last_active":"2023-05-16T07:18:17.076845+0000","last_peered":"2023-05-16T07:18:17.076845+0000","last_clean":"2023-05-16T07:18:17.076845+0000","last_became_active":"2023-05-16T07:18:17.071022+0000","last_became_peered":"2023-05-16T07:18:17.071022+0000","last_unstale":"2023-05-16T07:18:17.076845+0000","last_undegraded":"2023-05-16T07:18:17.076845+0000","last_fullsized":"2023-05-16T07:18:17.076845+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440633+0000","last_change":"2023-05-16T07:18:17.440633+0000","last_active":"2023-05-16T07:18:17.440633+0000","last_peered":"2023-05-16T07:18:17.440633+0000","last_clean":"2023-05-16T07:18:17.440633+0000","last_became_active":"2023-05-16T07:18:17.440374+0000","last_became_peered":"2023-05-16T07:18:17.440374+0000","last_unstale":"2023-05-16T07:18:17.440633+0000","last_undegraded":"2023-05-16T07:18:17.440633+0000","last_fullsized":"2023-05-16T07:18:17.440633+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.075290+0000","last_change":"2023-05-16T07:18:17.072234+0000","last_active":"2023-05-16T07:18:17.075290+0000","last_peered":"2023-05-16T07:18:17.075290+0000","last_clean":"2023-05-16T07:18:17.075290+0000","last_became_active":"2023-05-16T07:18:17.071749+0000","last_became_peered":"2023-05-16T07:18:17.071749+0000","last_unstale":"2023-05-16T07:18:17.075290+0000","last_undegraded":"2023-05-16T07:18:17.075290+0000","last_fullsized":"2023-05-16T07:18:17.075290+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444506+0000","last_change":"2023-05-16T07:18:17.444506+0000","last_active":"2023-05-16T07:18:17.444506+0000","last_peered":"2023-05-16T07:18:17.444506+0000","last_clean":"2023-05-16T07:18:17.444506+0000","last_became_active":"2023-05-16T07:18:17.444274+0000","last_became_peered":"2023-05-16T07:18:17.444274+0000","last_unstale":"2023-05-16T07:18:17.444506+0000","last_undegraded":"2023-05-16T07:18:17.444506+0000","last_fullsized":"2023-05-16T07:18:17.444506+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076862+0000","last_change":"2023-05-16T07:18:17.070734+0000","last_active":"2023-05-16T07:18:17.076862+0000","last_peered":"2023-05-16T07:18:17.076862+0000","last_clean":"2023-05-16T07:18:17.076862+0000","last_became_active":"2023-05-16T07:18:17.069874+0000","last_became_peered":"2023-05-16T07:18:17.069874+0000","last_unstale":"2023-05-16T07:18:17.076862+0000","last_undegraded":"2023-05-16T07:18:17.076862+0000","last_fullsized":"2023-05-16T07:18:17.076862+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444576+0000","last_change":"2023-05-16T07:18:17.444576+0000","last_active":"2023-05-16T07:18:17.444576+0000","last_peered":"2023-05-16T07:18:17.444576+0000","last_clean":"2023-05-16T07:18:17.444576+0000","last_became_active":"2023-05-16T07:18:17.444046+0000","last_became_peered":"2023-05-16T07:18:17.444046+0000","last_unstale":"2023-05-16T07:18:17.444576+0000","last_undegraded":"2023-05-16T07:18:17.444576+0000","last_fullsized":"2023-05-16T07:18:17.444576+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.084150+0000","last_change":"2023-05-16T07:18:17.083583+0000","last_active":"2023-05-16T07:18:17.084150+0000","last_peered":"2023-05-16T07:18:17.084150+0000","last_clean":"2023-05-16T07:18:17.084150+0000","last_became_active":"2023-05-16T07:18:17.083355+0000","last_became_peered":"2023-05-16T07:18:17.083355+0000","last_unstale":"2023-05-16T07:18:17.084150+0000","last_undegraded":"2023-05-16T07:18:17.084150+0000","last_fullsized":"2023-05-16T07:18:17.084150+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.443633+0000","last_change":"2023-05-16T07:18:17.443633+0000","last_active":"2023-05-16T07:18:17.443633+0000","last_peered":"2023-05-16T07:18:17.443633+0000","last_clean":"2023-05-16T07:18:17.443633+0000","last_became_active":"2023-05-16T07:18:17.443435+0000","last_became_peered":"2023-05-16T07:18:17.443435+0000","last_unstale":"2023-05-16T07:18:17.443633+0000","last_undegraded":"2023-05-16T07:18:17.443633+0000","last_fullsized":"2023-05-16T07:18:17.443633+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.432721+0000","last_change":"2023-05-16T07:18:17.072746+0000","last_active":"2023-05-16T07:18:17.432721+0000","last_peered":"2023-05-16T07:18:17.432721+0000","last_clean":"2023-05-16T07:18:17.432721+0000","last_became_active":"2023-05-16T07:18:17.072432+0000","last_became_peered":"2023-05-16T07:18:17.072432+0000","last_unstale":"2023-05-16T07:18:17.432721+0000","last_undegraded":"2023-05-16T07:18:17.432721+0000","last_fullsized":"2023-05-16T07:18:17.432721+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076473+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:18:17.076473+0000","last_peered":"2023-05-16T07:18:17.076473+0000","last_clean":"2023-05-16T07:18:17.076473+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:18:17.076473+0000","last_undegraded":"2023-05-16T07:18:17.076473+0000","last_fullsized":"2023-05-16T07:18:17.076473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.442676+0000","last_change":"2023-05-16T07:18:17.442676+0000","last_active":"2023-05-16T07:18:17.442676+0000","last_peered":"2023-05-16T07:18:17.442676+0000","last_clean":"2023-05-16T07:18:17.442676+0000","last_became_active":"2023-05-16T07:18:17.441971+0000","last_became_peered":"2023-05-16T07:18:17.441971+0000","last_unstale":"2023-05-16T07:18:17.442676+0000","last_undegraded":"2023-05-16T07:18:17.442676+0000","last_fullsized":"2023-05-16T07:18:17.442676+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080471+0000","last_change":"2023-05-16T07:18:17.079288+0000","last_active":"2023-05-16T07:18:17.080471+0000","last_peered":"2023-05-16T07:18:17.080471+0000","last_clean":"2023-05-16T07:18:17.080471+0000","last_became_active":"2023-05-16T07:18:17.078986+0000","last_became_peered":"2023-05-16T07:18:17.078986+0000","last_unstale":"2023-05-16T07:18:17.080471+0000","last_undegraded":"2023-05-16T07:18:17.080471+0000","last_fullsized":"2023-05-16T07:18:17.080471+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":81,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.063535+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:18:17.063535+0000","last_peered":"2023-05-16T07:18:17.063535+0000","last_clean":"2023-05-16T07:18:17.063535+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:18:17.063535+0000","last_undegraded":"2023-05-16T07:18:17.063535+0000","last_fullsized":"2023-05-16T07:18:17.063535+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440521+0000","last_change":"2023-05-16T07:18:17.440521+0000","last_active":"2023-05-16T07:18:17.440521+0000","last_peered":"2023-05-16T07:18:17.440521+0000","last_clean":"2023-05-16T07:18:17.440521+0000","last_became_active":"2023-05-16T07:18:17.440206+0000","last_became_peered":"2023-05-16T07:18:17.440206+0000","last_unstale":"2023-05-16T07:18:17.440521+0000","last_undegraded":"2023-05-16T07:18:17.440521+0000","last_fullsized":"2023-05-16T07:18:17.440521+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447368+0000","last_change":"2023-05-16T07:18:17.074712+0000","last_active":"2023-05-16T07:18:17.447368+0000","last_peered":"2023-05-16T07:18:17.447368+0000","last_clean":"2023-05-16T07:18:17.447368+0000","last_became_active":"2023-05-16T07:18:17.074231+0000","last_became_peered":"2023-05-16T07:18:17.074231+0000","last_unstale":"2023-05-16T07:18:17.447368+0000","last_undegraded":"2023-05-16T07:18:17.447368+0000","last_fullsized":"2023-05-16T07:18:17.447368+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.066141+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:18:17.066141+0000","last_peered":"2023-05-16T07:18:17.066141+0000","last_clean":"2023-05-16T07:18:17.066141+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:18:17.066141+0000","last_undegraded":"2023-05-16T07:18:17.066141+0000","last_fullsized":"2023-05-16T07:18:17.066141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.438877+0000","last_change":"2023-05-16T07:18:17.438877+0000","last_active":"2023-05-16T07:18:17.438877+0000","last_peered":"2023-05-16T07:18:17.438877+0000","last_clean":"2023-05-16T07:18:17.438877+0000","last_became_active":"2023-05-16T07:18:17.438573+0000","last_became_peered":"2023-05-16T07:18:17.438573+0000","last_unstale":"2023-05-16T07:18:17.438877+0000","last_undegraded":"2023-05-16T07:18:17.438877+0000","last_fullsized":"2023-05-16T07:18:17.438877+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082126+0000","last_change":"2023-05-16T07:18:17.076985+0000","last_active":"2023-05-16T07:18:17.082126+0000","last_peered":"2023-05-16T07:18:17.082126+0000","last_clean":"2023-05-16T07:18:17.082126+0000","last_became_active":"2023-05-16T07:18:17.076696+0000","last_became_peered":"2023-05-16T07:18:17.076696+0000","last_unstale":"2023-05-16T07:18:17.082126+0000","last_undegraded":"2023-05-16T07:18:17.082126+0000","last_fullsized":"2023-05-16T07:18:17.082126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":81,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.061665+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:18:17.061665+0000","last_peered":"2023-05-16T07:18:17.061665+0000","last_clean":"2023-05-16T07:18:17.061665+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:18:17.061665+0000","last_undegraded":"2023-05-16T07:18:17.061665+0000","last_fullsized":"2023-05-16T07:18:17.061665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.068277+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:18:17.068277+0000","last_peered":"2023-05-16T07:18:17.068277+0000","last_clean":"2023-05-16T07:18:17.068277+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:18:17.068277+0000","last_undegraded":"2023-05-16T07:18:17.068277+0000","last_fullsized":"2023-05-16T07:18:17.068277+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.442748+0000","last_change":"2023-05-16T07:18:17.442748+0000","last_active":"2023-05-16T07:18:17.442748+0000","last_peered":"2023-05-16T07:18:17.442748+0000","last_clean":"2023-05-16T07:18:17.442748+0000","last_became_active":"2023-05-16T07:18:17.442494+0000","last_became_peered":"2023-05-16T07:18:17.442494+0000","last_unstale":"2023-05-16T07:18:17.442748+0000","last_undegraded":"2023-05-16T07:18:17.442748+0000","last_fullsized":"2023-05-16T07:18:17.442748+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.428932+0000","last_change":"2023-05-16T07:18:17.075860+0000","last_active":"2023-05-16T07:18:17.428932+0000","last_peered":"2023-05-16T07:18:17.428932+0000","last_clean":"2023-05-16T07:18:17.428932+0000","last_became_active":"2023-05-16T07:18:17.075525+0000","last_became_peered":"2023-05-16T07:18:17.075525+0000","last_unstale":"2023-05-16T07:18:17.428932+0000","last_undegraded":"2023-05-16T07:18:17.428932+0000","last_fullsized":"2023-05-16T07:18:17.428932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.068353+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:18:17.068353+0000","last_peered":"2023-05-16T07:18:17.068353+0000","last_clean":"2023-05-16T07:18:17.068353+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:18:17.068353+0000","last_undegraded":"2023-05-16T07:18:17.068353+0000","last_fullsized":"2023-05-16T07:18:17.068353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446910+0000","last_change":"2023-05-16T07:18:17.446910+0000","last_active":"2023-05-16T07:18:17.446910+0000","last_peered":"2023-05-16T07:18:17.446910+0000","last_clean":"2023-05-16T07:18:17.446910+0000","last_became_active":"2023-05-16T07:18:17.446507+0000","last_became_peered":"2023-05-16T07:18:17.446507+0000","last_unstale":"2023-05-16T07:18:17.446910+0000","last_undegraded":"2023-05-16T07:18:17.446910+0000","last_fullsized":"2023-05-16T07:18:17.446910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.085865+0000","last_change":"2023-05-16T07:18:17.082252+0000","last_active":"2023-05-16T07:18:17.085865+0000","last_peered":"2023-05-16T07:18:17.085865+0000","last_clean":"2023-05-16T07:18:17.085865+0000","last_became_active":"2023-05-16T07:18:17.081778+0000","last_became_peered":"2023-05-16T07:18:17.081778+0000","last_unstale":"2023-05-16T07:18:17.085865+0000","last_undegraded":"2023-05-16T07:18:17.085865+0000","last_fullsized":"2023-05-16T07:18:17.085865+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.069334+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:18:17.069334+0000","last_peered":"2023-05-16T07:18:17.069334+0000","last_clean":"2023-05-16T07:18:17.069334+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:18:17.069334+0000","last_undegraded":"2023-05-16T07:18:17.069334+0000","last_fullsized":"2023-05-16T07:18:17.069334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448644+0000","last_change":"2023-05-16T07:18:17.448644+0000","last_active":"2023-05-16T07:18:17.448644+0000","last_peered":"2023-05-16T07:18:17.448644+0000","last_clean":"2023-05-16T07:18:17.448644+0000","last_became_active":"2023-05-16T07:18:17.448114+0000","last_became_peered":"2023-05-16T07:18:17.448114+0000","last_unstale":"2023-05-16T07:18:17.448644+0000","last_undegraded":"2023-05-16T07:18:17.448644+0000","last_fullsized":"2023-05-16T07:18:17.448644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073370+0000","last_change":"2023-05-16T07:18:17.070927+0000","last_active":"2023-05-16T07:18:17.073370+0000","last_peered":"2023-05-16T07:18:17.073370+0000","last_clean":"2023-05-16T07:18:17.073370+0000","last_became_active":"2023-05-16T07:18:17.070460+0000","last_became_peered":"2023-05-16T07:18:17.070460+0000","last_unstale":"2023-05-16T07:18:17.073370+0000","last_undegraded":"2023-05-16T07:18:17.073370+0000","last_fullsized":"2023-05-16T07:18:17.073370+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.065800+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:18:17.065800+0000","last_peered":"2023-05-16T07:18:17.065800+0000","last_clean":"2023-05-16T07:18:17.065800+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:18:17.065800+0000","last_undegraded":"2023-05-16T07:18:17.065800+0000","last_fullsized":"2023-05-16T07:18:17.065800+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444765+0000","last_change":"2023-05-16T07:18:17.444765+0000","last_active":"2023-05-16T07:18:17.444765+0000","last_peered":"2023-05-16T07:18:17.444765+0000","last_clean":"2023-05-16T07:18:17.444765+0000","last_became_active":"2023-05-16T07:18:17.444327+0000","last_became_peered":"2023-05-16T07:18:17.444327+0000","last_unstale":"2023-05-16T07:18:17.444765+0000","last_undegraded":"2023-05-16T07:18:17.444765+0000","last_fullsized":"2023-05-16T07:18:17.444765+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.072927+0000","last_change":"2023-05-16T07:18:17.068695+0000","last_active":"2023-05-16T07:18:17.072927+0000","last_peered":"2023-05-16T07:18:17.072927+0000","last_clean":"2023-05-16T07:18:17.072927+0000","last_became_active":"2023-05-16T07:18:17.068166+0000","last_became_peered":"2023-05-16T07:18:17.068166+0000","last_unstale":"2023-05-16T07:18:17.072927+0000","last_undegraded":"2023-05-16T07:18:17.072927+0000","last_fullsized":"2023-05-16T07:18:17.072927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.058423+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:18:17.058423+0000","last_peered":"2023-05-16T07:18:17.058423+0000","last_clean":"2023-05-16T07:18:17.058423+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:18:17.058423+0000","last_undegraded":"2023-05-16T07:18:17.058423+0000","last_fullsized":"2023-05-16T07:18:17.058423+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18120,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":16832,"kb_avail":94353720,"statfs":{"total":96636764160,"available":96618209280,"internally_reserved":0,"allocated":1310720,"data_stored":675719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17235968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.77700000000000002,"15min":0.84199999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67400000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":0.873},"last":0.503},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70499999999999996,"5min":0.876,"15min":0.876},"last":0.495}]},{"osd":2,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.441},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.35899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.46200000000000002},"min":{"1min":0.308,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47399999999999998,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.752,"15min":0.81999999999999995},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47599999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66300000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73899999999999999,"5min":0.748,"15min":0.999},"last":0.32300000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.879,"15min":0.95099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.38900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21480,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350360,"statfs":{"total":96636764160,"available":96614768640,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.317,"15min":0.26400000000000001},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66500000000000004,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76000000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.311,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.89400000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.503,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":4.048,"15min":4.048},"last":0.41799999999999998}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.749,"5min":0.749,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575039,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17512,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":16256,"kb_avail":94354328,"statfs":{"total":96636764160,"available":96618831872,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16646144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.69899999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.48199999999999998,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36499999999999999,"15min":0.378},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26000000000000001},"max":{"1min":0.49399999999999999,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.52700000000000002,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38300000000000001,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.501,"5min":0.622,"15min":0.64000000000000001},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71199999999999997,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76100000000000001,"5min":0.94699999999999995,"15min":1.087},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.93700000000000006,"15min":1.204},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.627,"15min":0.64100000000000001},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41299999999999998},"max":{"1min":0.76900000000000002,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575039,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16564,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":15296,"kb_avail":94355276,"statfs":{"total":96636764160,"available":96619802624,"internally_reserved":0,"allocated":1290240,"data_stored":667900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15663104},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33900000000000002,"15min":0.35699999999999998},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47099999999999997,"5min":0.53700000000000003,"15min":0.60199999999999998},"last":0.222},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.38600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.35299999999999998,"15min":0.373},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.40300000000000002},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.50600000000000001,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.251}]},{"osd":3,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38600000000000001,"15min":0.40699999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.24199999999999999},"max":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.28599999999999998}]},{"osd":4,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60599999999999998,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":1.157},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.56799999999999995}]},{"osd":6,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.65000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85499999999999998,"5min":0.93700000000000006,"15min":1.196},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18344,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":17088,"kb_avail":94353496,"statfs":{"total":96636764160,"available":96617979904,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17498112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.5,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.249},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.375,"15min":0.376},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.215}]},{"osd":1,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.434,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.29999999999999999}]},{"osd":3,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.64700000000000002},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.432,"15min":0.439},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60899999999999999,"5min":0.71399999999999997,"15min":0.72899999999999998},"last":0.29199999999999998}]},{"osd":4,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.274},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82599999999999996,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.37},"max":{"1min":0.872,"5min":0.88100000000000001,"15min":1.5609999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.47},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88200000000000001,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21448,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":20160,"kb_avail":94350392,"statfs":{"total":96636764160,"available":96614801408,"internally_reserved":0,"allocated":1310720,"data_stored":676625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20643840},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36099999999999999},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53300000000000003,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.36699999999999999,"15min":0.375},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46400000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45000000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40799999999999997,"15min":0.40899999999999997},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51300000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48199999999999998,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.52000000000000002}]},{"osd":5,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85799999999999998,"5min":0.91600000000000004,"15min":1.0489999999999999},"last":0.498},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70199999999999996,"5min":0.93799999999999994,"15min":1.0409999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.97399999999999998,"15min":1.0780000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.73099999999999998,"5min":0.94299999999999995,"15min":1.0049999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.40100000000000002},"max":{"1min":0.76400000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.93600000000000005,"15min":1.02},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17576,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":16320,"kb_avail":94354264,"statfs":{"total":96636764160,"available":96618766336,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16711680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.82099999999999995,"15min":0.83199999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.81200000000000006},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.627,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.67500000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.90500000000000003,"15min":0.999},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.97999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.629,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84099999999999997,"5min":0.875,"15min":0.93100000000000005},"last":0.626}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":1.0640000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70699999999999996,"5min":0.79200000000000004,"15min":0.94599999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.86199999999999999,"15min":1.2689999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":1.48,"15min":1.48},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":1.046},"last":0.67200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24948,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":23680,"kb_avail":94346892,"statfs":{"total":96636764160,"available":96611217408,"internally_reserved":0,"allocated":1290240,"data_stored":667900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24248320},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.36599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81599999999999995,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":3.641},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.81000000000000005,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":2.847}]},{"osd":1,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":0.96099999999999997,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86399999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":3.6059999999999999}]},{"osd":2,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71399999999999997,"5min":2.278,"15min":2.278},"last":2.8799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.45100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.748,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.127},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":2.806}]},{"osd":4,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.33900000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":1.333,"15min":1.333},"last":0.318},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.309},"max":{"1min":0.84899999999999998,"5min":2.081,"15min":2.081},"last":3.5659999999999998}]},{"osd":6,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":2.694},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":1.024,"15min":1.024},"last":1.554}]},{"osd":7,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.878,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.86599999999999999,"15min":0.878},"last":2.6429999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:18:20.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:18:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.764+0000 7f018ce35700 1 Processor -- start 2023-05-16T07:18:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.764+0000 7f018ce35700 1 -- start start 2023-05-16T07:18:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f018ce35700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01880c7380 0x7f0188154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f018ce35700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f01880c7760 0x7f01880acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f018ce35700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0188158970 con 0x7f01880ad040 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f018ce35700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0188155340 con 0x7f01880c7380 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f018ce35700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0188157d60 con 0x7f01880c7760 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01880c7380 0x7f0188154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f0187fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f01880ad040 0x7f018814dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58500/0 (socket says 172.21.15.92:58500) 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f0187fff700 1 -- 172.21.15.92:0/865727392 learned_addr learned my addr 172.21.15.92:0/865727392 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f0186ffd700 1 --2- 172.21.15.92:0/865727392 >> v2:172.21.15.158:3300/0 conn(0x7f01880c7760 0x7f01880acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01877fe700 1 -- 172.21.15.92:0/865727392 >> v2:172.21.15.158:3300/0 conn(0x7f01880c7760 msgr2=0x7f01880acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01877fe700 1 --2- 172.21.15.92:0/865727392 >> v2:172.21.15.158:3300/0 conn(0x7f01880c7760 0x7f01880acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01877fe700 1 -- 172.21.15.92:0/865727392 >> v1:172.21.15.92:6789/0 conn(0x7f01880ad040 legacy=0x7f018814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01877fe700 1 -- 172.21.15.92:0/865727392 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188158040 con 0x7f01880c7380 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01867fc700 1 -- 172.21.15.92:0/865727392 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558448170 0 0) 0x7f0188158970 con 0x7f01880ad040 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f0186ffd700 1 --2- 172.21.15.92:0/865727392 >> v2:172.21.15.158:3300/0 conn(0x7f01880c7760 0x7f01880acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01877fe700 1 --2- 172.21.15.92:0/865727392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880c7380 0x7f0188154e50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f0188084350 tx=0x7f017c009060).ready entity=mon.0 client_cookie=7a8be86804602c02 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01867fc700 1 -- 172.21.15.92:0/865727392 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c00b580 con 0x7f01880c7380 2023-05-16T07:18:20.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01867fc700 1 -- 172.21.15.92:0/865727392 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c00b6e0 con 0x7f01880c7380 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.765+0000 7f01867fc700 1 -- 172.21.15.92:0/865727392 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c00ba60 con 0x7f01880c7380 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 -- 172.21.15.92:0/865727392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880c7380 msgr2=0x7f0188154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 --2- 172.21.15.92:0/865727392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880c7380 0x7f0188154e50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f0188084350 tx=0x7f017c009060).stop 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 -- 172.21.15.92:0/865727392 shutdown_connections 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 --2- 172.21.15.92:0/865727392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880c7380 0x7f0188154e50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 --2- 172.21.15.92:0/865727392 >> v2:172.21.15.158:3300/0 conn(0x7f01880c7760 0x7f01880acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 -- 172.21.15.92:0/865727392 >> 172.21.15.92:0/865727392 conn(0x7f0188068330 msgr2=0x7f0188068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 -- 172.21.15.92:0/865727392 shutdown_connections 2023-05-16T07:18:20.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.766+0000 7f018ce35700 1 -- 172.21.15.92:0/865727392 wait complete. 2023-05-16T07:18:20.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.767+0000 7f018ce35700 1 Processor -- start 2023-05-16T07:18:20.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.767+0000 7f018ce35700 1 -- start start 2023-05-16T07:18:20.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 0x7f01880c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01880c7380 0x7f0188164010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01880c7760 0x7f01881501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01880c1c10 con 0x7f01880ad040 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01880c1ef0 con 0x7f01880c7760 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0188164870 con 0x7f01880c7380 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 0x7f01880c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 0x7f01880c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47756/0 (socket says 172.21.15.92:47756) 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 -- 172.21.15.92:0/926840852 learned_addr learned my addr 172.21.15.92:0/926840852 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 -- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01880c7380 msgr2=0x7f0188164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01880c7380 0x7f0188164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 -- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01880c7760 msgr2=0x7f01881501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01880c7760 0x7f01881501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 -- 172.21.15.92:0/926840852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0092a0 con 0x7f01880ad040 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f01877fe700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 0x7f01880c8cd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f017c009fb0 tx=0x7f017c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c00ba60 con 0x7f01880ad040 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c000c90 con 0x7f01880ad040 2023-05-16T07:18:20.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0188164b50 con 0x7f01880ad040 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c01b640 con 0x7f01880ad040 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.768+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0188164500 con 0x7f01880ad040 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.769+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0188047150 con 0x7f01880ad040 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.770+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f017c02b070 con 0x7f01880ad040 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.770+0000 7f016f7fe700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f017405ed90 0x7f0174061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.770+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f017c07b820 con 0x7f01880ad040 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.770+0000 7f0186ffd700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f017405ed90 0x7f0174061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:20.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.771+0000 7f0186ffd700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f017405ed90 0x7f0174061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f018814ef60 tx=0x7f0178008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:20.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.774+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f017c04cb70 con 0x7f01880ad040 2023-05-16T07:18:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.933+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01880bfb90 con 0x7f017405ed90 2023-05-16T07:18:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.943+0000 7f016f7fe700 1 -- 172.21.15.92:0/926840852 <== mgr.6373 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361751 (secure 0 0 0) 0x7f01880bfb90 con 0x7f017405ed90 2023-05-16T07:18:20.950 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:20.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f017405ed90 msgr2=0x7f0174061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f017405ed90 0x7f0174061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f018814ef60 tx=0x7f0178008040).stop 2023-05-16T07:18:20.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 msgr2=0x7f01880c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 0x7f01880c8cd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f017c009fb0 tx=0x7f017c017000).stop 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 shutdown_connections 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f017405ed90 0x7f0174061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01880c7760 0x7f01881501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01880ad040 0x7f01880c8cd0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 --2- 172.21.15.92:0/926840852 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01880c7380 0x7f0188164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:20.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 >> 172.21.15.92:0/926840852 conn(0x7f0188068330 msgr2=0x7f018814f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:20.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 shutdown_connections 2023-05-16T07:18:20.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:20.947+0000 7f018ce35700 1 -- 172.21.15.92:0/926840852 wait complete. 2023-05-16T07:18:20.955 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:18:20.962 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-05-16T07:18:20.899801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":155992,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":145856,"kb_avail":754818728,"statfs":{"total":773094113280,"available":772934377472,"internally_reserved":0,"allocated":10313728,"data_stored":5330692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":149356544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4751,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-53,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.597568"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441008+0000","last_change":"2023-05-16T07:18:17.441008+0000","last_active":"2023-05-16T07:18:17.441008+0000","last_peered":"2023-05-16T07:18:17.441008+0000","last_clean":"2023-05-16T07:18:17.441008+0000","last_became_active":"2023-05-16T07:18:17.440798+0000","last_became_peered":"2023-05-16T07:18:17.440798+0000","last_unstale":"2023-05-16T07:18:17.441008+0000","last_undegraded":"2023-05-16T07:18:17.441008+0000","last_fullsized":"2023-05-16T07:18:17.441008+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"106'3","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449285+0000","last_change":"2023-05-16T07:18:17.064002+0000","last_active":"2023-05-16T07:18:17.449285+0000","last_peered":"2023-05-16T07:18:17.449285+0000","last_clean":"2023-05-16T07:18:17.449285+0000","last_became_active":"2023-05-16T07:18:17.063705+0000","last_became_peered":"2023-05-16T07:18:17.063705+0000","last_unstale":"2023-05-16T07:18:17.449285+0000","last_undegraded":"2023-05-16T07:18:17.449285+0000","last_fullsized":"2023-05-16T07:18:17.449285+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.433114+0000","last_change":"2023-05-16T07:18:17.433114+0000","last_active":"2023-05-16T07:18:17.433114+0000","last_peered":"2023-05-16T07:18:17.433114+0000","last_clean":"2023-05-16T07:18:17.433114+0000","last_became_active":"2023-05-16T07:18:17.432833+0000","last_became_peered":"2023-05-16T07:18:17.432833+0000","last_unstale":"2023-05-16T07:18:17.433114+0000","last_undegraded":"2023-05-16T07:18:17.433114+0000","last_fullsized":"2023-05-16T07:18:17.433114+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082128+0000","last_change":"2023-05-16T07:18:17.076423+0000","last_active":"2023-05-16T07:18:17.082128+0000","last_peered":"2023-05-16T07:18:17.082128+0000","last_clean":"2023-05-16T07:18:17.082128+0000","last_became_active":"2023-05-16T07:18:17.075266+0000","last_became_peered":"2023-05-16T07:18:17.075266+0000","last_unstale":"2023-05-16T07:18:17.082128+0000","last_undegraded":"2023-05-16T07:18:17.082128+0000","last_fullsized":"2023-05-16T07:18:17.082128+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445829+0000","last_change":"2023-05-16T07:18:17.445829+0000","last_active":"2023-05-16T07:18:17.445829+0000","last_peered":"2023-05-16T07:18:17.445829+0000","last_clean":"2023-05-16T07:18:17.445829+0000","last_became_active":"2023-05-16T07:18:17.445573+0000","last_became_peered":"2023-05-16T07:18:17.445573+0000","last_unstale":"2023-05-16T07:18:17.445829+0000","last_undegraded":"2023-05-16T07:18:17.445829+0000","last_fullsized":"2023-05-16T07:18:17.445829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.078239+0000","last_change":"2023-05-16T07:18:17.076072+0000","last_active":"2023-05-16T07:18:17.078239+0000","last_peered":"2023-05-16T07:18:17.078239+0000","last_clean":"2023-05-16T07:18:17.078239+0000","last_became_active":"2023-05-16T07:18:17.075802+0000","last_became_peered":"2023-05-16T07:18:17.075802+0000","last_unstale":"2023-05-16T07:18:17.078239+0000","last_undegraded":"2023-05-16T07:18:17.078239+0000","last_fullsized":"2023-05-16T07:18:17.078239+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448115+0000","last_change":"2023-05-16T07:18:17.448115+0000","last_active":"2023-05-16T07:18:17.448115+0000","last_peered":"2023-05-16T07:18:17.448115+0000","last_clean":"2023-05-16T07:18:17.448115+0000","last_became_active":"2023-05-16T07:18:17.447681+0000","last_became_peered":"2023-05-16T07:18:17.447681+0000","last_unstale":"2023-05-16T07:18:17.448115+0000","last_undegraded":"2023-05-16T07:18:17.448115+0000","last_fullsized":"2023-05-16T07:18:17.448115+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3c","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.450659+0000","last_change":"2023-05-16T07:18:17.082046+0000","last_active":"2023-05-16T07:18:17.450659+0000","last_peered":"2023-05-16T07:18:17.450659+0000","last_clean":"2023-05-16T07:18:17.450659+0000","last_became_active":"2023-05-16T07:18:17.081825+0000","last_became_peered":"2023-05-16T07:18:17.081825+0000","last_unstale":"2023-05-16T07:18:17.450659+0000","last_undegraded":"2023-05-16T07:18:17.450659+0000","last_fullsized":"2023-05-16T07:18:17.450659+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449706+0000","last_change":"2023-05-16T07:18:17.449706+0000","last_active":"2023-05-16T07:18:17.449706+0000","last_peered":"2023-05-16T07:18:17.449706+0000","last_clean":"2023-05-16T07:18:17.449706+0000","last_became_active":"2023-05-16T07:18:17.449409+0000","last_became_peered":"2023-05-16T07:18:17.449409+0000","last_unstale":"2023-05-16T07:18:17.449706+0000","last_undegraded":"2023-05-16T07:18:17.449706+0000","last_fullsized":"2023-05-16T07:18:17.449706+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3b","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.431110+0000","last_change":"2023-05-16T07:18:17.079457+0000","last_active":"2023-05-16T07:18:17.431110+0000","last_peered":"2023-05-16T07:18:17.431110+0000","last_clean":"2023-05-16T07:18:17.431110+0000","last_became_active":"2023-05-16T07:18:17.079187+0000","last_became_peered":"2023-05-16T07:18:17.079187+0000","last_unstale":"2023-05-16T07:18:17.431110+0000","last_undegraded":"2023-05-16T07:18:17.431110+0000","last_fullsized":"2023-05-16T07:18:17.431110+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.442170+0000","last_change":"2023-05-16T07:18:17.442170+0000","last_active":"2023-05-16T07:18:17.442170+0000","last_peered":"2023-05-16T07:18:17.442170+0000","last_clean":"2023-05-16T07:18:17.442170+0000","last_became_active":"2023-05-16T07:18:17.441943+0000","last_became_peered":"2023-05-16T07:18:17.441943+0000","last_unstale":"2023-05-16T07:18:17.442170+0000","last_undegraded":"2023-05-16T07:18:17.442170+0000","last_fullsized":"2023-05-16T07:18:17.442170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3a","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.439179+0000","last_change":"2023-05-16T07:18:17.072055+0000","last_active":"2023-05-16T07:18:17.439179+0000","last_peered":"2023-05-16T07:18:17.439179+0000","last_clean":"2023-05-16T07:18:17.439179+0000","last_became_active":"2023-05-16T07:18:17.070976+0000","last_became_peered":"2023-05-16T07:18:17.070976+0000","last_unstale":"2023-05-16T07:18:17.439179+0000","last_undegraded":"2023-05-16T07:18:17.439179+0000","last_fullsized":"2023-05-16T07:18:17.439179+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449864+0000","last_change":"2023-05-16T07:18:17.449864+0000","last_active":"2023-05-16T07:18:17.449864+0000","last_peered":"2023-05-16T07:18:17.449864+0000","last_clean":"2023-05-16T07:18:17.449864+0000","last_became_active":"2023-05-16T07:18:17.449465+0000","last_became_peered":"2023-05-16T07:18:17.449465+0000","last_unstale":"2023-05-16T07:18:17.449864+0000","last_undegraded":"2023-05-16T07:18:17.449864+0000","last_fullsized":"2023-05-16T07:18:17.449864+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083278+0000","last_change":"2023-05-16T07:18:17.075912+0000","last_active":"2023-05-16T07:18:17.083278+0000","last_peered":"2023-05-16T07:18:17.083278+0000","last_clean":"2023-05-16T07:18:17.083278+0000","last_became_active":"2023-05-16T07:18:17.075626+0000","last_became_peered":"2023-05-16T07:18:17.075626+0000","last_unstale":"2023-05-16T07:18:17.083278+0000","last_undegraded":"2023-05-16T07:18:17.083278+0000","last_fullsized":"2023-05-16T07:18:17.083278+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.434268+0000","last_change":"2023-05-16T07:18:17.434268+0000","last_active":"2023-05-16T07:18:17.434268+0000","last_peered":"2023-05-16T07:18:17.434268+0000","last_clean":"2023-05-16T07:18:17.434268+0000","last_became_active":"2023-05-16T07:18:17.433897+0000","last_became_peered":"2023-05-16T07:18:17.433897+0000","last_unstale":"2023-05-16T07:18:17.434268+0000","last_undegraded":"2023-05-16T07:18:17.434268+0000","last_fullsized":"2023-05-16T07:18:17.434268+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082364+0000","last_change":"2023-05-16T07:18:17.081786+0000","last_active":"2023-05-16T07:18:17.082364+0000","last_peered":"2023-05-16T07:18:17.082364+0000","last_clean":"2023-05-16T07:18:17.082364+0000","last_became_active":"2023-05-16T07:18:17.081481+0000","last_became_peered":"2023-05-16T07:18:17.081481+0000","last_unstale":"2023-05-16T07:18:17.082364+0000","last_undegraded":"2023-05-16T07:18:17.082364+0000","last_fullsized":"2023-05-16T07:18:17.082364+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.437966+0000","last_change":"2023-05-16T07:18:17.437966+0000","last_active":"2023-05-16T07:18:17.437966+0000","last_peered":"2023-05-16T07:18:17.437966+0000","last_clean":"2023-05-16T07:18:17.437966+0000","last_became_active":"2023-05-16T07:18:17.437656+0000","last_became_peered":"2023-05-16T07:18:17.437656+0000","last_unstale":"2023-05-16T07:18:17.437966+0000","last_undegraded":"2023-05-16T07:18:17.437966+0000","last_fullsized":"2023-05-16T07:18:17.437966+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073450+0000","last_change":"2023-05-16T07:18:17.070729+0000","last_active":"2023-05-16T07:18:17.073450+0000","last_peered":"2023-05-16T07:18:17.073450+0000","last_clean":"2023-05-16T07:18:17.073450+0000","last_became_active":"2023-05-16T07:18:17.070299+0000","last_became_peered":"2023-05-16T07:18:17.070299+0000","last_unstale":"2023-05-16T07:18:17.073450+0000","last_undegraded":"2023-05-16T07:18:17.073450+0000","last_fullsized":"2023-05-16T07:18:17.073450+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.451546+0000","last_change":"2023-05-16T07:18:17.451546+0000","last_active":"2023-05-16T07:18:17.451546+0000","last_peered":"2023-05-16T07:18:17.451546+0000","last_clean":"2023-05-16T07:18:17.451546+0000","last_became_active":"2023-05-16T07:18:17.451294+0000","last_became_peered":"2023-05-16T07:18:17.451294+0000","last_unstale":"2023-05-16T07:18:17.451546+0000","last_undegraded":"2023-05-16T07:18:17.451546+0000","last_fullsized":"2023-05-16T07:18:17.451546+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076953+0000","last_change":"2023-05-16T07:18:17.071331+0000","last_active":"2023-05-16T07:18:17.076953+0000","last_peered":"2023-05-16T07:18:17.076953+0000","last_clean":"2023-05-16T07:18:17.076953+0000","last_became_active":"2023-05-16T07:18:17.070948+0000","last_became_peered":"2023-05-16T07:18:17.070948+0000","last_unstale":"2023-05-16T07:18:17.076953+0000","last_undegraded":"2023-05-16T07:18:17.076953+0000","last_fullsized":"2023-05-16T07:18:17.076953+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445514+0000","last_change":"2023-05-16T07:18:17.445514+0000","last_active":"2023-05-16T07:18:17.445514+0000","last_peered":"2023-05-16T07:18:17.445514+0000","last_clean":"2023-05-16T07:18:17.445514+0000","last_became_active":"2023-05-16T07:18:17.441495+0000","last_became_peered":"2023-05-16T07:18:17.441495+0000","last_unstale":"2023-05-16T07:18:17.445514+0000","last_undegraded":"2023-05-16T07:18:17.445514+0000","last_fullsized":"2023-05-16T07:18:17.445514+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083045+0000","last_change":"2023-05-16T07:18:17.079104+0000","last_active":"2023-05-16T07:18:17.083045+0000","last_peered":"2023-05-16T07:18:17.083045+0000","last_clean":"2023-05-16T07:18:17.083045+0000","last_became_active":"2023-05-16T07:18:17.078833+0000","last_became_peered":"2023-05-16T07:18:17.078833+0000","last_unstale":"2023-05-16T07:18:17.083045+0000","last_undegraded":"2023-05-16T07:18:17.083045+0000","last_fullsized":"2023-05-16T07:18:17.083045+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444167+0000","last_change":"2023-05-16T07:18:17.444167+0000","last_active":"2023-05-16T07:18:17.444167+0000","last_peered":"2023-05-16T07:18:17.444167+0000","last_clean":"2023-05-16T07:18:17.444167+0000","last_became_active":"2023-05-16T07:18:17.443907+0000","last_became_peered":"2023-05-16T07:18:17.443907+0000","last_unstale":"2023-05-16T07:18:17.444167+0000","last_undegraded":"2023-05-16T07:18:17.444167+0000","last_fullsized":"2023-05-16T07:18:17.444167+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.34","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.450671+0000","last_change":"2023-05-16T07:18:17.074971+0000","last_active":"2023-05-16T07:18:17.450671+0000","last_peered":"2023-05-16T07:18:17.450671+0000","last_clean":"2023-05-16T07:18:17.450671+0000","last_became_active":"2023-05-16T07:18:17.074091+0000","last_became_peered":"2023-05-16T07:18:17.074091+0000","last_unstale":"2023-05-16T07:18:17.450671+0000","last_undegraded":"2023-05-16T07:18:17.450671+0000","last_fullsized":"2023-05-16T07:18:17.450671+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440344+0000","last_change":"2023-05-16T07:18:17.440344+0000","last_active":"2023-05-16T07:18:17.440344+0000","last_peered":"2023-05-16T07:18:17.440344+0000","last_clean":"2023-05-16T07:18:17.440344+0000","last_became_active":"2023-05-16T07:18:17.439931+0000","last_became_peered":"2023-05-16T07:18:17.439931+0000","last_unstale":"2023-05-16T07:18:17.440344+0000","last_undegraded":"2023-05-16T07:18:17.440344+0000","last_fullsized":"2023-05-16T07:18:17.440344+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.431266+0000","last_change":"2023-05-16T07:18:17.075712+0000","last_active":"2023-05-16T07:18:17.431266+0000","last_peered":"2023-05-16T07:18:17.431266+0000","last_clean":"2023-05-16T07:18:17.431266+0000","last_became_active":"2023-05-16T07:18:17.074991+0000","last_became_peered":"2023-05-16T07:18:17.074991+0000","last_unstale":"2023-05-16T07:18:17.431266+0000","last_undegraded":"2023-05-16T07:18:17.431266+0000","last_fullsized":"2023-05-16T07:18:17.431266+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441254+0000","last_change":"2023-05-16T07:18:17.441254+0000","last_active":"2023-05-16T07:18:17.441254+0000","last_peered":"2023-05-16T07:18:17.441254+0000","last_clean":"2023-05-16T07:18:17.441254+0000","last_became_active":"2023-05-16T07:18:17.440951+0000","last_became_peered":"2023-05-16T07:18:17.440951+0000","last_unstale":"2023-05-16T07:18:17.441254+0000","last_undegraded":"2023-05-16T07:18:17.441254+0000","last_fullsized":"2023-05-16T07:18:17.441254+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.085887+0000","last_change":"2023-05-16T07:18:17.082754+0000","last_active":"2023-05-16T07:18:17.085887+0000","last_peered":"2023-05-16T07:18:17.085887+0000","last_clean":"2023-05-16T07:18:17.085887+0000","last_became_active":"2023-05-16T07:18:17.082463+0000","last_became_peered":"2023-05-16T07:18:17.082463+0000","last_unstale":"2023-05-16T07:18:17.085887+0000","last_undegraded":"2023-05-16T07:18:17.085887+0000","last_fullsized":"2023-05-16T07:18:17.085887+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446101+0000","last_change":"2023-05-16T07:18:17.446101+0000","last_active":"2023-05-16T07:18:17.446101+0000","last_peered":"2023-05-16T07:18:17.446101+0000","last_clean":"2023-05-16T07:18:17.446101+0000","last_became_active":"2023-05-16T07:18:17.445777+0000","last_became_peered":"2023-05-16T07:18:17.445777+0000","last_unstale":"2023-05-16T07:18:17.446101+0000","last_undegraded":"2023-05-16T07:18:17.446101+0000","last_fullsized":"2023-05-16T07:18:17.446101+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077966+0000","last_change":"2023-05-16T07:18:17.074634+0000","last_active":"2023-05-16T07:18:17.077966+0000","last_peered":"2023-05-16T07:18:17.077966+0000","last_clean":"2023-05-16T07:18:17.077966+0000","last_became_active":"2023-05-16T07:18:17.074417+0000","last_became_peered":"2023-05-16T07:18:17.074417+0000","last_unstale":"2023-05-16T07:18:17.077966+0000","last_undegraded":"2023-05-16T07:18:17.077966+0000","last_fullsized":"2023-05-16T07:18:17.077966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449312+0000","last_change":"2023-05-16T07:18:17.449312+0000","last_active":"2023-05-16T07:18:17.449312+0000","last_peered":"2023-05-16T07:18:17.449312+0000","last_clean":"2023-05-16T07:18:17.449312+0000","last_became_active":"2023-05-16T07:18:17.449021+0000","last_became_peered":"2023-05-16T07:18:17.449021+0000","last_unstale":"2023-05-16T07:18:17.449312+0000","last_undegraded":"2023-05-16T07:18:17.449312+0000","last_fullsized":"2023-05-16T07:18:17.449312+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449262+0000","last_change":"2023-05-16T07:18:17.065614+0000","last_active":"2023-05-16T07:18:17.449262+0000","last_peered":"2023-05-16T07:18:17.449262+0000","last_clean":"2023-05-16T07:18:17.449262+0000","last_became_active":"2023-05-16T07:18:17.065308+0000","last_became_peered":"2023-05-16T07:18:17.065308+0000","last_unstale":"2023-05-16T07:18:17.449262+0000","last_undegraded":"2023-05-16T07:18:17.449262+0000","last_fullsized":"2023-05-16T07:18:17.449262+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440043+0000","last_change":"2023-05-16T07:18:17.440043+0000","last_active":"2023-05-16T07:18:17.440043+0000","last_peered":"2023-05-16T07:18:17.440043+0000","last_clean":"2023-05-16T07:18:17.440043+0000","last_became_active":"2023-05-16T07:18:17.439759+0000","last_became_peered":"2023-05-16T07:18:17.439759+0000","last_unstale":"2023-05-16T07:18:17.440043+0000","last_undegraded":"2023-05-16T07:18:17.440043+0000","last_fullsized":"2023-05-16T07:18:17.440043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.072974+0000","last_change":"2023-05-16T07:18:17.069028+0000","last_active":"2023-05-16T07:18:17.072974+0000","last_peered":"2023-05-16T07:18:17.072974+0000","last_clean":"2023-05-16T07:18:17.072974+0000","last_became_active":"2023-05-16T07:18:17.068704+0000","last_became_peered":"2023-05-16T07:18:17.068704+0000","last_unstale":"2023-05-16T07:18:17.072974+0000","last_undegraded":"2023-05-16T07:18:17.072974+0000","last_fullsized":"2023-05-16T07:18:17.072974+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.443753+0000","last_change":"2023-05-16T07:18:17.443753+0000","last_active":"2023-05-16T07:18:17.443753+0000","last_peered":"2023-05-16T07:18:17.443753+0000","last_clean":"2023-05-16T07:18:17.443753+0000","last_became_active":"2023-05-16T07:18:17.443368+0000","last_became_peered":"2023-05-16T07:18:17.443368+0000","last_unstale":"2023-05-16T07:18:17.443753+0000","last_undegraded":"2023-05-16T07:18:17.443753+0000","last_fullsized":"2023-05-16T07:18:17.443753+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.074803+0000","last_change":"2023-05-16T07:18:17.070554+0000","last_active":"2023-05-16T07:18:17.074803+0000","last_peered":"2023-05-16T07:18:17.074803+0000","last_clean":"2023-05-16T07:18:17.074803+0000","last_became_active":"2023-05-16T07:18:17.070216+0000","last_became_peered":"2023-05-16T07:18:17.070216+0000","last_unstale":"2023-05-16T07:18:17.074803+0000","last_undegraded":"2023-05-16T07:18:17.074803+0000","last_fullsized":"2023-05-16T07:18:17.074803+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.439496+0000","last_change":"2023-05-16T07:18:17.439496+0000","last_active":"2023-05-16T07:18:17.439496+0000","last_peered":"2023-05-16T07:18:17.439496+0000","last_clean":"2023-05-16T07:18:17.439496+0000","last_became_active":"2023-05-16T07:18:17.439319+0000","last_became_peered":"2023-05-16T07:18:17.439319+0000","last_unstale":"2023-05-16T07:18:17.439496+0000","last_undegraded":"2023-05-16T07:18:17.439496+0000","last_fullsized":"2023-05-16T07:18:17.439496+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.436779+0000","last_change":"2023-05-16T07:18:17.070160+0000","last_active":"2023-05-16T07:18:17.436779+0000","last_peered":"2023-05-16T07:18:17.436779+0000","last_clean":"2023-05-16T07:18:17.436779+0000","last_became_active":"2023-05-16T07:18:17.069862+0000","last_became_peered":"2023-05-16T07:18:17.069862+0000","last_unstale":"2023-05-16T07:18:17.436779+0000","last_undegraded":"2023-05-16T07:18:17.436779+0000","last_fullsized":"2023-05-16T07:18:17.436779+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447639+0000","last_change":"2023-05-16T07:18:17.447639+0000","last_active":"2023-05-16T07:18:17.447639+0000","last_peered":"2023-05-16T07:18:17.447639+0000","last_clean":"2023-05-16T07:18:17.447639+0000","last_became_active":"2023-05-16T07:18:17.447383+0000","last_became_peered":"2023-05-16T07:18:17.447383+0000","last_unstale":"2023-05-16T07:18:17.447639+0000","last_undegraded":"2023-05-16T07:18:17.447639+0000","last_fullsized":"2023-05-16T07:18:17.447639+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.070466+0000","last_change":"2023-05-16T07:18:17.068150+0000","last_active":"2023-05-16T07:18:17.070466+0000","last_peered":"2023-05-16T07:18:17.070466+0000","last_clean":"2023-05-16T07:18:17.070466+0000","last_became_active":"2023-05-16T07:18:17.067885+0000","last_became_peered":"2023-05-16T07:18:17.067885+0000","last_unstale":"2023-05-16T07:18:17.070466+0000","last_undegraded":"2023-05-16T07:18:17.070466+0000","last_fullsized":"2023-05-16T07:18:17.070466+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.433159+0000","last_change":"2023-05-16T07:18:17.433159+0000","last_active":"2023-05-16T07:18:17.433159+0000","last_peered":"2023-05-16T07:18:17.433159+0000","last_clean":"2023-05-16T07:18:17.433159+0000","last_became_active":"2023-05-16T07:18:17.432906+0000","last_became_peered":"2023-05-16T07:18:17.432906+0000","last_unstale":"2023-05-16T07:18:17.433159+0000","last_undegraded":"2023-05-16T07:18:17.433159+0000","last_fullsized":"2023-05-16T07:18:17.433159+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2b","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.433443+0000","last_change":"2023-05-16T07:18:17.073894+0000","last_active":"2023-05-16T07:18:17.433443+0000","last_peered":"2023-05-16T07:18:17.433443+0000","last_clean":"2023-05-16T07:18:17.433443+0000","last_became_active":"2023-05-16T07:18:17.073587+0000","last_became_peered":"2023-05-16T07:18:17.073587+0000","last_unstale":"2023-05-16T07:18:17.433443+0000","last_undegraded":"2023-05-16T07:18:17.433443+0000","last_fullsized":"2023-05-16T07:18:17.433443+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440803+0000","last_change":"2023-05-16T07:18:17.440803+0000","last_active":"2023-05-16T07:18:17.440803+0000","last_peered":"2023-05-16T07:18:17.440803+0000","last_clean":"2023-05-16T07:18:17.440803+0000","last_became_active":"2023-05-16T07:18:17.440339+0000","last_became_peered":"2023-05-16T07:18:17.440339+0000","last_unstale":"2023-05-16T07:18:17.440803+0000","last_undegraded":"2023-05-16T07:18:17.440803+0000","last_fullsized":"2023-05-16T07:18:17.440803+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083302+0000","last_change":"2023-05-16T07:18:17.079203+0000","last_active":"2023-05-16T07:18:17.083302+0000","last_peered":"2023-05-16T07:18:17.083302+0000","last_clean":"2023-05-16T07:18:17.083302+0000","last_became_active":"2023-05-16T07:18:17.078786+0000","last_became_peered":"2023-05-16T07:18:17.078786+0000","last_unstale":"2023-05-16T07:18:17.083302+0000","last_undegraded":"2023-05-16T07:18:17.083302+0000","last_fullsized":"2023-05-16T07:18:17.083302+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445963+0000","last_change":"2023-05-16T07:18:17.445963+0000","last_active":"2023-05-16T07:18:17.445963+0000","last_peered":"2023-05-16T07:18:17.445963+0000","last_clean":"2023-05-16T07:18:17.445963+0000","last_became_active":"2023-05-16T07:18:17.445678+0000","last_became_peered":"2023-05-16T07:18:17.445678+0000","last_unstale":"2023-05-16T07:18:17.445963+0000","last_undegraded":"2023-05-16T07:18:17.445963+0000","last_fullsized":"2023-05-16T07:18:17.445963+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.074938+0000","last_change":"2023-05-16T07:18:17.069343+0000","last_active":"2023-05-16T07:18:17.074938+0000","last_peered":"2023-05-16T07:18:17.074938+0000","last_clean":"2023-05-16T07:18:17.074938+0000","last_became_active":"2023-05-16T07:18:17.068386+0000","last_became_peered":"2023-05-16T07:18:17.068386+0000","last_unstale":"2023-05-16T07:18:17.074938+0000","last_undegraded":"2023-05-16T07:18:17.074938+0000","last_fullsized":"2023-05-16T07:18:17.074938+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.438135+0000","last_change":"2023-05-16T07:18:17.438135+0000","last_active":"2023-05-16T07:18:17.438135+0000","last_peered":"2023-05-16T07:18:17.438135+0000","last_clean":"2023-05-16T07:18:17.438135+0000","last_became_active":"2023-05-16T07:18:17.437725+0000","last_became_peered":"2023-05-16T07:18:17.437725+0000","last_unstale":"2023-05-16T07:18:17.438135+0000","last_undegraded":"2023-05-16T07:18:17.438135+0000","last_fullsized":"2023-05-16T07:18:17.438135+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.072948+0000","last_change":"2023-05-16T07:18:17.068861+0000","last_active":"2023-05-16T07:18:17.072948+0000","last_peered":"2023-05-16T07:18:17.072948+0000","last_clean":"2023-05-16T07:18:17.072948+0000","last_became_active":"2023-05-16T07:18:17.068442+0000","last_became_peered":"2023-05-16T07:18:17.068442+0000","last_unstale":"2023-05-16T07:18:17.072948+0000","last_undegraded":"2023-05-16T07:18:17.072948+0000","last_fullsized":"2023-05-16T07:18:17.072948+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445332+0000","last_change":"2023-05-16T07:18:17.445332+0000","last_active":"2023-05-16T07:18:17.445332+0000","last_peered":"2023-05-16T07:18:17.445332+0000","last_clean":"2023-05-16T07:18:17.445332+0000","last_became_active":"2023-05-16T07:18:17.445106+0000","last_became_peered":"2023-05-16T07:18:17.445106+0000","last_unstale":"2023-05-16T07:18:17.445332+0000","last_undegraded":"2023-05-16T07:18:17.445332+0000","last_fullsized":"2023-05-16T07:18:17.445332+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083039+0000","last_change":"2023-05-16T07:18:17.079895+0000","last_active":"2023-05-16T07:18:17.083039+0000","last_peered":"2023-05-16T07:18:17.083039+0000","last_clean":"2023-05-16T07:18:17.083039+0000","last_became_active":"2023-05-16T07:18:17.079651+0000","last_became_peered":"2023-05-16T07:18:17.079651+0000","last_unstale":"2023-05-16T07:18:17.083039+0000","last_undegraded":"2023-05-16T07:18:17.083039+0000","last_fullsized":"2023-05-16T07:18:17.083039+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.436610+0000","last_change":"2023-05-16T07:18:17.436610+0000","last_active":"2023-05-16T07:18:17.436610+0000","last_peered":"2023-05-16T07:18:17.436610+0000","last_clean":"2023-05-16T07:18:17.436610+0000","last_became_active":"2023-05-16T07:18:17.436332+0000","last_became_peered":"2023-05-16T07:18:17.436332+0000","last_unstale":"2023-05-16T07:18:17.436610+0000","last_undegraded":"2023-05-16T07:18:17.436610+0000","last_fullsized":"2023-05-16T07:18:17.436610+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080490+0000","last_change":"2023-05-16T07:18:17.079450+0000","last_active":"2023-05-16T07:18:17.080490+0000","last_peered":"2023-05-16T07:18:17.080490+0000","last_clean":"2023-05-16T07:18:17.080490+0000","last_became_active":"2023-05-16T07:18:17.079149+0000","last_became_peered":"2023-05-16T07:18:17.079149+0000","last_unstale":"2023-05-16T07:18:17.080490+0000","last_undegraded":"2023-05-16T07:18:17.080490+0000","last_fullsized":"2023-05-16T07:18:17.080490+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.443664+0000","last_change":"2023-05-16T07:18:17.443664+0000","last_active":"2023-05-16T07:18:17.443664+0000","last_peered":"2023-05-16T07:18:17.443664+0000","last_clean":"2023-05-16T07:18:17.443664+0000","last_became_active":"2023-05-16T07:18:17.443389+0000","last_became_peered":"2023-05-16T07:18:17.443389+0000","last_unstale":"2023-05-16T07:18:17.443664+0000","last_undegraded":"2023-05-16T07:18:17.443664+0000","last_fullsized":"2023-05-16T07:18:17.443664+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073645+0000","last_change":"2023-05-16T07:18:17.071912+0000","last_active":"2023-05-16T07:18:17.073645+0000","last_peered":"2023-05-16T07:18:17.073645+0000","last_clean":"2023-05-16T07:18:17.073645+0000","last_became_active":"2023-05-16T07:18:17.071611+0000","last_became_peered":"2023-05-16T07:18:17.071611+0000","last_unstale":"2023-05-16T07:18:17.073645+0000","last_undegraded":"2023-05-16T07:18:17.073645+0000","last_fullsized":"2023-05-16T07:18:17.073645+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446935+0000","last_change":"2023-05-16T07:18:17.446935+0000","last_active":"2023-05-16T07:18:17.446935+0000","last_peered":"2023-05-16T07:18:17.446935+0000","last_clean":"2023-05-16T07:18:17.446935+0000","last_became_active":"2023-05-16T07:18:17.446070+0000","last_became_peered":"2023-05-16T07:18:17.446070+0000","last_unstale":"2023-05-16T07:18:17.446935+0000","last_undegraded":"2023-05-16T07:18:17.446935+0000","last_fullsized":"2023-05-16T07:18:17.446935+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077133+0000","last_change":"2023-05-16T07:18:17.073490+0000","last_active":"2023-05-16T07:18:17.077133+0000","last_peered":"2023-05-16T07:18:17.077133+0000","last_clean":"2023-05-16T07:18:17.077133+0000","last_became_active":"2023-05-16T07:18:17.072900+0000","last_became_peered":"2023-05-16T07:18:17.072900+0000","last_unstale":"2023-05-16T07:18:17.077133+0000","last_undegraded":"2023-05-16T07:18:17.077133+0000","last_fullsized":"2023-05-16T07:18:17.077133+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440186+0000","last_change":"2023-05-16T07:18:17.440186+0000","last_active":"2023-05-16T07:18:17.440186+0000","last_peered":"2023-05-16T07:18:17.440186+0000","last_clean":"2023-05-16T07:18:17.440186+0000","last_became_active":"2023-05-16T07:18:17.439583+0000","last_became_peered":"2023-05-16T07:18:17.439583+0000","last_unstale":"2023-05-16T07:18:17.440186+0000","last_undegraded":"2023-05-16T07:18:17.440186+0000","last_fullsized":"2023-05-16T07:18:17.440186+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083444+0000","last_change":"2023-05-16T07:18:17.080943+0000","last_active":"2023-05-16T07:18:17.083444+0000","last_peered":"2023-05-16T07:18:17.083444+0000","last_clean":"2023-05-16T07:18:17.083444+0000","last_became_active":"2023-05-16T07:18:17.080691+0000","last_became_peered":"2023-05-16T07:18:17.080691+0000","last_unstale":"2023-05-16T07:18:17.083444+0000","last_undegraded":"2023-05-16T07:18:17.083444+0000","last_fullsized":"2023-05-16T07:18:17.083444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445398+0000","last_change":"2023-05-16T07:18:17.445398+0000","last_active":"2023-05-16T07:18:17.445398+0000","last_peered":"2023-05-16T07:18:17.445398+0000","last_clean":"2023-05-16T07:18:17.445398+0000","last_became_active":"2023-05-16T07:18:17.445117+0000","last_became_peered":"2023-05-16T07:18:17.445117+0000","last_unstale":"2023-05-16T07:18:17.445398+0000","last_undegraded":"2023-05-16T07:18:17.445398+0000","last_fullsized":"2023-05-16T07:18:17.445398+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083319+0000","last_change":"2023-05-16T07:18:17.079286+0000","last_active":"2023-05-16T07:18:17.083319+0000","last_peered":"2023-05-16T07:18:17.083319+0000","last_clean":"2023-05-16T07:18:17.083319+0000","last_became_active":"2023-05-16T07:18:17.079026+0000","last_became_peered":"2023-05-16T07:18:17.079026+0000","last_unstale":"2023-05-16T07:18:17.083319+0000","last_undegraded":"2023-05-16T07:18:17.083319+0000","last_fullsized":"2023-05-16T07:18:17.083319+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447130+0000","last_change":"2023-05-16T07:18:17.447130+0000","last_active":"2023-05-16T07:18:17.447130+0000","last_peered":"2023-05-16T07:18:17.447130+0000","last_clean":"2023-05-16T07:18:17.447130+0000","last_became_active":"2023-05-16T07:18:17.446863+0000","last_became_peered":"2023-05-16T07:18:17.446863+0000","last_unstale":"2023-05-16T07:18:17.447130+0000","last_undegraded":"2023-05-16T07:18:17.447130+0000","last_fullsized":"2023-05-16T07:18:17.447130+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073021+0000","last_change":"2023-05-16T07:18:17.069695+0000","last_active":"2023-05-16T07:18:17.073021+0000","last_peered":"2023-05-16T07:18:17.073021+0000","last_clean":"2023-05-16T07:18:17.073021+0000","last_became_active":"2023-05-16T07:18:17.069412+0000","last_became_peered":"2023-05-16T07:18:17.069412+0000","last_unstale":"2023-05-16T07:18:17.073021+0000","last_undegraded":"2023-05-16T07:18:17.073021+0000","last_fullsized":"2023-05-16T07:18:17.073021+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.437779+0000","last_change":"2023-05-16T07:18:17.437779+0000","last_active":"2023-05-16T07:18:17.437779+0000","last_peered":"2023-05-16T07:18:17.437779+0000","last_clean":"2023-05-16T07:18:17.437779+0000","last_became_active":"2023-05-16T07:18:17.437483+0000","last_became_peered":"2023-05-16T07:18:17.437483+0000","last_unstale":"2023-05-16T07:18:17.437779+0000","last_undegraded":"2023-05-16T07:18:17.437779+0000","last_fullsized":"2023-05-16T07:18:17.437779+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080411+0000","last_change":"2023-05-16T07:18:17.079735+0000","last_active":"2023-05-16T07:18:17.080411+0000","last_peered":"2023-05-16T07:18:17.080411+0000","last_clean":"2023-05-16T07:18:17.080411+0000","last_became_active":"2023-05-16T07:18:17.079437+0000","last_became_peered":"2023-05-16T07:18:17.079437+0000","last_unstale":"2023-05-16T07:18:17.080411+0000","last_undegraded":"2023-05-16T07:18:17.080411+0000","last_fullsized":"2023-05-16T07:18:17.080411+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448903+0000","last_change":"2023-05-16T07:18:17.448903+0000","last_active":"2023-05-16T07:18:17.448903+0000","last_peered":"2023-05-16T07:18:17.448903+0000","last_clean":"2023-05-16T07:18:17.448903+0000","last_became_active":"2023-05-16T07:18:17.448649+0000","last_became_peered":"2023-05-16T07:18:17.448649+0000","last_unstale":"2023-05-16T07:18:17.448903+0000","last_undegraded":"2023-05-16T07:18:17.448903+0000","last_fullsized":"2023-05-16T07:18:17.448903+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.428980+0000","last_change":"2023-05-16T07:18:17.069782+0000","last_active":"2023-05-16T07:18:17.428980+0000","last_peered":"2023-05-16T07:18:17.428980+0000","last_clean":"2023-05-16T07:18:17.428980+0000","last_became_active":"2023-05-16T07:18:17.069495+0000","last_became_peered":"2023-05-16T07:18:17.069495+0000","last_unstale":"2023-05-16T07:18:17.428980+0000","last_undegraded":"2023-05-16T07:18:17.428980+0000","last_fullsized":"2023-05-16T07:18:17.428980+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446676+0000","last_change":"2023-05-16T07:18:17.446676+0000","last_active":"2023-05-16T07:18:17.446676+0000","last_peered":"2023-05-16T07:18:17.446676+0000","last_clean":"2023-05-16T07:18:17.446676+0000","last_became_active":"2023-05-16T07:18:17.446399+0000","last_became_peered":"2023-05-16T07:18:17.446399+0000","last_unstale":"2023-05-16T07:18:17.446676+0000","last_undegraded":"2023-05-16T07:18:17.446676+0000","last_fullsized":"2023-05-16T07:18:17.446676+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.429012+0000","last_change":"2023-05-16T07:18:17.075494+0000","last_active":"2023-05-16T07:18:17.429012+0000","last_peered":"2023-05-16T07:18:17.429012+0000","last_clean":"2023-05-16T07:18:17.429012+0000","last_became_active":"2023-05-16T07:18:17.075186+0000","last_became_peered":"2023-05-16T07:18:17.075186+0000","last_unstale":"2023-05-16T07:18:17.429012+0000","last_undegraded":"2023-05-16T07:18:17.429012+0000","last_fullsized":"2023-05-16T07:18:17.429012+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441741+0000","last_change":"2023-05-16T07:18:17.441741+0000","last_active":"2023-05-16T07:18:17.441741+0000","last_peered":"2023-05-16T07:18:17.441741+0000","last_clean":"2023-05-16T07:18:17.441741+0000","last_became_active":"2023-05-16T07:18:17.441499+0000","last_became_peered":"2023-05-16T07:18:17.441499+0000","last_unstale":"2023-05-16T07:18:17.441741+0000","last_undegraded":"2023-05-16T07:18:17.441741+0000","last_fullsized":"2023-05-16T07:18:17.441741+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.450222+0000","last_change":"2023-05-16T07:18:17.074477+0000","last_active":"2023-05-16T07:18:17.450222+0000","last_peered":"2023-05-16T07:18:17.450222+0000","last_clean":"2023-05-16T07:18:17.450222+0000","last_became_active":"2023-05-16T07:18:17.074197+0000","last_became_peered":"2023-05-16T07:18:17.074197+0000","last_unstale":"2023-05-16T07:18:17.450222+0000","last_undegraded":"2023-05-16T07:18:17.450222+0000","last_fullsized":"2023-05-16T07:18:17.450222+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447125+0000","last_change":"2023-05-16T07:18:17.447125+0000","last_active":"2023-05-16T07:18:17.447125+0000","last_peered":"2023-05-16T07:18:17.447125+0000","last_clean":"2023-05-16T07:18:17.447125+0000","last_became_active":"2023-05-16T07:18:17.446348+0000","last_became_peered":"2023-05-16T07:18:17.446348+0000","last_unstale":"2023-05-16T07:18:17.447125+0000","last_undegraded":"2023-05-16T07:18:17.447125+0000","last_fullsized":"2023-05-16T07:18:17.447125+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.079927+0000","last_change":"2023-05-16T07:18:17.077674+0000","last_active":"2023-05-16T07:18:17.079927+0000","last_peered":"2023-05-16T07:18:17.079927+0000","last_clean":"2023-05-16T07:18:17.079927+0000","last_became_active":"2023-05-16T07:18:17.077366+0000","last_became_peered":"2023-05-16T07:18:17.077366+0000","last_unstale":"2023-05-16T07:18:17.079927+0000","last_undegraded":"2023-05-16T07:18:17.079927+0000","last_fullsized":"2023-05-16T07:18:17.079927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440306+0000","last_change":"2023-05-16T07:18:17.440306+0000","last_active":"2023-05-16T07:18:17.440306+0000","last_peered":"2023-05-16T07:18:17.440306+0000","last_clean":"2023-05-16T07:18:17.440306+0000","last_became_active":"2023-05-16T07:18:17.440054+0000","last_became_peered":"2023-05-16T07:18:17.440054+0000","last_unstale":"2023-05-16T07:18:17.440306+0000","last_undegraded":"2023-05-16T07:18:17.440306+0000","last_fullsized":"2023-05-16T07:18:17.440306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073649+0000","last_change":"2023-05-16T07:18:17.071694+0000","last_active":"2023-05-16T07:18:17.073649+0000","last_peered":"2023-05-16T07:18:17.073649+0000","last_clean":"2023-05-16T07:18:17.073649+0000","last_became_active":"2023-05-16T07:18:17.071461+0000","last_became_peered":"2023-05-16T07:18:17.071461+0000","last_unstale":"2023-05-16T07:18:17.073649+0000","last_undegraded":"2023-05-16T07:18:17.073649+0000","last_fullsized":"2023-05-16T07:18:17.073649+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440621+0000","last_change":"2023-05-16T07:18:17.440621+0000","last_active":"2023-05-16T07:18:17.440621+0000","last_peered":"2023-05-16T07:18:17.440621+0000","last_clean":"2023-05-16T07:18:17.440621+0000","last_became_active":"2023-05-16T07:18:17.439959+0000","last_became_peered":"2023-05-16T07:18:17.439959+0000","last_unstale":"2023-05-16T07:18:17.440621+0000","last_undegraded":"2023-05-16T07:18:17.440621+0000","last_fullsized":"2023-05-16T07:18:17.440621+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.083282+0000","last_change":"2023-05-16T07:18:17.079082+0000","last_active":"2023-05-16T07:18:17.083282+0000","last_peered":"2023-05-16T07:18:17.083282+0000","last_clean":"2023-05-16T07:18:17.083282+0000","last_became_active":"2023-05-16T07:18:17.078439+0000","last_became_peered":"2023-05-16T07:18:17.078439+0000","last_unstale":"2023-05-16T07:18:17.083282+0000","last_undegraded":"2023-05-16T07:18:17.083282+0000","last_fullsized":"2023-05-16T07:18:17.083282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449702+0000","last_change":"2023-05-16T07:18:17.449702+0000","last_active":"2023-05-16T07:18:17.449702+0000","last_peered":"2023-05-16T07:18:17.449702+0000","last_clean":"2023-05-16T07:18:17.449702+0000","last_became_active":"2023-05-16T07:18:17.449141+0000","last_became_peered":"2023-05-16T07:18:17.449141+0000","last_unstale":"2023-05-16T07:18:17.449702+0000","last_undegraded":"2023-05-16T07:18:17.449702+0000","last_fullsized":"2023-05-16T07:18:17.449702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077985+0000","last_change":"2023-05-16T07:18:17.075129+0000","last_active":"2023-05-16T07:18:17.077985+0000","last_peered":"2023-05-16T07:18:17.077985+0000","last_clean":"2023-05-16T07:18:17.077985+0000","last_became_active":"2023-05-16T07:18:17.074646+0000","last_became_peered":"2023-05-16T07:18:17.074646+0000","last_unstale":"2023-05-16T07:18:17.077985+0000","last_undegraded":"2023-05-16T07:18:17.077985+0000","last_fullsized":"2023-05-16T07:18:17.077985+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440935+0000","last_change":"2023-05-16T07:18:17.440935+0000","last_active":"2023-05-16T07:18:17.440935+0000","last_peered":"2023-05-16T07:18:17.440935+0000","last_clean":"2023-05-16T07:18:17.440935+0000","last_became_active":"2023-05-16T07:18:17.440668+0000","last_became_peered":"2023-05-16T07:18:17.440668+0000","last_unstale":"2023-05-16T07:18:17.440935+0000","last_undegraded":"2023-05-16T07:18:17.440935+0000","last_fullsized":"2023-05-16T07:18:17.440935+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080445+0000","last_change":"2023-05-16T07:18:17.078439+0000","last_active":"2023-05-16T07:18:17.080445+0000","last_peered":"2023-05-16T07:18:17.080445+0000","last_clean":"2023-05-16T07:18:17.080445+0000","last_became_active":"2023-05-16T07:18:17.078154+0000","last_became_peered":"2023-05-16T07:18:17.078154+0000","last_unstale":"2023-05-16T07:18:17.080445+0000","last_undegraded":"2023-05-16T07:18:17.080445+0000","last_fullsized":"2023-05-16T07:18:17.080445+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.441562+0000","last_change":"2023-05-16T07:18:17.441562+0000","last_active":"2023-05-16T07:18:17.441562+0000","last_peered":"2023-05-16T07:18:17.441562+0000","last_clean":"2023-05-16T07:18:17.441562+0000","last_became_active":"2023-05-16T07:18:17.441280+0000","last_became_peered":"2023-05-16T07:18:17.441280+0000","last_unstale":"2023-05-16T07:18:17.441562+0000","last_undegraded":"2023-05-16T07:18:17.441562+0000","last_fullsized":"2023-05-16T07:18:17.441562+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073469+0000","last_change":"2023-05-16T07:18:17.071277+0000","last_active":"2023-05-16T07:18:17.073469+0000","last_peered":"2023-05-16T07:18:17.073469+0000","last_clean":"2023-05-16T07:18:17.073469+0000","last_became_active":"2023-05-16T07:18:17.070993+0000","last_became_peered":"2023-05-16T07:18:17.070993+0000","last_unstale":"2023-05-16T07:18:17.073469+0000","last_undegraded":"2023-05-16T07:18:17.073469+0000","last_fullsized":"2023-05-16T07:18:17.073469+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.437669+0000","last_change":"2023-05-16T07:18:17.437669+0000","last_active":"2023-05-16T07:18:17.437669+0000","last_peered":"2023-05-16T07:18:17.437669+0000","last_clean":"2023-05-16T07:18:17.437669+0000","last_became_active":"2023-05-16T07:18:17.437387+0000","last_became_peered":"2023-05-16T07:18:17.437387+0000","last_unstale":"2023-05-16T07:18:17.437669+0000","last_undegraded":"2023-05-16T07:18:17.437669+0000","last_fullsized":"2023-05-16T07:18:17.437669+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080390+0000","last_change":"2023-05-16T07:18:17.078210+0000","last_active":"2023-05-16T07:18:17.080390+0000","last_peered":"2023-05-16T07:18:17.080390+0000","last_clean":"2023-05-16T07:18:17.080390+0000","last_became_active":"2023-05-16T07:18:17.077324+0000","last_became_peered":"2023-05-16T07:18:17.077324+0000","last_unstale":"2023-05-16T07:18:17.080390+0000","last_undegraded":"2023-05-16T07:18:17.080390+0000","last_fullsized":"2023-05-16T07:18:17.080390+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447448+0000","last_change":"2023-05-16T07:18:17.447448+0000","last_active":"2023-05-16T07:18:17.447448+0000","last_peered":"2023-05-16T07:18:17.447448+0000","last_clean":"2023-05-16T07:18:17.447448+0000","last_became_active":"2023-05-16T07:18:17.447120+0000","last_became_peered":"2023-05-16T07:18:17.447120+0000","last_unstale":"2023-05-16T07:18:17.447448+0000","last_undegraded":"2023-05-16T07:18:17.447448+0000","last_fullsized":"2023-05-16T07:18:17.447448+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"106'3","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.451251+0000","last_change":"2023-05-16T07:18:17.078005+0000","last_active":"2023-05-16T07:18:17.451251+0000","last_peered":"2023-05-16T07:18:17.451251+0000","last_clean":"2023-05-16T07:18:17.451251+0000","last_became_active":"2023-05-16T07:18:17.077298+0000","last_became_peered":"2023-05-16T07:18:17.077298+0000","last_unstale":"2023-05-16T07:18:17.451251+0000","last_undegraded":"2023-05-16T07:18:17.451251+0000","last_fullsized":"2023-05-16T07:18:17.451251+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448543+0000","last_change":"2023-05-16T07:18:17.448543+0000","last_active":"2023-05-16T07:18:17.448543+0000","last_peered":"2023-05-16T07:18:17.448543+0000","last_clean":"2023-05-16T07:18:17.448543+0000","last_became_active":"2023-05-16T07:18:17.448270+0000","last_became_peered":"2023-05-16T07:18:17.448270+0000","last_unstale":"2023-05-16T07:18:17.448543+0000","last_undegraded":"2023-05-16T07:18:17.448543+0000","last_fullsized":"2023-05-16T07:18:17.448543+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446783+0000","last_change":"2023-05-16T07:18:17.070416+0000","last_active":"2023-05-16T07:18:17.446783+0000","last_peered":"2023-05-16T07:18:17.446783+0000","last_clean":"2023-05-16T07:18:17.446783+0000","last_became_active":"2023-05-16T07:18:17.069703+0000","last_became_peered":"2023-05-16T07:18:17.069703+0000","last_unstale":"2023-05-16T07:18:17.446783+0000","last_undegraded":"2023-05-16T07:18:17.446783+0000","last_fullsized":"2023-05-16T07:18:17.446783+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.449318+0000","last_change":"2023-05-16T07:18:17.449318+0000","last_active":"2023-05-16T07:18:17.449318+0000","last_peered":"2023-05-16T07:18:17.449318+0000","last_clean":"2023-05-16T07:18:17.449318+0000","last_became_active":"2023-05-16T07:18:17.448949+0000","last_became_peered":"2023-05-16T07:18:17.448949+0000","last_unstale":"2023-05-16T07:18:17.449318+0000","last_undegraded":"2023-05-16T07:18:17.449318+0000","last_fullsized":"2023-05-16T07:18:17.449318+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.431385+0000","last_change":"2023-05-16T07:18:17.077857+0000","last_active":"2023-05-16T07:18:17.431385+0000","last_peered":"2023-05-16T07:18:17.431385+0000","last_clean":"2023-05-16T07:18:17.431385+0000","last_became_active":"2023-05-16T07:18:17.077540+0000","last_became_peered":"2023-05-16T07:18:17.077540+0000","last_unstale":"2023-05-16T07:18:17.431385+0000","last_undegraded":"2023-05-16T07:18:17.431385+0000","last_fullsized":"2023-05-16T07:18:17.431385+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.434362+0000","last_change":"2023-05-16T07:18:17.434362+0000","last_active":"2023-05-16T07:18:17.434362+0000","last_peered":"2023-05-16T07:18:17.434362+0000","last_clean":"2023-05-16T07:18:17.434362+0000","last_became_active":"2023-05-16T07:18:17.433890+0000","last_became_peered":"2023-05-16T07:18:17.433890+0000","last_unstale":"2023-05-16T07:18:17.434362+0000","last_undegraded":"2023-05-16T07:18:17.434362+0000","last_fullsized":"2023-05-16T07:18:17.434362+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076801+0000","last_change":"2023-05-16T07:18:17.071296+0000","last_active":"2023-05-16T07:18:17.076801+0000","last_peered":"2023-05-16T07:18:17.076801+0000","last_clean":"2023-05-16T07:18:17.076801+0000","last_became_active":"2023-05-16T07:18:17.071047+0000","last_became_peered":"2023-05-16T07:18:17.071047+0000","last_unstale":"2023-05-16T07:18:17.076801+0000","last_undegraded":"2023-05-16T07:18:17.076801+0000","last_fullsized":"2023-05-16T07:18:17.076801+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446038+0000","last_change":"2023-05-16T07:18:17.446038+0000","last_active":"2023-05-16T07:18:17.446038+0000","last_peered":"2023-05-16T07:18:17.446038+0000","last_clean":"2023-05-16T07:18:17.446038+0000","last_became_active":"2023-05-16T07:18:17.445783+0000","last_became_peered":"2023-05-16T07:18:17.445783+0000","last_unstale":"2023-05-16T07:18:17.446038+0000","last_undegraded":"2023-05-16T07:18:17.446038+0000","last_fullsized":"2023-05-16T07:18:17.446038+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082875+0000","last_change":"2023-05-16T07:18:17.077641+0000","last_active":"2023-05-16T07:18:17.082875+0000","last_peered":"2023-05-16T07:18:17.082875+0000","last_clean":"2023-05-16T07:18:17.082875+0000","last_became_active":"2023-05-16T07:18:17.077383+0000","last_became_peered":"2023-05-16T07:18:17.077383+0000","last_unstale":"2023-05-16T07:18:17.082875+0000","last_undegraded":"2023-05-16T07:18:17.082875+0000","last_fullsized":"2023-05-16T07:18:17.082875+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.445432+0000","last_change":"2023-05-16T07:18:17.445432+0000","last_active":"2023-05-16T07:18:17.445432+0000","last_peered":"2023-05-16T07:18:17.445432+0000","last_clean":"2023-05-16T07:18:17.445432+0000","last_became_active":"2023-05-16T07:18:17.445208+0000","last_became_peered":"2023-05-16T07:18:17.445208+0000","last_unstale":"2023-05-16T07:18:17.445432+0000","last_undegraded":"2023-05-16T07:18:17.445432+0000","last_fullsized":"2023-05-16T07:18:17.445432+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.451663+0000","last_change":"2023-05-16T07:18:17.077534+0000","last_active":"2023-05-16T07:18:17.451663+0000","last_peered":"2023-05-16T07:18:17.451663+0000","last_clean":"2023-05-16T07:18:17.451663+0000","last_became_active":"2023-05-16T07:18:17.077247+0000","last_became_peered":"2023-05-16T07:18:17.077247+0000","last_unstale":"2023-05-16T07:18:17.451663+0000","last_undegraded":"2023-05-16T07:18:17.451663+0000","last_fullsized":"2023-05-16T07:18:17.451663+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.434819+0000","last_change":"2023-05-16T07:18:17.434819+0000","last_active":"2023-05-16T07:18:17.434819+0000","last_peered":"2023-05-16T07:18:17.434819+0000","last_clean":"2023-05-16T07:18:17.434819+0000","last_became_active":"2023-05-16T07:18:17.434534+0000","last_became_peered":"2023-05-16T07:18:17.434534+0000","last_unstale":"2023-05-16T07:18:17.434819+0000","last_undegraded":"2023-05-16T07:18:17.434819+0000","last_fullsized":"2023-05-16T07:18:17.434819+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448921+0000","last_change":"2023-05-16T07:18:17.074502+0000","last_active":"2023-05-16T07:18:17.448921+0000","last_peered":"2023-05-16T07:18:17.448921+0000","last_clean":"2023-05-16T07:18:17.448921+0000","last_became_active":"2023-05-16T07:18:17.074176+0000","last_became_peered":"2023-05-16T07:18:17.074176+0000","last_unstale":"2023-05-16T07:18:17.448921+0000","last_undegraded":"2023-05-16T07:18:17.448921+0000","last_fullsized":"2023-05-16T07:18:17.448921+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448702+0000","last_change":"2023-05-16T07:18:17.448702+0000","last_active":"2023-05-16T07:18:17.448702+0000","last_peered":"2023-05-16T07:18:17.448702+0000","last_clean":"2023-05-16T07:18:17.448702+0000","last_became_active":"2023-05-16T07:18:17.448435+0000","last_became_peered":"2023-05-16T07:18:17.448435+0000","last_unstale":"2023-05-16T07:18:17.448702+0000","last_undegraded":"2023-05-16T07:18:17.448702+0000","last_fullsized":"2023-05-16T07:18:17.448702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.067294+0000","last_change":"2023-05-16T07:18:17.065247+0000","last_active":"2023-05-16T07:18:17.067294+0000","last_peered":"2023-05-16T07:18:17.067294+0000","last_clean":"2023-05-16T07:18:17.067294+0000","last_became_active":"2023-05-16T07:18:17.064966+0000","last_became_peered":"2023-05-16T07:18:17.064966+0000","last_unstale":"2023-05-16T07:18:17.067294+0000","last_undegraded":"2023-05-16T07:18:17.067294+0000","last_fullsized":"2023-05-16T07:18:17.067294+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444722+0000","last_change":"2023-05-16T07:18:17.444722+0000","last_active":"2023-05-16T07:18:17.444722+0000","last_peered":"2023-05-16T07:18:17.444722+0000","last_clean":"2023-05-16T07:18:17.444722+0000","last_became_active":"2023-05-16T07:18:17.444477+0000","last_became_peered":"2023-05-16T07:18:17.444477+0000","last_unstale":"2023-05-16T07:18:17.444722+0000","last_undegraded":"2023-05-16T07:18:17.444722+0000","last_fullsized":"2023-05-16T07:18:17.444722+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.074975+0000","last_change":"2023-05-16T07:18:17.072555+0000","last_active":"2023-05-16T07:18:17.074975+0000","last_peered":"2023-05-16T07:18:17.074975+0000","last_clean":"2023-05-16T07:18:17.074975+0000","last_became_active":"2023-05-16T07:18:17.072276+0000","last_became_peered":"2023-05-16T07:18:17.072276+0000","last_unstale":"2023-05-16T07:18:17.074975+0000","last_undegraded":"2023-05-16T07:18:17.074975+0000","last_fullsized":"2023-05-16T07:18:17.074975+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448772+0000","last_change":"2023-05-16T07:18:17.448772+0000","last_active":"2023-05-16T07:18:17.448772+0000","last_peered":"2023-05-16T07:18:17.448772+0000","last_clean":"2023-05-16T07:18:17.448772+0000","last_became_active":"2023-05-16T07:18:17.446518+0000","last_became_peered":"2023-05-16T07:18:17.446518+0000","last_unstale":"2023-05-16T07:18:17.448772+0000","last_undegraded":"2023-05-16T07:18:17.448772+0000","last_fullsized":"2023-05-16T07:18:17.448772+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.077152+0000","last_change":"2023-05-16T07:18:17.073523+0000","last_active":"2023-05-16T07:18:17.077152+0000","last_peered":"2023-05-16T07:18:17.077152+0000","last_clean":"2023-05-16T07:18:17.077152+0000","last_became_active":"2023-05-16T07:18:17.073223+0000","last_became_peered":"2023-05-16T07:18:17.073223+0000","last_unstale":"2023-05-16T07:18:17.077152+0000","last_undegraded":"2023-05-16T07:18:17.077152+0000","last_fullsized":"2023-05-16T07:18:17.077152+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440599+0000","last_change":"2023-05-16T07:18:17.440599+0000","last_active":"2023-05-16T07:18:17.440599+0000","last_peered":"2023-05-16T07:18:17.440599+0000","last_clean":"2023-05-16T07:18:17.440599+0000","last_became_active":"2023-05-16T07:18:17.437506+0000","last_became_peered":"2023-05-16T07:18:17.437506+0000","last_unstale":"2023-05-16T07:18:17.440599+0000","last_undegraded":"2023-05-16T07:18:17.440599+0000","last_fullsized":"2023-05-16T07:18:17.440599+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076845+0000","last_change":"2023-05-16T07:18:17.071698+0000","last_active":"2023-05-16T07:18:17.076845+0000","last_peered":"2023-05-16T07:18:17.076845+0000","last_clean":"2023-05-16T07:18:17.076845+0000","last_became_active":"2023-05-16T07:18:17.071022+0000","last_became_peered":"2023-05-16T07:18:17.071022+0000","last_unstale":"2023-05-16T07:18:17.076845+0000","last_undegraded":"2023-05-16T07:18:17.076845+0000","last_fullsized":"2023-05-16T07:18:17.076845+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440633+0000","last_change":"2023-05-16T07:18:17.440633+0000","last_active":"2023-05-16T07:18:17.440633+0000","last_peered":"2023-05-16T07:18:17.440633+0000","last_clean":"2023-05-16T07:18:17.440633+0000","last_became_active":"2023-05-16T07:18:17.440374+0000","last_became_peered":"2023-05-16T07:18:17.440374+0000","last_unstale":"2023-05-16T07:18:17.440633+0000","last_undegraded":"2023-05-16T07:18:17.440633+0000","last_fullsized":"2023-05-16T07:18:17.440633+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.075290+0000","last_change":"2023-05-16T07:18:17.072234+0000","last_active":"2023-05-16T07:18:17.075290+0000","last_peered":"2023-05-16T07:18:17.075290+0000","last_clean":"2023-05-16T07:18:17.075290+0000","last_became_active":"2023-05-16T07:18:17.071749+0000","last_became_peered":"2023-05-16T07:18:17.071749+0000","last_unstale":"2023-05-16T07:18:17.075290+0000","last_undegraded":"2023-05-16T07:18:17.075290+0000","last_fullsized":"2023-05-16T07:18:17.075290+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444506+0000","last_change":"2023-05-16T07:18:17.444506+0000","last_active":"2023-05-16T07:18:17.444506+0000","last_peered":"2023-05-16T07:18:17.444506+0000","last_clean":"2023-05-16T07:18:17.444506+0000","last_became_active":"2023-05-16T07:18:17.444274+0000","last_became_peered":"2023-05-16T07:18:17.444274+0000","last_unstale":"2023-05-16T07:18:17.444506+0000","last_undegraded":"2023-05-16T07:18:17.444506+0000","last_fullsized":"2023-05-16T07:18:17.444506+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076862+0000","last_change":"2023-05-16T07:18:17.070734+0000","last_active":"2023-05-16T07:18:17.076862+0000","last_peered":"2023-05-16T07:18:17.076862+0000","last_clean":"2023-05-16T07:18:17.076862+0000","last_became_active":"2023-05-16T07:18:17.069874+0000","last_became_peered":"2023-05-16T07:18:17.069874+0000","last_unstale":"2023-05-16T07:18:17.076862+0000","last_undegraded":"2023-05-16T07:18:17.076862+0000","last_fullsized":"2023-05-16T07:18:17.076862+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444576+0000","last_change":"2023-05-16T07:18:17.444576+0000","last_active":"2023-05-16T07:18:17.444576+0000","last_peered":"2023-05-16T07:18:17.444576+0000","last_clean":"2023-05-16T07:18:17.444576+0000","last_became_active":"2023-05-16T07:18:17.444046+0000","last_became_peered":"2023-05-16T07:18:17.444046+0000","last_unstale":"2023-05-16T07:18:17.444576+0000","last_undegraded":"2023-05-16T07:18:17.444576+0000","last_fullsized":"2023-05-16T07:18:17.444576+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.084150+0000","last_change":"2023-05-16T07:18:17.083583+0000","last_active":"2023-05-16T07:18:17.084150+0000","last_peered":"2023-05-16T07:18:17.084150+0000","last_clean":"2023-05-16T07:18:17.084150+0000","last_became_active":"2023-05-16T07:18:17.083355+0000","last_became_peered":"2023-05-16T07:18:17.083355+0000","last_unstale":"2023-05-16T07:18:17.084150+0000","last_undegraded":"2023-05-16T07:18:17.084150+0000","last_fullsized":"2023-05-16T07:18:17.084150+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.443633+0000","last_change":"2023-05-16T07:18:17.443633+0000","last_active":"2023-05-16T07:18:17.443633+0000","last_peered":"2023-05-16T07:18:17.443633+0000","last_clean":"2023-05-16T07:18:17.443633+0000","last_became_active":"2023-05-16T07:18:17.443435+0000","last_became_peered":"2023-05-16T07:18:17.443435+0000","last_unstale":"2023-05-16T07:18:17.443633+0000","last_undegraded":"2023-05-16T07:18:17.443633+0000","last_fullsized":"2023-05-16T07:18:17.443633+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.432721+0000","last_change":"2023-05-16T07:18:17.072746+0000","last_active":"2023-05-16T07:18:17.432721+0000","last_peered":"2023-05-16T07:18:17.432721+0000","last_clean":"2023-05-16T07:18:17.432721+0000","last_became_active":"2023-05-16T07:18:17.072432+0000","last_became_peered":"2023-05-16T07:18:17.072432+0000","last_unstale":"2023-05-16T07:18:17.432721+0000","last_undegraded":"2023-05-16T07:18:17.432721+0000","last_fullsized":"2023-05-16T07:18:17.432721+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.076473+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:18:17.076473+0000","last_peered":"2023-05-16T07:18:17.076473+0000","last_clean":"2023-05-16T07:18:17.076473+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:18:17.076473+0000","last_undegraded":"2023-05-16T07:18:17.076473+0000","last_fullsized":"2023-05-16T07:18:17.076473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.442676+0000","last_change":"2023-05-16T07:18:17.442676+0000","last_active":"2023-05-16T07:18:17.442676+0000","last_peered":"2023-05-16T07:18:17.442676+0000","last_clean":"2023-05-16T07:18:17.442676+0000","last_became_active":"2023-05-16T07:18:17.441971+0000","last_became_peered":"2023-05-16T07:18:17.441971+0000","last_unstale":"2023-05-16T07:18:17.442676+0000","last_undegraded":"2023-05-16T07:18:17.442676+0000","last_fullsized":"2023-05-16T07:18:17.442676+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.080471+0000","last_change":"2023-05-16T07:18:17.079288+0000","last_active":"2023-05-16T07:18:17.080471+0000","last_peered":"2023-05-16T07:18:17.080471+0000","last_clean":"2023-05-16T07:18:17.080471+0000","last_became_active":"2023-05-16T07:18:17.078986+0000","last_became_peered":"2023-05-16T07:18:17.078986+0000","last_unstale":"2023-05-16T07:18:17.080471+0000","last_undegraded":"2023-05-16T07:18:17.080471+0000","last_fullsized":"2023-05-16T07:18:17.080471+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":81,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.063535+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:18:17.063535+0000","last_peered":"2023-05-16T07:18:17.063535+0000","last_clean":"2023-05-16T07:18:17.063535+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:18:17.063535+0000","last_undegraded":"2023-05-16T07:18:17.063535+0000","last_fullsized":"2023-05-16T07:18:17.063535+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.440521+0000","last_change":"2023-05-16T07:18:17.440521+0000","last_active":"2023-05-16T07:18:17.440521+0000","last_peered":"2023-05-16T07:18:17.440521+0000","last_clean":"2023-05-16T07:18:17.440521+0000","last_became_active":"2023-05-16T07:18:17.440206+0000","last_became_peered":"2023-05-16T07:18:17.440206+0000","last_unstale":"2023-05-16T07:18:17.440521+0000","last_undegraded":"2023-05-16T07:18:17.440521+0000","last_fullsized":"2023-05-16T07:18:17.440521+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.447368+0000","last_change":"2023-05-16T07:18:17.074712+0000","last_active":"2023-05-16T07:18:17.447368+0000","last_peered":"2023-05-16T07:18:17.447368+0000","last_clean":"2023-05-16T07:18:17.447368+0000","last_became_active":"2023-05-16T07:18:17.074231+0000","last_became_peered":"2023-05-16T07:18:17.074231+0000","last_unstale":"2023-05-16T07:18:17.447368+0000","last_undegraded":"2023-05-16T07:18:17.447368+0000","last_fullsized":"2023-05-16T07:18:17.447368+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.066141+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:18:17.066141+0000","last_peered":"2023-05-16T07:18:17.066141+0000","last_clean":"2023-05-16T07:18:17.066141+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:18:17.066141+0000","last_undegraded":"2023-05-16T07:18:17.066141+0000","last_fullsized":"2023-05-16T07:18:17.066141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.438877+0000","last_change":"2023-05-16T07:18:17.438877+0000","last_active":"2023-05-16T07:18:17.438877+0000","last_peered":"2023-05-16T07:18:17.438877+0000","last_clean":"2023-05-16T07:18:17.438877+0000","last_became_active":"2023-05-16T07:18:17.438573+0000","last_became_peered":"2023-05-16T07:18:17.438573+0000","last_unstale":"2023-05-16T07:18:17.438877+0000","last_undegraded":"2023-05-16T07:18:17.438877+0000","last_fullsized":"2023-05-16T07:18:17.438877+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.082126+0000","last_change":"2023-05-16T07:18:17.076985+0000","last_active":"2023-05-16T07:18:17.082126+0000","last_peered":"2023-05-16T07:18:17.082126+0000","last_clean":"2023-05-16T07:18:17.082126+0000","last_became_active":"2023-05-16T07:18:17.076696+0000","last_became_peered":"2023-05-16T07:18:17.076696+0000","last_unstale":"2023-05-16T07:18:17.082126+0000","last_undegraded":"2023-05-16T07:18:17.082126+0000","last_fullsized":"2023-05-16T07:18:17.082126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":81,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.061665+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:18:17.061665+0000","last_peered":"2023-05-16T07:18:17.061665+0000","last_clean":"2023-05-16T07:18:17.061665+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:18:17.061665+0000","last_undegraded":"2023-05-16T07:18:17.061665+0000","last_fullsized":"2023-05-16T07:18:17.061665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.068277+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:18:17.068277+0000","last_peered":"2023-05-16T07:18:17.068277+0000","last_clean":"2023-05-16T07:18:17.068277+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:18:17.068277+0000","last_undegraded":"2023-05-16T07:18:17.068277+0000","last_fullsized":"2023-05-16T07:18:17.068277+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.442748+0000","last_change":"2023-05-16T07:18:17.442748+0000","last_active":"2023-05-16T07:18:17.442748+0000","last_peered":"2023-05-16T07:18:17.442748+0000","last_clean":"2023-05-16T07:18:17.442748+0000","last_became_active":"2023-05-16T07:18:17.442494+0000","last_became_peered":"2023-05-16T07:18:17.442494+0000","last_unstale":"2023-05-16T07:18:17.442748+0000","last_undegraded":"2023-05-16T07:18:17.442748+0000","last_fullsized":"2023-05-16T07:18:17.442748+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.428932+0000","last_change":"2023-05-16T07:18:17.075860+0000","last_active":"2023-05-16T07:18:17.428932+0000","last_peered":"2023-05-16T07:18:17.428932+0000","last_clean":"2023-05-16T07:18:17.428932+0000","last_became_active":"2023-05-16T07:18:17.075525+0000","last_became_peered":"2023-05-16T07:18:17.075525+0000","last_unstale":"2023-05-16T07:18:17.428932+0000","last_undegraded":"2023-05-16T07:18:17.428932+0000","last_fullsized":"2023-05-16T07:18:17.428932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.068353+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:18:17.068353+0000","last_peered":"2023-05-16T07:18:17.068353+0000","last_clean":"2023-05-16T07:18:17.068353+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:18:17.068353+0000","last_undegraded":"2023-05-16T07:18:17.068353+0000","last_fullsized":"2023-05-16T07:18:17.068353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.446910+0000","last_change":"2023-05-16T07:18:17.446910+0000","last_active":"2023-05-16T07:18:17.446910+0000","last_peered":"2023-05-16T07:18:17.446910+0000","last_clean":"2023-05-16T07:18:17.446910+0000","last_became_active":"2023-05-16T07:18:17.446507+0000","last_became_peered":"2023-05-16T07:18:17.446507+0000","last_unstale":"2023-05-16T07:18:17.446910+0000","last_undegraded":"2023-05-16T07:18:17.446910+0000","last_fullsized":"2023-05-16T07:18:17.446910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.085865+0000","last_change":"2023-05-16T07:18:17.082252+0000","last_active":"2023-05-16T07:18:17.085865+0000","last_peered":"2023-05-16T07:18:17.085865+0000","last_clean":"2023-05-16T07:18:17.085865+0000","last_became_active":"2023-05-16T07:18:17.081778+0000","last_became_peered":"2023-05-16T07:18:17.081778+0000","last_unstale":"2023-05-16T07:18:17.085865+0000","last_undegraded":"2023-05-16T07:18:17.085865+0000","last_fullsized":"2023-05-16T07:18:17.085865+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.069334+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:18:17.069334+0000","last_peered":"2023-05-16T07:18:17.069334+0000","last_clean":"2023-05-16T07:18:17.069334+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:18:17.069334+0000","last_undegraded":"2023-05-16T07:18:17.069334+0000","last_fullsized":"2023-05-16T07:18:17.069334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.448644+0000","last_change":"2023-05-16T07:18:17.448644+0000","last_active":"2023-05-16T07:18:17.448644+0000","last_peered":"2023-05-16T07:18:17.448644+0000","last_clean":"2023-05-16T07:18:17.448644+0000","last_became_active":"2023-05-16T07:18:17.448114+0000","last_became_peered":"2023-05-16T07:18:17.448114+0000","last_unstale":"2023-05-16T07:18:17.448644+0000","last_undegraded":"2023-05-16T07:18:17.448644+0000","last_fullsized":"2023-05-16T07:18:17.448644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.073370+0000","last_change":"2023-05-16T07:18:17.070927+0000","last_active":"2023-05-16T07:18:17.073370+0000","last_peered":"2023-05-16T07:18:17.073370+0000","last_clean":"2023-05-16T07:18:17.073370+0000","last_became_active":"2023-05-16T07:18:17.070460+0000","last_became_peered":"2023-05-16T07:18:17.070460+0000","last_unstale":"2023-05-16T07:18:17.073370+0000","last_undegraded":"2023-05-16T07:18:17.073370+0000","last_fullsized":"2023-05-16T07:18:17.073370+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.065800+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:18:17.065800+0000","last_peered":"2023-05-16T07:18:17.065800+0000","last_clean":"2023-05-16T07:18:17.065800+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:18:17.065800+0000","last_undegraded":"2023-05-16T07:18:17.065800+0000","last_fullsized":"2023-05-16T07:18:17.065800+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.444765+0000","last_change":"2023-05-16T07:18:17.444765+0000","last_active":"2023-05-16T07:18:17.444765+0000","last_peered":"2023-05-16T07:18:17.444765+0000","last_clean":"2023-05-16T07:18:17.444765+0000","last_became_active":"2023-05-16T07:18:17.444327+0000","last_became_peered":"2023-05-16T07:18:17.444327+0000","last_unstale":"2023-05-16T07:18:17.444765+0000","last_undegraded":"2023-05-16T07:18:17.444765+0000","last_fullsized":"2023-05-16T07:18:17.444765+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:17.048995+0000","last_clean_scrub_stamp":"2023-05-16T07:18:17.048995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.072927+0000","last_change":"2023-05-16T07:18:17.068695+0000","last_active":"2023-05-16T07:18:17.072927+0000","last_peered":"2023-05-16T07:18:17.072927+0000","last_clean":"2023-05-16T07:18:17.072927+0000","last_became_active":"2023-05-16T07:18:17.068166+0000","last_became_peered":"2023-05-16T07:18:17.068166+0000","last_unstale":"2023-05-16T07:18:17.072927+0000","last_undegraded":"2023-05-16T07:18:17.072927+0000","last_fullsized":"2023-05-16T07:18:17.072927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:18:16.043857+0000","last_clean_scrub_stamp":"2023-05-16T07:18:16.043857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-05-16T07:18:17.058423+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:18:17.058423+0000","last_peered":"2023-05-16T07:18:17.058423+0000","last_clean":"2023-05-16T07:18:17.058423+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:18:17.058423+0000","last_undegraded":"2023-05-16T07:18:17.058423+0000","last_fullsized":"2023-05-16T07:18:17.058423+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18120,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":16832,"kb_avail":94353720,"statfs":{"total":96636764160,"available":96618209280,"internally_reserved":0,"allocated":1310720,"data_stored":675719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17235968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.77700000000000002,"15min":0.84199999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67400000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.312,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":0.873,"15min":0.873},"last":0.503},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70499999999999996,"5min":0.876,"15min":0.876},"last":0.495}]},{"osd":2,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.441},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.35899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.79300000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45000000000000001,"15min":0.46200000000000002},"min":{"1min":0.308,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47399999999999998,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.752,"15min":0.81999999999999995},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47599999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66300000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73899999999999999,"5min":0.748,"15min":0.999},"last":0.32300000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.879,"15min":0.95099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.38900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21480,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350360,"statfs":{"total":96636764160,"available":96614768640,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.317,"15min":0.26400000000000001},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.85099999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66500000000000004,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76000000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.311,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.89400000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.503,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":4.048,"15min":4.048},"last":0.41799999999999998}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.749,"5min":0.749,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575039,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17512,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":16256,"kb_avail":94354328,"statfs":{"total":96636764160,"available":96618831872,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16646144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.69899999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.48199999999999998,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36499999999999999,"15min":0.378},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26000000000000001},"max":{"1min":0.49399999999999999,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.52700000000000002,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38300000000000001,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.501,"5min":0.622,"15min":0.64000000000000001},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71199999999999997,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76100000000000001,"5min":0.94699999999999995,"15min":1.087},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.93700000000000006,"15min":1.204},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.627,"15min":0.64100000000000001},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41299999999999998},"max":{"1min":0.76900000000000002,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575039,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16564,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":15296,"kb_avail":94355276,"statfs":{"total":96636764160,"available":96619802624,"internally_reserved":0,"allocated":1290240,"data_stored":667900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15663104},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33900000000000002,"15min":0.35699999999999998},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47099999999999997,"5min":0.53700000000000003,"15min":0.60199999999999998},"last":0.222},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.38600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.35299999999999998,"15min":0.373},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.40300000000000002},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.50600000000000001,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.251}]},{"osd":3,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38600000000000001,"15min":0.40699999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.24199999999999999},"max":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.28599999999999998}]},{"osd":4,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60599999999999998,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":1.157},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.56799999999999995}]},{"osd":6,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue May 16 07:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.65000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85499999999999998,"5min":0.93700000000000006,"15min":1.196},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18344,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":17088,"kb_avail":94353496,"statfs":{"total":96636764160,"available":96617979904,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17498112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.184},"max":{"1min":0.5,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.249},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.375,"15min":0.376},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.215}]},{"osd":1,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.68899999999999995,"15min":0.71199999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.434,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.29999999999999999}]},{"osd":3,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.64700000000000002},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.432,"15min":0.439},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60899999999999999,"5min":0.71399999999999997,"15min":0.72899999999999998},"last":0.29199999999999998}]},{"osd":4,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.274},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82599999999999996,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.37},"max":{"1min":0.872,"5min":0.88100000000000001,"15min":1.5609999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.47},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue May 16 07:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88200000000000001,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21448,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":20160,"kb_avail":94350392,"statfs":{"total":96636764160,"available":96614801408,"internally_reserved":0,"allocated":1310720,"data_stored":676625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20643840},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36099999999999999},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53300000000000003,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.36699999999999999,"15min":0.375},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46400000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45000000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40799999999999997,"15min":0.40899999999999997},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51300000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48199999999999998,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.91200000000000003,"15min":0.96399999999999997},"last":0.52000000000000002}]},{"osd":5,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85799999999999998,"5min":0.91600000000000004,"15min":1.0489999999999999},"last":0.498},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70199999999999996,"5min":0.93799999999999994,"15min":1.0409999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.97399999999999998,"15min":1.0780000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.73099999999999998,"5min":0.94299999999999995,"15min":1.0049999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Tue May 16 07:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.40100000000000002},"max":{"1min":0.76400000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.93600000000000005,"15min":1.02},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17576,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":16320,"kb_avail":94354264,"statfs":{"total":96636764160,"available":96618766336,"internally_reserved":0,"allocated":1277952,"data_stored":660637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16711680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.67300000000000004,"5min":0.82099999999999995,"15min":0.83199999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.81200000000000006},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.627,"5min":0.85399999999999998,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.67500000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.90500000000000003,"15min":0.999},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.84599999999999997,"15min":0.97999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.629,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84099999999999997,"5min":0.875,"15min":0.93100000000000005},"last":0.626}]},{"osd":5,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":1.0640000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.86699999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70699999999999996,"5min":0.79200000000000004,"15min":0.94599999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.86199999999999999,"15min":1.2689999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue May 16 07:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":1.48,"15min":1.48},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":1.046},"last":0.67200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24948,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":23680,"kb_avail":94346892,"statfs":{"total":96636764160,"available":96611217408,"internally_reserved":0,"allocated":1290240,"data_stored":667900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24248320},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.36599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81599999999999995,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":3.641},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.81000000000000005,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":2.847}]},{"osd":1,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.30299999999999999},"max":{"1min":0.96099999999999997,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.38700000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86399999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":3.6059999999999999}]},{"osd":2,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71399999999999997,"5min":2.278,"15min":2.278},"last":2.8799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.45100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.748,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.127},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":2.806}]},{"osd":4,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.33900000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":1.333,"15min":1.333},"last":0.318},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.309},"max":{"1min":0.84899999999999998,"5min":2.081,"15min":2.081},"last":3.5659999999999998}]},{"osd":6,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":2.694},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":1.024,"15min":1.024},"last":1.554}]},{"osd":7,"last update":"Tue May 16 07:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.878,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.86599999999999999,"15min":0.878},"last":2.6429999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:18:20.972 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:18:20.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:18:21.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.115+0000 7feb6b59e700 1 Processor -- start 2023-05-16T07:18:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.115+0000 7feb6b59e700 1 -- start start 2023-05-16T07:18:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb6c066e50 0x7feb6c0672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6c0677b0 con 0x7feb6c153cd0 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6c0a7370 con 0x7feb6c066e50 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0a6760 con 0x7feb6c1540b0 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6a59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7feb6c153cd0 0x7feb6c064f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58510/0 (socket says 172.21.15.92:58510) 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb6c066e50 0x7feb6c0672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6a59c700 1 -- 172.21.15.92:0/2516605028 learned_addr learned my addr 172.21.15.92:0/2516605028 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb61d9b700 1 --2- 172.21.15.92:0/2516605028 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6ad9d700 1 -- 172.21.15.92:0/2516605028 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 msgr2=0x7feb6c0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6ad9d700 1 --2- 172.21.15.92:0/2516605028 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c0a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6ad9d700 1 -- 172.21.15.92:0/2516605028 >> v1:172.21.15.92:6789/0 conn(0x7feb6c153cd0 legacy=0x7feb6c064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb69d9b700 1 -- 172.21.15.92:0/2516605028 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497649908 0 0) 0x7feb6c0677b0 con 0x7feb6c153cd0 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6ad9d700 1 -- 172.21.15.92:0/2516605028 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0a6a40 con 0x7feb6c066e50 2023-05-16T07:18:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb6ad9d700 1 --2- 172.21.15.92:0/2516605028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c0672c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7feb64009e60 tx=0x7feb64009780).ready entity=mon.0 client_cookie=305630f813f9c491 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.116+0000 7feb69d9b700 1 -- 172.21.15.92:0/2516605028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400e070 con 0x7feb6c066e50 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb69d9b700 1 -- 172.21.15.92:0/2516605028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6400bc30 con 0x7feb6c066e50 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb69d9b700 1 -- 172.21.15.92:0/2516605028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64012410 con 0x7feb6c066e50 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 -- 172.21.15.92:0/2516605028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 msgr2=0x7feb6c0672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 --2- 172.21.15.92:0/2516605028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c0672c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7feb64009e60 tx=0x7feb64009780).stop 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 -- 172.21.15.92:0/2516605028 shutdown_connections 2023-05-16T07:18:21.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 --2- 172.21.15.92:0/2516605028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c0672c0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 --2- 172.21.15.92:0/2516605028 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c0a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 -- 172.21.15.92:0/2516605028 >> 172.21.15.92:0/2516605028 conn(0x7feb6c068a30 msgr2=0x7feb6c0c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:21.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 -- 172.21.15.92:0/2516605028 shutdown_connections 2023-05-16T07:18:21.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.117+0000 7feb6b59e700 1 -- 172.21.15.92:0/2516605028 wait complete. 2023-05-16T07:18:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.118+0000 7feb6b59e700 1 Processor -- start 2023-05-16T07:18:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.118+0000 7feb6b59e700 1 -- start start 2023-05-16T07:18:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c166f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6b59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb6c153cd0 0x7feb6c0aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c167e20 con 0x7feb6c066e50 2023-05-16T07:18:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6b59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0ab0e0 con 0x7feb6c153cd0 2023-05-16T07:18:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c167580 con 0x7feb6c1540b0 2023-05-16T07:18:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c166f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47768/0 (socket says 172.21.15.92:47768) 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 -- 172.21.15.92:0/1472101728 learned_addr learned my addr 172.21.15.92:0/1472101728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 -- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 msgr2=0x7feb6c150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 -- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb6c153cd0 msgr2=0x7feb6c0aabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb6c153cd0 0x7feb6c0aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 -- 172.21.15.92:0/1472101728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64009a20 con 0x7feb6c066e50 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.119+0000 7feb6a59c700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c166f30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7feb5c00c490 tx=0x7feb5c007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.120+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0072e0 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.120+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c007440 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.120+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c167860 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.120+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0057a0 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.120+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c13ec70 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.121+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c047150 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.121+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7feb5c010070 con 0x7feb6c066e50 2023-05-16T07:18:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.121+0000 7feb62ffd700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feb4c05ed90 0x7feb4c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.121+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7feb5c0793a0 con 0x7feb6c066e50 2023-05-16T07:18:21.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.121+0000 7feb61d9b700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feb4c05ed90 0x7feb4c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.122+0000 7feb61d9b700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feb4c05ed90 0x7feb4c061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7feb6c157970 tx=0x7feb54008040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:21.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.125+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7feb5c04a6f0 con 0x7feb6c066e50 2023-05-16T07:18:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.349+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7feb6c0c8480 con 0x7feb6c066e50 2023-05-16T07:18:21.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.352+0000 7feb62ffd700 1 -- 172.21.15.92:0/1472101728 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7feb5c04dd10 con 0x7feb6c066e50 2023-05-16T07:18:21.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feb4c05ed90 msgr2=0x7feb4c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feb4c05ed90 0x7feb4c061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7feb6c157970 tx=0x7feb54008040).stop 2023-05-16T07:18:21.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 msgr2=0x7feb6c166f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c166f30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7feb5c00c490 tx=0x7feb5c007e40).stop 2023-05-16T07:18:21.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 shutdown_connections 2023-05-16T07:18:21.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7feb4c05ed90 0x7feb4c061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7feb6c1540b0 0x7feb6c150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb6c066e50 0x7feb6c166f30 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 --2- 172.21.15.92:0/1472101728 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb6c153cd0 0x7feb6c0aabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 >> 172.21.15.92:0/1472101728 conn(0x7feb6c068a30 msgr2=0x7feb6c0a9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 shutdown_connections 2023-05-16T07:18:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.354+0000 7feb6b59e700 1 -- 172.21.15.92:0/1472101728 wait complete. 2023-05-16T07:18:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:18:21.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:18:21.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.479+0000 7f80d3c2e700 1 Processor -- start 2023-05-16T07:18:21.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.480+0000 7f80d3c2e700 1 -- start start 2023-05-16T07:18:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.480+0000 7f80d3c2e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80cc154170 0x7f80cc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.480+0000 7f80d3c2e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.480+0000 7f80d3c2e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0c0660 con 0x7f80cc154550 2023-05-16T07:18:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.480+0000 7f80d3c2e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc0c1b00 con 0x7f80cc154170 2023-05-16T07:18:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.480+0000 7f80d3c2e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0bfd30 con 0x7f80cc069db0 2023-05-16T07:18:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80cc154170 0x7f80cc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80cc154170 0x7f80cc0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47776/0 (socket says 172.21.15.92:47776) 2023-05-16T07:18:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 -- 172.21.15.92:0/3475351804 learned_addr learned my addr 172.21.15.92:0/3475351804 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d21cb700 1 --2- 172.21.15.92:0/3475351804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 -- 172.21.15.92:0/3475351804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 msgr2=0x7f80cc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 --2- 172.21.15.92:0/3475351804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 -- 172.21.15.92:0/3475351804 >> v1:172.21.15.158:6789/0 conn(0x7f80cc154550 legacy=0x7f80cc0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 -- 172.21.15.92:0/3475351804 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0c0010 con 0x7f80cc154170 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d19ca700 1 --2- 172.21.15.92:0/3475351804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc0c5c70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f80bc009f20 tx=0x7f80bc009840).ready entity=mon.0 client_cookie=fa54fd745a47b417 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d09c8700 1 -- 172.21.15.92:0/3475351804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00e070 con 0x7f80cc154170 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d09c8700 1 -- 172.21.15.92:0/3475351804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc00ba80 con 0x7f80cc154170 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.481+0000 7f80d09c8700 1 -- 172.21.15.92:0/3475351804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00be00 con 0x7f80cc154170 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 -- 172.21.15.92:0/3475351804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 msgr2=0x7f80cc0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/3475351804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc0c5c70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f80bc009f20 tx=0x7f80bc009840).stop 2023-05-16T07:18:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 -- 172.21.15.92:0/3475351804 shutdown_connections 2023-05-16T07:18:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/3475351804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc0c5c70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/3475351804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 -- 172.21.15.92:0/3475351804 >> 172.21.15.92:0/3475351804 conn(0x7f80cc0cb700 msgr2=0x7f80cc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 -- 172.21.15.92:0/3475351804 shutdown_connections 2023-05-16T07:18:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.482+0000 7f80d3c2e700 1 -- 172.21.15.92:0/3475351804 wait complete. 2023-05-16T07:18:21.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.483+0000 7f80d3c2e700 1 Processor -- start 2023-05-16T07:18:21.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.483+0000 7f80d3c2e700 1 -- start start 2023-05-16T07:18:21.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f80cc154550 0x7f80cc0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0c25e0 con 0x7f80cc154170 2023-05-16T07:18:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1487d0 con 0x7f80cc154550 2023-05-16T07:18:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc148210 con 0x7f80cc069db0 2023-05-16T07:18:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47784/0 (socket says 172.21.15.92:47784) 2023-05-16T07:18:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 -- 172.21.15.92:0/116343977 learned_addr learned my addr 172.21.15.92:0/116343977 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d19ca700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d21cb700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f80cc154550 0x7f80cc0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 -- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 msgr2=0x7f80cc144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 -- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f80cc154550 msgr2=0x7f80cc0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f80cc154550 0x7f80cc0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 -- 172.21.15.92:0/116343977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009ae0 con 0x7f80cc154170 2023-05-16T07:18:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d11c9700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc147830 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f80c000c440 tx=0x7f80c0007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c000ce40 con 0x7f80cc154170 2023-05-16T07:18:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c000f470 con 0x7f80cc154170 2023-05-16T07:18:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc1484f0 con 0x7f80cc154170 2023-05-16T07:18:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c000f830 con 0x7f80cc154170 2023-05-16T07:18:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.484+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc145130 con 0x7f80cc154170 2023-05-16T07:18:21.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.485+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc0666c0 con 0x7f80cc154170 2023-05-16T07:18:21.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.486+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f80c0010070 con 0x7f80cc154170 2023-05-16T07:18:21.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.486+0000 7f80c9ffb700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f80b805ed90 0x7f80b8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:21.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.486+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f80c0079010 con 0x7f80cc154170 2023-05-16T07:18:21.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.486+0000 7f80d19ca700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f80b805ed90 0x7f80b8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:21.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.486+0000 7f80d19ca700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f80b805ed90 0x7f80b8061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f80bc00c010 tx=0x7f80bc009840).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:21.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.490+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f80c00478f0 con 0x7f80cc154170 2023-05-16T07:18:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.692+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80cc06c350 con 0x7f80cc154170 2023-05-16T07:18:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.692+0000 7f80c9ffb700 1 -- 172.21.15.92:0/116343977 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f80c004a360 con 0x7f80cc154170 2023-05-16T07:18:21.696 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:21.696 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:18:17.423562+0000","modified":"2023-05-16T07:18:19.645369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7017":{"gid":7017,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1226709881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1226709881},{"type":"v1","addr":"172.21.15.92:6835","nonce":1226709881}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-05-16T07:18:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f80b805ed90 msgr2=0x7f80b8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f80b805ed90 0x7f80b8061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f80bc00c010 tx=0x7f80bc009840).stop 2023-05-16T07:18:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 msgr2=0x7f80cc147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc147830 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f80c000c440 tx=0x7f80c0007d80).stop 2023-05-16T07:18:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 shutdown_connections 2023-05-16T07:18:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f80b805ed90 0x7f80b8061250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f80cc069db0 0x7f80cc144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f80cc154550 0x7f80cc0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 --2- 172.21.15.92:0/116343977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80cc154170 0x7f80cc147830 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 >> 172.21.15.92:0/116343977 conn(0x7f80cc0cb700 msgr2=0x7f80cc0ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 shutdown_connections 2023-05-16T07:18:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:21.694+0000 7f80d3c2e700 1 -- 172.21.15.92:0/116343977 wait complete. 2023-05-16T07:18:21.700 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-05-16T07:18:22.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:18:17.423562+0000', 'modified': '2023-05-16T07:18:19.645369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7017}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7017': {'gid': 7017, 'name': 'a', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/1226709881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6833', 'nonce': 1226709881}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1226709881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:18:22.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:18:22.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-16T07:18:22.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.222+0000 7f9aba261700 1 Processor -- start 2023-05-16T07:18:22.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.222+0000 7f9aba261700 1 -- start start 2023-05-16T07:18:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9aba261700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 0x7f9ab40a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9aba261700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9ab40a7be0 0x7f9ab40bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9aba261700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40c5ce0 con 0x7f9ab40bf2d0 2023-05-16T07:18:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9aba261700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab40c51d0 con 0x7f9ab40a7be0 2023-05-16T07:18:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9aba261700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c9370 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab8a5e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 0x7f9ab40a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab9a60700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9ab40a7be0 0x7f9ab40bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab8a5e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 0x7f9ab40a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:40364/0 (socket says 172.21.15.92:40364) 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab8a5e700 1 -- 172.21.15.92:0/3204052762 learned_addr learned my addr 172.21.15.92:0/3204052762 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab8a5e700 1 -- 172.21.15.92:0/3204052762 >> v2:172.21.15.158:3300/0 conn(0x7f9ab40a7be0 msgr2=0x7f9ab40bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/3204052762 >> v2:172.21.15.158:3300/0 conn(0x7f9ab40a7be0 0x7f9ab40bd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.223+0000 7f9ab8a5e700 1 -- 172.21.15.92:0/3204052762 >> v1:172.21.15.158:6789/0 conn(0x7f9ab40bf2d0 legacy=0x7f9ab40c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9ab8a5e700 1 -- 172.21.15.92:0/3204052762 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40a8070 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/3204052762 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 0x7f9ab40a76f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f9aa4006fd0 tx=0x7f9aa40090f0).ready entity=mon.2 client_cookie=8a9a3a44fc6880a9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aabfff700 1 -- 172.21.15.92:0/3204052762 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400b580 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aabfff700 1 -- 172.21.15.92:0/3204052762 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa400b6e0 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aabfff700 1 -- 172.21.15.92:0/3204052762 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400ba60 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 -- 172.21.15.92:0/3204052762 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 msgr2=0x7f9ab40a76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 --2- 172.21.15.92:0/3204052762 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 0x7f9ab40a76f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f9aa4006fd0 tx=0x7f9aa40090f0).stop 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 -- 172.21.15.92:0/3204052762 shutdown_connections 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 --2- 172.21.15.92:0/3204052762 >> v2:172.21.15.158:3300/0 conn(0x7f9ab40a7be0 0x7f9ab40bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 --2- 172.21.15.92:0/3204052762 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40c8ea0 0x7f9ab40a76f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 -- 172.21.15.92:0/3204052762 >> 172.21.15.92:0/3204052762 conn(0x7f9ab40622b0 msgr2=0x7f9ab40626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 -- 172.21.15.92:0/3204052762 shutdown_connections 2023-05-16T07:18:22.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.224+0000 7f9aba261700 1 -- 172.21.15.92:0/3204052762 wait complete. 2023-05-16T07:18:22.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.225+0000 7f9aba261700 1 Processor -- start 2023-05-16T07:18:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.225+0000 7f9aba261700 1 -- start start 2023-05-16T07:18:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9aba261700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40a7be0 0x7f9ab40c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9aba261700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ab40bf2d0 0x7f9ab40c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9aba261700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 0x7f9ab40a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9aba261700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c23f0 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9aba261700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c1570 con 0x7f9ab40bf2d0 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9aba261700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab41645b0 con 0x7f9ab40a7be0 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.226+0000 7f9ab9a60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 0x7f9ab40a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 0x7f9ab40a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47800/0 (socket says 172.21.15.92:47800) 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 -- 172.21.15.92:0/2157883694 learned_addr learned my addr 172.21.15.92:0/2157883694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ab40bf2d0 0x7f9ab40c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab925f700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40a7be0 0x7f9ab40c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 -- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40a7be0 msgr2=0x7f9ab40c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40a7be0 0x7f9ab40c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 -- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ab40bf2d0 msgr2=0x7f9ab40c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ab40bf2d0 0x7f9ab40c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 -- 172.21.15.92:0/2157883694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa4009370 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 0x7f9ab40a54c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f9aac009df0 tx=0x7f9aac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9aa97fa700 1 -- 172.21.15.92:0/2157883694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00caa0 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9aa97fa700 1 -- 172.21.15.92:0/2157883694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac00f040 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9aa97fa700 1 -- 172.21.15.92:0/2157883694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00ce10 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab40c1770 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.227+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab40c1d30 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.228+0000 7f9aa97fa700 1 -- 172.21.15.92:0/2157883694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9aac013610 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.229+0000 7f9aa97fa700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9a9805ed90 0x7f9a98061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.229+0000 7f9aa97fa700 1 -- 172.21.15.92:0/2157883694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f9aac07a650 con 0x7f9ab40c8ea0 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.229+0000 7f9ab925f700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9a9805ed90 0x7f9a98061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.229+0000 7f9ab925f700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9a9805ed90 0x7f9a98061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9ab0007fd0 tx=0x7f9ab0007d30).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 Processor -- start 2023-05-16T07:18:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 -- start start 2023-05-16T07:18:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 0x55ac4e8aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e883a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e87e660 0x55ac4e87eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8495e0 con 0x55ac4e8836b0 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e7c08a0 con 0x55ac4e8cc350 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.231+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e7d5900 con 0x55ac4e87e660 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 0x55ac4e8aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 0x55ac4e8aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47804/0 (socket says 172.21.15.92:47804) 2023-05-16T07:18:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 -- 172.21.15.92:0/3517312697 learned_addr learned my addr 172.21.15.92:0/3517312697 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e883a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e87e660 0x55ac4e87eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 -- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e87e660 msgr2=0x55ac4e87eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e87e660 0x55ac4e87eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 -- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 msgr2=0x55ac4e883a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e883a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 -- 172.21.15.92:0/3517312697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac4e678310 con 0x55ac4e8836b0 2023-05-16T07:18:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e87e660 0x55ac4e87eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e883a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9ab925f700 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 0x55ac4e8aba90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f9ab000f820 tx=0x7f9ab000a000).ready entity=mon.0 client_cookie=ca8560f3cb2e011c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9a9e7fc700 1 -- 172.21.15.92:0/3517312697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab001c070 con 0x55ac4e8836b0 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9a9e7fc700 1 -- 172.21.15.92:0/3517312697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0003570 con 0x55ac4e8836b0 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.232+0000 7f9a9e7fc700 1 -- 172.21.15.92:0/3517312697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0014870 con 0x55ac4e8836b0 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 -- 172.21.15.92:0/3517312697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 msgr2=0x55ac4e8aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 0x55ac4e8aba90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f9ab000f820 tx=0x7f9ab000a000).stop 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 -- 172.21.15.92:0/3517312697 shutdown_connections 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e87e660 0x55ac4e87eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e8836b0 0x55ac4e8aba90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3517312697 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e883a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 -- 172.21.15.92:0/3517312697 >> 172.21.15.92:0/3517312697 conn(0x55ac4e640e40 msgr2=0x55ac4e8cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 -- 172.21.15.92:0/3517312697 shutdown_connections 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 -- 172.21.15.92:0/3517312697 wait complete. 2023-05-16T07:18:22.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 -- start start 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.233+0000 7f9acce55b80 1 Processor -- start 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e8b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e8d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e79ced0 con 0x55ac4e87e660 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8ce110 con 0x55ac4e8836b0 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8b1200 con 0x55ac4e8cc350 2023-05-16T07:18:22.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9acce55b80 10 client.0 ll_register_callbacks cb 0x55ac4e8433b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:18:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47810/0 (socket says 172.21.15.92:47810) 2023-05-16T07:18:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9ab925f700 1 -- 172.21.15.92:0/2932835044 learned_addr learned my addr 172.21.15.92:0/2932835044 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e8b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.234+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e8d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab925f700 1 -- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 msgr2=0x55ac4e8d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab925f700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e8d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab925f700 1 -- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 msgr2=0x55ac4e8b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab925f700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e8b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab925f700 1 -- 172.21.15.92:0/2932835044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab40c5ce0 con 0x55ac4e87e660 2023-05-16T07:18:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e8b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:22.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab925f700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8abbb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f9ab0007780 tx=0x7f9ab001f000).ready entity=mon.0 client_cookie=ecf264bb87a0d5e2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9a937fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9a937fe700 1 -- 172.21.15.92:0/2932835044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0016480 con 0x55ac4e87e660 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9a937fe700 1 -- 172.21.15.92:0/2932835044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f9ab00165e0 con 0x55ac4e87e660 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac4e87ba40 con 0x55ac4e87e660 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9a937fe700 1 -- 172.21.15.92:0/2932835044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab00169b0 con 0x55ac4e87e660 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 10 client.7176 fetch_fsmap learned FSMap version 37 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55ac4e7ce880 con 0x55ac4e87e660 2023-05-16T07:18:22.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9a937fe700 1 -- 172.21.15.92:0/2932835044 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac4e87ba40 con 0x55ac4e87e660 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9a937fe700 1 -- 172.21.15.92:0/2932835044 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9ab002b060 con 0x55ac4e87e660 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 10 client.7176 fetch_fsmap finished waiting for FSMap version 37 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 10 client.7176 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 20 client.7176 populate_metadata read hostname 'smithi092' 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e8ad4f0 0x55ac4e8cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 4 client.7176 mds_command: new command op to 7017 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:18:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 --> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac4e63cca0 con 0x55ac4e8ad4f0 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.235+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e8ad4f0 0x55ac4e8cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9a937fe700 10 client.7176 ms_handle_connect on v2:172.21.15.92:6833/1226709881 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e8ad4f0 0x55ac4e8cfc00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8d04825211fffde5 server_cookie=d87a078e8527893b in_seq=0 out_seq=0 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9a937fe700 1 -- 172.21.15.92:0/2932835044 <== mds.0 v2:172.21.15.92:6833/1226709881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55ac4e63cca0 con 0x55ac4e8ad4f0 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9a937fe700 10 client.7176 handle_command_reply: tid=0 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 client.7176 shutdown 2023-05-16T07:18:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 msgr2=0x55ac4e8abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8abbb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f9ab0007780 tx=0x7f9ab001f000).stop 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 shutdown_connections 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e8ad4f0 0x55ac4e8cfc00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e8d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8abbb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 --2- 172.21.15.92:0/2932835044 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e8b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9a937fe700 0 client.7176 ms_handle_reset on v2:172.21.15.92:6833/1226709881 2023-05-16T07:18:22.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 >> 172.21.15.92:0/2932835044 conn(0x55ac4e640e40 msgr2=0x55ac4e884980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 shutdown_connections 2023-05-16T07:18:22.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.236+0000 7f9acce55b80 1 -- 172.21.15.92:0/2932835044 wait complete. 2023-05-16T07:18:22.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.237+0000 7f9acce55b80 20 client.7176 trim_cache size 0 max 16384 2023-05-16T07:18:22.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 Processor -- start 2023-05-16T07:18:22.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 -- start start 2023-05-16T07:18:22.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8836b0 0x55ac4e8aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e8b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8cccf0 con 0x55ac4e87e660 2023-05-16T07:18:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8ccf60 con 0x55ac4e8cc350 2023-05-16T07:18:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8c9b70 con 0x55ac4e8836b0 2023-05-16T07:18:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47818/0 (socket says 172.21.15.92:47818) 2023-05-16T07:18:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.253+0000 7f9ab925f700 1 -- 172.21.15.92:0/3839765065 learned_addr learned my addr 172.21.15.92:0/3839765065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8836b0 0x55ac4e8aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e8b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab925f700 1 -- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8836b0 msgr2=0x55ac4e8aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab925f700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8836b0 0x55ac4e8aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab925f700 1 -- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 msgr2=0x55ac4e8b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab925f700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e8b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab925f700 1 -- 172.21.15.92:0/3839765065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4069820 con 0x55ac4e87e660 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8836b0 0x55ac4e8aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e8b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9ab925f700 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8b1d60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f9ab0003300 tx=0x7f9ab0029000).ready entity=mon.0 client_cookie=b392c729045a523d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9a9effd700 1 -- 172.21.15.92:0/3839765065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00169b0 con 0x55ac4e87e660 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9a9effd700 1 -- 172.21.15.92:0/3839765065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0026040 con 0x55ac4e87e660 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9a9effd700 1 -- 172.21.15.92:0/3839765065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000f6d0 con 0x55ac4e87e660 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 -- 172.21.15.92:0/3839765065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 msgr2=0x55ac4e8b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8b1d60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f9ab0003300 tx=0x7f9ab0029000).stop 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 -- 172.21.15.92:0/3839765065 shutdown_connections 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8cc350 0x55ac4e8b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e8b1d60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 --2- 172.21.15.92:0/3839765065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8836b0 0x55ac4e8aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 -- 172.21.15.92:0/3839765065 >> 172.21.15.92:0/3839765065 conn(0x55ac4e640e40 msgr2=0x55ac4e8ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 -- 172.21.15.92:0/3839765065 shutdown_connections 2023-05-16T07:18:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.254+0000 7f9acce55b80 1 -- 172.21.15.92:0/3839765065 wait complete. 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.255+0000 7f9acce55b80 1 -- start start 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.255+0000 7f9acce55b80 1 Processor -- start 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e87db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e9163b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e9168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8d55c0 con 0x55ac4e87e660 2023-05-16T07:18:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8da860 con 0x55ac4e8836b0 2023-05-16T07:18:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac4e8dae20 con 0x55ac4e8cc350 2023-05-16T07:18:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9acce55b80 10 client.0 ll_register_callbacks cb 0x55ac4e8433b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:18:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e87db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e87db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47824/0 (socket says 172.21.15.92:47824) 2023-05-16T07:18:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 -- 172.21.15.92:0/1725117537 learned_addr learned my addr 172.21.15.92:0/1725117537 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab9a60700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e9168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e9163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 -- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 msgr2=0x55ac4e9168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e9168a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 -- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 msgr2=0x55ac4e9163b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e9163b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 -- 172.21.15.92:0/1725117537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac4e7ce880 con 0x55ac4e87e660 2023-05-16T07:18:22.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9ab925f700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e87db10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f9ab0015010 tx=0x7f9ab0036000).ready entity=mon.0 client_cookie=d7cb240f542fc8d2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.256+0000 7f9a937fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:18:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9a937fe700 1 -- 172.21.15.92:0/1725117537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab001a040 con 0x55ac4e87e660 2023-05-16T07:18:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9a937fe700 1 -- 172.21.15.92:0/1725117537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f9ab0021410 con 0x55ac4e87e660 2023-05-16T07:18:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac4e8adbf0 con 0x55ac4e87e660 2023-05-16T07:18:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9a937fe700 1 -- 172.21.15.92:0/1725117537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab00169b0 con 0x55ac4e87e660 2023-05-16T07:18:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 10 client.7182 fetch_fsmap learned FSMap version 37 2023-05-16T07:18:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55ac4e7ca070 con 0x55ac4e87e660 2023-05-16T07:18:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9a937fe700 1 -- 172.21.15.92:0/1725117537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac4e8adbf0 con 0x55ac4e87e660 2023-05-16T07:18:22.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9a937fe700 1 -- 172.21.15.92:0/1725117537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9ab002b070 con 0x55ac4e87e660 2023-05-16T07:18:22.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 10 client.7182 fetch_fsmap finished waiting for FSMap version 37 2023-05-16T07:18:22.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 10 client.7182 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:18:22.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 20 client.7182 populate_metadata read hostname 'smithi092' 2023-05-16T07:18:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e87ebf0 0x55ac4e87efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 4 client.7182 mds_command: new command op to 7017 tid=0[{"prefix": "status"}] 2023-05-16T07:18:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 --> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9ab0003570 con 0x55ac4e87ebf0 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e87ebf0 0x55ac4e87efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9a937fe700 10 client.7182 ms_handle_connect on v2:172.21.15.92:6833/1226709881 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.257+0000 7f9ab8a5e700 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e87ebf0 0x55ac4e87efd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b610291465b4255e server_cookie=9a8a2e7c61bd8f8d in_seq=0 out_seq=0 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9a937fe700 1 -- 172.21.15.92:0/1725117537 <== mds.0 v2:172.21.15.92:6833/1226709881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f9ab0003570 con 0x55ac4e87ebf0 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9a937fe700 10 client.7182 handle_command_reply: tid=0 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 client.7182 shutdown 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 msgr2=0x55ac4e87db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e87db10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f9ab0015010 tx=0x7f9ab0036000).stop 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 shutdown_connections 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:6833/1226709881,v1:172.21.15.92:6835/1226709881] conn(0x55ac4e87ebf0 0x55ac4e87efd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ac4e8cc350 0x55ac4e9168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac4e87e660 0x55ac4e87db10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 --2- 172.21.15.92:0/1725117537 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ac4e8836b0 0x55ac4e9163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 >> 172.21.15.92:0/1725117537 conn(0x55ac4e640e40 msgr2=0x55ac4e8815e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9a937fe700 0 client.7182 ms_handle_reset on v2:172.21.15.92:6833/1226709881 2023-05-16T07:18:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 shutdown_connections 2023-05-16T07:18:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 1 -- 172.21.15.92:0/1725117537 wait complete. 2023-05-16T07:18:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9acce55b80 20 client.7182 trim_cache size 0 max 16384 2023-05-16T07:18:22.276 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:18:22.276 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:18:22.277 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:18:22.277 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7017, 2023-05-16T07:18:22.277 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.8336593199999998, 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 37, 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 107, 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 107, 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.3721982189999995 2023-05-16T07:18:22.278 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:18:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9a9805ed90 msgr2=0x7f9a98061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9a9805ed90 0x7f9a98061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9ab0007fd0 tx=0x7f9ab0007d30).stop 2023-05-16T07:18:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 msgr2=0x7f9ab40a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 0x7f9ab40a54c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f9aac009df0 tx=0x7f9aac00b040).stop 2023-05-16T07:18:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 shutdown_connections 2023-05-16T07:18:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9a9805ed90 0x7f9a98061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9ab40a7be0 0x7f9ab40c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab40c8ea0 0x7f9ab40a54c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 --2- 172.21.15.92:0/2157883694 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9ab40bf2d0 0x7f9ab40c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 >> 172.21.15.92:0/2157883694 conn(0x7f9ab40622b0 msgr2=0x7f9ab4062690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 shutdown_connections 2023-05-16T07:18:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.258+0000 7f9aba261700 1 -- 172.21.15.92:0/2157883694 wait complete. 2023-05-16T07:18:22.283 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:18:22.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.380+0000 7f6499126700 1 Processor -- start 2023-05-16T07:18:22.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.380+0000 7f6499126700 1 -- start start 2023-05-16T07:18:22.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f6499126700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 0x7f6494155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f6499126700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64940c4480 con 0x7f64940c3300 2023-05-16T07:18:22.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f6499126700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64940c3870 con 0x7f6494066930 2023-05-16T07:18:22.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f6499126700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6494068ba0 con 0x7f64940c4910 2023-05-16T07:18:22.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f64940c3300 0x7f6494152470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58582/0 (socket says 172.21.15.92:58582) 2023-05-16T07:18:22.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649359e700 1 -- 172.21.15.92:0/2915347928 learned_addr learned my addr 172.21.15.92:0/2915347928 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f648a59c700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 0x7f6494155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301603553 0 0) 0x7f64940c4480 con 0x7f64940c3300 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64800033f0 con 0x7f64940c3300 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2740167277 0 0) 0x7f64800033f0 con 0x7f64940c3300 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 msgr2=0x7f6494155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 0x7f6494155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 >> v1:172.21.15.158:6789/0 conn(0x7f6494066930 legacy=0x7f6494068490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940c3b50 con 0x7f64940c3300 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.381+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f648c0017f0 con 0x7f64940c3300 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 >> v1:172.21.15.92:6789/0 conn(0x7f64940c3300 legacy=0x7f6494152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480003910 0x7f6480003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 0x7f6480004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6480004b80 0x7f6480005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648000b8e0 con 0x7f6480004b80 2023-05-16T07:18:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648000ba50 con 0x7f6480003910 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648000bc10 con 0x7f64940c4910 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6480004b80 0x7f6480005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f6492d9d700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480003910 0x7f6480003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f648a59c700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 0x7f6480004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 -- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 msgr2=0x7f6480004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940c4910 0x7f6480004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 -- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480003910 msgr2=0x7f6480003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480003910 0x7f6480003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 -- 172.21.15.92:0/2915347928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800033f0 con 0x7f6480004b80 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649359e700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6480004b80 0x7f6480005000 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f648c00c480 tx=0x7f648c00c790).ready entity=mon.0 client_cookie=40b7aac0b1792620 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c002030 con 0x7f6480004b80 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c00ce20 con 0x7f6480004b80 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.382+0000 7f649259c700 1 -- 172.21.15.92:0/2915347928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c005d30 con 0x7f6480004b80 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f6499126700 1 -- 172.21.15.92:0/2915347928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6480004b80 msgr2=0x7f6480005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f6499126700 1 --2- 172.21.15.92:0/2915347928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6480004b80 0x7f6480005000 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f648c00c480 tx=0x7f648c00c790).stop 2023-05-16T07:18:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f649359e700 1 -- 172.21.15.92:0/2915347928 reap_dead start 2023-05-16T07:18:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f6499126700 1 -- 172.21.15.92:0/2915347928 shutdown_connections 2023-05-16T07:18:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f6499126700 1 -- 172.21.15.92:0/2915347928 >> 172.21.15.92:0/2915347928 conn(0x7f6494062db0 msgr2=0x7f64940c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f6499126700 1 -- 172.21.15.92:0/2915347928 shutdown_connections 2023-05-16T07:18:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.383+0000 7f6499126700 1 -- 172.21.15.92:0/2915347928 wait complete. 2023-05-16T07:18:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.384+0000 7f6499126700 1 Processor -- start 2023-05-16T07:18:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.384+0000 7f6499126700 1 -- start start 2023-05-16T07:18:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480004b80 0x7f64940c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 0x7f64940abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940abf90 0x7f6494165380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64940c7de0 con 0x7f649413a020 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649414d0e0 con 0x7f6480004b80 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64940c7790 con 0x7f64940abf90 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 0x7f64940abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 0x7f64940abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47844/0 (socket says 172.21.15.92:47844) 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 -- 172.21.15.92:0/4060911574 learned_addr learned my addr 172.21.15.92:0/4060911574 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6492d9d700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480004b80 0x7f64940c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 -- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940abf90 msgr2=0x7f6494165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f649359e700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940abf90 0x7f6494165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940abf90 0x7f6494165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 -- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480004b80 msgr2=0x7f64940c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480004b80 0x7f64940c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 -- 172.21.15.92:0/4060911574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940c7a70 con 0x7f649413a020 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6492d9d700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480004b80 0x7f64940c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f649359e700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940abf90 0x7f6494165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648a59c700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 0x7f64940abaa0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f647c0119c0 tx=0x7f647c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c00f8e0 con 0x7f649413a020 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c00fa40 con 0x7f649413a020 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649414d3c0 con 0x7f649413a020 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c00fdc0 con 0x7f649413a020 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.385+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649413a660 con 0x7f649413a020 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.386+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649405efa0 con 0x7f649413a020 2023-05-16T07:18:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.387+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f647c024070 con 0x7f649413a020 2023-05-16T07:18:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.387+0000 7f648b7fe700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f647405edc0 0x7f6474061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.387+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f647c07a850 con 0x7f649413a020 2023-05-16T07:18:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.387+0000 7f6492d9d700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f647405edc0 0x7f6474061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.388+0000 7f6492d9d700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f647405edc0 0x7f6474061280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6484000c00 tx=0x7f6484009040).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.391+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f647c04bba0 con 0x7f649413a020 2023-05-16T07:18:22.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.595+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64940c3e30 con 0x7f649413a020 2023-05-16T07:18:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.595+0000 7f648b7fe700 1 -- 172.21.15.92:0/4060911574 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f647c04f1c0 con 0x7f649413a020 2023-05-16T07:18:22.599 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:18:22.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f647405edc0 msgr2=0x7f6474061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f647405edc0 0x7f6474061280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6484000c00 tx=0x7f6484009040).stop 2023-05-16T07:18:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 msgr2=0x7f64940abaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 0x7f64940abaa0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f647c0119c0 tx=0x7f647c009060).stop 2023-05-16T07:18:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 shutdown_connections 2023-05-16T07:18:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f647405edc0 0x7f6474061280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f64940abf90 0x7f6494165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6480004b80 0x7f64940c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 --2- 172.21.15.92:0/4060911574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649413a020 0x7f64940abaa0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 >> 172.21.15.92:0/4060911574 conn(0x7f6494062db0 msgr2=0x7f6494154b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 shutdown_connections 2023-05-16T07:18:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:22.597+0000 7f6499126700 1 -- 172.21.15.92:0/4060911574 wait complete. 2023-05-16T07:18:22.611 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:18:22.612 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:18:22.612 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:18:22.613 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:18:22.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:18:22.613 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:18:22.614 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:18:22.614 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:18:22.615 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:18:22.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.616 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41052sec preferred_lft 41052sec 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:18:22.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.638 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:18:22.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.697 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:18:22.754 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:18:22.755 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:18:22.755 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:18:22.755 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:18:22.755 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:18:22.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.756 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:18:22.757 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:18:22.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:18:22.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:18:22.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:18:22.758 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:18:22.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.852 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:18:22.866 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:18:22.867 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:18:22.867 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:18:22.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:18:22.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:22.930 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:18:22.985 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:18:22.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:18:23.062 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:18:23.063 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:18:23.105 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:18:23.105 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:18:23.107 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:18:23.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:18:23.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:18:23.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:18:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.318+0000 7f4d329ac700 1 Processor -- start 2023-05-16T07:18:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.319+0000 7f4d329ac700 1 -- start start 2023-05-16T07:18:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.319+0000 7f4d329ac700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 0x7f4d2c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.319+0000 7f4d329ac700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4d2c153b00 0x7f4d2c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.319+0000 7f4d329ac700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2c06a920 con 0x7f4d2c0aab40 2023-05-16T07:18:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.319+0000 7f4d329ac700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c0c65f0 con 0x7f4d2c153b00 2023-05-16T07:18:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.319+0000 7f4d329ac700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0bfa30 con 0x7f4d2c069750 2023-05-16T07:18:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 0x7f4d2c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d321ab700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4d2c153b00 0x7f4d2c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 0x7f4d2c069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:40430/0 (socket says 172.21.15.92:40430) 2023-05-16T07:18:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 -- 172.21.15.92:0/350547341 learned_addr learned my addr 172.21.15.92:0/350547341 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 -- 172.21.15.92:0/350547341 >> v2:172.21.15.158:3300/0 conn(0x7f4d2c153b00 msgr2=0x7f4d2c153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 --2- 172.21.15.92:0/350547341 >> v2:172.21.15.158:3300/0 conn(0x7f4d2c153b00 0x7f4d2c153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 -- 172.21.15.92:0/350547341 >> v1:172.21.15.158:6789/0 conn(0x7f4d2c0aab40 legacy=0x7f4d2c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 -- 172.21.15.92:0/350547341 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0bfc80 con 0x7f4d2c069750 2023-05-16T07:18:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d319aa700 1 --2- 172.21.15.92:0/350547341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 0x7f4d2c069b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4d1c009f20 tx=0x7f4d1c009840).ready entity=mon.2 client_cookie=4d0721fd6a75e163 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:23.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d309a8700 1 -- 172.21.15.92:0/350547341 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00e070 con 0x7f4d2c069750 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.320+0000 7f4d309a8700 1 -- 172.21.15.92:0/350547341 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00b9d0 con 0x7f4d2c069750 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d309a8700 1 -- 172.21.15.92:0/350547341 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00bd50 con 0x7f4d2c069750 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 -- 172.21.15.92:0/350547341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 msgr2=0x7f4d2c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 --2- 172.21.15.92:0/350547341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 0x7f4d2c069b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4d1c009f20 tx=0x7f4d1c009840).stop 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 -- 172.21.15.92:0/350547341 shutdown_connections 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 --2- 172.21.15.92:0/350547341 >> v2:172.21.15.158:3300/0 conn(0x7f4d2c153b00 0x7f4d2c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 --2- 172.21.15.92:0/350547341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c069750 0x7f4d2c069b30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 -- 172.21.15.92:0/350547341 >> 172.21.15.92:0/350547341 conn(0x7f4d2c0cb550 msgr2=0x7f4d2c0c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:23.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 -- 172.21.15.92:0/350547341 shutdown_connections 2023-05-16T07:18:23.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.321+0000 7f4d329ac700 1 -- 172.21.15.92:0/350547341 wait complete. 2023-05-16T07:18:23.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.322+0000 7f4d329ac700 1 Processor -- start 2023-05-16T07:18:23.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.322+0000 7f4d329ac700 1 -- start start 2023-05-16T07:18:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d329ac700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d2c069750 0x7f4d2c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d329ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 0x7f4d2c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d329ac700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c153b00 0x7f4d2c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d329ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c152c20 con 0x7f4d2c0aab40 2023-05-16T07:18:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d329ac700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1535a0 con 0x7f4d2c069750 2023-05-16T07:18:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d329ac700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c152010 con 0x7f4d2c153b00 2023-05-16T07:18:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 0x7f4d2c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 0x7f4d2c1530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47864/0 (socket says 172.21.15.92:47864) 2023-05-16T07:18:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 -- 172.21.15.92:0/3737629327 learned_addr learned my addr 172.21.15.92:0/3737629327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d321ab700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c153b00 0x7f4d2c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d319aa700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d2c069750 0x7f4d2c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 -- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c153b00 msgr2=0x7f4d2c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c153b00 0x7f4d2c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 -- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d2c069750 msgr2=0x7f4d2c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d2c069750 0x7f4d2c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d311a9700 1 -- 172.21.15.92:0/3737629327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c009ae0 con 0x7f4d2c0aab40 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.323+0000 7f4d319aa700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d2c069750 0x7f4d2c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.324+0000 7f4d311a9700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 0x7f4d2c1530b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f4d2000c440 tx=0x7f4d20007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.324+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20007020 con 0x7f4d2c0aab40 2023-05-16T07:18:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.324+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20007180 con 0x7f4d2c0aab40 2023-05-16T07:18:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.324+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c1522f0 con 0x7f4d2c0aab40 2023-05-16T07:18:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.324+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20007520 con 0x7f4d2c0aab40 2023-05-16T07:18:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.324+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c1528b0 con 0x7f4d2c0aab40 2023-05-16T07:18:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.325+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c083040 con 0x7f4d2c0aab40 2023-05-16T07:18:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.325+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4d20010070 con 0x7f4d2c0aab40 2023-05-16T07:18:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.325+0000 7f4d29ffb700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1805ed90 0x7f4d18061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.325+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f4d20079050 con 0x7f4d2c0aab40 2023-05-16T07:18:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.326+0000 7f4d319aa700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1805ed90 0x7f4d18061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.326+0000 7f4d319aa700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1805ed90 0x7f4d18061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4d1c0072d0 tx=0x7f4d1c009840).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:23.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.329+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4d200478f0 con 0x7f4d2c0aab40 2023-05-16T07:18:23.350 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:18:23.346+0000 7f900c740700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:18:23.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.518+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d2c0bff60 con 0x7f4d1805ed90 2023-05-16T07:18:23.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.520+0000 7f4d29ffb700 1 -- 172.21.15.92:0/3737629327 <== mgr.6373 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4d2c0bff60 con 0x7f4d1805ed90 2023-05-16T07:18:23.525 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:18:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:18:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:18:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:18:23.525 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:18:23.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1805ed90 msgr2=0x7f4d18061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1805ed90 0x7f4d18061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4d1c0072d0 tx=0x7f4d1c009840).stop 2023-05-16T07:18:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 msgr2=0x7f4d2c1530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 0x7f4d2c1530b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f4d2000c440 tx=0x7f4d20007da0).stop 2023-05-16T07:18:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 shutdown_connections 2023-05-16T07:18:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1805ed90 0x7f4d18061250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d2c153b00 0x7f4d2c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d2c069750 0x7f4d2c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 --2- 172.21.15.92:0/3737629327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d2c0aab40 0x7f4d2c1530b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 >> 172.21.15.92:0/3737629327 conn(0x7f4d2c0cb550 msgr2=0x7f4d2c0aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 shutdown_connections 2023-05-16T07:18:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.522+0000 7f4d329ac700 1 -- 172.21.15.92:0/3737629327 wait complete. 2023-05-16T07:18:23.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:18:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.646+0000 7fc6f685c700 1 Processor -- start 2023-05-16T07:18:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.646+0000 7fc6f685c700 1 -- start start 2023-05-16T07:18:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6f685c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc6f0154470 0x7fc6f00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6f685c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f00c94d0 con 0x7fc6f0154090 2023-05-16T07:18:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6f685c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f00cb7e0 con 0x7fc6f0068160 2023-05-16T07:18:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6f685c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00c88c0 con 0x7fc6f0154470 2023-05-16T07:18:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6effff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc6f0154090 0x7fc6f00a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58618/0 (socket says 172.21.15.92:58618) 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6effff700 1 -- 172.21.15.92:0/2022049950 learned_addr learned my addr 172.21.15.92:0/2022049950 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.647+0000 7fc6ef7fe700 1 --2- 172.21.15.92:0/2022049950 >> v2:172.21.15.158:3300/0 conn(0x7fc6f0154470 0x7fc6f00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279917045 0 0) 0x7fc6f00c94d0 con 0x7fc6f0154090 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d80033f0 con 0x7fc6f0154090 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356783506 0 0) 0x7fc6f00cb7e0 con 0x7fc6f0068160 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f00c94d0 con 0x7fc6f0068160 2023-05-16T07:18:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834037336 0 0) 0x7fc6d80033f0 con 0x7fc6f0154090 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 >> v2:172.21.15.158:3300/0 conn(0x7fc6f0154470 msgr2=0x7fc6f00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 --2- 172.21.15.92:0/2022049950 >> v2:172.21.15.158:3300/0 conn(0x7fc6f0154470 0x7fc6f00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 >> v1:172.21.15.158:6789/0 conn(0x7fc6f0068160 legacy=0x7fc6f00c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00c8ba0 con 0x7fc6f0154090 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc6e0001470 con 0x7fc6f0154090 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 >> v1:172.21.15.92:6789/0 conn(0x7fc6f0154090 legacy=0x7fc6f00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6d8003a70 0x7fc6d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8004690 0x7fc6d8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6d8005000 0x7fc6d800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00c88c0 con 0x7fc6d8005000 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800bf60 con 0x7fc6d8003a70 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800c0d0 con 0x7fc6d8004690 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6d8005000 0x7fc6d800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 -- 172.21.15.92:0/2022049950 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8004690 msgr2=0x7fc6d8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8004690 0x7fc6d8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 -- 172.21.15.92:0/2022049950 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6d8003a70 msgr2=0x7fc6d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6d8003a70 0x7fc6d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 -- 172.21.15.92:0/2022049950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80033f0 con 0x7fc6d8005000 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 -- 172.21.15.92:0/2022049950 reap_dead start 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.648+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6d8005000 0x7fc6d800b8e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7fc6e8001f90 tx=0x7fc6e8007f30).ready entity=mon.0 client_cookie=410d7b1333e195d1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8006a20 con 0x7fc6d8005000 2023-05-16T07:18:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8006b80 con 0x7fc6d8005000 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6eeffd700 1 -- 172.21.15.92:0/2022049950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800c8b0 con 0x7fc6d8005000 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 -- 172.21.15.92:0/2022049950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6d8005000 msgr2=0x7fc6d800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6d8005000 0x7fc6d800b8e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7fc6e8001f90 tx=0x7fc6e8007f30).stop 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 -- 172.21.15.92:0/2022049950 shutdown_connections 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 --2- 172.21.15.92:0/2022049950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6d8005000 0x7fc6d800b8e0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 -- 172.21.15.92:0/2022049950 >> 172.21.15.92:0/2022049950 conn(0x7fc6f0067920 msgr2=0x7fc6f00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 -- 172.21.15.92:0/2022049950 shutdown_connections 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.649+0000 7fc6f685c700 1 -- 172.21.15.92:0/2022049950 wait complete. 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.650+0000 7fc6f685c700 1 Processor -- start 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.650+0000 7fc6f685c700 1 -- start start 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f685c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8005000 0x7fc6f01417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f685c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6f0156a10 0x7fc6f0164320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f685c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 0x7fc6f00abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f685c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0154850 con 0x7fc6f00c25d0 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f685c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0154e10 con 0x7fc6f0156a10 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f685c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0156f50 con 0x7fc6d8005000 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f4df9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 0x7fc6f00abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f4df9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 0x7fc6f00abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47886/0 (socket says 172.21.15.92:47886) 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6f4df9700 1 -- 172.21.15.92:0/3023973969 learned_addr learned my addr 172.21.15.92:0/3023973969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6ef7fe700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6f0156a10 0x7fc6f0164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.651+0000 7fc6effff700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8005000 0x7fc6f01417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f4df9700 1 -- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8005000 msgr2=0x7fc6f01417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8005000 0x7fc6f01417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f4df9700 1 -- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6f0156a10 msgr2=0x7fc6f0164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6f0156a10 0x7fc6f0164320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f4df9700 1 -- 172.21.15.92:0/3023973969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f0157230 con 0x7fc6f00c25d0 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6effff700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8005000 0x7fc6f01417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f4df9700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 0x7fc6f00abc90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7fc6f00cbca0 tx=0x7fc6e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8006a20 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e800c8b0 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e801f690 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f0154aa0 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.652+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f0164810 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.653+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f0066730 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.653+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc6e8010070 con 0x7fc6f00c25d0 2023-05-16T07:18:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.653+0000 7fc6d7fff700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6dc05ed90 0x7fc6dc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:23.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.654+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7fc6e807b1c0 con 0x7fc6f00c25d0 2023-05-16T07:18:23.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.654+0000 7fc6effff700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6dc05ed90 0x7fc6dc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:23.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.654+0000 7fc6effff700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6dc05ed90 0x7fc6dc061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc6e0001660 tx=0x7fc6e0001500).ready entity=mgr.6373 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:23.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.658+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc6e804c510 con 0x7fc6f00c25d0 2023-05-16T07:18:23.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:23.861+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc6f0154f80 con 0x7fc6f00c25d0 2023-05-16T07:18:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.669+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 39) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fc6e8078e70 con 0x7fc6f00c25d0 2023-05-16T07:18:24.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.671+0000 7fc6d7fff700 1 -- 172.21.15.92:0/3023973969 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v39) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc6e8040ed0 con 0x7fc6f00c25d0 2023-05-16T07:18:24.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.673+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6dc05ed90 msgr2=0x7fc6dc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:24.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.673+0000 7fc6f685c700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6dc05ed90 0x7fc6dc061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc6e0001660 tx=0x7fc6e0001500).stop 2023-05-16T07:18:24.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.673+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 msgr2=0x7fc6f00abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:24.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.673+0000 7fc6f685c700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 0x7fc6f00abc90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7fc6f00cbca0 tx=0x7fc6e8015000).stop 2023-05-16T07:18:24.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 shutdown_connections 2023-05-16T07:18:24.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6dc05ed90 0x7fc6dc061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6d8005000 0x7fc6f01417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6f00c25d0 0x7fc6f00abc90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 --2- 172.21.15.92:0/3023973969 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6f0156a10 0x7fc6f0164320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 >> 172.21.15.92:0/3023973969 conn(0x7fc6f0067920 msgr2=0x7fc6f014e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 shutdown_connections 2023-05-16T07:18:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.674+0000 7fc6f685c700 1 -- 172.21.15.92:0/3023973969 wait complete. 2023-05-16T07:18:24.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:18:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.808+0000 7f598667f700 1 Processor -- start 2023-05-16T07:18:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.809+0000 7f598667f700 1 -- start start 2023-05-16T07:18:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.809+0000 7f598667f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5980153f90 0x7f59800a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:24.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598667f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5980155380 0x7f59800c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598667f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800c66a0 con 0x7f59800c5d00 2023-05-16T07:18:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598667f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5980069f60 con 0x7f5980153f90 2023-05-16T07:18:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598667f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598006a9e0 con 0x7f5980155380 2023-05-16T07:18:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f5984e7c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f59800c5d00 0x7f59800c0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58664/0 (socket says 172.21.15.92:58664) 2023-05-16T07:18:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f5984e7c700 1 -- 172.21.15.92:0/2951771738 learned_addr learned my addr 172.21.15.92:0/2951771738 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598567d700 1 --2- 172.21.15.92:0/2951771738 >> v2:172.21.15.158:3300/0 conn(0x7f5980153f90 0x7f59800a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f5985e7e700 1 --2- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5980155380 0x7f59800c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f5973fff700 1 -- 172.21.15.92:0/2951771738 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921997725 0 0) 0x7f597c001110 con 0x7f59800c5d00 2023-05-16T07:18:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f5973fff700 1 -- 172.21.15.92:0/2951771738 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c0033f0 con 0x7f59800c5d00 2023-05-16T07:18:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598567d700 1 -- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5980155380 msgr2=0x7f59800c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598567d700 1 --2- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5980155380 0x7f59800c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598567d700 1 -- 172.21.15.92:0/2951771738 >> v1:172.21.15.92:6789/0 conn(0x7f59800c5d00 legacy=0x7f59800c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598567d700 1 -- 172.21.15.92:0/2951771738 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598006a140 con 0x7f5980153f90 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f598567d700 1 --2- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800a9d20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f5974009760 tx=0x7f5974009060).ready entity=mon.1 client_cookie=b22c3f84a54dedc0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.810+0000 7f5973fff700 1 -- 172.21.15.92:0/2951771738 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400d4a0 con 0x7f5980153f90 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f5973fff700 1 -- 172.21.15.92:0/2951771738 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597400d600 con 0x7f5980153f90 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f5973fff700 1 -- 172.21.15.92:0/2951771738 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400d980 con 0x7f5980153f90 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 -- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 msgr2=0x7f59800a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 --2- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800a9d20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f5974009760 tx=0x7f5974009060).stop 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 -- 172.21.15.92:0/2951771738 shutdown_connections 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 --2- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5980155380 0x7f59800c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 --2- 172.21.15.92:0/2951771738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800a9d20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 -- 172.21.15.92:0/2951771738 >> 172.21.15.92:0/2951771738 conn(0x7f59800cb600 msgr2=0x7f59800c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 -- 172.21.15.92:0/2951771738 shutdown_connections 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.811+0000 7f598667f700 1 -- 172.21.15.92:0/2951771738 wait complete. 2023-05-16T07:18:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.812+0000 7f598667f700 1 Processor -- start 2023-05-16T07:18:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.812+0000 7f598667f700 1 -- start start 2023-05-16T07:18:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598667f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59800c5d00 0x7f5980143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598667f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598667f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 0x7f59800ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598667f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800c2540 con 0x7f5980155380 2023-05-16T07:18:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598667f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800c2820 con 0x7f5980153f90 2023-05-16T07:18:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598667f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800c2b00 con 0x7f59800c5d00 2023-05-16T07:18:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f5985e7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 0x7f59800ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f5984e7c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f5985e7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 0x7f59800ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47924/0 (socket says 172.21.15.92:47924) 2023-05-16T07:18:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f598567d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59800c5d00 0x7f5980143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.813+0000 7f5985e7e700 1 -- 172.21.15.92:0/151224382 learned_addr learned my addr 172.21.15.92:0/151224382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5985e7e700 1 -- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59800c5d00 msgr2=0x7f5980143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5985e7e700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59800c5d00 0x7f5980143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5985e7e700 1 -- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 msgr2=0x7f59800c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5985e7e700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5985e7e700 1 -- 172.21.15.92:0/151224382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740092a0 con 0x7f5980155380 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5984e7c700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:18:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f5985e7e700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 0x7f59800ac510 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f5978009df0 tx=0x7f597800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597800caa0 con 0x7f5980155380 2023-05-16T07:18:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597800f040 con 0x7f5980155380 2023-05-16T07:18:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597800ce10 con 0x7f5980155380 2023-05-16T07:18:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59800c36f0 con 0x7f5980155380 2023-05-16T07:18:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.814+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5980143c50 con 0x7f5980155380 2023-05-16T07:18:24.827 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:24.815+0000 7f3b18568680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:18:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.815+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5980069d20 con 0x7f5980155380 2023-05-16T07:18:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.815+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f5978013610 con 0x7f5980155380 2023-05-16T07:18:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.816+0000 7f59717fa700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f596405ede0 0x7f59640612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.816+0000 7f598567d700 1 -- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f596405ede0 msgr2=0x7f59640612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.816+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f597807a730 con 0x7f5980155380 2023-05-16T07:18:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.816+0000 7f598567d700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f596405ede0 0x7f59640612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.820+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f597804b970 con 0x7f5980155380 2023-05-16T07:18:24.829 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:24.812+0000 7f12e5e10680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:18:24.878 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:24.875+0000 7f12e5e10680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:18:24.883 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:24.879+0000 7f3b18568680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:18:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.991+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f5980083120 con 0x7f5980155380 2023-05-16T07:18:24.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.994+0000 7f59717fa700 1 -- 172.21.15.92:0/151224382 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v19)=0 v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f597804ef90 con 0x7f5980155380 2023-05-16T07:18:25.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.996+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f596405ede0 msgr2=0x7f59640612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:25.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.996+0000 7f598667f700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f596405ede0 0x7f59640612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:25.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.996+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 msgr2=0x7f59800ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:25.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.996+0000 7f598667f700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 0x7f59800ac510 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f5978009df0 tx=0x7f597800b040).stop 2023-05-16T07:18:25.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.996+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 shutdown_connections 2023-05-16T07:18:25.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.996+0000 7f598667f700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f596405ede0 0x7f59640612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:25.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.997+0000 7f598667f700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59800c5d00 0x7f5980143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.997+0000 7f598667f700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5980155380 0x7f59800ac510 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.997+0000 7f598667f700 1 --2- 172.21.15.92:0/151224382 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5980153f90 0x7f59800c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.997+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 >> 172.21.15.92:0/151224382 conn(0x7f59800cb600 msgr2=0x7f59800c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:25.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.997+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 shutdown_connections 2023-05-16T07:18:25.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:24.997+0000 7f598667f700 1 -- 172.21.15.92:0/151224382 wait complete. 2023-05-16T07:18:25.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:25.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:18:25.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-05-16T07:18:25.151 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:25.147+0000 7f12e5e10680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:18:25.168 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:25.165+0000 7f3b18568680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:18:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.190+0000 7f76e542b700 1 Processor -- start 2023-05-16T07:18:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.190+0000 7f76e542b700 1 -- start start 2023-05-16T07:18:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.190+0000 7f76e542b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76e0151e40 0x7f76e00bd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.190+0000 7f76e542b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e0152220 0x7f76e00c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76e542b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e005e3a0 con 0x7f76e00a6680 2023-05-16T07:18:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76e542b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00c38e0 con 0x7f76e0151e40 2023-05-16T07:18:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76e542b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00bc5b0 con 0x7f76e0152220 2023-05-16T07:18:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76e0151e40 0x7f76e00bd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76e0151e40 0x7f76e00bd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47972/0 (socket says 172.21.15.92:47972) 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 -- 172.21.15.92:0/3076476910 learned_addr learned my addr 172.21.15.92:0/3076476910 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76df7fe700 1 --1- 172.21.15.92:0/3076476910 >> v1:172.21.15.92:6789/0 conn(0x7f76e00a6680 0x7f76e0068ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58724/0 (socket says 172.21.15.92:58724) 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76de7fc700 1 --2- 172.21.15.92:0/3076476910 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e0152220 0x7f76e00c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 -- 172.21.15.92:0/3076476910 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e0152220 msgr2=0x7f76e00c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 --2- 172.21.15.92:0/3076476910 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e0152220 0x7f76e00c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 -- 172.21.15.92:0/3076476910 >> v1:172.21.15.92:6789/0 conn(0x7f76e00a6680 legacy=0x7f76e0068ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 -- 172.21.15.92:0/3076476910 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00bbff0 con 0x7f76e0151e40 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76ddffb700 1 -- 172.21.15.92:0/3076476910 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263656154 0 0) 0x7f76e005e3a0 con 0x7f76e00a6680 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76deffd700 1 --2- 172.21.15.92:0/3076476910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0151e40 0x7f76e00bd9a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f76d0006fd0 tx=0x7f76d00090f0).ready entity=mon.0 client_cookie=8d71d0f2a0d4da3a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76ddffb700 1 -- 172.21.15.92:0/3076476910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000b600 con 0x7f76e0151e40 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76ddffb700 1 -- 172.21.15.92:0/3076476910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d000b760 con 0x7f76e0151e40 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.191+0000 7f76ddffb700 1 -- 172.21.15.92:0/3076476910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000bae0 con 0x7f76e0151e40 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 -- 172.21.15.92:0/3076476910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0151e40 msgr2=0x7f76e00bd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 --2- 172.21.15.92:0/3076476910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0151e40 0x7f76e00bd9a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f76d0006fd0 tx=0x7f76d00090f0).stop 2023-05-16T07:18:25.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 -- 172.21.15.92:0/3076476910 shutdown_connections 2023-05-16T07:18:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 --2- 172.21.15.92:0/3076476910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0151e40 0x7f76e00bd9a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 --2- 172.21.15.92:0/3076476910 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e0152220 0x7f76e00c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 -- 172.21.15.92:0/3076476910 >> 172.21.15.92:0/3076476910 conn(0x7f76e00c4600 msgr2=0x7f76e0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 -- 172.21.15.92:0/3076476910 shutdown_connections 2023-05-16T07:18:25.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.192+0000 7f76e542b700 1 -- 172.21.15.92:0/3076476910 wait complete. 2023-05-16T07:18:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.193+0000 7f76e542b700 1 Processor -- start 2023-05-16T07:18:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.193+0000 7f76e542b700 1 -- start start 2023-05-16T07:18:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76e542b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e00a6680 0x7f76e0152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76e542b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f76e0151e40 0x7f76e0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76e542b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 0x7f76e014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76e542b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00aa080 con 0x7f76e0152220 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76e542b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e014d030 con 0x7f76e0151e40 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76e542b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00acc90 con 0x7f76e00a6680 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 0x7f76e014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 0x7f76e014bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47982/0 (socket says 172.21.15.92:47982) 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 -- 172.21.15.92:0/1110019241 learned_addr learned my addr 172.21.15.92:0/1110019241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e00a6680 0x7f76e0152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76de7fc700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f76e0151e40 0x7f76e0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e00a6680 msgr2=0x7f76e0152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e00a6680 0x7f76e0152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f76e0151e40 msgr2=0x7f76e0154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f76e0151e40 0x7f76e0154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 -- 172.21.15.92:0/1110019241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0009370 con 0x7f76e0152220 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.194+0000 7f76df7fe700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 0x7f76e014bf90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f76d800c810 tx=0x7f76d8007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:25.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.195+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8005c70 con 0x7f76e0152220 2023-05-16T07:18:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.195+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8005dd0 con 0x7f76e0152220 2023-05-16T07:18:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.195+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e00acf70 con 0x7f76e0152220 2023-05-16T07:18:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.195+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8007360 con 0x7f76e0152220 2023-05-16T07:18:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.195+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e00aa600 con 0x7f76e0152220 2023-05-16T07:18:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.195+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e006c350 con 0x7f76e0152220 2023-05-16T07:18:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.196+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f76d8010030 con 0x7f76e0152220 2023-05-16T07:18:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.196+0000 7f76c6ffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.196+0000 7f76deffd700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.196+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f76d8079c90 con 0x7f76e0152220 2023-05-16T07:18:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.196+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:18:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.200+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f76d804af60 con 0x7f76e0152220 2023-05-16T07:18:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.396+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f76e006a540 con 0x7f76cc05ede0 2023-05-16T07:18:25.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.397+0000 7f76deffd700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.397+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:18:25.757 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:25.753+0000 7f12e5e10680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:18:25.783 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:25.779+0000 7f3b18568680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:18:25.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.798+0000 7f76deffd700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:25.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:25.798+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:18:25.828 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:25.824+0000 7f12e5e10680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:18:25.853 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:25.850+0000 7f3b18568680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:18:26.012 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:26.008+0000 7f12e5e10680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:18:26.036 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:26.033+0000 7f3b18568680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:18:26.425 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:26.422+0000 7f12e5e10680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:18:26.447 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:26.443+0000 7f3b18568680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:18:26.526 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:26.523+0000 7f12e5e10680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:26.547 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:26.544+0000 7f3b18568680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:26.599+0000 7f76deffd700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:26.599+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:18:26.677 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:26.674+0000 7f12e5e10680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:26.697 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:26.693+0000 7f3b18568680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:26.750 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:26.747+0000 7f12e5e10680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:18:26.769 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:26.766+0000 7f3b18568680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:18:26.816 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:26.812+0000 7f12e5e10680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:18:26.834 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:26.831+0000 7f3b18568680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:18:27.068 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:27.065+0000 7f12e5e10680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:18:27.088 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:27.084+0000 7f3b18568680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:18:27.249 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:27.246+0000 7f12e5e10680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:27.268 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:27.264+0000 7f3b18568680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:18:27.887 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:27.884+0000 7f12e5e10680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:18:27.904 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:27.900+0000 7f3b18568680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:18:27.955 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:27.951+0000 7f12e5e10680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:18:27.971 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:27.967+0000 7f3b18568680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:18:28.028 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.025+0000 7f12e5e10680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:18:28.045 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.041+0000 7f3b18568680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:18:28.172 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.168+0000 7f12e5e10680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:18:28.186 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.183+0000 7f3b18568680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:18:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:28.201+0000 7f76deffd700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:18:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:28.201+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:18:28.240 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.237+0000 7f12e5e10680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:18:28.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.252+0000 7f3b18568680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:18:28.352 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.349+0000 7f12e5e10680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:18:28.368 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.365+0000 7f3b18568680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:18:28.458 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.454+0000 7f12e5e10680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:28.474 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.470+0000 7f3b18568680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:18:28.747 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.743+0000 7f12e5e10680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:18:28.759 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.755+0000 7f3b18568680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:18:28.816 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:28.813+0000 7f12e5e10680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:18:28.827 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:18:28.824+0000 7f3b18568680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:18:29.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:29.656+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 40) v1 ==== 75109+0+0 (secure 0 0 0) 0x7f76d80778c0 con 0x7f76e0152220 2023-05-16T07:18:29.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:29.657+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:18:29.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:29.657+0000 7f76c6ffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:18:29.721 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12645f2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.721 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12615ec700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.721 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12645f2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.721 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12615ec700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.721 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12645f2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.721 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12615ec700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.722 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12645f2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.722 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12645f2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.722 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12615ec700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:29.722 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:18:29.717+0000 7f12615ec700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:18:30.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.661+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 41) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f76d8077c40 con 0x7f76e0152220 2023-05-16T07:18:30.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.661+0000 7f76c6ffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc02fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.661+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f76e006a540 con 0x7f76cc05ede0 2023-05-16T07:18:30.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.662+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc02fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.663+0000 7f76deffd700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc02fd40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f76d0009130 tx=0x7f76d0017000).ready entity=mgr.6167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.678+0000 7f76c6ffd700 1 -- 172.21.15.92:0/1110019241 <== mgr.6167 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f76e006a540 con 0x7f76cc05ede0 2023-05-16T07:18:30.682 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path snap_test_dir1 2023-05-16T07:18:30.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 msgr2=0x7f76cc02fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc02fd40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f76d0009130 tx=0x7f76d0017000).stop 2023-05-16T07:18:30.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 msgr2=0x7f76e014bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 0x7f76e014bf90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f76d800c810 tx=0x7f76d8007f10).stop 2023-05-16T07:18:30.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 shutdown_connections 2023-05-16T07:18:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f76cc05ede0 0x7f76cc02fd40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f76e00a6680 0x7f76e0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e0152220 0x7f76e014bf90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 --2- 172.21.15.92:0/1110019241 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f76e0151e40 0x7f76e0154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.680+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 >> 172.21.15.92:0/1110019241 conn(0x7f76e00c4600 msgr2=0x7f76e014eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:30.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.681+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 shutdown_connections 2023-05-16T07:18:30.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.681+0000 7f76e542b700 1 -- 172.21.15.92:0/1110019241 wait complete. 2023-05-16T07:18:30.698 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-05-16T07:18:30.698 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-05-16-07_19_00 in ~30s... 2023-05-16T07:18:30.699 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-05-16T07:18:30.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-05-16T07:18:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.816+0000 7f0e3d473700 1 Processor -- start 2023-05-16T07:18:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.817+0000 7f0e3d473700 1 -- start start 2023-05-16T07:18:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.817+0000 7f0e3d473700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e380c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.817+0000 7f0e3d473700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e380c6660 con 0x7f0e380c61f0 2023-05-16T07:18:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.817+0000 7f0e3d473700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3806a140 con 0x7f0e38153f90 2023-05-16T07:18:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.817+0000 7f0e3d473700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38069f60 con 0x7f0e380c1160 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e3cc72700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0e380c61f0 0x7f0e38154370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58856/0 (socket says 172.21.15.92:58856) 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e3cc72700 1 -- 172.21.15.92:0/2330565484 learned_addr learned my addr 172.21.15.92:0/2330565484 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e380c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687981937 0 0) 0x7f0e380c6660 con 0x7f0e380c61f0 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e200033f0 con 0x7f0e380c61f0 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1348588678 0 0) 0x7f0e200033f0 con 0x7f0e380c61f0 2023-05-16T07:18:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 msgr2=0x7f0e380c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e380c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 >> v1:172.21.15.158:6789/0 conn(0x7f0e38153f90 legacy=0x7f0e380a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3806a420 con 0x7f0e380c61f0 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f0e300017f0 con 0x7f0e380c61f0 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 >> v1:172.21.15.92:6789/0 conn(0x7f0e380c61f0 legacy=0x7f0e38154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e20003910 0x7f0e20003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e20004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e20005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38069f60 con 0x7f0e20004690 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000b8e0 con 0x7f0e20003910 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000ba50 con 0x7f0e380c1160 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e20004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e37fff700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e20003910 0x7f0e20003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e3cc72700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e20005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 -- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 msgr2=0x7f0e20005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e20005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 -- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e20003910 msgr2=0x7f0e20003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e20003910 0x7f0e20003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e377fe700 1 -- 172.21.15.92:0/2330565484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200033f0 con 0x7f0e20004690 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e37fff700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e20003910 0x7f0e20003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.818+0000 7f0e3cc72700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380c1160 0x7f0e20005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e377fe700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e20004b10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f0e2c009a60 tx=0x7f0e2c009060).ready entity=mon.0 client_cookie=8a5e173e2519cb65 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c009ea0 con 0x7f0e20004690 2023-05-16T07:18:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c009430 con 0x7f0e20004690 2023-05-16T07:18:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e36ffd700 1 -- 172.21.15.92:0/2330565484 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00e810 con 0x7f0e20004690 2023-05-16T07:18:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3d473700 1 -- 172.21.15.92:0/2330565484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 msgr2=0x7f0e20004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3d473700 1 --2- 172.21.15.92:0/2330565484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e20004b10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f0e2c009a60 tx=0x7f0e2c009060).stop 2023-05-16T07:18:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3cc72700 1 -- 172.21.15.92:0/2330565484 reap_dead start 2023-05-16T07:18:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3d473700 1 -- 172.21.15.92:0/2330565484 shutdown_connections 2023-05-16T07:18:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3d473700 1 -- 172.21.15.92:0/2330565484 >> 172.21.15.92:0/2330565484 conn(0x7f0e380cb600 msgr2=0x7f0e380c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3d473700 1 -- 172.21.15.92:0/2330565484 shutdown_connections 2023-05-16T07:18:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.819+0000 7f0e3d473700 1 -- 172.21.15.92:0/2330565484 wait complete. 2023-05-16T07:18:30.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.820+0000 7f0e3d473700 1 Processor -- start 2023-05-16T07:18:30.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 -- start start 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e3813bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e38156ce0 0x7f0e381584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380abf70 0x7f0e380ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380ac8c0 con 0x7f0e20004690 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38158e40 con 0x7f0e38156ce0 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3d473700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38159120 con 0x7f0e380abf70 2023-05-16T07:18:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e3813bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e3813bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48116/0 (socket says 172.21.15.92:48116) 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e37fff700 1 -- 172.21.15.92:0/4130111230 learned_addr learned my addr 172.21.15.92:0/4130111230 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e3cc72700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380abf70 0x7f0e380ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.821+0000 7f0e377fe700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e38156ce0 0x7f0e381584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e37fff700 1 -- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380abf70 msgr2=0x7f0e380ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e37fff700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380abf70 0x7f0e380ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e37fff700 1 -- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e38156ce0 msgr2=0x7f0e381584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e37fff700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e38156ce0 0x7f0e381584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e37fff700 1 -- 172.21.15.92:0/4130111230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38159400 con 0x7f0e20004690 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e377fe700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e38156ce0 0x7f0e381584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e37fff700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e3813bea0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f0e2800e660 tx=0x7f0e28009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28010520 con 0x7f0e20004690 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28010680 con 0x7f0e20004690 2023-05-16T07:18:30.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e381596e0 con 0x7f0e20004690 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28010a00 con 0x7f0e20004690 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.822+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3813c390 con 0x7f0e20004690 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.823+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e38083120 con 0x7f0e20004690 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.823+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 41) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f0e28014430 con 0x7f0e20004690 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.823+0000 7f0e1ffff700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e2405ed40 0x7f0e24061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.824+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f0e2807a070 con 0x7f0e20004690 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.824+0000 7f0e377fe700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e2405ed40 0x7f0e24061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:18:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.824+0000 7f0e377fe700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e2405ed40 0x7f0e24061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0e38167940 tx=0x7f0e2c01a000).ready entity=mgr.6167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:18:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:30.827+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0e2804b360 con 0x7f0e20004690 2023-05-16T07:18:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.014+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e38047150 con 0x7f0e2405ed40 2023-05-16T07:18:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.017+0000 7f0e1ffff700 1 -- 172.21.15.92:0/4130111230 <== mgr.6167 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f0e38047150 con 0x7f0e2405ed40 2023-05-16T07:18:31.021 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:18:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-05-16T07:18:31.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e2405ed40 msgr2=0x7f0e24061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e2405ed40 0x7f0e24061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0e38167940 tx=0x7f0e2c01a000).stop 2023-05-16T07:18:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 msgr2=0x7f0e3813bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:18:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e3813bea0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f0e2800e660 tx=0x7f0e28009040).stop 2023-05-16T07:18:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 shutdown_connections 2023-05-16T07:18:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0e2405ed40 0x7f0e24061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0e380abf70 0x7f0e380ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e20004690 0x7f0e3813bea0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 --2- 172.21.15.92:0/4130111230 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0e38156ce0 0x7f0e381584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:18:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 >> 172.21.15.92:0/4130111230 conn(0x7f0e380cb600 msgr2=0x7f0e3815a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:18:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 shutdown_connections 2023-05-16T07:18:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:18:31.019+0000 7f0e3d473700 1 -- 172.21.15.92:0/4130111230 wait complete. 2023-05-16T07:18:31.035 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-05-16T07:18:31.036 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-05-16T07:18:31.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:31.037 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:32.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:32.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:33.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:33.072 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:34.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:34.091 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:35.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:35.111 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:36.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:36.131 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:37.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:37.152 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:37.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:18:37.274 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:18:38.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:38.170 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:39.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:39.190 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:40.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:40.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:41.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:41.231 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:42.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:42.251 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:43.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:43.272 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:44.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:44.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:45.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:45.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:46.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:46.332 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:47.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:47.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:48.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:48.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:49.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:49.393 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:50.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:50.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:51.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:51.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:52.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:52.452 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:53.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:53.471 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:54.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:54.491 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:55.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:55.511 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:56.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:56.529 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:57.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:57.549 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:58.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:58.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:18:59.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:18:59.589 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:19:00.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:00.608 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:19:00.686 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:00.683+0000 7f12545d2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:00.687 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:00.683+0000 7f12545d2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:00.687 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:00.683+0000 7f12545d2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:00.687 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:00.683+0000 7f12545d2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:00.687 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:00.683+0000 7f12545d2700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:01.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:01.628 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:19:01.647 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_19_00_UTC 2023-05-16T07:19:01.647 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-05-16-07_19_00_UTC'] 2023-05-16T07:19:01.647 DEBUG:tasks.cephfs.test_snap_schedules:exec=1684221510.6988785, now = 1684221541.647957, timo = 30 2023-05-16T07:19:02.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-05-16T07:19:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.769+0000 7f6ac8cb5700 1 Processor -- start 2023-05-16T07:19:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.769+0000 7f6ac8cb5700 1 -- start start 2023-05-16T07:19:02.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac8cb5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154090 0x7f6ac40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:02.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac8cb5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ac4068160 0x7f6ac4068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:02.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac8cb5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac40c94d0 con 0x7f6ac4154470 2023-05-16T07:19:02.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac8cb5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac40cb7e0 con 0x7f6ac4068160 2023-05-16T07:19:02.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac8cb5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4068a70 con 0x7f6ac4154090 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ac4068160 0x7f6ac4068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6abbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ac4154470 0x7f6ac40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58872/0 (socket says 172.21.15.92:58872) 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ac4068160 0x7f6ac4068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48128/0 (socket says 172.21.15.92:48128) 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6abbfff700 1 -- 172.21.15.92:0/1944499419 learned_addr learned my addr 172.21.15.92:0/1944499419 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac259c700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154090 0x7f6ac40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 -- 172.21.15.92:0/1944499419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154090 msgr2=0x7f6ac40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:02.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154090 0x7f6ac40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 -- 172.21.15.92:0/1944499419 >> v1:172.21.15.92:6789/0 conn(0x7f6ac4154470 legacy=0x7f6ac40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac1d9b700 1 -- 172.21.15.92:0/1944499419 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270523178 0 0) 0x7f6ac40c94d0 con 0x7f6ac4154470 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 -- 172.21.15.92:0/1944499419 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40c88c0 con 0x7f6ac4068160 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac259c700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154090 0x7f6ac40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.770+0000 7f6ac2d9d700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4068160 0x7f6ac4068580 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f6abc009e60 tx=0x7f6abc009780).ready entity=mon.0 client_cookie=b20a2d892c41086a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac1d9b700 1 -- 172.21.15.92:0/1944499419 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00e070 con 0x7f6ac4068160 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac1d9b700 1 -- 172.21.15.92:0/1944499419 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc00bc30 con 0x7f6ac4068160 2023-05-16T07:19:02.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac1d9b700 1 -- 172.21.15.92:0/1944499419 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc012410 con 0x7f6ac4068160 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/1944499419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4068160 msgr2=0x7f6ac4068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4068160 0x7f6ac4068580 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f6abc009e60 tx=0x7f6abc009780).stop 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/1944499419 shutdown_connections 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154090 0x7f6ac40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/1944499419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4068160 0x7f6ac4068580 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/1944499419 >> 172.21.15.92:0/1944499419 conn(0x7f6ac4067920 msgr2=0x7f6ac40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:02.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/1944499419 shutdown_connections 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.771+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/1944499419 wait complete. 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.772+0000 7f6ac8cb5700 1 Processor -- start 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.772+0000 7f6ac8cb5700 1 -- start start 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6ac8cb5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ac4068160 0x7f6ac4168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6ac8cb5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 0x7f6ac414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6ac8cb5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154470 0x7f6ac4150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6ac8cb5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4150f80 con 0x7f6ac4154090 2023-05-16T07:19:02.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6ac8cb5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4151260 con 0x7f6ac4068160 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6ac8cb5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4151540 con 0x7f6ac4154470 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 0x7f6ac414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 0x7f6ac414fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48130/0 (socket says 172.21.15.92:48130) 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 -- 172.21.15.92:0/3677163447 learned_addr learned my addr 172.21.15.92:0/3677163447 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 -- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154470 msgr2=0x7f6ac4150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154470 0x7f6ac4150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:02.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 -- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ac4068160 msgr2=0x7f6ac4168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ac4068160 0x7f6ac4168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 -- 172.21.15.92:0/3677163447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc009a20 con 0x7f6ac4154090 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.773+0000 7f6abbfff700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 0x7f6ac414fa90 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f6aac00c490 tx=0x7f6aac007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.774+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0072e0 con 0x7f6ac4154090 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.774+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac007440 con 0x7f6ac4154090 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.774+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac4151820 con 0x7f6ac4154090 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.774+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0057a0 con 0x7f6ac4154090 2023-05-16T07:19:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.774+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac414ff80 con 0x7f6ac4154090 2023-05-16T07:19:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.774+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac4066730 con 0x7f6ac4154090 2023-05-16T07:19:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.775+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 43) v1 ==== 75419+0+0 (secure 0 0 0) 0x7f6aac01a060 con 0x7f6ac4154090 2023-05-16T07:19:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.775+0000 7f6aba7fc700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aa405ef00 0x7f6aa40613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.775+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f6aac079650 con 0x7f6ac4154090 2023-05-16T07:19:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.775+0000 7f6ac259c700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aa405ef00 0x7f6aa40613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.776+0000 7f6ac259c700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aa405ef00 0x7f6aa40613c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6ab4006fd0 tx=0x7f6ab4008040).ready entity=mgr.6167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:02.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.779+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6aac04a880 con 0x7f6ac4154090 2023-05-16T07:19:02.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.967+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ac406c350 con 0x7f6aa405ef00 2023-05-16T07:19:02.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.971+0000 7f6aba7fc700 1 -- 172.21.15.92:0/3677163447 <== mgr.6167 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6ac406c350 con 0x7f6aa405ef00 2023-05-16T07:19:02.976 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path snap_test_dir1 2023-05-16T07:19:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aa405ef00 msgr2=0x7f6aa40613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:02.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aa405ef00 0x7f6aa40613c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6ab4006fd0 tx=0x7f6ab4008040).stop 2023-05-16T07:19:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 msgr2=0x7f6ac414fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 0x7f6ac414fa90 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f6aac00c490 tx=0x7f6aac007d70).stop 2023-05-16T07:19:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 shutdown_connections 2023-05-16T07:19:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6aa405ef00 0x7f6aa40613c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ac4154470 0x7f6ac4150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ac4068160 0x7f6ac4168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 --2- 172.21.15.92:0/3677163447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac4154090 0x7f6ac414fa90 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 >> 172.21.15.92:0/3677163447 conn(0x7f6ac4067920 msgr2=0x7f6ac40c1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 shutdown_connections 2023-05-16T07:19:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:02.973+0000 7f6ac8cb5700 1 -- 172.21.15.92:0/3677163447 wait complete. 2023-05-16T07:19:02.988 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-05-16T07:19:02.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:02.989 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:19:03.003 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_19_00_UTC 2023-05-16T07:19:03.004 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-05-16-07_19_00_UTC 2023-05-16T07:19:03.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:03.005 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-05-16-07_19_00_UTC) 2023-05-16T07:19:03.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:03.066 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-05-16T07:19:03.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:19:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.272+0000 7f89db599700 1 Processor -- start 2023-05-16T07:19:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.272+0000 7f89db599700 1 -- start start 2023-05-16T07:19:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.273+0000 7f89db599700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f89d4153f90 0x7f89d40a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.273+0000 7f89db599700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d4155780 con 0x7f89d40c5d00 2023-05-16T07:19:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.273+0000 7f89db599700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d406a140 con 0x7f89d4155380 2023-05-16T07:19:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.273+0000 7f89db599700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d4069f60 con 0x7f89d4153f90 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.273+0000 7f89d9d96700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89d40c5d00 0x7f89d40c0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58888/0 (socket says 172.21.15.92:58888) 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.273+0000 7f89d9d96700 1 -- 172.21.15.92:0/3345137887 learned_addr learned my addr 172.21.15.92:0/3345137887 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 --2- 172.21.15.92:0/3345137887 >> v2:172.21.15.158:3300/0 conn(0x7f89d4153f90 0x7f89d40a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234965578 0 0) 0x7f89d4155780 con 0x7f89d40c5d00 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89bc0033f0 con 0x7f89d40c5d00 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038522722 0 0) 0x7f89bc0033f0 con 0x7f89d40c5d00 2023-05-16T07:19:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 >> v2:172.21.15.158:3300/0 conn(0x7f89d4153f90 msgr2=0x7f89d40a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:03.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 --2- 172.21.15.92:0/3345137887 >> v2:172.21.15.158:3300/0 conn(0x7f89d4153f90 0x7f89d40a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 >> v1:172.21.15.158:6789/0 conn(0x7f89d4155380 legacy=0x7f89d40c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d406a420 con 0x7f89d40c5d00 2023-05-16T07:19:03.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f89c8001470 con 0x7f89d40c5d00 2023-05-16T07:19:03.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 >> v1:172.21.15.92:6789/0 conn(0x7f89d40c5d00 legacy=0x7f89d40c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89bc0038e0 0x7f89bc003d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc004690 0x7f89bc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89bc005000 0x7f89bc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4069f60 con 0x7f89bc0038e0 2023-05-16T07:19:03.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc00beb0 con 0x7f89bc004690 2023-05-16T07:19:03.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc00c070 con 0x7f89bc005000 2023-05-16T07:19:03.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89bc0038e0 0x7f89bc003d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9d96700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc004690 0x7f89bc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89dad98700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89bc005000 0x7f89bc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 -- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89bc005000 msgr2=0x7f89bc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89bc005000 0x7f89bc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 -- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc004690 msgr2=0x7f89bc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc004690 0x7f89bc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89da597700 1 -- 172.21.15.92:0/3345137887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc0033f0 con 0x7f89bc0038e0 2023-05-16T07:19:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89dad98700 1 -- 172.21.15.92:0/3345137887 reap_dead start 2023-05-16T07:19:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.274+0000 7f89d9d96700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc004690 0x7f89bc004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89da597700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89bc0038e0 0x7f89bc003d20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f89c400e210 tx=0x7f89c400b040).ready entity=mon.0 client_cookie=ba1622159dd9e86e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4009ac0 con 0x7f89bc0038e0 2023-05-16T07:19:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400f040 con 0x7f89bc0038e0 2023-05-16T07:19:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89d9595700 1 -- 172.21.15.92:0/3345137887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400ecd0 con 0x7f89bc0038e0 2023-05-16T07:19:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 -- 172.21.15.92:0/3345137887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89bc0038e0 msgr2=0x7f89bc003d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89bc0038e0 0x7f89bc003d20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f89c400e210 tx=0x7f89c400b040).stop 2023-05-16T07:19:03.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 -- 172.21.15.92:0/3345137887 shutdown_connections 2023-05-16T07:19:03.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 --2- 172.21.15.92:0/3345137887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89bc0038e0 0x7f89bc003d20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 -- 172.21.15.92:0/3345137887 >> 172.21.15.92:0/3345137887 conn(0x7f89d40cb600 msgr2=0x7f89d40c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:03.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 -- 172.21.15.92:0/3345137887 shutdown_connections 2023-05-16T07:19:03.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.275+0000 7f89db599700 1 -- 172.21.15.92:0/3345137887 wait complete. 2023-05-16T07:19:03.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.276+0000 7f89db599700 1 Processor -- start 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.276+0000 7f89db599700 1 -- start start 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89db599700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc0038e0 0x7f89d40c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89db599700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 0x7f89d40ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89db599700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89d40adae0 0x7f89d40ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89db599700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40c42a0 con 0x7f89d40c4730 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89db599700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40afff0 con 0x7f89bc0038e0 2023-05-16T07:19:03.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89db599700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40c3690 con 0x7f89d40adae0 2023-05-16T07:19:03.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89d9d96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 0x7f89d40ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89d9d96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 0x7f89d40ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48150/0 (socket says 172.21.15.92:48150) 2023-05-16T07:19:03.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89d9d96700 1 -- 172.21.15.92:0/2872032682 learned_addr learned my addr 172.21.15.92:0/2872032682 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:03.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89da597700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc0038e0 0x7f89d40c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.277+0000 7f89dad98700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89d40adae0 0x7f89d40ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d9d96700 1 -- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89d40adae0 msgr2=0x7f89d40ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d9d96700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89d40adae0 0x7f89d40ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d9d96700 1 -- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc0038e0 msgr2=0x7f89d40c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d9d96700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc0038e0 0x7f89d40c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d9d96700 1 -- 172.21.15.92:0/2872032682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d40c3970 con 0x7f89d40c4730 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d9d96700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 0x7f89d40ae2e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f89c8000fd0 tx=0x7f89c8001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:03.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8010520 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c8010680 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8010a00 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d40c3c50 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.278+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d414fe50 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.279+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4083120 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.279+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 43) v1 ==== 75419+0+0 (secure 0 0 0) 0x7f89c801a020 con 0x7f89d40c4730 2023-05-16T07:19:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.279+0000 7f89d27fc700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89c005ef00 0x7f89c00613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.279+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f89c8079950 con 0x7f89d40c4730 2023-05-16T07:19:03.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.279+0000 7f89da597700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89c005ef00 0x7f89c00613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.280+0000 7f89da597700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89c005ef00 0x7f89c00613c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f89c400afd0 tx=0x7f89c401a000).ready entity=mgr.6167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:03.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.283+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f89c804ab80 con 0x7f89d40c4730 2023-05-16T07:19:03.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.492+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f89d4047150 con 0x7f89d40c4730 2023-05-16T07:19:03.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.547+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f89c8010d70 con 0x7f89d40c4730 2023-05-16T07:19:03.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.548+0000 7f89d27fc700 1 -- 172.21.15.92:0/2872032682 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v44) v1 ==== 93+0+0 (secure 0 0 0) 0x7f89c8072d60 con 0x7f89d40c4730 2023-05-16T07:19:03.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89c005ef00 msgr2=0x7f89c00613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89c005ef00 0x7f89c00613c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f89c400afd0 tx=0x7f89c401a000).stop 2023-05-16T07:19:03.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 msgr2=0x7f89d40ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 0x7f89d40ae2e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f89c8000fd0 tx=0x7f89c8001d90).stop 2023-05-16T07:19:03.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 shutdown_connections 2023-05-16T07:19:03.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89c005ef00 0x7f89c00613c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89d40adae0 0x7f89d40ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89bc0038e0 0x7f89d40c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 --2- 172.21.15.92:0/2872032682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d40c4730 0x7f89d40ae2e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 >> 172.21.15.92:0/2872032682 conn(0x7f89d40cb600 msgr2=0x7f89d40c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:03.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 shutdown_connections 2023-05-16T07:19:03.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.551+0000 7f89db599700 1 -- 172.21.15.92:0/2872032682 wait complete. 2023-05-16T07:19:03.566 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:19:03.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:03.595 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:03.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:19:03.624 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:19:03.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:03.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:03.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:19:03.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:03.677 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:19:03.690 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:03.687+0000 7fb8bd688680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:03.699 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:03.696+0000 7f95c378c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:03.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:03.755 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:03.752+0000 7fb8bd688680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:03.760 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:03.756+0000 7f95c378c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:03.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.856+0000 7f4324ebc700 1 Processor -- start 2023-05-16T07:19:03.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.856+0000 7f4324ebc700 1 -- start start 2023-05-16T07:19:03.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f4324ebc700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4320153eb0 0x7f4320154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f4324ebc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4320068c70 0x7f4320066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f4324ebc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320065000 0x7f4320065480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f4324ebc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4320065970 con 0x7f4320068c70 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f4324ebc700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43200bc9b0 con 0x7f4320153eb0 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f4324ebc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200bcc90 con 0x7f4320065000 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f431dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4320068c70 0x7f4320066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f431dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4320068c70 0x7f4320066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48218/0 (socket says 172.21.15.92:48218) 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f431dd9b700 1 -- 172.21.15.92:0/2882754064 learned_addr learned my addr 172.21.15.92:0/2882754064 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f431ed9d700 1 --2- 172.21.15.92:0/2882754064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320065000 0x7f4320065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.857+0000 7f431e59c700 1 --2- 172.21.15.92:0/2882754064 >> v2:172.21.15.158:3300/0 conn(0x7f4320153eb0 0x7f4320154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431dd9b700 1 -- 172.21.15.92:0/2882754064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320065000 msgr2=0x7f4320065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431dd9b700 1 --2- 172.21.15.92:0/2882754064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320065000 0x7f4320065480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431dd9b700 1 -- 172.21.15.92:0/2882754064 >> v2:172.21.15.158:3300/0 conn(0x7f4320153eb0 msgr2=0x7f4320154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:03.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431dd9b700 1 --2- 172.21.15.92:0/2882754064 >> v2:172.21.15.158:3300/0 conn(0x7f4320153eb0 0x7f4320154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431dd9b700 1 -- 172.21.15.92:0/2882754064 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200bcf70 con 0x7f4320068c70 2023-05-16T07:19:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431dd9b700 1 --2- 172.21.15.92:0/2882754064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320068c70 0x7f4320066e50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f4320064430 tx=0x7f43100090f0).ready entity=mon.0 client_cookie=ef8203c6438772b1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431d59a700 1 -- 172.21.15.92:0/2882754064 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000f510 con 0x7f4320068c70 2023-05-16T07:19:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431d59a700 1 -- 172.21.15.92:0/2882754064 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431000f670 con 0x7f4320068c70 2023-05-16T07:19:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f431d59a700 1 -- 172.21.15.92:0/2882754064 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000f9f0 con 0x7f4320068c70 2023-05-16T07:19:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f4324ebc700 1 -- 172.21.15.92:0/2882754064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320068c70 msgr2=0x7f4320066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.858+0000 7f4324ebc700 1 --2- 172.21.15.92:0/2882754064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320068c70 0x7f4320066e50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f4320064430 tx=0x7f43100090f0).stop 2023-05-16T07:19:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 -- 172.21.15.92:0/2882754064 shutdown_connections 2023-05-16T07:19:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 --2- 172.21.15.92:0/2882754064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320065000 0x7f4320065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 --2- 172.21.15.92:0/2882754064 >> v2:172.21.15.158:3300/0 conn(0x7f4320153eb0 0x7f4320154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 --2- 172.21.15.92:0/2882754064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320068c70 0x7f4320066e50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 -- 172.21.15.92:0/2882754064 >> 172.21.15.92:0/2882754064 conn(0x7f43200c4430 msgr2=0x7f43200c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 -- 172.21.15.92:0/2882754064 shutdown_connections 2023-05-16T07:19:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.859+0000 7f4324ebc700 1 -- 172.21.15.92:0/2882754064 wait complete. 2023-05-16T07:19:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 Processor -- start 2023-05-16T07:19:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 -- start start 2023-05-16T07:19:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 0x7f4320149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320153eb0 0x7f4320163b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43200c17a0 0x7f43200ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320164060 con 0x7f4320065000 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200bebb0 con 0x7f43200c17a0 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.860+0000 7f4324ebc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200bee90 con 0x7f4320153eb0 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 0x7f4320149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 0x7f4320149490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48224/0 (socket says 172.21.15.92:48224) 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 -- 172.21.15.92:0/1775004310 learned_addr learned my addr 172.21.15.92:0/1775004310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431ed9d700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43200c17a0 0x7f43200ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431dd9b700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320153eb0 0x7f4320163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 -- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320153eb0 msgr2=0x7f4320163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320153eb0 0x7f4320163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 -- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43200c17a0 msgr2=0x7f43200ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43200c17a0 0x7f43200ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 -- 172.21.15.92:0/1775004310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310009370 con 0x7f4320065000 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f431e59c700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 0x7f4320149490 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f431400e900 tx=0x7f4314009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314012600 con 0x7f4320065000 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314012760 con 0x7f4320065000 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314012ae0 con 0x7f4320065000 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43200bf170 con 0x7f4320065000 2023-05-16T07:19:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.861+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43200bf730 con 0x7f4320065000 2023-05-16T07:19:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.862+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f4314022070 con 0x7f4320065000 2023-05-16T07:19:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.862+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200bfaa0 con 0x7f4320065000 2023-05-16T07:19:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.863+0000 7f430e7fc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 0x7f4300061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.863+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f431407a110 con 0x7f4320065000 2023-05-16T07:19:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.863+0000 7f431dd9b700 1 -- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 msgr2=0x7f4300061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.863+0000 7f431dd9b700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 0x7f4300061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:03.867+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f431404b2c0 con 0x7f4320065000 2023-05-16T07:19:04.028 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:04.025+0000 7f95c378c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:04.039 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:04.036+0000 7fb8bd688680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:04.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.064+0000 7f431dd9b700 1 -- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 msgr2=0x7f4300061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:04.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.064+0000 7f431dd9b700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 0x7f4300061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:04.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.080+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f432006c350 con 0x7f4320065000 2023-05-16T07:19:04.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.081+0000 7f430e7fc700 1 -- 172.21.15.92:0/1775004310 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f4314078320 con 0x7f4320065000 2023-05-16T07:19:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:04.085 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:18:17.423562+0000","modified":"2023-05-16T07:18:19.645369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7017":{"gid":7017,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1226709881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1226709881},{"type":"v1","addr":"172.21.15.92:6835","nonce":1226709881}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-05-16T07:19:04.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 msgr2=0x7f4300061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 0x7f4300061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 msgr2=0x7f4320149490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 0x7f4320149490 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f431400e900 tx=0x7f4314009060).stop 2023-05-16T07:19:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 shutdown_connections 2023-05-16T07:19:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f430005eeb0 0x7f4300061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43200c17a0 0x7f43200ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4320065000 0x7f4320149490 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 --2- 172.21.15.92:0/1775004310 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4320153eb0 0x7f4320163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 >> 172.21.15.92:0/1775004310 conn(0x7f43200c4430 msgr2=0x7f43200622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 shutdown_connections 2023-05-16T07:19:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.083+0000 7f4324ebc700 1 -- 172.21.15.92:0/1775004310 wait complete. 2023-05-16T07:19:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-05-16T07:19:04.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.250+0000 7f893ac62700 1 Processor -- start 2023-05-16T07:19:04.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.250+0000 7f893ac62700 1 -- start start 2023-05-16T07:19:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f893ac62700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f893413a190 0x7f8934064e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f893ac62700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89340c83c0 0x7f8934066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f893ac62700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8934067340 0x7f89341578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f893ac62700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89340c51d0 con 0x7f893413a190 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f893ac62700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89340677f0 con 0x7f8934067340 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f893ac62700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340c41c0 con 0x7f89340c83c0 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.251+0000 7f89389fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f893413a190 0x7f8934064e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f893413a190 0x7f8934064e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48278/0 (socket says 172.21.15.92:48278) 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 -- 172.21.15.92:0/2990797030 learned_addr learned my addr 172.21.15.92:0/2990797030 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f892bfff700 1 --2- 172.21.15.92:0/2990797030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89340c83c0 0x7f8934066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89391ff700 1 --2- 172.21.15.92:0/2990797030 >> v2:172.21.15.158:3300/0 conn(0x7f8934067340 0x7f89341578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 -- 172.21.15.92:0/2990797030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89340c83c0 msgr2=0x7f8934066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 --2- 172.21.15.92:0/2990797030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89340c83c0 0x7f8934066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 -- 172.21.15.92:0/2990797030 >> v2:172.21.15.158:3300/0 conn(0x7f8934067340 msgr2=0x7f89341578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 --2- 172.21.15.92:0/2990797030 >> v2:172.21.15.158:3300/0 conn(0x7f8934067340 0x7f89341578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 -- 172.21.15.92:0/2990797030 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340c4d40 con 0x7f893413a190 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89391ff700 1 --2- 172.21.15.92:0/2990797030 >> v2:172.21.15.158:3300/0 conn(0x7f8934067340 0x7f89341578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f89389fe700 1 --2- 172.21.15.92:0/2990797030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f893413a190 0x7f8934064e00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f893406a900 tx=0x7f89300090f0).ready entity=mon.0 client_cookie=10c6239a3d813d9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f892b7fe700 1 -- 172.21.15.92:0/2990797030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893000b520 con 0x7f893413a190 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f892b7fe700 1 -- 172.21.15.92:0/2990797030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893000b680 con 0x7f893413a190 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.252+0000 7f892b7fe700 1 -- 172.21.15.92:0/2990797030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893000ba00 con 0x7f893413a190 2023-05-16T07:19:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 -- 172.21.15.92:0/2990797030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f893413a190 msgr2=0x7f8934064e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 --2- 172.21.15.92:0/2990797030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f893413a190 0x7f8934064e00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f893406a900 tx=0x7f89300090f0).stop 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 -- 172.21.15.92:0/2990797030 shutdown_connections 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 --2- 172.21.15.92:0/2990797030 >> v2:172.21.15.158:3300/0 conn(0x7f8934067340 0x7f89341578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 --2- 172.21.15.92:0/2990797030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f893413a190 0x7f8934064e00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 --2- 172.21.15.92:0/2990797030 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89340c83c0 0x7f8934066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 -- 172.21.15.92:0/2990797030 >> 172.21.15.92:0/2990797030 conn(0x7f89340689e0 msgr2=0x7f89340695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 -- 172.21.15.92:0/2990797030 shutdown_connections 2023-05-16T07:19:04.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.253+0000 7f893ac62700 1 -- 172.21.15.92:0/2990797030 wait complete. 2023-05-16T07:19:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.255+0000 7f893ac62700 1 Processor -- start 2023-05-16T07:19:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.255+0000 7f893ac62700 1 -- start start 2023-05-16T07:19:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f893ac62700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 0x7f8934167720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f893ac62700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89340c83c0 0x7f89340a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f893ac62700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f893413a190 0x7f89340a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f893ac62700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340a05a0 con 0x7f8934067340 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f893ac62700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340a0880 con 0x7f89340c83c0 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f893ac62700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340a0b60 con 0x7f893413a190 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f89389fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 0x7f8934167720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f89389fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 0x7f8934167720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48284/0 (socket says 172.21.15.92:48284) 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.256+0000 7f89389fe700 1 -- 172.21.15.92:0/871523151 learned_addr learned my addr 172.21.15.92:0/871523151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f892bfff700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89340c83c0 0x7f89340a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89391ff700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f893413a190 0x7f89340a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89389fe700 1 -- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f893413a190 msgr2=0x7f89340a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89389fe700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f893413a190 0x7f89340a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89389fe700 1 -- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89340c83c0 msgr2=0x7f89340a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89389fe700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89340c83c0 0x7f89340a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89389fe700 1 -- 172.21.15.92:0/871523151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930009370 con 0x7f8934067340 2023-05-16T07:19:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f89389fe700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 0x7f8934167720 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f893406bb90 tx=0x7f8930017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893000be90 con 0x7f8934067340 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893000b7f0 con 0x7f8934067340 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89340a0e40 con 0x7f8934067340 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893001c630 con 0x7f8934067340 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.257+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8934167c10 con 0x7f8934067340 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.258+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8934047150 con 0x7f8934067340 2023-05-16T07:19:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.258+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f893000f070 con 0x7f8934067340 2023-05-16T07:19:04.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.259+0000 7f8928ff9700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f892005eeb0 0x7f8920061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.259+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f893007b5a0 con 0x7f8934067340 2023-05-16T07:19:04.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.259+0000 7f892bfff700 1 -- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f892005eeb0 msgr2=0x7f8920061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:04.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.259+0000 7f892bfff700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f892005eeb0 0x7f8920061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:04.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.263+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f893004c7d0 con 0x7f8934067340 2023-05-16T07:19:04.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.433+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8934069eb0 con 0x7f8934067340 2023-05-16T07:19:04.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.434+0000 7f8928ff9700 1 -- 172.21.15.92:0/871523151 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+18118 (secure 0 0 0) 0x7f893004fdf0 con 0x7f8934067340 2023-05-16T07:19:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:04.438 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":108,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:28.827155+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:18:15.946822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-05-16T07:18:16.384841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3022628291":"2023-05-17T07:18:28.827125+0000","172.21.15.158:6833/86301":"2023-05-17T07:18:28.827125+0000","172.21.15.158:6832/86301":"2023-05-17T07:18:28.827125+0000","172.21.15.158:0/157465347":"2023-05-17T07:18:28.827125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:04.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f892005eeb0 msgr2=0x7f8920061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f892005eeb0 0x7f8920061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 msgr2=0x7f8934167720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 0x7f8934167720 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f893406bb90 tx=0x7f8930017000).stop 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 shutdown_connections 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f892005eeb0 0x7f8920061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f893413a190 0x7f89340a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8934067340 0x7f8934167720 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 --2- 172.21.15.92:0/871523151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89340c83c0 0x7f89340a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 >> 172.21.15.92:0/871523151 conn(0x7f89340689e0 msgr2=0x7f8934069780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 shutdown_connections 2023-05-16T07:19:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.436+0000 7f893ac62700 1 -- 172.21.15.92:0/871523151 wait complete. 2023-05-16T07:19:04.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:18:15.946822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:18:16.384841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:19:04.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:19:04.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.568+0000 7f4d40fb1700 1 Processor -- start 2023-05-16T07:19:04.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.568+0000 7f4d40fb1700 1 -- start start 2023-05-16T07:19:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d40fb1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4d3c153d00 0x7f4d3c063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d40fb1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0bf950 con 0x7f4d3c1540e0 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d40fb1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0a7c90 con 0x7f4d3c0622b0 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d40fb1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c067f60 con 0x7f4d3c153d00 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d39d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4d3c1540e0 0x7f4d3c0bf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59040/0 (socket says 172.21.15.92:59040) 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d39d9b700 1 -- 172.21.15.92:0/558473696 learned_addr learned my addr 172.21.15.92:0/558473696 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3a59c700 1 --2- 172.21.15.92:0/558473696 >> v2:172.21.15.158:3300/0 conn(0x7f4d3c153d00 0x7f4d3c063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3959a700 1 -- 172.21.15.92:0/558473696 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671414571 0 0) 0x7f4d3c0bf950 con 0x7f4d3c1540e0 2023-05-16T07:19:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3959a700 1 -- 172.21.15.92:0/558473696 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d240033f0 con 0x7f4d3c1540e0 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3a59c700 1 -- 172.21.15.92:0/558473696 >> v1:172.21.15.158:6789/0 conn(0x7f4d3c0622b0 legacy=0x7f4d3c0683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3959a700 1 -- 172.21.15.92:0/558473696 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037459316 0 0) 0x7f4d3c0a7c90 con 0x7f4d3c0622b0 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3a59c700 1 -- 172.21.15.92:0/558473696 >> v1:172.21.15.92:6789/0 conn(0x7f4d3c1540e0 legacy=0x7f4d3c0bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.569+0000 7f4d3a59c700 1 -- 172.21.15.92:0/558473696 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0a7f00 con 0x7f4d3c153d00 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d3a59c700 1 --2- 172.21.15.92:0/558473696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c153d00 0x7f4d3c063fd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f4d30009760 tx=0x7f4d30009060).ready entity=mon.1 client_cookie=8d275979702b65f4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d3959a700 1 -- 172.21.15.92:0/558473696 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000f510 con 0x7f4d3c153d00 2023-05-16T07:19:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d3959a700 1 -- 172.21.15.92:0/558473696 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3000f670 con 0x7f4d3c153d00 2023-05-16T07:19:04.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d3959a700 1 -- 172.21.15.92:0/558473696 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000f9f0 con 0x7f4d3c153d00 2023-05-16T07:19:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 -- 172.21.15.92:0/558473696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c153d00 msgr2=0x7f4d3c063fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/558473696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c153d00 0x7f4d3c063fd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f4d30009760 tx=0x7f4d30009060).stop 2023-05-16T07:19:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 -- 172.21.15.92:0/558473696 shutdown_connections 2023-05-16T07:19:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/558473696 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c153d00 0x7f4d3c063fd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 -- 172.21.15.92:0/558473696 >> 172.21.15.92:0/558473696 conn(0x7f4d3c05e820 msgr2=0x7f4d3c0c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 -- 172.21.15.92:0/558473696 shutdown_connections 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.570+0000 7f4d40fb1700 1 -- 172.21.15.92:0/558473696 wait complete. 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.571+0000 7f4d40fb1700 1 Processor -- start 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 -- start start 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d3c0622b0 0x7f4d3c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 0x7f4d3c14e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c1540e0 0x7f4d3c14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0a8e90 con 0x7f4d3c153d00 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c14ee80 con 0x7f4d3c1540e0 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d40fb1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0abd60 con 0x7f4d3c0622b0 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d39d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 0x7f4d3c14e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d39d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 0x7f4d3c14e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48298/0 (socket says 172.21.15.92:48298) 2023-05-16T07:19:04.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d39d9b700 1 -- 172.21.15.92:0/257159345 learned_addr learned my addr 172.21.15.92:0/257159345 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d3a59c700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d3c0622b0 0x7f4d3c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.572+0000 7f4d3ad9d700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c1540e0 0x7f4d3c14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d39d9b700 1 -- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d3c0622b0 msgr2=0x7f4d3c14c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d39d9b700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d3c0622b0 0x7f4d3c14c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d39d9b700 1 -- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c1540e0 msgr2=0x7f4d3c14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d39d9b700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c1540e0 0x7f4d3c14e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d39d9b700 1 -- 172.21.15.92:0/257159345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300092a0 con 0x7f4d3c153d00 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d39d9b700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 0x7f4d3c14e4a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f4d2c00c410 tx=0x7f4d2c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0070d0 con 0x7f4d3c153d00 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c007230 con 0x7f4d3c153d00 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c0a57e0 con 0x7f4d3c153d00 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0075d0 con 0x7f4d3c153d00 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.573+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0a87d0 con 0x7f4d3c153d00 2023-05-16T07:19:04.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.574+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c0a8fd0 con 0x7f4d3c153d00 2023-05-16T07:19:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.574+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f4d2c010070 con 0x7f4d3c153d00 2023-05-16T07:19:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.575+0000 7f4d2a7fc700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 0x7f4d1c061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.575+0000 7f4d3a59c700 1 -- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 msgr2=0x7f4d1c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.575+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f4d2c079390 con 0x7f4d3c153d00 2023-05-16T07:19:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.575+0000 7f4d3a59c700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 0x7f4d1c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:04.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.579+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4d2c04a5c0 con 0x7f4d3c153d00 2023-05-16T07:19:04.629 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:04.626+0000 7f95c378c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:04.694 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:04.691+0000 7fb8bd688680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:04.700 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:04.697+0000 7f95c378c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:04.760 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:04.756+0000 7fb8bd688680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:04.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.776+0000 7f4d3a59c700 1 -- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 msgr2=0x7f4d1c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:04.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.776+0000 7f4d3a59c700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 0x7f4d1c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.780+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4d3c0a81e0 con 0x7f4d3c153d00 2023-05-16T07:19:04.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.780+0000 7f4d2a7fc700 1 -- 172.21.15.92:0/257159345 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v108) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4d2c018070 con 0x7f4d3c153d00 2023-05-16T07:19:04.784 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:19:04.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 msgr2=0x7f4d1c061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:04.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 0x7f4d1c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:04.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 msgr2=0x7f4d3c14e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 0x7f4d3c14e4a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f4d2c00c410 tx=0x7f4d2c007c70).stop 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 shutdown_connections 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4d1c05eeb0 0x7f4d1c061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d3c0622b0 0x7f4d3c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d3c1540e0 0x7f4d3c14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 --2- 172.21.15.92:0/257159345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d3c153d00 0x7f4d3c14e4a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 >> 172.21.15.92:0/257159345 conn(0x7f4d3c05e820 msgr2=0x7f4d3c0c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 shutdown_connections 2023-05-16T07:19:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.782+0000 7f4d40fb1700 1 -- 172.21.15.92:0/257159345 wait complete. 2023-05-16T07:19:04.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:19:04.884 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:04.880+0000 7f95c378c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.913+0000 7f1339477700 1 Processor -- start 2023-05-16T07:19:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.913+0000 7f1339477700 1 -- start start 2023-05-16T07:19:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1339477700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f133414e620 0x7f13340c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1339477700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f133414ea00 0x7f1334154760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1339477700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1334065bf0 con 0x7f13340c4610 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1339477700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1334065ed0 con 0x7f133414ea00 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1339477700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13340661b0 con 0x7f133414e620 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f13337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f133414ea00 0x7f1334154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1338c76700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f13340c4610 0x7f1334151190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59056/0 (socket says 172.21.15.92:59056) 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1338c76700 1 -- 172.21.15.92:0/2865285668 learned_addr learned my addr 172.21.15.92:0/2865285668 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1333fff700 1 --2- 172.21.15.92:0/2865285668 >> v2:172.21.15.158:3300/0 conn(0x7f133414e620 0x7f13340c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1332ffd700 1 -- 172.21.15.92:0/2865285668 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292473627 0 0) 0x7f1334065bf0 con 0x7f13340c4610 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f1332ffd700 1 -- 172.21.15.92:0/2865285668 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c0033f0 con 0x7f13340c4610 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f13337fe700 1 -- 172.21.15.92:0/2865285668 >> v2:172.21.15.158:3300/0 conn(0x7f133414e620 msgr2=0x7f13340c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f13337fe700 1 --2- 172.21.15.92:0/2865285668 >> v2:172.21.15.158:3300/0 conn(0x7f133414e620 0x7f13340c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f13337fe700 1 -- 172.21.15.92:0/2865285668 >> v1:172.21.15.92:6789/0 conn(0x7f13340c4610 legacy=0x7f1334151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.914+0000 7f13337fe700 1 -- 172.21.15.92:0/2865285668 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1334066490 con 0x7f133414ea00 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f13337fe700 1 --2- 172.21.15.92:0/2865285668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f133414ea00 0x7f1334154760 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f133405fc40 tx=0x7f1324009060).ready entity=mon.0 client_cookie=261e1ea76e653d51 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1332ffd700 1 -- 172.21.15.92:0/2865285668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400d4a0 con 0x7f133414ea00 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1332ffd700 1 -- 172.21.15.92:0/2865285668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132400d600 con 0x7f133414ea00 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1332ffd700 1 -- 172.21.15.92:0/2865285668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400d980 con 0x7f133414ea00 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 -- 172.21.15.92:0/2865285668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f133414ea00 msgr2=0x7f1334154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 --2- 172.21.15.92:0/2865285668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f133414ea00 0x7f1334154760 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f133405fc40 tx=0x7f1324009060).stop 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 -- 172.21.15.92:0/2865285668 shutdown_connections 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 --2- 172.21.15.92:0/2865285668 >> v2:172.21.15.158:3300/0 conn(0x7f133414e620 0x7f13340c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 --2- 172.21.15.92:0/2865285668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f133414ea00 0x7f1334154760 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 -- 172.21.15.92:0/2865285668 >> 172.21.15.92:0/2865285668 conn(0x7f13340c7a80 msgr2=0x7f13340c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 -- 172.21.15.92:0/2865285668 shutdown_connections 2023-05-16T07:19:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.915+0000 7f1339477700 1 -- 172.21.15.92:0/2865285668 wait complete. 2023-05-16T07:19:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.917+0000 7f1339477700 1 Processor -- start 2023-05-16T07:19:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.917+0000 7f1339477700 1 -- start start 2023-05-16T07:19:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 0x7f13340aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f133414e620 0x7f13341503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f133414ea00 0x7f1334142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340aa870 con 0x7f13340c4610 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1334150da0 con 0x7f133414e620 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340a9f40 con 0x7f133414ea00 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 0x7f13340aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 0x7f13340aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48312/0 (socket says 172.21.15.92:48312) 2023-05-16T07:19:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 -- 172.21.15.92:0/1656733166 learned_addr learned my addr 172.21.15.92:0/1656733166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f13337fe700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f133414e620 0x7f13341503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1338c76700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f133414ea00 0x7f1334142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 -- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f133414ea00 msgr2=0x7f1334142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f133414ea00 0x7f1334142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 -- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f133414e620 msgr2=0x7f13341503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f133414e620 0x7f13341503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 -- 172.21.15.92:0/1656733166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240092a0 con 0x7f13340c4610 2023-05-16T07:19:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f13337fe700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f133414e620 0x7f13341503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1338c76700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f133414ea00 0x7f1334142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1333fff700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 0x7f13340aad00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f132800e930 tx=0x7f1328009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800eed0 con 0x7f13340c4610 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132800c850 con 0x7f13340c4610 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800cbd0 con 0x7f13340c4610 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13340aa220 con 0x7f13340c4610 2023-05-16T07:19:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.919+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13340ab1f0 con 0x7f13340c4610 2023-05-16T07:19:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.920+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13340aab50 con 0x7f13340c4610 2023-05-16T07:19:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.921+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f1328022070 con 0x7f13340c4610 2023-05-16T07:19:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.921+0000 7f131bfff700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f132005eeb0 0x7f1320061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.921+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f132807a3a0 con 0x7f13340c4610 2023-05-16T07:19:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.921+0000 7f13337fe700 1 -- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f132005eeb0 msgr2=0x7f1320061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.921+0000 7f13337fe700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f132005eeb0 0x7f1320061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:04.925+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f132804b5d0 con 0x7f13340c4610 2023-05-16T07:19:04.956 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:04.952+0000 7fb8bd688680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:05.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.099+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f13340c5b60 con 0x7f13340c4610 2023-05-16T07:19:05.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.099+0000 7f131bfff700 1 -- 172.21.15.92:0/1656733166 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f132804ebf0 con 0x7f13340c4610 2023-05-16T07:19:05.103 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:19:05.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f132005eeb0 msgr2=0x7f1320061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:05.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f132005eeb0 0x7f1320061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:05.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 msgr2=0x7f13340aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 0x7f13340aad00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f132800e930 tx=0x7f1328009060).stop 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 shutdown_connections 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f132005eeb0 0x7f1320061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f133414ea00 0x7f1334142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13340c4610 0x7f13340aad00 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 --2- 172.21.15.92:0/1656733166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f133414e620 0x7f13341503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 >> 172.21.15.92:0/1656733166 conn(0x7f13340c7a80 msgr2=0x7f13340c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 shutdown_connections 2023-05-16T07:19:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.101+0000 7f1339477700 1 -- 172.21.15.92:0/1656733166 wait complete. 2023-05-16T07:19:05.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:19:05.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.233+0000 7f389f59e700 1 Processor -- start 2023-05-16T07:19:05.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.233+0000 7f389f59e700 1 -- start start 2023-05-16T07:19:05.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.234+0000 7f389f59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f38a01540b0 0x7f38a00a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.234+0000 7f389f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00a6760 con 0x7f38a0066e50 2023-05-16T07:19:05.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.234+0000 7f389f59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00a6a40 con 0x7f38a0153cd0 2023-05-16T07:19:05.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.234+0000 7f389f59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a00a7370 con 0x7f38a01540b0 2023-05-16T07:19:05.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.234+0000 7f389ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f38a0066e50 0x7f38a0157d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59102/0 (socket says 172.21.15.92:59102) 2023-05-16T07:19:05.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 -- 172.21.15.92:0/1830858055 learned_addr learned my addr 172.21.15.92:0/1830858055 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:05.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389dd9b700 1 --2- 172.21.15.92:0/1830858055 >> v2:172.21.15.158:3300/0 conn(0x7f38a01540b0 0x7f38a00a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983701961 0 0) 0x7f38a00a6760 con 0x7f38a0066e50 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38880033f0 con 0x7f38a0066e50 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433868621 0 0) 0x7f38880033f0 con 0x7f38a0066e50 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 >> v2:172.21.15.158:3300/0 conn(0x7f38a01540b0 msgr2=0x7f38a00a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 --2- 172.21.15.92:0/1830858055 >> v2:172.21.15.158:3300/0 conn(0x7f38a01540b0 0x7f38a00a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 >> v1:172.21.15.158:6789/0 conn(0x7f38a0153cd0 legacy=0x7f38a0064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00a6d20 con 0x7f38a0066e50 2023-05-16T07:19:05.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f38980017f0 con 0x7f38a0066e50 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389dd9b700 1 --2- 172.21.15.92:0/1830858055 >> v2:172.21.15.158:3300/0 conn(0x7f38a01540b0 0x7f38a00a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 >> v1:172.21.15.92:6789/0 conn(0x7f38a0066e50 legacy=0x7f38a0157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38880038e0 0x7f3888003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3888004690 0x7f3888004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888005000 0x7f388800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00a7370 con 0x7f3888005000 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388800beb0 con 0x7f38880038e0 2023-05-16T07:19:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388800c070 con 0x7f3888004690 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888005000 0x7f388800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389dd9b700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3888004690 0x7f3888004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389e59c700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38880038e0 0x7f3888003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 -- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3888004690 msgr2=0x7f3888004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3888004690 0x7f3888004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 -- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38880038e0 msgr2=0x7f3888003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38880038e0 0x7f3888003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.235+0000 7f389ed9d700 1 -- 172.21.15.92:0/1830858055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880033f0 con 0x7f3888005000 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389ed9d700 1 -- 172.21.15.92:0/1830858055 reap_dead start 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389ed9d700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888005000 0x7f388800b8e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f389800c5f0 tx=0x7f38a00a72e0).ready entity=mon.0 client_cookie=90b439e40dbac7e1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800cdf0 con 0x7f3888005000 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898005b20 con 0x7f3888005000 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389d59a700 1 -- 172.21.15.92:0/1830858055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898005ea0 con 0x7f3888005000 2023-05-16T07:19:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 -- 172.21.15.92:0/1830858055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888005000 msgr2=0x7f388800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888005000 0x7f388800b8e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f389800c5f0 tx=0x7f38a00a72e0).stop 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 -- 172.21.15.92:0/1830858055 shutdown_connections 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 --2- 172.21.15.92:0/1830858055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888005000 0x7f388800b8e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 -- 172.21.15.92:0/1830858055 >> 172.21.15.92:0/1830858055 conn(0x7f38a0068a30 msgr2=0x7f38a00c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 -- 172.21.15.92:0/1830858055 shutdown_connections 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.236+0000 7f389f59e700 1 -- 172.21.15.92:0/1830858055 wait complete. 2023-05-16T07:19:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.237+0000 7f389f59e700 1 Processor -- start 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 -- start start 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3888005000 0x7f38a0167310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f38a014bbd0 0x7f38a014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 0x7f38a00ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00ac190 con 0x7f38a014b7c0 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a014c3e0 con 0x7f3888005000 2023-05-16T07:19:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01679a0 con 0x7f38a014bbd0 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 0x7f38a00ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 0x7f38a00ac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48368/0 (socket says 172.21.15.92:48368) 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389ed9d700 1 -- 172.21.15.92:0/2164695589 learned_addr learned my addr 172.21.15.92:0/2164695589 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389e59c700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3888005000 0x7f38a0167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.238+0000 7f389dd9b700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f38a014bbd0 0x7f38a014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389ed9d700 1 -- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f38a014bbd0 msgr2=0x7f38a014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389ed9d700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f38a014bbd0 0x7f38a014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389ed9d700 1 -- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3888005000 msgr2=0x7f38a0167310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389ed9d700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3888005000 0x7f38a0167310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389ed9d700 1 -- 172.21.15.92:0/2164695589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0167c80 con 0x7f38a014b7c0 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389ed9d700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 0x7f38a00ac620 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f38a00aa710 tx=0x7f3898015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38980017f0 con 0x7f38a014b7c0 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898013450 con 0x7f38a014b7c0 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a0167f60 con 0x7f38a014b7c0 2023-05-16T07:19:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898005ea0 con 0x7f38a014b7c0 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.239+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00accd0 con 0x7f38a014b7c0 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.240+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a00abf60 con 0x7f38a014b7c0 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.240+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f3898027070 con 0x7f38a014b7c0 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.241+0000 7f388e7fc700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f388005eee0 0x7f38800613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.241+0000 7f389e59c700 1 -- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f388005eee0 msgr2=0x7f38800613a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.241+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f389807ad20 con 0x7f38a014b7c0 2023-05-16T07:19:05.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.241+0000 7f389e59c700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f388005eee0 0x7f38800613a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:05.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.248+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f38a00abf60 con 0x7f38a014b7c0 2023-05-16T07:19:05.301 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:05.297+0000 7f95c378c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:05.392 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:05.388+0000 7fb8bd688680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:05.401 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:05.398+0000 7f95c378c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:05.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.418+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38a00abf60 con 0x7f38a014b7c0 2023-05-16T07:19:05.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.418+0000 7f388e7fc700 1 -- 172.21.15.92:0/2164695589 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38a00abf60 con 0x7f38a014b7c0 2023-05-16T07:19:05.422 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:19:05.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f388005eee0 msgr2=0x7f38800613a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:05.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f388005eee0 0x7f38800613a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:05.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 msgr2=0x7f38a00ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 0x7f38a00ac620 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f38a00aa710 tx=0x7f3898015000).stop 2023-05-16T07:19:05.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 shutdown_connections 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f388005eee0 0x7f38800613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a014b7c0 0x7f38a00ac620 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3888005000 0x7f38a0167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 --2- 172.21.15.92:0/2164695589 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f38a014bbd0 0x7f38a014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 >> 172.21.15.92:0/2164695589 conn(0x7f38a0068a30 msgr2=0x7f38a0069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 shutdown_connections 2023-05-16T07:19:05.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.420+0000 7f389f59e700 1 -- 172.21.15.92:0/2164695589 wait complete. 2023-05-16T07:19:05.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:19:05.502 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:05.499+0000 7fb8bd688680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:05.555 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:05.552+0000 7f95c378c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:05.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.552+0000 7f71ce21f700 1 Processor -- start 2023-05-16T07:19:05.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.552+0000 7f71ce21f700 1 -- start start 2023-05-16T07:19:05.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71ce21f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c80a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71ce21f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71c8062980 0x7f71c8062e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71ce21f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c8063320 con 0x7f71c8150d00 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71ce21f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c8151520 con 0x7f71c8062980 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71ce21f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80a7aa0 con 0x7f71c8150920 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71c8062980 0x7f71c8062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71c8062980 0x7f71c8062e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48376/0 (socket says 172.21.15.92:48376) 2023-05-16T07:19:05.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 -- 172.21.15.92:0/1456856072 learned_addr learned my addr 172.21.15.92:0/1456856072 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:05.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cd21d700 1 --2- 172.21.15.92:0/1456856072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c80a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 -- 172.21.15.92:0/1456856072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 msgr2=0x7f71c80a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:05.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 --2- 172.21.15.92:0/1456856072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c80a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 -- 172.21.15.92:0/1456856072 >> v1:172.21.15.92:6789/0 conn(0x7f71c8150d00 legacy=0x7f71c81519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.553+0000 7f71cda1e700 1 -- 172.21.15.92:0/1456856072 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c81520c0 con 0x7f71c8062980 2023-05-16T07:19:05.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.554+0000 7f71cda1e700 1 --2- 172.21.15.92:0/1456856072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c8062e30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f71c0009f20 tx=0x7f71c0009840).ready entity=mon.0 client_cookie=5d64648b5e79d938 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:05.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.555+0000 7f71bffff700 1 -- 172.21.15.92:0/1456856072 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000e070 con 0x7f71c8062980 2023-05-16T07:19:05.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.555+0000 7f71bffff700 1 -- 172.21.15.92:0/1456856072 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c000b9d0 con 0x7f71c8062980 2023-05-16T07:19:05.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.555+0000 7f71bffff700 1 -- 172.21.15.92:0/1456856072 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000bd50 con 0x7f71c8062980 2023-05-16T07:19:05.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.555+0000 7f71ce21f700 1 -- 172.21.15.92:0/1456856072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 msgr2=0x7f71c8062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.555+0000 7f71ce21f700 1 --2- 172.21.15.92:0/1456856072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c8062e30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f71c0009f20 tx=0x7f71c0009840).stop 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.556+0000 7f71ce21f700 1 -- 172.21.15.92:0/1456856072 shutdown_connections 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.556+0000 7f71ce21f700 1 --2- 172.21.15.92:0/1456856072 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c80a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.556+0000 7f71ce21f700 1 --2- 172.21.15.92:0/1456856072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c8062e30 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.556+0000 7f71ce21f700 1 -- 172.21.15.92:0/1456856072 >> 172.21.15.92:0/1456856072 conn(0x7f71c8068a30 msgr2=0x7f71c80c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.556+0000 7f71ce21f700 1 -- 172.21.15.92:0/1456856072 shutdown_connections 2023-05-16T07:19:05.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.556+0000 7f71ce21f700 1 -- 172.21.15.92:0/1456856072 wait complete. 2023-05-16T07:19:05.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.557+0000 7f71ce21f700 1 Processor -- start 2023-05-16T07:19:05.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.557+0000 7f71ce21f700 1 -- start start 2023-05-16T07:19:05.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c81422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c8145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71c8150d00 0x7f71c814a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81460b0 con 0x7f71c8062980 2023-05-16T07:19:05.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80abc90 con 0x7f71c8150d00 2023-05-16T07:19:05.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80abf70 con 0x7f71c8150920 2023-05-16T07:19:05.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c81422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c81422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48378/0 (socket says 172.21.15.92:48378) 2023-05-16T07:19:05.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 -- 172.21.15.92:0/982841967 learned_addr learned my addr 172.21.15.92:0/982841967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:05.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cda1e700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71c8150d00 0x7f71c814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cca1c700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c8145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 -- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 msgr2=0x7f71c8145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c8145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 -- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71c8150d00 msgr2=0x7f71c814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71c8150d00 0x7f71c814a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 -- 172.21.15.92:0/982841967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0009ae0 con 0x7f71c8062980 2023-05-16T07:19:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71cd21d700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c81422a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f71c400e950 tx=0x7f71c4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4012600 con 0x7f71c8062980 2023-05-16T07:19:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4012760 con 0x7f71c8062980 2023-05-16T07:19:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c80ac250 con 0x7f71c8062980 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4012ae0 con 0x7f71c8062980 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.558+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c8142790 con 0x7f71c8062980 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.560+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c80ac770 con 0x7f71c8062980 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.560+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f71c4022070 con 0x7f71c8062980 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.560+0000 7f71bd7fa700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f71ac05eeb0 0x7f71ac061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.560+0000 7f71cca1c700 1 -- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f71ac05eeb0 msgr2=0x7f71ac061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.560+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f71c407a2d0 con 0x7f71c8062980 2023-05-16T07:19:05.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.560+0000 7f71cca1c700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f71ac05eeb0 0x7f71ac061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:05.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.568+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f71c80ac770 con 0x7f71c8062980 2023-05-16T07:19:05.630 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:05.627+0000 7f95c378c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:05.662 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:05.659+0000 7fb8bd688680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:05.701 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:05.693+0000 7f95c378c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:05.738 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:05.734+0000 7fb8bd688680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:05.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.740+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71c80ac770 con 0x7f71c8062980 2023-05-16T07:19:05.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.740+0000 7f71bd7fa700 1 -- 172.21.15.92:0/982841967 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f71c80ac770 con 0x7f71c8062980 2023-05-16T07:19:05.744 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:19:05.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f71ac05eeb0 msgr2=0x7f71ac061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:05.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f71ac05eeb0 0x7f71ac061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:05.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 msgr2=0x7f71c81422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c81422a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f71c400e950 tx=0x7f71c4009060).stop 2023-05-16T07:19:05.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 shutdown_connections 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f71ac05eeb0 0x7f71ac061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71c8150d00 0x7f71c814a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c8062980 0x7f71c81422a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 --2- 172.21.15.92:0/982841967 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f71c8150920 0x7f71c8145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 >> 172.21.15.92:0/982841967 conn(0x7f71c8068a30 msgr2=0x7f71c814bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 shutdown_connections 2023-05-16T07:19:05.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.742+0000 7f71ce21f700 1 -- 172.21.15.92:0/982841967 wait complete. 2023-05-16T07:19:05.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:05.809 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:05.806+0000 7fb8bd688680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:05.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 Processor -- start 2023-05-16T07:19:05.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 -- start start 2023-05-16T07:19:05.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154170 0x7f9cf00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9cf0154550 0x7f9cf00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf00c0660 con 0x7f9cf00696a0 2023-05-16T07:19:05.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf00c1b00 con 0x7f9cf0154550 2023-05-16T07:19:05.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.869+0000 7f9cf69a8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00bfd30 con 0x7f9cf0154170 2023-05-16T07:19:05.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9cf0154550 0x7f9cf00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9cf0154550 0x7f9cf00c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48386/0 (socket says 172.21.15.92:48386) 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 -- 172.21.15.92:0/3883192099 learned_addr learned my addr 172.21.15.92:0/3883192099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9ceffff700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154170 0x7f9cf00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 -- 172.21.15.92:0/3883192099 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154170 msgr2=0x7f9cf00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154170 0x7f9cf00c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 -- 172.21.15.92:0/3883192099 >> v1:172.21.15.158:6789/0 conn(0x7f9cf00696a0 legacy=0x7f9cf0157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 -- 172.21.15.92:0/3883192099 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00c0010 con 0x7f9cf0154550 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9ceffff700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154170 0x7f9cf00c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9cef7fe700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154550 0x7f9cf00c6160 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f9ce4006fd0 tx=0x7f9ce40090f0).ready entity=mon.0 client_cookie=901900bff29ed6da server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9ceeffd700 1 -- 172.21.15.92:0/3883192099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400b600 con 0x7f9cf0154550 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9ceeffd700 1 -- 172.21.15.92:0/3883192099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce400b760 con 0x7f9cf0154550 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.870+0000 7f9ceeffd700 1 -- 172.21.15.92:0/3883192099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400bae0 con 0x7f9cf0154550 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 -- 172.21.15.92:0/3883192099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154550 msgr2=0x7f9cf00c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154550 0x7f9cf00c6160 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f9ce4006fd0 tx=0x7f9ce40090f0).stop 2023-05-16T07:19:05.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 -- 172.21.15.92:0/3883192099 shutdown_connections 2023-05-16T07:19:05.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154170 0x7f9cf00c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/3883192099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154550 0x7f9cf00c6160 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 -- 172.21.15.92:0/3883192099 >> 172.21.15.92:0/3883192099 conn(0x7f9cf00cb750 msgr2=0x7f9cf00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:05.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 -- 172.21.15.92:0/3883192099 shutdown_connections 2023-05-16T07:19:05.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.871+0000 7f9cf69a8700 1 -- 172.21.15.92:0/3883192099 wait complete. 2023-05-16T07:19:05.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.872+0000 7f9cf69a8700 1 Processor -- start 2023-05-16T07:19:05.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.872+0000 7f9cf69a8700 1 -- start start 2023-05-16T07:19:05.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cf00696a0 0x7f9cf00b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 0x7f9cf013f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154550 0x7f9cf00c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00aef50 con 0x7f9cf0154170 2023-05-16T07:19:05.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00af230 con 0x7f9cf00696a0 2023-05-16T07:19:05.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00af510 con 0x7f9cf0154550 2023-05-16T07:19:05.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 0x7f9cf013f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 0x7f9cf013f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48394/0 (socket says 172.21.15.92:48394) 2023-05-16T07:19:05.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 -- 172.21.15.92:0/1666495703 learned_addr learned my addr 172.21.15.92:0/1666495703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:05.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf4f45700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154550 0x7f9cf00c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9ceffff700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cf00696a0 0x7f9cf00b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 -- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154550 msgr2=0x7f9cf00c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154550 0x7f9cf00c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 -- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cf00696a0 msgr2=0x7f9cf00b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cf00696a0 0x7f9cf00b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 -- 172.21.15.92:0/1666495703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009370 con 0x7f9cf0154170 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cef7fe700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 0x7f9cf013f710 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f9cf0083060 tx=0x7f9ce4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400bea0 con 0x7f9cf0154170 2023-05-16T07:19:05.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce400b8d0 con 0x7f9cf0154170 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf00af7f0 con 0x7f9cf0154170 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce401b640 con 0x7f9cf0154170 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.873+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf00ae8e0 con 0x7f9cf0154170 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.874+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf00666c0 con 0x7f9cf0154170 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.875+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f9ce4021020 con 0x7f9cf0154170 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.875+0000 7f9cd7fff700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 0x7f9cdc061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:05.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.875+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f9ce407b4c0 con 0x7f9cf0154170 2023-05-16T07:19:05.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.875+0000 7f9ceffff700 1 -- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 msgr2=0x7f9cdc061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:05.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.875+0000 7f9ceffff700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 0x7f9cdc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:05.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:05.883+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9ce404c820 con 0x7f9cf0154170 2023-05-16T07:19:05.949 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:05.946+0000 7f95c378c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:06.074 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:06.071+0000 7fb8bd688680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.076+0000 7f9ceffff700 1 -- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 msgr2=0x7f9cdc061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.076+0000 7f9ceffff700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 0x7f9cdc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.090+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cf006c350 con 0x7f9cf0154170 2023-05-16T07:19:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.090+0000 7f9cd7fff700 1 -- 172.21.15.92:0/1666495703 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f9ce404fe40 con 0x7f9cf0154170 2023-05-16T07:19:06.098 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:06.098 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:18:17.423562+0000","modified":"2023-05-16T07:18:19.645369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7017":{"gid":7017,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1226709881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1226709881},{"type":"v1","addr":"172.21.15.92:6835","nonce":1226709881}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-05-16T07:19:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 msgr2=0x7f9cdc061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 0x7f9cdc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 msgr2=0x7f9cf013f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 0x7f9cf013f710 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f9cf0083060 tx=0x7f9ce4017000).stop 2023-05-16T07:19:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 shutdown_connections 2023-05-16T07:19:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cdc05eeb0 0x7f9cdc061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9cf0154550 0x7f9cf00c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cf00696a0 0x7f9cf00b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 --2- 172.21.15.92:0/1666495703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf0154170 0x7f9cf013f710 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 >> 172.21.15.92:0/1666495703 conn(0x7f9cf00cb750 msgr2=0x7f9cf00ab150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 shutdown_connections 2023-05-16T07:19:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.093+0000 7f9cf69a8700 1 -- 172.21.15.92:0/1666495703 wait complete. 2023-05-16T07:19:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-05-16T07:19:06.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:06.133 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:06.129+0000 7f95c378c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:06.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.223+0000 7fbc9aa31700 1 Processor -- start 2023-05-16T07:19:06.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.223+0000 7fbc9aa31700 1 -- start start 2023-05-16T07:19:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc9aa31700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc94154170 0x7fbc940c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc9aa31700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc940c0660 con 0x7fbc94069db0 2023-05-16T07:19:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc9aa31700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc940bfd30 con 0x7fbc94154550 2023-05-16T07:19:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc9aa31700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc940c0010 con 0x7fbc94154170 2023-05-16T07:19:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc94154170 0x7fbc940c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc94154170 0x7fbc940c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48402/0 (socket says 172.21.15.92:48402) 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 -- 172.21.15.92:0/3514582225 learned_addr learned my addr 172.21.15.92:0/3514582225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc98fce700 1 --1- 172.21.15.92:0/3514582225 >> v1:172.21.15.92:6789/0 conn(0x7fbc94069db0 0x7fbc940c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59154/0 (socket says 172.21.15.92:59154) 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 -- 172.21.15.92:0/3514582225 >> v1:172.21.15.158:6789/0 conn(0x7fbc94154550 legacy=0x7fbc940696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 -- 172.21.15.92:0/3514582225 >> v1:172.21.15.92:6789/0 conn(0x7fbc94069db0 legacy=0x7fbc940c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 -- 172.21.15.92:0/3514582225 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940c02f0 con 0x7fbc94154170 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc92ffd700 1 -- 172.21.15.92:0/3514582225 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854934042 0 0) 0x7fbc940c0660 con 0x7fbc94069db0 2023-05-16T07:19:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc93fff700 1 --2- 172.21.15.92:0/3514582225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154170 0x7fbc940c5c70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fbc7c006fd0 tx=0x7fbc7c0090f0).ready entity=mon.0 client_cookie=b792de1e382099cf server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.224+0000 7fbc92ffd700 1 -- 172.21.15.92:0/3514582225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00b600 con 0x7fbc94154170 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc92ffd700 1 -- 172.21.15.92:0/3514582225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c00b760 con 0x7fbc94154170 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc92ffd700 1 -- 172.21.15.92:0/3514582225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00bb00 con 0x7fbc94154170 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 -- 172.21.15.92:0/3514582225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154170 msgr2=0x7fbc940c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/3514582225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154170 0x7fbc940c5c70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fbc7c006fd0 tx=0x7fbc7c0090f0).stop 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 -- 172.21.15.92:0/3514582225 shutdown_connections 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/3514582225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154170 0x7fbc940c5c70 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 -- 172.21.15.92:0/3514582225 >> 172.21.15.92:0/3514582225 conn(0x7fbc940cb700 msgr2=0x7fbc940c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:06.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 -- 172.21.15.92:0/3514582225 shutdown_connections 2023-05-16T07:19:06.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.225+0000 7fbc9aa31700 1 -- 172.21.15.92:0/3514582225 wait complete. 2023-05-16T07:19:06.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.226+0000 7fbc9aa31700 1 Processor -- start 2023-05-16T07:19:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 -- start start 2023-05-16T07:19:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc94069db0 0x7fbc94158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc94154170 0x7fbc9414ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 0x7fbc940c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941581f0 con 0x7fbc94154550 2023-05-16T07:19:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941578c0 con 0x7fbc94069db0 2023-05-16T07:19:06.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc9aa31700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc94157ba0 con 0x7fbc94154170 2023-05-16T07:19:06.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc98fce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 0x7fbc940c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc98fce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 0x7fbc940c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48412/0 (socket says 172.21.15.92:48412) 2023-05-16T07:19:06.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc94069db0 0x7fbc94158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc98fce700 1 -- 172.21.15.92:0/1030020078 learned_addr learned my addr 172.21.15.92:0/1030020078 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.227+0000 7fbc937fe700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc94154170 0x7fbc9414ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc98fce700 1 -- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc94154170 msgr2=0x7fbc9414ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc98fce700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc94154170 0x7fbc9414ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc98fce700 1 -- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc94069db0 msgr2=0x7fbc94158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc98fce700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc94069db0 0x7fbc94158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc98fce700 1 -- 172.21.15.92:0/1030020078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c009370 con 0x7fbc94154550 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc98fce700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 0x7fbc940c9c50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fbc8c00c7e0 tx=0x7fbc8c007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:06.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c005c70 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c005dd0 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc94157e80 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c007320 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.228+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9414cf40 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.229+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc941583a0 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.229+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7fbc8c010030 con 0x7fbc94154550 2023-05-16T07:19:06.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.230+0000 7fbc8bfff700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc7405eeb0 0x7fbc74061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.230+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fbc8c079d90 con 0x7fbc94154550 2023-05-16T07:19:06.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.230+0000 7fbc93fff700 1 -- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc7405eeb0 msgr2=0x7fbc74061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:06.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.230+0000 7fbc93fff700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc7405eeb0 0x7fbc74061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:06.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.234+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbc941583a0 con 0x7fbc94154550 2023-05-16T07:19:06.266 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:06.262+0000 7fb8bd688680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:06.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.403+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc940666c0 con 0x7fbc94154550 2023-05-16T07:19:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.404+0000 7fbc8bfff700 1 -- 172.21.15.92:0/1030020078 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+18118 (secure 0 0 0) 0x7fbc8c04afc0 con 0x7fbc94154550 2023-05-16T07:19:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:06.409 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":108,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:28.827155+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:18:15.946822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-05-16T07:18:16.384841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3022628291":"2023-05-17T07:18:28.827125+0000","172.21.15.158:6833/86301":"2023-05-17T07:18:28.827125+0000","172.21.15.158:6832/86301":"2023-05-17T07:18:28.827125+0000","172.21.15.158:0/157465347":"2023-05-17T07:18:28.827125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc7405eeb0 msgr2=0x7fbc74061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc7405eeb0 0x7fbc74061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 msgr2=0x7fbc940c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 0x7fbc940c9c50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fbc8c00c7e0 tx=0x7fbc8c007f60).stop 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 shutdown_connections 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc7405eeb0 0x7fbc74061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc94154550 0x7fbc940c9c50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbc94069db0 0x7fbc94158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 --2- 172.21.15.92:0/1030020078 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbc94154170 0x7fbc9414ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.406+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 >> 172.21.15.92:0/1030020078 conn(0x7fbc940cb700 msgr2=0x7fbc940abfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.407+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 shutdown_connections 2023-05-16T07:19:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.407+0000 7fbc9aa31700 1 -- 172.21.15.92:0/1030020078 wait complete. 2023-05-16T07:19:06.423 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:19:06.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:06.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.535+0000 7f18065aa700 1 Processor -- start 2023-05-16T07:19:06.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.535+0000 7f18065aa700 1 -- start start 2023-05-16T07:19:06.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f18065aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18000c1550 0x7f18000c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f18065aa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18000bc5f0 con 0x7f1800151160 2023-05-16T07:19:06.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f18065aa700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1800066670 con 0x7f1800150d80 2023-05-16T07:19:06.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f18065aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18000bbfe0 con 0x7f18000c1550 2023-05-16T07:19:06.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1804da7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18000c1550 0x7f18000c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1805da9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1800151160 0x7f18001515f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59166/0 (socket says 172.21.15.92:59166) 2023-05-16T07:19:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1805da9700 1 -- 172.21.15.92:0/1888955057 learned_addr learned my addr 172.21.15.92:0/1888955057 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1804da7700 1 --2- 172.21.15.92:0/1888955057 >> v2:172.21.15.92:3300/0 conn(0x7f18000c1550 0x7f18000c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48418/0 (socket says 172.21.15.92:48418) 2023-05-16T07:19:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1804da7700 1 -- 172.21.15.92:0/1888955057 >> v1:172.21.15.158:6789/0 conn(0x7f1800150d80 legacy=0x7f1800065bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f17f7fff700 1 -- 172.21.15.92:0/1888955057 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286713733 0 0) 0x7f18000bc5f0 con 0x7f1800151160 2023-05-16T07:19:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1804da7700 1 -- 172.21.15.92:0/1888955057 >> v1:172.21.15.92:6789/0 conn(0x7f1800151160 legacy=0x7f18001515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1804da7700 1 -- 172.21.15.92:0/1888955057 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1800066390 con 0x7f18000c1550 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.536+0000 7f1804da7700 1 --2- 172.21.15.92:0/1888955057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f18000c7740 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f17f0009760 tx=0x7f17f0009060).ready entity=mon.0 client_cookie=c5b5fe9ec7ea109b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f17f7fff700 1 -- 172.21.15.92:0/1888955057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000b580 con 0x7f18000c1550 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f17f7fff700 1 -- 172.21.15.92:0/1888955057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f000b6e0 con 0x7f18000c1550 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f17f7fff700 1 -- 172.21.15.92:0/1888955057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000ba60 con 0x7f18000c1550 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 -- 172.21.15.92:0/1888955057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 msgr2=0x7f18000c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 --2- 172.21.15.92:0/1888955057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f18000c7740 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f17f0009760 tx=0x7f17f0009060).stop 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 -- 172.21.15.92:0/1888955057 shutdown_connections 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 --2- 172.21.15.92:0/1888955057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f18000c7740 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 -- 172.21.15.92:0/1888955057 >> 172.21.15.92:0/1888955057 conn(0x7f1800069940 msgr2=0x7f18000c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 -- 172.21.15.92:0/1888955057 shutdown_connections 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.537+0000 7f18065aa700 1 -- 172.21.15.92:0/1888955057 wait complete. 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.538+0000 7f18065aa700 1 Processor -- start 2023-05-16T07:19:06.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.538+0000 7f18065aa700 1 -- start start 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18065aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f1800067040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18065aa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1800150d80 0x7f180014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18065aa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1800151160 0x7f180014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18065aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18001653e0 con 0x7f18000c1550 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18065aa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000c03f0 con 0x7f1800150d80 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18065aa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000bff60 con 0x7f1800151160 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18055a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f1800067040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18055a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f1800067040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48422/0 (socket says 172.21.15.92:48422) 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f18055a8700 1 -- 172.21.15.92:0/4032862291 learned_addr learned my addr 172.21.15.92:0/4032862291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.539+0000 7f1804da7700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1800150d80 0x7f180014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f1805da9700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1800151160 0x7f180014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18055a8700 1 -- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1800151160 msgr2=0x7f180014fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18055a8700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1800151160 0x7f180014fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18055a8700 1 -- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1800150d80 msgr2=0x7f180014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18055a8700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1800150d80 0x7f180014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18055a8700 1 -- 172.21.15.92:0/4032862291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00092a0 con 0x7f18000c1550 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18055a8700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f1800067040 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f17fc00c4c0 tx=0x7f17fc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007020 con 0x7f18000c1550 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc001e90 con 0x7f18000c1550 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1800150290 con 0x7f18000c1550 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0073b0 con 0x7f18000c1550 2023-05-16T07:19:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.540+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18000bfbf0 con 0x7f18000c1550 2023-05-16T07:19:06.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.541+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18000c0240 con 0x7f18000c1550 2023-05-16T07:19:06.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.541+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f17fc010070 con 0x7f18000c1550 2023-05-16T07:19:06.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.542+0000 7f17f57fa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 0x7f17e4061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.542+0000 7f1804da7700 1 -- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 msgr2=0x7f17e4061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:06.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.542+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f17fc0798b0 con 0x7f18000c1550 2023-05-16T07:19:06.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.542+0000 7f1804da7700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 0x7f17e4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:06.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.546+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f17fc04a9d0 con 0x7f18000c1550 2023-05-16T07:19:06.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.743+0000 7f1804da7700 1 -- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 msgr2=0x7f17e4061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:06.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.743+0000 7f1804da7700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 0x7f17e4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.756+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1800047150 con 0x7f18000c1550 2023-05-16T07:19:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.757+0000 7f17f57fa700 1 -- 172.21.15.92:0/4032862291 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f17fc077a30 con 0x7f18000c1550 2023-05-16T07:19:06.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:06.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:18:17.423562+0000","modified":"2023-05-16T07:18:19.645369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7017":{"gid":7017,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1226709881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1226709881},{"type":"v1","addr":"172.21.15.92:6835","nonce":1226709881}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-05-16T07:19:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 msgr2=0x7f17e4061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 0x7f17e4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 msgr2=0x7f1800067040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f1800067040 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f17fc00c4c0 tx=0x7f17fc007e00).stop 2023-05-16T07:19:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 shutdown_connections 2023-05-16T07:19:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17e405eeb0 0x7f17e4061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1800151160 0x7f180014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18000c1550 0x7f1800067040 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 --2- 172.21.15.92:0/4032862291 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1800150d80 0x7f180014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 >> 172.21.15.92:0/4032862291 conn(0x7f1800069940 msgr2=0x7f18000c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 shutdown_connections 2023-05-16T07:19:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.759+0000 7f18065aa700 1 -- 172.21.15.92:0/4032862291 wait complete. 2023-05-16T07:19:06.766 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-05-16T07:19:06.774 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:06.769+0000 7f95c378c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:06.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:06.839 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:06.835+0000 7f95c378c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.890+0000 7f381a2da700 1 Processor -- start 2023-05-16T07:19:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.890+0000 7f381a2da700 1 -- start start 2023-05-16T07:19:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f381a2da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3814153eb0 0x7f3814154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f381a2da700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3814065000 0x7f3814065480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f381a2da700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3814067560 con 0x7f3814068c70 2023-05-16T07:19:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f381a2da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3814065970 con 0x7f3814153eb0 2023-05-16T07:19:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f381a2da700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38140bc9b0 con 0x7f3814065000 2023-05-16T07:19:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f3813fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3814153eb0 0x7f3814154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f3813fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3814153eb0 0x7f3814154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48464/0 (socket says 172.21.15.92:48464) 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f3813fff700 1 -- 172.21.15.92:0/3396475587 learned_addr learned my addr 172.21.15.92:0/3396475587 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.891+0000 7f3818877700 1 --2- 172.21.15.92:0/3396475587 >> v2:172.21.15.158:3300/0 conn(0x7f3814065000 0x7f3814065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3813fff700 1 -- 172.21.15.92:0/3396475587 >> v2:172.21.15.158:3300/0 conn(0x7f3814065000 msgr2=0x7f3814065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3813fff700 1 --2- 172.21.15.92:0/3396475587 >> v2:172.21.15.158:3300/0 conn(0x7f3814065000 0x7f3814065480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3813fff700 1 -- 172.21.15.92:0/3396475587 >> v1:172.21.15.158:6789/0 conn(0x7f3814068c70 legacy=0x7f3814066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3813fff700 1 -- 172.21.15.92:0/3396475587 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140bcc90 con 0x7f3814153eb0 2023-05-16T07:19:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3813fff700 1 --2- 172.21.15.92:0/3396475587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814153eb0 0x7f3814154290 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f3814064dd0 tx=0x7f38080090f0).ready entity=mon.0 client_cookie=dad44224df377ea6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3812ffd700 1 -- 172.21.15.92:0/3396475587 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380800b520 con 0x7f3814153eb0 2023-05-16T07:19:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3812ffd700 1 -- 172.21.15.92:0/3396475587 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380800b680 con 0x7f3814153eb0 2023-05-16T07:19:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f3812ffd700 1 -- 172.21.15.92:0/3396475587 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380800ba00 con 0x7f3814153eb0 2023-05-16T07:19:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f381a2da700 1 -- 172.21.15.92:0/3396475587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814153eb0 msgr2=0x7f3814154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.892+0000 7f381a2da700 1 --2- 172.21.15.92:0/3396475587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814153eb0 0x7f3814154290 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f3814064dd0 tx=0x7f38080090f0).stop 2023-05-16T07:19:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.893+0000 7f381a2da700 1 -- 172.21.15.92:0/3396475587 shutdown_connections 2023-05-16T07:19:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.893+0000 7f381a2da700 1 --2- 172.21.15.92:0/3396475587 >> v2:172.21.15.158:3300/0 conn(0x7f3814065000 0x7f3814065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.893+0000 7f381a2da700 1 --2- 172.21.15.92:0/3396475587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814153eb0 0x7f3814154290 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.893+0000 7f381a2da700 1 -- 172.21.15.92:0/3396475587 >> 172.21.15.92:0/3396475587 conn(0x7f38140c4430 msgr2=0x7f38140c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.893+0000 7f381a2da700 1 -- 172.21.15.92:0/3396475587 shutdown_connections 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.893+0000 7f381a2da700 1 -- 172.21.15.92:0/3396475587 wait complete. 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.894+0000 7f381a2da700 1 Processor -- start 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.894+0000 7f381a2da700 1 -- start start 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.894+0000 7f381a2da700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3814065000 0x7f38140be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3814153eb0 0x7f38140c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 0x7f381414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381414b3f0 con 0x7f3814163ca0 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140c2470 con 0x7f3814065000 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381414a540 con 0x7f3814153eb0 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3818877700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 0x7f381414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3818877700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 0x7f381414a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48470/0 (socket says 172.21.15.92:48470) 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3818877700 1 -- 172.21.15.92:0/4195028462 learned_addr learned my addr 172.21.15.92:0/4195028462 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3814065000 0x7f38140be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f38137fe700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3814153eb0 0x7f38140c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 -- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3814153eb0 msgr2=0x7f38140c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3814153eb0 0x7f38140c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 -- 172.21.15.92:0/4195028462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 msgr2=0x7f381414a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 0x7f381414a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 -- 172.21.15.92:0/4195028462 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808009370 con 0x7f3814065000 2023-05-16T07:19:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3818877700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 0x7f381414a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f3813fff700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3814065000 0x7f38140be4e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f3808014000 tx=0x7f3808017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808006d90 con 0x7f3814065000 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3808000bc0 con 0x7f3814065000 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381414a7e0 con 0x7f3814065000 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380801b410 con 0x7f3814065000 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.895+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381414ada0 con 0x7f3814065000 2023-05-16T07:19:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.896+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381406c350 con 0x7f3814065000 2023-05-16T07:19:06.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.897+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f380802b070 con 0x7f3814065000 2023-05-16T07:19:06.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.897+0000 7f37fbfff700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 0x7f3800061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:06.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.897+0000 7f38137fe700 1 -- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 msgr2=0x7f3800061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.897+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f380807b6e0 con 0x7f3814065000 2023-05-16T07:19:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.897+0000 7f38137fe700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 0x7f3800061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:06.902+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f380804c910 con 0x7f3814065000 2023-05-16T07:19:06.914 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:06.909+0000 7f95c378c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:06.940 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:06.937+0000 7fb8bd688680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:07.009 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.005+0000 7fb8bd688680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:07.056 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:07.052+0000 7f95c378c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:07.083 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.080+0000 7fb8bd688680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.098+0000 7f38137fe700 1 -- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 msgr2=0x7f3800061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.098+0000 7f38137fe700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 0x7f3800061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.111+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38140bcf70 con 0x7f3814065000 2023-05-16T07:19:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.112+0000 7f37fbfff700 1 -- 172.21.15.92:0/4195028462 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f380801f020 con 0x7f3814065000 2023-05-16T07:19:07.116 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:07.116 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:18:17.423562+0000","modified":"2023-05-16T07:18:19.645369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7017":{"gid":7017,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1226709881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":1226709881},{"type":"v1","addr":"172.21.15.92:6835","nonce":1226709881}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-05-16T07:19:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 msgr2=0x7f3800061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 0x7f3800061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3814065000 msgr2=0x7f38140be4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3814065000 0x7f38140be4e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f3808014000 tx=0x7f3808017000).stop 2023-05-16T07:19:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 shutdown_connections 2023-05-16T07:19:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f380005eeb0 0x7f3800061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3814163ca0 0x7f381414a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3814065000 0x7f38140be4e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 --2- 172.21.15.92:0/4195028462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3814153eb0 0x7f38140c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 >> 172.21.15.92:0/4195028462 conn(0x7f38140c4430 msgr2=0x7f38140c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 shutdown_connections 2023-05-16T07:19:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.114+0000 7f381a2da700 1 -- 172.21.15.92:0/4195028462 wait complete. 2023-05-16T07:19:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2023-05-16T07:19:07.126 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:07.121+0000 7f95c378c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:07.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:07.237 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.234+0000 7fb8bd688680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:07.238 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:07.234+0000 7f95c378c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:07.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.241+0000 7f52748a6700 1 Processor -- start 2023-05-16T07:19:07.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.241+0000 7f52748a6700 1 -- start start 2023-05-16T07:19:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f52748a6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f52701386a0 0x7f52700c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f52748a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52700c30a0 0x7f52700622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f52748a6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52700c37f0 con 0x7f52700627a0 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f52748a6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5270138a80 con 0x7f52700c30a0 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f52748a6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52700be3d0 con 0x7f52701386a0 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f526dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52700c30a0 0x7f52700622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f526dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52700c30a0 0x7f52700622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48476/0 (socket says 172.21.15.92:48476) 2023-05-16T07:19:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f526dd9b700 1 -- 172.21.15.92:0/1176721411 learned_addr learned my addr 172.21.15.92:0/1176721411 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.242+0000 7f526e59c700 1 --2- 172.21.15.92:0/1176721411 >> v2:172.21.15.158:3300/0 conn(0x7f52701386a0 0x7f52700c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526dd9b700 1 -- 172.21.15.92:0/1176721411 >> v2:172.21.15.158:3300/0 conn(0x7f52701386a0 msgr2=0x7f52700c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526dd9b700 1 --2- 172.21.15.92:0/1176721411 >> v2:172.21.15.158:3300/0 conn(0x7f52701386a0 0x7f52700c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526dd9b700 1 -- 172.21.15.92:0/1176721411 >> v1:172.21.15.158:6789/0 conn(0x7f52700627a0 legacy=0x7f5270063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526dd9b700 1 -- 172.21.15.92:0/1176721411 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52700bef90 con 0x7f52700c30a0 2023-05-16T07:19:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526dd9b700 1 --2- 172.21.15.92:0/1176721411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f52700622b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f52700c89b0 tx=0x7f52600090f0).ready entity=mon.0 client_cookie=67cdffd6eb78abdf server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526d59a700 1 -- 172.21.15.92:0/1176721411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000b520 con 0x7f52700c30a0 2023-05-16T07:19:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526d59a700 1 -- 172.21.15.92:0/1176721411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000b680 con 0x7f52700c30a0 2023-05-16T07:19:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f526d59a700 1 -- 172.21.15.92:0/1176721411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000ba00 con 0x7f52700c30a0 2023-05-16T07:19:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f52748a6700 1 -- 172.21.15.92:0/1176721411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 msgr2=0x7f52700622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.243+0000 7f52748a6700 1 --2- 172.21.15.92:0/1176721411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f52700622b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f52700c89b0 tx=0x7f52600090f0).stop 2023-05-16T07:19:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.244+0000 7f52748a6700 1 -- 172.21.15.92:0/1176721411 shutdown_connections 2023-05-16T07:19:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.244+0000 7f52748a6700 1 --2- 172.21.15.92:0/1176721411 >> v2:172.21.15.158:3300/0 conn(0x7f52701386a0 0x7f52700c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.244+0000 7f52748a6700 1 --2- 172.21.15.92:0/1176721411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f52700622b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.244+0000 7f52748a6700 1 -- 172.21.15.92:0/1176721411 >> 172.21.15.92:0/1176721411 conn(0x7f52700c8aa0 msgr2=0x7f52700c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.244+0000 7f52748a6700 1 -- 172.21.15.92:0/1176721411 shutdown_connections 2023-05-16T07:19:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.244+0000 7f52748a6700 1 -- 172.21.15.92:0/1176721411 wait complete. 2023-05-16T07:19:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.245+0000 7f52748a6700 1 Processor -- start 2023-05-16T07:19:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.245+0000 7f52748a6700 1 -- start start 2023-05-16T07:19:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f52748a6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52700627a0 0x7f5270135970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f52748a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f5270156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f52748a6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52701386a0 0x7f5270156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f52748a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700ac6f0 con 0x7f52700c30a0 2023-05-16T07:19:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f52748a6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700ac9d0 con 0x7f52701386a0 2023-05-16T07:19:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f52748a6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5270157380 con 0x7f52700627a0 2023-05-16T07:19:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f5270156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f5270156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48484/0 (socket says 172.21.15.92:48484) 2023-05-16T07:19:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 -- 172.21.15.92:0/1199088918 learned_addr learned my addr 172.21.15.92:0/1199088918 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526e59c700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52700627a0 0x7f5270135970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526ed9d700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52701386a0 0x7f5270156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 -- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52700627a0 msgr2=0x7f5270135970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52700627a0 0x7f5270135970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 -- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52701386a0 msgr2=0x7f5270156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52701386a0 0x7f5270156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.246+0000 7f526dd9b700 1 -- 172.21.15.92:0/1199088918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260009370 con 0x7f52700c30a0 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.247+0000 7f526dd9b700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f5270156870 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f5260009130 tx=0x7f5260017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.247+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260000bc0 con 0x7f52700c30a0 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.247+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000b7f0 con 0x7f52700c30a0 2023-05-16T07:19:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.247+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260000d20 con 0x7f52700c30a0 2023-05-16T07:19:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.247+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5270157660 con 0x7f52700c30a0 2023-05-16T07:19:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.247+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527014f840 con 0x7f52700c30a0 2023-05-16T07:19:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.248+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f526002c070 con 0x7f52700c30a0 2023-05-16T07:19:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.248+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527006c350 con 0x7f52700c30a0 2023-05-16T07:19:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.248+0000 7f525e7fc700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f525005eee0 0x7f52500613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.248+0000 7f526e59c700 1 -- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f525005eee0 msgr2=0x7f52500613a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.249+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f526007b9f0 con 0x7f52700c30a0 2023-05-16T07:19:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.249+0000 7f526e59c700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f525005eee0 0x7f52500613a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:07.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.252+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f526007de60 con 0x7f52700c30a0 2023-05-16T07:19:07.312 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.308+0000 7fb8bd688680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:07.345 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:07.341+0000 7f95c378c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:07.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:19:07.354 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:19:07.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.424+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f527006bcb0 con 0x7f52700c30a0 2023-05-16T07:19:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.425+0000 7f525e7fc700 1 -- 172.21.15.92:0/1199088918 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+18118 (secure 0 0 0) 0x7f526002a020 con 0x7f52700c30a0 2023-05-16T07:19:07.429 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:07.429 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":108,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:18:28.827155+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:18:15.946822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-05-16T07:18:16.384841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3022628291":"2023-05-17T07:18:28.827125+0000","172.21.15.158:6833/86301":"2023-05-17T07:18:28.827125+0000","172.21.15.158:6832/86301":"2023-05-17T07:18:28.827125+0000","172.21.15.158:0/157465347":"2023-05-17T07:18:28.827125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:07.430 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.426+0000 7fb8bd688680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f525005eee0 msgr2=0x7f52500613a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f525005eee0 0x7f52500613a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 msgr2=0x7f5270156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f5270156870 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f5260009130 tx=0x7f5260017000).stop 2023-05-16T07:19:07.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 shutdown_connections 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f525005eee0 0x7f52500613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f52700627a0 0x7f5270135970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52701386a0 0x7f5270156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 --2- 172.21.15.92:0/1199088918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52700c30a0 0x7f5270156870 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 >> 172.21.15.92:0/1199088918 conn(0x7f52700c8aa0 msgr2=0x7f5270136330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 shutdown_connections 2023-05-16T07:19:07.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.427+0000 7f52748a6700 1 -- 172.21.15.92:0/1199088918 wait complete. 2023-05-16T07:19:07.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:19:07.542 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.538+0000 7fb8bd688680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.595+0000 7fb341b96700 1 Processor -- start 2023-05-16T07:19:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.595+0000 7fb341b96700 1 -- start start 2023-05-16T07:19:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb341b96700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb33c0629c0 0x7fb33c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb341b96700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb341b96700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c0c37f0 con 0x7fb33c1386a0 2023-05-16T07:19:07.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb341b96700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c138a80 con 0x7fb33c0629c0 2023-05-16T07:19:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb341b96700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0be3d0 con 0x7fb33c0c30a0 2023-05-16T07:19:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb332ffd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb33c0629c0 0x7fb33c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33b7fe700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fb33c1386a0 0x7fb33c0622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:34606/0 (socket says 172.21.15.92:34606) 2023-05-16T07:19:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33b7fe700 1 -- 172.21.15.92:0/2882063453 learned_addr learned my addr 172.21.15.92:0/2882063453 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33affd700 1 -- 172.21.15.92:0/2882063453 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273679586 0 0) 0x7fb33c0c37f0 con 0x7fb33c1386a0 2023-05-16T07:19:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33affd700 1 -- 172.21.15.92:0/2882063453 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3280033f0 con 0x7fb33c1386a0 2023-05-16T07:19:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33bfff700 1 -- 172.21.15.92:0/2882063453 >> v2:172.21.15.158:3300/0 conn(0x7fb33c0629c0 msgr2=0x7fb33c0c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33bfff700 1 --2- 172.21.15.92:0/2882063453 >> v2:172.21.15.158:3300/0 conn(0x7fb33c0629c0 0x7fb33c0c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33bfff700 1 -- 172.21.15.92:0/2882063453 >> v1:172.21.15.158:6789/0 conn(0x7fb33c1386a0 legacy=0x7fb33c0622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33bfff700 1 -- 172.21.15.92:0/2882063453 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0bef90 con 0x7fb33c0c30a0 2023-05-16T07:19:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb332ffd700 1 --2- 172.21.15.92:0/2882063453 >> v2:172.21.15.158:3300/0 conn(0x7fb33c0629c0 0x7fb33c0c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33bfff700 1 --2- 172.21.15.92:0/2882063453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c138fa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb334009e60 tx=0x7fb334009780).ready entity=mon.2 client_cookie=f64290533f1bbe2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.597+0000 7fb33affd700 1 -- 172.21.15.92:0/2882063453 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400e070 con 0x7fb33c0c30a0 2023-05-16T07:19:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb33affd700 1 -- 172.21.15.92:0/2882063453 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33400bab0 con 0x7fb33c0c30a0 2023-05-16T07:19:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb33affd700 1 -- 172.21.15.92:0/2882063453 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400be30 con 0x7fb33c0c30a0 2023-05-16T07:19:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 -- 172.21.15.92:0/2882063453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 msgr2=0x7fb33c138fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 --2- 172.21.15.92:0/2882063453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c138fa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb334009e60 tx=0x7fb334009780).stop 2023-05-16T07:19:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 -- 172.21.15.92:0/2882063453 shutdown_connections 2023-05-16T07:19:07.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 --2- 172.21.15.92:0/2882063453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c138fa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 --2- 172.21.15.92:0/2882063453 >> v2:172.21.15.158:3300/0 conn(0x7fb33c0629c0 0x7fb33c0c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 -- 172.21.15.92:0/2882063453 >> 172.21.15.92:0/2882063453 conn(0x7fb33c0c8aa0 msgr2=0x7fb33c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:07.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 -- 172.21.15.92:0/2882063453 shutdown_connections 2023-05-16T07:19:07.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.598+0000 7fb341b96700 1 -- 172.21.15.92:0/2882063453 wait complete. 2023-05-16T07:19:07.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.600+0000 7fb341b96700 1 Processor -- start 2023-05-16T07:19:07.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.600+0000 7fb341b96700 1 -- start start 2023-05-16T07:19:07.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.601+0000 7fb341b96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c0629c0 0x7fb33c1631a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.601+0000 7fb341b96700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c0aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.601+0000 7fb341b96700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c1386a0 0x7fb33c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.601+0000 7fb341b96700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c155260 con 0x7fb33c0629c0 2023-05-16T07:19:07.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.601+0000 7fb341b96700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c154650 con 0x7fb33c1386a0 2023-05-16T07:19:07.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.601+0000 7fb341b96700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c154930 con 0x7fb33c0c30a0 2023-05-16T07:19:07.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c0629c0 0x7fb33c1631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb332ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c0aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c1386a0 0x7fb33c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:07.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c0629c0 0x7fb33c1631a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48496/0 (socket says 172.21.15.92:48496) 2023-05-16T07:19:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33b7fe700 1 -- 172.21.15.92:0/1506154306 learned_addr learned my addr 172.21.15.92:0/1506154306 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 msgr2=0x7fb33c0aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c0aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c0629c0 msgr2=0x7fb33c1631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c0629c0 0x7fb33c1631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 -- 172.21.15.92:0/1506154306 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240092a0 con 0x7fb33c1386a0 2023-05-16T07:19:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb33bfff700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c1386a0 0x7fb33c1556f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fb334014000 tx=0x7fb334009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334021070 con 0x7fb33c1386a0 2023-05-16T07:19:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334013e40 con 0x7fb33c1386a0 2023-05-16T07:19:07.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33401d860 con 0x7fb33c1386a0 2023-05-16T07:19:07.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb334009a20 con 0x7fb33c1386a0 2023-05-16T07:19:07.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.602+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c154ef0 con 0x7fb33c1386a0 2023-05-16T07:19:07.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.603+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c1553d0 con 0x7fb33c1386a0 2023-05-16T07:19:07.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.604+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 44) v1 ==== 75402+0+0 (secure 0 0 0) 0x7fb33401dc90 con 0x7fb33c1386a0 2023-05-16T07:19:07.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.604+0000 7fb333fff700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 0x7fb31c061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:07.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.604+0000 7fb33b7fe700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 msgr2=0x7fb31c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:07.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.604+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fb33407bf00 con 0x7fb33c1386a0 2023-05-16T07:19:07.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.604+0000 7fb33b7fe700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 0x7fb31c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:07.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.608+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb33404d000 con 0x7fb33c1386a0 2023-05-16T07:19:07.639 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:07.635+0000 7f95c378c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:19:07.709 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:07.704+0000 7f95c378c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:19:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.805+0000 7fb33b7fe700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 msgr2=0x7fb31c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.805+0000 7fb33b7fe700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 0x7fb31c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:07.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:07.820+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb33c06c350 con 0x7fb33c1386a0 2023-05-16T07:19:07.841 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.838+0000 7fb8bd688680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:19:07.892 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:19:07.888+0000 7f9008f39700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:19:07.897 INFO:tasks.ceph.mds.a.smithi092.stderr: -6076> 2023-05-16T07:18:23.346+0000 7f900c740700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:19:07.904 INFO:tasks.ceph.mds.a.smithi092.stderr: -1> 2023-05-16T07:19:07.888+0000 7f9008f39700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:19:07.917 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:07.914+0000 7fb8bd688680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:19:07.922 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:19:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.206+0000 7fb33b7fe700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 msgr2=0x7fb31c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:08.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.206+0000 7fb33b7fe700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 0x7fb31c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:19:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.566+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mgrmap(e 45) v1 ==== 75092+0+0 (secure 0 0 0) 0x7fb334079960 con 0x7fb33c1386a0 2023-05-16T07:19:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.566+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 msgr2=0x7fb31c061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.566+0000 7fb333fff700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 0x7fb31c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:08.621 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9546777700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.621 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9546777700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.621 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9546777700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.621 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9546777700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9542f70700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9546777700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9542f70700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9542f70700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9542f70700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:08.616+0000 7f9542f70700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:08.724 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:19:08.721+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:19:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.724+0000 7fb333fff700 1 -- 172.21.15.92:0/1506154306 <== mon.1 v2:172.21.15.158:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb334055a60 con 0x7fb33c1386a0 2023-05-16T07:19:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c1386a0 msgr2=0x7fb33c1556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c1386a0 0x7fb33c1556f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fb334014000 tx=0x7fb334009780).stop 2023-05-16T07:19:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 shutdown_connections 2023-05-16T07:19:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb31c05eeb0 0x7fb31c061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb33c1386a0 0x7fb33c1556f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb33c0629c0 0x7fb33c1631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 --2- 172.21.15.92:0/1506154306 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb33c0c30a0 0x7fb33c0aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 >> 172.21.15.92:0/1506154306 conn(0x7fb33c0c8aa0 msgr2=0x7fb33c135de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 shutdown_connections 2023-05-16T07:19:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.726+0000 7fb341b96700 1 -- 172.21.15.92:0/1506154306 wait complete. 2023-05-16T07:19:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:19:08.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:19:08.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.852+0000 7f7b12f3c700 1 Processor -- start 2023-05-16T07:19:08.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.852+0000 7f7b12f3c700 1 -- start start 2023-05-16T07:19:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b12f3c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c154090 0x7f7b0c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b12f3c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7b0c068160 0x7f7b0c068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b12f3c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0c94d0 con 0x7f7b0c154470 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b12f3c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c0cb7e0 con 0x7f7b0c068160 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b12f3c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c068a70 con 0x7f7b0c154090 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b114d9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7b0c068160 0x7f7b0c068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b10cd8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c154090 0x7f7b0c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b0bfff700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f7b0c154470 0x7f7b0c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:34640/0 (socket says 172.21.15.92:34640) 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.853+0000 7f7b0bfff700 1 -- 172.21.15.92:0/3414078086 learned_addr learned my addr 172.21.15.92:0/3414078086 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:08.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b114d9700 1 -- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c154090 msgr2=0x7f7b0c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b114d9700 1 --2- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c154090 0x7f7b0c0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b114d9700 1 -- 172.21.15.92:0/3414078086 >> v1:172.21.15.158:6789/0 conn(0x7f7b0c154470 legacy=0x7f7b0c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b114d9700 1 -- 172.21.15.92:0/3414078086 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0c88c0 con 0x7f7b0c068160 2023-05-16T07:19:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b0b7fe700 1 -- 172.21.15.92:0/3414078086 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763002626 0 0) 0x7f7b0c0c94d0 con 0x7f7b0c154470 2023-05-16T07:19:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b10cd8700 1 --2- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c154090 0x7f7b0c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b114d9700 1 --2- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c068160 0x7f7b0c068580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f7b04009e60 tx=0x7f7b04009780).ready entity=mon.1 client_cookie=28c5dcb589235a16 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b0b7fe700 1 -- 172.21.15.92:0/3414078086 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400e070 con 0x7f7b0c068160 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b0b7fe700 1 -- 172.21.15.92:0/3414078086 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0400bb60 con 0x7f7b0c068160 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b0b7fe700 1 -- 172.21.15.92:0/3414078086 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04012410 con 0x7f7b0c068160 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c068160 msgr2=0x7f7b0c068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c068160 0x7f7b0c068580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f7b04009e60 tx=0x7f7b04009780).stop 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3414078086 shutdown_connections 2023-05-16T07:19:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c068160 0x7f7b0c068580 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3414078086 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c154090 0x7f7b0c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3414078086 >> 172.21.15.92:0/3414078086 conn(0x7f7b0c067920 msgr2=0x7f7b0c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:08.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3414078086 shutdown_connections 2023-05-16T07:19:08.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.854+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3414078086 wait complete. 2023-05-16T07:19:08.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.855+0000 7f7b12f3c700 1 Processor -- start 2023-05-16T07:19:08.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 -- start start 2023-05-16T07:19:08.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c068160 0x7f7b0c152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:08.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 0x7f7b0c144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:08.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c154470 0x7f7b0c0c0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:08.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1513c0 con 0x7f7b0c154090 2023-05-16T07:19:08.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1516a0 con 0x7f7b0c154470 2023-05-16T07:19:08.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.856+0000 7f7b12f3c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c151980 con 0x7f7b0c068160 2023-05-16T07:19:08.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 0x7f7b0c144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:08.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 0x7f7b0c144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48532/0 (socket says 172.21.15.92:48532) 2023-05-16T07:19:08.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 -- 172.21.15.92:0/3516290641 learned_addr learned my addr 172.21.15.92:0/3516290641 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:08.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b10cd8700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c068160 0x7f7b0c152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:08.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b114d9700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c154470 0x7f7b0c0c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:08.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 -- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c068160 msgr2=0x7f7b0c152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:08.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c068160 0x7f7b0c152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 -- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c154470 msgr2=0x7f7b0c0c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:08.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c154470 0x7f7b0c0c0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:08.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 -- 172.21.15.92:0/3516290641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0092a0 con 0x7f7b0c154090 2023-05-16T07:19:08.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b0bfff700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 0x7f7b0c144800 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f7b0000c490 tx=0x7f7b00007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:08.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000072e0 con 0x7f7b0c154090 2023-05-16T07:19:08.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00007440 con 0x7f7b0c154090 2023-05-16T07:19:08.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b04009a20 con 0x7f7b0c154090 2023-05-16T07:19:08.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000057a0 con 0x7f7b0c154090 2023-05-16T07:19:08.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.857+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c151f40 con 0x7f7b0c154090 2023-05-16T07:19:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.858+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c152160 con 0x7f7b0c154090 2023-05-16T07:19:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.858+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 45) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f7b0001a060 con 0x7f7b0c154090 2023-05-16T07:19:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.859+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7277+0+0 (secure 0 0 0) 0x7f7b00079530 con 0x7f7b0c154090 2023-05-16T07:19:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:08.863+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7b0004a6f0 con 0x7f7b0c154090 2023-05-16T07:19:09.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.067+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7b0c06c350 con 0x7f7b0c154090 2023-05-16T07:19:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.569+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 46) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f7b00010070 con 0x7f7b0c154090 2023-05-16T07:19:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.570+0000 7f7b08ff9700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7af80301b0 0x7f7af8067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.572+0000 7f7b10cd8700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7af80301b0 0x7f7af8067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.574+0000 7f7b10cd8700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7af80301b0 0x7f7af8067e30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f7afc009270 tx=0x7f7afc008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.727+0000 7f7b08ff9700 1 -- 172.21.15.92:0/3516290641 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7b00072750 con 0x7f7b0c154090 2023-05-16T07:19:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7af80301b0 msgr2=0x7f7af8067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7af80301b0 0x7f7af8067e30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f7afc009270 tx=0x7f7afc008040).stop 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 msgr2=0x7f7b0c144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 0x7f7b0c144800 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f7b0000c490 tx=0x7f7b00007d70).stop 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 shutdown_connections 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7af80301b0 0x7f7af8067e30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7b0c068160 0x7f7b0c152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7b0c154470 0x7f7b0c0c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.729+0000 7f7b12f3c700 1 --2- 172.21.15.92:0/3516290641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c154090 0x7f7b0c144800 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.730+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 >> 172.21.15.92:0/3516290641 conn(0x7f7b0c067920 msgr2=0x7f7b0c0a9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.730+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 shutdown_connections 2023-05-16T07:19:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.730+0000 7f7b12f3c700 1 -- 172.21.15.92:0/3516290641 wait complete. 2023-05-16T07:19:09.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:19:09.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.865+0000 7f6f350ee700 1 Processor -- start 2023-05-16T07:19:09.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.865+0000 7f6f350ee700 1 -- start start 2023-05-16T07:19:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f350ee700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f30153ee0 0x7f6f301542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f350ee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f300be280 0x7f6f30066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f350ee700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f30155870 con 0x7f6f30062cc0 2023-05-16T07:19:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f350ee700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f30067340 con 0x7f6f300be280 2023-05-16T07:19:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f350ee700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30155670 con 0x7f6f30153ee0 2023-05-16T07:19:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f2f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f300be280 0x7f6f30066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f2f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f300be280 0x7f6f30066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48556/0 (socket says 172.21.15.92:48556) 2023-05-16T07:19:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f2f59e700 1 -- 172.21.15.92:0/3822385571 learned_addr learned my addr 172.21.15.92:0/3822385571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.866+0000 7f6f2ed9d700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f30153ee0 0x7f6f301542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2f59e700 1 -- 172.21.15.92:0/3822385571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f30153ee0 msgr2=0x7f6f301542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2f59e700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f30153ee0 0x7f6f301542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2f59e700 1 -- 172.21.15.92:0/3822385571 >> v1:172.21.15.92:6789/0 conn(0x7f6f30062cc0 legacy=0x7f6f300bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:09.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2f59e700 1 -- 172.21.15.92:0/3822385571 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f30063120 con 0x7f6f300be280 2023-05-16T07:19:09.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2ed9d700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f30153ee0 0x7f6f301542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:09.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2f59e700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f300be280 0x7f6f30066e50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f6f28009f50 tx=0x7f6f28009840).ready entity=mon.0 client_cookie=9d5b1e2b9cc9a9ac server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2e59c700 1 -- 172.21.15.92:0/3822385571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800e070 con 0x7f6f300be280 2023-05-16T07:19:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2e59c700 1 -- 172.21.15.92:0/3822385571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2800ba70 con 0x7f6f300be280 2023-05-16T07:19:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f2e59c700 1 -- 172.21.15.92:0/3822385571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800bdf0 con 0x7f6f300be280 2023-05-16T07:19:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 -- 172.21.15.92:0/3822385571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f300be280 msgr2=0x7f6f30066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f300be280 0x7f6f30066e50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f6f28009f50 tx=0x7f6f28009840).stop 2023-05-16T07:19:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 -- 172.21.15.92:0/3822385571 shutdown_connections 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f30153ee0 0x7f6f301542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 --2- 172.21.15.92:0/3822385571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f300be280 0x7f6f30066e50 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 -- 172.21.15.92:0/3822385571 >> 172.21.15.92:0/3822385571 conn(0x7f6f300c8420 msgr2=0x7f6f300c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 -- 172.21.15.92:0/3822385571 shutdown_connections 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.867+0000 7f6f350ee700 1 -- 172.21.15.92:0/3822385571 wait complete. 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 Processor -- start 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 -- start start 2023-05-16T07:19:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 0x7f6f3014c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f300be280 0x7f6f30156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f30153ee0 0x7f6f3014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30155a00 con 0x7f6f30062cc0 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30156210 con 0x7f6f30153ee0 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f350ee700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300c6610 con 0x7f6f300be280 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f2ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 0x7f6f3014c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f2ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 0x7f6f3014c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48562/0 (socket says 172.21.15.92:48562) 2023-05-16T07:19:09.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.869+0000 7f6f2ed9d700 1 -- 172.21.15.92:0/4094563630 learned_addr learned my addr 172.21.15.92:0/4094563630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:09.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2659c700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f300be280 0x7f6f30156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2f59e700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f30153ee0 0x7f6f3014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2ed9d700 1 -- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f300be280 msgr2=0x7f6f30156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:09.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2ed9d700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f300be280 0x7f6f30156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2ed9d700 1 -- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f30153ee0 msgr2=0x7f6f3014f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:09.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2ed9d700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f30153ee0 0x7f6f3014f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:09.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2ed9d700 1 -- 172.21.15.92:0/4094563630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28009ae0 con 0x7f6f30062cc0 2023-05-16T07:19:09.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f2ed9d700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 0x7f6f3014c7b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f6f2000c8b0 tx=0x7f6f20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:09.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20010580 con 0x7f6f30062cc0 2023-05-16T07:19:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f200106e0 con 0x7f6f30062cc0 2023-05-16T07:19:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20010a60 con 0x7f6f30062cc0 2023-05-16T07:19:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f300c4070 con 0x7f6f30062cc0 2023-05-16T07:19:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.870+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f30155f30 con 0x7f6f30062cc0 2023-05-16T07:19:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.871+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f300632c0 con 0x7f6f30062cc0 2023-05-16T07:19:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.871+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 46) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f6f2001b460 con 0x7f6f30062cc0 2023-05-16T07:19:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.872+0000 7f6f277fe700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f1005ecf0 0x7f6f100611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.872+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7277+0+0 (secure 0 0 0) 0x7f6f20079fa0 con 0x7f6f30062cc0 2023-05-16T07:19:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.872+0000 7f6f2659c700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f1005ecf0 0x7f6f100611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.872+0000 7f6f2659c700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f1005ecf0 0x7f6f100611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6f30157bd0 tx=0x7f6f18009040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:09.876+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6f2004b100 con 0x7f6f30062cc0 2023-05-16T07:19:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.041+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f3006c350 con 0x7f6f30062cc0 2023-05-16T07:19:10.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.728+0000 7f6f277fe700 1 -- 172.21.15.92:0/4094563630 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6f2004e720 con 0x7f6f30062cc0 2023-05-16T07:19:10.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.731+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f1005ecf0 msgr2=0x7f6f100611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:10.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.731+0000 7f6f350ee700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f1005ecf0 0x7f6f100611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6f30157bd0 tx=0x7f6f18009040).stop 2023-05-16T07:19:10.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.731+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 msgr2=0x7f6f3014c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:10.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.731+0000 7f6f350ee700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 0x7f6f3014c7b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f6f2000c8b0 tx=0x7f6f20008040).stop 2023-05-16T07:19:10.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 shutdown_connections 2023-05-16T07:19:10.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f1005ecf0 0x7f6f100611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6f30153ee0 0x7f6f3014f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f30062cc0 0x7f6f3014c7b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 --2- 172.21.15.92:0/4094563630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6f300be280 0x7f6f30156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 >> 172.21.15.92:0/4094563630 conn(0x7f6f300c8420 msgr2=0x7f6f30152ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:10.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 shutdown_connections 2023-05-16T07:19:10.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.732+0000 7f6f350ee700 1 -- 172.21.15.92:0/4094563630 wait complete. 2023-05-16T07:19:10.737 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:19:10.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:19:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.868+0000 7f7f07d2e700 1 Processor -- start 2023-05-16T07:19:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.869+0000 7f7f07d2e700 1 -- start start 2023-05-16T07:19:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.869+0000 7f7f07d2e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f00154090 0x7f7f000aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.869+0000 7f7f07d2e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7f00154470 0x7f7f000a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.869+0000 7f7f07d2e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f000c94d0 con 0x7f7f000aa150 2023-05-16T07:19:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.869+0000 7f7f07d2e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f000aa5a0 con 0x7f7f00154090 2023-05-16T07:19:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.869+0000 7f7f07d2e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f000cb7e0 con 0x7f7f00154470 2023-05-16T07:19:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f00154090 0x7f7f000aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f00154090 0x7f7f000aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48566/0 (socket says 172.21.15.92:48566) 2023-05-16T07:19:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 -- 172.21.15.92:0/99613117 learned_addr learned my addr 172.21.15.92:0/99613117 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f052c9700 1 --2- 172.21.15.92:0/99613117 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154470 0x7f7f000a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 -- 172.21.15.92:0/99613117 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154470 msgr2=0x7f7f000a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 --2- 172.21.15.92:0/99613117 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154470 0x7f7f000a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 -- 172.21.15.92:0/99613117 >> v1:172.21.15.158:6789/0 conn(0x7f7f000aa150 legacy=0x7f7f00068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 -- 172.21.15.92:0/99613117 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f000c88c0 con 0x7f7f00154090 2023-05-16T07:19:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f05aca700 1 --2- 172.21.15.92:0/99613117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000aaca0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f7f00084320 tx=0x7f7ef0009840).ready entity=mon.0 client_cookie=ffe3429242c60e76 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f04ac8700 1 -- 172.21.15.92:0/99613117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000e070 con 0x7f7f00154090 2023-05-16T07:19:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f04ac8700 1 -- 172.21.15.92:0/99613117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef000ba50 con 0x7f7f00154090 2023-05-16T07:19:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.870+0000 7f7f04ac8700 1 -- 172.21.15.92:0/99613117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000bdd0 con 0x7f7f00154090 2023-05-16T07:19:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 -- 172.21.15.92:0/99613117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 msgr2=0x7f7f000aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/99613117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000aaca0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f7f00084320 tx=0x7f7ef0009840).stop 2023-05-16T07:19:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 -- 172.21.15.92:0/99613117 shutdown_connections 2023-05-16T07:19:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/99613117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000aaca0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/99613117 >> v2:172.21.15.158:3300/0 conn(0x7f7f00154470 0x7f7f000a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 -- 172.21.15.92:0/99613117 >> 172.21.15.92:0/99613117 conn(0x7f7f00067920 msgr2=0x7f7f000cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 -- 172.21.15.92:0/99613117 shutdown_connections 2023-05-16T07:19:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.871+0000 7f7f07d2e700 1 -- 172.21.15.92:0/99613117 wait complete. 2023-05-16T07:19:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.872+0000 7f7f07d2e700 1 Processor -- start 2023-05-16T07:19:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.872+0000 7f7f07d2e700 1 -- start start 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f000aa150 0x7f7f0014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00154470 0x7f7f000c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f001505a0 con 0x7f7f00154090 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00150e40 con 0x7f7f00154470 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000c4d70 con 0x7f7f000aa150 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000c41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48572/0 (socket says 172.21.15.92:48572) 2023-05-16T07:19:10.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 -- 172.21.15.92:0/1213733372 learned_addr learned my addr 172.21.15.92:0/1213733372 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:10.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 -- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f000aa150 msgr2=0x7f7f0014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:10.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f05aca700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f000aa150 0x7f7f0014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:10.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f062cb700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00154470 0x7f7f000c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:10.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f000aa150 0x7f7f0014ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 -- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00154470 msgr2=0x7f7f000c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00154470 0x7f7f000c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 -- 172.21.15.92:0/1213733372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0009ae0 con 0x7f7f00154090 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f062cb700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00154470 0x7f7f000c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f052c9700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000c41d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f7ef400d890 tx=0x7f7ef4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40125f0 con 0x7f7f00154090 2023-05-16T07:19:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4012750 con 0x7f7f00154090 2023-05-16T07:19:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f001507f0 con 0x7f7f00154090 2023-05-16T07:19:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.873+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4012ad0 con 0x7f7f00154090 2023-05-16T07:19:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.874+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f000c2920 con 0x7f7f00154090 2023-05-16T07:19:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.874+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00066730 con 0x7f7f00154090 2023-05-16T07:19:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.875+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7ef4013480 con 0x7f7f00154090 2023-05-16T07:19:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.875+0000 7f7efdffb700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7eec05ed90 0x7f7eec061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.875+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6811+0+0 (secure 0 0 0) 0x7f7ef4079c40 con 0x7f7f00154090 2023-05-16T07:19:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.875+0000 7f7f05aca700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7eec05ed90 0x7f7eec061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.875+0000 7f7f05aca700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7eec05ed90 0x7f7eec061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7f00084320 tx=0x7f7ef0009840).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:10.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:10.879+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7ef404aed0 con 0x7f7f00154090 2023-05-16T07:19:11.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.042+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7f000c8ba0 con 0x7f7f00154090 2023-05-16T07:19:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.741+0000 7f7efdffb700 1 -- 172.21.15.92:0/1213733372 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7ef404e4f0 con 0x7f7f00154090 2023-05-16T07:19:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7eec05ed90 msgr2=0x7f7eec061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7eec05ed90 0x7f7eec061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7f00084320 tx=0x7f7ef0009840).stop 2023-05-16T07:19:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 msgr2=0x7f7f000c41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000c41d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f7ef400d890 tx=0x7f7ef4008040).stop 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 shutdown_connections 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f7eec05ed90 0x7f7eec061250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f000aa150 0x7f7f0014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f00154470 0x7f7f000c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 --2- 172.21.15.92:0/1213733372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f00154090 0x7f7f000c41d0 secure :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0x7f7ef400d890 tx=0x7f7ef4008040).stop 2023-05-16T07:19:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 >> 172.21.15.92:0/1213733372 conn(0x7f7f00067920 msgr2=0x7f7f000c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.743+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 shutdown_connections 2023-05-16T07:19:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.744+0000 7f7f07d2e700 1 -- 172.21.15.92:0/1213733372 wait complete. 2023-05-16T07:19:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:19:11.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:19:11.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.871+0000 7fcfafb07700 1 Processor -- start 2023-05-16T07:19:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.871+0000 7fcfafb07700 1 -- start start 2023-05-16T07:19:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfafb07700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcfa814b4e0 0x7fcfa80bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfafb07700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 0x7fcfa81512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfafb07700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa8065710 con 0x7fcfa8069cb0 2023-05-16T07:19:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfafb07700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa814b8c0 con 0x7fcfa814b4e0 2023-05-16T07:19:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfafb07700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80bd8f0 con 0x7fcfa80c1a30 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfad0a2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 0x7fcfa81512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfad8a3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcfa814b4e0 0x7fcfa80bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfad0a2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 0x7fcfa81512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:41150/0 (socket says 172.21.15.92:41150) 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.872+0000 7fcfad0a2700 1 -- 172.21.15.92:0/917125109 learned_addr learned my addr 172.21.15.92:0/917125109 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfac8a1700 1 -- 172.21.15.92:0/917125109 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170545643 0 0) 0x7fcfa8065710 con 0x7fcfa8069cb0 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfac8a1700 1 -- 172.21.15.92:0/917125109 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf900033f0 con 0x7fcfa8069cb0 2023-05-16T07:19:11.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfad8a3700 1 -- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 msgr2=0x7fcfa81512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfad8a3700 1 --2- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 0x7fcfa81512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfad8a3700 1 -- 172.21.15.92:0/917125109 >> v1:172.21.15.158:6789/0 conn(0x7fcfa8069cb0 legacy=0x7fcfa8065ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfad8a3700 1 -- 172.21.15.92:0/917125109 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8151b50 con 0x7fcfa814b4e0 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfad0a2700 1 --2- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 0x7fcfa81512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfad8a3700 1 --2- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa814b4e0 0x7fcfa80bd400 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fcf98009e60 tx=0x7fcf98009780).ready entity=mon.1 client_cookie=9afb6541f17e7598 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfac8a1700 1 -- 172.21.15.92:0/917125109 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800e070 con 0x7fcfa814b4e0 2023-05-16T07:19:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfac8a1700 1 -- 172.21.15.92:0/917125109 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9800bab0 con 0x7fcfa814b4e0 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.873+0000 7fcfac8a1700 1 -- 172.21.15.92:0/917125109 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800be30 con 0x7fcfa814b4e0 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 -- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa814b4e0 msgr2=0x7fcfa80bd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 --2- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa814b4e0 0x7fcfa80bd400 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fcf98009e60 tx=0x7fcf98009780).stop 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 -- 172.21.15.92:0/917125109 shutdown_connections 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 --2- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa814b4e0 0x7fcfa80bd400 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 --2- 172.21.15.92:0/917125109 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa80c1a30 0x7fcfa81512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 -- 172.21.15.92:0/917125109 >> 172.21.15.92:0/917125109 conn(0x7fcfa80c4600 msgr2=0x7fcfa8066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 -- 172.21.15.92:0/917125109 shutdown_connections 2023-05-16T07:19:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.874+0000 7fcfafb07700 1 -- 172.21.15.92:0/917125109 wait complete. 2023-05-16T07:19:11.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.875+0000 7fcfafb07700 1 Processor -- start 2023-05-16T07:19:11.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.875+0000 7fcfafb07700 1 -- start start 2023-05-16T07:19:11.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.875+0000 7fcfafb07700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa8069cb0 0x7fcfa80a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:11.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.875+0000 7fcfafb07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 0x7fcfa80a7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:11.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.875+0000 7fcfafb07700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa814b4e0 0x7fcfa80a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:11.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfafb07700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8138650 con 0x7fcfa80c1a30 2023-05-16T07:19:11.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfafb07700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80be7a0 con 0x7fcfa8069cb0 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfafb07700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80bf2e0 con 0x7fcfa814b4e0 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 0x7fcfa80a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 0x7fcfa80a7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48584/0 (socket says 172.21.15.92:48584) 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 -- 172.21.15.92:0/1409244882 learned_addr learned my addr 172.21.15.92:0/1409244882 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad8a3700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa8069cb0 0x7fcfa80a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfae0a4700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa814b4e0 0x7fcfa80a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:11.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 -- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa814b4e0 msgr2=0x7fcfa80a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa814b4e0 0x7fcfa80a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 -- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa8069cb0 msgr2=0x7fcfa80a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa8069cb0 0x7fcfa80a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 -- 172.21.15.92:0/1409244882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c009370 con 0x7fcfa80c1a30 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfad0a2700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 0x7fcfa80a7100 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fcf9c00ea20 tx=0x7fcf9c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00c850 con 0x7fcfa80c1a30 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c00c9b0 con 0x7fcfa80c1a30 2023-05-16T07:19:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf98009a20 con 0x7fcfa80c1a30 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00cd30 con 0x7fcfa80c1a30 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.876+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa80be9b0 con 0x7fcfa80c1a30 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.877+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa80bf470 con 0x7fcfa80c1a30 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.877+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcf9c022070 con 0x7fcfa80c1a30 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.878+0000 7fcfa5ffb700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf9405ed90 0x7fcf94061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.878+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fcf9c07a1b0 con 0x7fcfa80c1a30 2023-05-16T07:19:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.878+0000 7fcfad8a3700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf9405ed90 0x7fcf94061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:11.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.878+0000 7fcfad8a3700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf9405ed90 0x7fcf94061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fcf98009e60 tx=0x7fcf98009780).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:11.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:11.882+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcf9c04b640 con 0x7fcfa80c1a30 2023-05-16T07:19:12.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.046+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcfa8151e30 con 0x7fcfa80c1a30 2023-05-16T07:19:12.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.050+0000 7fcfa5ffb700 1 -- 172.21.15.92:0/1409244882 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf9c04ec60 con 0x7fcfa80c1a30 2023-05-16T07:19:12.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf9405ed90 msgr2=0x7fcf94061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf9405ed90 0x7fcf94061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fcf98009e60 tx=0x7fcf98009780).stop 2023-05-16T07:19:12.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 msgr2=0x7fcfa80a7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 0x7fcfa80a7100 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fcf9c00ea20 tx=0x7fcf9c0090f0).stop 2023-05-16T07:19:12.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 shutdown_connections 2023-05-16T07:19:12.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcf9405ed90 0x7fcf94061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcfa814b4e0 0x7fcfa80a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcfa8069cb0 0x7fcfa80a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 --2- 172.21.15.92:0/1409244882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfa80c1a30 0x7fcfa80a7100 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 >> 172.21.15.92:0/1409244882 conn(0x7fcfa80c4600 msgr2=0x7fcfa80bd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:12.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 shutdown_connections 2023-05-16T07:19:12.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.052+0000 7fcfafb07700 1 -- 172.21.15.92:0/1409244882 wait complete. 2023-05-16T07:19:12.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:19:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.176+0000 7fccc2209700 1 Processor -- start 2023-05-16T07:19:12.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.176+0000 7fccc2209700 1 -- start start 2023-05-16T07:19:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc2209700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 0x7fccbc0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc2209700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccbc0c6660 con 0x7fccbc153f90 2023-05-16T07:19:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc2209700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccbc06a140 con 0x7fccbc0c61f0 2023-05-16T07:19:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc2209700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc069f60 con 0x7fccbc0c1160 2023-05-16T07:19:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc1207700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fccbc153f90 0x7fccbc0a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59340/0 (socket says 172.21.15.92:59340) 2023-05-16T07:19:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc1207700 1 -- 172.21.15.92:0/2771453396 learned_addr learned my addr 172.21.15.92:0/2771453396 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccc0a06700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 0x7fccbc0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492991478 0 0) 0x7fccbc0c6660 con 0x7fccbc153f90 2023-05-16T07:19:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca40033f0 con 0x7fccbc153f90 2023-05-16T07:19:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448433987 0 0) 0x7fccbc06a140 con 0x7fccbc0c61f0 2023-05-16T07:19:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccbc0c6660 con 0x7fccbc0c61f0 2023-05-16T07:19:12.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939638998 0 0) 0x7fcca40033f0 con 0x7fccbc153f90 2023-05-16T07:19:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 msgr2=0x7fccbc0c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 0x7fccbc0c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 >> v1:172.21.15.158:6789/0 conn(0x7fccbc0c61f0 legacy=0x7fccbc154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc06a420 con 0x7fccbc153f90 2023-05-16T07:19:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fccac0017f0 con 0x7fccbc153f90 2023-05-16T07:19:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 >> v1:172.21.15.92:6789/0 conn(0x7fccbc153f90 legacy=0x7fccbc0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcca4003aa0 0x7fcca4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 0x7fcca4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca4004b80 0x7fcca4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc069f60 con 0x7fcca4004b80 2023-05-16T07:19:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca400b990 con 0x7fcca4003aa0 2023-05-16T07:19:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.177+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca400bb00 con 0x7fccbc0c1160 2023-05-16T07:19:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca4004b80 0x7fcca4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1207700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcca4003aa0 0x7fcca4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc0a06700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 0x7fcca4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 -- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 msgr2=0x7fcca4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccbc0c1160 0x7fcca4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 -- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcca4003aa0 msgr2=0x7fcca4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcca4003aa0 0x7fcca4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 -- 172.21.15.92:0/2771453396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40033f0 con 0x7fcca4004b80 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccc1a08700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca4004b80 0x7fcca4005000 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fccb4001890 tx=0x7fccb4007bb0).ready entity=mon.0 client_cookie=ff8242c483d5f42c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4007020 con 0x7fcca4004b80 2023-05-16T07:19:12.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4007180 con 0x7fcca4004b80 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.178+0000 7fccbbfff700 1 -- 172.21.15.92:0/2771453396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4007500 con 0x7fcca4004b80 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc2209700 1 -- 172.21.15.92:0/2771453396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca4004b80 msgr2=0x7fcca4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc2209700 1 --2- 172.21.15.92:0/2771453396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca4004b80 0x7fcca4005000 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fccb4001890 tx=0x7fccb4007bb0).stop 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc1a08700 1 -- 172.21.15.92:0/2771453396 reap_dead start 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc2209700 1 -- 172.21.15.92:0/2771453396 shutdown_connections 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc2209700 1 -- 172.21.15.92:0/2771453396 >> 172.21.15.92:0/2771453396 conn(0x7fccbc0cb600 msgr2=0x7fccbc0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc2209700 1 -- 172.21.15.92:0/2771453396 shutdown_connections 2023-05-16T07:19:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.179+0000 7fccc2209700 1 -- 172.21.15.92:0/2771453396 wait complete. 2023-05-16T07:19:12.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.180+0000 7fccc2209700 1 Processor -- start 2023-05-16T07:19:12.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.180+0000 7fccc2209700 1 -- start start 2023-05-16T07:19:12.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcca4004b80 0x7fccbc148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 0x7fccbc0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccbc0ae080 0x7fccbc0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0c97a0 con 0x7fccbc0adc00 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0aff10 con 0x7fccbc0ae080 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0b03a0 con 0x7fcca4004b80 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 0x7fccbc0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 0x7fccbc0c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48606/0 (socket says 172.21.15.92:48606) 2023-05-16T07:19:12.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 -- 172.21.15.92:0/4174411346 learned_addr learned my addr 172.21.15.92:0/4174411346 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 -- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcca4004b80 msgr2=0x7fccbc148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcca4004b80 0x7fccbc148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 -- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccbc0ae080 msgr2=0x7fccbc0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccbc0ae080 0x7fccbc0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 -- 172.21.15.92:0/4174411346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc0af300 con 0x7fccbc0adc00 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc0a06700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 0x7fccbc0c8dc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fccb00119c0 tx=0x7fccb0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000f8e0 con 0x7fccbc0adc00 2023-05-16T07:19:12.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb000fa40 con 0x7fccbc0adc00 2023-05-16T07:19:12.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccbc0af5e0 con 0x7fccbc0adc00 2023-05-16T07:19:12.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000fdc0 con 0x7fccbc0adc00 2023-05-16T07:19:12.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.181+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccbc0afba0 con 0x7fccbc0adc00 2023-05-16T07:19:12.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.182+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccbc083120 con 0x7fccbc0adc00 2023-05-16T07:19:12.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.182+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fccb0024070 con 0x7fccbc0adc00 2023-05-16T07:19:12.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.183+0000 7fccb97fa700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca805ed90 0x7fcca8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.183+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fccb007a710 con 0x7fccbc0adc00 2023-05-16T07:19:12.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.183+0000 7fccc1207700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca805ed90 0x7fcca8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.184+0000 7fccc1207700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca805ed90 0x7fcca8061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fccac007fd0 tx=0x7fccac007f20).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.187+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fccb004bba0 con 0x7fccbc0adc00 2023-05-16T07:19:12.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.352+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fccbc047150 con 0x7fccbc0adc00 2023-05-16T07:19:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.355+0000 7fccb97fa700 1 -- 172.21.15.92:0/4174411346 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7fccb004f1c0 con 0x7fccbc0adc00 2023-05-16T07:19:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca805ed90 msgr2=0x7fcca8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca805ed90 0x7fcca8061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fccac007fd0 tx=0x7fccac007f20).stop 2023-05-16T07:19:12.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 msgr2=0x7fccbc0c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 0x7fccbc0c8dc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fccb00119c0 tx=0x7fccb0009060).stop 2023-05-16T07:19:12.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 shutdown_connections 2023-05-16T07:19:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcca805ed90 0x7fcca8061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcca4004b80 0x7fccbc148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fccbc0ae080 0x7fccbc0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 --2- 172.21.15.92:0/4174411346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccbc0adc00 0x7fccbc0c8dc0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 >> 172.21.15.92:0/4174411346 conn(0x7fccbc0cb600 msgr2=0x7fccbc0c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:12.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 shutdown_connections 2023-05-16T07:19:12.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.357+0000 7fccc2209700 1 -- 172.21.15.92:0/4174411346 wait complete. 2023-05-16T07:19:12.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-05-16T07:19:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.480+0000 7f242d183700 1 Processor -- start 2023-05-16T07:19:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.480+0000 7f242d183700 1 -- start start 2023-05-16T07:19:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f242d183700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f24280c7380 0x7f2428154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f242d183700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2428158970 con 0x7f24280ad260 2023-05-16T07:19:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f242d183700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280ad6c0 con 0x7f24280c7760 2023-05-16T07:19:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f242d183700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2428155340 con 0x7f24280c7380 2023-05-16T07:19:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f242c982700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24280ad260 0x7f242814dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59364/0 (socket says 172.21.15.92:59364) 2023-05-16T07:19:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f242c982700 1 -- 172.21.15.92:0/3629627994 learned_addr learned my addr 172.21.15.92:0/3629627994 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f2427fff700 1 --2- 172.21.15.92:0/3629627994 >> v2:172.21.15.158:3300/0 conn(0x7f24280c7380 0x7f2428154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304854091 0 0) 0x7f2428158970 con 0x7f24280ad260 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.481+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24140033f0 con 0x7f24280ad260 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013839602 0 0) 0x7f24140033f0 con 0x7f24280ad260 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 >> v2:172.21.15.158:3300/0 conn(0x7f24280c7380 msgr2=0x7f2428154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 --2- 172.21.15.92:0/3629627994 >> v2:172.21.15.158:3300/0 conn(0x7f24280c7380 0x7f2428154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 >> v1:172.21.15.158:6789/0 conn(0x7f24280c7760 legacy=0x7f24280acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428157d60 con 0x7f24280ad260 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f24200017f0 con 0x7f24280ad260 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 >> v1:172.21.15.92:6789/0 conn(0x7f24280ad260 legacy=0x7f242814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f2414003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2414004690 0x7f2414004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2414005000 0x7f241400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428155340 con 0x7f24140038e0 2023-05-16T07:19:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241400beb0 con 0x7f2414005000 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241400c070 con 0x7f2414004690 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f2414003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f242c982700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2414005000 0x7f241400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f241ffff700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2414004690 0x7f2414004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 -- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2414004690 msgr2=0x7f2414004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2414004690 0x7f2414004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 -- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2414005000 msgr2=0x7f241400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2414005000 0x7f241400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 -- 172.21.15.92:0/3629627994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140033f0 con 0x7f24140038e0 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f242c982700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2414005000 0x7f241400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f242c982700 1 -- 172.21.15.92:0/3629627994 reap_dead start 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f2427fff700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f2414003d00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f2418009a60 tx=0x7f2418009060).ready entity=mon.0 client_cookie=ff2710063e81cd3d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418009ea0 con 0x7f24140038e0 2023-05-16T07:19:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418009430 con 0x7f24140038e0 2023-05-16T07:19:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.482+0000 7f24277fe700 1 -- 172.21.15.92:0/3629627994 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800e870 con 0x7f24140038e0 2023-05-16T07:19:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 -- 172.21.15.92:0/3629627994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 msgr2=0x7f2414003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f2414003d00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f2418009a60 tx=0x7f2418009060).stop 2023-05-16T07:19:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 -- 172.21.15.92:0/3629627994 shutdown_connections 2023-05-16T07:19:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 --2- 172.21.15.92:0/3629627994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f2414003d00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 -- 172.21.15.92:0/3629627994 >> 172.21.15.92:0/3629627994 conn(0x7f2428068330 msgr2=0x7f2428068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 -- 172.21.15.92:0/3629627994 shutdown_connections 2023-05-16T07:19:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.483+0000 7f242d183700 1 -- 172.21.15.92:0/3629627994 wait complete. 2023-05-16T07:19:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.484+0000 7f242d183700 1 Processor -- start 2023-05-16T07:19:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.484+0000 7f242d183700 1 -- start start 2023-05-16T07:19:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f242814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2428146910 0x7f24280c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24280c29a0 0x7f24280c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c2020 con 0x7f24140038e0 2023-05-16T07:19:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428146d90 con 0x7f2428146910 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242814e120 con 0x7f24280c29a0 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f242814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f242814ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48624/0 (socket says 172.21.15.92:48624) 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 -- 172.21.15.92:0/1165704778 learned_addr learned my addr 172.21.15.92:0/1165704778 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 -- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24280c29a0 msgr2=0x7f24280c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24280c29a0 0x7f24280c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 -- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2428146910 msgr2=0x7f24280c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2428146910 0x7f24280c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 -- 172.21.15.92:0/1165704778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242814db60 con 0x7f24140038e0 2023-05-16T07:19:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2427fff700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f242814ef00 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f241400be80 tx=0x7f241801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241801bcc0 con 0x7f24140038e0 2023-05-16T07:19:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241801be40 con 0x7f24140038e0 2023-05-16T07:19:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242814de40 con 0x7f24140038e0 2023-05-16T07:19:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418025690 con 0x7f24140038e0 2023-05-16T07:19:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.485+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280c8dd0 con 0x7f24140038e0 2023-05-16T07:19:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.486+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24280c2300 con 0x7f24140038e0 2023-05-16T07:19:12.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.487+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f241802b020 con 0x7f24140038e0 2023-05-16T07:19:12.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.487+0000 7f2424ff9700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f240805ed90 0x7f2408061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.487+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f241807b3a0 con 0x7f24140038e0 2023-05-16T07:19:12.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.487+0000 7f241ffff700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f240805ed90 0x7f2408061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.488+0000 7f241ffff700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f240805ed90 0x7f2408061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2410007fd0 tx=0x7f2410009040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.491+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f241804c830 con 0x7f24140038e0 2023-05-16T07:19:12.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.717+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f2428047150 con 0x7f24140038e0 2023-05-16T07:19:12.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.741+0000 7f2424ff9700 1 -- 172.21.15.92:0/1165704778 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v619) v1 ==== 148+0+0 (secure 0 0 0) 0x7f2418016020 con 0x7f24140038e0 2023-05-16T07:19:12.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f240805ed90 msgr2=0x7f2408061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f240805ed90 0x7f2408061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2410007fd0 tx=0x7f2410009040).stop 2023-05-16T07:19:12.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 msgr2=0x7f242814ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f242814ef00 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f241400be80 tx=0x7f241801a000).stop 2023-05-16T07:19:12.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 shutdown_connections 2023-05-16T07:19:12.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f240805ed90 0x7f2408061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24280c29a0 0x7f24280c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24140038e0 0x7f242814ef00 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 --2- 172.21.15.92:0/1165704778 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2428146910 0x7f24280c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:12.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 >> 172.21.15.92:0/1165704778 conn(0x7f2428068330 msgr2=0x7f24280bfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:12.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 shutdown_connections 2023-05-16T07:19:12.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.743+0000 7f242d183700 1 -- 172.21.15.92:0/1165704778 wait complete. 2023-05-16T07:19:12.759 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-05-16T07:19:12.760 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-05-16T07:19:12.760 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:19:12.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-05-16T07:19:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.868+0000 7f33d0555700 1 Processor -- start 2023-05-16T07:19:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.868+0000 7f33d0555700 1 -- start start 2023-05-16T07:19:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33d0555700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f33c8153da0 0x7f33c8069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33d0555700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f33c8063e30 0x7f33c80bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33d0555700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c8062b10 con 0x7f33c8154180 2023-05-16T07:19:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33d0555700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c8068620 con 0x7f33c8153da0 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33d0555700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c80622b0 con 0x7f33c8063e30 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f33c8153da0 0x7f33c8069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f33c8153da0 0x7f33c8069a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48632/0 (socket says 172.21.15.92:48632) 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 -- 172.21.15.92:0/577936529 learned_addr learned my addr 172.21.15.92:0/577936529 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ceaf2700 1 --2- 172.21.15.92:0/577936529 >> v2:172.21.15.158:3300/0 conn(0x7f33c8063e30 0x7f33c80bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 -- 172.21.15.92:0/577936529 >> v2:172.21.15.158:3300/0 conn(0x7f33c8063e30 msgr2=0x7f33c80bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 --2- 172.21.15.92:0/577936529 >> v2:172.21.15.158:3300/0 conn(0x7f33c8063e30 0x7f33c80bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 -- 172.21.15.92:0/577936529 >> v1:172.21.15.92:6789/0 conn(0x7f33c8154180 legacy=0x7f33c8063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.869+0000 7f33ce2f1700 1 -- 172.21.15.92:0/577936529 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c8068340 con 0x7f33c8153da0 2023-05-16T07:19:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33ce2f1700 1 --2- 172.21.15.92:0/577936529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8153da0 0x7f33c8069a10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f33c80c9750 tx=0x7f33b8009840).ready entity=mon.0 client_cookie=a3842dbd679fa48c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33cd2ef700 1 -- 172.21.15.92:0/577936529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b800e070 con 0x7f33c8153da0 2023-05-16T07:19:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33cd2ef700 1 -- 172.21.15.92:0/577936529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b800ba50 con 0x7f33c8153da0 2023-05-16T07:19:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33cd2ef700 1 -- 172.21.15.92:0/577936529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b800bdd0 con 0x7f33c8153da0 2023-05-16T07:19:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 -- 172.21.15.92:0/577936529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8153da0 msgr2=0x7f33c8069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 --2- 172.21.15.92:0/577936529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8153da0 0x7f33c8069a10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f33c80c9750 tx=0x7f33b8009840).stop 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 -- 172.21.15.92:0/577936529 shutdown_connections 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 --2- 172.21.15.92:0/577936529 >> v2:172.21.15.158:3300/0 conn(0x7f33c8063e30 0x7f33c80bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 --2- 172.21.15.92:0/577936529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8153da0 0x7f33c8069a10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 -- 172.21.15.92:0/577936529 >> 172.21.15.92:0/577936529 conn(0x7f33c80c8aa0 msgr2=0x7f33c80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 -- 172.21.15.92:0/577936529 shutdown_connections 2023-05-16T07:19:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.870+0000 7f33d0555700 1 -- 172.21.15.92:0/577936529 wait complete. 2023-05-16T07:19:12.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.871+0000 7f33d0555700 1 Processor -- start 2023-05-16T07:19:12.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.871+0000 7f33d0555700 1 -- start start 2023-05-16T07:19:12.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33d0555700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 0x7f33c81671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33d0555700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33c8153da0 0x7f33c814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33d0555700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f33c8154180 0x7f33c814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33d0555700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80c2400 con 0x7f33c8063e30 2023-05-16T07:19:12.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33d0555700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c8150350 con 0x7f33c8153da0 2023-05-16T07:19:12.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33d0555700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c8165450 con 0x7f33c8154180 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 0x7f33c81671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 0x7f33c81671f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48640/0 (socket says 172.21.15.92:48640) 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 -- 172.21.15.92:0/1390003779 learned_addr learned my addr 172.21.15.92:0/1390003779 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33cdaf0700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33c8153da0 0x7f33c814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ceaf2700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f33c8154180 0x7f33c814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 -- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f33c8154180 msgr2=0x7f33c814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f33c8154180 0x7f33c814fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 -- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33c8153da0 msgr2=0x7f33c814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33c8153da0 0x7f33c814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.872+0000 7f33ce2f1700 1 -- 172.21.15.92:0/1390003779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8009ae0 con 0x7f33c8063e30 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.873+0000 7f33ce2f1700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 0x7f33c81671f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f33b8009fd0 tx=0x7f33b8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.873+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80092e0 con 0x7f33c8063e30 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.873+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b800bbc0 con 0x7f33c8063e30 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.873+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c80c0230 con 0x7f33c8063e30 2023-05-16T07:19:12.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.873+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b801d410 con 0x7f33c8063e30 2023-05-16T07:19:12.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.873+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c80bfbe0 con 0x7f33c8063e30 2023-05-16T07:19:12.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.874+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c806c350 con 0x7f33c8063e30 2023-05-16T07:19:12.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.874+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f33b801b040 con 0x7f33c8063e30 2023-05-16T07:19:12.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.874+0000 7f33c67fc700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f33b405ed90 0x7f33b4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:12.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.874+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f33b807b4d0 con 0x7f33c8063e30 2023-05-16T07:19:12.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.875+0000 7f33cdaf0700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f33b405ed90 0x7f33b4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:12.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.875+0000 7f33cdaf0700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f33b405ed90 0x7f33b4061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f33c80c1cd0 tx=0x7f33bc007d40).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:12.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:12.878+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f33b804c960 con 0x7f33c8063e30 2023-05-16T07:19:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.107+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f33c80bf5a0 con 0x7f33c8063e30 2023-05-16T07:19:13.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.742+0000 7f33c67fc700 1 -- 172.21.15.92:0/1390003779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v620) v1 ==== 177+0+0 (secure 0 0 0) 0x7f33b804ff80 con 0x7f33c8063e30 2023-05-16T07:19:13.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f33b405ed90 msgr2=0x7f33b4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:13.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f33b405ed90 0x7f33b4061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f33c80c1cd0 tx=0x7f33bc007d40).stop 2023-05-16T07:19:13.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 msgr2=0x7f33c81671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:13.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 0x7f33c81671f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f33b8009fd0 tx=0x7f33b8009840).stop 2023-05-16T07:19:13.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 shutdown_connections 2023-05-16T07:19:13.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f33b405ed90 0x7f33b4061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f33c8154180 0x7f33c814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c8063e30 0x7f33c81671f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 --2- 172.21.15.92:0/1390003779 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f33c8153da0 0x7f33c814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 >> 172.21.15.92:0/1390003779 conn(0x7f33c80c8aa0 msgr2=0x7f33c80aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:13.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 shutdown_connections 2023-05-16T07:19:13.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.744+0000 7f33d0555700 1 -- 172.21.15.92:0/1390003779 wait complete. 2023-05-16T07:19:13.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:19:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.874+0000 7f77bc83b700 1 Processor -- start 2023-05-16T07:19:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.875+0000 7f77bc83b700 1 -- start start 2023-05-16T07:19:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.875+0000 7f77bc83b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f77b8153d40 0x7f77b8154120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.875+0000 7f77bc83b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b80a6e10 0x7f77b81584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.875+0000 7f77bc83b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b80a7290 con 0x7f77b8066e50 2023-05-16T07:19:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.875+0000 7f77bc83b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b80a6980 con 0x7f77b8153d40 2023-05-16T07:19:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.875+0000 7f77bc83b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81589c0 con 0x7f77b80a6e10 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b77fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f77b8153d40 0x7f77b8154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b6ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b80a6e10 0x7f77b81584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b7fff700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f77b8066e50 0x7f77b8064f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:34766/0 (socket says 172.21.15.92:34766) 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b7fff700 1 -- 172.21.15.92:0/1464964860 learned_addr learned my addr 172.21.15.92:0/1464964860 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b77fe700 1 -- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b80a6e10 msgr2=0x7f77b81584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b77fe700 1 --2- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b80a6e10 0x7f77b81584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b77fe700 1 -- 172.21.15.92:0/1464964860 >> v1:172.21.15.158:6789/0 conn(0x7f77b8066e50 legacy=0x7f77b8064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b77fe700 1 -- 172.21.15.92:0/1464964860 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8069ad0 con 0x7f77b8153d40 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b67fc700 1 -- 172.21.15.92:0/1464964860 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952060104 0 0) 0x7f77b0001110 con 0x7f77b8066e50 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b6ffd700 1 --2- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b80a6e10 0x7f77b81584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b77fe700 1 --2- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b8153d40 0x7f77b8154120 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f77a8009e60 tx=0x7f77a8009780).ready entity=mon.1 client_cookie=e040c9cffa033ca4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.876+0000 7f77b67fc700 1 -- 172.21.15.92:0/1464964860 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a800e070 con 0x7f77b8153d40 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77b67fc700 1 -- 172.21.15.92:0/1464964860 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a800bb60 con 0x7f77b8153d40 2023-05-16T07:19:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77b67fc700 1 -- 172.21.15.92:0/1464964860 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8012410 con 0x7f77b8153d40 2023-05-16T07:19:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 -- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b8153d40 msgr2=0x7f77b8154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 --2- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b8153d40 0x7f77b8154120 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f77a8009e60 tx=0x7f77a8009780).stop 2023-05-16T07:19:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 -- 172.21.15.92:0/1464964860 shutdown_connections 2023-05-16T07:19:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 --2- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b8153d40 0x7f77b8154120 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 --2- 172.21.15.92:0/1464964860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b80a6e10 0x7f77b81584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 -- 172.21.15.92:0/1464964860 >> 172.21.15.92:0/1464964860 conn(0x7f77b8068a30 msgr2=0x7f77b80c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 -- 172.21.15.92:0/1464964860 shutdown_connections 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.877+0000 7f77bc83b700 1 -- 172.21.15.92:0/1464964860 wait complete. 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.878+0000 7f77bc83b700 1 Processor -- start 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.878+0000 7f77bc83b700 1 -- start start 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77bc83b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 0x7f77b80c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77bc83b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b80a6e10 0x7f77b8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77bc83b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b8153d40 0x7f77b8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77bc83b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80c51e0 con 0x7f77b8066e50 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77bc83b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8155510 con 0x7f77b80a6e10 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77bc83b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8156b90 con 0x7f77b8153d40 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 0x7f77b80c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 0x7f77b80c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48652/0 (socket says 172.21.15.92:48652) 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 -- 172.21.15.92:0/3237687302 learned_addr learned my addr 172.21.15.92:0/3237687302 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b6ffd700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b80a6e10 0x7f77b8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b7fff700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b8153d40 0x7f77b8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 -- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b8153d40 msgr2=0x7f77b8155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b8153d40 0x7f77b8155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 -- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b80a6e10 msgr2=0x7f77b8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b80a6e10 0x7f77b8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b77fe700 1 -- 172.21.15.92:0/3237687302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0092a0 con 0x7f77b8066e50 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.879+0000 7f77b7fff700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b8153d40 0x7f77b8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.880+0000 7f77b77fe700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 0x7f77b80c4cf0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f77a800b6c0 tx=0x7f77a8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.880+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a801d420 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.880+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a801d580 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.880+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a8009a20 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.880+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a801d940 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.880+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b8155710 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.881+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b8047150 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.881+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f77a8026410 con 0x7f77b8066e50 2023-05-16T07:19:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.881+0000 7f779f7fe700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77a405ed60 0x7f77a4061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:13.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.881+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f77a807bc50 con 0x7f77b8066e50 2023-05-16T07:19:13.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.881+0000 7f77b6ffd700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77a405ed60 0x7f77a4061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:13.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.882+0000 7f77b6ffd700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77a405ed60 0x7f77a4061220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f77ac009270 tx=0x7f77ac008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:13.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:13.885+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f77a804d080 con 0x7f77b8066e50 2023-05-16T07:19:14.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.051+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f77b8156960 con 0x7f77b8066e50 2023-05-16T07:19:14.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.055+0000 7f779f7fe700 1 -- 172.21.15.92:0/3237687302 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f77b8156960 con 0x7f77b8066e50 2023-05-16T07:19:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77a405ed60 msgr2=0x7f77a4061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77a405ed60 0x7f77a4061220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f77ac009270 tx=0x7f77ac008040).stop 2023-05-16T07:19:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 msgr2=0x7f77b80c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 0x7f77b80c4cf0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f77a800b6c0 tx=0x7f77a8009780).stop 2023-05-16T07:19:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 shutdown_connections 2023-05-16T07:19:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f77a405ed60 0x7f77a4061220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77b8153d40 0x7f77b8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77b8066e50 0x7f77b80c4cf0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 --2- 172.21.15.92:0/3237687302 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f77b80a6e10 0x7f77b8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 >> 172.21.15.92:0/3237687302 conn(0x7f77b8068a30 msgr2=0x7f77b80a7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:14.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 shutdown_connections 2023-05-16T07:19:14.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.057+0000 7f77bc83b700 1 -- 172.21.15.92:0/3237687302 wait complete. 2023-05-16T07:19:14.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:19:14.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:14.100 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:14.133 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:19:14.133 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:19:14.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.247+0000 7fe80edb0700 1 Processor -- start 2023-05-16T07:19:14.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.247+0000 7fe80edb0700 1 -- start start 2023-05-16T07:19:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80edb0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe808154170 0x7fe8080c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80edb0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe808154550 0x7fe8080c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80edb0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8080c0660 con 0x7fe8080696a0 2023-05-16T07:19:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80edb0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8080c1b00 con 0x7fe808154170 2023-05-16T07:19:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80edb0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8080bfd30 con 0x7fe808154550 2023-05-16T07:19:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe808154170 0x7fe8080c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe808154170 0x7fe8080c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48660/0 (socket says 172.21.15.92:48660) 2023-05-16T07:19:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3418349804 learned_addr learned my addr 172.21.15.92:0/3418349804 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3418349804 >> v2:172.21.15.158:3300/0 conn(0x7fe808154550 msgr2=0x7fe8080c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 --2- 172.21.15.92:0/3418349804 >> v2:172.21.15.158:3300/0 conn(0x7fe808154550 0x7fe8080c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3418349804 >> v1:172.21.15.158:6789/0 conn(0x7fe8080696a0 legacy=0x7fe808157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.248+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3418349804 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8080c0010 con 0x7fe808154170 2023-05-16T07:19:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80cb4c700 1 --2- 172.21.15.92:0/3418349804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe808154170 0x7fe8080c5c70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fe8080844e0 tx=0x7fe7f80090f0).ready entity=mon.0 client_cookie=8346b9db238fe18 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe8077fe700 1 -- 172.21.15.92:0/3418349804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800b5a0 con 0x7fe808154170 2023-05-16T07:19:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe8077fe700 1 -- 172.21.15.92:0/3418349804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f800b700 con 0x7fe808154170 2023-05-16T07:19:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe8077fe700 1 -- 172.21.15.92:0/3418349804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800baa0 con 0x7fe808154170 2023-05-16T07:19:14.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 -- 172.21.15.92:0/3418349804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe808154170 msgr2=0x7fe8080c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3418349804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe808154170 0x7fe8080c5c70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fe8080844e0 tx=0x7fe7f80090f0).stop 2023-05-16T07:19:14.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 -- 172.21.15.92:0/3418349804 shutdown_connections 2023-05-16T07:19:14.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3418349804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe808154170 0x7fe8080c5c70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3418349804 >> v2:172.21.15.158:3300/0 conn(0x7fe808154550 0x7fe8080c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 -- 172.21.15.92:0/3418349804 >> 172.21.15.92:0/3418349804 conn(0x7fe8080cb700 msgr2=0x7fe8080c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 -- 172.21.15.92:0/3418349804 shutdown_connections 2023-05-16T07:19:14.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.249+0000 7fe80edb0700 1 -- 172.21.15.92:0/3418349804 wait complete. 2023-05-16T07:19:14.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.250+0000 7fe80edb0700 1 Processor -- start 2023-05-16T07:19:14.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.250+0000 7fe80edb0700 1 -- start start 2023-05-16T07:19:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80edb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 0x7fe808156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80edb0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe808154170 0x7fe8081686e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80edb0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe808154550 0x7fe80814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80edb0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe808155ea0 con 0x7fe8080696a0 2023-05-16T07:19:14.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80edb0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe808156160 con 0x7fe808154170 2023-05-16T07:19:14.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80edb0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe808156440 con 0x7fe808154550 2023-05-16T07:19:14.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80cb4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 0x7fe808156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80cb4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 0x7fe808156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48664/0 (socket says 172.21.15.92:48664) 2023-05-16T07:19:14.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3779426733 learned_addr learned my addr 172.21.15.92:0/3779426733 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.251+0000 7fe807fff700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe808154170 0x7fe8081686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80d34d700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe808154550 0x7fe80814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe808154550 msgr2=0x7fe80814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80cb4c700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe808154550 0x7fe80814d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe808154170 msgr2=0x7fe8081686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80cb4c700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe808154170 0x7fe8081686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80cb4c700 1 -- 172.21.15.92:0/3779426733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8009370 con 0x7fe8080696a0 2023-05-16T07:19:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe807fff700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe808154170 0x7fe8081686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80cb4c700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 0x7fe808156f20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fe808083060 tx=0x7fe7f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800bee0 con 0x7fe8080696a0 2023-05-16T07:19:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f800b870 con 0x7fe8080696a0 2023-05-16T07:19:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe808156720 con 0x7fe8080696a0 2023-05-16T07:19:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80246c0 con 0x7fe8080696a0 2023-05-16T07:19:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.252+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe808167ff0 con 0x7fe8080696a0 2023-05-16T07:19:14.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.253+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8080666c0 con 0x7fe8080696a0 2023-05-16T07:19:14.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.253+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe7f800f070 con 0x7fe8080696a0 2023-05-16T07:19:14.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.254+0000 7fe804ff9700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe7e805ed90 0x7fe7e8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.254+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fe7f807b9b0 con 0x7fe8080696a0 2023-05-16T07:19:14.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.254+0000 7fe807fff700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe7e805ed90 0x7fe7e8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.254+0000 7fe807fff700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe7e805ed90 0x7fe7e8061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe8080c60a0 tx=0x7fe7f00096a0).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:14.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.258+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe7f804ce40 con 0x7fe8080696a0 2023-05-16T07:19:14.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.461+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe80806c350 con 0x7fe8080696a0 2023-05-16T07:19:14.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.461+0000 7fe804ff9700 1 -- 172.21.15.92:0/3779426733 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe7f8029070 con 0x7fe8080696a0 2023-05-16T07:19:14.465 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:14.466 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1910037752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1910037752},{"type":"v1","addr":"172.21.15.158:6835","nonce":1910037752}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/1331375849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":1331375849},{"type":"v1","addr":"172.21.15.158:6837","nonce":1331375849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/921461900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":921461900},{"type":"v1","addr":"172.21.15.92:6834","nonce":921461900}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3134498028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3134498028},{"type":"v1","addr":"172.21.15.92:6835","nonce":3134498028}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[]} 2023-05-16T07:19:14.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe7e805ed90 msgr2=0x7fe7e8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe7e805ed90 0x7fe7e8061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe8080c60a0 tx=0x7fe7f00096a0).stop 2023-05-16T07:19:14.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 msgr2=0x7fe808156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 0x7fe808156f20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fe808083060 tx=0x7fe7f8015000).stop 2023-05-16T07:19:14.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 shutdown_connections 2023-05-16T07:19:14.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe7e805ed90 0x7fe7e8061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe808154550 0x7fe80814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8080696a0 0x7fe808156f20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 --2- 172.21.15.92:0/3779426733 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe808154170 0x7fe8081686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 >> 172.21.15.92:0/3779426733 conn(0x7fe8080cb700 msgr2=0x7fe8080ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:14.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 shutdown_connections 2023-05-16T07:19:14.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.463+0000 7fe80edb0700 1 -- 172.21.15.92:0/3779426733 wait complete. 2023-05-16T07:19:14.469 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 39 2023-05-16T07:19:14.479 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:19:14.479 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:19:14.479 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:19:14.479 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:14.480 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:19:14.480 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:19:14.480 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:19:14.480 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:14.481 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:19:14.482 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:19:14.482 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:19:14.482 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:14.482 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:19:14.483 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:19:14.483 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:19:14.483 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:14.487 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:19:14.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:19:14.490 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:19:14.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:19:14.492 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:19:14.493 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:19:14.494 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:19:14.494 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:19:14.530 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:19:14.530 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:14.531 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:19:14.532 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:14.536 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:19:14.537 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:14.538 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:19:14.538 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:14.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:19:14.600 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:19:14.601 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:19:14.618 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:19:14.618 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:19:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.656+0000 7f06ba5a4700 1 Processor -- start 2023-05-16T07:19:14.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.656+0000 7f06ba5a4700 1 -- start start 2023-05-16T07:19:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06ba5a4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f06b4154550 0x7f06b40696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06ba5a4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b40c0660 con 0x7f06b4069b90 2023-05-16T07:19:14.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06ba5a4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b4069fc0 con 0x7f06b4154170 2023-05-16T07:19:14.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06ba5a4700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b40bfd30 con 0x7f06b4154550 2023-05-16T07:19:14.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06b8b41700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f06b4069b90 0x7f06b40c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59446/0 (socket says 172.21.15.92:59446) 2023-05-16T07:19:14.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06b8b41700 1 -- 172.21.15.92:0/1281769734 learned_addr learned my addr 172.21.15.92:0/1281769734 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06b37fe700 1 --2- 172.21.15.92:0/1281769734 >> v2:172.21.15.158:3300/0 conn(0x7f06b4154550 0x7f06b40696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.657+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833594409 0 0) 0x7f06b40c0660 con 0x7f06b4069b90 2023-05-16T07:19:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069c0033f0 con 0x7f06b4069b90 2023-05-16T07:19:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309249905 0 0) 0x7f06b4069fc0 con 0x7f06b4154170 2023-05-16T07:19:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b40c0660 con 0x7f06b4154170 2023-05-16T07:19:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 59256827 0 0) 0x7f069c0033f0 con 0x7f06b4069b90 2023-05-16T07:19:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 >> v2:172.21.15.158:3300/0 conn(0x7f06b4154550 msgr2=0x7f06b40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:14.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 --2- 172.21.15.92:0/1281769734 >> v2:172.21.15.158:3300/0 conn(0x7f06b4154550 0x7f06b40696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 >> v1:172.21.15.158:6789/0 conn(0x7f06b4154170 legacy=0x7f06b40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b40c0010 con 0x7f06b4069b90 2023-05-16T07:19:14.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f06ac0017f0 con 0x7f06b4069b90 2023-05-16T07:19:14.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 --2- 172.21.15.92:0/1281769734 >> v2:172.21.15.158:3300/0 conn(0x7f06b4154550 0x7f06b40696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:14.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 >> v1:172.21.15.92:6789/0 conn(0x7f06b4069b90 legacy=0x7f06b40c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f069c003a70 0x7f069c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c004690 0x7f069c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c005050 0x7f069c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40bfd30 con 0x7f069c004690 2023-05-16T07:19:14.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c00bfc0 con 0x7f069c003a70 2023-05-16T07:19:14.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c00c180 con 0x7f069c005050 2023-05-16T07:19:14.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c004690 0x7f069c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b3fff700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f069c003a70 0x7f069c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b8b41700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c005050 0x7f069c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 -- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c005050 msgr2=0x7f069c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:14.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c005050 0x7f069c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 -- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f069c003a70 msgr2=0x7f069c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:14.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f069c003a70 0x7f069c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b8b41700 1 -- 172.21.15.92:0/1281769734 reap_dead start 2023-05-16T07:19:14.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 -- 172.21.15.92:0/1281769734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0033f0 con 0x7f069c004690 2023-05-16T07:19:14.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b3fff700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f069c003a70 0x7f069c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.658+0000 7f06b37fe700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c004690 0x7f069c004b10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f06b4154930 tx=0x7f06a8009060).ready entity=mon.0 client_cookie=b8a2dcaeb604222b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80095c0 con 0x7f069c004690 2023-05-16T07:19:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800e710 con 0x7f069c004690 2023-05-16T07:19:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06b2ffd700 1 -- 172.21.15.92:0/1281769734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800ea90 con 0x7f069c004690 2023-05-16T07:19:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 -- 172.21.15.92:0/1281769734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c004690 msgr2=0x7f069c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c004690 0x7f069c004b10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f06b4154930 tx=0x7f06a8009060).stop 2023-05-16T07:19:14.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 -- 172.21.15.92:0/1281769734 shutdown_connections 2023-05-16T07:19:14.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/1281769734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f069c004690 0x7f069c004b10 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 -- 172.21.15.92:0/1281769734 >> 172.21.15.92:0/1281769734 conn(0x7f06b40cb700 msgr2=0x7f06b40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:14.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 -- 172.21.15.92:0/1281769734 shutdown_connections 2023-05-16T07:19:14.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.659+0000 7f06ba5a4700 1 -- 172.21.15.92:0/1281769734 wait complete. 2023-05-16T07:19:14.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.660+0000 7f06ba5a4700 1 Processor -- start 2023-05-16T07:19:14.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.660+0000 7f06ba5a4700 1 -- start start 2023-05-16T07:19:14.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06ba5a4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c004690 0x7f06b40afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06ba5a4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 0x7f06b414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06ba5a4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06b40aec20 0x7f06b40af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06ba5a4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41519e0 con 0x7f06b40c1d40 2023-05-16T07:19:14.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06ba5a4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40af590 con 0x7f06b40aec20 2023-05-16T07:19:14.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06ba5a4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40c24b0 con 0x7f069c004690 2023-05-16T07:19:14.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 0x7f06b414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 0x7f06b414cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48708/0 (socket says 172.21.15.92:48708) 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 -- 172.21.15.92:0/703733389 learned_addr learned my addr 172.21.15.92:0/703733389 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b3fff700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c004690 0x7f06b40afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b8b41700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06b40aec20 0x7f06b40af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 -- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c004690 msgr2=0x7f06b40afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c004690 0x7f06b40afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 -- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06b40aec20 msgr2=0x7f06b40af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:14.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06b40aec20 0x7f06b40af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:14.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.661+0000 7f06b37fe700 1 -- 172.21.15.92:0/703733389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b41510b0 con 0x7f06b40c1d40 2023-05-16T07:19:14.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f06b8b41700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06b40aec20 0x7f06b40af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:14.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f06b37fe700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 0x7f06b414cb60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f06a800e4b0 tx=0x7f06a801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:14.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800e880 con 0x7f06b40c1d40 2023-05-16T07:19:14.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800e9e0 con 0x7f06b40c1d40 2023-05-16T07:19:14.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b4151390 con 0x7f06b40c1d40 2023-05-16T07:19:14.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8024600 con 0x7f06b40c1d40 2023-05-16T07:19:14.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.662+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b40c21c0 con 0x7f06b40c1d40 2023-05-16T07:19:14.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.663+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b40666c0 con 0x7f06b40c1d40 2023-05-16T07:19:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.663+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f06a802b030 con 0x7f06b40c1d40 2023-05-16T07:19:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.663+0000 7f069bfff700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f06a005ed90 0x7f06a0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.664+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f06a807b1e0 con 0x7f06b40c1d40 2023-05-16T07:19:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.664+0000 7f06b3fff700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f06a005ed90 0x7f06a0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.664+0000 7f06b3fff700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f06a005ed90 0x7f06a0061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f06a40010e0 tx=0x7f06a4007bb0).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:14.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.668+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f06a804c610 con 0x7f06b40c1d40 2023-05-16T07:19:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:14.846+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f06b406c350 con 0x7f06b40c1d40 2023-05-16T07:19:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.750+0000 7f069bfff700 1 -- 172.21.15.92:0/703733389 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7f06a804fc30 con 0x7f06b40c1d40 2023-05-16T07:19:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f06a005ed90 msgr2=0x7f06a0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:15.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f06a005ed90 0x7f06a0061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f06a40010e0 tx=0x7f06a4007bb0).stop 2023-05-16T07:19:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 msgr2=0x7f06b414cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 0x7f06b414cb60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f06a800e4b0 tx=0x7f06a801a000).stop 2023-05-16T07:19:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 shutdown_connections 2023-05-16T07:19:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f06a005ed90 0x7f06a0061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f069c004690 0x7f06b40afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.752+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f06b40aec20 0x7f06b40af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.753+0000 7f06ba5a4700 1 --2- 172.21.15.92:0/703733389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b40c1d40 0x7f06b414cb60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.753+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 >> 172.21.15.92:0/703733389 conn(0x7f06b40cb700 msgr2=0x7f06b40ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.753+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 shutdown_connections 2023-05-16T07:19:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.753+0000 7f06ba5a4700 1 -- 172.21.15.92:0/703733389 wait complete. 2023-05-16T07:19:15.759 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:19:15.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:19:15.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 Processor -- start 2023-05-16T07:19:15.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 -- start start 2023-05-16T07:19:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f82d8154170 0x7f82d80c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82d8069db0 0x7f82d80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d80c0660 con 0x7f82d8154550 2023-05-16T07:19:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d80c1b00 con 0x7f82d8069db0 2023-05-16T07:19:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.885+0000 7f82de7d8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d80bfd30 con 0x7f82d8154170 2023-05-16T07:19:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82d8069db0 0x7f82d80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82d8069db0 0x7f82d80c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48722/0 (socket says 172.21.15.92:48722) 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 -- 172.21.15.92:0/144293656 learned_addr learned my addr 172.21.15.92:0/144293656 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82d77fe700 1 --1- 172.21.15.92:0/144293656 >> v1:172.21.15.92:6789/0 conn(0x7f82d8154550 0x7f82d80696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59472/0 (socket says 172.21.15.92:59472) 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 -- 172.21.15.92:0/144293656 >> v2:172.21.15.158:3300/0 conn(0x7f82d8154170 msgr2=0x7f82d80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 --2- 172.21.15.92:0/144293656 >> v2:172.21.15.158:3300/0 conn(0x7f82d8154170 0x7f82d80c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 -- 172.21.15.92:0/144293656 >> v1:172.21.15.92:6789/0 conn(0x7f82d8154550 legacy=0x7f82d80696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82d6ffd700 1 -- 172.21.15.92:0/144293656 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572538622 0 0) 0x7f82d80c0660 con 0x7f82d8154550 2023-05-16T07:19:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 -- 172.21.15.92:0/144293656 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80c0010 con 0x7f82d8069db0 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82dcd75700 1 --2- 172.21.15.92:0/144293656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8069db0 0x7f82d80c6160 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f82d8083030 tx=0x7f82d0009840).ready entity=mon.0 client_cookie=6d72add0d219e9ed server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82d6ffd700 1 -- 172.21.15.92:0/144293656 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000e070 con 0x7f82d8069db0 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82d6ffd700 1 -- 172.21.15.92:0/144293656 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d000ba50 con 0x7f82d8069db0 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.886+0000 7f82d6ffd700 1 -- 172.21.15.92:0/144293656 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000bdd0 con 0x7f82d8069db0 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 -- 172.21.15.92:0/144293656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8069db0 msgr2=0x7f82d80c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 --2- 172.21.15.92:0/144293656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8069db0 0x7f82d80c6160 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f82d8083030 tx=0x7f82d0009840).stop 2023-05-16T07:19:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 -- 172.21.15.92:0/144293656 shutdown_connections 2023-05-16T07:19:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 --2- 172.21.15.92:0/144293656 >> v2:172.21.15.158:3300/0 conn(0x7f82d8154170 0x7f82d80c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 --2- 172.21.15.92:0/144293656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8069db0 0x7f82d80c6160 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 -- 172.21.15.92:0/144293656 >> 172.21.15.92:0/144293656 conn(0x7f82d80cb700 msgr2=0x7f82d80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 -- 172.21.15.92:0/144293656 shutdown_connections 2023-05-16T07:19:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.887+0000 7f82de7d8700 1 -- 172.21.15.92:0/144293656 wait complete. 2023-05-16T07:19:15.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.888+0000 7f82de7d8700 1 Processor -- start 2023-05-16T07:19:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.888+0000 7f82de7d8700 1 -- start start 2023-05-16T07:19:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f82d8069db0 0x7f82d8147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82d8154170 0x7f82d8148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 0x7f82d80c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80c2f10 con 0x7f82d8154550 2023-05-16T07:19:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8147f70 con 0x7f82d8069db0 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80c2670 con 0x7f82d8154170 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 0x7f82d80c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 0x7f82d80c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48730/0 (socket says 172.21.15.92:48730) 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 -- 172.21.15.92:0/4029809046 learned_addr learned my addr 172.21.15.92:0/4029809046 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82d7fff700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f82d8069db0 0x7f82d8147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82d77fe700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82d8154170 0x7f82d8148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 -- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82d8154170 msgr2=0x7f82d8148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82d8154170 0x7f82d8148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 -- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f82d8069db0 msgr2=0x7f82d8147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f82d8069db0 0x7f82d8147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 -- 172.21.15.92:0/4029809046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0009ae0 con 0x7f82d8154550 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82d7fff700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f82d8069db0 0x7f82d8147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82dcd75700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 0x7f82d80c1d40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f82d0009580 tx=0x7f82d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00092e0 con 0x7f82d8154550 2023-05-16T07:19:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d001d410 con 0x7f82d8154550 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d80c2950 con 0x7f82d8154550 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d001d7b0 con 0x7f82d8154550 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.889+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d80add60 con 0x7f82d8154550 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.890+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d80666c0 con 0x7f82d8154550 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.891+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f82d001b040 con 0x7f82d8154550 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.891+0000 7f82bffff700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f82c405ed90 0x7f82c4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.891+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f82d007b4a0 con 0x7f82d8154550 2023-05-16T07:19:15.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.891+0000 7f82d7fff700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f82c405ed90 0x7f82c4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:15.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.892+0000 7f82d7fff700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f82c405ed90 0x7f82c4061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f82c8009990 tx=0x7f82c8008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:15.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:15.895+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f82d004cb30 con 0x7f82d8154550 2023-05-16T07:19:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.121+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f82d806c350 con 0x7f82d8154550 2023-05-16T07:19:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.122+0000 7f82bffff700 1 -- 172.21.15.92:0/4029809046 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f82d0050150 con 0x7f82d8154550 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:19:16.126 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:19:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:19:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:19:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:19:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:19:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:19:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:19:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:19:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:19:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:19:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:19:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.141 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:19:16.141 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:19:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:19:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:19:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:19:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:19:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:19:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:19:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:19:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:19:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:19:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:19:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:19:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:19:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:19:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:19:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:19:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:19:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:19:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:19:16.156 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:19:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f82c405ed90 msgr2=0x7f82c4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f82c405ed90 0x7f82c4061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f82c8009990 tx=0x7f82c8008040).stop 2023-05-16T07:19:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 msgr2=0x7f82d80c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 0x7f82d80c1d40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f82d0009580 tx=0x7f82d0009840).stop 2023-05-16T07:19:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 shutdown_connections 2023-05-16T07:19:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f82c405ed90 0x7f82c4061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82d8154550 0x7f82d80c1d40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f82d8069db0 0x7f82d8147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 --2- 172.21.15.92:0/4029809046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f82d8154170 0x7f82d8148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 >> 172.21.15.92:0/4029809046 conn(0x7f82d80cb700 msgr2=0x7f82d814fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 shutdown_connections 2023-05-16T07:19:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.124+0000 7f82de7d8700 1 -- 172.21.15.92:0/4029809046 wait complete. 2023-05-16T07:19:16.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.266+0000 7fe0c6f52700 1 Processor -- start 2023-05-16T07:19:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.266+0000 7fe0c6f52700 1 -- start start 2023-05-16T07:19:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0c6f52700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0c013b550 0x7fe0c00c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0c6f52700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0c6f52700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c00bdc30 con 0x7fe0c013b930 2023-05-16T07:19:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0c6f52700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00c5400 con 0x7fe0c013b550 2023-05-16T07:19:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0c6f52700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c4e40 con 0x7fe0c0063e30 2023-05-16T07:19:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0c4cee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0c013b550 0x7fe0c00c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0bffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe0c013b930 0x7fe0c0063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59484/0 (socket says 172.21.15.92:59484) 2023-05-16T07:19:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.267+0000 7fe0bffff700 1 -- 172.21.15.92:0/2981122922 learned_addr learned my addr 172.21.15.92:0/2981122922 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c4cee700 1 -- 172.21.15.92:0/2981122922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 msgr2=0x7fe0c00abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c54ef700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c4cee700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c4cee700 1 -- 172.21.15.92:0/2981122922 >> v1:172.21.15.92:6789/0 conn(0x7fe0c013b930 legacy=0x7fe0c0063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2981122922 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635883263 0 0) 0x7fe0c00bdc30 con 0x7fe0c013b930 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c4cee700 1 -- 172.21.15.92:0/2981122922 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00c5120 con 0x7fe0c013b550 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c54ef700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c4cee700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b550 0x7fe0c00c31f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fe0b0009760 tx=0x7fe0b0009060).ready entity=mon.0 client_cookie=9b8b012817c87a4d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2981122922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000b580 con 0x7fe0c013b550 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2981122922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b000b6e0 con 0x7fe0c013b550 2023-05-16T07:19:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0bf7fe700 1 -- 172.21.15.92:0/2981122922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000ba60 con 0x7fe0c013b550 2023-05-16T07:19:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 -- 172.21.15.92:0/2981122922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b550 msgr2=0x7fe0c00c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b550 0x7fe0c00c31f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fe0b0009760 tx=0x7fe0b0009060).stop 2023-05-16T07:19:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 -- 172.21.15.92:0/2981122922 shutdown_connections 2023-05-16T07:19:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/2981122922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b550 0x7fe0c00c31f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 -- 172.21.15.92:0/2981122922 >> 172.21.15.92:0/2981122922 conn(0x7fe0c00c8aa0 msgr2=0x7fe0c00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 -- 172.21.15.92:0/2981122922 shutdown_connections 2023-05-16T07:19:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.268+0000 7fe0c6f52700 1 -- 172.21.15.92:0/2981122922 wait complete. 2023-05-16T07:19:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.269+0000 7fe0c6f52700 1 Processor -- start 2023-05-16T07:19:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 -- start start 2023-05-16T07:19:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c013b550 0x7fe0c00c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 0x7fe0c00c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c8000 con 0x7fe0c013b930 2023-05-16T07:19:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0157b90 con 0x7fe0c013b550 2023-05-16T07:19:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.270+0000 7fe0c6f52700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0158a00 con 0x7fe0c0063e30 2023-05-16T07:19:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 0x7fe0c00c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 0x7fe0c00c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48744/0 (socket says 172.21.15.92:48744) 2023-05-16T07:19:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 -- 172.21.15.92:0/1485138825 learned_addr learned my addr 172.21.15.92:0/1485138825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c4cee700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0bffff700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c013b550 0x7fe0c00c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 -- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 msgr2=0x7fe0c00c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 -- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c013b550 msgr2=0x7fe0c00c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c013b550 0x7fe0c00c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 -- 172.21.15.92:0/1485138825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00092a0 con 0x7fe0c013b930 2023-05-16T07:19:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c54ef700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 0x7fe0c00c7b10 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fe0b800cf90 tx=0x7fe0b80078a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800ae60 con 0x7fe0c013b930 2023-05-16T07:19:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b8016470 con 0x7fe0c013b930 2023-05-16T07:19:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c0158440 con 0x7fe0c013b930 2023-05-16T07:19:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8016810 con 0x7fe0c013b930 2023-05-16T07:19:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.271+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c0157390 con 0x7fe0c013b930 2023-05-16T07:19:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.272+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c006c350 con 0x7fe0c013b930 2023-05-16T07:19:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.273+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe0b801d060 con 0x7fe0c013b930 2023-05-16T07:19:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.273+0000 7fe0bcff9700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.273+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe0b807a530 con 0x7fe0c013b930 2023-05-16T07:19:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.273+0000 7fe0c4cee700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.273+0000 7fe0c4cee700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe0b000b340 tx=0x7fe0b0017000).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.277+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe0b804bbc0 con 0x7fe0c013b930 2023-05-16T07:19:16.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.443+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0c00c4b50 con 0x7fe0c013b930 2023-05-16T07:19:16.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.444+0000 7fe0bcff9700 1 -- 172.21.15.92:0/1485138825 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+14698 (secure 0 0 0) 0x7fe0b801b070 con 0x7fe0c013b930 2023-05-16T07:19:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:16.449 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":117,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:15.744157+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:16.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 msgr2=0x7fe0a0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe0b000b340 tx=0x7fe0b0017000).stop 2023-05-16T07:19:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 msgr2=0x7fe0c00c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 0x7fe0c00c7b10 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fe0b800cf90 tx=0x7fe0b80078a0).stop 2023-05-16T07:19:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 shutdown_connections 2023-05-16T07:19:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe0a005ed90 0x7fe0a0061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe0c0063e30 0x7fe0c00c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0c013b930 0x7fe0c00c7b10 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 --2- 172.21.15.92:0/1485138825 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe0c013b550 0x7fe0c00c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 >> 172.21.15.92:0/1485138825 conn(0x7fe0c00c8aa0 msgr2=0x7fe0c0154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:16.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 shutdown_connections 2023-05-16T07:19:16.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.446+0000 7fe0c6f52700 1 -- 172.21.15.92:0/1485138825 wait complete. 2023-05-16T07:19:16.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:19:16.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:19:16.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.572+0000 7ff45b351700 1 Processor -- start 2023-05-16T07:19:16.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.572+0000 7ff45b351700 1 -- start start 2023-05-16T07:19:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff45b351700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff4540aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff45b351700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff454154470 0x7ff4540a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff45b351700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4540aa570 con 0x7ff4540aa150 2023-05-16T07:19:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff45b351700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540c94d0 con 0x7ff454154470 2023-05-16T07:19:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff45b351700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540cb7e0 con 0x7ff454154090 2023-05-16T07:19:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff4598ee700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff4540aa150 0x7ff454068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59498/0 (socket says 172.21.15.92:59498) 2023-05-16T07:19:16.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff4598ee700 1 -- 172.21.15.92:0/2238034398 learned_addr learned my addr 172.21.15.92:0/2238034398 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036925246 0 0) 0x7ff44c001110 con 0x7ff4540aa150 2023-05-16T07:19:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff43c0033f0 con 0x7ff4540aa150 2023-05-16T07:19:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff4540aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.573+0000 7ff4588ec700 1 --2- 172.21.15.92:0/2238034398 >> v2:172.21.15.158:3300/0 conn(0x7ff454154470 0x7ff4540a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2203253053 0 0) 0x7ff43c0033f0 con 0x7ff4540aa150 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 msgr2=0x7ff4540aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff4540aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 >> v2:172.21.15.158:3300/0 conn(0x7ff454154470 msgr2=0x7ff4540a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 --2- 172.21.15.92:0/2238034398 >> v2:172.21.15.158:3300/0 conn(0x7ff454154470 0x7ff4540a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540c88c0 con 0x7ff4540aa150 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7ff44c001600 con 0x7ff4540aa150 2023-05-16T07:19:16.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 >> v1:172.21.15.92:6789/0 conn(0x7ff4540aa150 legacy=0x7ff454068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff43c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff43c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff43c004b80 0x7ff43c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540c94d0 con 0x7ff43c0038e0 2023-05-16T07:19:16.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540cb7e0 con 0x7ff43c004b80 2023-05-16T07:19:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c00b8e0 con 0x7ff454154090 2023-05-16T07:19:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff43c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4588ec700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff43c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4598ee700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff43c004b80 0x7ff43c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 -- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 msgr2=0x7ff43c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff43c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 -- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff43c004b80 msgr2=0x7ff43c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff43c004b80 0x7ff43c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 -- 172.21.15.92:0/2238034398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0033f0 con 0x7ff43c0038e0 2023-05-16T07:19:16.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4588ec700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454154090 0x7ff43c004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff43c003d00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7ff44400e1c0 tx=0x7ff44400b040).ready entity=mon.0 client_cookie=3b42d69dd525a3c2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444009ac0 con 0x7ff43c0038e0 2023-05-16T07:19:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400f040 con 0x7ff43c0038e0 2023-05-16T07:19:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.574+0000 7ff453fff700 1 -- 172.21.15.92:0/2238034398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400eca0 con 0x7ff43c0038e0 2023-05-16T07:19:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff45b351700 1 -- 172.21.15.92:0/2238034398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 msgr2=0x7ff43c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff45b351700 1 --2- 172.21.15.92:0/2238034398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff43c003d00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7ff44400e1c0 tx=0x7ff44400b040).stop 2023-05-16T07:19:16.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff4598ee700 1 -- 172.21.15.92:0/2238034398 reap_dead start 2023-05-16T07:19:16.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff45b351700 1 -- 172.21.15.92:0/2238034398 shutdown_connections 2023-05-16T07:19:16.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff45b351700 1 -- 172.21.15.92:0/2238034398 >> 172.21.15.92:0/2238034398 conn(0x7ff454067920 msgr2=0x7ff4540cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:16.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff45b351700 1 -- 172.21.15.92:0/2238034398 shutdown_connections 2023-05-16T07:19:16.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.575+0000 7ff45b351700 1 -- 172.21.15.92:0/2238034398 wait complete. 2023-05-16T07:19:16.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.576+0000 7ff45b351700 1 Processor -- start 2023-05-16T07:19:16.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.576+0000 7ff45b351700 1 -- start start 2023-05-16T07:19:16.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff45b351700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff4540adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff45b351700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4540ae0d0 0x7ff454150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff45b351700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454150a70 0x7ff454150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff45b351700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540aee40 con 0x7ff43c0038e0 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff45b351700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454154e10 con 0x7ff4540ae0d0 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff45b351700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541550d0 con 0x7ff454150a70 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff4540adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff4540adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48760/0 (socket says 172.21.15.92:48760) 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 -- 172.21.15.92:0/2704248417 learned_addr learned my addr 172.21.15.92:0/2704248417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 -- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454150a70 msgr2=0x7ff454150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454150a70 0x7ff454150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:16.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 -- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4540ae0d0 msgr2=0x7ff454150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4540ae0d0 0x7ff454150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 -- 172.21.15.92:0/2704248417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4541513e0 con 0x7ff43c0038e0 2023-05-16T07:19:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.577+0000 7ff4590ed700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff4540adbe0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7ff444013ab0 tx=0x7ff44401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.578+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444027420 con 0x7ff43c0038e0 2023-05-16T07:19:16.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.578+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444027580 con 0x7ff43c0038e0 2023-05-16T07:19:16.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.578+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4540ae5a0 con 0x7ff43c0038e0 2023-05-16T07:19:16.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.578+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444027730 con 0x7ff43c0038e0 2023-05-16T07:19:16.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.578+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4540aeb60 con 0x7ff43c0038e0 2023-05-16T07:19:16.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.579+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454066730 con 0x7ff43c0038e0 2023-05-16T07:19:16.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.579+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff44402e070 con 0x7ff43c0038e0 2023-05-16T07:19:16.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.580+0000 7ff4517fa700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff44005ed60 0x7ff440061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.580+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7ff44407c170 con 0x7ff43c0038e0 2023-05-16T07:19:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.580+0000 7ff4588ec700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff44005ed60 0x7ff440061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.580+0000 7ff4588ec700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff44005ed60 0x7ff440061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff4540ae4d0 tx=0x7ff448008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.583+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff44404d880 con 0x7ff43c0038e0 2023-05-16T07:19:16.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.745+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff45406c350 con 0x7ff43c0038e0 2023-05-16T07:19:16.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.746+0000 7ff4517fa700 1 -- 172.21.15.92:0/2704248417 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v117) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff44402c020 con 0x7ff43c0038e0 2023-05-16T07:19:16.750 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:19:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff44005ed60 msgr2=0x7ff440061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff44005ed60 0x7ff440061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff4540ae4d0 tx=0x7ff448008040).stop 2023-05-16T07:19:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 msgr2=0x7ff4540adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff4540adbe0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7ff444013ab0 tx=0x7ff44401a000).stop 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 shutdown_connections 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff44005ed60 0x7ff440061220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff454150a70 0x7ff454150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff43c0038e0 0x7ff4540adbe0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 --2- 172.21.15.92:0/2704248417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff4540ae0d0 0x7ff454150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 >> 172.21.15.92:0/2704248417 conn(0x7ff454067920 msgr2=0x7ff4540c9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 shutdown_connections 2023-05-16T07:19:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.747+0000 7ff45b351700 1 -- 172.21.15.92:0/2704248417 wait complete. 2023-05-16T07:19:16.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:19:16.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 Processor -- start 2023-05-16T07:19:16.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 -- start start 2023-05-16T07:19:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 0x7ff2300698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff2300bdf20 0x7ff230067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23005e760 con 0x7ff230153f90 2023-05-16T07:19:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff230069ef0 con 0x7ff2300bdf20 2023-05-16T07:19:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.874+0000 7ff234d20700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300bf540 con 0x7ff230153bb0 2023-05-16T07:19:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff230153f90 0x7ff2300bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59518/0 (socket says 172.21.15.92:59518) 2023-05-16T07:19:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22effd700 1 -- 172.21.15.92:0/2451902298 learned_addr learned my addr 172.21.15.92:0/2451902298 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491742631 0 0) 0x7ff23005e760 con 0x7ff230153f90 2023-05-16T07:19:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff21c0033f0 con 0x7ff230153f90 2023-05-16T07:19:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22f7fe700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 0x7ff2300698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22ffff700 1 --2- 172.21.15.92:0/2451902298 >> v2:172.21.15.158:3300/0 conn(0x7ff2300bdf20 0x7ff230067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4207154236 0 0) 0x7ff21c0033f0 con 0x7ff230153f90 2023-05-16T07:19:16.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 msgr2=0x7ff2300698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 0x7ff2300698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 >> v2:172.21.15.158:3300/0 conn(0x7ff2300bdf20 msgr2=0x7ff230067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 --2- 172.21.15.92:0/2451902298 >> v2:172.21.15.158:3300/0 conn(0x7ff2300bdf20 0x7ff230067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230068450 con 0x7ff230153f90 2023-05-16T07:19:16.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7ff218001470 con 0x7ff230153f90 2023-05-16T07:19:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 >> v1:172.21.15.92:6789/0 conn(0x7ff230153f90 legacy=0x7ff2300bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff21c0038e0 0x7ff21c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff21c004690 0x7ff21c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 0x7ff21c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff230069ef0 con 0x7ff21c004690 2023-05-16T07:19:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300bf540 con 0x7ff21c0038e0 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c00b8e0 con 0x7ff230153bb0 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22effd700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff21c004690 0x7ff21c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22f7fe700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff21c0038e0 0x7ff21c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.875+0000 7ff22ffff700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 0x7ff21c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22effd700 1 -- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 msgr2=0x7ff21c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22effd700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff230153bb0 0x7ff21c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22effd700 1 -- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff21c0038e0 msgr2=0x7ff21c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22effd700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff21c0038e0 0x7ff21c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22effd700 1 -- 172.21.15.92:0/2451902298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0033f0 con 0x7ff21c004690 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22effd700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff21c004690 0x7ff21c004b10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7ff2300be7f0 tx=0x7ff2180077c0).ready entity=mon.0 client_cookie=39bda0b3ba0d4e8f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21800ca40 con 0x7ff21c004690 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21800cba0 con 0x7ff21c004690 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22e7fc700 1 -- 172.21.15.92:0/2451902298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180070d0 con 0x7ff21c004690 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff234d20700 1 -- 172.21.15.92:0/2451902298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff21c004690 msgr2=0x7ff21c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff234d20700 1 --2- 172.21.15.92:0/2451902298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff21c004690 0x7ff21c004b10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7ff2300be7f0 tx=0x7ff2180077c0).stop 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff22ffff700 1 -- 172.21.15.92:0/2451902298 reap_dead start 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff234d20700 1 -- 172.21.15.92:0/2451902298 shutdown_connections 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff234d20700 1 -- 172.21.15.92:0/2451902298 >> 172.21.15.92:0/2451902298 conn(0x7ff230062d80 msgr2=0x7ff23005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff234d20700 1 -- 172.21.15.92:0/2451902298 shutdown_connections 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.876+0000 7ff234d20700 1 -- 172.21.15.92:0/2451902298 wait complete. 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 Processor -- start 2023-05-16T07:19:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 -- start start 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff21c004690 0x7ff230152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff230144b90 0x7ff2300c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 0x7ff2300ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300ad840 con 0x7ff2300c4cf0 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300c05a0 con 0x7ff230144b90 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff234d20700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff230152c80 con 0x7ff21c004690 2023-05-16T07:19:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff22ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 0x7ff2300ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.878+0000 7ff22ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 0x7ff2300ac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48784/0 (socket says 172.21.15.92:48784) 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 -- 172.21.15.92:0/3396879153 learned_addr learned my addr 172.21.15.92:0/3396879153 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22f7fe700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff21c004690 0x7ff230152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22effd700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff230144b90 0x7ff2300c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 -- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff21c004690 msgr2=0x7ff230152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff21c004690 0x7ff230152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 -- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff230144b90 msgr2=0x7ff2300c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff230144b90 0x7ff2300c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 -- 172.21.15.92:0/3396879153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300c5170 con 0x7ff2300c4cf0 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff22ffff700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 0x7ff2300ac830 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7ff2300bf4b0 tx=0x7ff228009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228013680 con 0x7ff2300c4cf0 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2280137e0 con 0x7ff2300c4cf0 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2300ad560 con 0x7ff2300c4cf0 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228013b80 con 0x7ff2300c4cf0 2023-05-16T07:19:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.879+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2300c0230 con 0x7ff2300c4cf0 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.880+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff230047150 con 0x7ff2300c4cf0 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.880+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff228024070 con 0x7ff2300c4cf0 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.881+0000 7ff2277fe700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff21005ed90 0x7ff210061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.881+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7ff22807a0e0 con 0x7ff2300c4cf0 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.881+0000 7ff22f7fe700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff21005ed90 0x7ff210061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.881+0000 7ff22f7fe700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff21005ed90 0x7ff210061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff2200094d0 tx=0x7ff22000b040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:16.885+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff22804b770 con 0x7ff2300c4cf0 2023-05-16T07:19:17.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.047+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff230158460 con 0x7ff2300c4cf0 2023-05-16T07:19:17.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.047+0000 7ff2277fe700 1 -- 172.21.15.92:0/3396879153 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff228018020 con 0x7ff2300c4cf0 2023-05-16T07:19:17.051 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:19:17.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff21005ed90 msgr2=0x7ff210061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff21005ed90 0x7ff210061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff2200094d0 tx=0x7ff22000b040).stop 2023-05-16T07:19:17.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 msgr2=0x7ff2300ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 0x7ff2300ac830 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7ff2300bf4b0 tx=0x7ff228009060).stop 2023-05-16T07:19:17.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 shutdown_connections 2023-05-16T07:19:17.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff21005ed90 0x7ff210061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff21c004690 0x7ff230152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2300c4cf0 0x7ff2300ac830 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 --2- 172.21.15.92:0/3396879153 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff230144b90 0x7ff2300c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 >> 172.21.15.92:0/3396879153 conn(0x7ff230062d80 msgr2=0x7ff2300c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:17.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 shutdown_connections 2023-05-16T07:19:17.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.049+0000 7ff234d20700 1 -- 172.21.15.92:0/3396879153 wait complete. 2023-05-16T07:19:17.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:19:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.171+0000 7fca51375700 1 Processor -- start 2023-05-16T07:19:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.171+0000 7fca51375700 1 -- start start 2023-05-16T07:19:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca51375700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fca4c154550 0x7fca4c0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca51375700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c0c0660 con 0x7fca4c154170 2023-05-16T07:19:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca51375700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c069fc0 con 0x7fca4c069b90 2023-05-16T07:19:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca51375700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c0bfd30 con 0x7fca4c154550 2023-05-16T07:19:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca4affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fca4c154170 0x7fca4c0c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59536/0 (socket says 172.21.15.92:59536) 2023-05-16T07:19:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca4affd700 1 -- 172.21.15.92:0/620936429 learned_addr learned my addr 172.21.15.92:0/620936429 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/620936429 >> v2:172.21.15.158:3300/0 conn(0x7fca4c154550 0x7fca4c0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489415501 0 0) 0x7fca4c0c0660 con 0x7fca4c154170 2023-05-16T07:19:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.172+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca380033f0 con 0x7fca4c154170 2023-05-16T07:19:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544530857 0 0) 0x7fca380033f0 con 0x7fca4c154170 2023-05-16T07:19:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 >> v2:172.21.15.158:3300/0 conn(0x7fca4c154550 msgr2=0x7fca4c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 --2- 172.21.15.92:0/620936429 >> v2:172.21.15.158:3300/0 conn(0x7fca4c154550 0x7fca4c0696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 >> v1:172.21.15.158:6789/0 conn(0x7fca4c069b90 legacy=0x7fca4c0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0c0010 con 0x7fca4c154170 2023-05-16T07:19:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fca3c0017f0 con 0x7fca4c154170 2023-05-16T07:19:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 >> v1:172.21.15.92:6789/0 conn(0x7fca4c154170 legacy=0x7fca4c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca380038e0 0x7fca38003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca38004690 0x7fca38004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca38005050 0x7fca3800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0bfd30 con 0x7fca38004690 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3800bfe0 con 0x7fca380038e0 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3800c1a0 con 0x7fca38005050 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca38004690 0x7fca38004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4affd700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca380038e0 0x7fca38003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4b7fe700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca38005050 0x7fca3800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 -- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca38005050 msgr2=0x7fca3800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca38005050 0x7fca3800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 -- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca380038e0 msgr2=0x7fca38003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca380038e0 0x7fca38003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 -- 172.21.15.92:0/620936429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380033f0 con 0x7fca38004690 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4b7fe700 1 -- 172.21.15.92:0/620936429 reap_dead start 2023-05-16T07:19:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4affd700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca380038e0 0x7fca38003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca38004690 0x7fca38004b10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7fca34006d80 tx=0x7fca34009060).ready entity=mon.0 client_cookie=ada8da9f9ab3ded server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34009e70 con 0x7fca38004690 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400e610 con 0x7fca38004690 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.173+0000 7fca49ffb700 1 -- 172.21.15.92:0/620936429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400e9b0 con 0x7fca38004690 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 -- 172.21.15.92:0/620936429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca38004690 msgr2=0x7fca38004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca38004690 0x7fca38004b10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7fca34006d80 tx=0x7fca34009060).stop 2023-05-16T07:19:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 -- 172.21.15.92:0/620936429 shutdown_connections 2023-05-16T07:19:17.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 --2- 172.21.15.92:0/620936429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca38004690 0x7fca38004b10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 -- 172.21.15.92:0/620936429 >> 172.21.15.92:0/620936429 conn(0x7fca4c0cb700 msgr2=0x7fca4c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:17.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 -- 172.21.15.92:0/620936429 shutdown_connections 2023-05-16T07:19:17.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.174+0000 7fca51375700 1 -- 172.21.15.92:0/620936429 wait complete. 2023-05-16T07:19:17.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.175+0000 7fca51375700 1 Processor -- start 2023-05-16T07:19:17.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.175+0000 7fca51375700 1 -- start start 2023-05-16T07:19:17.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca51375700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca38004690 0x7fca4c14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca51375700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 0x7fca4c0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca51375700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca4c0adc30 0x7fca4c0ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca51375700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c150ec0 con 0x7fca4c0c3da0 2023-05-16T07:19:17.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca51375700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1519d0 con 0x7fca38004690 2023-05-16T07:19:17.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca51375700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0ae5a0 con 0x7fca4c0adc30 2023-05-16T07:19:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 0x7fca4c0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 0x7fca4c0c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48800/0 (socket says 172.21.15.92:48800) 2023-05-16T07:19:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 -- 172.21.15.92:0/4168534535 learned_addr learned my addr 172.21.15.92:0/4168534535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4b7fe700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca4c0adc30 0x7fca4c0ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4affd700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca38004690 0x7fca4c14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 -- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca4c0adc30 msgr2=0x7fca4c0ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca4c0adc30 0x7fca4c0ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 -- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca38004690 msgr2=0x7fca4c14eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca38004690 0x7fca4c14eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 -- 172.21.15.92:0/4168534535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c1510a0 con 0x7fca4c0c3da0 2023-05-16T07:19:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.176+0000 7fca4a7fc700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 0x7fca4c0c4220 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fca3401bfd0 tx=0x7fca3401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.177+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400e780 con 0x7fca4c0c3da0 2023-05-16T07:19:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.177+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400e8e0 con 0x7fca4c0c3da0 2023-05-16T07:19:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.177+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c151380 con 0x7fca4c0c3da0 2023-05-16T07:19:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.177+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34024640 con 0x7fca4c0c3da0 2023-05-16T07:19:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.177+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c0c3160 con 0x7fca4c0c3da0 2023-05-16T07:19:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.178+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c0666c0 con 0x7fca4c0c3da0 2023-05-16T07:19:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.178+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fca3402b030 con 0x7fca4c0c3da0 2023-05-16T07:19:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.178+0000 7fca42ffd700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca2c05ed90 0x7fca2c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.178+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fca3407b030 con 0x7fca4c0c3da0 2023-05-16T07:19:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.178+0000 7fca4affd700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca2c05ed90 0x7fca2c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.179+0000 7fca4affd700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca2c05ed90 0x7fca2c061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fca3c007e10 tx=0x7fca3c009040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.182+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fca3404c6c0 con 0x7fca4c0c3da0 2023-05-16T07:19:17.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.347+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fca4c06c350 con 0x7fca4c0c3da0 2023-05-16T07:19:17.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.347+0000 7fca42ffd700 1 -- 172.21.15.92:0/4168534535 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+143 (secure 0 0 0) 0x7fca34029070 con 0x7fca4c0c3da0 2023-05-16T07:19:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:17.352 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:19:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:19:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:19:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:19:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:19:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:17.354 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:19:17.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca2c05ed90 msgr2=0x7fca2c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca2c05ed90 0x7fca2c061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fca3c007e10 tx=0x7fca3c009040).stop 2023-05-16T07:19:17.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 msgr2=0x7fca4c0c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 0x7fca4c0c4220 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fca3401bfd0 tx=0x7fca3401a000).stop 2023-05-16T07:19:17.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 shutdown_connections 2023-05-16T07:19:17.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fca2c05ed90 0x7fca2c061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fca4c0adc30 0x7fca4c0ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fca38004690 0x7fca4c14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 --2- 172.21.15.92:0/4168534535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4c0c3da0 0x7fca4c0c4220 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 >> 172.21.15.92:0/4168534535 conn(0x7fca4c0cb700 msgr2=0x7fca4c0ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:17.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 shutdown_connections 2023-05-16T07:19:17.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.349+0000 7fca51375700 1 -- 172.21.15.92:0/4168534535 wait complete. 2023-05-16T07:19:17.364 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:19:17.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:19:17.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.477+0000 7f44ae419700 1 Processor -- start 2023-05-16T07:19:17.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.477+0000 7f44ae419700 1 -- start start 2023-05-16T07:19:17.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44ae419700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44a8154090 0x7f44a80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44ae419700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80c94d0 con 0x7f44a8068160 2023-05-16T07:19:17.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44ae419700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80cb7e0 con 0x7f44a8154470 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44ae419700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a80c88c0 con 0x7f44a8154090 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44a7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44a8154090 0x7f44a80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44ac9b6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44a8068160 0x7f44a80c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59558/0 (socket says 172.21.15.92:59558) 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44ac9b6700 1 -- 172.21.15.92:0/2609904982 learned_addr learned my addr 172.21.15.92:0/2609904982 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44a7fff700 1 -- 172.21.15.92:0/2609904982 >> v1:172.21.15.158:6789/0 conn(0x7f44a8154470 legacy=0x7f44a80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44a7fff700 1 -- 172.21.15.92:0/2609904982 >> v1:172.21.15.92:6789/0 conn(0x7f44a8068160 legacy=0x7f44a80c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44a7fff700 1 -- 172.21.15.92:0/2609904982 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80c8ba0 con 0x7f44a8154090 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.478+0000 7f44a7fff700 1 --2- 172.21.15.92:0/2609904982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8154090 0x7f44a80aaca0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f4498009760 tx=0x7f4498009060).ready entity=mon.0 client_cookie=f7cfd12a7a1f061d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44a6ffd700 1 -- 172.21.15.92:0/2609904982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800b520 con 0x7f44a8154090 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44a6ffd700 1 -- 172.21.15.92:0/2609904982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449800b680 con 0x7f44a8154090 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44a6ffd700 1 -- 172.21.15.92:0/2609904982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800ba00 con 0x7f44a8154090 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 -- 172.21.15.92:0/2609904982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8154090 msgr2=0x7f44a80aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 --2- 172.21.15.92:0/2609904982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8154090 0x7f44a80aaca0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f4498009760 tx=0x7f4498009060).stop 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 -- 172.21.15.92:0/2609904982 shutdown_connections 2023-05-16T07:19:17.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 --2- 172.21.15.92:0/2609904982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8154090 0x7f44a80aaca0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 -- 172.21.15.92:0/2609904982 >> 172.21.15.92:0/2609904982 conn(0x7f44a8067920 msgr2=0x7f44a80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:17.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 -- 172.21.15.92:0/2609904982 shutdown_connections 2023-05-16T07:19:17.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.479+0000 7f44ae419700 1 -- 172.21.15.92:0/2609904982 wait complete. 2023-05-16T07:19:17.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.480+0000 7f44ae419700 1 Processor -- start 2023-05-16T07:19:17.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.480+0000 7f44ae419700 1 -- start start 2023-05-16T07:19:17.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ae419700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 0x7f44a8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ae419700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8154090 0x7f44a80b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ae419700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44a8154470 0x7f44a814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ae419700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8150d80 con 0x7f44a8068160 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ae419700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80b0c30 con 0x7f44a8154090 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ae419700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80adff0 con 0x7f44a8154470 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 0x7f44a8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 0x7f44a8150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48812/0 (socket says 172.21.15.92:48812) 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 -- 172.21.15.92:0/3865709176 learned_addr learned my addr 172.21.15.92:0/3865709176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ac9b6700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44a8154470 0x7f44a814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a77fe700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8154090 0x7f44a80b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 -- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44a8154470 msgr2=0x7f44a814cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44a8154470 0x7f44a814cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 -- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8154090 msgr2=0x7f44a80b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8154090 0x7f44a80b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 -- 172.21.15.92:0/3865709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44980092a0 con 0x7f44a8068160 2023-05-16T07:19:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44ac9b6700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44a8154470 0x7f44a814cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:17.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.481+0000 7f44a7fff700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 0x7f44a8150890 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f449800b520 tx=0x7f4498017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.482+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800ba00 con 0x7f44a8068160 2023-05-16T07:19:17.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.482+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498000c90 con 0x7f44a8068160 2023-05-16T07:19:17.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.482+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a8150f90 con 0x7f44a8068160 2023-05-16T07:19:17.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.482+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449801b620 con 0x7f44a8068160 2023-05-16T07:19:17.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.482+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a8151550 con 0x7f44a8068160 2023-05-16T07:19:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.483+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a8066730 con 0x7f44a8068160 2023-05-16T07:19:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.483+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f449802b070 con 0x7f44a8068160 2023-05-16T07:19:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.483+0000 7f448ffff700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f449405edc0 0x7f4494061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.484+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f449807b500 con 0x7f44a8068160 2023-05-16T07:19:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.484+0000 7f44a77fe700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f449405edc0 0x7f4494061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.484+0000 7f44a77fe700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f449405edc0 0x7f4494061280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f44a8151980 tx=0x7f449c008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.487+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f449804cb90 con 0x7f44a8068160 2023-05-16T07:19:17.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.650+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f44a806c380 con 0x7f44a8068160 2023-05-16T07:19:17.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.758+0000 7f448ffff700 1 -- 172.21.15.92:0/3865709176 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 152+0+0 (secure 0 0 0) 0x7f449801f0c0 con 0x7f44a8068160 2023-05-16T07:19:17.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f449405edc0 msgr2=0x7f4494061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f449405edc0 0x7f4494061280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f44a8151980 tx=0x7f449c008040).stop 2023-05-16T07:19:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 msgr2=0x7f44a8150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 0x7f44a8150890 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f449800b520 tx=0x7f4498017000).stop 2023-05-16T07:19:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 shutdown_connections 2023-05-16T07:19:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f449405edc0 0x7f4494061280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f44a8154470 0x7f44a814cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8068160 0x7f44a8150890 secure :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0x7f449800b520 tx=0x7f4498017000).stop 2023-05-16T07:19:17.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 --2- 172.21.15.92:0/3865709176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8154090 0x7f44a80b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.765+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 >> 172.21.15.92:0/3865709176 conn(0x7f44a8067920 msgr2=0x7f44a814dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:17.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.767+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 shutdown_connections 2023-05-16T07:19:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.767+0000 7f44ae419700 1 -- 172.21.15.92:0/3865709176 wait complete. 2023-05-16T07:19:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:19:17.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:19:17.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.897+0000 7eff22999700 1 Processor -- start 2023-05-16T07:19:17.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.897+0000 7eff22999700 1 -- start start 2023-05-16T07:19:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff22999700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff1c153bb0 0x7eff1c0698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff22999700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff22999700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1c05e760 con 0x7eff1c0bd810 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff22999700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c069ef0 con 0x7eff1c153bb0 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff22999700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0bf540 con 0x7eff1c153f90 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff21997700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff1c153bb0 0x7eff1c0698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff21997700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff1c153bb0 0x7eff1c0698c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48820/0 (socket says 172.21.15.92:48820) 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.898+0000 7eff21997700 1 -- 172.21.15.92:0/3514065219 learned_addr learned my addr 172.21.15.92:0/3514065219 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21196700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21997700 1 -- 172.21.15.92:0/3514065219 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 msgr2=0x7eff1c067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21997700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21997700 1 -- 172.21.15.92:0/3514065219 >> v1:172.21.15.158:6789/0 conn(0x7eff1c0bd810 legacy=0x7eff1c0634d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21997700 1 -- 172.21.15.92:0/3514065219 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c068450 con 0x7eff1c153bb0 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21196700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff21997700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c153bb0 0x7eff1c0698c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7eff0c009f20 tx=0x7eff0c009840).ready entity=mon.0 client_cookie=d9fbe76f6a4264d1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff20995700 1 -- 172.21.15.92:0/3514065219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00e070 con 0x7eff1c153bb0 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff20995700 1 -- 172.21.15.92:0/3514065219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c00ba40 con 0x7eff1c153bb0 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff20995700 1 -- 172.21.15.92:0/3514065219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00bdc0 con 0x7eff1c153bb0 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 -- 172.21.15.92:0/3514065219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c153bb0 msgr2=0x7eff1c0698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c153bb0 0x7eff1c0698c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7eff0c009f20 tx=0x7eff0c009840).stop 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 -- 172.21.15.92:0/3514065219 shutdown_connections 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c153bb0 0x7eff1c0698c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 --2- 172.21.15.92:0/3514065219 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 -- 172.21.15.92:0/3514065219 >> 172.21.15.92:0/3514065219 conn(0x7eff1c062d80 msgr2=0x7eff1c05dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.899+0000 7eff22999700 1 -- 172.21.15.92:0/3514065219 shutdown_connections 2023-05-16T07:19:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.900+0000 7eff22999700 1 -- 172.21.15.92:0/3514065219 wait complete. 2023-05-16T07:19:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 Processor -- start 2023-05-16T07:19:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 -- start start 2023-05-16T07:19:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 0x7eff1c147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7eff1c153bb0 0x7eff1c0c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c164790 con 0x7eff1c0bd810 2023-05-16T07:19:17.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c163e80 con 0x7eff1c153bb0 2023-05-16T07:19:17.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff22999700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1478f0 con 0x7eff1c153f90 2023-05-16T07:19:17.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff21997700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 0x7eff1c147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff21997700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 0x7eff1c147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48826/0 (socket says 172.21.15.92:48826) 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.901+0000 7eff21997700 1 -- 172.21.15.92:0/1562547211 learned_addr learned my addr 172.21.15.92:0/1562547211 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff21997700 1 -- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 msgr2=0x7eff1c0ad870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff21997700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c0ad870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff21997700 1 -- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7eff1c153bb0 msgr2=0x7eff1c0c1470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff21997700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7eff1c153bb0 0x7eff1c0c1470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff21997700 1 -- 172.21.15.92:0/1562547211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c009ae0 con 0x7eff1c0bd810 2023-05-16T07:19:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff21997700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 0x7eff1c147400 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7eff0c00b6c0 tx=0x7eff0c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00bdc0 con 0x7eff1c0bd810 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c001010 con 0x7eff1c0bd810 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c0c1dc0 con 0x7eff1c0bd810 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c01d6b0 con 0x7eff1c0bd810 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.902+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c164420 con 0x7eff1c0bd810 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.903+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c047150 con 0x7eff1c0bd810 2023-05-16T07:19:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.904+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7eff0c024030 con 0x7eff1c0bd810 2023-05-16T07:19:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.904+0000 7eff19ffb700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7eff0805ed90 0x7eff08061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.904+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6251+0+0 (secure 0 0 0) 0x7eff0c07b830 con 0x7eff1c0bd810 2023-05-16T07:19:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.904+0000 7eff21196700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7eff0805ed90 0x7eff08061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.905+0000 7eff21196700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7eff0805ed90 0x7eff08061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7eff10006fd0 tx=0x7eff10008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:17.908+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7eff0c04ce80 con 0x7eff1c0bd810 2023-05-16T07:19:18.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.073+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7eff1c158460 con 0x7eff1c0bd810 2023-05-16T07:19:18.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.771+0000 7eff19ffb700 1 -- 172.21.15.92:0/1562547211 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7eff0c02c020 con 0x7eff1c0bd810 2023-05-16T07:19:18.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.783+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7eff0805ed90 msgr2=0x7eff08061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:18.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.783+0000 7eff22999700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7eff0805ed90 0x7eff08061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7eff10006fd0 tx=0x7eff10008040).stop 2023-05-16T07:19:18.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.783+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 msgr2=0x7eff1c147400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:18.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.783+0000 7eff22999700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 0x7eff1c147400 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7eff0c00b6c0 tx=0x7eff0c009840).stop 2023-05-16T07:19:18.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 shutdown_connections 2023-05-16T07:19:18.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7eff0805ed90 0x7eff08061250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7eff1c153f90 0x7eff1c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0bd810 0x7eff1c147400 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 --2- 172.21.15.92:0/1562547211 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7eff1c153bb0 0x7eff1c0c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 >> 172.21.15.92:0/1562547211 conn(0x7eff1c062d80 msgr2=0x7eff1c05dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 shutdown_connections 2023-05-16T07:19:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.785+0000 7eff22999700 1 -- 172.21.15.92:0/1562547211 wait complete. 2023-05-16T07:19:18.793 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:19:18.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:19:18.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.915+0000 7fefc2651700 1 Processor -- start 2023-05-16T07:19:18.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.915+0000 7fefc2651700 1 -- start start 2023-05-16T07:19:18.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc2651700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc154090 0x7fefbc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:18.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc2651700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefbc068160 0x7fefbc068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:18.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc2651700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc0c94d0 con 0x7fefbc154470 2023-05-16T07:19:18.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc2651700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc0cb7e0 con 0x7fefbc068160 2023-05-16T07:19:18.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc2651700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc068a70 con 0x7fefbc154090 2023-05-16T07:19:18.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc0bee700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefbc068160 0x7fefbc068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:18.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefbbfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc154090 0x7fefbc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:18.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefbb7fe700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fefbc154470 0x7fefbc0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:34952/0 (socket says 172.21.15.92:34952) 2023-05-16T07:19:18.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc0bee700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefbc068160 0x7fefbc068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:52474/0 (socket says 172.21.15.92:52474) 2023-05-16T07:19:18.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefbb7fe700 1 -- 172.21.15.92:0/728747740 learned_addr learned my addr 172.21.15.92:0/728747740 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:18.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc0bee700 1 -- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc154090 msgr2=0x7fefbc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:18.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.916+0000 7fefc0bee700 1 --2- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc154090 0x7fefbc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc0bee700 1 -- 172.21.15.92:0/728747740 >> v1:172.21.15.158:6789/0 conn(0x7fefbc154470 legacy=0x7fefbc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:18.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefbaffd700 1 -- 172.21.15.92:0/728747740 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890393897 0 0) 0x7fefbc0c94d0 con 0x7fefbc154470 2023-05-16T07:19:18.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc0bee700 1 -- 172.21.15.92:0/728747740 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0c88c0 con 0x7fefbc068160 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc0bee700 1 --2- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc068160 0x7fefbc068580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fefb4009e60 tx=0x7fefb4009780).ready entity=mon.1 client_cookie=1a1ef625e4e8f56e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefbaffd700 1 -- 172.21.15.92:0/728747740 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400e070 con 0x7fefbc068160 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefbaffd700 1 -- 172.21.15.92:0/728747740 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb400bb60 con 0x7fefbc068160 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefbaffd700 1 -- 172.21.15.92:0/728747740 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4012410 con 0x7fefbc068160 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 -- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc068160 msgr2=0x7fefbc068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 --2- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc068160 0x7fefbc068580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fefb4009e60 tx=0x7fefb4009780).stop 2023-05-16T07:19:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 -- 172.21.15.92:0/728747740 shutdown_connections 2023-05-16T07:19:18.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 --2- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc068160 0x7fefbc068580 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 --2- 172.21.15.92:0/728747740 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc154090 0x7fefbc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 -- 172.21.15.92:0/728747740 >> 172.21.15.92:0/728747740 conn(0x7fefbc067920 msgr2=0x7fefbc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:18.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 -- 172.21.15.92:0/728747740 shutdown_connections 2023-05-16T07:19:18.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.917+0000 7fefc2651700 1 -- 172.21.15.92:0/728747740 wait complete. 2023-05-16T07:19:18.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.918+0000 7fefc2651700 1 Processor -- start 2023-05-16T07:19:18.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 -- start start 2023-05-16T07:19:18.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc068160 0x7fefbc14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:18.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc154090 0x7fefbc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:18.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 0x7fefbc150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:18.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0c3330 con 0x7fefbc154470 2023-05-16T07:19:18.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc151430 con 0x7fefbc154090 2023-05-16T07:19:18.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.919+0000 7fefc2651700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc151710 con 0x7fefbc068160 2023-05-16T07:19:18.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 0x7fefbc150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:18.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 0x7fefbc150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48844/0 (socket says 172.21.15.92:48844) 2023-05-16T07:19:18.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 -- 172.21.15.92:0/2907924464 learned_addr learned my addr 172.21.15.92:0/2907924464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:18.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefbbfff700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc068160 0x7fefbc14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:18.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefbb7fe700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc154090 0x7fefbc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:18.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 -- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc068160 msgr2=0x7fefbc14f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:18.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc068160 0x7fefbc14f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 -- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc154090 msgr2=0x7fefbc150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:18.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc154090 0x7fefbc150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:18.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 -- 172.21.15.92:0/2907924464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4009a20 con 0x7fefbc154470 2023-05-16T07:19:18.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc0bee700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 0x7fefbc150e60 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fefb4009480 tx=0x7fefb4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:18.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb401c660 con 0x7fefbc154470 2023-05-16T07:19:18.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb401c7c0 con 0x7fefbc154470 2023-05-16T07:19:18.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb401cb80 con 0x7fefbc154470 2023-05-16T07:19:18.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc1519f0 con 0x7fefbc154470 2023-05-16T07:19:18.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.920+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc14fe40 con 0x7fefbc154470 2023-05-16T07:19:18.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.921+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc066730 con 0x7fefbc154470 2023-05-16T07:19:18.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.922+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fefb401d420 con 0x7fefbc154470 2023-05-16T07:19:18.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.922+0000 7fefa3fff700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefa805ed90 0x7fefa8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:18.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.922+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6633+0+0 (secure 0 0 0) 0x7fefb407c090 con 0x7fefbc154470 2023-05-16T07:19:18.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.922+0000 7fefbbfff700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefa805ed90 0x7fefa8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:18.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.923+0000 7fefbbfff700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefa805ed90 0x7fefa8061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fefb0009bc0 tx=0x7fefb0008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:18.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:18.926+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fefb404d3d0 con 0x7fefbc154470 2023-05-16T07:19:19.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.131+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fefbc06c380 con 0x7fefbc154470 2023-05-16T07:19:19.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.138+0000 7fefa3fff700 1 -- 172.21.15.92:0/2907924464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v41) v1 ==== 174+0+0 (secure 0 0 0) 0x7fefb40509f0 con 0x7fefbc154470 2023-05-16T07:19:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.145+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefa805ed90 msgr2=0x7fefa8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.145+0000 7fefc2651700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefa805ed90 0x7fefa8061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fefb0009bc0 tx=0x7fefb0008040).stop 2023-05-16T07:19:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.145+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 msgr2=0x7fefbc150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.145+0000 7fefc2651700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 0x7fefbc150e60 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fefb4009480 tx=0x7fefb4009780).stop 2023-05-16T07:19:19.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.146+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 shutdown_connections 2023-05-16T07:19:19.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.146+0000 7fefc2651700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fefa805ed90 0x7fefa8061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.146+0000 7fefc2651700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefbc068160 0x7fefbc14f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.146+0000 7fefc2651700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc154470 0x7fefbc150e60 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.146+0000 7fefc2651700 1 --2- 172.21.15.92:0/2907924464 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefbc154090 0x7fefbc150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.146+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 >> 172.21.15.92:0/2907924464 conn(0x7fefbc067920 msgr2=0x7fefbc0abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:19.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.148+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 shutdown_connections 2023-05-16T07:19:19.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.151+0000 7fefc2651700 1 -- 172.21.15.92:0/2907924464 wait complete. 2023-05-16T07:19:19.161 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 15 and data pool 16 2023-05-16T07:19:19.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.280+0000 7ff31a876700 1 Processor -- start 2023-05-16T07:19:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.280+0000 7ff31a876700 1 -- start start 2023-05-16T07:19:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff31a876700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff314153da0 0x7ff314069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff31a876700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314062b10 con 0x7ff314154180 2023-05-16T07:19:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff31a876700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314068340 con 0x7ff314063e30 2023-05-16T07:19:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff31a876700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff314068620 con 0x7ff314153da0 2023-05-16T07:19:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff313fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff314153da0 0x7ff314069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff313fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff314153da0 0x7ff314069a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48860/0 (socket says 172.21.15.92:48860) 2023-05-16T07:19:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff313fff700 1 -- 172.21.15.92:0/2076577735 learned_addr learned my addr 172.21.15.92:0/2076577735 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.281+0000 7ff3137fe700 1 --1- 172.21.15.92:0/2076577735 >> v1:172.21.15.92:6789/0 conn(0x7ff314154180 0x7ff314063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59612/0 (socket says 172.21.15.92:59612) 2023-05-16T07:19:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff313fff700 1 -- 172.21.15.92:0/2076577735 >> v1:172.21.15.158:6789/0 conn(0x7ff314063e30 legacy=0x7ff3140bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff313fff700 1 -- 172.21.15.92:0/2076577735 >> v1:172.21.15.92:6789/0 conn(0x7ff314154180 legacy=0x7ff314063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff313fff700 1 -- 172.21.15.92:0/2076577735 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140624c0 con 0x7ff314153da0 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff312ffd700 1 -- 172.21.15.92:0/2076577735 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562928125 0 0) 0x7ff314062b10 con 0x7ff314154180 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff313fff700 1 --2- 172.21.15.92:0/2076577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314153da0 0x7ff314069a10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7ff304006fd0 tx=0x7ff3040090f0).ready entity=mon.0 client_cookie=ffe4fff9a732c4e3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff312ffd700 1 -- 172.21.15.92:0/2076577735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400b600 con 0x7ff314153da0 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff312ffd700 1 -- 172.21.15.92:0/2076577735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30400b760 con 0x7ff314153da0 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff312ffd700 1 -- 172.21.15.92:0/2076577735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400bae0 con 0x7ff314153da0 2023-05-16T07:19:19.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff31a876700 1 -- 172.21.15.92:0/2076577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314153da0 msgr2=0x7ff314069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff31a876700 1 --2- 172.21.15.92:0/2076577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314153da0 0x7ff314069a10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7ff304006fd0 tx=0x7ff3040090f0).stop 2023-05-16T07:19:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff31a876700 1 -- 172.21.15.92:0/2076577735 shutdown_connections 2023-05-16T07:19:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff31a876700 1 --2- 172.21.15.92:0/2076577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314153da0 0x7ff314069a10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.282+0000 7ff31a876700 1 -- 172.21.15.92:0/2076577735 >> 172.21.15.92:0/2076577735 conn(0x7ff3140c8aa0 msgr2=0x7ff3140c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.283+0000 7ff31a876700 1 -- 172.21.15.92:0/2076577735 shutdown_connections 2023-05-16T07:19:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.283+0000 7ff31a876700 1 -- 172.21.15.92:0/2076577735 wait complete. 2023-05-16T07:19:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.284+0000 7ff31a876700 1 Processor -- start 2023-05-16T07:19:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.284+0000 7ff31a876700 1 -- start start 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.284+0000 7ff31a876700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 0x7ff314156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff314153da0 0x7ff3141582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff314154180 0x7ff3141587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314158cc0 con 0x7ff314063e30 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140c1e70 con 0x7ff314153da0 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140c2150 con 0x7ff314154180 2023-05-16T07:19:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 0x7ff314156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 0x7ff314156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48866/0 (socket says 172.21.15.92:48866) 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 -- 172.21.15.92:0/2388379167 learned_addr learned my addr 172.21.15.92:0/2388379167 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff3137fe700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff314153da0 0x7ff3141582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff318e13700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff314154180 0x7ff3141587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 -- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff314154180 msgr2=0x7ff3141587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff314154180 0x7ff3141587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 -- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff314153da0 msgr2=0x7ff3141582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff314153da0 0x7ff3141582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 -- 172.21.15.92:0/2388379167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009370 con 0x7ff314063e30 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff313fff700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 0x7ff314156c20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7ff304009130 tx=0x7ff304017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400bae0 con 0x7ff314063e30 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30401c420 con 0x7ff314063e30 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3140c2430 con 0x7ff314063e30 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30401c800 con 0x7ff314063e30 2023-05-16T07:19:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.285+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff314164430 con 0x7ff314063e30 2023-05-16T07:19:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.286+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31406c350 con 0x7ff314063e30 2023-05-16T07:19:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.287+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff304022030 con 0x7ff314063e30 2023-05-16T07:19:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.287+0000 7ff2fbfff700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff30005ed90 0x7ff300061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.287+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6733+0+0 (secure 0 0 0) 0x7ff30407bf80 con 0x7ff314063e30 2023-05-16T07:19:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.287+0000 7ff3137fe700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff30005ed90 0x7ff300061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.288+0000 7ff3137fe700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff30005ed90 0x7ff300061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff3141581e0 tx=0x7ff308007d50).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.291+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff30404d2c0 con 0x7ff314063e30 2023-05-16T07:19:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.456+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3140bf5a0 con 0x7ff314063e30 2023-05-16T07:19:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.457+0000 7ff2fbfff700 1 -- 172.21.15.92:0/2388379167 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17931 (secure 0 0 0) 0x7ff30402a070 con 0x7ff314063e30 2023-05-16T07:19:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:19.462 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":120,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:19.133891+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:17.674878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:18.097566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:19.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff30005ed90 msgr2=0x7ff300061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff30005ed90 0x7ff300061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff3141581e0 tx=0x7ff308007d50).stop 2023-05-16T07:19:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 msgr2=0x7ff314156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 0x7ff314156c20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7ff304009130 tx=0x7ff304017000).stop 2023-05-16T07:19:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 shutdown_connections 2023-05-16T07:19:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff30005ed90 0x7ff300061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff314154180 0x7ff3141587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff314063e30 0x7ff314156c20 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 --2- 172.21.15.92:0/2388379167 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff314153da0 0x7ff3141582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 >> 172.21.15.92:0/2388379167 conn(0x7ff3140c8aa0 msgr2=0x7ff3140ac810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 shutdown_connections 2023-05-16T07:19:19.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.459+0000 7ff31a876700 1 -- 172.21.15.92:0/2388379167 wait complete. 2023-05-16T07:19:19.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:19.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.582+0000 7ff3174fb700 1 Processor -- start 2023-05-16T07:19:19.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.582+0000 7ff3174fb700 1 -- start start 2023-05-16T07:19:19.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff3174fb700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff310154170 0x7ff3100c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff3174fb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff3100c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff3174fb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3100c0660 con 0x7ff3100696a0 2023-05-16T07:19:19.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff3174fb700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3100c1b00 con 0x7ff310154170 2023-05-16T07:19:19.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff3174fb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100bfd30 con 0x7ff310154550 2023-05-16T07:19:19.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff315a98700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff3100696a0 0x7ff310157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59624/0 (socket says 172.21.15.92:59624) 2023-05-16T07:19:19.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff315a98700 1 -- 172.21.15.92:0/1139866343 learned_addr learned my addr 172.21.15.92:0/1139866343 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:19.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff315297700 1 --2- 172.21.15.92:0/1139866343 >> v2:172.21.15.158:3300/0 conn(0x7ff310154170 0x7ff3100c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752472697 0 0) 0x7ff3100c0660 con 0x7ff3100696a0 2023-05-16T07:19:19.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f80033f0 con 0x7ff3100696a0 2023-05-16T07:19:19.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.583+0000 7ff314a96700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff3100c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432157118 0 0) 0x7ff2f80033f0 con 0x7ff3100696a0 2023-05-16T07:19:19.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 msgr2=0x7ff3100c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff3100c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 >> v2:172.21.15.158:3300/0 conn(0x7ff310154170 msgr2=0x7ff3100c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 --2- 172.21.15.92:0/1139866343 >> v2:172.21.15.158:3300/0 conn(0x7ff310154170 0x7ff3100c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100c0010 con 0x7ff3100696a0 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7ff3080017f0 con 0x7ff3100696a0 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 >> v1:172.21.15.92:6789/0 conn(0x7ff3100696a0 legacy=0x7ff310157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f80038e0 0x7ff2f8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2f8004690 0x7ff2f8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff2f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100c1b00 con 0x7ff2f8004690 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100bfd30 con 0x7ff2f80038e0 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800b8e0 con 0x7ff310154550 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2f8004690 0x7ff2f8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff315297700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f80038e0 0x7ff2f8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff315a98700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff2f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 -- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 msgr2=0x7ff2f8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:19.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff2f8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 -- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f80038e0 msgr2=0x7ff2f8003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:19.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f80038e0 0x7ff2f8003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 -- 172.21.15.92:0/1139866343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80033f0 con 0x7ff2f8004690 2023-05-16T07:19:19.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff315a98700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff310154550 0x7ff2f8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:19.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff314a96700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2f8004690 0x7ff2f8004b10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7ff310154930 tx=0x7ff304009060).ready entity=mon.0 client_cookie=e08da483e42490fb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304009430 con 0x7ff2f8004690 2023-05-16T07:19:19.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304009e70 con 0x7ff2f8004690 2023-05-16T07:19:19.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.584+0000 7ff30ffff700 1 -- 172.21.15.92:0/1139866343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400e850 con 0x7ff2f8004690 2023-05-16T07:19:19.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff3174fb700 1 -- 172.21.15.92:0/1139866343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2f8004690 msgr2=0x7ff2f8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1139866343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2f8004690 0x7ff2f8004b10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7ff310154930 tx=0x7ff304009060).stop 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff315a98700 1 -- 172.21.15.92:0/1139866343 reap_dead start 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff3174fb700 1 -- 172.21.15.92:0/1139866343 shutdown_connections 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff3174fb700 1 -- 172.21.15.92:0/1139866343 >> 172.21.15.92:0/1139866343 conn(0x7ff3100cb700 msgr2=0x7ff3100c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff3174fb700 1 -- 172.21.15.92:0/1139866343 shutdown_connections 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.585+0000 7ff3174fb700 1 -- 172.21.15.92:0/1139866343 wait complete. 2023-05-16T07:19:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.586+0000 7ff3174fb700 1 Processor -- start 2023-05-16T07:19:19.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.586+0000 7ff3174fb700 1 -- start start 2023-05-16T07:19:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff3174fb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f8004690 0x7ff31014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff3174fb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 0x7ff310154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff3174fb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff31014d2e0 0x7ff310152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff3174fb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31014e9b0 con 0x7ff3100c2d10 2023-05-16T07:19:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff3174fb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff310151130 con 0x7ff2f8004690 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff3174fb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31014ebc0 con 0x7ff31014d2e0 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 0x7ff310154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 0x7ff310154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48888/0 (socket says 172.21.15.92:48888) 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 -- 172.21.15.92:0/1778454314 learned_addr learned my addr 172.21.15.92:0/1778454314 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff315a98700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff31014d2e0 0x7ff310152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff315297700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f8004690 0x7ff31014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 -- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff31014d2e0 msgr2=0x7ff310152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff31014d2e0 0x7ff310152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 -- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f8004690 msgr2=0x7ff31014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f8004690 0x7ff31014e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 -- 172.21.15.92:0/1778454314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff310150ae0 con 0x7ff3100c2d10 2023-05-16T07:19:19.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.587+0000 7ff314a96700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 0x7ff310154960 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7ff30401b750 tx=0x7ff30401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.588+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304020070 con 0x7ff3100c2d10 2023-05-16T07:19:19.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.588+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30401bca0 con 0x7ff3100c2d10 2023-05-16T07:19:19.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.588+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff310150dc0 con 0x7ff3100c2d10 2023-05-16T07:19:19.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.588+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304009430 con 0x7ff3100c2d10 2023-05-16T07:19:19.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.588+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3100c3190 con 0x7ff3100c2d10 2023-05-16T07:19:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.589+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3100666c0 con 0x7ff3100c2d10 2023-05-16T07:19:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.589+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff304025900 con 0x7ff3100c2d10 2023-05-16T07:19:19.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.589+0000 7ff30d7fa700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff2fc05ed90 0x7ff2fc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.589+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6733+0+0 (secure 0 0 0) 0x7ff30407b5f0 con 0x7ff3100c2d10 2023-05-16T07:19:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.590+0000 7ff315297700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff2fc05ed90 0x7ff2fc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.590+0000 7ff315297700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff2fc05ed90 0x7ff2fc061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff300006d80 tx=0x7ff300008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.593+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff30404c930 con 0x7ff3100c2d10 2023-05-16T07:19:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.755+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3101512c0 con 0x7ff3100c2d10 2023-05-16T07:19:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.756+0000 7ff30d7fa700 1 -- 172.21.15.92:0/1778454314 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17931 (secure 0 0 0) 0x7ff30402a070 con 0x7ff3100c2d10 2023-05-16T07:19:19.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":120,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:19.133891+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:17.674878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:18.097566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff2fc05ed90 msgr2=0x7ff2fc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff2fc05ed90 0x7ff2fc061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff300006d80 tx=0x7ff300008040).stop 2023-05-16T07:19:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 msgr2=0x7ff310154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 0x7ff310154960 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7ff30401b750 tx=0x7ff30401a000).stop 2023-05-16T07:19:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 shutdown_connections 2023-05-16T07:19:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff2fc05ed90 0x7ff2fc061250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff31014d2e0 0x7ff310152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2f8004690 0x7ff31014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 --2- 172.21.15.92:0/1778454314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3100c2d10 0x7ff310154960 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 >> 172.21.15.92:0/1778454314 conn(0x7ff3100cb700 msgr2=0x7ff31014f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 shutdown_connections 2023-05-16T07:19:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.758+0000 7ff3174fb700 1 -- 172.21.15.92:0/1778454314 wait complete. 2023-05-16T07:19:19.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:19:19.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.884+0000 7fc5a6d55700 1 Processor -- start 2023-05-16T07:19:19.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.885+0000 7fc5a6d55700 1 -- start start 2023-05-16T07:19:19.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.885+0000 7fc5a6d55700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc5a0153bb0 0x7fc5a00698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.885+0000 7fc5a6d55700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a00bdf20 0x7fc5a0067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.885+0000 7fc5a6d55700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a005e760 con 0x7fc5a0153f90 2023-05-16T07:19:19.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.885+0000 7fc5a6d55700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a0069ef0 con 0x7fc5a0153bb0 2023-05-16T07:19:19.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.885+0000 7fc5a6d55700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00bf540 con 0x7fc5a00bdf20 2023-05-16T07:19:19.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a6554700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a00bdf20 0x7fc5a0067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5d53700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc5a0153bb0 0x7fc5a00698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5552700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fc5a0153f90 0x7fc5a00bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:35012/0 (socket says 172.21.15.92:35012) 2023-05-16T07:19:19.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5552700 1 -- 172.21.15.92:0/266364362 learned_addr learned my addr 172.21.15.92:0/266364362 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:19.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a4d51700 1 -- 172.21.15.92:0/266364362 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448401911 0 0) 0x7fc5a005e760 con 0x7fc5a0153f90 2023-05-16T07:19:19.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a4d51700 1 -- 172.21.15.92:0/266364362 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5880033f0 con 0x7fc5a0153f90 2023-05-16T07:19:19.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5d53700 1 -- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a00bdf20 msgr2=0x7fc5a0067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:19.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5d53700 1 --2- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a00bdf20 0x7fc5a0067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5d53700 1 -- 172.21.15.92:0/266364362 >> v1:172.21.15.158:6789/0 conn(0x7fc5a0153f90 legacy=0x7fc5a00bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5d53700 1 -- 172.21.15.92:0/266364362 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0068450 con 0x7fc5a0153bb0 2023-05-16T07:19:19.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a6554700 1 --2- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a00bdf20 0x7fc5a0067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:19.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.886+0000 7fc5a5d53700 1 --2- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a0153bb0 0x7fc5a00698c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fc590009e60 tx=0x7fc590009780).ready entity=mon.1 client_cookie=6ad3b9703919a032 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a4d51700 1 -- 172.21.15.92:0/266364362 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000e070 con 0x7fc5a0153bb0 2023-05-16T07:19:19.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a4d51700 1 -- 172.21.15.92:0/266364362 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59000bab0 con 0x7fc5a0153bb0 2023-05-16T07:19:19.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a4d51700 1 -- 172.21.15.92:0/266364362 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000be30 con 0x7fc5a0153bb0 2023-05-16T07:19:19.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 -- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a0153bb0 msgr2=0x7fc5a00698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a0153bb0 0x7fc5a00698c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fc590009e60 tx=0x7fc590009780).stop 2023-05-16T07:19:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 -- 172.21.15.92:0/266364362 shutdown_connections 2023-05-16T07:19:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a0153bb0 0x7fc5a00698c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/266364362 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a00bdf20 0x7fc5a0067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 -- 172.21.15.92:0/266364362 >> 172.21.15.92:0/266364362 conn(0x7fc5a0062d80 msgr2=0x7fc5a005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 -- 172.21.15.92:0/266364362 shutdown_connections 2023-05-16T07:19:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.887+0000 7fc5a6d55700 1 -- 172.21.15.92:0/266364362 wait complete. 2023-05-16T07:19:19.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.888+0000 7fc5a6d55700 1 Processor -- start 2023-05-16T07:19:19.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.888+0000 7fc5a6d55700 1 -- start start 2023-05-16T07:19:19.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6d55700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a00bdf20 0x7fc5a00ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6d55700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a0153bb0 0x7fc5a00c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6d55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 0x7fc5a00c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6d55700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00bf9e0 con 0x7fc5a0153f90 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6d55700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00ace80 con 0x7fc5a00bdf20 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6d55700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00ad160 con 0x7fc5a0153bb0 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 0x7fc5a00c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 0x7fc5a00c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48904/0 (socket says 172.21.15.92:48904) 2023-05-16T07:19:19.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 -- 172.21.15.92:0/126860066 learned_addr learned my addr 172.21.15.92:0/126860066 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:19.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a5d53700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a00bdf20 0x7fc5a00ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a5552700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a0153bb0 0x7fc5a00c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 -- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a0153bb0 msgr2=0x7fc5a00c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a0153bb0 0x7fc5a00c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 -- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a00bdf20 msgr2=0x7fc5a00ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a00bdf20 0x7fc5a00ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 -- 172.21.15.92:0/126860066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5980092a0 con 0x7fc5a0153f90 2023-05-16T07:19:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.889+0000 7fc5a6554700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 0x7fc5a00c4c20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7fc59800e8d0 tx=0x7fc598009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.890+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ee70 con 0x7fc5a0153f90 2023-05-16T07:19:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.890+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59800c7f0 con 0x7fc5a0153f90 2023-05-16T07:19:19.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.890+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc590009a20 con 0x7fc5a0153f90 2023-05-16T07:19:19.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.890+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800cb90 con 0x7fc5a0153f90 2023-05-16T07:19:19.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.890+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a00ad720 con 0x7fc5a0153f90 2023-05-16T07:19:19.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.891+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a0047150 con 0x7fc5a0153f90 2023-05-16T07:19:19.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.891+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc598022070 con 0x7fc5a0153f90 2023-05-16T07:19:19.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.891+0000 7fc59dffb700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc58c05ed90 0x7fc58c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:19.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.891+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fc59807a1b0 con 0x7fc5a0153f90 2023-05-16T07:19:19.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.891+0000 7fc5a5d53700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc58c05ed90 0x7fc58c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:19.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.892+0000 7fc5a5d53700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc58c05ed90 0x7fc58c061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc590009f90 tx=0x7fc590009780).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:19.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:19.895+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc59804b490 con 0x7fc5a0153f90 2023-05-16T07:19:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.099+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5a00bfb20 con 0x7fc5a0153f90 2023-05-16T07:19:20.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.136+0000 7fc59dffb700 1 -- 172.21.15.92:0/126860066 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc598016020 con 0x7fc5a0153f90 2023-05-16T07:19:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc58c05ed90 msgr2=0x7fc58c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc58c05ed90 0x7fc58c061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc590009f90 tx=0x7fc590009780).stop 2023-05-16T07:19:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 msgr2=0x7fc5a00c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 0x7fc5a00c4c20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7fc59800e8d0 tx=0x7fc598009060).stop 2023-05-16T07:19:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 shutdown_connections 2023-05-16T07:19:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc58c05ed90 0x7fc58c061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a0153f90 0x7fc5a00c4c20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc5a00bdf20 0x7fc5a00ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 --2- 172.21.15.92:0/126860066 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc5a0153bb0 0x7fc5a00c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 >> 172.21.15.92:0/126860066 conn(0x7fc5a0062d80 msgr2=0x7fc5a005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 shutdown_connections 2023-05-16T07:19:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.139+0000 7fc5a6d55700 1 -- 172.21.15.92:0/126860066 wait complete. 2023-05-16T07:19:20.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:19:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.268+0000 7f9349c6a700 1 Processor -- start 2023-05-16T07:19:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.268+0000 7f9349c6a700 1 -- start start 2023-05-16T07:19:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.269+0000 7f9349c6a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 0x7f9344150650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.269+0000 7f9349c6a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f93441546f0 0x7f9344063d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.269+0000 7f9349c6a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440c3130 con 0x7f9344154310 2023-05-16T07:19:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.269+0000 7f9349c6a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9344154b20 con 0x7f93441546f0 2023-05-16T07:19:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.269+0000 7f9349c6a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9344068540 con 0x7f934406a0f0 2023-05-16T07:19:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f93441546f0 0x7f9344063d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9342ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 0x7f9344150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f93441546f0 0x7f9344063d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:52548/0 (socket says 172.21.15.92:52548) 2023-05-16T07:19:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f93437fe700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f9344154310 0x7f934414ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:35024/0 (socket says 172.21.15.92:35024) 2023-05-16T07:19:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 -- 172.21.15.92:0/1923819829 learned_addr learned my addr 172.21.15.92:0/1923819829 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9342ffd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 0x7f9344150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:41482/0 (socket says 172.21.15.92:41482) 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 -- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 msgr2=0x7f9344150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 --2- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 0x7f9344150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 -- 172.21.15.92:0/1923819829 >> v1:172.21.15.158:6789/0 conn(0x7f9344154310 legacy=0x7f934414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f93427fc700 1 -- 172.21.15.92:0/1923819829 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452256503 0 0) 0x7f93440c3130 con 0x7f9344154310 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 -- 172.21.15.92:0/1923819829 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344064240 con 0x7f93441546f0 2023-05-16T07:19:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9342ffd700 1 --2- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 0x7f9344150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f9343fff700 1 --2- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93441546f0 0x7f9344063d50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f933c009f20 tx=0x7f933c009840).ready entity=mon.1 client_cookie=914027fc51042506 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f93427fc700 1 -- 172.21.15.92:0/1923819829 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00e070 con 0x7f93441546f0 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f93427fc700 1 -- 172.21.15.92:0/1923819829 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00b9d0 con 0x7f93441546f0 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.270+0000 7f93427fc700 1 -- 172.21.15.92:0/1923819829 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00bd50 con 0x7f93441546f0 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 -- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93441546f0 msgr2=0x7f9344063d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 --2- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93441546f0 0x7f9344063d50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f933c009f20 tx=0x7f933c009840).stop 2023-05-16T07:19:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 -- 172.21.15.92:0/1923819829 shutdown_connections 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 --2- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f93441546f0 0x7f9344063d50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 --2- 172.21.15.92:0/1923819829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f934406a0f0 0x7f9344150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 -- 172.21.15.92:0/1923819829 >> 172.21.15.92:0/1923819829 conn(0x7f93440c3d90 msgr2=0x7f93440c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 -- 172.21.15.92:0/1923819829 shutdown_connections 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.271+0000 7f9349c6a700 1 -- 172.21.15.92:0/1923819829 wait complete. 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.272+0000 7f9349c6a700 1 Processor -- start 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.272+0000 7f9349c6a700 1 -- start start 2023-05-16T07:19:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 0x7f93440aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9344154310 0x7f93440c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f93441546f0 0x7f93440c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440c5fe0 con 0x7f934406a0f0 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440c4630 con 0x7f9344154310 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93440c5170 con 0x7f93441546f0 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 0x7f93440aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 0x7f93440aab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48914/0 (socket says 172.21.15.92:48914) 2023-05-16T07:19:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 -- 172.21.15.92:0/864703406 learned_addr learned my addr 172.21.15.92:0/864703406 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9343fff700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f93441546f0 0x7f93440c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9342ffd700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9344154310 0x7f93440c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 -- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f93441546f0 msgr2=0x7f93440c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f93441546f0 0x7f93440c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 -- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9344154310 msgr2=0x7f93440c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9344154310 0x7f93440c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 -- 172.21.15.92:0/864703406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380092a0 con 0x7f934406a0f0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f93437fe700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 0x7f93440aab80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f933400c560 tx=0x7f9334007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340072e0 con 0x7f934406a0f0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334007440 con 0x7f934406a0f0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c009ae0 con 0x7f934406a0f0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.273+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340058f0 con 0x7f934406a0f0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.274+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440ab070 con 0x7f934406a0f0 2023-05-16T07:19:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.274+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93440c5450 con 0x7f934406a0f0 2023-05-16T07:19:20.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.275+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9334010070 con 0x7f934406a0f0 2023-05-16T07:19:20.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.275+0000 7f932b7fe700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f933005ed90 0x7f9330061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.275+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f9334079660 con 0x7f934406a0f0 2023-05-16T07:19:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.275+0000 7f9342ffd700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f933005ed90 0x7f9330061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.276+0000 7f9342ffd700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f933005ed90 0x7f9330061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f9338009270 tx=0x7f9338008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.279+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f933404a890 con 0x7f934406a0f0 2023-05-16T07:19:20.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:20.481+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f934406c350 con 0x7f934406a0f0 2023-05-16T07:19:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.144+0000 7f932b7fe700 1 -- 172.21.15.92:0/864703406 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7f933404deb0 con 0x7f934406a0f0 2023-05-16T07:19:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f933005ed90 msgr2=0x7f9330061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f933005ed90 0x7f9330061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f9338009270 tx=0x7f9338008040).stop 2023-05-16T07:19:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 msgr2=0x7f93440aab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 0x7f93440aab80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f933400c560 tx=0x7f9334007c90).stop 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 shutdown_connections 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f933005ed90 0x7f9330061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f93441546f0 0x7f93440c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f934406a0f0 0x7f93440aab80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 --2- 172.21.15.92:0/864703406 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9344154310 0x7f93440c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 >> 172.21.15.92:0/864703406 conn(0x7f93440c3d90 msgr2=0x7f9344152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 shutdown_connections 2023-05-16T07:19:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.146+0000 7f9349c6a700 1 -- 172.21.15.92:0/864703406 wait complete. 2023-05-16T07:19:21.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:21.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.272+0000 7f957710c700 1 Processor -- start 2023-05-16T07:19:21.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f957710c700 1 -- start start 2023-05-16T07:19:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f957710c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f95700696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f957710c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9570069b90 0x7f95700c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f957710c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95700c0660 con 0x7f9570154170 2023-05-16T07:19:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f957710c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957006a020 con 0x7f9570069b90 2023-05-16T07:19:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f957710c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95700bfd30 con 0x7f9570154550 2023-05-16T07:19:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f9574ea8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9570154170 0x7f95700c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59670/0 (socket says 172.21.15.92:59670) 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.273+0000 7f9574ea8700 1 -- 172.21.15.92:0/2999877313 learned_addr learned my addr 172.21.15.92:0/2999877313 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372899980 0 0) 0x7f95700c0660 con 0x7f9570154170 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95580033f0 con 0x7f9570154170 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95756a9700 1 --2- 172.21.15.92:0/2999877313 >> v2:172.21.15.158:3300/0 conn(0x7f9570069b90 0x7f95700c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f9567fff700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f95700696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059988190 0 0) 0x7f95580033f0 con 0x7f9570154170 2023-05-16T07:19:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 msgr2=0x7f95700696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f95700696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 >> v2:172.21.15.158:3300/0 conn(0x7f9570069b90 msgr2=0x7f95700c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 --2- 172.21.15.92:0/2999877313 >> v2:172.21.15.158:3300/0 conn(0x7f9570069b90 0x7f95700c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700c0010 con 0x7f9570154170 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f956c0017f0 con 0x7f9570154170 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 >> v1:172.21.15.92:6789/0 conn(0x7f9570154170 legacy=0x7f95700c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95580038e0 0x7f9558003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9558004690 0x7f9558004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f9558005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957006a020 con 0x7f95580038e0 2023-05-16T07:19:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700bfd30 con 0x7f9558004690 2023-05-16T07:19:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955800b8e0 con 0x7f9570154550 2023-05-16T07:19:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f9574ea8700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95580038e0 0x7f9558003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f9567fff700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9558004690 0x7f9558004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f95756a9700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f9558005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f9574ea8700 1 -- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 msgr2=0x7f9558005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.274+0000 7f9574ea8700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f9558005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f9574ea8700 1 -- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9558004690 msgr2=0x7f9558004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f9574ea8700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9558004690 0x7f9558004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f9574ea8700 1 -- 172.21.15.92:0/2999877313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95580033f0 con 0x7f95580038e0 2023-05-16T07:19:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f95756a9700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9570154550 0x7f9558005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f9574ea8700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95580038e0 0x7f9558003d00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f956c00c530 tx=0x7f956c001d90).ready entity=mon.0 client_cookie=2ad9e80364b78ac4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c00ce60 con 0x7f95580038e0 2023-05-16T07:19:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c005b20 con 0x7f95580038e0 2023-05-16T07:19:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f95677fe700 1 -- 172.21.15.92:0/2999877313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0070d0 con 0x7f95580038e0 2023-05-16T07:19:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f957710c700 1 -- 172.21.15.92:0/2999877313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95580038e0 msgr2=0x7f9558003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f957710c700 1 --2- 172.21.15.92:0/2999877313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95580038e0 0x7f9558003d00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f956c00c530 tx=0x7f956c001d90).stop 2023-05-16T07:19:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f95756a9700 1 -- 172.21.15.92:0/2999877313 reap_dead start 2023-05-16T07:19:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f957710c700 1 -- 172.21.15.92:0/2999877313 shutdown_connections 2023-05-16T07:19:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f957710c700 1 -- 172.21.15.92:0/2999877313 >> 172.21.15.92:0/2999877313 conn(0x7f95700cb700 msgr2=0x7f95700c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f957710c700 1 -- 172.21.15.92:0/2999877313 shutdown_connections 2023-05-16T07:19:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.275+0000 7f957710c700 1 -- 172.21.15.92:0/2999877313 wait complete. 2023-05-16T07:19:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.276+0000 7f957710c700 1 Processor -- start 2023-05-16T07:19:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 -- start start 2023-05-16T07:19:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95580038e0 0x7f9570157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 0x7f95700c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95700a9380 0x7f95700a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570168100 con 0x7f9570164af0 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570157b70 con 0x7f95700a9380 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f957710c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570165030 con 0x7f95580038e0 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f9567fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 0x7f95700c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f9567fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 0x7f95700c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48936/0 (socket says 172.21.15.92:48936) 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f9567fff700 1 -- 172.21.15.92:0/217547843 learned_addr learned my addr 172.21.15.92:0/217547843 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.277+0000 7f9574ea8700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95580038e0 0x7f9570157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f95756a9700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95700a9380 0x7f95700a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9567fff700 1 -- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95580038e0 msgr2=0x7f9570157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9567fff700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95580038e0 0x7f9570157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9567fff700 1 -- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95700a9380 msgr2=0x7f95700a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9567fff700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95700a9380 0x7f95700a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9567fff700 1 -- 172.21.15.92:0/217547843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570165310 con 0x7f9570164af0 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9574ea8700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95580038e0 0x7f9570157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f95756a9700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95700a9380 0x7f95700a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9567fff700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 0x7f95700c1d40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f95600119f0 tx=0x7f9560009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000f940 con 0x7f9570164af0 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000faa0 con 0x7f9570164af0 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95701655f0 con 0x7f9570164af0 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000fe20 con 0x7f9570164af0 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.278+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9570159840 con 0x7f9570164af0 2023-05-16T07:19:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.279+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570168240 con 0x7f9570164af0 2023-05-16T07:19:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.279+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9560024070 con 0x7f9570164af0 2023-05-16T07:19:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.280+0000 7f9564ff9700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f955c05ed90 0x7f955c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.280+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f956007a870 con 0x7f9570164af0 2023-05-16T07:19:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.280+0000 7f9574ea8700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f955c05ed90 0x7f955c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.280+0000 7f9574ea8700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f955c05ed90 0x7f955c061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f956c0014a0 tx=0x7f956c015000).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.284+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9570168240 con 0x7f9570164af0 2023-05-16T07:19:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.486+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9570168240 con 0x7f9570164af0 2023-05-16T07:19:21.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.487+0000 7f9564ff9700 1 -- 172.21.15.92:0/217547843 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f9570168240 con 0x7f9570164af0 2023-05-16T07:19:21.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:21.492 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:19.134223+0000","modified":"2023-05-16T07:19:21.140319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7461":{"gid":7461,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1482305851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1482305851},{"type":"v1","addr":"172.21.15.92:6835","nonce":1482305851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-05-16T07:19:21.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f955c05ed90 msgr2=0x7f955c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f955c05ed90 0x7f955c061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f956c0014a0 tx=0x7f956c015000).stop 2023-05-16T07:19:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 msgr2=0x7f95700c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 0x7f95700c1d40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f95600119f0 tx=0x7f9560009060).stop 2023-05-16T07:19:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 shutdown_connections 2023-05-16T07:19:21.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f955c05ed90 0x7f955c061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95580038e0 0x7f9570157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95700a9380 0x7f95700a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 --2- 172.21.15.92:0/217547843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9570164af0 0x7f95700c1d40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 >> 172.21.15.92:0/217547843 conn(0x7f95700cb700 msgr2=0x7f95700c09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 shutdown_connections 2023-05-16T07:19:21.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.489+0000 7f957710c700 1 -- 172.21.15.92:0/217547843 wait complete. 2023-05-16T07:19:21.496 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-05-16T07:19:21.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:21.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.613+0000 7f681de72700 1 Processor -- start 2023-05-16T07:19:21.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.613+0000 7f681de72700 1 -- start start 2023-05-16T07:19:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.614+0000 7f681de72700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 0x7f68180696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.614+0000 7f681de72700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68180c0660 con 0x7f6818069b90 2023-05-16T07:19:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.614+0000 7f681de72700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681806a020 con 0x7f6818154170 2023-05-16T07:19:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.614+0000 7f681de72700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68180bfd30 con 0x7f6818154550 2023-05-16T07:19:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.614+0000 7f6817fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6818069b90 0x7f68180c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59692/0 (socket says 172.21.15.92:59692) 2023-05-16T07:19:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.614+0000 7f6817fff700 1 -- 172.21.15.92:0/3046850424 learned_addr learned my addr 172.21.15.92:0/3046850424 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6816ffd700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 0x7f68180696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512626021 0 0) 0x7f68180c0660 con 0x7f6818069b90 2023-05-16T07:19:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68000033f0 con 0x7f6818069b90 2023-05-16T07:19:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2211193741 0 0) 0x7f68000033f0 con 0x7f6818069b90 2023-05-16T07:19:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 msgr2=0x7f68180696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 0x7f68180696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 >> v1:172.21.15.158:6789/0 conn(0x7f6818154170 legacy=0x7f68180c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180c0010 con 0x7f6818069b90 2023-05-16T07:19:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6810001470 con 0x7f6818069b90 2023-05-16T07:19:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 >> v1:172.21.15.92:6789/0 conn(0x7f6818069b90 legacy=0x7f68180c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 0x7f6800003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6800003e00 0x7f6800004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f6800005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68180bfd30 con 0x7f6800004b80 2023-05-16T07:19:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680000b990 con 0x7f6800003e00 2023-05-16T07:19:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680000bb50 con 0x7f6818154550 2023-05-16T07:19:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f6800005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68177fe700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 0x7f6800003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6816ffd700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6800003e00 0x7f6800004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 -- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 msgr2=0x7f6800003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6818154550 0x7f6800003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 -- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6800003e00 msgr2=0x7f6800004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6800003e00 0x7f6800004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 -- 172.21.15.92:0/3046850424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000033f0 con 0x7f6800004b80 2023-05-16T07:19:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6816ffd700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6800003e00 0x7f6800004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f6817fff700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f6800005000 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f681000c110 tx=0x7f6810001df0).ready entity=mon.0 client_cookie=9c0b9abb0b8ad6a9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.615+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000ca40 con 0x7f6800004b80 2023-05-16T07:19:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681000cba0 con 0x7f6800004b80 2023-05-16T07:19:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f68167fc700 1 -- 172.21.15.92:0/3046850424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100070d0 con 0x7f6800004b80 2023-05-16T07:19:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f681de72700 1 -- 172.21.15.92:0/3046850424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 msgr2=0x7f6800005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f681de72700 1 --2- 172.21.15.92:0/3046850424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f6800005000 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f681000c110 tx=0x7f6810001df0).stop 2023-05-16T07:19:21.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f6817fff700 1 -- 172.21.15.92:0/3046850424 reap_dead start 2023-05-16T07:19:21.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f681de72700 1 -- 172.21.15.92:0/3046850424 shutdown_connections 2023-05-16T07:19:21.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f681de72700 1 -- 172.21.15.92:0/3046850424 >> 172.21.15.92:0/3046850424 conn(0x7f68180cb700 msgr2=0x7f68180c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:21.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f681de72700 1 -- 172.21.15.92:0/3046850424 shutdown_connections 2023-05-16T07:19:21.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.616+0000 7f681de72700 1 -- 172.21.15.92:0/3046850424 wait complete. 2023-05-16T07:19:21.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.617+0000 7f681de72700 1 Processor -- start 2023-05-16T07:19:21.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.617+0000 7f681de72700 1 -- start start 2023-05-16T07:19:21.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f681de72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f68180aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f681de72700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68180af480 0x7f6818152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f681de72700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6818153490 0x7f68181556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f681de72700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68180c2de0 con 0x7f6800004b80 2023-05-16T07:19:21.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f681de72700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6818153910 con 0x7f6818153490 2023-05-16T07:19:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f681de72700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68180c1d40 con 0x7f68180af480 2023-05-16T07:19:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f68180aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f68180aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48954/0 (socket says 172.21.15.92:48954) 2023-05-16T07:19:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 -- 172.21.15.92:0/3759256848 learned_addr learned my addr 172.21.15.92:0/3759256848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 -- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68180af480 msgr2=0x7f6818152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68180af480 0x7f6818152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 -- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6818153490 msgr2=0x7f68181556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f6817fff700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6818153490 0x7f68181556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6818153490 0x7f68181556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.618+0000 7f68177fe700 1 -- 172.21.15.92:0/3759256848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180c2020 con 0x7f6800004b80 2023-05-16T07:19:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.619+0000 7f68177fe700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f68180aef90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f680800ebc0 tx=0x7f6808007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.619+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080040e0 con 0x7f6800004b80 2023-05-16T07:19:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.619+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680800c830 con 0x7f6800004b80 2023-05-16T07:19:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.619+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68180c2300 con 0x7f6800004b80 2023-05-16T07:19:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.619+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800cbf0 con 0x7f6800004b80 2023-05-16T07:19:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.619+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68180c28c0 con 0x7f6800004b80 2023-05-16T07:19:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.620+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68180666c0 con 0x7f6800004b80 2023-05-16T07:19:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.620+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f680801d460 con 0x7f6800004b80 2023-05-16T07:19:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.620+0000 7f67ff7fe700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f680405ed90 0x7f6804061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.620+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f680807a2a0 con 0x7f6800004b80 2023-05-16T07:19:21.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.621+0000 7f6816ffd700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f680405ed90 0x7f6804061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.621+0000 7f6816ffd700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f680405ed90 0x7f6804061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f680c006ac0 tx=0x7f680c008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.624+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f680804b580 con 0x7f6800004b80 2023-05-16T07:19:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.789+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f681806c350 con 0x7f6800004b80 2023-05-16T07:19:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.790+0000 7f67ff7fe700 1 -- 172.21.15.92:0/3759256848 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17918 (secure 0 0 0) 0x7f680804eba0 con 0x7f6800004b80 2023-05-16T07:19:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:21.796 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":121,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:20.135903+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:17.674878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:18.097566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f680405ed90 msgr2=0x7f6804061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f680405ed90 0x7f6804061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f680c006ac0 tx=0x7f680c008040).stop 2023-05-16T07:19:21.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 msgr2=0x7f68180aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f68180aef90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f680800ebc0 tx=0x7f6808007f30).stop 2023-05-16T07:19:21.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 shutdown_connections 2023-05-16T07:19:21.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f680405ed90 0x7f6804061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6818153490 0x7f68181556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6800004b80 0x7f68180aef90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 --2- 172.21.15.92:0/3759256848 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f68180af480 0x7f6818152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:21.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 >> 172.21.15.92:0/3759256848 conn(0x7f68180cb700 msgr2=0x7f68180c0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:21.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.792+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 shutdown_connections 2023-05-16T07:19:21.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.793+0000 7f681de72700 1 -- 172.21.15.92:0/3759256848 wait complete. 2023-05-16T07:19:21.809 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:19:21.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:19:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.916+0000 7fd4f72e6700 1 Processor -- start 2023-05-16T07:19:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.916+0000 7fd4f72e6700 1 -- start start 2023-05-16T07:19:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.917+0000 7fd4f72e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 0x7fd4f015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.917+0000 7fd4f72e6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00c0660 con 0x7fd4f0154550 2023-05-16T07:19:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.917+0000 7fd4f72e6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00c1b00 con 0x7fd4f0154170 2023-05-16T07:19:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.917+0000 7fd4f72e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00bfd30 con 0x7fd4f0069db0 2023-05-16T07:19:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.917+0000 7fd4f4881700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd4f0154550 0x7fd4f00696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59712/0 (socket says 172.21.15.92:59712) 2023-05-16T07:19:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.917+0000 7fd4f4881700 1 -- 172.21.15.92:0/2965999823 learned_addr learned my addr 172.21.15.92:0/2965999823 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f5883700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 0x7fd4f015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408006246 0 0) 0x7fd4f00c0660 con 0x7fd4f0154550 2023-05-16T07:19:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d80033f0 con 0x7fd4f0154550 2023-05-16T07:19:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821200248 0 0) 0x7fd4f00c1b00 con 0x7fd4f0154170 2023-05-16T07:19:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f00c0660 con 0x7fd4f0154170 2023-05-16T07:19:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602297254 0 0) 0x7fd4d80033f0 con 0x7fd4f0154550 2023-05-16T07:19:21.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 msgr2=0x7fd4f015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 0x7fd4f015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 >> v1:172.21.15.158:6789/0 conn(0x7fd4f0154170 legacy=0x7fd4f00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00c0010 con 0x7fd4f0154550 2023-05-16T07:19:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd4e4001470 con 0x7fd4f0154550 2023-05-16T07:19:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 >> v1:172.21.15.92:6789/0 conn(0x7fd4f0154550 legacy=0x7fd4f00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4d8003aa0 0x7fd4d8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4d8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 0x7fd4d8005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00bfd30 con 0x7fd4d8004690 2023-05-16T07:19:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800b990 con 0x7fd4d8003aa0 2023-05-16T07:19:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800bb50 con 0x7fd4f0069db0 2023-05-16T07:19:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4d8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f5082700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4d8003aa0 0x7fd4d8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f5883700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 0x7fd4d8005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 -- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 msgr2=0x7fd4d8005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0069db0 0x7fd4d8005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 -- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4d8003aa0 msgr2=0x7fd4d8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4d8003aa0 0x7fd4d8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 -- 172.21.15.92:0/2965999823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80033f0 con 0x7fd4d8004690 2023-05-16T07:19:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4f4881700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4d8004b10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fd4e400c0e0 tx=0x7fd4e4001e10).ready entity=mon.0 client_cookie=fd8bcc91d41bae6a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.918+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400ca10 con 0x7fd4d8004690 2023-05-16T07:19:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e400cb70 con 0x7fd4d8004690 2023-05-16T07:19:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4effff700 1 -- 172.21.15.92:0/2965999823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40070d0 con 0x7fd4d8004690 2023-05-16T07:19:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f72e6700 1 -- 172.21.15.92:0/2965999823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 msgr2=0x7fd4d8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/2965999823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4d8004b10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fd4e400c0e0 tx=0x7fd4e4001e10).stop 2023-05-16T07:19:21.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f5883700 1 -- 172.21.15.92:0/2965999823 reap_dead start 2023-05-16T07:19:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f72e6700 1 -- 172.21.15.92:0/2965999823 shutdown_connections 2023-05-16T07:19:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f72e6700 1 -- 172.21.15.92:0/2965999823 >> 172.21.15.92:0/2965999823 conn(0x7fd4f00cb700 msgr2=0x7fd4f00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f72e6700 1 -- 172.21.15.92:0/2965999823 shutdown_connections 2023-05-16T07:19:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.919+0000 7fd4f72e6700 1 -- 172.21.15.92:0/2965999823 wait complete. 2023-05-16T07:19:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.920+0000 7fd4f72e6700 1 Processor -- start 2023-05-16T07:19:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.920+0000 7fd4f72e6700 1 -- start start 2023-05-16T07:19:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f72e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4f0168820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f72e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0151210 0x7fd4f0153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f72e6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4f0153770 0x7fd4f00aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f72e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01516d0 con 0x7fd4d8004690 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f72e6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01519b0 con 0x7fd4f0153770 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f72e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0168040 con 0x7fd4f0151210 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4f0168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4f0168820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48976/0 (socket says 172.21.15.92:48976) 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 -- 172.21.15.92:0/3482167419 learned_addr learned my addr 172.21.15.92:0/3482167419 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f4881700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0151210 0x7fd4f0153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5883700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4f0153770 0x7fd4f00aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 -- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0151210 msgr2=0x7fd4f0153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0151210 0x7fd4f0153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 -- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4f0153770 msgr2=0x7fd4f00aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4f0153770 0x7fd4f00aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.921+0000 7fd4f5082700 1 -- 172.21.15.92:0/3482167419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0168320 con 0x7fd4d8004690 2023-05-16T07:19:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.922+0000 7fd4f5082700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4f0168820 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7fd4e000eb90 tx=0x7fd4e0007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.922+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000eea0 con 0x7fd4d8004690 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.922+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000c7e0 con 0x7fd4d8004690 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.922+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f0168d10 con 0x7fd4d8004690 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.922+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cb80 con 0x7fd4d8004690 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.922+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f0151e50 con 0x7fd4d8004690 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.923+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f00666c0 con 0x7fd4d8004690 2023-05-16T07:19:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.923+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd4e001d460 con 0x7fd4d8004690 2023-05-16T07:19:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.923+0000 7fd4ed7fa700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd4dc05ed90 0x7fd4dc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.923+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fd4e007a110 con 0x7fd4d8004690 2023-05-16T07:19:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.924+0000 7fd4f4881700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd4dc05ed90 0x7fd4dc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.924+0000 7fd4f4881700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd4dc05ed90 0x7fd4dc061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd4e4001120 tx=0x7fd4e4015000).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:21.927+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd4e004b3f0 con 0x7fd4d8004690 2023-05-16T07:19:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.086+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4f006c350 con 0x7fd4dc05ed90 2023-05-16T07:19:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.097+0000 7fd4ed7fa700 1 -- 172.21.15.92:0/3482167419 <== mgr.6236 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361465 (secure 0 0 0) 0x7fd4f006c350 con 0x7fd4dc05ed90 2023-05-16T07:19:22.106 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd4dc05ed90 msgr2=0x7fd4dc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd4dc05ed90 0x7fd4dc061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd4e4001120 tx=0x7fd4e4015000).stop 2023-05-16T07:19:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 msgr2=0x7fd4f0168820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4f0168820 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7fd4e000eb90 tx=0x7fd4e0007f30).stop 2023-05-16T07:19:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 shutdown_connections 2023-05-16T07:19:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd4dc05ed90 0x7fd4dc061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4f0153770 0x7fd4f00aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8004690 0x7fd4f0168820 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 --2- 172.21.15.92:0/3482167419 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd4f0151210 0x7fd4f0153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.100+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 >> 172.21.15.92:0/3482167419 conn(0x7fd4f00cb700 msgr2=0x7fd4f00ab890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.101+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 shutdown_connections 2023-05-16T07:19:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.101+0000 7fd4f72e6700 1 -- 172.21.15.92:0/3482167419 wait complete. 2023-05-16T07:19:22.111 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:19:22.117 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:19:20.576626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":169004,"kb_used_data":11500,"kb_used_omap":0,"kb_used_meta":157440,"kb_avail":754805716,"statfs":{"total":773094113280,"available":772921053184,"internally_reserved":0,"allocated":11776000,"data_stored":6149015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":161218560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19801,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.585523"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160631+0000","last_change":"2023-05-16T07:19:19.160631+0000","last_active":"2023-05-16T07:19:19.160631+0000","last_peered":"2023-05-16T07:19:19.160631+0000","last_clean":"2023-05-16T07:19:19.160631+0000","last_became_active":"2023-05-16T07:19:19.160384+0000","last_became_peered":"2023-05-16T07:19:19.160384+0000","last_unstale":"2023-05-16T07:19:19.160631+0000","last_undegraded":"2023-05-16T07:19:19.160631+0000","last_fullsized":"2023-05-16T07:19:19.160631+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3f","version":"120'3","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160574+0000","last_change":"2023-05-16T07:19:18.782301+0000","last_active":"2023-05-16T07:19:19.160574+0000","last_peered":"2023-05-16T07:19:19.160574+0000","last_clean":"2023-05-16T07:19:19.160574+0000","last_became_active":"2023-05-16T07:19:18.782063+0000","last_became_peered":"2023-05-16T07:19:18.782063+0000","last_unstale":"2023-05-16T07:19:19.160574+0000","last_undegraded":"2023-05-16T07:19:19.160574+0000","last_fullsized":"2023-05-16T07:19:19.160574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156268+0000","last_change":"2023-05-16T07:19:19.156268+0000","last_active":"2023-05-16T07:19:19.156268+0000","last_peered":"2023-05-16T07:19:19.156268+0000","last_clean":"2023-05-16T07:19:19.156268+0000","last_became_active":"2023-05-16T07:19:19.156004+0000","last_became_peered":"2023-05-16T07:19:19.156004+0000","last_unstale":"2023-05-16T07:19:19.156268+0000","last_undegraded":"2023-05-16T07:19:19.156268+0000","last_fullsized":"2023-05-16T07:19:19.156268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785162+0000","last_change":"2023-05-16T07:19:18.781986+0000","last_active":"2023-05-16T07:19:18.785162+0000","last_peered":"2023-05-16T07:19:18.785162+0000","last_clean":"2023-05-16T07:19:18.785162+0000","last_became_active":"2023-05-16T07:19:18.781544+0000","last_became_peered":"2023-05-16T07:19:18.781544+0000","last_unstale":"2023-05-16T07:19:18.785162+0000","last_undegraded":"2023-05-16T07:19:18.785162+0000","last_fullsized":"2023-05-16T07:19:18.785162+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155389+0000","last_change":"2023-05-16T07:19:19.155389+0000","last_active":"2023-05-16T07:19:19.155389+0000","last_peered":"2023-05-16T07:19:19.155389+0000","last_clean":"2023-05-16T07:19:19.155389+0000","last_became_active":"2023-05-16T07:19:19.155088+0000","last_became_peered":"2023-05-16T07:19:19.155088+0000","last_unstale":"2023-05-16T07:19:19.155389+0000","last_undegraded":"2023-05-16T07:19:19.155389+0000","last_fullsized":"2023-05-16T07:19:19.155389+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788064+0000","last_change":"2023-05-16T07:19:18.781229+0000","last_active":"2023-05-16T07:19:18.788064+0000","last_peered":"2023-05-16T07:19:18.788064+0000","last_clean":"2023-05-16T07:19:18.788064+0000","last_became_active":"2023-05-16T07:19:18.780926+0000","last_became_peered":"2023-05-16T07:19:18.780926+0000","last_unstale":"2023-05-16T07:19:18.788064+0000","last_undegraded":"2023-05-16T07:19:18.788064+0000","last_fullsized":"2023-05-16T07:19:18.788064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160344+0000","last_change":"2023-05-16T07:19:19.160344+0000","last_active":"2023-05-16T07:19:19.160344+0000","last_peered":"2023-05-16T07:19:19.160344+0000","last_clean":"2023-05-16T07:19:19.160344+0000","last_became_active":"2023-05-16T07:19:19.160072+0000","last_became_peered":"2023-05-16T07:19:19.160072+0000","last_unstale":"2023-05-16T07:19:19.160344+0000","last_undegraded":"2023-05-16T07:19:19.160344+0000","last_fullsized":"2023-05-16T07:19:19.160344+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3c","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162021+0000","last_change":"2023-05-16T07:19:18.786413+0000","last_active":"2023-05-16T07:19:19.162021+0000","last_peered":"2023-05-16T07:19:19.162021+0000","last_clean":"2023-05-16T07:19:19.162021+0000","last_became_active":"2023-05-16T07:19:18.786150+0000","last_became_peered":"2023-05-16T07:19:18.786150+0000","last_unstale":"2023-05-16T07:19:19.162021+0000","last_undegraded":"2023-05-16T07:19:19.162021+0000","last_fullsized":"2023-05-16T07:19:19.162021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.143335+0000","last_change":"2023-05-16T07:19:19.143335+0000","last_active":"2023-05-16T07:19:19.143335+0000","last_peered":"2023-05-16T07:19:19.143335+0000","last_clean":"2023-05-16T07:19:19.143335+0000","last_became_active":"2023-05-16T07:19:19.143087+0000","last_became_peered":"2023-05-16T07:19:19.143087+0000","last_unstale":"2023-05-16T07:19:19.143335+0000","last_undegraded":"2023-05-16T07:19:19.143335+0000","last_fullsized":"2023-05-16T07:19:19.143335+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.138956+0000","last_change":"2023-05-16T07:19:18.778239+0000","last_active":"2023-05-16T07:19:19.138956+0000","last_peered":"2023-05-16T07:19:19.138956+0000","last_clean":"2023-05-16T07:19:19.138956+0000","last_became_active":"2023-05-16T07:19:18.777948+0000","last_became_peered":"2023-05-16T07:19:18.777948+0000","last_unstale":"2023-05-16T07:19:19.138956+0000","last_undegraded":"2023-05-16T07:19:19.138956+0000","last_fullsized":"2023-05-16T07:19:19.138956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154029+0000","last_change":"2023-05-16T07:19:19.154029+0000","last_active":"2023-05-16T07:19:19.154029+0000","last_peered":"2023-05-16T07:19:19.154029+0000","last_clean":"2023-05-16T07:19:19.154029+0000","last_became_active":"2023-05-16T07:19:19.153229+0000","last_became_peered":"2023-05-16T07:19:19.153229+0000","last_unstale":"2023-05-16T07:19:19.154029+0000","last_undegraded":"2023-05-16T07:19:19.154029+0000","last_fullsized":"2023-05-16T07:19:19.154029+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3a","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.145943+0000","last_change":"2023-05-16T07:19:18.782257+0000","last_active":"2023-05-16T07:19:19.145943+0000","last_peered":"2023-05-16T07:19:19.145943+0000","last_clean":"2023-05-16T07:19:19.145943+0000","last_became_active":"2023-05-16T07:19:18.781979+0000","last_became_peered":"2023-05-16T07:19:18.781979+0000","last_unstale":"2023-05-16T07:19:19.145943+0000","last_undegraded":"2023-05-16T07:19:19.145943+0000","last_fullsized":"2023-05-16T07:19:19.145943+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163572+0000","last_change":"2023-05-16T07:19:19.163572+0000","last_active":"2023-05-16T07:19:19.163572+0000","last_peered":"2023-05-16T07:19:19.163572+0000","last_clean":"2023-05-16T07:19:19.163572+0000","last_became_active":"2023-05-16T07:19:19.163287+0000","last_became_peered":"2023-05-16T07:19:19.163287+0000","last_unstale":"2023-05-16T07:19:19.163572+0000","last_undegraded":"2023-05-16T07:19:19.163572+0000","last_fullsized":"2023-05-16T07:19:19.163572+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782744+0000","last_change":"2023-05-16T07:19:18.781649+0000","last_active":"2023-05-16T07:19:18.782744+0000","last_peered":"2023-05-16T07:19:18.782744+0000","last_clean":"2023-05-16T07:19:18.782744+0000","last_became_active":"2023-05-16T07:19:18.781374+0000","last_became_peered":"2023-05-16T07:19:18.781374+0000","last_unstale":"2023-05-16T07:19:18.782744+0000","last_undegraded":"2023-05-16T07:19:18.782744+0000","last_fullsized":"2023-05-16T07:19:18.782744+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154825+0000","last_change":"2023-05-16T07:19:19.154825+0000","last_active":"2023-05-16T07:19:19.154825+0000","last_peered":"2023-05-16T07:19:19.154825+0000","last_clean":"2023-05-16T07:19:19.154825+0000","last_became_active":"2023-05-16T07:19:19.154536+0000","last_became_peered":"2023-05-16T07:19:19.154536+0000","last_unstale":"2023-05-16T07:19:19.154825+0000","last_undegraded":"2023-05-16T07:19:19.154825+0000","last_fullsized":"2023-05-16T07:19:19.154825+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782952+0000","last_change":"2023-05-16T07:19:18.777066+0000","last_active":"2023-05-16T07:19:18.782952+0000","last_peered":"2023-05-16T07:19:18.782952+0000","last_clean":"2023-05-16T07:19:18.782952+0000","last_became_active":"2023-05-16T07:19:18.776803+0000","last_became_peered":"2023-05-16T07:19:18.776803+0000","last_unstale":"2023-05-16T07:19:18.782952+0000","last_undegraded":"2023-05-16T07:19:18.782952+0000","last_fullsized":"2023-05-16T07:19:18.782952+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158474+0000","last_change":"2023-05-16T07:19:19.158474+0000","last_active":"2023-05-16T07:19:19.158474+0000","last_peered":"2023-05-16T07:19:19.158474+0000","last_clean":"2023-05-16T07:19:19.158474+0000","last_became_active":"2023-05-16T07:19:19.158233+0000","last_became_peered":"2023-05-16T07:19:19.158233+0000","last_unstale":"2023-05-16T07:19:19.158474+0000","last_undegraded":"2023-05-16T07:19:19.158474+0000","last_fullsized":"2023-05-16T07:19:19.158474+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789462+0000","last_change":"2023-05-16T07:19:18.787944+0000","last_active":"2023-05-16T07:19:18.789462+0000","last_peered":"2023-05-16T07:19:18.789462+0000","last_clean":"2023-05-16T07:19:18.789462+0000","last_became_active":"2023-05-16T07:19:18.786678+0000","last_became_peered":"2023-05-16T07:19:18.786678+0000","last_unstale":"2023-05-16T07:19:18.789462+0000","last_undegraded":"2023-05-16T07:19:18.789462+0000","last_fullsized":"2023-05-16T07:19:18.789462+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158028+0000","last_change":"2023-05-16T07:19:19.158028+0000","last_active":"2023-05-16T07:19:19.158028+0000","last_peered":"2023-05-16T07:19:19.158028+0000","last_clean":"2023-05-16T07:19:19.158028+0000","last_became_active":"2023-05-16T07:19:19.156920+0000","last_became_peered":"2023-05-16T07:19:19.156920+0000","last_unstale":"2023-05-16T07:19:19.158028+0000","last_undegraded":"2023-05-16T07:19:19.158028+0000","last_fullsized":"2023-05-16T07:19:19.158028+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.786887+0000","last_change":"2023-05-16T07:19:18.782689+0000","last_active":"2023-05-16T07:19:18.786887+0000","last_peered":"2023-05-16T07:19:18.786887+0000","last_clean":"2023-05-16T07:19:18.786887+0000","last_became_active":"2023-05-16T07:19:18.782360+0000","last_became_peered":"2023-05-16T07:19:18.782360+0000","last_unstale":"2023-05-16T07:19:18.786887+0000","last_undegraded":"2023-05-16T07:19:18.786887+0000","last_fullsized":"2023-05-16T07:19:18.786887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150377+0000","last_change":"2023-05-16T07:19:19.150377+0000","last_active":"2023-05-16T07:19:19.150377+0000","last_peered":"2023-05-16T07:19:19.150377+0000","last_clean":"2023-05-16T07:19:19.150377+0000","last_became_active":"2023-05-16T07:19:19.147947+0000","last_became_peered":"2023-05-16T07:19:19.147947+0000","last_unstale":"2023-05-16T07:19:19.150377+0000","last_undegraded":"2023-05-16T07:19:19.150377+0000","last_fullsized":"2023-05-16T07:19:19.150377+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.786833+0000","last_change":"2023-05-16T07:19:18.785233+0000","last_active":"2023-05-16T07:19:18.786833+0000","last_peered":"2023-05-16T07:19:18.786833+0000","last_clean":"2023-05-16T07:19:18.786833+0000","last_became_active":"2023-05-16T07:19:18.784906+0000","last_became_peered":"2023-05-16T07:19:18.784906+0000","last_unstale":"2023-05-16T07:19:18.786833+0000","last_undegraded":"2023-05-16T07:19:18.786833+0000","last_fullsized":"2023-05-16T07:19:18.786833+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.161966+0000","last_change":"2023-05-16T07:19:19.161966+0000","last_active":"2023-05-16T07:19:19.161966+0000","last_peered":"2023-05-16T07:19:19.161966+0000","last_clean":"2023-05-16T07:19:19.161966+0000","last_became_active":"2023-05-16T07:19:19.161677+0000","last_became_peered":"2023-05-16T07:19:19.161677+0000","last_unstale":"2023-05-16T07:19:19.161966+0000","last_undegraded":"2023-05-16T07:19:19.161966+0000","last_fullsized":"2023-05-16T07:19:19.161966+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.34","version":"120'4","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.166911+0000","last_change":"2023-05-16T07:19:18.784263+0000","last_active":"2023-05-16T07:19:19.166911+0000","last_peered":"2023-05-16T07:19:19.166911+0000","last_clean":"2023-05-16T07:19:19.166911+0000","last_became_active":"2023-05-16T07:19:18.783977+0000","last_became_peered":"2023-05-16T07:19:18.783977+0000","last_unstale":"2023-05-16T07:19:19.166911+0000","last_undegraded":"2023-05-16T07:19:19.166911+0000","last_fullsized":"2023-05-16T07:19:19.166911+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155157+0000","last_change":"2023-05-16T07:19:19.155157+0000","last_active":"2023-05-16T07:19:19.155157+0000","last_peered":"2023-05-16T07:19:19.155157+0000","last_clean":"2023-05-16T07:19:19.155157+0000","last_became_active":"2023-05-16T07:19:19.154884+0000","last_became_peered":"2023-05-16T07:19:19.154884+0000","last_unstale":"2023-05-16T07:19:19.155157+0000","last_undegraded":"2023-05-16T07:19:19.155157+0000","last_fullsized":"2023-05-16T07:19:19.155157+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.147016+0000","last_change":"2023-05-16T07:19:18.784767+0000","last_active":"2023-05-16T07:19:19.147016+0000","last_peered":"2023-05-16T07:19:19.147016+0000","last_clean":"2023-05-16T07:19:19.147016+0000","last_became_active":"2023-05-16T07:19:18.784496+0000","last_became_peered":"2023-05-16T07:19:18.784496+0000","last_unstale":"2023-05-16T07:19:19.147016+0000","last_undegraded":"2023-05-16T07:19:19.147016+0000","last_fullsized":"2023-05-16T07:19:19.147016+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162896+0000","last_change":"2023-05-16T07:19:19.162896+0000","last_active":"2023-05-16T07:19:19.162896+0000","last_peered":"2023-05-16T07:19:19.162896+0000","last_clean":"2023-05-16T07:19:19.162896+0000","last_became_active":"2023-05-16T07:19:19.162627+0000","last_became_peered":"2023-05-16T07:19:19.162627+0000","last_unstale":"2023-05-16T07:19:19.162896+0000","last_undegraded":"2023-05-16T07:19:19.162896+0000","last_fullsized":"2023-05-16T07:19:19.162896+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785787+0000","last_change":"2023-05-16T07:19:18.775367+0000","last_active":"2023-05-16T07:19:18.785787+0000","last_peered":"2023-05-16T07:19:18.785787+0000","last_clean":"2023-05-16T07:19:18.785787+0000","last_became_active":"2023-05-16T07:19:18.775120+0000","last_became_peered":"2023-05-16T07:19:18.775120+0000","last_unstale":"2023-05-16T07:19:18.785787+0000","last_undegraded":"2023-05-16T07:19:18.785787+0000","last_fullsized":"2023-05-16T07:19:18.785787+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156980+0000","last_change":"2023-05-16T07:19:19.156980+0000","last_active":"2023-05-16T07:19:19.156980+0000","last_peered":"2023-05-16T07:19:19.156980+0000","last_clean":"2023-05-16T07:19:19.156980+0000","last_became_active":"2023-05-16T07:19:19.155806+0000","last_became_peered":"2023-05-16T07:19:19.155806+0000","last_unstale":"2023-05-16T07:19:19.156980+0000","last_undegraded":"2023-05-16T07:19:19.156980+0000","last_fullsized":"2023-05-16T07:19:19.156980+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.793085+0000","last_change":"2023-05-16T07:19:18.791619+0000","last_active":"2023-05-16T07:19:18.793085+0000","last_peered":"2023-05-16T07:19:18.793085+0000","last_clean":"2023-05-16T07:19:18.793085+0000","last_became_active":"2023-05-16T07:19:18.791283+0000","last_became_peered":"2023-05-16T07:19:18.791283+0000","last_unstale":"2023-05-16T07:19:18.793085+0000","last_undegraded":"2023-05-16T07:19:18.793085+0000","last_fullsized":"2023-05-16T07:19:18.793085+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154166+0000","last_change":"2023-05-16T07:19:19.154166+0000","last_active":"2023-05-16T07:19:19.154166+0000","last_peered":"2023-05-16T07:19:19.154166+0000","last_clean":"2023-05-16T07:19:19.154166+0000","last_became_active":"2023-05-16T07:19:19.153660+0000","last_became_peered":"2023-05-16T07:19:19.153660+0000","last_unstale":"2023-05-16T07:19:19.154166+0000","last_undegraded":"2023-05-16T07:19:19.154166+0000","last_fullsized":"2023-05-16T07:19:19.154166+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"120'4","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.157632+0000","last_change":"2023-05-16T07:19:18.778671+0000","last_active":"2023-05-16T07:19:19.157632+0000","last_peered":"2023-05-16T07:19:19.157632+0000","last_clean":"2023-05-16T07:19:19.157632+0000","last_became_active":"2023-05-16T07:19:18.778177+0000","last_became_peered":"2023-05-16T07:19:18.778177+0000","last_unstale":"2023-05-16T07:19:19.157632+0000","last_undegraded":"2023-05-16T07:19:19.157632+0000","last_fullsized":"2023-05-16T07:19:19.157632+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.157155+0000","last_change":"2023-05-16T07:19:19.157155+0000","last_active":"2023-05-16T07:19:19.157155+0000","last_peered":"2023-05-16T07:19:19.157155+0000","last_clean":"2023-05-16T07:19:19.157155+0000","last_became_active":"2023-05-16T07:19:19.156870+0000","last_became_peered":"2023-05-16T07:19:19.156870+0000","last_unstale":"2023-05-16T07:19:19.157155+0000","last_undegraded":"2023-05-16T07:19:19.157155+0000","last_fullsized":"2023-05-16T07:19:19.157155+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.791000+0000","last_change":"2023-05-16T07:19:18.787300+0000","last_active":"2023-05-16T07:19:18.791000+0000","last_peered":"2023-05-16T07:19:18.791000+0000","last_clean":"2023-05-16T07:19:18.791000+0000","last_became_active":"2023-05-16T07:19:18.786965+0000","last_became_peered":"2023-05-16T07:19:18.786965+0000","last_unstale":"2023-05-16T07:19:18.791000+0000","last_undegraded":"2023-05-16T07:19:18.791000+0000","last_fullsized":"2023-05-16T07:19:18.791000+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156626+0000","last_change":"2023-05-16T07:19:19.156626+0000","last_active":"2023-05-16T07:19:19.156626+0000","last_peered":"2023-05-16T07:19:19.156626+0000","last_clean":"2023-05-16T07:19:19.156626+0000","last_became_active":"2023-05-16T07:19:19.156350+0000","last_became_peered":"2023-05-16T07:19:19.156350+0000","last_unstale":"2023-05-16T07:19:19.156626+0000","last_undegraded":"2023-05-16T07:19:19.156626+0000","last_fullsized":"2023-05-16T07:19:19.156626+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789644+0000","last_change":"2023-05-16T07:19:18.786358+0000","last_active":"2023-05-16T07:19:18.789644+0000","last_peered":"2023-05-16T07:19:18.789644+0000","last_clean":"2023-05-16T07:19:18.789644+0000","last_became_active":"2023-05-16T07:19:18.786158+0000","last_became_peered":"2023-05-16T07:19:18.786158+0000","last_unstale":"2023-05-16T07:19:18.789644+0000","last_undegraded":"2023-05-16T07:19:18.789644+0000","last_fullsized":"2023-05-16T07:19:18.789644+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151693+0000","last_change":"2023-05-16T07:19:19.151693+0000","last_active":"2023-05-16T07:19:19.151693+0000","last_peered":"2023-05-16T07:19:19.151693+0000","last_clean":"2023-05-16T07:19:19.151693+0000","last_became_active":"2023-05-16T07:19:19.151384+0000","last_became_peered":"2023-05-16T07:19:19.151384+0000","last_unstale":"2023-05-16T07:19:19.151693+0000","last_undegraded":"2023-05-16T07:19:19.151693+0000","last_fullsized":"2023-05-16T07:19:19.151693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2d","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.149215+0000","last_change":"2023-05-16T07:19:18.782930+0000","last_active":"2023-05-16T07:19:19.149215+0000","last_peered":"2023-05-16T07:19:19.149215+0000","last_clean":"2023-05-16T07:19:19.149215+0000","last_became_active":"2023-05-16T07:19:18.782642+0000","last_became_peered":"2023-05-16T07:19:18.782642+0000","last_unstale":"2023-05-16T07:19:19.149215+0000","last_undegraded":"2023-05-16T07:19:19.149215+0000","last_fullsized":"2023-05-16T07:19:19.149215+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151297+0000","last_change":"2023-05-16T07:19:19.151297+0000","last_active":"2023-05-16T07:19:19.151297+0000","last_peered":"2023-05-16T07:19:19.151297+0000","last_clean":"2023-05-16T07:19:19.151297+0000","last_became_active":"2023-05-16T07:19:19.151054+0000","last_became_peered":"2023-05-16T07:19:19.151054+0000","last_unstale":"2023-05-16T07:19:19.151297+0000","last_undegraded":"2023-05-16T07:19:19.151297+0000","last_fullsized":"2023-05-16T07:19:19.151297+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789440+0000","last_change":"2023-05-16T07:19:18.786711+0000","last_active":"2023-05-16T07:19:18.789440+0000","last_peered":"2023-05-16T07:19:18.789440+0000","last_clean":"2023-05-16T07:19:18.789440+0000","last_became_active":"2023-05-16T07:19:18.786313+0000","last_became_peered":"2023-05-16T07:19:18.786313+0000","last_unstale":"2023-05-16T07:19:18.789440+0000","last_undegraded":"2023-05-16T07:19:18.789440+0000","last_fullsized":"2023-05-16T07:19:18.789440+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156954+0000","last_change":"2023-05-16T07:19:19.156954+0000","last_active":"2023-05-16T07:19:19.156954+0000","last_peered":"2023-05-16T07:19:19.156954+0000","last_clean":"2023-05-16T07:19:19.156954+0000","last_became_active":"2023-05-16T07:19:19.156586+0000","last_became_peered":"2023-05-16T07:19:19.156586+0000","last_unstale":"2023-05-16T07:19:19.156954+0000","last_undegraded":"2023-05-16T07:19:19.156954+0000","last_fullsized":"2023-05-16T07:19:19.156954+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155920+0000","last_change":"2023-05-16T07:19:18.780770+0000","last_active":"2023-05-16T07:19:19.155920+0000","last_peered":"2023-05-16T07:19:19.155920+0000","last_clean":"2023-05-16T07:19:19.155920+0000","last_became_active":"2023-05-16T07:19:18.780297+0000","last_became_peered":"2023-05-16T07:19:18.780297+0000","last_unstale":"2023-05-16T07:19:19.155920+0000","last_undegraded":"2023-05-16T07:19:19.155920+0000","last_fullsized":"2023-05-16T07:19:19.155920+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.144377+0000","last_change":"2023-05-16T07:19:19.144377+0000","last_active":"2023-05-16T07:19:19.144377+0000","last_peered":"2023-05-16T07:19:19.144377+0000","last_clean":"2023-05-16T07:19:19.144377+0000","last_became_active":"2023-05-16T07:19:19.143744+0000","last_became_peered":"2023-05-16T07:19:19.143744+0000","last_unstale":"2023-05-16T07:19:19.144377+0000","last_undegraded":"2023-05-16T07:19:19.144377+0000","last_fullsized":"2023-05-16T07:19:19.144377+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785096+0000","last_change":"2023-05-16T07:19:18.781983+0000","last_active":"2023-05-16T07:19:18.785096+0000","last_peered":"2023-05-16T07:19:18.785096+0000","last_clean":"2023-05-16T07:19:18.785096+0000","last_became_active":"2023-05-16T07:19:18.781154+0000","last_became_peered":"2023-05-16T07:19:18.781154+0000","last_unstale":"2023-05-16T07:19:18.785096+0000","last_undegraded":"2023-05-16T07:19:18.785096+0000","last_fullsized":"2023-05-16T07:19:18.785096+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163919+0000","last_change":"2023-05-16T07:19:19.163919+0000","last_active":"2023-05-16T07:19:19.163919+0000","last_peered":"2023-05-16T07:19:19.163919+0000","last_clean":"2023-05-16T07:19:19.163919+0000","last_became_active":"2023-05-16T07:19:19.163660+0000","last_became_peered":"2023-05-16T07:19:19.163660+0000","last_unstale":"2023-05-16T07:19:19.163919+0000","last_undegraded":"2023-05-16T07:19:19.163919+0000","last_fullsized":"2023-05-16T07:19:19.163919+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785118+0000","last_change":"2023-05-16T07:19:18.781673+0000","last_active":"2023-05-16T07:19:18.785118+0000","last_peered":"2023-05-16T07:19:18.785118+0000","last_clean":"2023-05-16T07:19:18.785118+0000","last_became_active":"2023-05-16T07:19:18.781407+0000","last_became_peered":"2023-05-16T07:19:18.781407+0000","last_unstale":"2023-05-16T07:19:18.785118+0000","last_undegraded":"2023-05-16T07:19:18.785118+0000","last_fullsized":"2023-05-16T07:19:18.785118+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.159888+0000","last_change":"2023-05-16T07:19:19.159888+0000","last_active":"2023-05-16T07:19:19.159888+0000","last_peered":"2023-05-16T07:19:19.159888+0000","last_clean":"2023-05-16T07:19:19.159888+0000","last_became_active":"2023-05-16T07:19:19.159612+0000","last_became_peered":"2023-05-16T07:19:19.159612+0000","last_unstale":"2023-05-16T07:19:19.159888+0000","last_undegraded":"2023-05-16T07:19:19.159888+0000","last_fullsized":"2023-05-16T07:19:19.159888+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785856+0000","last_change":"2023-05-16T07:19:18.783986+0000","last_active":"2023-05-16T07:19:18.785856+0000","last_peered":"2023-05-16T07:19:18.785856+0000","last_clean":"2023-05-16T07:19:18.785856+0000","last_became_active":"2023-05-16T07:19:18.783715+0000","last_became_peered":"2023-05-16T07:19:18.783715+0000","last_unstale":"2023-05-16T07:19:18.785856+0000","last_undegraded":"2023-05-16T07:19:18.785856+0000","last_fullsized":"2023-05-16T07:19:18.785856+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163156+0000","last_change":"2023-05-16T07:19:19.163156+0000","last_active":"2023-05-16T07:19:19.163156+0000","last_peered":"2023-05-16T07:19:19.163156+0000","last_clean":"2023-05-16T07:19:19.163156+0000","last_became_active":"2023-05-16T07:19:19.162908+0000","last_became_peered":"2023-05-16T07:19:19.162908+0000","last_unstale":"2023-05-16T07:19:19.163156+0000","last_undegraded":"2023-05-16T07:19:19.163156+0000","last_fullsized":"2023-05-16T07:19:19.163156+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.786995+0000","last_change":"2023-05-16T07:19:18.784135+0000","last_active":"2023-05-16T07:19:18.786995+0000","last_peered":"2023-05-16T07:19:18.786995+0000","last_clean":"2023-05-16T07:19:18.786995+0000","last_became_active":"2023-05-16T07:19:18.783826+0000","last_became_peered":"2023-05-16T07:19:18.783826+0000","last_unstale":"2023-05-16T07:19:18.786995+0000","last_undegraded":"2023-05-16T07:19:18.786995+0000","last_fullsized":"2023-05-16T07:19:18.786995+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155467+0000","last_change":"2023-05-16T07:19:19.155467+0000","last_active":"2023-05-16T07:19:19.155467+0000","last_peered":"2023-05-16T07:19:19.155467+0000","last_clean":"2023-05-16T07:19:19.155467+0000","last_became_active":"2023-05-16T07:19:19.154352+0000","last_became_peered":"2023-05-16T07:19:19.154352+0000","last_unstale":"2023-05-16T07:19:19.155467+0000","last_undegraded":"2023-05-16T07:19:19.155467+0000","last_fullsized":"2023-05-16T07:19:19.155467+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.796515+0000","last_change":"2023-05-16T07:19:18.792552+0000","last_active":"2023-05-16T07:19:18.796515+0000","last_peered":"2023-05-16T07:19:18.796515+0000","last_clean":"2023-05-16T07:19:18.796515+0000","last_became_active":"2023-05-16T07:19:18.792307+0000","last_became_peered":"2023-05-16T07:19:18.792307+0000","last_unstale":"2023-05-16T07:19:18.796515+0000","last_undegraded":"2023-05-16T07:19:18.796515+0000","last_fullsized":"2023-05-16T07:19:18.796515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156553+0000","last_change":"2023-05-16T07:19:19.156553+0000","last_active":"2023-05-16T07:19:19.156553+0000","last_peered":"2023-05-16T07:19:19.156553+0000","last_clean":"2023-05-16T07:19:19.156553+0000","last_became_active":"2023-05-16T07:19:19.155443+0000","last_became_peered":"2023-05-16T07:19:19.155443+0000","last_unstale":"2023-05-16T07:19:19.156553+0000","last_undegraded":"2023-05-16T07:19:19.156553+0000","last_fullsized":"2023-05-16T07:19:19.156553+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.772684+0000","last_change":"2023-05-16T07:19:18.771682+0000","last_active":"2023-05-16T07:19:18.772684+0000","last_peered":"2023-05-16T07:19:18.772684+0000","last_clean":"2023-05-16T07:19:18.772684+0000","last_became_active":"2023-05-16T07:19:18.771355+0000","last_became_peered":"2023-05-16T07:19:18.771355+0000","last_unstale":"2023-05-16T07:19:18.772684+0000","last_undegraded":"2023-05-16T07:19:18.772684+0000","last_fullsized":"2023-05-16T07:19:18.772684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156376+0000","last_change":"2023-05-16T07:19:19.156376+0000","last_active":"2023-05-16T07:19:19.156376+0000","last_peered":"2023-05-16T07:19:19.156376+0000","last_clean":"2023-05-16T07:19:19.156376+0000","last_became_active":"2023-05-16T07:19:19.156089+0000","last_became_peered":"2023-05-16T07:19:19.156089+0000","last_unstale":"2023-05-16T07:19:19.156376+0000","last_undegraded":"2023-05-16T07:19:19.156376+0000","last_fullsized":"2023-05-16T07:19:19.156376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.787284+0000","last_change":"2023-05-16T07:19:18.786458+0000","last_active":"2023-05-16T07:19:18.787284+0000","last_peered":"2023-05-16T07:19:18.787284+0000","last_clean":"2023-05-16T07:19:18.787284+0000","last_became_active":"2023-05-16T07:19:18.786185+0000","last_became_peered":"2023-05-16T07:19:18.786185+0000","last_unstale":"2023-05-16T07:19:18.787284+0000","last_undegraded":"2023-05-16T07:19:18.787284+0000","last_fullsized":"2023-05-16T07:19:18.787284+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156922+0000","last_change":"2023-05-16T07:19:19.156922+0000","last_active":"2023-05-16T07:19:19.156922+0000","last_peered":"2023-05-16T07:19:19.156922+0000","last_clean":"2023-05-16T07:19:19.156922+0000","last_became_active":"2023-05-16T07:19:19.156658+0000","last_became_peered":"2023-05-16T07:19:19.156658+0000","last_unstale":"2023-05-16T07:19:19.156922+0000","last_undegraded":"2023-05-16T07:19:19.156922+0000","last_fullsized":"2023-05-16T07:19:19.156922+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.796823+0000","last_change":"2023-05-16T07:19:18.794426+0000","last_active":"2023-05-16T07:19:18.796823+0000","last_peered":"2023-05-16T07:19:18.796823+0000","last_clean":"2023-05-16T07:19:18.796823+0000","last_became_active":"2023-05-16T07:19:18.794165+0000","last_became_peered":"2023-05-16T07:19:18.794165+0000","last_unstale":"2023-05-16T07:19:18.796823+0000","last_undegraded":"2023-05-16T07:19:18.796823+0000","last_fullsized":"2023-05-16T07:19:18.796823+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158694+0000","last_change":"2023-05-16T07:19:19.158694+0000","last_active":"2023-05-16T07:19:19.158694+0000","last_peered":"2023-05-16T07:19:19.158694+0000","last_clean":"2023-05-16T07:19:19.158694+0000","last_became_active":"2023-05-16T07:19:19.158445+0000","last_became_peered":"2023-05-16T07:19:19.158445+0000","last_unstale":"2023-05-16T07:19:19.158694+0000","last_undegraded":"2023-05-16T07:19:19.158694+0000","last_fullsized":"2023-05-16T07:19:19.158694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.793943+0000","last_change":"2023-05-16T07:19:18.789342+0000","last_active":"2023-05-16T07:19:18.793943+0000","last_peered":"2023-05-16T07:19:18.793943+0000","last_clean":"2023-05-16T07:19:18.793943+0000","last_became_active":"2023-05-16T07:19:18.789071+0000","last_became_peered":"2023-05-16T07:19:18.789071+0000","last_unstale":"2023-05-16T07:19:18.793943+0000","last_undegraded":"2023-05-16T07:19:18.793943+0000","last_fullsized":"2023-05-16T07:19:18.793943+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158198+0000","last_change":"2023-05-16T07:19:19.158198+0000","last_active":"2023-05-16T07:19:19.158198+0000","last_peered":"2023-05-16T07:19:19.158198+0000","last_clean":"2023-05-16T07:19:19.158198+0000","last_became_active":"2023-05-16T07:19:19.157914+0000","last_became_peered":"2023-05-16T07:19:19.157914+0000","last_unstale":"2023-05-16T07:19:19.158198+0000","last_undegraded":"2023-05-16T07:19:19.158198+0000","last_fullsized":"2023-05-16T07:19:19.158198+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788294+0000","last_change":"2023-05-16T07:19:18.780567+0000","last_active":"2023-05-16T07:19:18.788294+0000","last_peered":"2023-05-16T07:19:18.788294+0000","last_clean":"2023-05-16T07:19:18.788294+0000","last_became_active":"2023-05-16T07:19:18.780313+0000","last_became_peered":"2023-05-16T07:19:18.780313+0000","last_unstale":"2023-05-16T07:19:18.788294+0000","last_undegraded":"2023-05-16T07:19:18.788294+0000","last_fullsized":"2023-05-16T07:19:18.788294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154324+0000","last_change":"2023-05-16T07:19:19.154324+0000","last_active":"2023-05-16T07:19:19.154324+0000","last_peered":"2023-05-16T07:19:19.154324+0000","last_clean":"2023-05-16T07:19:19.154324+0000","last_became_active":"2023-05-16T07:19:19.153951+0000","last_became_peered":"2023-05-16T07:19:19.153951+0000","last_unstale":"2023-05-16T07:19:19.154324+0000","last_undegraded":"2023-05-16T07:19:19.154324+0000","last_fullsized":"2023-05-16T07:19:19.154324+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.794010+0000","last_change":"2023-05-16T07:19:18.788992+0000","last_active":"2023-05-16T07:19:18.794010+0000","last_peered":"2023-05-16T07:19:18.794010+0000","last_clean":"2023-05-16T07:19:18.794010+0000","last_became_active":"2023-05-16T07:19:18.788602+0000","last_became_peered":"2023-05-16T07:19:18.788602+0000","last_unstale":"2023-05-16T07:19:18.794010+0000","last_undegraded":"2023-05-16T07:19:18.794010+0000","last_fullsized":"2023-05-16T07:19:18.794010+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.149760+0000","last_change":"2023-05-16T07:19:19.149760+0000","last_active":"2023-05-16T07:19:19.149760+0000","last_peered":"2023-05-16T07:19:19.149760+0000","last_clean":"2023-05-16T07:19:19.149760+0000","last_became_active":"2023-05-16T07:19:19.149503+0000","last_became_peered":"2023-05-16T07:19:19.149503+0000","last_unstale":"2023-05-16T07:19:19.149760+0000","last_undegraded":"2023-05-16T07:19:19.149760+0000","last_fullsized":"2023-05-16T07:19:19.149760+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.146233+0000","last_change":"2023-05-16T07:19:18.786400+0000","last_active":"2023-05-16T07:19:19.146233+0000","last_peered":"2023-05-16T07:19:19.146233+0000","last_clean":"2023-05-16T07:19:19.146233+0000","last_became_active":"2023-05-16T07:19:18.786032+0000","last_became_peered":"2023-05-16T07:19:18.786032+0000","last_unstale":"2023-05-16T07:19:19.146233+0000","last_undegraded":"2023-05-16T07:19:19.146233+0000","last_fullsized":"2023-05-16T07:19:19.146233+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.168118+0000","last_change":"2023-05-16T07:19:19.168118+0000","last_active":"2023-05-16T07:19:19.168118+0000","last_peered":"2023-05-16T07:19:19.168118+0000","last_clean":"2023-05-16T07:19:19.168118+0000","last_became_active":"2023-05-16T07:19:19.167945+0000","last_became_peered":"2023-05-16T07:19:19.167945+0000","last_unstale":"2023-05-16T07:19:19.168118+0000","last_undegraded":"2023-05-16T07:19:19.168118+0000","last_fullsized":"2023-05-16T07:19:19.168118+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.149164+0000","last_change":"2023-05-16T07:19:18.775801+0000","last_active":"2023-05-16T07:19:19.149164+0000","last_peered":"2023-05-16T07:19:19.149164+0000","last_clean":"2023-05-16T07:19:19.149164+0000","last_became_active":"2023-05-16T07:19:18.774934+0000","last_became_peered":"2023-05-16T07:19:18.774934+0000","last_unstale":"2023-05-16T07:19:19.149164+0000","last_undegraded":"2023-05-16T07:19:19.149164+0000","last_fullsized":"2023-05-16T07:19:19.149164+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154346+0000","last_change":"2023-05-16T07:19:19.154346+0000","last_active":"2023-05-16T07:19:19.154346+0000","last_peered":"2023-05-16T07:19:19.154346+0000","last_clean":"2023-05-16T07:19:19.154346+0000","last_became_active":"2023-05-16T07:19:19.154008+0000","last_became_peered":"2023-05-16T07:19:19.154008+0000","last_unstale":"2023-05-16T07:19:19.154346+0000","last_undegraded":"2023-05-16T07:19:19.154346+0000","last_fullsized":"2023-05-16T07:19:19.154346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158443+0000","last_change":"2023-05-16T07:19:18.784982+0000","last_active":"2023-05-16T07:19:19.158443+0000","last_peered":"2023-05-16T07:19:19.158443+0000","last_clean":"2023-05-16T07:19:19.158443+0000","last_became_active":"2023-05-16T07:19:18.783374+0000","last_became_peered":"2023-05-16T07:19:18.783374+0000","last_unstale":"2023-05-16T07:19:19.158443+0000","last_undegraded":"2023-05-16T07:19:19.158443+0000","last_fullsized":"2023-05-16T07:19:19.158443+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.159159+0000","last_change":"2023-05-16T07:19:19.159159+0000","last_active":"2023-05-16T07:19:19.159159+0000","last_peered":"2023-05-16T07:19:19.159159+0000","last_clean":"2023-05-16T07:19:19.159159+0000","last_became_active":"2023-05-16T07:19:19.158843+0000","last_became_peered":"2023-05-16T07:19:19.158843+0000","last_unstale":"2023-05-16T07:19:19.159159+0000","last_undegraded":"2023-05-16T07:19:19.159159+0000","last_fullsized":"2023-05-16T07:19:19.159159+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.792030+0000","last_change":"2023-05-16T07:19:18.785276+0000","last_active":"2023-05-16T07:19:18.792030+0000","last_peered":"2023-05-16T07:19:18.792030+0000","last_clean":"2023-05-16T07:19:18.792030+0000","last_became_active":"2023-05-16T07:19:18.785016+0000","last_became_peered":"2023-05-16T07:19:18.785016+0000","last_unstale":"2023-05-16T07:19:18.792030+0000","last_undegraded":"2023-05-16T07:19:18.792030+0000","last_fullsized":"2023-05-16T07:19:18.792030+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.167188+0000","last_change":"2023-05-16T07:19:19.167188+0000","last_active":"2023-05-16T07:19:19.167188+0000","last_peered":"2023-05-16T07:19:19.167188+0000","last_clean":"2023-05-16T07:19:19.167188+0000","last_became_active":"2023-05-16T07:19:19.166919+0000","last_became_peered":"2023-05-16T07:19:19.166919+0000","last_unstale":"2023-05-16T07:19:19.167188+0000","last_undegraded":"2023-05-16T07:19:19.167188+0000","last_fullsized":"2023-05-16T07:19:19.167188+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.772657+0000","last_change":"2023-05-16T07:19:18.771030+0000","last_active":"2023-05-16T07:19:18.772657+0000","last_peered":"2023-05-16T07:19:18.772657+0000","last_clean":"2023-05-16T07:19:18.772657+0000","last_became_active":"2023-05-16T07:19:18.770748+0000","last_became_peered":"2023-05-16T07:19:18.770748+0000","last_unstale":"2023-05-16T07:19:18.772657+0000","last_undegraded":"2023-05-16T07:19:18.772657+0000","last_fullsized":"2023-05-16T07:19:18.772657+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162746+0000","last_change":"2023-05-16T07:19:19.162746+0000","last_active":"2023-05-16T07:19:19.162746+0000","last_peered":"2023-05-16T07:19:19.162746+0000","last_clean":"2023-05-16T07:19:19.162746+0000","last_became_active":"2023-05-16T07:19:19.162307+0000","last_became_peered":"2023-05-16T07:19:19.162307+0000","last_unstale":"2023-05-16T07:19:19.162746+0000","last_undegraded":"2023-05-16T07:19:19.162746+0000","last_fullsized":"2023-05-16T07:19:19.162746+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785438+0000","last_change":"2023-05-16T07:19:18.781482+0000","last_active":"2023-05-16T07:19:18.785438+0000","last_peered":"2023-05-16T07:19:18.785438+0000","last_clean":"2023-05-16T07:19:18.785438+0000","last_became_active":"2023-05-16T07:19:18.781175+0000","last_became_peered":"2023-05-16T07:19:18.781175+0000","last_unstale":"2023-05-16T07:19:18.785438+0000","last_undegraded":"2023-05-16T07:19:18.785438+0000","last_fullsized":"2023-05-16T07:19:18.785438+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.143623+0000","last_change":"2023-05-16T07:19:19.143623+0000","last_active":"2023-05-16T07:19:19.143623+0000","last_peered":"2023-05-16T07:19:19.143623+0000","last_clean":"2023-05-16T07:19:19.143623+0000","last_became_active":"2023-05-16T07:19:19.143136+0000","last_became_peered":"2023-05-16T07:19:19.143136+0000","last_unstale":"2023-05-16T07:19:19.143623+0000","last_undegraded":"2023-05-16T07:19:19.143623+0000","last_fullsized":"2023-05-16T07:19:19.143623+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788901+0000","last_change":"2023-05-16T07:19:18.783900+0000","last_active":"2023-05-16T07:19:18.788901+0000","last_peered":"2023-05-16T07:19:18.788901+0000","last_clean":"2023-05-16T07:19:18.788901+0000","last_became_active":"2023-05-16T07:19:18.783167+0000","last_became_peered":"2023-05-16T07:19:18.783167+0000","last_unstale":"2023-05-16T07:19:18.788901+0000","last_undegraded":"2023-05-16T07:19:18.788901+0000","last_fullsized":"2023-05-16T07:19:18.788901+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155583+0000","last_change":"2023-05-16T07:19:19.155583+0000","last_active":"2023-05-16T07:19:19.155583+0000","last_peered":"2023-05-16T07:19:19.155583+0000","last_clean":"2023-05-16T07:19:19.155583+0000","last_became_active":"2023-05-16T07:19:19.155167+0000","last_became_peered":"2023-05-16T07:19:19.155167+0000","last_unstale":"2023-05-16T07:19:19.155583+0000","last_undegraded":"2023-05-16T07:19:19.155583+0000","last_fullsized":"2023-05-16T07:19:19.155583+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.793084+0000","last_change":"2023-05-16T07:19:18.791521+0000","last_active":"2023-05-16T07:19:18.793084+0000","last_peered":"2023-05-16T07:19:18.793084+0000","last_clean":"2023-05-16T07:19:18.793084+0000","last_became_active":"2023-05-16T07:19:18.791257+0000","last_became_peered":"2023-05-16T07:19:18.791257+0000","last_unstale":"2023-05-16T07:19:18.793084+0000","last_undegraded":"2023-05-16T07:19:18.793084+0000","last_fullsized":"2023-05-16T07:19:18.793084+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.153842+0000","last_change":"2023-05-16T07:19:19.153842+0000","last_active":"2023-05-16T07:19:19.153842+0000","last_peered":"2023-05-16T07:19:19.153842+0000","last_clean":"2023-05-16T07:19:19.153842+0000","last_became_active":"2023-05-16T07:19:19.153581+0000","last_became_peered":"2023-05-16T07:19:19.153581+0000","last_unstale":"2023-05-16T07:19:19.153842+0000","last_undegraded":"2023-05-16T07:19:19.153842+0000","last_fullsized":"2023-05-16T07:19:19.153842+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788842+0000","last_change":"2023-05-16T07:19:18.783043+0000","last_active":"2023-05-16T07:19:18.788842+0000","last_peered":"2023-05-16T07:19:18.788842+0000","last_clean":"2023-05-16T07:19:18.788842+0000","last_became_active":"2023-05-16T07:19:18.782750+0000","last_became_peered":"2023-05-16T07:19:18.782750+0000","last_unstale":"2023-05-16T07:19:18.788842+0000","last_undegraded":"2023-05-16T07:19:18.788842+0000","last_fullsized":"2023-05-16T07:19:18.788842+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160849+0000","last_change":"2023-05-16T07:19:19.160849+0000","last_active":"2023-05-16T07:19:19.160849+0000","last_peered":"2023-05-16T07:19:19.160849+0000","last_clean":"2023-05-16T07:19:19.160849+0000","last_became_active":"2023-05-16T07:19:19.160582+0000","last_became_peered":"2023-05-16T07:19:19.160582+0000","last_unstale":"2023-05-16T07:19:19.160849+0000","last_undegraded":"2023-05-16T07:19:19.160849+0000","last_fullsized":"2023-05-16T07:19:19.160849+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785197+0000","last_change":"2023-05-16T07:19:18.783798+0000","last_active":"2023-05-16T07:19:18.785197+0000","last_peered":"2023-05-16T07:19:18.785197+0000","last_clean":"2023-05-16T07:19:18.785197+0000","last_became_active":"2023-05-16T07:19:18.783609+0000","last_became_peered":"2023-05-16T07:19:18.783609+0000","last_unstale":"2023-05-16T07:19:18.785197+0000","last_undegraded":"2023-05-16T07:19:18.785197+0000","last_fullsized":"2023-05-16T07:19:18.785197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.167255+0000","last_change":"2023-05-16T07:19:19.167255+0000","last_active":"2023-05-16T07:19:19.167255+0000","last_peered":"2023-05-16T07:19:19.167255+0000","last_clean":"2023-05-16T07:19:19.167255+0000","last_became_active":"2023-05-16T07:19:19.166971+0000","last_became_peered":"2023-05-16T07:19:19.166971+0000","last_unstale":"2023-05-16T07:19:19.167255+0000","last_undegraded":"2023-05-16T07:19:19.167255+0000","last_fullsized":"2023-05-16T07:19:19.167255+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"120'3","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.159042+0000","last_change":"2023-05-16T07:19:18.778412+0000","last_active":"2023-05-16T07:19:19.159042+0000","last_peered":"2023-05-16T07:19:19.159042+0000","last_clean":"2023-05-16T07:19:19.159042+0000","last_became_active":"2023-05-16T07:19:18.778125+0000","last_became_peered":"2023-05-16T07:19:18.778125+0000","last_unstale":"2023-05-16T07:19:19.159042+0000","last_undegraded":"2023-05-16T07:19:19.159042+0000","last_fullsized":"2023-05-16T07:19:19.159042+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163143+0000","last_change":"2023-05-16T07:19:19.163143+0000","last_active":"2023-05-16T07:19:19.163143+0000","last_peered":"2023-05-16T07:19:19.163143+0000","last_clean":"2023-05-16T07:19:19.163143+0000","last_became_active":"2023-05-16T07:19:19.162874+0000","last_became_peered":"2023-05-16T07:19:19.162874+0000","last_unstale":"2023-05-16T07:19:19.163143+0000","last_undegraded":"2023-05-16T07:19:19.163143+0000","last_fullsized":"2023-05-16T07:19:19.163143+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.164511+0000","last_change":"2023-05-16T07:19:18.780209+0000","last_active":"2023-05-16T07:19:19.164511+0000","last_peered":"2023-05-16T07:19:19.164511+0000","last_clean":"2023-05-16T07:19:19.164511+0000","last_became_active":"2023-05-16T07:19:18.779771+0000","last_became_peered":"2023-05-16T07:19:18.779771+0000","last_unstale":"2023-05-16T07:19:19.164511+0000","last_undegraded":"2023-05-16T07:19:19.164511+0000","last_fullsized":"2023-05-16T07:19:19.164511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155766+0000","last_change":"2023-05-16T07:19:19.155766+0000","last_active":"2023-05-16T07:19:19.155766+0000","last_peered":"2023-05-16T07:19:19.155766+0000","last_clean":"2023-05-16T07:19:19.155766+0000","last_became_active":"2023-05-16T07:19:19.155446+0000","last_became_peered":"2023-05-16T07:19:19.155446+0000","last_unstale":"2023-05-16T07:19:19.155766+0000","last_undegraded":"2023-05-16T07:19:19.155766+0000","last_fullsized":"2023-05-16T07:19:19.155766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.140489+0000","last_change":"2023-05-16T07:19:18.789954+0000","last_active":"2023-05-16T07:19:19.140489+0000","last_peered":"2023-05-16T07:19:19.140489+0000","last_clean":"2023-05-16T07:19:19.140489+0000","last_became_active":"2023-05-16T07:19:18.789671+0000","last_became_peered":"2023-05-16T07:19:18.789671+0000","last_unstale":"2023-05-16T07:19:19.140489+0000","last_undegraded":"2023-05-16T07:19:19.140489+0000","last_fullsized":"2023-05-16T07:19:19.140489+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150536+0000","last_change":"2023-05-16T07:19:19.150536+0000","last_active":"2023-05-16T07:19:19.150536+0000","last_peered":"2023-05-16T07:19:19.150536+0000","last_clean":"2023-05-16T07:19:19.150536+0000","last_became_active":"2023-05-16T07:19:19.149715+0000","last_became_peered":"2023-05-16T07:19:19.149715+0000","last_unstale":"2023-05-16T07:19:19.150536+0000","last_undegraded":"2023-05-16T07:19:19.150536+0000","last_fullsized":"2023-05-16T07:19:19.150536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785505+0000","last_change":"2023-05-16T07:19:18.777062+0000","last_active":"2023-05-16T07:19:18.785505+0000","last_peered":"2023-05-16T07:19:18.785505+0000","last_clean":"2023-05-16T07:19:18.785505+0000","last_became_active":"2023-05-16T07:19:18.776808+0000","last_became_peered":"2023-05-16T07:19:18.776808+0000","last_unstale":"2023-05-16T07:19:18.785505+0000","last_undegraded":"2023-05-16T07:19:18.785505+0000","last_fullsized":"2023-05-16T07:19:18.785505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156081+0000","last_change":"2023-05-16T07:19:19.156081+0000","last_active":"2023-05-16T07:19:19.156081+0000","last_peered":"2023-05-16T07:19:19.156081+0000","last_clean":"2023-05-16T07:19:19.156081+0000","last_became_active":"2023-05-16T07:19:19.155813+0000","last_became_peered":"2023-05-16T07:19:19.155813+0000","last_unstale":"2023-05-16T07:19:19.156081+0000","last_undegraded":"2023-05-16T07:19:19.156081+0000","last_fullsized":"2023-05-16T07:19:19.156081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788826+0000","last_change":"2023-05-16T07:19:18.785319+0000","last_active":"2023-05-16T07:19:18.788826+0000","last_peered":"2023-05-16T07:19:18.788826+0000","last_clean":"2023-05-16T07:19:18.788826+0000","last_became_active":"2023-05-16T07:19:18.784968+0000","last_became_peered":"2023-05-16T07:19:18.784968+0000","last_unstale":"2023-05-16T07:19:18.788826+0000","last_undegraded":"2023-05-16T07:19:18.788826+0000","last_fullsized":"2023-05-16T07:19:18.788826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155403+0000","last_change":"2023-05-16T07:19:19.155403+0000","last_active":"2023-05-16T07:19:19.155403+0000","last_peered":"2023-05-16T07:19:19.155403+0000","last_clean":"2023-05-16T07:19:19.155403+0000","last_became_active":"2023-05-16T07:19:19.154491+0000","last_became_peered":"2023-05-16T07:19:19.154491+0000","last_unstale":"2023-05-16T07:19:19.155403+0000","last_undegraded":"2023-05-16T07:19:19.155403+0000","last_fullsized":"2023-05-16T07:19:19.155403+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"120'4","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158691+0000","last_change":"2023-05-16T07:19:18.783503+0000","last_active":"2023-05-16T07:19:19.158691+0000","last_peered":"2023-05-16T07:19:19.158691+0000","last_clean":"2023-05-16T07:19:19.158691+0000","last_became_active":"2023-05-16T07:19:18.783119+0000","last_became_peered":"2023-05-16T07:19:18.783119+0000","last_unstale":"2023-05-16T07:19:19.158691+0000","last_undegraded":"2023-05-16T07:19:19.158691+0000","last_fullsized":"2023-05-16T07:19:19.158691+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150270+0000","last_change":"2023-05-16T07:19:19.150270+0000","last_active":"2023-05-16T07:19:19.150270+0000","last_peered":"2023-05-16T07:19:19.150270+0000","last_clean":"2023-05-16T07:19:19.150270+0000","last_became_active":"2023-05-16T07:19:19.149962+0000","last_became_peered":"2023-05-16T07:19:19.149962+0000","last_unstale":"2023-05-16T07:19:19.150270+0000","last_undegraded":"2023-05-16T07:19:19.150270+0000","last_fullsized":"2023-05-16T07:19:19.150270+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163309+0000","last_change":"2023-05-16T07:19:18.790080+0000","last_active":"2023-05-16T07:19:19.163309+0000","last_peered":"2023-05-16T07:19:19.163309+0000","last_clean":"2023-05-16T07:19:19.163309+0000","last_became_active":"2023-05-16T07:19:18.789727+0000","last_became_peered":"2023-05-16T07:19:18.789727+0000","last_unstale":"2023-05-16T07:19:19.163309+0000","last_undegraded":"2023-05-16T07:19:19.163309+0000","last_fullsized":"2023-05-16T07:19:19.163309+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162517+0000","last_change":"2023-05-16T07:19:19.162517+0000","last_active":"2023-05-16T07:19:19.162517+0000","last_peered":"2023-05-16T07:19:19.162517+0000","last_clean":"2023-05-16T07:19:19.162517+0000","last_became_active":"2023-05-16T07:19:19.162168+0000","last_became_peered":"2023-05-16T07:19:19.162168+0000","last_unstale":"2023-05-16T07:19:19.162517+0000","last_undegraded":"2023-05-16T07:19:19.162517+0000","last_fullsized":"2023-05-16T07:19:19.162517+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785179+0000","last_change":"2023-05-16T07:19:18.781799+0000","last_active":"2023-05-16T07:19:18.785179+0000","last_peered":"2023-05-16T07:19:18.785179+0000","last_clean":"2023-05-16T07:19:18.785179+0000","last_became_active":"2023-05-16T07:19:18.781226+0000","last_became_peered":"2023-05-16T07:19:18.781226+0000","last_unstale":"2023-05-16T07:19:18.785179+0000","last_undegraded":"2023-05-16T07:19:18.785179+0000","last_fullsized":"2023-05-16T07:19:18.785179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158073+0000","last_change":"2023-05-16T07:19:19.158073+0000","last_active":"2023-05-16T07:19:19.158073+0000","last_peered":"2023-05-16T07:19:19.158073+0000","last_clean":"2023-05-16T07:19:19.158073+0000","last_became_active":"2023-05-16T07:19:19.157801+0000","last_became_peered":"2023-05-16T07:19:19.157801+0000","last_unstale":"2023-05-16T07:19:19.158073+0000","last_undegraded":"2023-05-16T07:19:19.158073+0000","last_fullsized":"2023-05-16T07:19:19.158073+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782138+0000","last_change":"2023-05-16T07:19:18.779784+0000","last_active":"2023-05-16T07:19:18.782138+0000","last_peered":"2023-05-16T07:19:18.782138+0000","last_clean":"2023-05-16T07:19:18.782138+0000","last_became_active":"2023-05-16T07:19:18.779477+0000","last_became_peered":"2023-05-16T07:19:18.779477+0000","last_unstale":"2023-05-16T07:19:18.782138+0000","last_undegraded":"2023-05-16T07:19:18.782138+0000","last_fullsized":"2023-05-16T07:19:18.782138+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160828+0000","last_change":"2023-05-16T07:19:19.160828+0000","last_active":"2023-05-16T07:19:19.160828+0000","last_peered":"2023-05-16T07:19:19.160828+0000","last_clean":"2023-05-16T07:19:19.160828+0000","last_became_active":"2023-05-16T07:19:19.160609+0000","last_became_peered":"2023-05-16T07:19:19.160609+0000","last_unstale":"2023-05-16T07:19:19.160828+0000","last_undegraded":"2023-05-16T07:19:19.160828+0000","last_fullsized":"2023-05-16T07:19:19.160828+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789373+0000","last_change":"2023-05-16T07:19:18.781076+0000","last_active":"2023-05-16T07:19:18.789373+0000","last_peered":"2023-05-16T07:19:18.789373+0000","last_clean":"2023-05-16T07:19:18.789373+0000","last_became_active":"2023-05-16T07:19:18.780730+0000","last_became_peered":"2023-05-16T07:19:18.780730+0000","last_unstale":"2023-05-16T07:19:18.789373+0000","last_undegraded":"2023-05-16T07:19:18.789373+0000","last_fullsized":"2023-05-16T07:19:18.789373+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.167741+0000","last_change":"2023-05-16T07:19:19.167741+0000","last_active":"2023-05-16T07:19:19.167741+0000","last_peered":"2023-05-16T07:19:19.167741+0000","last_clean":"2023-05-16T07:19:19.167741+0000","last_became_active":"2023-05-16T07:19:19.167510+0000","last_became_peered":"2023-05-16T07:19:19.167510+0000","last_unstale":"2023-05-16T07:19:19.167741+0000","last_undegraded":"2023-05-16T07:19:19.167741+0000","last_fullsized":"2023-05-16T07:19:19.167741+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785990+0000","last_change":"2023-05-16T07:19:18.780689+0000","last_active":"2023-05-16T07:19:18.785990+0000","last_peered":"2023-05-16T07:19:18.785990+0000","last_clean":"2023-05-16T07:19:18.785990+0000","last_became_active":"2023-05-16T07:19:18.780391+0000","last_became_peered":"2023-05-16T07:19:18.780391+0000","last_unstale":"2023-05-16T07:19:18.785990+0000","last_undegraded":"2023-05-16T07:19:18.785990+0000","last_fullsized":"2023-05-16T07:19:18.785990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.161302+0000","last_change":"2023-05-16T07:19:19.161302+0000","last_active":"2023-05-16T07:19:19.161302+0000","last_peered":"2023-05-16T07:19:19.161302+0000","last_clean":"2023-05-16T07:19:19.161302+0000","last_became_active":"2023-05-16T07:19:19.160781+0000","last_became_peered":"2023-05-16T07:19:19.160781+0000","last_unstale":"2023-05-16T07:19:19.161302+0000","last_undegraded":"2023-05-16T07:19:19.161302+0000","last_fullsized":"2023-05-16T07:19:19.161302+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785825+0000","last_change":"2023-05-16T07:19:18.781734+0000","last_active":"2023-05-16T07:19:18.785825+0000","last_peered":"2023-05-16T07:19:18.785825+0000","last_clean":"2023-05-16T07:19:18.785825+0000","last_became_active":"2023-05-16T07:19:18.781534+0000","last_became_peered":"2023-05-16T07:19:18.781534+0000","last_unstale":"2023-05-16T07:19:18.785825+0000","last_undegraded":"2023-05-16T07:19:18.785825+0000","last_fullsized":"2023-05-16T07:19:18.785825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151782+0000","last_change":"2023-05-16T07:19:19.151782+0000","last_active":"2023-05-16T07:19:19.151782+0000","last_peered":"2023-05-16T07:19:19.151782+0000","last_clean":"2023-05-16T07:19:19.151782+0000","last_became_active":"2023-05-16T07:19:19.151498+0000","last_became_peered":"2023-05-16T07:19:19.151498+0000","last_unstale":"2023-05-16T07:19:19.151782+0000","last_undegraded":"2023-05-16T07:19:19.151782+0000","last_fullsized":"2023-05-16T07:19:19.151782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782084+0000","last_change":"2023-05-16T07:19:18.779031+0000","last_active":"2023-05-16T07:19:18.782084+0000","last_peered":"2023-05-16T07:19:18.782084+0000","last_clean":"2023-05-16T07:19:18.782084+0000","last_became_active":"2023-05-16T07:19:18.778746+0000","last_became_peered":"2023-05-16T07:19:18.778746+0000","last_unstale":"2023-05-16T07:19:18.782084+0000","last_undegraded":"2023-05-16T07:19:18.782084+0000","last_fullsized":"2023-05-16T07:19:18.782084+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150276+0000","last_change":"2023-05-16T07:19:19.150276+0000","last_active":"2023-05-16T07:19:19.150276+0000","last_peered":"2023-05-16T07:19:19.150276+0000","last_clean":"2023-05-16T07:19:19.150276+0000","last_became_active":"2023-05-16T07:19:19.149677+0000","last_became_peered":"2023-05-16T07:19:19.149677+0000","last_unstale":"2023-05-16T07:19:19.150276+0000","last_undegraded":"2023-05-16T07:19:19.150276+0000","last_fullsized":"2023-05-16T07:19:19.150276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.791729+0000","last_change":"2023-05-16T07:19:18.785742+0000","last_active":"2023-05-16T07:19:18.791729+0000","last_peered":"2023-05-16T07:19:18.791729+0000","last_clean":"2023-05-16T07:19:18.791729+0000","last_became_active":"2023-05-16T07:19:18.785006+0000","last_became_peered":"2023-05-16T07:19:18.785006+0000","last_unstale":"2023-05-16T07:19:18.791729+0000","last_undegraded":"2023-05-16T07:19:18.791729+0000","last_fullsized":"2023-05-16T07:19:18.791729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162534+0000","last_change":"2023-05-16T07:19:19.162534+0000","last_active":"2023-05-16T07:19:19.162534+0000","last_peered":"2023-05-16T07:19:19.162534+0000","last_clean":"2023-05-16T07:19:19.162534+0000","last_became_active":"2023-05-16T07:19:19.162285+0000","last_became_peered":"2023-05-16T07:19:19.162285+0000","last_unstale":"2023-05-16T07:19:19.162534+0000","last_undegraded":"2023-05-16T07:19:19.162534+0000","last_fullsized":"2023-05-16T07:19:19.162534+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782127+0000","last_change":"2023-05-16T07:19:18.780923+0000","last_active":"2023-05-16T07:19:18.782127+0000","last_peered":"2023-05-16T07:19:18.782127+0000","last_clean":"2023-05-16T07:19:18.782127+0000","last_became_active":"2023-05-16T07:19:18.780278+0000","last_became_peered":"2023-05-16T07:19:18.780278+0000","last_unstale":"2023-05-16T07:19:18.782127+0000","last_undegraded":"2023-05-16T07:19:18.782127+0000","last_fullsized":"2023-05-16T07:19:18.782127+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.772575+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:19:18.772575+0000","last_peered":"2023-05-16T07:19:18.772575+0000","last_clean":"2023-05-16T07:19:18.772575+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:19:18.772575+0000","last_undegraded":"2023-05-16T07:19:18.772575+0000","last_fullsized":"2023-05-16T07:19:18.772575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156445+0000","last_change":"2023-05-16T07:19:19.156445+0000","last_active":"2023-05-16T07:19:19.156445+0000","last_peered":"2023-05-16T07:19:19.156445+0000","last_clean":"2023-05-16T07:19:19.156445+0000","last_became_active":"2023-05-16T07:19:19.156172+0000","last_became_peered":"2023-05-16T07:19:19.156172+0000","last_unstale":"2023-05-16T07:19:19.156445+0000","last_undegraded":"2023-05-16T07:19:19.156445+0000","last_fullsized":"2023-05-16T07:19:19.156445+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.140491+0000","last_change":"2023-05-16T07:19:18.784697+0000","last_active":"2023-05-16T07:19:19.140491+0000","last_peered":"2023-05-16T07:19:19.140491+0000","last_clean":"2023-05-16T07:19:19.140491+0000","last_became_active":"2023-05-16T07:19:18.784412+0000","last_became_peered":"2023-05-16T07:19:18.784412+0000","last_unstale":"2023-05-16T07:19:19.140491+0000","last_undegraded":"2023-05-16T07:19:19.140491+0000","last_fullsized":"2023-05-16T07:19:19.140491+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.769464+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:19:18.769464+0000","last_peered":"2023-05-16T07:19:18.769464+0000","last_clean":"2023-05-16T07:19:18.769464+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:19:18.769464+0000","last_undegraded":"2023-05-16T07:19:18.769464+0000","last_fullsized":"2023-05-16T07:19:18.769464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150204+0000","last_change":"2023-05-16T07:19:19.150204+0000","last_active":"2023-05-16T07:19:19.150204+0000","last_peered":"2023-05-16T07:19:19.150204+0000","last_clean":"2023-05-16T07:19:19.150204+0000","last_became_active":"2023-05-16T07:19:19.150015+0000","last_became_peered":"2023-05-16T07:19:19.150015+0000","last_unstale":"2023-05-16T07:19:19.150204+0000","last_undegraded":"2023-05-16T07:19:19.150204+0000","last_fullsized":"2023-05-16T07:19:19.150204+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.780422+0000","last_change":"2023-05-16T07:19:18.775232+0000","last_active":"2023-05-16T07:19:18.780422+0000","last_peered":"2023-05-16T07:19:18.780422+0000","last_clean":"2023-05-16T07:19:18.780422+0000","last_became_active":"2023-05-16T07:19:18.774959+0000","last_became_peered":"2023-05-16T07:19:18.774959+0000","last_unstale":"2023-05-16T07:19:18.780422+0000","last_undegraded":"2023-05-16T07:19:18.780422+0000","last_fullsized":"2023-05-16T07:19:18.780422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.767926+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:19:18.767926+0000","last_peered":"2023-05-16T07:19:18.767926+0000","last_clean":"2023-05-16T07:19:18.767926+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:19:18.767926+0000","last_undegraded":"2023-05-16T07:19:18.767926+0000","last_fullsized":"2023-05-16T07:19:18.767926+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160610+0000","last_change":"2023-05-16T07:19:19.160610+0000","last_active":"2023-05-16T07:19:19.160610+0000","last_peered":"2023-05-16T07:19:19.160610+0000","last_clean":"2023-05-16T07:19:19.160610+0000","last_became_active":"2023-05-16T07:19:19.160295+0000","last_became_peered":"2023-05-16T07:19:19.160295+0000","last_unstale":"2023-05-16T07:19:19.160610+0000","last_undegraded":"2023-05-16T07:19:19.160610+0000","last_fullsized":"2023-05-16T07:19:19.160610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160883+0000","last_change":"2023-05-16T07:19:18.779399+0000","last_active":"2023-05-16T07:19:19.160883+0000","last_peered":"2023-05-16T07:19:19.160883+0000","last_clean":"2023-05-16T07:19:19.160883+0000","last_became_active":"2023-05-16T07:19:18.779143+0000","last_became_peered":"2023-05-16T07:19:18.779143+0000","last_unstale":"2023-05-16T07:19:19.160883+0000","last_undegraded":"2023-05-16T07:19:19.160883+0000","last_fullsized":"2023-05-16T07:19:19.160883+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.771627+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:19:18.771627+0000","last_peered":"2023-05-16T07:19:18.771627+0000","last_clean":"2023-05-16T07:19:18.771627+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:19:18.771627+0000","last_undegraded":"2023-05-16T07:19:18.771627+0000","last_fullsized":"2023-05-16T07:19:18.771627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.767289+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:19:18.767289+0000","last_peered":"2023-05-16T07:19:18.767289+0000","last_clean":"2023-05-16T07:19:18.767289+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:19:18.767289+0000","last_undegraded":"2023-05-16T07:19:18.767289+0000","last_fullsized":"2023-05-16T07:19:18.767289+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156080+0000","last_change":"2023-05-16T07:19:19.156080+0000","last_active":"2023-05-16T07:19:19.156080+0000","last_peered":"2023-05-16T07:19:19.156080+0000","last_clean":"2023-05-16T07:19:19.156080+0000","last_became_active":"2023-05-16T07:19:19.155820+0000","last_became_peered":"2023-05-16T07:19:19.155820+0000","last_unstale":"2023-05-16T07:19:19.156080+0000","last_undegraded":"2023-05-16T07:19:19.156080+0000","last_fullsized":"2023-05-16T07:19:19.156080+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.787533+0000","last_change":"2023-05-16T07:19:18.785576+0000","last_active":"2023-05-16T07:19:18.787533+0000","last_peered":"2023-05-16T07:19:18.787533+0000","last_clean":"2023-05-16T07:19:18.787533+0000","last_became_active":"2023-05-16T07:19:18.784910+0000","last_became_peered":"2023-05-16T07:19:18.784910+0000","last_unstale":"2023-05-16T07:19:18.787533+0000","last_undegraded":"2023-05-16T07:19:18.787533+0000","last_fullsized":"2023-05-16T07:19:18.787533+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.767375+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:19:18.767375+0000","last_peered":"2023-05-16T07:19:18.767375+0000","last_clean":"2023-05-16T07:19:18.767375+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:19:18.767375+0000","last_undegraded":"2023-05-16T07:19:18.767375+0000","last_fullsized":"2023-05-16T07:19:18.767375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.168720+0000","last_change":"2023-05-16T07:19:19.168720+0000","last_active":"2023-05-16T07:19:19.168720+0000","last_peered":"2023-05-16T07:19:19.168720+0000","last_clean":"2023-05-16T07:19:19.168720+0000","last_became_active":"2023-05-16T07:19:19.168458+0000","last_became_peered":"2023-05-16T07:19:19.168458+0000","last_unstale":"2023-05-16T07:19:19.168720+0000","last_undegraded":"2023-05-16T07:19:19.168720+0000","last_fullsized":"2023-05-16T07:19:19.168720+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785660+0000","last_change":"2023-05-16T07:19:18.777463+0000","last_active":"2023-05-16T07:19:18.785660+0000","last_peered":"2023-05-16T07:19:18.785660+0000","last_clean":"2023-05-16T07:19:18.785660+0000","last_became_active":"2023-05-16T07:19:18.776988+0000","last_became_peered":"2023-05-16T07:19:18.776988+0000","last_unstale":"2023-05-16T07:19:18.785660+0000","last_undegraded":"2023-05-16T07:19:18.785660+0000","last_fullsized":"2023-05-16T07:19:18.785660+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.766609+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:19:18.766609+0000","last_peered":"2023-05-16T07:19:18.766609+0000","last_clean":"2023-05-16T07:19:18.766609+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:19:18.766609+0000","last_undegraded":"2023-05-16T07:19:18.766609+0000","last_fullsized":"2023-05-16T07:19:18.766609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151508+0000","last_change":"2023-05-16T07:19:19.151508+0000","last_active":"2023-05-16T07:19:19.151508+0000","last_peered":"2023-05-16T07:19:19.151508+0000","last_clean":"2023-05-16T07:19:19.151508+0000","last_became_active":"2023-05-16T07:19:19.151255+0000","last_became_peered":"2023-05-16T07:19:19.151255+0000","last_unstale":"2023-05-16T07:19:19.151508+0000","last_undegraded":"2023-05-16T07:19:19.151508+0000","last_fullsized":"2023-05-16T07:19:19.151508+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155094+0000","last_change":"2023-05-16T07:19:18.777969+0000","last_active":"2023-05-16T07:19:19.155094+0000","last_peered":"2023-05-16T07:19:19.155094+0000","last_clean":"2023-05-16T07:19:19.155094+0000","last_became_active":"2023-05-16T07:19:18.777538+0000","last_became_peered":"2023-05-16T07:19:18.777538+0000","last_unstale":"2023-05-16T07:19:19.155094+0000","last_undegraded":"2023-05-16T07:19:19.155094+0000","last_fullsized":"2023-05-16T07:19:19.155094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.781395+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:19:18.781395+0000","last_peered":"2023-05-16T07:19:18.781395+0000","last_clean":"2023-05-16T07:19:18.781395+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:19:18.781395+0000","last_undegraded":"2023-05-16T07:19:18.781395+0000","last_fullsized":"2023-05-16T07:19:18.781395+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.161931+0000","last_change":"2023-05-16T07:19:19.161931+0000","last_active":"2023-05-16T07:19:19.161931+0000","last_peered":"2023-05-16T07:19:19.161931+0000","last_clean":"2023-05-16T07:19:19.161931+0000","last_became_active":"2023-05-16T07:19:19.161242+0000","last_became_peered":"2023-05-16T07:19:19.161242+0000","last_unstale":"2023-05-16T07:19:19.161931+0000","last_undegraded":"2023-05-16T07:19:19.161931+0000","last_fullsized":"2023-05-16T07:19:19.161931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785806+0000","last_change":"2023-05-16T07:19:18.778237+0000","last_active":"2023-05-16T07:19:18.785806+0000","last_peered":"2023-05-16T07:19:18.785806+0000","last_clean":"2023-05-16T07:19:18.785806+0000","last_became_active":"2023-05-16T07:19:18.777975+0000","last_became_peered":"2023-05-16T07:19:18.777975+0000","last_unstale":"2023-05-16T07:19:18.785806+0000","last_undegraded":"2023-05-16T07:19:18.785806+0000","last_fullsized":"2023-05-16T07:19:18.785806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":111,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.770701+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:19:18.770701+0000","last_peered":"2023-05-16T07:19:18.770701+0000","last_clean":"2023-05-16T07:19:18.770701+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:19:18.770701+0000","last_undegraded":"2023-05-16T07:19:18.770701+0000","last_fullsized":"2023-05-16T07:19:18.770701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22564,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349276,"statfs":{"total":96636764160,"available":96613658624,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.56200000000000006},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.77700000000000002,"15min":0.84199999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.54200000000000004,"15min":0.56499999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73299999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38900000000000001}]},{"osd":1,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.55800000000000005},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.54400000000000004,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.749,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56499999999999995},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.50800000000000001}]},{"osd":3,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.79300000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58899999999999997}]},{"osd":4,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54500000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65000000000000002,"5min":0.752,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.621},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68200000000000005,"5min":0.748,"15min":0.751},"last":0.628}]},{"osd":6,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65700000000000003,"5min":0.879,"15min":0.879},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.53100000000000003,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22244,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349596,"statfs":{"total":96636764160,"available":96613986304,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53600000000000003,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.85099999999999998},"last":0.435},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71499999999999997,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78400000000000003,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38},"max":{"1min":0.73999999999999999,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70999999999999996,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.377,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.73899999999999999,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57699999999999996,"5min":0.749,"15min":0.89400000000000002},"last":0.372},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":4.048,"15min":4.048},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.92300000000000004},"last":0.48999999999999999}]},{"osd":7,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.5},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70699999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18212,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":16768,"kb_avail":94353628,"statfs":{"total":96636764160,"available":96618115072,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17170432},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.69899999999999995},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.39800000000000002,"15min":0.42099999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.5,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35699999999999998,"15min":0.379},"min":{"1min":0.27200000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41099999999999998,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38400000000000001,"15min":0.40500000000000003},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52100000000000002,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.27100000000000002}]},{"osd":3,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.377,"15min":0.40000000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46300000000000002,"5min":0.622,"15min":0.64000000000000001},"last":0.26000000000000001}]},{"osd":4,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.80000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.623},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61099999999999999,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77400000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.63500000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85599999999999998,"5min":0.94699999999999995,"15min":1.087},"last":0.434},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.93700000000000006,"15min":1.204},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17188,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":15744,"kb_avail":94354652,"statfs":{"total":96636764160,"available":96619163648,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16121856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.36299999999999999,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34599999999999997,"15min":0.372},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.437,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.40500000000000003},"min":{"1min":0.251,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.25}]},{"osd":3,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.40799999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.49399999999999999,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.434},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.499},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65500000000000003,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0669999999999999},"last":0.47899999999999998}]},{"osd":7,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18980,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":17536,"kb_avail":94352860,"statfs":{"total":96636764160,"available":96617328640,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17956864},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.35099999999999998,"15min":0.36499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.184},"max":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.64900000000000002},"last":0.31},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.372,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.47699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.376}]},{"osd":1,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.40000000000000002,"15min":0.41199999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.71199999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.42099999999999999,"15min":0.437},"min":{"1min":0.29999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.5,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.436}]},{"osd":3,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.378,"15min":0.39500000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22700000000000001},"max":{"1min":0.51400000000000001,"5min":0.58299999999999996,"15min":0.64700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.41099999999999998,"15min":0.438},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51400000000000001,"5min":0.64900000000000002,"15min":0.72899999999999998},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.378,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.625},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.91400000000000003,"15min":1.113},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61299999999999999,"15min":0.63200000000000001},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.82599999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":0.88100000000000001,"15min":1.5609999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":1.0369999999999999,"15min":1.47},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81299999999999994,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.63300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26096,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":24640,"kb_avail":94345744,"statfs":{"total":96636764160,"available":96610041856,"internally_reserved":0,"allocated":1482752,"data_stored":775444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25231360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35799999999999998,"15min":0.379},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.5,"5min":0.54900000000000004,"15min":0.58799999999999997},"last":0.307}]},{"osd":1,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.48299999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.41599999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41399999999999998,"15min":0.433},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.53300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":0.877,"15min":0.91600000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.61399999999999999},"min":{"1min":0.40300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96399999999999997},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0489999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":1.0409999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0780000000000001},"last":0.62},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.877,"15min":1.0049999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61299999999999999,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.02},"last":0.67400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18276,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":16832,"kb_avail":94353564,"statfs":{"total":96636764160,"available":96618049536,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17235968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36199999999999999,"15min":0.35999999999999999},"max":{"1min":2.5600000000000001,"5min":2.5600000000000001,"15min":2.5600000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.628},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575049,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25444,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":24000,"kb_avail":94346396,"statfs":{"total":96636764160,"available":96610709504,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24576000},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.32500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.641,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":2.847,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.434}]},{"osd":1,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.499},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.6059999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73799999999999999,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.127,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.503,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1150000000000002,"5min":3.1150000000000002,"15min":3.1150000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":3.5659999999999998,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.52600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.76600000000000001}]},{"osd":7,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:19:22.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:19:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.263+0000 7fc34890e700 1 Processor -- start 2023-05-16T07:19:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.263+0000 7fc34890e700 1 -- start start 2023-05-16T07:19:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc34890e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc3441517f0 0x7fc344156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc34890e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc344151bd0 0x7fc344152a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc34890e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3440a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc34890e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440c8750 con 0x7fc344151bd0 2023-05-16T07:19:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc34890e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc344155d70 con 0x7fc3441517f0 2023-05-16T07:19:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc34890e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440a8030 con 0x7fc3440a76f0 2023-05-16T07:19:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc344151bd0 0x7fc344152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc344151bd0 0x7fc344152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48984/0 (socket says 172.21.15.92:48984) 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 -- 172.21.15.92:0/3008830940 learned_addr learned my addr 172.21.15.92:0/3008830940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc343fff700 1 --2- 172.21.15.92:0/3008830940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3440a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc3437fe700 1 --2- 172.21.15.92:0/3008830940 >> v2:172.21.15.158:3300/0 conn(0x7fc3441517f0 0x7fc344156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 -- 172.21.15.92:0/3008830940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 msgr2=0x7fc3440a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 --2- 172.21.15.92:0/3008830940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3440a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 -- 172.21.15.92:0/3008830940 >> v2:172.21.15.158:3300/0 conn(0x7fc3441517f0 msgr2=0x7fc344156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 --2- 172.21.15.92:0/3008830940 >> v2:172.21.15.158:3300/0 conn(0x7fc3441517f0 0x7fc344156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 -- 172.21.15.92:0/3008830940 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440c8470 con 0x7fc344151bd0 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc342ffd700 1 --2- 172.21.15.92:0/3008830940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bd0 0x7fc344152a60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fc344060670 tx=0x7fc3380090f0).ready entity=mon.0 client_cookie=f1d51c87f537a182 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.264+0000 7fc3427fc700 1 -- 172.21.15.92:0/3008830940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800b600 con 0x7fc344151bd0 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc3427fc700 1 -- 172.21.15.92:0/3008830940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800b760 con 0x7fc344151bd0 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc3427fc700 1 -- 172.21.15.92:0/3008830940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800bb00 con 0x7fc344151bd0 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 -- 172.21.15.92:0/3008830940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bd0 msgr2=0x7fc344152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 --2- 172.21.15.92:0/3008830940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bd0 0x7fc344152a60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fc344060670 tx=0x7fc3380090f0).stop 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 -- 172.21.15.92:0/3008830940 shutdown_connections 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 --2- 172.21.15.92:0/3008830940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3440a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 --2- 172.21.15.92:0/3008830940 >> v2:172.21.15.158:3300/0 conn(0x7fc3441517f0 0x7fc344156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 --2- 172.21.15.92:0/3008830940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc344151bd0 0x7fc344152a60 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 -- 172.21.15.92:0/3008830940 >> 172.21.15.92:0/3008830940 conn(0x7fc34405e820 msgr2=0x7fc3440622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 -- 172.21.15.92:0/3008830940 shutdown_connections 2023-05-16T07:19:22.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.265+0000 7fc34890e700 1 -- 172.21.15.92:0/3008830940 wait complete. 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.266+0000 7fc34890e700 1 Processor -- start 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.266+0000 7fc34890e700 1 -- start start 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc34890e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3441676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc34890e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 0x7fc34414a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc34890e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc344151bd0 0x7fc3440c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc34890e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344068700 con 0x7fc3441517f0 2023-05-16T07:19:22.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc34890e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440683d0 con 0x7fc344151bd0 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc34890e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc344167fb0 con 0x7fc3440a76f0 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 0x7fc34414a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 0x7fc34414a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48990/0 (socket says 172.21.15.92:48990) 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 -- 172.21.15.92:0/1630798090 learned_addr learned my addr 172.21.15.92:0/1630798090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc3437fe700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3441676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc343fff700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc344151bd0 0x7fc3440c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 -- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 msgr2=0x7fc3441676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3441676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 -- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc344151bd0 msgr2=0x7fc3440c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc344151bd0 0x7fc3440c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.267+0000 7fc342ffd700 1 -- 172.21.15.92:0/1630798090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009370 con 0x7fc3441517f0 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.268+0000 7fc342ffd700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 0x7fc34414a590 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fc34405fd20 tx=0x7fc338017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.268+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338006d90 con 0x7fc3441517f0 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.268+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800bec0 con 0x7fc3441517f0 2023-05-16T07:19:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.268+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc344168290 con 0x7fc3441517f0 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.268+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800b8d0 con 0x7fc3441517f0 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.268+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344167ba0 con 0x7fc3441517f0 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.269+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344047150 con 0x7fc3441517f0 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.269+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc33802c070 con 0x7fc3441517f0 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.269+0000 7fc32b7fe700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc33005ed90 0x7fc330061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.269+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fc33807b630 con 0x7fc3441517f0 2023-05-16T07:19:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.269+0000 7fc3437fe700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc33005ed90 0x7fc330061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.270+0000 7fc3437fe700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc33005ed90 0x7fc330061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc334007680 tx=0x7fc3340098d0).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.273+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc33804c970 con 0x7fc3441517f0 2023-05-16T07:19:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.430+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc344155720 con 0x7fc33005ed90 2023-05-16T07:19:22.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.441+0000 7fc32b7fe700 1 -- 172.21.15.92:0/1630798090 <== mgr.6236 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361465 (secure 0 0 0) 0x7fc344155720 con 0x7fc33005ed90 2023-05-16T07:19:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc33005ed90 msgr2=0x7fc330061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc33005ed90 0x7fc330061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc334007680 tx=0x7fc3340098d0).stop 2023-05-16T07:19:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 msgr2=0x7fc34414a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 0x7fc34414a590 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fc34405fd20 tx=0x7fc338017000).stop 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 shutdown_connections 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc33005ed90 0x7fc330061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc3440a76f0 0x7fc3441676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc344151bd0 0x7fc3440c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 --2- 172.21.15.92:0/1630798090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3441517f0 0x7fc34414a590 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 >> 172.21.15.92:0/1630798090 conn(0x7fc34405e820 msgr2=0x7fc34414b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 shutdown_connections 2023-05-16T07:19:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.444+0000 7fc34890e700 1 -- 172.21.15.92:0/1630798090 wait complete. 2023-05-16T07:19:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:19:22.460 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:19:20.576626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":169004,"kb_used_data":11500,"kb_used_omap":0,"kb_used_meta":157440,"kb_avail":754805716,"statfs":{"total":773094113280,"available":772921053184,"internally_reserved":0,"allocated":11776000,"data_stored":6149015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":161218560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19801,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.585523"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160631+0000","last_change":"2023-05-16T07:19:19.160631+0000","last_active":"2023-05-16T07:19:19.160631+0000","last_peered":"2023-05-16T07:19:19.160631+0000","last_clean":"2023-05-16T07:19:19.160631+0000","last_became_active":"2023-05-16T07:19:19.160384+0000","last_became_peered":"2023-05-16T07:19:19.160384+0000","last_unstale":"2023-05-16T07:19:19.160631+0000","last_undegraded":"2023-05-16T07:19:19.160631+0000","last_fullsized":"2023-05-16T07:19:19.160631+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3f","version":"120'3","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160574+0000","last_change":"2023-05-16T07:19:18.782301+0000","last_active":"2023-05-16T07:19:19.160574+0000","last_peered":"2023-05-16T07:19:19.160574+0000","last_clean":"2023-05-16T07:19:19.160574+0000","last_became_active":"2023-05-16T07:19:18.782063+0000","last_became_peered":"2023-05-16T07:19:18.782063+0000","last_unstale":"2023-05-16T07:19:19.160574+0000","last_undegraded":"2023-05-16T07:19:19.160574+0000","last_fullsized":"2023-05-16T07:19:19.160574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156268+0000","last_change":"2023-05-16T07:19:19.156268+0000","last_active":"2023-05-16T07:19:19.156268+0000","last_peered":"2023-05-16T07:19:19.156268+0000","last_clean":"2023-05-16T07:19:19.156268+0000","last_became_active":"2023-05-16T07:19:19.156004+0000","last_became_peered":"2023-05-16T07:19:19.156004+0000","last_unstale":"2023-05-16T07:19:19.156268+0000","last_undegraded":"2023-05-16T07:19:19.156268+0000","last_fullsized":"2023-05-16T07:19:19.156268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785162+0000","last_change":"2023-05-16T07:19:18.781986+0000","last_active":"2023-05-16T07:19:18.785162+0000","last_peered":"2023-05-16T07:19:18.785162+0000","last_clean":"2023-05-16T07:19:18.785162+0000","last_became_active":"2023-05-16T07:19:18.781544+0000","last_became_peered":"2023-05-16T07:19:18.781544+0000","last_unstale":"2023-05-16T07:19:18.785162+0000","last_undegraded":"2023-05-16T07:19:18.785162+0000","last_fullsized":"2023-05-16T07:19:18.785162+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155389+0000","last_change":"2023-05-16T07:19:19.155389+0000","last_active":"2023-05-16T07:19:19.155389+0000","last_peered":"2023-05-16T07:19:19.155389+0000","last_clean":"2023-05-16T07:19:19.155389+0000","last_became_active":"2023-05-16T07:19:19.155088+0000","last_became_peered":"2023-05-16T07:19:19.155088+0000","last_unstale":"2023-05-16T07:19:19.155389+0000","last_undegraded":"2023-05-16T07:19:19.155389+0000","last_fullsized":"2023-05-16T07:19:19.155389+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788064+0000","last_change":"2023-05-16T07:19:18.781229+0000","last_active":"2023-05-16T07:19:18.788064+0000","last_peered":"2023-05-16T07:19:18.788064+0000","last_clean":"2023-05-16T07:19:18.788064+0000","last_became_active":"2023-05-16T07:19:18.780926+0000","last_became_peered":"2023-05-16T07:19:18.780926+0000","last_unstale":"2023-05-16T07:19:18.788064+0000","last_undegraded":"2023-05-16T07:19:18.788064+0000","last_fullsized":"2023-05-16T07:19:18.788064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160344+0000","last_change":"2023-05-16T07:19:19.160344+0000","last_active":"2023-05-16T07:19:19.160344+0000","last_peered":"2023-05-16T07:19:19.160344+0000","last_clean":"2023-05-16T07:19:19.160344+0000","last_became_active":"2023-05-16T07:19:19.160072+0000","last_became_peered":"2023-05-16T07:19:19.160072+0000","last_unstale":"2023-05-16T07:19:19.160344+0000","last_undegraded":"2023-05-16T07:19:19.160344+0000","last_fullsized":"2023-05-16T07:19:19.160344+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3c","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162021+0000","last_change":"2023-05-16T07:19:18.786413+0000","last_active":"2023-05-16T07:19:19.162021+0000","last_peered":"2023-05-16T07:19:19.162021+0000","last_clean":"2023-05-16T07:19:19.162021+0000","last_became_active":"2023-05-16T07:19:18.786150+0000","last_became_peered":"2023-05-16T07:19:18.786150+0000","last_unstale":"2023-05-16T07:19:19.162021+0000","last_undegraded":"2023-05-16T07:19:19.162021+0000","last_fullsized":"2023-05-16T07:19:19.162021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.143335+0000","last_change":"2023-05-16T07:19:19.143335+0000","last_active":"2023-05-16T07:19:19.143335+0000","last_peered":"2023-05-16T07:19:19.143335+0000","last_clean":"2023-05-16T07:19:19.143335+0000","last_became_active":"2023-05-16T07:19:19.143087+0000","last_became_peered":"2023-05-16T07:19:19.143087+0000","last_unstale":"2023-05-16T07:19:19.143335+0000","last_undegraded":"2023-05-16T07:19:19.143335+0000","last_fullsized":"2023-05-16T07:19:19.143335+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.138956+0000","last_change":"2023-05-16T07:19:18.778239+0000","last_active":"2023-05-16T07:19:19.138956+0000","last_peered":"2023-05-16T07:19:19.138956+0000","last_clean":"2023-05-16T07:19:19.138956+0000","last_became_active":"2023-05-16T07:19:18.777948+0000","last_became_peered":"2023-05-16T07:19:18.777948+0000","last_unstale":"2023-05-16T07:19:19.138956+0000","last_undegraded":"2023-05-16T07:19:19.138956+0000","last_fullsized":"2023-05-16T07:19:19.138956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154029+0000","last_change":"2023-05-16T07:19:19.154029+0000","last_active":"2023-05-16T07:19:19.154029+0000","last_peered":"2023-05-16T07:19:19.154029+0000","last_clean":"2023-05-16T07:19:19.154029+0000","last_became_active":"2023-05-16T07:19:19.153229+0000","last_became_peered":"2023-05-16T07:19:19.153229+0000","last_unstale":"2023-05-16T07:19:19.154029+0000","last_undegraded":"2023-05-16T07:19:19.154029+0000","last_fullsized":"2023-05-16T07:19:19.154029+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3a","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.145943+0000","last_change":"2023-05-16T07:19:18.782257+0000","last_active":"2023-05-16T07:19:19.145943+0000","last_peered":"2023-05-16T07:19:19.145943+0000","last_clean":"2023-05-16T07:19:19.145943+0000","last_became_active":"2023-05-16T07:19:18.781979+0000","last_became_peered":"2023-05-16T07:19:18.781979+0000","last_unstale":"2023-05-16T07:19:19.145943+0000","last_undegraded":"2023-05-16T07:19:19.145943+0000","last_fullsized":"2023-05-16T07:19:19.145943+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163572+0000","last_change":"2023-05-16T07:19:19.163572+0000","last_active":"2023-05-16T07:19:19.163572+0000","last_peered":"2023-05-16T07:19:19.163572+0000","last_clean":"2023-05-16T07:19:19.163572+0000","last_became_active":"2023-05-16T07:19:19.163287+0000","last_became_peered":"2023-05-16T07:19:19.163287+0000","last_unstale":"2023-05-16T07:19:19.163572+0000","last_undegraded":"2023-05-16T07:19:19.163572+0000","last_fullsized":"2023-05-16T07:19:19.163572+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782744+0000","last_change":"2023-05-16T07:19:18.781649+0000","last_active":"2023-05-16T07:19:18.782744+0000","last_peered":"2023-05-16T07:19:18.782744+0000","last_clean":"2023-05-16T07:19:18.782744+0000","last_became_active":"2023-05-16T07:19:18.781374+0000","last_became_peered":"2023-05-16T07:19:18.781374+0000","last_unstale":"2023-05-16T07:19:18.782744+0000","last_undegraded":"2023-05-16T07:19:18.782744+0000","last_fullsized":"2023-05-16T07:19:18.782744+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154825+0000","last_change":"2023-05-16T07:19:19.154825+0000","last_active":"2023-05-16T07:19:19.154825+0000","last_peered":"2023-05-16T07:19:19.154825+0000","last_clean":"2023-05-16T07:19:19.154825+0000","last_became_active":"2023-05-16T07:19:19.154536+0000","last_became_peered":"2023-05-16T07:19:19.154536+0000","last_unstale":"2023-05-16T07:19:19.154825+0000","last_undegraded":"2023-05-16T07:19:19.154825+0000","last_fullsized":"2023-05-16T07:19:19.154825+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782952+0000","last_change":"2023-05-16T07:19:18.777066+0000","last_active":"2023-05-16T07:19:18.782952+0000","last_peered":"2023-05-16T07:19:18.782952+0000","last_clean":"2023-05-16T07:19:18.782952+0000","last_became_active":"2023-05-16T07:19:18.776803+0000","last_became_peered":"2023-05-16T07:19:18.776803+0000","last_unstale":"2023-05-16T07:19:18.782952+0000","last_undegraded":"2023-05-16T07:19:18.782952+0000","last_fullsized":"2023-05-16T07:19:18.782952+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158474+0000","last_change":"2023-05-16T07:19:19.158474+0000","last_active":"2023-05-16T07:19:19.158474+0000","last_peered":"2023-05-16T07:19:19.158474+0000","last_clean":"2023-05-16T07:19:19.158474+0000","last_became_active":"2023-05-16T07:19:19.158233+0000","last_became_peered":"2023-05-16T07:19:19.158233+0000","last_unstale":"2023-05-16T07:19:19.158474+0000","last_undegraded":"2023-05-16T07:19:19.158474+0000","last_fullsized":"2023-05-16T07:19:19.158474+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789462+0000","last_change":"2023-05-16T07:19:18.787944+0000","last_active":"2023-05-16T07:19:18.789462+0000","last_peered":"2023-05-16T07:19:18.789462+0000","last_clean":"2023-05-16T07:19:18.789462+0000","last_became_active":"2023-05-16T07:19:18.786678+0000","last_became_peered":"2023-05-16T07:19:18.786678+0000","last_unstale":"2023-05-16T07:19:18.789462+0000","last_undegraded":"2023-05-16T07:19:18.789462+0000","last_fullsized":"2023-05-16T07:19:18.789462+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158028+0000","last_change":"2023-05-16T07:19:19.158028+0000","last_active":"2023-05-16T07:19:19.158028+0000","last_peered":"2023-05-16T07:19:19.158028+0000","last_clean":"2023-05-16T07:19:19.158028+0000","last_became_active":"2023-05-16T07:19:19.156920+0000","last_became_peered":"2023-05-16T07:19:19.156920+0000","last_unstale":"2023-05-16T07:19:19.158028+0000","last_undegraded":"2023-05-16T07:19:19.158028+0000","last_fullsized":"2023-05-16T07:19:19.158028+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.786887+0000","last_change":"2023-05-16T07:19:18.782689+0000","last_active":"2023-05-16T07:19:18.786887+0000","last_peered":"2023-05-16T07:19:18.786887+0000","last_clean":"2023-05-16T07:19:18.786887+0000","last_became_active":"2023-05-16T07:19:18.782360+0000","last_became_peered":"2023-05-16T07:19:18.782360+0000","last_unstale":"2023-05-16T07:19:18.786887+0000","last_undegraded":"2023-05-16T07:19:18.786887+0000","last_fullsized":"2023-05-16T07:19:18.786887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150377+0000","last_change":"2023-05-16T07:19:19.150377+0000","last_active":"2023-05-16T07:19:19.150377+0000","last_peered":"2023-05-16T07:19:19.150377+0000","last_clean":"2023-05-16T07:19:19.150377+0000","last_became_active":"2023-05-16T07:19:19.147947+0000","last_became_peered":"2023-05-16T07:19:19.147947+0000","last_unstale":"2023-05-16T07:19:19.150377+0000","last_undegraded":"2023-05-16T07:19:19.150377+0000","last_fullsized":"2023-05-16T07:19:19.150377+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.786833+0000","last_change":"2023-05-16T07:19:18.785233+0000","last_active":"2023-05-16T07:19:18.786833+0000","last_peered":"2023-05-16T07:19:18.786833+0000","last_clean":"2023-05-16T07:19:18.786833+0000","last_became_active":"2023-05-16T07:19:18.784906+0000","last_became_peered":"2023-05-16T07:19:18.784906+0000","last_unstale":"2023-05-16T07:19:18.786833+0000","last_undegraded":"2023-05-16T07:19:18.786833+0000","last_fullsized":"2023-05-16T07:19:18.786833+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.161966+0000","last_change":"2023-05-16T07:19:19.161966+0000","last_active":"2023-05-16T07:19:19.161966+0000","last_peered":"2023-05-16T07:19:19.161966+0000","last_clean":"2023-05-16T07:19:19.161966+0000","last_became_active":"2023-05-16T07:19:19.161677+0000","last_became_peered":"2023-05-16T07:19:19.161677+0000","last_unstale":"2023-05-16T07:19:19.161966+0000","last_undegraded":"2023-05-16T07:19:19.161966+0000","last_fullsized":"2023-05-16T07:19:19.161966+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.34","version":"120'4","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.166911+0000","last_change":"2023-05-16T07:19:18.784263+0000","last_active":"2023-05-16T07:19:19.166911+0000","last_peered":"2023-05-16T07:19:19.166911+0000","last_clean":"2023-05-16T07:19:19.166911+0000","last_became_active":"2023-05-16T07:19:18.783977+0000","last_became_peered":"2023-05-16T07:19:18.783977+0000","last_unstale":"2023-05-16T07:19:19.166911+0000","last_undegraded":"2023-05-16T07:19:19.166911+0000","last_fullsized":"2023-05-16T07:19:19.166911+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155157+0000","last_change":"2023-05-16T07:19:19.155157+0000","last_active":"2023-05-16T07:19:19.155157+0000","last_peered":"2023-05-16T07:19:19.155157+0000","last_clean":"2023-05-16T07:19:19.155157+0000","last_became_active":"2023-05-16T07:19:19.154884+0000","last_became_peered":"2023-05-16T07:19:19.154884+0000","last_unstale":"2023-05-16T07:19:19.155157+0000","last_undegraded":"2023-05-16T07:19:19.155157+0000","last_fullsized":"2023-05-16T07:19:19.155157+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.147016+0000","last_change":"2023-05-16T07:19:18.784767+0000","last_active":"2023-05-16T07:19:19.147016+0000","last_peered":"2023-05-16T07:19:19.147016+0000","last_clean":"2023-05-16T07:19:19.147016+0000","last_became_active":"2023-05-16T07:19:18.784496+0000","last_became_peered":"2023-05-16T07:19:18.784496+0000","last_unstale":"2023-05-16T07:19:19.147016+0000","last_undegraded":"2023-05-16T07:19:19.147016+0000","last_fullsized":"2023-05-16T07:19:19.147016+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162896+0000","last_change":"2023-05-16T07:19:19.162896+0000","last_active":"2023-05-16T07:19:19.162896+0000","last_peered":"2023-05-16T07:19:19.162896+0000","last_clean":"2023-05-16T07:19:19.162896+0000","last_became_active":"2023-05-16T07:19:19.162627+0000","last_became_peered":"2023-05-16T07:19:19.162627+0000","last_unstale":"2023-05-16T07:19:19.162896+0000","last_undegraded":"2023-05-16T07:19:19.162896+0000","last_fullsized":"2023-05-16T07:19:19.162896+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785787+0000","last_change":"2023-05-16T07:19:18.775367+0000","last_active":"2023-05-16T07:19:18.785787+0000","last_peered":"2023-05-16T07:19:18.785787+0000","last_clean":"2023-05-16T07:19:18.785787+0000","last_became_active":"2023-05-16T07:19:18.775120+0000","last_became_peered":"2023-05-16T07:19:18.775120+0000","last_unstale":"2023-05-16T07:19:18.785787+0000","last_undegraded":"2023-05-16T07:19:18.785787+0000","last_fullsized":"2023-05-16T07:19:18.785787+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156980+0000","last_change":"2023-05-16T07:19:19.156980+0000","last_active":"2023-05-16T07:19:19.156980+0000","last_peered":"2023-05-16T07:19:19.156980+0000","last_clean":"2023-05-16T07:19:19.156980+0000","last_became_active":"2023-05-16T07:19:19.155806+0000","last_became_peered":"2023-05-16T07:19:19.155806+0000","last_unstale":"2023-05-16T07:19:19.156980+0000","last_undegraded":"2023-05-16T07:19:19.156980+0000","last_fullsized":"2023-05-16T07:19:19.156980+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.793085+0000","last_change":"2023-05-16T07:19:18.791619+0000","last_active":"2023-05-16T07:19:18.793085+0000","last_peered":"2023-05-16T07:19:18.793085+0000","last_clean":"2023-05-16T07:19:18.793085+0000","last_became_active":"2023-05-16T07:19:18.791283+0000","last_became_peered":"2023-05-16T07:19:18.791283+0000","last_unstale":"2023-05-16T07:19:18.793085+0000","last_undegraded":"2023-05-16T07:19:18.793085+0000","last_fullsized":"2023-05-16T07:19:18.793085+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154166+0000","last_change":"2023-05-16T07:19:19.154166+0000","last_active":"2023-05-16T07:19:19.154166+0000","last_peered":"2023-05-16T07:19:19.154166+0000","last_clean":"2023-05-16T07:19:19.154166+0000","last_became_active":"2023-05-16T07:19:19.153660+0000","last_became_peered":"2023-05-16T07:19:19.153660+0000","last_unstale":"2023-05-16T07:19:19.154166+0000","last_undegraded":"2023-05-16T07:19:19.154166+0000","last_fullsized":"2023-05-16T07:19:19.154166+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"120'4","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.157632+0000","last_change":"2023-05-16T07:19:18.778671+0000","last_active":"2023-05-16T07:19:19.157632+0000","last_peered":"2023-05-16T07:19:19.157632+0000","last_clean":"2023-05-16T07:19:19.157632+0000","last_became_active":"2023-05-16T07:19:18.778177+0000","last_became_peered":"2023-05-16T07:19:18.778177+0000","last_unstale":"2023-05-16T07:19:19.157632+0000","last_undegraded":"2023-05-16T07:19:19.157632+0000","last_fullsized":"2023-05-16T07:19:19.157632+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.157155+0000","last_change":"2023-05-16T07:19:19.157155+0000","last_active":"2023-05-16T07:19:19.157155+0000","last_peered":"2023-05-16T07:19:19.157155+0000","last_clean":"2023-05-16T07:19:19.157155+0000","last_became_active":"2023-05-16T07:19:19.156870+0000","last_became_peered":"2023-05-16T07:19:19.156870+0000","last_unstale":"2023-05-16T07:19:19.157155+0000","last_undegraded":"2023-05-16T07:19:19.157155+0000","last_fullsized":"2023-05-16T07:19:19.157155+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.791000+0000","last_change":"2023-05-16T07:19:18.787300+0000","last_active":"2023-05-16T07:19:18.791000+0000","last_peered":"2023-05-16T07:19:18.791000+0000","last_clean":"2023-05-16T07:19:18.791000+0000","last_became_active":"2023-05-16T07:19:18.786965+0000","last_became_peered":"2023-05-16T07:19:18.786965+0000","last_unstale":"2023-05-16T07:19:18.791000+0000","last_undegraded":"2023-05-16T07:19:18.791000+0000","last_fullsized":"2023-05-16T07:19:18.791000+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156626+0000","last_change":"2023-05-16T07:19:19.156626+0000","last_active":"2023-05-16T07:19:19.156626+0000","last_peered":"2023-05-16T07:19:19.156626+0000","last_clean":"2023-05-16T07:19:19.156626+0000","last_became_active":"2023-05-16T07:19:19.156350+0000","last_became_peered":"2023-05-16T07:19:19.156350+0000","last_unstale":"2023-05-16T07:19:19.156626+0000","last_undegraded":"2023-05-16T07:19:19.156626+0000","last_fullsized":"2023-05-16T07:19:19.156626+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789644+0000","last_change":"2023-05-16T07:19:18.786358+0000","last_active":"2023-05-16T07:19:18.789644+0000","last_peered":"2023-05-16T07:19:18.789644+0000","last_clean":"2023-05-16T07:19:18.789644+0000","last_became_active":"2023-05-16T07:19:18.786158+0000","last_became_peered":"2023-05-16T07:19:18.786158+0000","last_unstale":"2023-05-16T07:19:18.789644+0000","last_undegraded":"2023-05-16T07:19:18.789644+0000","last_fullsized":"2023-05-16T07:19:18.789644+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151693+0000","last_change":"2023-05-16T07:19:19.151693+0000","last_active":"2023-05-16T07:19:19.151693+0000","last_peered":"2023-05-16T07:19:19.151693+0000","last_clean":"2023-05-16T07:19:19.151693+0000","last_became_active":"2023-05-16T07:19:19.151384+0000","last_became_peered":"2023-05-16T07:19:19.151384+0000","last_unstale":"2023-05-16T07:19:19.151693+0000","last_undegraded":"2023-05-16T07:19:19.151693+0000","last_fullsized":"2023-05-16T07:19:19.151693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2d","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.149215+0000","last_change":"2023-05-16T07:19:18.782930+0000","last_active":"2023-05-16T07:19:19.149215+0000","last_peered":"2023-05-16T07:19:19.149215+0000","last_clean":"2023-05-16T07:19:19.149215+0000","last_became_active":"2023-05-16T07:19:18.782642+0000","last_became_peered":"2023-05-16T07:19:18.782642+0000","last_unstale":"2023-05-16T07:19:19.149215+0000","last_undegraded":"2023-05-16T07:19:19.149215+0000","last_fullsized":"2023-05-16T07:19:19.149215+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151297+0000","last_change":"2023-05-16T07:19:19.151297+0000","last_active":"2023-05-16T07:19:19.151297+0000","last_peered":"2023-05-16T07:19:19.151297+0000","last_clean":"2023-05-16T07:19:19.151297+0000","last_became_active":"2023-05-16T07:19:19.151054+0000","last_became_peered":"2023-05-16T07:19:19.151054+0000","last_unstale":"2023-05-16T07:19:19.151297+0000","last_undegraded":"2023-05-16T07:19:19.151297+0000","last_fullsized":"2023-05-16T07:19:19.151297+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789440+0000","last_change":"2023-05-16T07:19:18.786711+0000","last_active":"2023-05-16T07:19:18.789440+0000","last_peered":"2023-05-16T07:19:18.789440+0000","last_clean":"2023-05-16T07:19:18.789440+0000","last_became_active":"2023-05-16T07:19:18.786313+0000","last_became_peered":"2023-05-16T07:19:18.786313+0000","last_unstale":"2023-05-16T07:19:18.789440+0000","last_undegraded":"2023-05-16T07:19:18.789440+0000","last_fullsized":"2023-05-16T07:19:18.789440+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156954+0000","last_change":"2023-05-16T07:19:19.156954+0000","last_active":"2023-05-16T07:19:19.156954+0000","last_peered":"2023-05-16T07:19:19.156954+0000","last_clean":"2023-05-16T07:19:19.156954+0000","last_became_active":"2023-05-16T07:19:19.156586+0000","last_became_peered":"2023-05-16T07:19:19.156586+0000","last_unstale":"2023-05-16T07:19:19.156954+0000","last_undegraded":"2023-05-16T07:19:19.156954+0000","last_fullsized":"2023-05-16T07:19:19.156954+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155920+0000","last_change":"2023-05-16T07:19:18.780770+0000","last_active":"2023-05-16T07:19:19.155920+0000","last_peered":"2023-05-16T07:19:19.155920+0000","last_clean":"2023-05-16T07:19:19.155920+0000","last_became_active":"2023-05-16T07:19:18.780297+0000","last_became_peered":"2023-05-16T07:19:18.780297+0000","last_unstale":"2023-05-16T07:19:19.155920+0000","last_undegraded":"2023-05-16T07:19:19.155920+0000","last_fullsized":"2023-05-16T07:19:19.155920+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.144377+0000","last_change":"2023-05-16T07:19:19.144377+0000","last_active":"2023-05-16T07:19:19.144377+0000","last_peered":"2023-05-16T07:19:19.144377+0000","last_clean":"2023-05-16T07:19:19.144377+0000","last_became_active":"2023-05-16T07:19:19.143744+0000","last_became_peered":"2023-05-16T07:19:19.143744+0000","last_unstale":"2023-05-16T07:19:19.144377+0000","last_undegraded":"2023-05-16T07:19:19.144377+0000","last_fullsized":"2023-05-16T07:19:19.144377+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785096+0000","last_change":"2023-05-16T07:19:18.781983+0000","last_active":"2023-05-16T07:19:18.785096+0000","last_peered":"2023-05-16T07:19:18.785096+0000","last_clean":"2023-05-16T07:19:18.785096+0000","last_became_active":"2023-05-16T07:19:18.781154+0000","last_became_peered":"2023-05-16T07:19:18.781154+0000","last_unstale":"2023-05-16T07:19:18.785096+0000","last_undegraded":"2023-05-16T07:19:18.785096+0000","last_fullsized":"2023-05-16T07:19:18.785096+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163919+0000","last_change":"2023-05-16T07:19:19.163919+0000","last_active":"2023-05-16T07:19:19.163919+0000","last_peered":"2023-05-16T07:19:19.163919+0000","last_clean":"2023-05-16T07:19:19.163919+0000","last_became_active":"2023-05-16T07:19:19.163660+0000","last_became_peered":"2023-05-16T07:19:19.163660+0000","last_unstale":"2023-05-16T07:19:19.163919+0000","last_undegraded":"2023-05-16T07:19:19.163919+0000","last_fullsized":"2023-05-16T07:19:19.163919+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785118+0000","last_change":"2023-05-16T07:19:18.781673+0000","last_active":"2023-05-16T07:19:18.785118+0000","last_peered":"2023-05-16T07:19:18.785118+0000","last_clean":"2023-05-16T07:19:18.785118+0000","last_became_active":"2023-05-16T07:19:18.781407+0000","last_became_peered":"2023-05-16T07:19:18.781407+0000","last_unstale":"2023-05-16T07:19:18.785118+0000","last_undegraded":"2023-05-16T07:19:18.785118+0000","last_fullsized":"2023-05-16T07:19:18.785118+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.159888+0000","last_change":"2023-05-16T07:19:19.159888+0000","last_active":"2023-05-16T07:19:19.159888+0000","last_peered":"2023-05-16T07:19:19.159888+0000","last_clean":"2023-05-16T07:19:19.159888+0000","last_became_active":"2023-05-16T07:19:19.159612+0000","last_became_peered":"2023-05-16T07:19:19.159612+0000","last_unstale":"2023-05-16T07:19:19.159888+0000","last_undegraded":"2023-05-16T07:19:19.159888+0000","last_fullsized":"2023-05-16T07:19:19.159888+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785856+0000","last_change":"2023-05-16T07:19:18.783986+0000","last_active":"2023-05-16T07:19:18.785856+0000","last_peered":"2023-05-16T07:19:18.785856+0000","last_clean":"2023-05-16T07:19:18.785856+0000","last_became_active":"2023-05-16T07:19:18.783715+0000","last_became_peered":"2023-05-16T07:19:18.783715+0000","last_unstale":"2023-05-16T07:19:18.785856+0000","last_undegraded":"2023-05-16T07:19:18.785856+0000","last_fullsized":"2023-05-16T07:19:18.785856+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163156+0000","last_change":"2023-05-16T07:19:19.163156+0000","last_active":"2023-05-16T07:19:19.163156+0000","last_peered":"2023-05-16T07:19:19.163156+0000","last_clean":"2023-05-16T07:19:19.163156+0000","last_became_active":"2023-05-16T07:19:19.162908+0000","last_became_peered":"2023-05-16T07:19:19.162908+0000","last_unstale":"2023-05-16T07:19:19.163156+0000","last_undegraded":"2023-05-16T07:19:19.163156+0000","last_fullsized":"2023-05-16T07:19:19.163156+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.786995+0000","last_change":"2023-05-16T07:19:18.784135+0000","last_active":"2023-05-16T07:19:18.786995+0000","last_peered":"2023-05-16T07:19:18.786995+0000","last_clean":"2023-05-16T07:19:18.786995+0000","last_became_active":"2023-05-16T07:19:18.783826+0000","last_became_peered":"2023-05-16T07:19:18.783826+0000","last_unstale":"2023-05-16T07:19:18.786995+0000","last_undegraded":"2023-05-16T07:19:18.786995+0000","last_fullsized":"2023-05-16T07:19:18.786995+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155467+0000","last_change":"2023-05-16T07:19:19.155467+0000","last_active":"2023-05-16T07:19:19.155467+0000","last_peered":"2023-05-16T07:19:19.155467+0000","last_clean":"2023-05-16T07:19:19.155467+0000","last_became_active":"2023-05-16T07:19:19.154352+0000","last_became_peered":"2023-05-16T07:19:19.154352+0000","last_unstale":"2023-05-16T07:19:19.155467+0000","last_undegraded":"2023-05-16T07:19:19.155467+0000","last_fullsized":"2023-05-16T07:19:19.155467+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.796515+0000","last_change":"2023-05-16T07:19:18.792552+0000","last_active":"2023-05-16T07:19:18.796515+0000","last_peered":"2023-05-16T07:19:18.796515+0000","last_clean":"2023-05-16T07:19:18.796515+0000","last_became_active":"2023-05-16T07:19:18.792307+0000","last_became_peered":"2023-05-16T07:19:18.792307+0000","last_unstale":"2023-05-16T07:19:18.796515+0000","last_undegraded":"2023-05-16T07:19:18.796515+0000","last_fullsized":"2023-05-16T07:19:18.796515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156553+0000","last_change":"2023-05-16T07:19:19.156553+0000","last_active":"2023-05-16T07:19:19.156553+0000","last_peered":"2023-05-16T07:19:19.156553+0000","last_clean":"2023-05-16T07:19:19.156553+0000","last_became_active":"2023-05-16T07:19:19.155443+0000","last_became_peered":"2023-05-16T07:19:19.155443+0000","last_unstale":"2023-05-16T07:19:19.156553+0000","last_undegraded":"2023-05-16T07:19:19.156553+0000","last_fullsized":"2023-05-16T07:19:19.156553+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.772684+0000","last_change":"2023-05-16T07:19:18.771682+0000","last_active":"2023-05-16T07:19:18.772684+0000","last_peered":"2023-05-16T07:19:18.772684+0000","last_clean":"2023-05-16T07:19:18.772684+0000","last_became_active":"2023-05-16T07:19:18.771355+0000","last_became_peered":"2023-05-16T07:19:18.771355+0000","last_unstale":"2023-05-16T07:19:18.772684+0000","last_undegraded":"2023-05-16T07:19:18.772684+0000","last_fullsized":"2023-05-16T07:19:18.772684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156376+0000","last_change":"2023-05-16T07:19:19.156376+0000","last_active":"2023-05-16T07:19:19.156376+0000","last_peered":"2023-05-16T07:19:19.156376+0000","last_clean":"2023-05-16T07:19:19.156376+0000","last_became_active":"2023-05-16T07:19:19.156089+0000","last_became_peered":"2023-05-16T07:19:19.156089+0000","last_unstale":"2023-05-16T07:19:19.156376+0000","last_undegraded":"2023-05-16T07:19:19.156376+0000","last_fullsized":"2023-05-16T07:19:19.156376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.787284+0000","last_change":"2023-05-16T07:19:18.786458+0000","last_active":"2023-05-16T07:19:18.787284+0000","last_peered":"2023-05-16T07:19:18.787284+0000","last_clean":"2023-05-16T07:19:18.787284+0000","last_became_active":"2023-05-16T07:19:18.786185+0000","last_became_peered":"2023-05-16T07:19:18.786185+0000","last_unstale":"2023-05-16T07:19:18.787284+0000","last_undegraded":"2023-05-16T07:19:18.787284+0000","last_fullsized":"2023-05-16T07:19:18.787284+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156922+0000","last_change":"2023-05-16T07:19:19.156922+0000","last_active":"2023-05-16T07:19:19.156922+0000","last_peered":"2023-05-16T07:19:19.156922+0000","last_clean":"2023-05-16T07:19:19.156922+0000","last_became_active":"2023-05-16T07:19:19.156658+0000","last_became_peered":"2023-05-16T07:19:19.156658+0000","last_unstale":"2023-05-16T07:19:19.156922+0000","last_undegraded":"2023-05-16T07:19:19.156922+0000","last_fullsized":"2023-05-16T07:19:19.156922+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.796823+0000","last_change":"2023-05-16T07:19:18.794426+0000","last_active":"2023-05-16T07:19:18.796823+0000","last_peered":"2023-05-16T07:19:18.796823+0000","last_clean":"2023-05-16T07:19:18.796823+0000","last_became_active":"2023-05-16T07:19:18.794165+0000","last_became_peered":"2023-05-16T07:19:18.794165+0000","last_unstale":"2023-05-16T07:19:18.796823+0000","last_undegraded":"2023-05-16T07:19:18.796823+0000","last_fullsized":"2023-05-16T07:19:18.796823+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158694+0000","last_change":"2023-05-16T07:19:19.158694+0000","last_active":"2023-05-16T07:19:19.158694+0000","last_peered":"2023-05-16T07:19:19.158694+0000","last_clean":"2023-05-16T07:19:19.158694+0000","last_became_active":"2023-05-16T07:19:19.158445+0000","last_became_peered":"2023-05-16T07:19:19.158445+0000","last_unstale":"2023-05-16T07:19:19.158694+0000","last_undegraded":"2023-05-16T07:19:19.158694+0000","last_fullsized":"2023-05-16T07:19:19.158694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.793943+0000","last_change":"2023-05-16T07:19:18.789342+0000","last_active":"2023-05-16T07:19:18.793943+0000","last_peered":"2023-05-16T07:19:18.793943+0000","last_clean":"2023-05-16T07:19:18.793943+0000","last_became_active":"2023-05-16T07:19:18.789071+0000","last_became_peered":"2023-05-16T07:19:18.789071+0000","last_unstale":"2023-05-16T07:19:18.793943+0000","last_undegraded":"2023-05-16T07:19:18.793943+0000","last_fullsized":"2023-05-16T07:19:18.793943+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158198+0000","last_change":"2023-05-16T07:19:19.158198+0000","last_active":"2023-05-16T07:19:19.158198+0000","last_peered":"2023-05-16T07:19:19.158198+0000","last_clean":"2023-05-16T07:19:19.158198+0000","last_became_active":"2023-05-16T07:19:19.157914+0000","last_became_peered":"2023-05-16T07:19:19.157914+0000","last_unstale":"2023-05-16T07:19:19.158198+0000","last_undegraded":"2023-05-16T07:19:19.158198+0000","last_fullsized":"2023-05-16T07:19:19.158198+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788294+0000","last_change":"2023-05-16T07:19:18.780567+0000","last_active":"2023-05-16T07:19:18.788294+0000","last_peered":"2023-05-16T07:19:18.788294+0000","last_clean":"2023-05-16T07:19:18.788294+0000","last_became_active":"2023-05-16T07:19:18.780313+0000","last_became_peered":"2023-05-16T07:19:18.780313+0000","last_unstale":"2023-05-16T07:19:18.788294+0000","last_undegraded":"2023-05-16T07:19:18.788294+0000","last_fullsized":"2023-05-16T07:19:18.788294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154324+0000","last_change":"2023-05-16T07:19:19.154324+0000","last_active":"2023-05-16T07:19:19.154324+0000","last_peered":"2023-05-16T07:19:19.154324+0000","last_clean":"2023-05-16T07:19:19.154324+0000","last_became_active":"2023-05-16T07:19:19.153951+0000","last_became_peered":"2023-05-16T07:19:19.153951+0000","last_unstale":"2023-05-16T07:19:19.154324+0000","last_undegraded":"2023-05-16T07:19:19.154324+0000","last_fullsized":"2023-05-16T07:19:19.154324+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.794010+0000","last_change":"2023-05-16T07:19:18.788992+0000","last_active":"2023-05-16T07:19:18.794010+0000","last_peered":"2023-05-16T07:19:18.794010+0000","last_clean":"2023-05-16T07:19:18.794010+0000","last_became_active":"2023-05-16T07:19:18.788602+0000","last_became_peered":"2023-05-16T07:19:18.788602+0000","last_unstale":"2023-05-16T07:19:18.794010+0000","last_undegraded":"2023-05-16T07:19:18.794010+0000","last_fullsized":"2023-05-16T07:19:18.794010+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.149760+0000","last_change":"2023-05-16T07:19:19.149760+0000","last_active":"2023-05-16T07:19:19.149760+0000","last_peered":"2023-05-16T07:19:19.149760+0000","last_clean":"2023-05-16T07:19:19.149760+0000","last_became_active":"2023-05-16T07:19:19.149503+0000","last_became_peered":"2023-05-16T07:19:19.149503+0000","last_unstale":"2023-05-16T07:19:19.149760+0000","last_undegraded":"2023-05-16T07:19:19.149760+0000","last_fullsized":"2023-05-16T07:19:19.149760+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.146233+0000","last_change":"2023-05-16T07:19:18.786400+0000","last_active":"2023-05-16T07:19:19.146233+0000","last_peered":"2023-05-16T07:19:19.146233+0000","last_clean":"2023-05-16T07:19:19.146233+0000","last_became_active":"2023-05-16T07:19:18.786032+0000","last_became_peered":"2023-05-16T07:19:18.786032+0000","last_unstale":"2023-05-16T07:19:19.146233+0000","last_undegraded":"2023-05-16T07:19:19.146233+0000","last_fullsized":"2023-05-16T07:19:19.146233+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.168118+0000","last_change":"2023-05-16T07:19:19.168118+0000","last_active":"2023-05-16T07:19:19.168118+0000","last_peered":"2023-05-16T07:19:19.168118+0000","last_clean":"2023-05-16T07:19:19.168118+0000","last_became_active":"2023-05-16T07:19:19.167945+0000","last_became_peered":"2023-05-16T07:19:19.167945+0000","last_unstale":"2023-05-16T07:19:19.168118+0000","last_undegraded":"2023-05-16T07:19:19.168118+0000","last_fullsized":"2023-05-16T07:19:19.168118+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.149164+0000","last_change":"2023-05-16T07:19:18.775801+0000","last_active":"2023-05-16T07:19:19.149164+0000","last_peered":"2023-05-16T07:19:19.149164+0000","last_clean":"2023-05-16T07:19:19.149164+0000","last_became_active":"2023-05-16T07:19:18.774934+0000","last_became_peered":"2023-05-16T07:19:18.774934+0000","last_unstale":"2023-05-16T07:19:19.149164+0000","last_undegraded":"2023-05-16T07:19:19.149164+0000","last_fullsized":"2023-05-16T07:19:19.149164+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.154346+0000","last_change":"2023-05-16T07:19:19.154346+0000","last_active":"2023-05-16T07:19:19.154346+0000","last_peered":"2023-05-16T07:19:19.154346+0000","last_clean":"2023-05-16T07:19:19.154346+0000","last_became_active":"2023-05-16T07:19:19.154008+0000","last_became_peered":"2023-05-16T07:19:19.154008+0000","last_unstale":"2023-05-16T07:19:19.154346+0000","last_undegraded":"2023-05-16T07:19:19.154346+0000","last_fullsized":"2023-05-16T07:19:19.154346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158443+0000","last_change":"2023-05-16T07:19:18.784982+0000","last_active":"2023-05-16T07:19:19.158443+0000","last_peered":"2023-05-16T07:19:19.158443+0000","last_clean":"2023-05-16T07:19:19.158443+0000","last_became_active":"2023-05-16T07:19:18.783374+0000","last_became_peered":"2023-05-16T07:19:18.783374+0000","last_unstale":"2023-05-16T07:19:19.158443+0000","last_undegraded":"2023-05-16T07:19:19.158443+0000","last_fullsized":"2023-05-16T07:19:19.158443+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.159159+0000","last_change":"2023-05-16T07:19:19.159159+0000","last_active":"2023-05-16T07:19:19.159159+0000","last_peered":"2023-05-16T07:19:19.159159+0000","last_clean":"2023-05-16T07:19:19.159159+0000","last_became_active":"2023-05-16T07:19:19.158843+0000","last_became_peered":"2023-05-16T07:19:19.158843+0000","last_unstale":"2023-05-16T07:19:19.159159+0000","last_undegraded":"2023-05-16T07:19:19.159159+0000","last_fullsized":"2023-05-16T07:19:19.159159+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.792030+0000","last_change":"2023-05-16T07:19:18.785276+0000","last_active":"2023-05-16T07:19:18.792030+0000","last_peered":"2023-05-16T07:19:18.792030+0000","last_clean":"2023-05-16T07:19:18.792030+0000","last_became_active":"2023-05-16T07:19:18.785016+0000","last_became_peered":"2023-05-16T07:19:18.785016+0000","last_unstale":"2023-05-16T07:19:18.792030+0000","last_undegraded":"2023-05-16T07:19:18.792030+0000","last_fullsized":"2023-05-16T07:19:18.792030+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.167188+0000","last_change":"2023-05-16T07:19:19.167188+0000","last_active":"2023-05-16T07:19:19.167188+0000","last_peered":"2023-05-16T07:19:19.167188+0000","last_clean":"2023-05-16T07:19:19.167188+0000","last_became_active":"2023-05-16T07:19:19.166919+0000","last_became_peered":"2023-05-16T07:19:19.166919+0000","last_unstale":"2023-05-16T07:19:19.167188+0000","last_undegraded":"2023-05-16T07:19:19.167188+0000","last_fullsized":"2023-05-16T07:19:19.167188+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.772657+0000","last_change":"2023-05-16T07:19:18.771030+0000","last_active":"2023-05-16T07:19:18.772657+0000","last_peered":"2023-05-16T07:19:18.772657+0000","last_clean":"2023-05-16T07:19:18.772657+0000","last_became_active":"2023-05-16T07:19:18.770748+0000","last_became_peered":"2023-05-16T07:19:18.770748+0000","last_unstale":"2023-05-16T07:19:18.772657+0000","last_undegraded":"2023-05-16T07:19:18.772657+0000","last_fullsized":"2023-05-16T07:19:18.772657+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162746+0000","last_change":"2023-05-16T07:19:19.162746+0000","last_active":"2023-05-16T07:19:19.162746+0000","last_peered":"2023-05-16T07:19:19.162746+0000","last_clean":"2023-05-16T07:19:19.162746+0000","last_became_active":"2023-05-16T07:19:19.162307+0000","last_became_peered":"2023-05-16T07:19:19.162307+0000","last_unstale":"2023-05-16T07:19:19.162746+0000","last_undegraded":"2023-05-16T07:19:19.162746+0000","last_fullsized":"2023-05-16T07:19:19.162746+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785438+0000","last_change":"2023-05-16T07:19:18.781482+0000","last_active":"2023-05-16T07:19:18.785438+0000","last_peered":"2023-05-16T07:19:18.785438+0000","last_clean":"2023-05-16T07:19:18.785438+0000","last_became_active":"2023-05-16T07:19:18.781175+0000","last_became_peered":"2023-05-16T07:19:18.781175+0000","last_unstale":"2023-05-16T07:19:18.785438+0000","last_undegraded":"2023-05-16T07:19:18.785438+0000","last_fullsized":"2023-05-16T07:19:18.785438+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.143623+0000","last_change":"2023-05-16T07:19:19.143623+0000","last_active":"2023-05-16T07:19:19.143623+0000","last_peered":"2023-05-16T07:19:19.143623+0000","last_clean":"2023-05-16T07:19:19.143623+0000","last_became_active":"2023-05-16T07:19:19.143136+0000","last_became_peered":"2023-05-16T07:19:19.143136+0000","last_unstale":"2023-05-16T07:19:19.143623+0000","last_undegraded":"2023-05-16T07:19:19.143623+0000","last_fullsized":"2023-05-16T07:19:19.143623+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788901+0000","last_change":"2023-05-16T07:19:18.783900+0000","last_active":"2023-05-16T07:19:18.788901+0000","last_peered":"2023-05-16T07:19:18.788901+0000","last_clean":"2023-05-16T07:19:18.788901+0000","last_became_active":"2023-05-16T07:19:18.783167+0000","last_became_peered":"2023-05-16T07:19:18.783167+0000","last_unstale":"2023-05-16T07:19:18.788901+0000","last_undegraded":"2023-05-16T07:19:18.788901+0000","last_fullsized":"2023-05-16T07:19:18.788901+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155583+0000","last_change":"2023-05-16T07:19:19.155583+0000","last_active":"2023-05-16T07:19:19.155583+0000","last_peered":"2023-05-16T07:19:19.155583+0000","last_clean":"2023-05-16T07:19:19.155583+0000","last_became_active":"2023-05-16T07:19:19.155167+0000","last_became_peered":"2023-05-16T07:19:19.155167+0000","last_unstale":"2023-05-16T07:19:19.155583+0000","last_undegraded":"2023-05-16T07:19:19.155583+0000","last_fullsized":"2023-05-16T07:19:19.155583+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.793084+0000","last_change":"2023-05-16T07:19:18.791521+0000","last_active":"2023-05-16T07:19:18.793084+0000","last_peered":"2023-05-16T07:19:18.793084+0000","last_clean":"2023-05-16T07:19:18.793084+0000","last_became_active":"2023-05-16T07:19:18.791257+0000","last_became_peered":"2023-05-16T07:19:18.791257+0000","last_unstale":"2023-05-16T07:19:18.793084+0000","last_undegraded":"2023-05-16T07:19:18.793084+0000","last_fullsized":"2023-05-16T07:19:18.793084+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.153842+0000","last_change":"2023-05-16T07:19:19.153842+0000","last_active":"2023-05-16T07:19:19.153842+0000","last_peered":"2023-05-16T07:19:19.153842+0000","last_clean":"2023-05-16T07:19:19.153842+0000","last_became_active":"2023-05-16T07:19:19.153581+0000","last_became_peered":"2023-05-16T07:19:19.153581+0000","last_unstale":"2023-05-16T07:19:19.153842+0000","last_undegraded":"2023-05-16T07:19:19.153842+0000","last_fullsized":"2023-05-16T07:19:19.153842+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788842+0000","last_change":"2023-05-16T07:19:18.783043+0000","last_active":"2023-05-16T07:19:18.788842+0000","last_peered":"2023-05-16T07:19:18.788842+0000","last_clean":"2023-05-16T07:19:18.788842+0000","last_became_active":"2023-05-16T07:19:18.782750+0000","last_became_peered":"2023-05-16T07:19:18.782750+0000","last_unstale":"2023-05-16T07:19:18.788842+0000","last_undegraded":"2023-05-16T07:19:18.788842+0000","last_fullsized":"2023-05-16T07:19:18.788842+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160849+0000","last_change":"2023-05-16T07:19:19.160849+0000","last_active":"2023-05-16T07:19:19.160849+0000","last_peered":"2023-05-16T07:19:19.160849+0000","last_clean":"2023-05-16T07:19:19.160849+0000","last_became_active":"2023-05-16T07:19:19.160582+0000","last_became_peered":"2023-05-16T07:19:19.160582+0000","last_unstale":"2023-05-16T07:19:19.160849+0000","last_undegraded":"2023-05-16T07:19:19.160849+0000","last_fullsized":"2023-05-16T07:19:19.160849+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785197+0000","last_change":"2023-05-16T07:19:18.783798+0000","last_active":"2023-05-16T07:19:18.785197+0000","last_peered":"2023-05-16T07:19:18.785197+0000","last_clean":"2023-05-16T07:19:18.785197+0000","last_became_active":"2023-05-16T07:19:18.783609+0000","last_became_peered":"2023-05-16T07:19:18.783609+0000","last_unstale":"2023-05-16T07:19:18.785197+0000","last_undegraded":"2023-05-16T07:19:18.785197+0000","last_fullsized":"2023-05-16T07:19:18.785197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.167255+0000","last_change":"2023-05-16T07:19:19.167255+0000","last_active":"2023-05-16T07:19:19.167255+0000","last_peered":"2023-05-16T07:19:19.167255+0000","last_clean":"2023-05-16T07:19:19.167255+0000","last_became_active":"2023-05-16T07:19:19.166971+0000","last_became_peered":"2023-05-16T07:19:19.166971+0000","last_unstale":"2023-05-16T07:19:19.167255+0000","last_undegraded":"2023-05-16T07:19:19.167255+0000","last_fullsized":"2023-05-16T07:19:19.167255+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"120'3","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.159042+0000","last_change":"2023-05-16T07:19:18.778412+0000","last_active":"2023-05-16T07:19:19.159042+0000","last_peered":"2023-05-16T07:19:19.159042+0000","last_clean":"2023-05-16T07:19:19.159042+0000","last_became_active":"2023-05-16T07:19:18.778125+0000","last_became_peered":"2023-05-16T07:19:18.778125+0000","last_unstale":"2023-05-16T07:19:19.159042+0000","last_undegraded":"2023-05-16T07:19:19.159042+0000","last_fullsized":"2023-05-16T07:19:19.159042+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163143+0000","last_change":"2023-05-16T07:19:19.163143+0000","last_active":"2023-05-16T07:19:19.163143+0000","last_peered":"2023-05-16T07:19:19.163143+0000","last_clean":"2023-05-16T07:19:19.163143+0000","last_became_active":"2023-05-16T07:19:19.162874+0000","last_became_peered":"2023-05-16T07:19:19.162874+0000","last_unstale":"2023-05-16T07:19:19.163143+0000","last_undegraded":"2023-05-16T07:19:19.163143+0000","last_fullsized":"2023-05-16T07:19:19.163143+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.164511+0000","last_change":"2023-05-16T07:19:18.780209+0000","last_active":"2023-05-16T07:19:19.164511+0000","last_peered":"2023-05-16T07:19:19.164511+0000","last_clean":"2023-05-16T07:19:19.164511+0000","last_became_active":"2023-05-16T07:19:18.779771+0000","last_became_peered":"2023-05-16T07:19:18.779771+0000","last_unstale":"2023-05-16T07:19:19.164511+0000","last_undegraded":"2023-05-16T07:19:19.164511+0000","last_fullsized":"2023-05-16T07:19:19.164511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155766+0000","last_change":"2023-05-16T07:19:19.155766+0000","last_active":"2023-05-16T07:19:19.155766+0000","last_peered":"2023-05-16T07:19:19.155766+0000","last_clean":"2023-05-16T07:19:19.155766+0000","last_became_active":"2023-05-16T07:19:19.155446+0000","last_became_peered":"2023-05-16T07:19:19.155446+0000","last_unstale":"2023-05-16T07:19:19.155766+0000","last_undegraded":"2023-05-16T07:19:19.155766+0000","last_fullsized":"2023-05-16T07:19:19.155766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.140489+0000","last_change":"2023-05-16T07:19:18.789954+0000","last_active":"2023-05-16T07:19:19.140489+0000","last_peered":"2023-05-16T07:19:19.140489+0000","last_clean":"2023-05-16T07:19:19.140489+0000","last_became_active":"2023-05-16T07:19:18.789671+0000","last_became_peered":"2023-05-16T07:19:18.789671+0000","last_unstale":"2023-05-16T07:19:19.140489+0000","last_undegraded":"2023-05-16T07:19:19.140489+0000","last_fullsized":"2023-05-16T07:19:19.140489+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150536+0000","last_change":"2023-05-16T07:19:19.150536+0000","last_active":"2023-05-16T07:19:19.150536+0000","last_peered":"2023-05-16T07:19:19.150536+0000","last_clean":"2023-05-16T07:19:19.150536+0000","last_became_active":"2023-05-16T07:19:19.149715+0000","last_became_peered":"2023-05-16T07:19:19.149715+0000","last_unstale":"2023-05-16T07:19:19.150536+0000","last_undegraded":"2023-05-16T07:19:19.150536+0000","last_fullsized":"2023-05-16T07:19:19.150536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785505+0000","last_change":"2023-05-16T07:19:18.777062+0000","last_active":"2023-05-16T07:19:18.785505+0000","last_peered":"2023-05-16T07:19:18.785505+0000","last_clean":"2023-05-16T07:19:18.785505+0000","last_became_active":"2023-05-16T07:19:18.776808+0000","last_became_peered":"2023-05-16T07:19:18.776808+0000","last_unstale":"2023-05-16T07:19:18.785505+0000","last_undegraded":"2023-05-16T07:19:18.785505+0000","last_fullsized":"2023-05-16T07:19:18.785505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156081+0000","last_change":"2023-05-16T07:19:19.156081+0000","last_active":"2023-05-16T07:19:19.156081+0000","last_peered":"2023-05-16T07:19:19.156081+0000","last_clean":"2023-05-16T07:19:19.156081+0000","last_became_active":"2023-05-16T07:19:19.155813+0000","last_became_peered":"2023-05-16T07:19:19.155813+0000","last_unstale":"2023-05-16T07:19:19.156081+0000","last_undegraded":"2023-05-16T07:19:19.156081+0000","last_fullsized":"2023-05-16T07:19:19.156081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.788826+0000","last_change":"2023-05-16T07:19:18.785319+0000","last_active":"2023-05-16T07:19:18.788826+0000","last_peered":"2023-05-16T07:19:18.788826+0000","last_clean":"2023-05-16T07:19:18.788826+0000","last_became_active":"2023-05-16T07:19:18.784968+0000","last_became_peered":"2023-05-16T07:19:18.784968+0000","last_unstale":"2023-05-16T07:19:18.788826+0000","last_undegraded":"2023-05-16T07:19:18.788826+0000","last_fullsized":"2023-05-16T07:19:18.788826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155403+0000","last_change":"2023-05-16T07:19:19.155403+0000","last_active":"2023-05-16T07:19:19.155403+0000","last_peered":"2023-05-16T07:19:19.155403+0000","last_clean":"2023-05-16T07:19:19.155403+0000","last_became_active":"2023-05-16T07:19:19.154491+0000","last_became_peered":"2023-05-16T07:19:19.154491+0000","last_unstale":"2023-05-16T07:19:19.155403+0000","last_undegraded":"2023-05-16T07:19:19.155403+0000","last_fullsized":"2023-05-16T07:19:19.155403+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"120'4","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158691+0000","last_change":"2023-05-16T07:19:18.783503+0000","last_active":"2023-05-16T07:19:19.158691+0000","last_peered":"2023-05-16T07:19:19.158691+0000","last_clean":"2023-05-16T07:19:19.158691+0000","last_became_active":"2023-05-16T07:19:18.783119+0000","last_became_peered":"2023-05-16T07:19:18.783119+0000","last_unstale":"2023-05-16T07:19:19.158691+0000","last_undegraded":"2023-05-16T07:19:19.158691+0000","last_fullsized":"2023-05-16T07:19:19.158691+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150270+0000","last_change":"2023-05-16T07:19:19.150270+0000","last_active":"2023-05-16T07:19:19.150270+0000","last_peered":"2023-05-16T07:19:19.150270+0000","last_clean":"2023-05-16T07:19:19.150270+0000","last_became_active":"2023-05-16T07:19:19.149962+0000","last_became_peered":"2023-05-16T07:19:19.149962+0000","last_unstale":"2023-05-16T07:19:19.150270+0000","last_undegraded":"2023-05-16T07:19:19.150270+0000","last_fullsized":"2023-05-16T07:19:19.150270+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.163309+0000","last_change":"2023-05-16T07:19:18.790080+0000","last_active":"2023-05-16T07:19:19.163309+0000","last_peered":"2023-05-16T07:19:19.163309+0000","last_clean":"2023-05-16T07:19:19.163309+0000","last_became_active":"2023-05-16T07:19:18.789727+0000","last_became_peered":"2023-05-16T07:19:18.789727+0000","last_unstale":"2023-05-16T07:19:19.163309+0000","last_undegraded":"2023-05-16T07:19:19.163309+0000","last_fullsized":"2023-05-16T07:19:19.163309+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162517+0000","last_change":"2023-05-16T07:19:19.162517+0000","last_active":"2023-05-16T07:19:19.162517+0000","last_peered":"2023-05-16T07:19:19.162517+0000","last_clean":"2023-05-16T07:19:19.162517+0000","last_became_active":"2023-05-16T07:19:19.162168+0000","last_became_peered":"2023-05-16T07:19:19.162168+0000","last_unstale":"2023-05-16T07:19:19.162517+0000","last_undegraded":"2023-05-16T07:19:19.162517+0000","last_fullsized":"2023-05-16T07:19:19.162517+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785179+0000","last_change":"2023-05-16T07:19:18.781799+0000","last_active":"2023-05-16T07:19:18.785179+0000","last_peered":"2023-05-16T07:19:18.785179+0000","last_clean":"2023-05-16T07:19:18.785179+0000","last_became_active":"2023-05-16T07:19:18.781226+0000","last_became_peered":"2023-05-16T07:19:18.781226+0000","last_unstale":"2023-05-16T07:19:18.785179+0000","last_undegraded":"2023-05-16T07:19:18.785179+0000","last_fullsized":"2023-05-16T07:19:18.785179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.158073+0000","last_change":"2023-05-16T07:19:19.158073+0000","last_active":"2023-05-16T07:19:19.158073+0000","last_peered":"2023-05-16T07:19:19.158073+0000","last_clean":"2023-05-16T07:19:19.158073+0000","last_became_active":"2023-05-16T07:19:19.157801+0000","last_became_peered":"2023-05-16T07:19:19.157801+0000","last_unstale":"2023-05-16T07:19:19.158073+0000","last_undegraded":"2023-05-16T07:19:19.158073+0000","last_fullsized":"2023-05-16T07:19:19.158073+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782138+0000","last_change":"2023-05-16T07:19:18.779784+0000","last_active":"2023-05-16T07:19:18.782138+0000","last_peered":"2023-05-16T07:19:18.782138+0000","last_clean":"2023-05-16T07:19:18.782138+0000","last_became_active":"2023-05-16T07:19:18.779477+0000","last_became_peered":"2023-05-16T07:19:18.779477+0000","last_unstale":"2023-05-16T07:19:18.782138+0000","last_undegraded":"2023-05-16T07:19:18.782138+0000","last_fullsized":"2023-05-16T07:19:18.782138+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160828+0000","last_change":"2023-05-16T07:19:19.160828+0000","last_active":"2023-05-16T07:19:19.160828+0000","last_peered":"2023-05-16T07:19:19.160828+0000","last_clean":"2023-05-16T07:19:19.160828+0000","last_became_active":"2023-05-16T07:19:19.160609+0000","last_became_peered":"2023-05-16T07:19:19.160609+0000","last_unstale":"2023-05-16T07:19:19.160828+0000","last_undegraded":"2023-05-16T07:19:19.160828+0000","last_fullsized":"2023-05-16T07:19:19.160828+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.789373+0000","last_change":"2023-05-16T07:19:18.781076+0000","last_active":"2023-05-16T07:19:18.789373+0000","last_peered":"2023-05-16T07:19:18.789373+0000","last_clean":"2023-05-16T07:19:18.789373+0000","last_became_active":"2023-05-16T07:19:18.780730+0000","last_became_peered":"2023-05-16T07:19:18.780730+0000","last_unstale":"2023-05-16T07:19:18.789373+0000","last_undegraded":"2023-05-16T07:19:18.789373+0000","last_fullsized":"2023-05-16T07:19:18.789373+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.167741+0000","last_change":"2023-05-16T07:19:19.167741+0000","last_active":"2023-05-16T07:19:19.167741+0000","last_peered":"2023-05-16T07:19:19.167741+0000","last_clean":"2023-05-16T07:19:19.167741+0000","last_became_active":"2023-05-16T07:19:19.167510+0000","last_became_peered":"2023-05-16T07:19:19.167510+0000","last_unstale":"2023-05-16T07:19:19.167741+0000","last_undegraded":"2023-05-16T07:19:19.167741+0000","last_fullsized":"2023-05-16T07:19:19.167741+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785990+0000","last_change":"2023-05-16T07:19:18.780689+0000","last_active":"2023-05-16T07:19:18.785990+0000","last_peered":"2023-05-16T07:19:18.785990+0000","last_clean":"2023-05-16T07:19:18.785990+0000","last_became_active":"2023-05-16T07:19:18.780391+0000","last_became_peered":"2023-05-16T07:19:18.780391+0000","last_unstale":"2023-05-16T07:19:18.785990+0000","last_undegraded":"2023-05-16T07:19:18.785990+0000","last_fullsized":"2023-05-16T07:19:18.785990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.161302+0000","last_change":"2023-05-16T07:19:19.161302+0000","last_active":"2023-05-16T07:19:19.161302+0000","last_peered":"2023-05-16T07:19:19.161302+0000","last_clean":"2023-05-16T07:19:19.161302+0000","last_became_active":"2023-05-16T07:19:19.160781+0000","last_became_peered":"2023-05-16T07:19:19.160781+0000","last_unstale":"2023-05-16T07:19:19.161302+0000","last_undegraded":"2023-05-16T07:19:19.161302+0000","last_fullsized":"2023-05-16T07:19:19.161302+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785825+0000","last_change":"2023-05-16T07:19:18.781734+0000","last_active":"2023-05-16T07:19:18.785825+0000","last_peered":"2023-05-16T07:19:18.785825+0000","last_clean":"2023-05-16T07:19:18.785825+0000","last_became_active":"2023-05-16T07:19:18.781534+0000","last_became_peered":"2023-05-16T07:19:18.781534+0000","last_unstale":"2023-05-16T07:19:18.785825+0000","last_undegraded":"2023-05-16T07:19:18.785825+0000","last_fullsized":"2023-05-16T07:19:18.785825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151782+0000","last_change":"2023-05-16T07:19:19.151782+0000","last_active":"2023-05-16T07:19:19.151782+0000","last_peered":"2023-05-16T07:19:19.151782+0000","last_clean":"2023-05-16T07:19:19.151782+0000","last_became_active":"2023-05-16T07:19:19.151498+0000","last_became_peered":"2023-05-16T07:19:19.151498+0000","last_unstale":"2023-05-16T07:19:19.151782+0000","last_undegraded":"2023-05-16T07:19:19.151782+0000","last_fullsized":"2023-05-16T07:19:19.151782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782084+0000","last_change":"2023-05-16T07:19:18.779031+0000","last_active":"2023-05-16T07:19:18.782084+0000","last_peered":"2023-05-16T07:19:18.782084+0000","last_clean":"2023-05-16T07:19:18.782084+0000","last_became_active":"2023-05-16T07:19:18.778746+0000","last_became_peered":"2023-05-16T07:19:18.778746+0000","last_unstale":"2023-05-16T07:19:18.782084+0000","last_undegraded":"2023-05-16T07:19:18.782084+0000","last_fullsized":"2023-05-16T07:19:18.782084+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150276+0000","last_change":"2023-05-16T07:19:19.150276+0000","last_active":"2023-05-16T07:19:19.150276+0000","last_peered":"2023-05-16T07:19:19.150276+0000","last_clean":"2023-05-16T07:19:19.150276+0000","last_became_active":"2023-05-16T07:19:19.149677+0000","last_became_peered":"2023-05-16T07:19:19.149677+0000","last_unstale":"2023-05-16T07:19:19.150276+0000","last_undegraded":"2023-05-16T07:19:19.150276+0000","last_fullsized":"2023-05-16T07:19:19.150276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.791729+0000","last_change":"2023-05-16T07:19:18.785742+0000","last_active":"2023-05-16T07:19:18.791729+0000","last_peered":"2023-05-16T07:19:18.791729+0000","last_clean":"2023-05-16T07:19:18.791729+0000","last_became_active":"2023-05-16T07:19:18.785006+0000","last_became_peered":"2023-05-16T07:19:18.785006+0000","last_unstale":"2023-05-16T07:19:18.791729+0000","last_undegraded":"2023-05-16T07:19:18.791729+0000","last_fullsized":"2023-05-16T07:19:18.791729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.162534+0000","last_change":"2023-05-16T07:19:19.162534+0000","last_active":"2023-05-16T07:19:19.162534+0000","last_peered":"2023-05-16T07:19:19.162534+0000","last_clean":"2023-05-16T07:19:19.162534+0000","last_became_active":"2023-05-16T07:19:19.162285+0000","last_became_peered":"2023-05-16T07:19:19.162285+0000","last_unstale":"2023-05-16T07:19:19.162534+0000","last_undegraded":"2023-05-16T07:19:19.162534+0000","last_fullsized":"2023-05-16T07:19:19.162534+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.782127+0000","last_change":"2023-05-16T07:19:18.780923+0000","last_active":"2023-05-16T07:19:18.782127+0000","last_peered":"2023-05-16T07:19:18.782127+0000","last_clean":"2023-05-16T07:19:18.782127+0000","last_became_active":"2023-05-16T07:19:18.780278+0000","last_became_peered":"2023-05-16T07:19:18.780278+0000","last_unstale":"2023-05-16T07:19:18.782127+0000","last_undegraded":"2023-05-16T07:19:18.782127+0000","last_fullsized":"2023-05-16T07:19:18.782127+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.772575+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:19:18.772575+0000","last_peered":"2023-05-16T07:19:18.772575+0000","last_clean":"2023-05-16T07:19:18.772575+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:19:18.772575+0000","last_undegraded":"2023-05-16T07:19:18.772575+0000","last_fullsized":"2023-05-16T07:19:18.772575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156445+0000","last_change":"2023-05-16T07:19:19.156445+0000","last_active":"2023-05-16T07:19:19.156445+0000","last_peered":"2023-05-16T07:19:19.156445+0000","last_clean":"2023-05-16T07:19:19.156445+0000","last_became_active":"2023-05-16T07:19:19.156172+0000","last_became_peered":"2023-05-16T07:19:19.156172+0000","last_unstale":"2023-05-16T07:19:19.156445+0000","last_undegraded":"2023-05-16T07:19:19.156445+0000","last_fullsized":"2023-05-16T07:19:19.156445+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.140491+0000","last_change":"2023-05-16T07:19:18.784697+0000","last_active":"2023-05-16T07:19:19.140491+0000","last_peered":"2023-05-16T07:19:19.140491+0000","last_clean":"2023-05-16T07:19:19.140491+0000","last_became_active":"2023-05-16T07:19:18.784412+0000","last_became_peered":"2023-05-16T07:19:18.784412+0000","last_unstale":"2023-05-16T07:19:19.140491+0000","last_undegraded":"2023-05-16T07:19:19.140491+0000","last_fullsized":"2023-05-16T07:19:19.140491+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.769464+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:19:18.769464+0000","last_peered":"2023-05-16T07:19:18.769464+0000","last_clean":"2023-05-16T07:19:18.769464+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:19:18.769464+0000","last_undegraded":"2023-05-16T07:19:18.769464+0000","last_fullsized":"2023-05-16T07:19:18.769464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.150204+0000","last_change":"2023-05-16T07:19:19.150204+0000","last_active":"2023-05-16T07:19:19.150204+0000","last_peered":"2023-05-16T07:19:19.150204+0000","last_clean":"2023-05-16T07:19:19.150204+0000","last_became_active":"2023-05-16T07:19:19.150015+0000","last_became_peered":"2023-05-16T07:19:19.150015+0000","last_unstale":"2023-05-16T07:19:19.150204+0000","last_undegraded":"2023-05-16T07:19:19.150204+0000","last_fullsized":"2023-05-16T07:19:19.150204+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.780422+0000","last_change":"2023-05-16T07:19:18.775232+0000","last_active":"2023-05-16T07:19:18.780422+0000","last_peered":"2023-05-16T07:19:18.780422+0000","last_clean":"2023-05-16T07:19:18.780422+0000","last_became_active":"2023-05-16T07:19:18.774959+0000","last_became_peered":"2023-05-16T07:19:18.774959+0000","last_unstale":"2023-05-16T07:19:18.780422+0000","last_undegraded":"2023-05-16T07:19:18.780422+0000","last_fullsized":"2023-05-16T07:19:18.780422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.767926+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:19:18.767926+0000","last_peered":"2023-05-16T07:19:18.767926+0000","last_clean":"2023-05-16T07:19:18.767926+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:19:18.767926+0000","last_undegraded":"2023-05-16T07:19:18.767926+0000","last_fullsized":"2023-05-16T07:19:18.767926+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160610+0000","last_change":"2023-05-16T07:19:19.160610+0000","last_active":"2023-05-16T07:19:19.160610+0000","last_peered":"2023-05-16T07:19:19.160610+0000","last_clean":"2023-05-16T07:19:19.160610+0000","last_became_active":"2023-05-16T07:19:19.160295+0000","last_became_peered":"2023-05-16T07:19:19.160295+0000","last_unstale":"2023-05-16T07:19:19.160610+0000","last_undegraded":"2023-05-16T07:19:19.160610+0000","last_fullsized":"2023-05-16T07:19:19.160610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"120'2","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.160883+0000","last_change":"2023-05-16T07:19:18.779399+0000","last_active":"2023-05-16T07:19:19.160883+0000","last_peered":"2023-05-16T07:19:19.160883+0000","last_clean":"2023-05-16T07:19:19.160883+0000","last_became_active":"2023-05-16T07:19:18.779143+0000","last_became_peered":"2023-05-16T07:19:18.779143+0000","last_unstale":"2023-05-16T07:19:19.160883+0000","last_undegraded":"2023-05-16T07:19:19.160883+0000","last_fullsized":"2023-05-16T07:19:19.160883+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.771627+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:19:18.771627+0000","last_peered":"2023-05-16T07:19:18.771627+0000","last_clean":"2023-05-16T07:19:18.771627+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:19:18.771627+0000","last_undegraded":"2023-05-16T07:19:18.771627+0000","last_fullsized":"2023-05-16T07:19:18.771627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.767289+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:19:18.767289+0000","last_peered":"2023-05-16T07:19:18.767289+0000","last_clean":"2023-05-16T07:19:18.767289+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:19:18.767289+0000","last_undegraded":"2023-05-16T07:19:18.767289+0000","last_fullsized":"2023-05-16T07:19:18.767289+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.156080+0000","last_change":"2023-05-16T07:19:19.156080+0000","last_active":"2023-05-16T07:19:19.156080+0000","last_peered":"2023-05-16T07:19:19.156080+0000","last_clean":"2023-05-16T07:19:19.156080+0000","last_became_active":"2023-05-16T07:19:19.155820+0000","last_became_peered":"2023-05-16T07:19:19.155820+0000","last_unstale":"2023-05-16T07:19:19.156080+0000","last_undegraded":"2023-05-16T07:19:19.156080+0000","last_fullsized":"2023-05-16T07:19:19.156080+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.787533+0000","last_change":"2023-05-16T07:19:18.785576+0000","last_active":"2023-05-16T07:19:18.787533+0000","last_peered":"2023-05-16T07:19:18.787533+0000","last_clean":"2023-05-16T07:19:18.787533+0000","last_became_active":"2023-05-16T07:19:18.784910+0000","last_became_peered":"2023-05-16T07:19:18.784910+0000","last_unstale":"2023-05-16T07:19:18.787533+0000","last_undegraded":"2023-05-16T07:19:18.787533+0000","last_fullsized":"2023-05-16T07:19:18.787533+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.767375+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:19:18.767375+0000","last_peered":"2023-05-16T07:19:18.767375+0000","last_clean":"2023-05-16T07:19:18.767375+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:19:18.767375+0000","last_undegraded":"2023-05-16T07:19:18.767375+0000","last_fullsized":"2023-05-16T07:19:18.767375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.168720+0000","last_change":"2023-05-16T07:19:19.168720+0000","last_active":"2023-05-16T07:19:19.168720+0000","last_peered":"2023-05-16T07:19:19.168720+0000","last_clean":"2023-05-16T07:19:19.168720+0000","last_became_active":"2023-05-16T07:19:19.168458+0000","last_became_peered":"2023-05-16T07:19:19.168458+0000","last_unstale":"2023-05-16T07:19:19.168720+0000","last_undegraded":"2023-05-16T07:19:19.168720+0000","last_fullsized":"2023-05-16T07:19:19.168720+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785660+0000","last_change":"2023-05-16T07:19:18.777463+0000","last_active":"2023-05-16T07:19:18.785660+0000","last_peered":"2023-05-16T07:19:18.785660+0000","last_clean":"2023-05-16T07:19:18.785660+0000","last_became_active":"2023-05-16T07:19:18.776988+0000","last_became_peered":"2023-05-16T07:19:18.776988+0000","last_unstale":"2023-05-16T07:19:18.785660+0000","last_undegraded":"2023-05-16T07:19:18.785660+0000","last_fullsized":"2023-05-16T07:19:18.785660+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.766609+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:19:18.766609+0000","last_peered":"2023-05-16T07:19:18.766609+0000","last_clean":"2023-05-16T07:19:18.766609+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:19:18.766609+0000","last_undegraded":"2023-05-16T07:19:18.766609+0000","last_fullsized":"2023-05-16T07:19:18.766609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.151508+0000","last_change":"2023-05-16T07:19:19.151508+0000","last_active":"2023-05-16T07:19:19.151508+0000","last_peered":"2023-05-16T07:19:19.151508+0000","last_clean":"2023-05-16T07:19:19.151508+0000","last_became_active":"2023-05-16T07:19:19.151255+0000","last_became_peered":"2023-05-16T07:19:19.151255+0000","last_unstale":"2023-05-16T07:19:19.151508+0000","last_undegraded":"2023-05-16T07:19:19.151508+0000","last_fullsized":"2023-05-16T07:19:19.151508+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"120'1","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.155094+0000","last_change":"2023-05-16T07:19:18.777969+0000","last_active":"2023-05-16T07:19:19.155094+0000","last_peered":"2023-05-16T07:19:19.155094+0000","last_clean":"2023-05-16T07:19:19.155094+0000","last_became_active":"2023-05-16T07:19:18.777538+0000","last_became_peered":"2023-05-16T07:19:18.777538+0000","last_unstale":"2023-05-16T07:19:19.155094+0000","last_undegraded":"2023-05-16T07:19:19.155094+0000","last_fullsized":"2023-05-16T07:19:19.155094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.781395+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:19:18.781395+0000","last_peered":"2023-05-16T07:19:18.781395+0000","last_clean":"2023-05-16T07:19:18.781395+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:19:18.781395+0000","last_undegraded":"2023-05-16T07:19:18.781395+0000","last_fullsized":"2023-05-16T07:19:18.781395+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-05-16T07:19:19.161931+0000","last_change":"2023-05-16T07:19:19.161931+0000","last_active":"2023-05-16T07:19:19.161931+0000","last_peered":"2023-05-16T07:19:19.161931+0000","last_clean":"2023-05-16T07:19:19.161931+0000","last_became_active":"2023-05-16T07:19:19.161242+0000","last_became_peered":"2023-05-16T07:19:19.161242+0000","last_unstale":"2023-05-16T07:19:19.161931+0000","last_undegraded":"2023-05-16T07:19:19.161931+0000","last_fullsized":"2023-05-16T07:19:19.161931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:18.758080+0000","last_clean_scrub_stamp":"2023-05-16T07:19:18.758080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.785806+0000","last_change":"2023-05-16T07:19:18.778237+0000","last_active":"2023-05-16T07:19:18.785806+0000","last_peered":"2023-05-16T07:19:18.785806+0000","last_clean":"2023-05-16T07:19:18.785806+0000","last_became_active":"2023-05-16T07:19:18.777975+0000","last_became_peered":"2023-05-16T07:19:18.777975+0000","last_unstale":"2023-05-16T07:19:18.785806+0000","last_undegraded":"2023-05-16T07:19:18.785806+0000","last_fullsized":"2023-05-16T07:19:18.785806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:17.753430+0000","last_clean_scrub_stamp":"2023-05-16T07:19:17.753430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":111,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-05-16T07:19:18.770701+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:19:18.770701+0000","last_peered":"2023-05-16T07:19:18.770701+0000","last_clean":"2023-05-16T07:19:18.770701+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:19:18.770701+0000","last_undegraded":"2023-05-16T07:19:18.770701+0000","last_fullsized":"2023-05-16T07:19:18.770701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22564,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349276,"statfs":{"total":96636764160,"available":96613658624,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.56200000000000006},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.77700000000000002,"15min":0.84199999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.54200000000000004,"15min":0.56499999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73299999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38900000000000001}]},{"osd":1,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.55800000000000005},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.54400000000000004,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.749,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56499999999999995},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.50800000000000001}]},{"osd":3,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.79300000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58899999999999997}]},{"osd":4,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54500000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65000000000000002,"5min":0.752,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.621},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68200000000000005,"5min":0.748,"15min":0.751},"last":0.628}]},{"osd":6,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65700000000000003,"5min":0.879,"15min":0.879},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.53100000000000003,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22244,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349596,"statfs":{"total":96636764160,"available":96613986304,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53600000000000003,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.85099999999999998},"last":0.435},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71499999999999997,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78400000000000003,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38},"max":{"1min":0.73999999999999999,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70999999999999996,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.377,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.73899999999999999,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57699999999999996,"5min":0.749,"15min":0.89400000000000002},"last":0.372},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":4.048,"15min":4.048},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.92300000000000004},"last":0.48999999999999999}]},{"osd":7,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.5},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70699999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18212,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":16768,"kb_avail":94353628,"statfs":{"total":96636764160,"available":96618115072,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17170432},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.69899999999999995},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.39800000000000002,"15min":0.42099999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.5,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35699999999999998,"15min":0.379},"min":{"1min":0.27200000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41099999999999998,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38400000000000001,"15min":0.40500000000000003},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52100000000000002,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.27100000000000002}]},{"osd":3,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.377,"15min":0.40000000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46300000000000002,"5min":0.622,"15min":0.64000000000000001},"last":0.26000000000000001}]},{"osd":4,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.80000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.623},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61099999999999999,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77400000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.63500000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85599999999999998,"5min":0.94699999999999995,"15min":1.087},"last":0.434},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.93700000000000006,"15min":1.204},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17188,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":15744,"kb_avail":94354652,"statfs":{"total":96636764160,"available":96619163648,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16121856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.36299999999999999,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34599999999999997,"15min":0.372},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.437,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.40500000000000003},"min":{"1min":0.251,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.25}]},{"osd":3,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.40799999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.49399999999999999,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.434},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.499},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65500000000000003,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0669999999999999},"last":0.47899999999999998}]},{"osd":7,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18980,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":17536,"kb_avail":94352860,"statfs":{"total":96636764160,"available":96617328640,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17956864},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.35099999999999998,"15min":0.36499999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.184},"max":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.64900000000000002},"last":0.31},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.372,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.47699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.376}]},{"osd":1,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.40000000000000002,"15min":0.41199999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.71199999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.42099999999999999,"15min":0.437},"min":{"1min":0.29999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.5,"5min":0.64200000000000002,"15min":0.72899999999999998},"last":0.436}]},{"osd":3,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.378,"15min":0.39500000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22700000000000001},"max":{"1min":0.51400000000000001,"5min":0.58299999999999996,"15min":0.64700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.41099999999999998,"15min":0.438},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51400000000000001,"5min":0.64900000000000002,"15min":0.72899999999999998},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.378,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.625},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.91400000000000003,"15min":1.113},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61299999999999999,"15min":0.63200000000000001},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.82599999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61699999999999999,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":0.88100000000000001,"15min":1.5609999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":1.0369999999999999,"15min":1.47},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue May 16 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.39200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83099999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81299999999999994,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.63300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26096,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":24640,"kb_avail":94345744,"statfs":{"total":96636764160,"available":96610041856,"internally_reserved":0,"allocated":1482752,"data_stored":775444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25231360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35799999999999998,"15min":0.379},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.5,"5min":0.54900000000000004,"15min":0.58799999999999997},"last":0.307}]},{"osd":1,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.48299999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.41599999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41399999999999998,"15min":0.433},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.53300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":0.877,"15min":0.91600000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.61399999999999999},"min":{"1min":0.40300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96399999999999997},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0489999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":1.0409999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0780000000000001},"last":0.62},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.877,"15min":1.0049999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61299999999999999,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.02},"last":0.67400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18276,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":16832,"kb_avail":94353564,"statfs":{"total":96636764160,"available":96618049536,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17235968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36199999999999999,"15min":0.35999999999999999},"max":{"1min":2.5600000000000001,"5min":2.5600000000000001,"15min":2.5600000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.628},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575049,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25444,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":24000,"kb_avail":94346396,"statfs":{"total":96636764160,"available":96610709504,"internally_reserved":0,"allocated":1470464,"data_stored":767653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24576000},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.32500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.641,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":2.847,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.434}]},{"osd":1,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.499},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.6059999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73799999999999999,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.127,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.503,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1150000000000002,"5min":3.1150000000000002,"15min":3.1150000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":3.5659999999999998,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.52600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.76600000000000001}]},{"osd":7,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:19:22.470 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:19:22.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:19:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.612+0000 7f9d13a6d700 1 Processor -- start 2023-05-16T07:19:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d13a6d700 1 -- start start 2023-05-16T07:19:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d13a6d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9d0c154470 0x7f9d0c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d13a6d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c068160 0x7f9d0c0685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d13a6d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0c94d0 con 0x7f9d0c154090 2023-05-16T07:19:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d13a6d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cb7e0 con 0x7f9d0c154470 2023-05-16T07:19:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d13a6d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c068aa0 con 0x7f9d0c068160 2023-05-16T07:19:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d11809700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9d0c154090 0x7f9d0c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59744/0 (socket says 172.21.15.92:59744) 2023-05-16T07:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d11809700 1 -- 172.21.15.92:0/3747357042 learned_addr learned my addr 172.21.15.92:0/3747357042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.613+0000 7f9d11008700 1 --2- 172.21.15.92:0/3747357042 >> v2:172.21.15.92:3300/0 conn(0x7f9d0c154470 0x7f9d0c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d1200a700 1 --2- 172.21.15.92:0/3747357042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c068160 0x7f9d0c0685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d10807700 1 -- 172.21.15.92:0/3747357042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408089520 0 0) 0x7f9d0c0c94d0 con 0x7f9d0c154090 2023-05-16T07:19:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d10807700 1 -- 172.21.15.92:0/3747357042 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf40033f0 con 0x7f9d0c154090 2023-05-16T07:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d11008700 1 -- 172.21.15.92:0/3747357042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c068160 msgr2=0x7f9d0c0685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d11008700 1 --2- 172.21.15.92:0/3747357042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c068160 0x7f9d0c0685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d11008700 1 -- 172.21.15.92:0/3747357042 >> v1:172.21.15.92:6789/0 conn(0x7f9d0c154090 legacy=0x7f9d0c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d11008700 1 -- 172.21.15.92:0/3747357042 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0c88c0 con 0x7f9d0c154470 2023-05-16T07:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d11008700 1 --2- 172.21.15.92:0/3747357042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154470 0x7f9d0c0aaca0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f9d00009760 tx=0x7f9d00009060).ready entity=mon.0 client_cookie=819d7b880dbe9dcf server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d10807700 1 -- 172.21.15.92:0/3747357042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000f510 con 0x7f9d0c154470 2023-05-16T07:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d10807700 1 -- 172.21.15.92:0/3747357042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0000f670 con 0x7f9d0c154470 2023-05-16T07:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d10807700 1 -- 172.21.15.92:0/3747357042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000f9f0 con 0x7f9d0c154470 2023-05-16T07:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d13a6d700 1 -- 172.21.15.92:0/3747357042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154470 msgr2=0x7f9d0c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.614+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/3747357042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154470 0x7f9d0c0aaca0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f9d00009760 tx=0x7f9d00009060).stop 2023-05-16T07:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.615+0000 7f9d13a6d700 1 -- 172.21.15.92:0/3747357042 shutdown_connections 2023-05-16T07:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.615+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/3747357042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c068160 0x7f9d0c0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.615+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/3747357042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154470 0x7f9d0c0aaca0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.615+0000 7f9d13a6d700 1 -- 172.21.15.92:0/3747357042 >> 172.21.15.92:0/3747357042 conn(0x7f9d0c067920 msgr2=0x7f9d0c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.615+0000 7f9d13a6d700 1 -- 172.21.15.92:0/3747357042 shutdown_connections 2023-05-16T07:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.615+0000 7f9d13a6d700 1 -- 172.21.15.92:0/3747357042 wait complete. 2023-05-16T07:19:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 Processor -- start 2023-05-16T07:19:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 -- start start 2023-05-16T07:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d0c068160 0x7f9d0c0c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 0x7f9d0c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c154470 0x7f9d0c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c155d90 con 0x7f9d0c154090 2023-05-16T07:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c4c20 con 0x7f9d0c068160 2023-05-16T07:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.616+0000 7f9d13a6d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c155fc0 con 0x7f9d0c154470 2023-05-16T07:19:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 0x7f9d0c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 0x7f9d0c0c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49002/0 (socket says 172.21.15.92:49002) 2023-05-16T07:19:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 -- 172.21.15.92:0/2435262620 learned_addr learned my addr 172.21.15.92:0/2435262620 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d1200a700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c154470 0x7f9d0c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11809700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d0c068160 0x7f9d0c0c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 -- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c154470 msgr2=0x7f9d0c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c154470 0x7f9d0c151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 -- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d0c068160 msgr2=0x7f9d0c0c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d0c068160 0x7f9d0c0c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 -- 172.21.15.92:0/2435262620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000092a0 con 0x7f9d0c154090 2023-05-16T07:19:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d11008700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 0x7f9d0c0c4280 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f9d0c06bb30 tx=0x7f9d00017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000f7e0 con 0x7f9d0c154090 2023-05-16T07:19:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0000f940 con 0x7f9d0c154090 2023-05-16T07:19:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c156260 con 0x7f9d0c154090 2023-05-16T07:19:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0001b690 con 0x7f9d0c154090 2023-05-16T07:19:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.617+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c156820 con 0x7f9d0c154090 2023-05-16T07:19:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.618+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c066730 con 0x7f9d0c154090 2023-05-16T07:19:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.618+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9d0000f070 con 0x7f9d0c154090 2023-05-16T07:19:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.619+0000 7f9d09ffb700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cf805ed90 0x7f9cf8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.619+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f9d0007b440 con 0x7f9d0c154090 2023-05-16T07:19:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.619+0000 7f9d11809700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cf805ed90 0x7f9cf8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.619+0000 7f9d11809700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cf805ed90 0x7f9cf8061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9cfc005ad0 tx=0x7f9cfc009040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.623+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9d0004c830 con 0x7f9d0c154090 2023-05-16T07:19:22.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.851+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f9d0c06c350 con 0x7f9d0c154090 2023-05-16T07:19:22.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.855+0000 7f9d09ffb700 1 -- 172.21.15.92:0/2435262620 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9d0004fe50 con 0x7f9d0c154090 2023-05-16T07:19:22.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cf805ed90 msgr2=0x7f9cf8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cf805ed90 0x7f9cf8061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9cfc005ad0 tx=0x7f9cfc009040).stop 2023-05-16T07:19:22.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 msgr2=0x7f9d0c0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 0x7f9d0c0c4280 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f9d0c06bb30 tx=0x7f9d00017000).stop 2023-05-16T07:19:22.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 shutdown_connections 2023-05-16T07:19:22.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9cf805ed90 0x7f9cf8061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9d0c154470 0x7f9d0c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9d0c068160 0x7f9d0c0c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 --2- 172.21.15.92:0/2435262620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c154090 0x7f9d0c0c4280 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 >> 172.21.15.92:0/2435262620 conn(0x7f9d0c067920 msgr2=0x7f9d0c14f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:22.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 shutdown_connections 2023-05-16T07:19:22.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.857+0000 7f9d13a6d700 1 -- 172.21.15.92:0/2435262620 wait complete. 2023-05-16T07:19:22.865 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:19:22.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.979+0000 7f50b5372700 1 Processor -- start 2023-05-16T07:19:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.979+0000 7f50b5372700 1 -- start start 2023-05-16T07:19:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50b5372700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f50b00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50b5372700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b00c0660 con 0x7f50b0069b90 2023-05-16T07:19:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50b5372700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b006a020 con 0x7f50b0154170 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50b5372700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00bfd30 con 0x7f50b0154550 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50af7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f50b0069b90 0x7f50b00c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59758/0 (socket says 172.21.15.92:59758) 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50af7fe700 1 -- 172.21.15.92:0/4009539221 learned_addr learned my addr 172.21.15.92:0/4009539221 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105910200 0 0) 0x7f50b00c0660 con 0x7f50b0069b90 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f509c0033f0 con 0x7f50b0069b90 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50a67fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f50b00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075795297 0 0) 0x7f509c0033f0 con 0x7f50b0069b90 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 msgr2=0x7f50b00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f50b00696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 >> v1:172.21.15.158:6789/0 conn(0x7f50b0154170 legacy=0x7f50b00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00c0010 con 0x7f50b0069b90 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f50a80017f0 con 0x7f50b0069b90 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 >> v1:172.21.15.92:6789/0 conn(0x7f50b0069b90 legacy=0x7f50b00c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f509c003910 0x7f509c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f509c004690 0x7f509c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f509c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00bfd30 con 0x7f509c004690 2023-05-16T07:19:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c00bac0 con 0x7f509c003910 2023-05-16T07:19:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c00bc80 con 0x7f50b0154550 2023-05-16T07:19:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50a67fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f509c004690 0x7f509c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.980+0000 7f50aeffd700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f509c003910 0x7f509c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50af7fe700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f509c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50a67fc700 1 -- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 msgr2=0x7f509c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50a67fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f509c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50a67fc700 1 -- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f509c003910 msgr2=0x7f509c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50a67fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f509c003910 0x7f509c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50a67fc700 1 -- 172.21.15.92:0/4009539221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0033f0 con 0x7f509c004690 2023-05-16T07:19:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50af7fe700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b0154550 0x7f509c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50a67fc700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f509c004690 0x7f509c004b10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f509800b9d0 tx=0x7f5098008040).ready entity=mon.0 client_cookie=6ba843a63942f51 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800b340 con 0x7f509c004690 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5098013480 con 0x7f509c004690 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50ae7fc700 1 -- 172.21.15.92:0/4009539221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098013800 con 0x7f509c004690 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50b5372700 1 -- 172.21.15.92:0/4009539221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f509c004690 msgr2=0x7f509c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50b5372700 1 --2- 172.21.15.92:0/4009539221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f509c004690 0x7f509c004b10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f509800b9d0 tx=0x7f5098008040).stop 2023-05-16T07:19:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50af7fe700 1 -- 172.21.15.92:0/4009539221 reap_dead start 2023-05-16T07:19:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50b5372700 1 -- 172.21.15.92:0/4009539221 shutdown_connections 2023-05-16T07:19:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50b5372700 1 -- 172.21.15.92:0/4009539221 >> 172.21.15.92:0/4009539221 conn(0x7f50b00cb700 msgr2=0x7f50b00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50b5372700 1 -- 172.21.15.92:0/4009539221 shutdown_connections 2023-05-16T07:19:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.981+0000 7f50b5372700 1 -- 172.21.15.92:0/4009539221 wait complete. 2023-05-16T07:19:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 Processor -- start 2023-05-16T07:19:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 -- start start 2023-05-16T07:19:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f509c004690 0x7f50b0143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b00c2c30 0x7f50b00c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 0x7f50b00af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0159270 con 0x7f50b00aedd0 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b01589b0 con 0x7f50b00c2c30 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50b5372700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00af720 con 0x7f509c004690 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.983+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 0x7f50b00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 0x7f50b00af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49024/0 (socket says 172.21.15.92:49024) 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 -- 172.21.15.92:0/3587396285 learned_addr learned my addr 172.21.15.92:0/3587396285 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50a67fc700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b00c2c30 0x7f50b00c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50aeffd700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f509c004690 0x7f50b0143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 -- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f509c004690 msgr2=0x7f50b0143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f509c004690 0x7f50b0143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 -- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b00c2c30 msgr2=0x7f50b00c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b00c2c30 0x7f50b00c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 -- 172.21.15.92:0/3587396285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0158c20 con 0x7f50b00aedd0 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50a67fc700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b00c2c30 0x7f50b00c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50af7fe700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 0x7f50b00af230 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f509c005670 tx=0x7f50a8007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8004a30 con 0x7f50b00aedd0 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8004b90 con 0x7f50b00aedd0 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800e8f0 con 0x7f50b00aedd0 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b0158f00 con 0x7f50b00aedd0 2023-05-16T07:19:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.984+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b0143d50 con 0x7f50b00aedd0 2023-05-16T07:19:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.985+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b00666c0 con 0x7f50b00aedd0 2023-05-16T07:19:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.985+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f50a801d070 con 0x7f50b00aedd0 2023-05-16T07:19:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.986+0000 7f50a77fe700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f509005edc0 0x7f5090061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.986+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f50a80795c0 con 0x7f50b00aedd0 2023-05-16T07:19:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.986+0000 7f50aeffd700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f509005edc0 0x7f5090061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:22.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.986+0000 7f50aeffd700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f509005edc0 0x7f5090061280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f50a0007a40 tx=0x7f50a0009040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:22.990+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f50a804a900 con 0x7f50b00aedd0 2023-05-16T07:19:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.194+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50b006c350 con 0x7f50b00aedd0 2023-05-16T07:19:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.194+0000 7f50a77fe700 1 -- 172.21.15.92:0/3587396285 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f50a801b070 con 0x7f50b00aedd0 2023-05-16T07:19:23.199 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:23.199 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:19.134223+0000","modified":"2023-05-16T07:19:21.140319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7461":{"gid":7461,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1482305851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1482305851},{"type":"v1","addr":"172.21.15.92:6835","nonce":1482305851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-05-16T07:19:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f509005edc0 msgr2=0x7f5090061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f509005edc0 0x7f5090061280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f50a0007a40 tx=0x7f50a0009040).stop 2023-05-16T07:19:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 msgr2=0x7f50b00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 0x7f50b00af230 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f509c005670 tx=0x7f50a8007ef0).stop 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 shutdown_connections 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f509005edc0 0x7f5090061280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f509c004690 0x7f50b0143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b00aedd0 0x7f50b00af230 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 --2- 172.21.15.92:0/3587396285 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b00c2c30 0x7f50b00c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 >> 172.21.15.92:0/3587396285 conn(0x7f50b00cb700 msgr2=0x7f50b0153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 shutdown_connections 2023-05-16T07:19:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.196+0000 7f50b5372700 1 -- 172.21.15.92:0/3587396285 wait complete. 2023-05-16T07:19:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-05-16T07:19:23.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:19:19.134223+0000', 'modified': '2023-05-16T07:19:21.140319+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7461}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7461': {'gid': 7461, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/1482305851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1482305851}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1482305851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:19:23.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:19:23.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-16T07:19:23.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.730+0000 7fdc5e1da700 1 Processor -- start 2023-05-16T07:19:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.730+0000 7fdc5e1da700 1 -- start start 2023-05-16T07:19:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc5e1da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc5e1da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc58069db0 0x7fdc580c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc5e1da700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc580c0660 con 0x7fdc58154550 2023-05-16T07:19:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc5e1da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580c1b00 con 0x7fdc58069db0 2023-05-16T07:19:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc5e1da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580bfd30 con 0x7fdc58154170 2023-05-16T07:19:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc57fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc58069db0 0x7fdc580c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc56ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc58154550 0x7fdc580696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59778/0 (socket says 172.21.15.92:59778) 2023-05-16T07:19:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc56ffd700 1 -- 172.21.15.92:0/2650529040 learned_addr learned my addr 172.21.15.92:0/2650529040 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc57fff700 1 -- 172.21.15.92:0/2650529040 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 msgr2=0x7fdc580c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc57fff700 1 --2- 172.21.15.92:0/2650529040 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc57fff700 1 -- 172.21.15.92:0/2650529040 >> v1:172.21.15.92:6789/0 conn(0x7fdc58154550 legacy=0x7fdc580696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc57fff700 1 -- 172.21.15.92:0/2650529040 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc580c0010 con 0x7fdc58069db0 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc567fc700 1 -- 172.21.15.92:0/2650529040 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210241651 0 0) 0x7fdc580c0660 con 0x7fdc58154550 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc57fff700 1 --2- 172.21.15.92:0/2650529040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58069db0 0x7fdc580c6160 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7fdc50009e60 tx=0x7fdc50009780).ready entity=mon.0 client_cookie=550a3a5bd0d3506c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc567fc700 1 -- 172.21.15.92:0/2650529040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000e070 con 0x7fdc58069db0 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc567fc700 1 -- 172.21.15.92:0/2650529040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5000bc30 con 0x7fdc58069db0 2023-05-16T07:19:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.731+0000 7fdc567fc700 1 -- 172.21.15.92:0/2650529040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50012410 con 0x7fdc58069db0 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 -- 172.21.15.92:0/2650529040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58069db0 msgr2=0x7fdc580c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/2650529040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58069db0 0x7fdc580c6160 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7fdc50009e60 tx=0x7fdc50009780).stop 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 -- 172.21.15.92:0/2650529040 shutdown_connections 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/2650529040 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/2650529040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58069db0 0x7fdc580c6160 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 -- 172.21.15.92:0/2650529040 >> 172.21.15.92:0/2650529040 conn(0x7fdc580cb700 msgr2=0x7fdc580c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 -- 172.21.15.92:0/2650529040 shutdown_connections 2023-05-16T07:19:23.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.732+0000 7fdc5e1da700 1 -- 172.21.15.92:0/2650529040 wait complete. 2023-05-16T07:19:23.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.733+0000 7fdc5e1da700 1 Processor -- start 2023-05-16T07:19:23.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.733+0000 7fdc5e1da700 1 -- start start 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc5e1da700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc58069db0 0x7fdc581472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc5e1da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc5e1da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 0x7fdc580c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc5e1da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc581477e0 con 0x7fdc58154550 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc5e1da700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580adc30 con 0x7fdc58069db0 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc5e1da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc581684f0 con 0x7fdc58154170 2023-05-16T07:19:23.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 0x7fdc580c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 0x7fdc580c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49038/0 (socket says 172.21.15.92:49038) 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 -- 172.21.15.92:0/549744003 learned_addr learned my addr 172.21.15.92:0/549744003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc577fe700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc58069db0 0x7fdc581472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 -- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 msgr2=0x7fdc580c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 -- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc58069db0 msgr2=0x7fdc581472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc58069db0 0x7fdc581472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 -- 172.21.15.92:0/549744003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50009a20 con 0x7fdc58154550 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc57fff700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 0x7fdc580c8d50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fdc58083060 tx=0x7fdc50009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.734+0000 7fdc3f7fe700 1 -- 172.21.15.92:0/549744003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500116d0 con 0x7fdc58154550 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.735+0000 7fdc3f7fe700 1 -- 172.21.15.92:0/549744003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc50011830 con 0x7fdc58154550 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.735+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc58147c10 con 0x7fdc58154550 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.735+0000 7fdc3f7fe700 1 -- 172.21.15.92:0/549744003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50011bf0 con 0x7fdc58154550 2023-05-16T07:19:23.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.735+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc58167bc0 con 0x7fdc58154550 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.736+0000 7fdc3f7fe700 1 -- 172.21.15.92:0/549744003 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdc50027460 con 0x7fdc58154550 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.736+0000 7fdc3f7fe700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdc4405ed90 0x7fdc44061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.736+0000 7fdc3f7fe700 1 -- 172.21.15.92:0/549744003 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fdc5007be40 con 0x7fdc58154550 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.736+0000 7fdc577fe700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdc4405ed90 0x7fdc44061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.737+0000 7fdc577fe700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdc4405ed90 0x7fdc44061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fdc580aa890 tx=0x7fdc48008040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.738+0000 7fdc6fb6cb80 1 Processor -- start 2023-05-16T07:19:23.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.738+0000 7fdc6fb6cb80 1 -- start start 2023-05-16T07:19:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 0x55ec5eb76c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb4f300 0x55ec5eb49710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eb145e0 con 0x55ec5eb93560 2023-05-16T07:19:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5ea8b8a0 con 0x55ec5eb4f300 2023-05-16T07:19:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eaa0900 con 0x55ec5eb98480 2023-05-16T07:19:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 0x55ec5eb76c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 0x55ec5eb76c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49042/0 (socket says 172.21.15.92:49042) 2023-05-16T07:19:23.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc577fe700 1 -- 172.21.15.92:0/723961304 learned_addr learned my addr 172.21.15.92:0/723961304 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:23.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc57fff700 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb4f300 0x55ec5eb49710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.739+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb93940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc577fe700 1 -- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 msgr2=0x55ec5eb93940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc577fe700 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb93940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc577fe700 1 -- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb4f300 msgr2=0x55ec5eb49710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc577fe700 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb4f300 0x55ec5eb49710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc577fe700 1 -- 172.21.15.92:0/723961304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec5eb14450 con 0x55ec5eb93560 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc577fe700 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 0x55ec5eb76c50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7fdc4800c8f0 tx=0x7fdc48002000).ready entity=mon.0 client_cookie=aec02566d6b3d5c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc3cff9700 1 -- 172.21.15.92:0/723961304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48016890 con 0x55ec5eb93560 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc3cff9700 1 -- 172.21.15.92:0/723961304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480169f0 con 0x55ec5eb93560 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc3cff9700 1 -- 172.21.15.92:0/723961304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48016d90 con 0x55ec5eb93560 2023-05-16T07:19:23.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/723961304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 msgr2=0x55ec5eb76c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 0x55ec5eb76c50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7fdc4800c8f0 tx=0x7fdc48002000).stop 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/723961304 shutdown_connections 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb4f300 0x55ec5eb49710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb93560 0x55ec5eb76c50 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/723961304 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.740+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/723961304 >> 172.21.15.92:0/723961304 conn(0x55ec5e90be40 msgr2=0x55ec5eb979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.741+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/723961304 shutdown_connections 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.741+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/723961304 wait complete. 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.741+0000 7fdc6fb6cb80 1 -- start start 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.741+0000 7fdc6fb6cb80 1 Processor -- start 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb77750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5ea67ed0 con 0x55ec5eb4f300 2023-05-16T07:19:23.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eb8ea50 con 0x55ec5eb93560 2023-05-16T07:19:23.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eba4920 con 0x55ec5eb98480 2023-05-16T07:19:23.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc6fb6cb80 10 client.0 ll_register_callbacks cb 0x55ec5eb0e1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:19:23.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb74150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49048/0 (socket says 172.21.15.92:49048) 2023-05-16T07:19:23.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 -- 172.21.15.92:0/3934242590 learned_addr learned my addr 172.21.15.92:0/3934242590 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:23.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc57fff700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb77750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 -- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 msgr2=0x55ec5eb77c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb77c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 -- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 msgr2=0x55ec5eb77750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb77750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 -- 172.21.15.92:0/3934242590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4800c5e0 con 0x55ec5eb4f300 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb77750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc577fe700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb74150 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7fdc48016400 tx=0x7fdc48011000).ready entity=mon.0 client_cookie=56ce203392110906 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.742+0000 7fdc35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc35ffb700 1 -- 172.21.15.92:0/3934242590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48016890 con 0x55ec5eb4f300 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc35ffb700 1 -- 172.21.15.92:0/3934242590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fdc48013e30 con 0x55ec5eb4f300 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec5eb42460 con 0x55ec5eb4f300 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc35ffb700 1 -- 172.21.15.92:0/3934242590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4801f640 con 0x55ec5eb4f300 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 10 client.7602 fetch_fsmap learned FSMap version 43 2023-05-16T07:19:23.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55ec5e943310 con 0x55ec5eb4f300 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc35ffb700 1 -- 172.21.15.92:0/3934242590 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec5eb42460 con 0x55ec5eb4f300 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc35ffb700 1 -- 172.21.15.92:0/3934242590 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 43) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdc4801f7a0 con 0x55ec5eb4f300 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 10 client.7602 fetch_fsmap finished waiting for FSMap version 43 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 10 client.7602 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 20 client.7602 populate_metadata read hostname 'smithi092' 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb75180 0x55ec5eb95fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 4 client.7602 mds_command: new command op to 7461 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:19:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 --> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec5e907ca0 con 0x55ec5eb75180 2023-05-16T07:19:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb75180 0x55ec5eb95fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc35ffb700 10 client.7602 ms_handle_connect on v2:172.21.15.92:6834/1482305851 2023-05-16T07:19:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.743+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb75180 0x55ec5eb95fd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=87ffd41c397ffb23 server_cookie=8d0a77ed20674db9 in_seq=0 out_seq=0 2023-05-16T07:19:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc35ffb700 1 -- 172.21.15.92:0/3934242590 <== mds.0 v2:172.21.15.92:6834/1482305851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55ec5e907ca0 con 0x55ec5eb75180 2023-05-16T07:19:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc35ffb700 10 client.7602 handle_command_reply: tid=0 2023-05-16T07:19:23.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 client.7602 shutdown 2023-05-16T07:19:23.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 msgr2=0x55ec5eb74150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb74150 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7fdc48016400 tx=0x7fdc48011000).stop 2023-05-16T07:19:23.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 shutdown_connections 2023-05-16T07:19:23.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb75180 0x55ec5eb95fd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb98480 0x55ec5eb77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc35ffb700 0 client.7602 ms_handle_reset on v2:172.21.15.92:6834/1482305851 2023-05-16T07:19:23.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb74150 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3934242590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb77750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 >> 172.21.15.92:0/3934242590 conn(0x55ec5e90be40 msgr2=0x55ec5eb4e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 shutdown_connections 2023-05-16T07:19:23.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.744+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3934242590 wait complete. 2023-05-16T07:19:23.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.745+0000 7fdc6fb6cb80 20 client.7602 trim_cache size 0 max 16384 2023-05-16T07:19:23.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 Processor -- start 2023-05-16T07:19:23.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 -- start start 2023-05-16T07:19:23.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb4c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb93560 0x55ec5eb4b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb98480 0x55ec5eb77d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eba3170 con 0x55ec5eb4f300 2023-05-16T07:19:23.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eb926d0 con 0x55ec5eb98480 2023-05-16T07:19:23.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.761+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eb4e030 con 0x55ec5eb93560 2023-05-16T07:19:23.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb4c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb4c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49056/0 (socket says 172.21.15.92:49056) 2023-05-16T07:19:23.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 -- 172.21.15.92:0/3415338000 learned_addr learned my addr 172.21.15.92:0/3415338000 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 -- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb93560 msgr2=0x55ec5eb4b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb93560 0x55ec5eb4b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 -- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb98480 msgr2=0x55ec5eb77d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc57fff700 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb98480 0x55ec5eb77d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb98480 0x55ec5eb77d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 -- 172.21.15.92:0/3415338000 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec5eba3330 con 0x55ec5eb4f300 2023-05-16T07:19:23.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc577fe700 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb4c5a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7fdc4800a530 tx=0x7fdc48023040).ready entity=mon.0 client_cookie=6669630a8826b1d4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc3d7fa700 1 -- 172.21.15.92:0/3415338000 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48013aa0 con 0x55ec5eb4f300 2023-05-16T07:19:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc3d7fa700 1 -- 172.21.15.92:0/3415338000 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48013e30 con 0x55ec5eb4f300 2023-05-16T07:19:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.762+0000 7fdc3d7fa700 1 -- 172.21.15.92:0/3415338000 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480164b0 con 0x55ec5eb4f300 2023-05-16T07:19:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3415338000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 msgr2=0x55ec5eb4c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb4c5a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7fdc4800a530 tx=0x7fdc48023040).stop 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3415338000 shutdown_connections 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb98480 0x55ec5eb77d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb4f300 0x55ec5eb4c5a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/3415338000 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb93560 0x55ec5eb4b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3415338000 >> 172.21.15.92:0/3415338000 conn(0x55ec5e90be40 msgr2=0x55ec5eb92b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3415338000 shutdown_connections 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/3415338000 wait complete. 2023-05-16T07:19:23.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 -- start start 2023-05-16T07:19:23.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.763+0000 7fdc6fb6cb80 1 Processor -- start 2023-05-16T07:19:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb4f300 0x55ec5ebb0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb95fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 0x55ec5eb96490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5eb96980 con 0x55ec5eb98480 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec5ebfa370 con 0x55ec5eb93560 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec5ebb1390 con 0x55ec5eb4f300 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc6fb6cb80 10 client.0 ll_register_callbacks cb 0x55ec5eb0e1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 0x55ec5eb96490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 0x55ec5eb96490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49064/0 (socket says 172.21.15.92:49064) 2023-05-16T07:19:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 -- 172.21.15.92:0/1729272455 learned_addr learned my addr 172.21.15.92:0/1729272455 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc56ffd700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb95fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc577fe700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb4f300 0x55ec5ebb0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 -- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb4f300 msgr2=0x55ec5ebb0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb4f300 0x55ec5ebb0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 -- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 msgr2=0x55ec5eb95fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb95fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 -- 172.21.15.92:0/1729272455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec5e943310 con 0x55ec5eb98480 2023-05-16T07:19:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc57fff700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 0x55ec5eb96490 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7fdc500099f0 tx=0x7fdc5001f040).ready entity=mon.0 client_cookie=ad8e9580a1bab7dc server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.764+0000 7fdc35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:19:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 1 -- 172.21.15.92:0/1729272455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500561d0 con 0x55ec5eb98480 2023-05-16T07:19:23.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 1 -- 172.21.15.92:0/1729272455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fdc5004d4f0 con 0x55ec5eb98480 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec5eb794f0 con 0x55ec5eb98480 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 1 -- 172.21.15.92:0/1729272455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5004d6b0 con 0x55ec5eb98480 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 10 client.7608 fetch_fsmap learned FSMap version 43 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55ec5ea99880 con 0x55ec5eb98480 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 1 -- 172.21.15.92:0/1729272455 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec5eb794f0 con 0x55ec5eb98480 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 1 -- 172.21.15.92:0/1729272455 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 43) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdc5004cbd0 con 0x55ec5eb98480 2023-05-16T07:19:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 10 client.7608 fetch_fsmap finished waiting for FSMap version 43 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 10 client.7608 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 20 client.7608 populate_metadata read hostname 'smithi092' 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb493e0 0x55ec5eb497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 4 client.7608 mds_command: new command op to 7461 tid=0[{"prefix": "status"}] 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 --> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fdc480169f0 con 0x55ec5eb493e0 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc577fe700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb493e0 0x55ec5eb497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 10 client.7608 ms_handle_connect on v2:172.21.15.92:6834/1482305851 2023-05-16T07:19:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc577fe700 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb493e0 0x55ec5eb497c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f75f5b5c4d548e1e server_cookie=2df2fe3d20576710 in_seq=0 out_seq=0 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 1 -- 172.21.15.92:0/1729272455 <== mds.0 v2:172.21.15.92:6834/1482305851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7fdc480169f0 con 0x55ec5eb493e0 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc35ffb700 10 client.7608 handle_command_reply: tid=0 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.765+0000 7fdc6fb6cb80 1 client.7608 shutdown 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 msgr2=0x55ec5eb96490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 0x55ec5eb96490 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7fdc500099f0 tx=0x7fdc5001f040).stop 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 shutdown_connections 2023-05-16T07:19:23.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:6834/1482305851,v1:172.21.15.92:6835/1482305851] conn(0x55ec5eb493e0 0x55ec5eb497c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x55ec5eb4f300 0x55ec5ebb0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc35ffb700 0 client.7608 ms_handle_reset on v2:172.21.15.92:6834/1482305851 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5eb98480 0x55ec5eb96490 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 --2- 172.21.15.92:0/1729272455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x55ec5eb93560 0x55ec5eb95fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 >> 172.21.15.92:0/1729272455 conn(0x55ec5e90be40 msgr2=0x55ec5eb73520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 shutdown_connections 2023-05-16T07:19:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 1 -- 172.21.15.92:0/1729272455 wait complete. 2023-05-16T07:19:23.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc6fb6cb80 20 client.7608 trim_cache size 0 max 16384 2023-05-16T07:19:23.784 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:19:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:19:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:19:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7461, 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.631667352, 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 43, 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 121, 2023-05-16T07:19:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 121, 2023-05-16T07:19:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.1679638220000008 2023-05-16T07:19:23.786 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:19:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdc4405ed90 msgr2=0x7fdc44061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdc4405ed90 0x7fdc44061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fdc580aa890 tx=0x7fdc48008040).stop 2023-05-16T07:19:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 msgr2=0x7fdc580c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 0x7fdc580c8d50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fdc58083060 tx=0x7fdc50009780).stop 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 shutdown_connections 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fdc4405ed90 0x7fdc44061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58154550 0x7fdc580c8d50 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc58069db0 0x7fdc581472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 --2- 172.21.15.92:0/549744003 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc58154170 0x7fdc580c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 >> 172.21.15.92:0/549744003 conn(0x7fdc580cb700 msgr2=0x7fdc580ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 shutdown_connections 2023-05-16T07:19:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.766+0000 7fdc5e1da700 1 -- 172.21.15.92:0/549744003 wait complete. 2023-05-16T07:19:23.789 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:19:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.883+0000 7f89facc0700 1 Processor -- start 2023-05-16T07:19:23.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.883+0000 7f89facc0700 1 -- start start 2023-05-16T07:19:23.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89facc0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4063d00 0x7f89f415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89facc0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f40aaf40 con 0x7f89f4153dc0 2023-05-16T07:19:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89facc0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f40c1450 con 0x7f89f41541a0 2023-05-16T07:19:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89facc0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40c1730 con 0x7f89f4063d00 2023-05-16T07:19:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89f8a5c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89f4153dc0 0x7f89f40c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59816/0 (socket says 172.21.15.92:59816) 2023-05-16T07:19:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89f8a5c700 1 -- 172.21.15.92:0/2988833160 learned_addr learned my addr 172.21.15.92:0/2988833160 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:23.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89f925d700 1 --2- 172.21.15.92:0/2988833160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4063d00 0x7f89f415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89f37fe700 1 -- 172.21.15.92:0/2988833160 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718332811 0 0) 0x7f89f40aaf40 con 0x7f89f4153dc0 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.884+0000 7f89f37fe700 1 -- 172.21.15.92:0/2988833160 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e00033f0 con 0x7f89f4153dc0 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f37fe700 1 -- 172.21.15.92:0/2988833160 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280463297 0 0) 0x7f89f40c1450 con 0x7f89f41541a0 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f925d700 1 -- 172.21.15.92:0/2988833160 >> v1:172.21.15.158:6789/0 conn(0x7f89f41541a0 legacy=0x7f89f40635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f925d700 1 -- 172.21.15.92:0/2988833160 >> v1:172.21.15.92:6789/0 conn(0x7f89f4153dc0 legacy=0x7f89f40c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f925d700 1 -- 172.21.15.92:0/2988833160 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f40c1a10 con 0x7f89f4063d00 2023-05-16T07:19:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f925d700 1 --2- 172.21.15.92:0/2988833160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4063d00 0x7f89f415eb30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f89ec009790 tx=0x7f89ec009060).ready entity=mon.2 client_cookie=2f4cb131ead88245 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f37fe700 1 -- 172.21.15.92:0/2988833160 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4001470 con 0x7f89f4063d00 2023-05-16T07:19:23.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f37fe700 1 -- 172.21.15.92:0/2988833160 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec012410 con 0x7f89f4063d00 2023-05-16T07:19:23.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89f37fe700 1 -- 172.21.15.92:0/2988833160 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec012790 con 0x7f89f4063d00 2023-05-16T07:19:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89facc0700 1 -- 172.21.15.92:0/2988833160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4063d00 msgr2=0x7f89f415eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.885+0000 7f89facc0700 1 --2- 172.21.15.92:0/2988833160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4063d00 0x7f89f415eb30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f89ec009790 tx=0x7f89ec009060).stop 2023-05-16T07:19:23.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.886+0000 7f89facc0700 1 -- 172.21.15.92:0/2988833160 shutdown_connections 2023-05-16T07:19:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.886+0000 7f89facc0700 1 --2- 172.21.15.92:0/2988833160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4063d00 0x7f89f415eb30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.886+0000 7f89facc0700 1 -- 172.21.15.92:0/2988833160 >> 172.21.15.92:0/2988833160 conn(0x7f89f40caca0 msgr2=0x7f89f40cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.886+0000 7f89facc0700 1 -- 172.21.15.92:0/2988833160 shutdown_connections 2023-05-16T07:19:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.886+0000 7f89facc0700 1 -- 172.21.15.92:0/2988833160 wait complete. 2023-05-16T07:19:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.887+0000 7f89facc0700 1 Processor -- start 2023-05-16T07:19:23.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.887+0000 7f89facc0700 1 -- start start 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89f4063d00 0x7f89f4158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4153dc0 0x7f89f40c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 0x7f89f4150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41591d0 con 0x7f89f41541a0 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40c3e40 con 0x7f89f4063d00 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40c4100 con 0x7f89f4153dc0 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 0x7f89f4150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 0x7f89f4150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49074/0 (socket says 172.21.15.92:49074) 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 -- 172.21.15.92:0/3922546409 learned_addr learned my addr 172.21.15.92:0/3922546409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f8a5c700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89f4063d00 0x7f89f4158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f3fff700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4153dc0 0x7f89f40c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 -- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4153dc0 msgr2=0x7f89f40c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4153dc0 0x7f89f40c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 -- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89f4063d00 msgr2=0x7f89f4158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:23.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89f4063d00 0x7f89f4158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 -- 172.21.15.92:0/3922546409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0092a0 con 0x7f89f41541a0 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f3fff700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4153dc0 0x7f89f40c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f925d700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 0x7f89f4150790 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f89ec009e30 tx=0x7f89ec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec021070 con 0x7f89f41541a0 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec011c70 con 0x7f89f41541a0 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f40c43e0 con 0x7f89f41541a0 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec01d3f0 con 0x7f89f41541a0 2023-05-16T07:19:23.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.888+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f40c49a0 con 0x7f89f41541a0 2023-05-16T07:19:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.889+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f405efa0 con 0x7f89f41541a0 2023-05-16T07:19:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.889+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f89ec01a030 con 0x7f89f41541a0 2023-05-16T07:19:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.890+0000 7f89f0ff9700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89d405ed90 0x7f89d4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.890+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f89ec07b370 con 0x7f89f41541a0 2023-05-16T07:19:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.890+0000 7f89f8a5c700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89d405ed90 0x7f89d4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.890+0000 7f89f8a5c700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89d405ed90 0x7f89d4061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f89e4000fd0 tx=0x7f89e4001b00).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:23.895+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f89ec04c6b0 con 0x7f89f41541a0 2023-05-16T07:19:24.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.095+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f89f405efa0 con 0x7f89f41541a0 2023-05-16T07:19:24.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.096+0000 7f89f0ff9700 1 -- 172.21.15.92:0/3922546409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89ec04fcd0 con 0x7f89f41541a0 2023-05-16T07:19:24.101 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:19:24.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89d405ed90 msgr2=0x7f89d4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:24.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89d405ed90 0x7f89d4061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f89e4000fd0 tx=0x7f89e4001b00).stop 2023-05-16T07:19:24.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 msgr2=0x7f89f4150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 0x7f89f4150790 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f89ec009e30 tx=0x7f89ec009060).stop 2023-05-16T07:19:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 shutdown_connections 2023-05-16T07:19:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f89d405ed90 0x7f89d4061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f41541a0 0x7f89f4150790 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89f4063d00 0x7f89f4158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 --2- 172.21.15.92:0/3922546409 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f89f4153dc0 0x7f89f40c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 >> 172.21.15.92:0/3922546409 conn(0x7f89f40caca0 msgr2=0x7f89f40ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 shutdown_connections 2023-05-16T07:19:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.097+0000 7f89facc0700 1 -- 172.21.15.92:0/3922546409 wait complete. 2023-05-16T07:19:24.113 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:19:24.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:19:24.114 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:19:24.114 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:19:24.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:24.116 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:19:24.117 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:19:24.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:19:24.119 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:19:24.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.120 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:19:24.135 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:19:24.136 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:24.137 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 40991sec preferred_lft 40991sec 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:19:24.139 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:19:24.140 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:24.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.140 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:19:24.198 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:19:24.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.199 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:19:24.255 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:19:24.255 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:19:24.255 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:19:24.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:19:24.258 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:19:24.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:19:24.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:19:24.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:19:24.259 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:19:24.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.353 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:19:24.367 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:19:24.368 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:19:24.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:19:24.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:19:24.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.431 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:24.485 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:19:24.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:19:24.563 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:19:24.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:19:24.605 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:19:24.606 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:19:24.607 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:19:24.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:24.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:24.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:19:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.819+0000 7fce1cc50700 1 Processor -- start 2023-05-16T07:19:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.819+0000 7fce1cc50700 1 -- start start 2023-05-16T07:19:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1cc50700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154170 0x7fce180c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1cc50700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce18154550 0x7fce180c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1cc50700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce180c0660 con 0x7fce180696a0 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1cc50700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce180c1b00 con 0x7fce18154550 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1cc50700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce180bfd30 con 0x7fce18154170 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce18154550 0x7fce180c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce18154550 0x7fce180c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49088/0 (socket says 172.21.15.92:49088) 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce16d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fce180696a0 0x7fce18157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59838/0 (socket says 172.21.15.92:59838) 2023-05-16T07:19:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 -- 172.21.15.92:0/478270012 learned_addr learned my addr 172.21.15.92:0/478270012 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1659c700 1 --2- 172.21.15.92:0/478270012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154170 0x7fce180c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 -- 172.21.15.92:0/478270012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154170 msgr2=0x7fce180c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 --2- 172.21.15.92:0/478270012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154170 0x7fce180c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 -- 172.21.15.92:0/478270012 >> v1:172.21.15.92:6789/0 conn(0x7fce180696a0 legacy=0x7fce18157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:24.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 -- 172.21.15.92:0/478270012 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180c0010 con 0x7fce18154550 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1559a700 1 -- 172.21.15.92:0/478270012 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907737720 0 0) 0x7fce180c0660 con 0x7fce180696a0 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce15d9b700 1 --2- 172.21.15.92:0/478270012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce18154550 0x7fce180c6160 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7fce0c009f20 tx=0x7fce0c009840).ready entity=mon.0 client_cookie=36629b59ae00b11a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.820+0000 7fce1559a700 1 -- 172.21.15.92:0/478270012 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00e070 con 0x7fce18154550 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1559a700 1 -- 172.21.15.92:0/478270012 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c00ba80 con 0x7fce18154550 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1559a700 1 -- 172.21.15.92:0/478270012 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00be00 con 0x7fce18154550 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 -- 172.21.15.92:0/478270012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce18154550 msgr2=0x7fce180c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 --2- 172.21.15.92:0/478270012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce18154550 0x7fce180c6160 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7fce0c009f20 tx=0x7fce0c009840).stop 2023-05-16T07:19:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 -- 172.21.15.92:0/478270012 shutdown_connections 2023-05-16T07:19:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 --2- 172.21.15.92:0/478270012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154170 0x7fce180c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 --2- 172.21.15.92:0/478270012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce18154550 0x7fce180c6160 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 -- 172.21.15.92:0/478270012 >> 172.21.15.92:0/478270012 conn(0x7fce180cb700 msgr2=0x7fce180c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 -- 172.21.15.92:0/478270012 shutdown_connections 2023-05-16T07:19:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.821+0000 7fce1cc50700 1 -- 172.21.15.92:0/478270012 wait complete. 2023-05-16T07:19:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.822+0000 7fce1cc50700 1 Processor -- start 2023-05-16T07:19:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.822+0000 7fce1cc50700 1 -- start start 2023-05-16T07:19:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1cc50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 0x7fce180c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1cc50700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce18154170 0x7fce180c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1cc50700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154550 0x7fce18157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1cc50700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce180c2720 con 0x7fce180696a0 2023-05-16T07:19:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1cc50700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce181490b0 con 0x7fce18154170 2023-05-16T07:19:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1cc50700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce18158e60 con 0x7fce18154550 2023-05-16T07:19:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 0x7fce180c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 0x7fce180c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49092/0 (socket says 172.21.15.92:49092) 2023-05-16T07:19:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 -- 172.21.15.92:0/1483080596 learned_addr learned my addr 172.21.15.92:0/1483080596 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce16d9d700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154550 0x7fce18157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce15d9b700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce18154170 0x7fce180c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 -- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154550 msgr2=0x7fce18157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154550 0x7fce18157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 -- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce18154170 msgr2=0x7fce180c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce18154170 0x7fce180c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 -- 172.21.15.92:0/1483080596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c009ae0 con 0x7fce180696a0 2023-05-16T07:19:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce1659c700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 0x7fce180c1d40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fce1000d8c0 tx=0x7fce10008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.823+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce100095b0 con 0x7fce180696a0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.824+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce10009710 con 0x7fce180696a0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.824+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce18159140 con 0x7fce180696a0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.824+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10009a90 con 0x7fce180696a0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.824+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce180aeb20 con 0x7fce180696a0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.824+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce180666c0 con 0x7fce180696a0 2023-05-16T07:19:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.825+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fce10013480 con 0x7fce180696a0 2023-05-16T07:19:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.825+0000 7fce067fc700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcdf805ed90 0x7fcdf8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.825+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fce10079e10 con 0x7fce180696a0 2023-05-16T07:19:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.825+0000 7fce15d9b700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcdf805ed90 0x7fcdf8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.826+0000 7fce15d9b700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcdf805ed90 0x7fcdf8061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fce0c009f20 tx=0x7fce0c009840).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:24.829+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fce1007d030 con 0x7fce180696a0 2023-05-16T07:19:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.019+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fce1806c350 con 0x7fcdf805ed90 2023-05-16T07:19:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.020+0000 7fce067fc700 1 -- 172.21.15.92:0/1483080596 <== mgr.6236 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fce1806c350 con 0x7fcdf805ed90 2023-05-16T07:19:25.026 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:19:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:19:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:25.027 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:19:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcdf805ed90 msgr2=0x7fcdf8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcdf805ed90 0x7fcdf8061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fce0c009f20 tx=0x7fce0c009840).stop 2023-05-16T07:19:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 msgr2=0x7fce180c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 0x7fce180c1d40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fce1000d8c0 tx=0x7fce10008040).stop 2023-05-16T07:19:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 shutdown_connections 2023-05-16T07:19:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fcdf805ed90 0x7fcdf8061250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fce18154550 0x7fce18157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce180696a0 0x7fce180c1d40 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 --2- 172.21.15.92:0/1483080596 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fce18154170 0x7fce180c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 >> 172.21.15.92:0/1483080596 conn(0x7fce180cb700 msgr2=0x7fce180accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 shutdown_connections 2023-05-16T07:19:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.022+0000 7fce1cc50700 1 -- 172.21.15.92:0/1483080596 wait complete. 2023-05-16T07:19:25.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:19:25.064 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-05-16T07:19:25.060+0000 7fb38d2db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:19:25.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.146+0000 7fc8e9960700 1 Processor -- start 2023-05-16T07:19:25.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.147+0000 7fc8e9960700 1 -- start start 2023-05-16T07:19:25.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.147+0000 7fc8e9960700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc8e4154090 0x7fc8e40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.147+0000 7fc8e9960700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40c94d0 con 0x7fc8e4154470 2023-05-16T07:19:25.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.147+0000 7fc8e9960700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40cb7e0 con 0x7fc8e4068160 2023-05-16T07:19:25.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.147+0000 7fc8e9960700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40c88c0 con 0x7fc8e4154090 2023-05-16T07:19:25.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.147+0000 7fc8e27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc8e4154470 0x7fc8e40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59852/0 (socket says 172.21.15.92:59852) 2023-05-16T07:19:25.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e27fc700 1 -- 172.21.15.92:0/2330708160 learned_addr learned my addr 172.21.15.92:0/2330708160 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:25.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e2ffd700 1 --2- 172.21.15.92:0/2330708160 >> v2:172.21.15.158:3300/0 conn(0x7fc8e4154090 0x7fc8e40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345837391 0 0) 0x7fc8e40c94d0 con 0x7fc8e4154470 2023-05-16T07:19:25.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8d00033f0 con 0x7fc8e4154470 2023-05-16T07:19:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 567067128 0 0) 0x7fc8d00033f0 con 0x7fc8e4154470 2023-05-16T07:19:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 >> v2:172.21.15.158:3300/0 conn(0x7fc8e4154090 msgr2=0x7fc8e40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 --2- 172.21.15.92:0/2330708160 >> v2:172.21.15.158:3300/0 conn(0x7fc8e4154090 0x7fc8e40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 >> v1:172.21.15.158:6789/0 conn(0x7fc8e4068160 legacy=0x7fc8e40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40c8ba0 con 0x7fc8e4154470 2023-05-16T07:19:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc8cc001470 con 0x7fc8e4154470 2023-05-16T07:19:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 >> v1:172.21.15.92:6789/0 conn(0x7fc8e4154470 legacy=0x7fc8e40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00038e0 0x7fc8d0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8d0004690 0x7fc8d0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8d000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c88c0 con 0x7fc8d0005000 2023-05-16T07:19:25.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d000bf60 con 0x7fc8d0004690 2023-05-16T07:19:25.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d000c0f0 con 0x7fc8d00038e0 2023-05-16T07:19:25.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e37fe700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8d000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e2ffd700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00038e0 0x7fc8d0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e27fc700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8d0004690 0x7fc8d0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.148+0000 7fc8e37fe700 1 -- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00038e0 msgr2=0x7fc8d0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e37fe700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8d00038e0 0x7fc8d0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e37fe700 1 -- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8d0004690 msgr2=0x7fc8d0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e37fe700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8d0004690 0x7fc8d0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e37fe700 1 -- 172.21.15.92:0/2330708160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00033f0 con 0x7fc8d0005000 2023-05-16T07:19:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e37fe700 1 -- 172.21.15.92:0/2330708160 reap_dead start 2023-05-16T07:19:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e37fe700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8d000b8e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fc8dc001730 tx=0x7fc8dc007bb0).ready entity=mon.0 client_cookie=e22aaac535beb84 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc007020 con 0x7fc8d0005000 2023-05-16T07:19:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc007180 con 0x7fc8d0005000 2023-05-16T07:19:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e1ffb700 1 -- 172.21.15.92:0/2330708160 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc007520 con 0x7fc8d0005000 2023-05-16T07:19:25.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 -- 172.21.15.92:0/2330708160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 msgr2=0x7fc8d000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8d000b8e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fc8dc001730 tx=0x7fc8dc007bb0).stop 2023-05-16T07:19:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 -- 172.21.15.92:0/2330708160 shutdown_connections 2023-05-16T07:19:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 --2- 172.21.15.92:0/2330708160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8d000b8e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 -- 172.21.15.92:0/2330708160 >> 172.21.15.92:0/2330708160 conn(0x7fc8e4067920 msgr2=0x7fc8e40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:25.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 -- 172.21.15.92:0/2330708160 shutdown_connections 2023-05-16T07:19:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.149+0000 7fc8e9960700 1 -- 172.21.15.92:0/2330708160 wait complete. 2023-05-16T07:19:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 Processor -- start 2023-05-16T07:19:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 -- start start 2023-05-16T07:19:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8e40adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8e41654e0 0x7fc8e414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8e40abb80 0x7fc8e40abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4164810 con 0x7fc8d0005000 2023-05-16T07:19:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e414e1c0 con 0x7fc8e40abb80 2023-05-16T07:19:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e9960700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40ac4d0 con 0x7fc8e41654e0 2023-05-16T07:19:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.151+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8e40adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8e40adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49112/0 (socket says 172.21.15.92:49112) 2023-05-16T07:19:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 -- 172.21.15.92:0/4059078679 learned_addr learned my addr 172.21.15.92:0/4059078679 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e37fe700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8e40abb80 0x7fc8e40abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e27fc700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8e41654e0 0x7fc8e414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 -- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8e41654e0 msgr2=0x7fc8e414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8e41654e0 0x7fc8e414dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 -- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8e40abb80 msgr2=0x7fc8e40abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8e40abb80 0x7fc8e40abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 -- 172.21.15.92:0/4059078679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40ae8f0 con 0x7fc8d0005000 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e27fc700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8e41654e0 0x7fc8e414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:25.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.152+0000 7fc8e2ffd700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8e40adbe0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fc8d4009e60 tx=0x7fc8d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:25.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.153+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400e040 con 0x7fc8d0005000 2023-05-16T07:19:25.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.153+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d400f040 con 0x7fc8d0005000 2023-05-16T07:19:25.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.153+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400fe50 con 0x7fc8d0005000 2023-05-16T07:19:25.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.153+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e4164a00 con 0x7fc8d0005000 2023-05-16T07:19:25.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.153+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e4164fc0 con 0x7fc8d0005000 2023-05-16T07:19:25.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.154+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e4066730 con 0x7fc8d0005000 2023-05-16T07:19:25.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.155+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 47) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc8d4015660 con 0x7fc8d0005000 2023-05-16T07:19:25.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.155+0000 7fc8daffd700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8c405ed90 0x7fc8c4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:25.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.155+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fc8d407b490 con 0x7fc8d0005000 2023-05-16T07:19:25.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.155+0000 7fc8e27fc700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8c405ed90 0x7fc8c4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:25.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.156+0000 7fc8e27fc700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8c405ed90 0x7fc8c4061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc8e4165940 tx=0x7fc8cc009040).ready entity=mgr.6236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:25.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.160+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc8d404c740 con 0x7fc8d0005000 2023-05-16T07:19:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:25.365+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc8e40c8e80 con 0x7fc8d0005000 2023-05-16T07:19:26.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.164+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 48) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fc8d40790a0 con 0x7fc8d0005000 2023-05-16T07:19:26.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.166+0000 7fc8daffd700 1 -- 172.21.15.92:0/4059078679 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v48) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc8d4047080 con 0x7fc8d0005000 2023-05-16T07:19:26.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8c405ed90 msgr2=0x7fc8c4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8c405ed90 0x7fc8c4061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc8e4165940 tx=0x7fc8cc009040).stop 2023-05-16T07:19:26.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 msgr2=0x7fc8e40adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8e40adbe0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fc8d4009e60 tx=0x7fc8d400b040).stop 2023-05-16T07:19:26.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 shutdown_connections 2023-05-16T07:19:26.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc8c405ed90 0x7fc8c4061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8e40abb80 0x7fc8e40abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d0005000 0x7fc8e40adbe0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 --2- 172.21.15.92:0/4059078679 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc8e41654e0 0x7fc8e414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.168+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 >> 172.21.15.92:0/4059078679 conn(0x7fc8e4067920 msgr2=0x7fc8e40bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:26.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.169+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 shutdown_connections 2023-05-16T07:19:26.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.169+0000 7fc8e9960700 1 -- 172.21.15.92:0/4059078679 wait complete. 2023-05-16T07:19:26.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:19:26.308 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:26.304+0000 7f556d430680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:26.314 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:26.311+0000 7f58a1fd7680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:26.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.343+0000 7f6fb6556700 1 Processor -- start 2023-05-16T07:19:26.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.343+0000 7f6fb6556700 1 -- start start 2023-05-16T07:19:26.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.344+0000 7f6fb6556700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 0x7f6fb00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.344+0000 7f6fb6556700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 0x7f6fb00685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.344+0000 7f6fb6556700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb00c94d0 con 0x7f6fb0154090 2023-05-16T07:19:26.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.344+0000 7f6fb6556700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb00cb7e0 con 0x7f6fb0068160 2023-05-16T07:19:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.344+0000 7f6fb6556700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0068ac0 con 0x7f6fb0154470 2023-05-16T07:19:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 0x7f6fb00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6fb4af3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 0x7f6fb00685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 0x7f6fb00aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:41720/0 (socket says 172.21.15.92:41720) 2023-05-16T07:19:26.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faffff700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f6fb0154090 0x7f6fb00a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:35262/0 (socket says 172.21.15.92:35262) 2023-05-16T07:19:26.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 -- 172.21.15.92:0/1165648058 learned_addr learned my addr 172.21.15.92:0/1165648058 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:26.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6fb4af3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 0x7f6fb00685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:52786/0 (socket says 172.21.15.92:52786) 2023-05-16T07:19:26.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faeffd700 1 -- 172.21.15.92:0/1165648058 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698011516 0 0) 0x7f6fb00c94d0 con 0x7f6fb0154090 2023-05-16T07:19:26.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faeffd700 1 -- 172.21.15.92:0/1165648058 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f980033f0 con 0x7f6fb0154090 2023-05-16T07:19:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 -- 172.21.15.92:0/1165648058 >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 msgr2=0x7f6fb00685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 --2- 172.21.15.92:0/1165648058 >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 0x7f6fb00685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 -- 172.21.15.92:0/1165648058 >> v1:172.21.15.158:6789/0 conn(0x7f6fb0154090 legacy=0x7f6fb00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 -- 172.21.15.92:0/1165648058 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00c88c0 con 0x7f6fb0154470 2023-05-16T07:19:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6fb4af3700 1 --2- 172.21.15.92:0/1165648058 >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 0x7f6fb00685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faf7fe700 1 --2- 172.21.15.92:0/1165648058 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 0x7f6fb00aaca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f6fa0006fd0 tx=0x7f6fa00090f0).ready entity=mon.2 client_cookie=f983d2ec150d8e4b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faeffd700 1 -- 172.21.15.92:0/1165648058 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000f510 con 0x7f6fb0154470 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faeffd700 1 -- 172.21.15.92:0/1165648058 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa000f670 con 0x7f6fb0154470 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.345+0000 7f6faeffd700 1 -- 172.21.15.92:0/1165648058 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000f9f0 con 0x7f6fb0154470 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 -- 172.21.15.92:0/1165648058 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 msgr2=0x7f6fb00aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 --2- 172.21.15.92:0/1165648058 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 0x7f6fb00aaca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f6fa0006fd0 tx=0x7f6fa00090f0).stop 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 -- 172.21.15.92:0/1165648058 shutdown_connections 2023-05-16T07:19:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 --2- 172.21.15.92:0/1165648058 >> v2:172.21.15.158:3300/0 conn(0x7f6fb0068160 0x7f6fb00685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 --2- 172.21.15.92:0/1165648058 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0154470 0x7f6fb00aaca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 -- 172.21.15.92:0/1165648058 >> 172.21.15.92:0/1165648058 conn(0x7f6fb0067920 msgr2=0x7f6fb00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:26.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 -- 172.21.15.92:0/1165648058 shutdown_connections 2023-05-16T07:19:26.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.346+0000 7f6fb6556700 1 -- 172.21.15.92:0/1165648058 wait complete. 2023-05-16T07:19:26.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.347+0000 7f6fb6556700 1 Processor -- start 2023-05-16T07:19:26.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.347+0000 7f6fb6556700 1 -- start start 2023-05-16T07:19:26.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb6556700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0068160 0x7f6fb01407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb6556700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb0154090 0x7f6fb00b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb6556700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fb0154470 0x7f6fb00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb6556700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01556f0 con 0x7f6fb0154090 2023-05-16T07:19:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb6556700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01559d0 con 0x7f6fb0154470 2023-05-16T07:19:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb6556700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0155cb0 con 0x7f6fb0068160 2023-05-16T07:19:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb0154090 0x7f6fb00b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0068160 0x7f6fb01407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb0154090 0x7f6fb00b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49154/0 (socket says 172.21.15.92:49154) 2023-05-16T07:19:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faf7fe700 1 -- 172.21.15.92:0/4277108429 learned_addr learned my addr 172.21.15.92:0/4277108429 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6fb4af3700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fb0154470 0x7f6fb00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faffff700 1 -- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fb0154470 msgr2=0x7f6fb00ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faffff700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fb0154470 0x7f6fb00ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faffff700 1 -- 172.21.15.92:0/4277108429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb0154090 msgr2=0x7f6fb00b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faffff700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb0154090 0x7f6fb00b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.348+0000 7f6faffff700 1 -- 172.21.15.92:0/4277108429 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009a20 con 0x7f6fb0068160 2023-05-16T07:19:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.349+0000 7f6faffff700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0068160 0x7f6fb01407c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f6fa400c440 tx=0x7f6fa4007bc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.349+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400cea0 con 0x7f6fb0068160 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.349+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa40072e0 con 0x7f6fb0068160 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.349+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa0009370 con 0x7f6fb0068160 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.349+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40056e0 con 0x7f6fb0068160 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.349+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb0156270 con 0x7f6fb0068160 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.350+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb0066730 con 0x7f6fb0068160 2023-05-16T07:19:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.350+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 4 ==== mgrmap(e 48) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f6fa401a060 con 0x7f6fb0068160 2023-05-16T07:19:26.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.350+0000 7f6f97fff700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f9c05edb0 0x7f6f9c061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.350+0000 7f6faf7fe700 1 -- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f9c05edb0 msgr2=0x7f6f9c061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:26.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.350+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f6fa40790b0 con 0x7f6fb0068160 2023-05-16T07:19:26.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.350+0000 7f6faf7fe700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f9c05edb0 0x7f6f9c061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:26.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.355+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6fa404a3f0 con 0x7f6fb0068160 2023-05-16T07:19:26.370 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:26.366+0000 7f556d430680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:26.380 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:26.376+0000 7f58a1fd7680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:26.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.523+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6fb00c1030 con 0x7f6fb0068160 2023-05-16T07:19:26.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.528+0000 7f6f97fff700 1 -- 172.21.15.92:0/4277108429 <== mon.2 v2:172.21.15.158:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6fa4018020 con 0x7f6fb0068160 2023-05-16T07:19:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f9c05edb0 msgr2=0x7f6f9c061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f9c05edb0 0x7f6f9c061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0068160 msgr2=0x7f6fb01407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0068160 0x7f6fb01407c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f6fa400c440 tx=0x7f6fa4007bc0).stop 2023-05-16T07:19:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 shutdown_connections 2023-05-16T07:19:26.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6f9c05edb0 0x7f6f9c061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6fb0068160 0x7f6fb01407c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6fb0154470 0x7f6fb00ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 --2- 172.21.15.92:0/4277108429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb0154090 0x7f6fb00b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 >> 172.21.15.92:0/4277108429 conn(0x7f6fb0067920 msgr2=0x7f6fb00c1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 shutdown_connections 2023-05-16T07:19:26.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.530+0000 7f6fb6556700 1 -- 172.21.15.92:0/4277108429 wait complete. 2023-05-16T07:19:26.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:26.547 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:19:26.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-05-16T07:19:26.639 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:26.635+0000 7f556d430680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:26.660 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:26.656+0000 7f58a1fd7680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:26.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.722+0000 7fc28f4d2700 1 Processor -- start 2023-05-16T07:19:26.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.723+0000 7fc28f4d2700 1 -- start start 2023-05-16T07:19:26.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.723+0000 7fc28f4d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 0x7fc288150650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.723+0000 7fc28f4d2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880c3130 con 0x7fc2881546f0 2023-05-16T07:19:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28f4d2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc288068540 con 0x7fc288154310 2023-05-16T07:19:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28f4d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288154b20 con 0x7fc28806a0f0 2023-05-16T07:19:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28da6f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc2881546f0 0x7fc2880c16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59928/0 (socket says 172.21.15.92:59928) 2023-05-16T07:19:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28da6f700 1 -- 172.21.15.92:0/1957032398 learned_addr learned my addr 172.21.15.92:0/1957032398 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28ca6d700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 0x7fc288150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765936862 0 0) 0x7fc2880c3130 con 0x7fc2881546f0 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc274004500 con 0x7fc2881546f0 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2264314389 0 0) 0x7fc274004500 con 0x7fc2881546f0 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 msgr2=0x7fc288150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 0x7fc288150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 >> v1:172.21.15.158:6789/0 conn(0x7fc288154310 legacy=0x7fc28814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880c2890 con 0x7fc2881546f0 2023-05-16T07:19:26.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc2800017f0 con 0x7fc2881546f0 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 >> v1:172.21.15.92:6789/0 conn(0x7fc2881546f0 legacy=0x7fc2880c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc274004a20 0x7fc274004e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 0x7fc2740057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc274005c90 0x7fc274006110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288154b20 con 0x7fc274004a20 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400c9f0 con 0x7fc274005c90 2023-05-16T07:19:26.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400cb80 con 0x7fc28806a0f0 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28d26e700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc274004a20 0x7fc274004e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28ca6d700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 0x7fc2740057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.724+0000 7fc28da6f700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc274005c90 0x7fc274006110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc28d26e700 1 -- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 msgr2=0x7fc2740057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc28d26e700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc28806a0f0 0x7fc2740057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc28d26e700 1 -- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc274005c90 msgr2=0x7fc274006110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc28d26e700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc274005c90 0x7fc274006110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc28d26e700 1 -- 172.21.15.92:0/1957032398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274004500 con 0x7fc274004a20 2023-05-16T07:19:26.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc28d26e700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc274004a20 0x7fc274004e00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fc278001810 tx=0x7fc278007b70).ready entity=mon.0 client_cookie=d640857cf2f8f964 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:26.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278013410 con 0x7fc274004a20 2023-05-16T07:19:26.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc278013570 con 0x7fc274004a20 2023-05-16T07:19:26.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.725+0000 7fc27ffff700 1 -- 172.21.15.92:0/1957032398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780138f0 con 0x7fc274004a20 2023-05-16T07:19:26.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28f4d2700 1 -- 172.21.15.92:0/1957032398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc274004a20 msgr2=0x7fc274004e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/1957032398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc274004a20 0x7fc274004e00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fc278001810 tx=0x7fc278007b70).stop 2023-05-16T07:19:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28da6f700 1 -- 172.21.15.92:0/1957032398 reap_dead start 2023-05-16T07:19:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28f4d2700 1 -- 172.21.15.92:0/1957032398 shutdown_connections 2023-05-16T07:19:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28f4d2700 1 -- 172.21.15.92:0/1957032398 >> 172.21.15.92:0/1957032398 conn(0x7fc2880c3d90 msgr2=0x7fc2880c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:26.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28f4d2700 1 -- 172.21.15.92:0/1957032398 shutdown_connections 2023-05-16T07:19:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.726+0000 7fc28f4d2700 1 -- 172.21.15.92:0/1957032398 wait complete. 2023-05-16T07:19:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.727+0000 7fc28f4d2700 1 Processor -- start 2023-05-16T07:19:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.727+0000 7fc28f4d2700 1 -- start start 2023-05-16T07:19:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28f4d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc274004a20 0x7fc2880a7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28f4d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 0x7fc2881526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28f4d2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2880c7380 0x7fc288152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28f4d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2881530d0 con 0x7fc2880c6f00 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28f4d2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c6a70 con 0x7fc2880c7380 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28f4d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288163bf0 con 0x7fc274004a20 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 0x7fc2881526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 0x7fc2881526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49192/0 (socket says 172.21.15.92:49192) 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 -- 172.21.15.92:0/746618327 learned_addr learned my addr 172.21.15.92:0/746618327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28da6f700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2880c7380 0x7fc288152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc274004a20 0x7fc2880a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc274004a20 msgr2=0x7fc2880a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc274004a20 0x7fc2880a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2880c7380 msgr2=0x7fc288152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2880c7380 0x7fc288152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 -- 172.21.15.92:0/746618327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880c6420 con 0x7fc2880c6f00 2023-05-16T07:19:26.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc28ca6d700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 0x7fc2881526f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7fc284011a20 tx=0x7fc284009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:26.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.728+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284011e60 con 0x7fc2880c6f00 2023-05-16T07:19:26.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.729+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28400f940 con 0x7fc2880c6f00 2023-05-16T07:19:26.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.729+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2880c6700 con 0x7fc2880c6f00 2023-05-16T07:19:26.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.729+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28400fcc0 con 0x7fc2880c6f00 2023-05-16T07:19:26.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.729+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880c6040 con 0x7fc2880c6f00 2023-05-16T07:19:26.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.729+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28806c350 con 0x7fc2880c6f00 2023-05-16T07:19:26.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.730+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 48) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fc284024070 con 0x7fc2880c6f00 2023-05-16T07:19:26.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.730+0000 7fc27d7fa700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:26.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.730+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fc28407a7d0 con 0x7fc2880c6f00 2023-05-16T07:19:26.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.730+0000 7fc28d26e700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:26.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.730+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:26.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.735+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc28404ba00 con 0x7fc2880c6f00 2023-05-16T07:19:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.931+0000 7fc28d26e700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.931+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:26.934+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2880c6bb0 con 0x7fc26c05ede0 2023-05-16T07:19:27.247 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:27.242+0000 7f556d430680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:27.290 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:27.276+0000 7f58a1fd7680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:27.318 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:27.313+0000 7f556d430680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:27.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:27.332+0000 7fc28d26e700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:27.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:27.332+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:19:27.351 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:27.347+0000 7f58a1fd7680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:27.500 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:27.496+0000 7f556d430680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:27.533 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:27.530+0000 7f58a1fd7680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:27.912 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:27.907+0000 7f556d430680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:27.944 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:27.941+0000 7f58a1fd7680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:28.013 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:28.009+0000 7f556d430680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:28.046 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:28.042+0000 7f58a1fd7680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:28.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:28.133+0000 7fc28d26e700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:28.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:28.133+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:19:28.165 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:28.161+0000 7f556d430680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:28.196 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:28.193+0000 7f58a1fd7680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:28.237 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:28.233+0000 7f556d430680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:28.268 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:28.265+0000 7f58a1fd7680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:28.304 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:28.300+0000 7f556d430680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:28.333 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:28.330+0000 7f58a1fd7680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:28.555 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:28.550+0000 7f556d430680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:28.586 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:28.582+0000 7f58a1fd7680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:28.737 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:28.733+0000 7f556d430680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:28.765 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:28.762+0000 7f58a1fd7680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:29.374 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.369+0000 7f556d430680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:29.399 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.396+0000 7f58a1fd7680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:29.439 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.435+0000 7f556d430680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:29.466 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.463+0000 7f58a1fd7680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:29.512 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.508+0000 7f556d430680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:29.539 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.536+0000 7f58a1fd7680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:29.654 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.650+0000 7f556d430680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:29.682 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.679+0000 7f58a1fd7680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:29.723 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.719+0000 7f556d430680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:29.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:29.735+0000 7fc28d26e700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:29.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:29.735+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:19:29.751 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.747+0000 7f58a1fd7680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:29.835 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.831+0000 7f556d430680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:29.862 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.859+0000 7f58a1fd7680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:29.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:29.936+0000 7f556d430680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:29.968 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:29.965+0000 7f58a1fd7680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:30.227 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.223+0000 7f556d430680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:19:30.252 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:30.249+0000 7f58a1fd7680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:19:30.296 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.292+0000 7f556d430680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:19:30.323 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:30.319+0000 7f58a1fd7680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:19:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:30.362+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 49) v1 ==== 75109+0+0 (secure 0 0 0) 0x7fc284078360 con 0x7fc2880c6f00 2023-05-16T07:19:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:30.362+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:30.362+0000 7fc27d7fa700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:30.425 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54e8c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.425 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54e8c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.425 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54e8c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54e8c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54e8c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54ebc12700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54ebc12700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54ebc12700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54ebc12700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:30.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:30.420+0000 7f54ebc12700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:31.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.373+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 50) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fc2840786e0 con 0x7fc2880c6f00 2023-05-16T07:19:31.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.374+0000 7fc27d7fa700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c02fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.374+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2880c6bb0 con 0x7fc26c05ede0 2023-05-16T07:19:31.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.376+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c02fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.384+0000 7fc28d26e700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c02fd40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc278007f70 tx=0x7fc278016000).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.389+0000 7fc27d7fa700 1 -- 172.21.15.92:0/746618327 <== mgr.6467 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc2880c6bb0 con 0x7fc26c05ede0 2023-05-16T07:19:31.394 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path snap_test_dir1 2023-05-16T07:19:31.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 msgr2=0x7fc26c02fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c02fd40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc278007f70 tx=0x7fc278016000).stop 2023-05-16T07:19:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 msgr2=0x7fc2881526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 0x7fc2881526f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7fc284011a20 tx=0x7fc284009060).stop 2023-05-16T07:19:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 shutdown_connections 2023-05-16T07:19:31.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc26c05ede0 0x7fc26c02fd40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc274004a20 0x7fc2880a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2880c7380 0x7fc288152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 --2- 172.21.15.92:0/746618327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2880c6f00 0x7fc2881526f0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 >> 172.21.15.92:0/746618327 conn(0x7fc2880c3d90 msgr2=0x7fc288063d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 shutdown_connections 2023-05-16T07:19:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.392+0000 7fc28f4d2700 1 -- 172.21.15.92:0/746618327 wait complete. 2023-05-16T07:19:31.410 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-05-16T07:19:31.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-05-16T07:19:31.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.525+0000 7f5c757fd700 1 Processor -- start 2023-05-16T07:19:31.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.525+0000 7f5c757fd700 1 -- start start 2023-05-16T07:19:31.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c757fd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c70153f90 0x7f5c700a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c757fd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5c700c5d00 0x7f5c700c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c757fd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c700c0150 0x7f5c700c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c757fd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c70069f60 con 0x7f5c70153f90 2023-05-16T07:19:31.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c757fd700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7006a9e0 con 0x7f5c700c5d00 2023-05-16T07:19:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c757fd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700c66a0 con 0x7f5c700c0150 2023-05-16T07:19:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c6ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c70153f90 0x7f5c700a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c6ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c70153f90 0x7f5c700a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49322/0 (socket says 172.21.15.92:49322) 2023-05-16T07:19:31.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.526+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3447393027 learned_addr learned my addr 172.21.15.92:0/3447393027 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c67fff700 1 --2- 172.21.15.92:0/3447393027 >> v2:172.21.15.158:3300/0 conn(0x7f5c700c5d00 0x7f5c700c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3447393027 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c700c0150 msgr2=0x7f5c700c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c74ffc700 1 --2- 172.21.15.92:0/3447393027 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c700c0150 0x7f5c700c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6ffff700 1 --2- 172.21.15.92:0/3447393027 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c700c0150 0x7f5c700c05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3447393027 >> v2:172.21.15.158:3300/0 conn(0x7f5c700c5d00 msgr2=0x7f5c700c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6ffff700 1 --2- 172.21.15.92:0/3447393027 >> v2:172.21.15.158:3300/0 conn(0x7f5c700c5d00 0x7f5c700c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3447393027 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7006a140 con 0x7f5c70153f90 2023-05-16T07:19:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c67fff700 1 --2- 172.21.15.92:0/3447393027 >> v2:172.21.15.158:3300/0 conn(0x7f5c700c5d00 0x7f5c700c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6ffff700 1 --2- 172.21.15.92:0/3447393027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c70153f90 0x7f5c700a9d20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f5c700c8240 tx=0x7f5c600090f0).ready entity=mon.0 client_cookie=d922bf750c07a609 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6f7fe700 1 -- 172.21.15.92:0/3447393027 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000b600 con 0x7f5c70153f90 2023-05-16T07:19:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6f7fe700 1 -- 172.21.15.92:0/3447393027 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6000b760 con 0x7f5c70153f90 2023-05-16T07:19:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c6f7fe700 1 -- 172.21.15.92:0/3447393027 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000bae0 con 0x7f5c70153f90 2023-05-16T07:19:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c757fd700 1 -- 172.21.15.92:0/3447393027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c70153f90 msgr2=0x7f5c700a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.527+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3447393027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c70153f90 0x7f5c700a9d20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f5c700c8240 tx=0x7f5c600090f0).stop 2023-05-16T07:19:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 -- 172.21.15.92:0/3447393027 shutdown_connections 2023-05-16T07:19:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3447393027 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c700c0150 0x7f5c700c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3447393027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c70153f90 0x7f5c700a9d20 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3447393027 >> v2:172.21.15.158:3300/0 conn(0x7f5c700c5d00 0x7f5c700c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 -- 172.21.15.92:0/3447393027 >> 172.21.15.92:0/3447393027 conn(0x7f5c700cb600 msgr2=0x7f5c700c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 -- 172.21.15.92:0/3447393027 shutdown_connections 2023-05-16T07:19:31.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.528+0000 7f5c757fd700 1 -- 172.21.15.92:0/3447393027 wait complete. 2023-05-16T07:19:31.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 Processor -- start 2023-05-16T07:19:31.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 -- start start 2023-05-16T07:19:31.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 0x7f5c70147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5c700c5d00 0x7f5c700a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c70153f90 0x7f5c700b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700adec0 con 0x7f5c700c0150 2023-05-16T07:19:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700ae1a0 con 0x7f5c700c5d00 2023-05-16T07:19:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.529+0000 7f5c757fd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700ae480 con 0x7f5c70153f90 2023-05-16T07:19:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 0x7f5c70147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 0x7f5c70147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49326/0 (socket says 172.21.15.92:49326) 2023-05-16T07:19:31.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3886002079 learned_addr learned my addr 172.21.15.92:0/3886002079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c67fff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5c700c5d00 0x7f5c700a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c74ffc700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c70153f90 0x7f5c700b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c70153f90 msgr2=0x7f5c700b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c70153f90 0x7f5c700b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5c700c5d00 msgr2=0x7f5c700a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5c700c5d00 0x7f5c700a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 -- 172.21.15.92:0/3886002079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009370 con 0x7f5c700c0150 2023-05-16T07:19:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c67fff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5c700c5d00 0x7f5c700a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6ffff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 0x7f5c70147020 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f5c60009130 tx=0x7f5c60017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000bae0 con 0x7f5c700c0150 2023-05-16T07:19:31.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60000c90 con 0x7f5c700c0150 2023-05-16T07:19:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c700ae760 con 0x7f5c700c0150 2023-05-16T07:19:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6001b660 con 0x7f5c700c0150 2023-05-16T07:19:31.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.530+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c700adae0 con 0x7f5c700c0150 2023-05-16T07:19:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.531+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c70167a40 con 0x7f5c700c0150 2023-05-16T07:19:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.531+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 50) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f5c6000f070 con 0x7f5c700c0150 2023-05-16T07:19:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.532+0000 7f5c6cff9700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5c5005ed40 0x7f5c50061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.532+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f5c6007b600 con 0x7f5c700c0150 2023-05-16T07:19:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.532+0000 7f5c67fff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5c5005ed40 0x7f5c50061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.532+0000 7f5c67fff700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5c5005ed40 0x7f5c50061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f5c700aee50 tx=0x7f5c58008040).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.536+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5c60050360 con 0x7f5c700c0150 2023-05-16T07:19:31.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.725+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c700ae310 con 0x7f5c5005ed40 2023-05-16T07:19:31.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.729+0000 7f5c6cff9700 1 -- 172.21.15.92:0/3886002079 <== mgr.6467 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5c700ae310 con 0x7f5c5005ed40 2023-05-16T07:19:31.734 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path snap_test_dir1 2023-05-16T07:19:31.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5c5005ed40 msgr2=0x7f5c50061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5c5005ed40 0x7f5c50061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f5c700aee50 tx=0x7f5c58008040).stop 2023-05-16T07:19:31.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 msgr2=0x7f5c70147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 0x7f5c70147020 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f5c60009130 tx=0x7f5c60017000).stop 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 shutdown_connections 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f5c5005ed40 0x7f5c50061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5c70153f90 0x7f5c700b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c700c0150 0x7f5c70147020 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 --2- 172.21.15.92:0/3886002079 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5c700c5d00 0x7f5c700a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.731+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 >> 172.21.15.92:0/3886002079 conn(0x7f5c700cb600 msgr2=0x7f5c700c05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:31.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.732+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 shutdown_connections 2023-05-16T07:19:31.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.732+0000 7f5c757fd700 1 -- 172.21.15.92:0/3886002079 wait complete. 2023-05-16T07:19:31.748 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-05-16T07:19:31.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-05-16T07:19:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.859+0000 7f531dd6a700 1 Processor -- start 2023-05-16T07:19:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.859+0000 7f531dd6a700 1 -- start start 2023-05-16T07:19:31.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f531dd6a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5318154090 0x7f53180aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f531dd6a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5318068160 0x7f53180685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f531dd6a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53180c94d0 con 0x7f5318154470 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f531dd6a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53180cb7e0 con 0x7f5318154090 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f531dd6a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5318068aa0 con 0x7f5318068160 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5318154090 0x7f53180aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5318154090 0x7f53180aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49334/0 (socket says 172.21.15.92:49334) 2023-05-16T07:19:31.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 -- 172.21.15.92:0/2220962423 learned_addr learned my addr 172.21.15.92:0/2220962423 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:31.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 -- 172.21.15.92:0/2220962423 >> v2:172.21.15.158:3300/0 conn(0x7f5318068160 msgr2=0x7f53180685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:31.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 --2- 172.21.15.92:0/2220962423 >> v2:172.21.15.158:3300/0 conn(0x7f5318068160 0x7f53180685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:31.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 -- 172.21.15.92:0/2220962423 >> v1:172.21.15.158:6789/0 conn(0x7f5318154470 legacy=0x7f53180a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:31.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 -- 172.21.15.92:0/2220962423 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180c88c0 con 0x7f5318154090 2023-05-16T07:19:31.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53177fe700 1 --2- 172.21.15.92:0/2220962423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154090 0x7f53180aaca0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f531805f9e0 tx=0x7f5308009840).ready entity=mon.0 client_cookie=4b1c12df13c6e51a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.860+0000 7f53167fc700 1 -- 172.21.15.92:0/2220962423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530800e070 con 0x7f5318154090 2023-05-16T07:19:31.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f53167fc700 1 -- 172.21.15.92:0/2220962423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530800ba10 con 0x7f5318154090 2023-05-16T07:19:31.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f53167fc700 1 -- 172.21.15.92:0/2220962423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530800bd90 con 0x7f5318154090 2023-05-16T07:19:31.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 -- 172.21.15.92:0/2220962423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154090 msgr2=0x7f53180aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 --2- 172.21.15.92:0/2220962423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154090 0x7f53180aaca0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f531805f9e0 tx=0x7f5308009840).stop 2023-05-16T07:19:31.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 -- 172.21.15.92:0/2220962423 shutdown_connections 2023-05-16T07:19:31.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 --2- 172.21.15.92:0/2220962423 >> v2:172.21.15.158:3300/0 conn(0x7f5318068160 0x7f53180685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 --2- 172.21.15.92:0/2220962423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154090 0x7f53180aaca0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 -- 172.21.15.92:0/2220962423 >> 172.21.15.92:0/2220962423 conn(0x7f5318067920 msgr2=0x7f53180cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 -- 172.21.15.92:0/2220962423 shutdown_connections 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.861+0000 7f531dd6a700 1 -- 172.21.15.92:0/2220962423 wait complete. 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.862+0000 7f531dd6a700 1 Processor -- start 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.862+0000 7f531dd6a700 1 -- start start 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f531dd6a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5318068160 0x7f531814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f531dd6a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5318154090 0x7f5318152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f531dd6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154470 0x7f53181532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f531dd6a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53181537b0 con 0x7f5318154470 2023-05-16T07:19:31.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f531dd6a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5318164f90 con 0x7f5318068160 2023-05-16T07:19:31.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f531dd6a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53181643c0 con 0x7f5318154090 2023-05-16T07:19:31.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154470 0x7f53181532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154470 0x7f53181532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49340/0 (socket says 172.21.15.92:49340) 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 -- 172.21.15.92:0/3512214167 learned_addr learned my addr 172.21.15.92:0/3512214167 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 -- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5318154090 msgr2=0x7f5318152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 --2- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5318154090 0x7f5318152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 -- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5318068160 msgr2=0x7f531814ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 --2- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5318068160 0x7f531814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.863+0000 7f5317fff700 1 -- 172.21.15.92:0/3512214167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308009ae0 con 0x7f5318154470 2023-05-16T07:19:31.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.864+0000 7f5317fff700 1 --2- 172.21.15.92:0/3512214167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5318154470 0x7f53181532c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f531000b5e0 tx=0x7f5310009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.864+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000d4a0 con 0x7f5318154470 2023-05-16T07:19:31.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.864+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531000d620 con 0x7f5318154470 2023-05-16T07:19:31.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.864+0000 7f531dd6a700 1 -- 172.21.15.92:0/3512214167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53180c1f70 con 0x7f5318154470 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.864+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000d9e0 con 0x7f5318154470 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.864+0000 7f531dd6a700 1 -- 172.21.15.92:0/3512214167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5318164c20 con 0x7f5318154470 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.865+0000 7f531dd6a700 1 -- 172.21.15.92:0/3512214167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5318066730 con 0x7f5318154470 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.865+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 50) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f531000f030 con 0x7f5318154470 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.865+0000 7f52ff7fe700 1 --2- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530405ed40 0x7f5304061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.865+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f5310079bd0 con 0x7f5318154470 2023-05-16T07:19:31.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.865+0000 7f53177fe700 1 --2- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530405ed40 0x7f5304061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:31.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.866+0000 7f53177fe700 1 --2- 172.21.15.92:0/3512214167 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f530405ed40 0x7f5304061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f530800b9e0 tx=0x7f5308009840).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:31.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:31.869+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f531004af00 con 0x7f5318154470 2023-05-16T07:19:32.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.053+0000 7f531dd6a700 1 -- 172.21.15.92:0/3512214167 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f53180c8ba0 con 0x7f530405ed40 2023-05-16T07:19:32.057 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:32.053+0000 7f551aebb700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-05-16T07:19:32.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.054+0000 7f52ff7fe700 1 -- 172.21.15.92:0/3512214167 <== mgr.6467 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f53180c8ba0 con 0x7f530405ed40 2023-05-16T07:19:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-05-16T07:19:32.062 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-05-16T07:19:32.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:32.064 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-05-16T07:19:32.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:19:32.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.230+0000 7f1da7852700 1 Processor -- start 2023-05-16T07:19:32.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.231+0000 7f1da7852700 1 -- start start 2023-05-16T07:19:32.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.231+0000 7f1da7852700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1da0154470 0x7f1da00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.231+0000 7f1da7852700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00c94d0 con 0x7f1da0154090 2023-05-16T07:19:32.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.231+0000 7f1da7852700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00cb7e0 con 0x7f1da0068160 2023-05-16T07:19:32.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.231+0000 7f1da7852700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00c88c0 con 0x7f1da0154470 2023-05-16T07:19:32.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1da55ee700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1da0154090 0x7f1da00a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60094/0 (socket says 172.21.15.92:60094) 2023-05-16T07:19:32.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1da55ee700 1 -- 172.21.15.92:0/2068244819 learned_addr learned my addr 172.21.15.92:0/2068244819 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:32.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1da4ded700 1 --2- 172.21.15.92:0/2068244819 >> v2:172.21.15.158:3300/0 conn(0x7f1da0154470 0x7f1da00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771244210 0 0) 0x7f1da00c94d0 con 0x7f1da0154090 2023-05-16T07:19:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d8c0033f0 con 0x7f1da0154090 2023-05-16T07:19:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223150634 0 0) 0x7f1d8c0033f0 con 0x7f1da0154090 2023-05-16T07:19:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 >> v2:172.21.15.158:3300/0 conn(0x7f1da0154470 msgr2=0x7f1da00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 --2- 172.21.15.92:0/2068244819 >> v2:172.21.15.158:3300/0 conn(0x7f1da0154470 0x7f1da00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 >> v1:172.21.15.158:6789/0 conn(0x7f1da0068160 legacy=0x7f1da00c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00c8ba0 con 0x7f1da0154090 2023-05-16T07:19:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f1d9c001470 con 0x7f1da0154090 2023-05-16T07:19:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 >> v1:172.21.15.92:6789/0 conn(0x7f1da0154090 legacy=0x7f1da00a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1d8c0038e0 0x7f1d8c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1d8c004690 0x7f1d8c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1d8c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c88c0 con 0x7f1d8c005000 2023-05-16T07:19:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00bf60 con 0x7f1d8c004690 2023-05-16T07:19:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00c0f0 con 0x7f1d8c0038e0 2023-05-16T07:19:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1da5def700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1d8c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1da4ded700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1d8c004690 0x7f1d8c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.232+0000 7f1da55ee700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1d8c0038e0 0x7f1d8c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 -- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1d8c0038e0 msgr2=0x7f1d8c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1d8c0038e0 0x7f1d8c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 -- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1d8c004690 msgr2=0x7f1d8c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1d8c004690 0x7f1d8c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 -- 172.21.15.92:0/2068244819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0033f0 con 0x7f1d8c005000 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 -- 172.21.15.92:0/2068244819 reap_dead start 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da5def700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1d8c00b8e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f1d98001e30 tx=0x7f1d98007f30).ready entity=mon.0 client_cookie=5b57773d294e5492 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98006a20 con 0x7f1d8c005000 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98006b80 con 0x7f1d8c005000 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1d97fff700 1 -- 172.21.15.92:0/2068244819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800c5d0 con 0x7f1d8c005000 2023-05-16T07:19:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 -- 172.21.15.92:0/2068244819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 msgr2=0x7f1d8c00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1d8c00b8e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f1d98001e30 tx=0x7f1d98007f30).stop 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 -- 172.21.15.92:0/2068244819 shutdown_connections 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 --2- 172.21.15.92:0/2068244819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1d8c00b8e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 -- 172.21.15.92:0/2068244819 >> 172.21.15.92:0/2068244819 conn(0x7f1da0067920 msgr2=0x7f1da00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 -- 172.21.15.92:0/2068244819 shutdown_connections 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.233+0000 7f1da7852700 1 -- 172.21.15.92:0/2068244819 wait complete. 2023-05-16T07:19:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.234+0000 7f1da7852700 1 Processor -- start 2023-05-16T07:19:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 -- start start 2023-05-16T07:19:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1da00c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1da0167b80 0x7f1da0152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1da0153320 0x7f1da00c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c1cd0 con 0x7f1d8c005000 2023-05-16T07:19:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0168080 con 0x7f1da0167b80 2023-05-16T07:19:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da7852700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0168360 con 0x7f1da0153320 2023-05-16T07:19:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da55ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1da00c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da55ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1da00c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49358/0 (socket says 172.21.15.92:49358) 2023-05-16T07:19:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.235+0000 7f1da55ee700 1 -- 172.21.15.92:0/2706320460 learned_addr learned my addr 172.21.15.92:0/2706320460 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da4ded700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1da0167b80 0x7f1da0152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da5def700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1da0153320 0x7f1da00c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da55ee700 1 -- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1da0153320 msgr2=0x7f1da00c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da55ee700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1da0153320 0x7f1da00c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da55ee700 1 -- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1da0167b80 msgr2=0x7f1da0152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da55ee700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1da0167b80 0x7f1da0152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da55ee700 1 -- 172.21.15.92:0/2706320460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0168640 con 0x7f1d8c005000 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da4ded700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1da0167b80 0x7f1da0152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da55ee700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1da00c5050 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f1d9c004730 tx=0x7f1d9c001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c000ed0 con 0x7f1d8c005000 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00f5f0 con 0x7f1d8c005000 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da0168920 con 0x7f1d8c005000 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00f990 con 0x7f1d8c005000 2023-05-16T07:19:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.236+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da00c1050 con 0x7f1d8c005000 2023-05-16T07:19:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.237+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da0066730 con 0x7f1d8c005000 2023-05-16T07:19:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.237+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 50) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f1d9c01b030 con 0x7f1d8c005000 2023-05-16T07:19:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.237+0000 7f1d957fa700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1d8405ed40 0x7f1d84061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.238+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f1d9c079540 con 0x7f1d8c005000 2023-05-16T07:19:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.238+0000 7f1da4ded700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1d8405ed40 0x7f1d84061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.238+0000 7f1da4ded700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1d8405ed40 0x7f1d84061200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1d90006d50 tx=0x7f1d90008040).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.242+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1d9c04e2f0 con 0x7f1d8c005000 2023-05-16T07:19:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.386+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 51) v1 ==== 75379+0+0 (secure 0 0 0) 0x7f1d9c0771d0 con 0x7f1d8c005000 2023-05-16T07:19:32.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:32.453+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f1da006c350 con 0x7f1d8c005000 2023-05-16T07:19:33.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.389+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f1d9c077550 con 0x7f1d8c005000 2023-05-16T07:19:33.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.391+0000 7f1d957fa700 1 -- 172.21.15.92:0/2706320460 <== mon.0 v2:172.21.15.92:3300/0 9 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v52) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1d9c04b8d0 con 0x7f1d8c005000 2023-05-16T07:19:33.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1d8405ed40 msgr2=0x7f1d84061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1d8405ed40 0x7f1d84061200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1d90006d50 tx=0x7f1d90008040).stop 2023-05-16T07:19:33.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 msgr2=0x7f1da00c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1da00c5050 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f1d9c004730 tx=0x7f1d9c001e50).stop 2023-05-16T07:19:33.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 shutdown_connections 2023-05-16T07:19:33.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1d8405ed40 0x7f1d84061200 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1da0153320 0x7f1da00c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c005000 0x7f1da00c5050 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 --2- 172.21.15.92:0/2706320460 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1da0167b80 0x7f1da0152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 >> 172.21.15.92:0/2706320460 conn(0x7f1da0067920 msgr2=0x7f1da00aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:33.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 shutdown_connections 2023-05-16T07:19:33.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.393+0000 7f1da7852700 1 -- 172.21.15.92:0/2706320460 wait complete. 2023-05-16T07:19:33.411 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:19:33.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:33.446 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:33.475 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:19:33.475 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:19:33.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:33.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:33.530 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:19:33.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:33.531 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:19:33.537 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:33.533+0000 7f2fb3812680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:33.537 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:33.533+0000 7fa92f2e3680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:33.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:33.598 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:33.594+0000 7fa92f2e3680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:33.601 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:33.598+0000 7f2fb3812680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:33.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.702+0000 7fe7ad94b700 1 Processor -- start 2023-05-16T07:19:33.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.702+0000 7fe7ad94b700 1 -- start start 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7ad94b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe7a8063e30 0x7fe7a8069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7ad94b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 0x7fe7a8062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7ad94b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a8062960 con 0x7fe7a8154040 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7ad94b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a8154420 con 0x7fe7a8063e30 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7ad94b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a8069fb0 con 0x7fe7a80bf4f0 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7a77fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 0x7fe7a8062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7a67fc700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe7a8063e30 0x7fe7a8069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:33.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7a77fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 0x7fe7a8062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:41994/0 (socket says 172.21.15.92:41994) 2023-05-16T07:19:33.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.703+0000 7fe7a77fe700 1 -- 172.21.15.92:0/1866775116 learned_addr learned my addr 172.21.15.92:0/1866775116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:33.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 msgr2=0x7fe7a8062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:33.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a67fc700 1 --2- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 0x7fe7a8062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1866775116 >> v1:172.21.15.158:6789/0 conn(0x7fe7a8154040 legacy=0x7fe7a8063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1866775116 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8062350 con 0x7fe7a8063e30 2023-05-16T07:19:33.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 0x7fe7a8062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:33.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a67fc700 1 --2- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a8063e30 0x7fe7a8069ac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe798009760 tx=0x7fe798009060).ready entity=mon.1 client_cookie=f7daddfe24e13771 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:33.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a5ffb700 1 -- 172.21.15.92:0/1866775116 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800f510 con 0x7fe7a8063e30 2023-05-16T07:19:33.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a5ffb700 1 -- 172.21.15.92:0/1866775116 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79800f670 con 0x7fe7a8063e30 2023-05-16T07:19:33.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7a5ffb700 1 -- 172.21.15.92:0/1866775116 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800f9f0 con 0x7fe7a8063e30 2023-05-16T07:19:33.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7ad94b700 1 -- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a8063e30 msgr2=0x7fe7a8069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.704+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a8063e30 0x7fe7a8069ac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe798009760 tx=0x7fe798009060).stop 2023-05-16T07:19:33.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.705+0000 7fe7ad94b700 1 -- 172.21.15.92:0/1866775116 shutdown_connections 2023-05-16T07:19:33.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.705+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a80bf4f0 0x7fe7a8062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.705+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/1866775116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a8063e30 0x7fe7a8069ac0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.705+0000 7fe7ad94b700 1 -- 172.21.15.92:0/1866775116 >> 172.21.15.92:0/1866775116 conn(0x7fe7a80c8aa0 msgr2=0x7fe7a80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:33.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.705+0000 7fe7ad94b700 1 -- 172.21.15.92:0/1866775116 shutdown_connections 2023-05-16T07:19:33.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.705+0000 7fe7ad94b700 1 -- 172.21.15.92:0/1866775116 wait complete. 2023-05-16T07:19:33.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.706+0000 7fe7ad94b700 1 Processor -- start 2023-05-16T07:19:33.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.706+0000 7fe7ad94b700 1 -- start start 2023-05-16T07:19:33.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a8063e30 0x7fe7a814f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:33.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a80bf4f0 0x7fe7a80aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:33.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 0x7fe7a80c09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:33.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80ad0c0 con 0x7fe7a8154040 2023-05-16T07:19:33.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80ad330 con 0x7fe7a80bf4f0 2023-05-16T07:19:33.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80ac280 con 0x7fe7a8063e30 2023-05-16T07:19:33.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 0x7fe7a80c09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:33.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 0x7fe7a80c09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49430/0 (socket says 172.21.15.92:49430) 2023-05-16T07:19:33.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 -- 172.21.15.92:0/933720105 learned_addr learned my addr 172.21.15.92:0/933720105 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:33.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a67fc700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a80bf4f0 0x7fe7a80aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:33.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a8063e30 0x7fe7a814f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:33.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 -- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a8063e30 msgr2=0x7fe7a814f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a8063e30 0x7fe7a814f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 -- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a80bf4f0 msgr2=0x7fe7a80aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a80bf4f0 0x7fe7a80aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 -- 172.21.15.92:0/933720105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0009ae0 con 0x7fe7a8154040 2023-05-16T07:19:33.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 0x7fe7a80c09b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fe7a0009f10 tx=0x7fe7a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:33.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000c960 con 0x7fe7a8154040 2023-05-16T07:19:33.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7980092a0 con 0x7fe7a8154040 2023-05-16T07:19:33.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.707+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80ac840 con 0x7fe7a8154040 2023-05-16T07:19:33.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.708+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a000f040 con 0x7fe7a8154040 2023-05-16T07:19:33.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.708+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000ccd0 con 0x7fe7a8154040 2023-05-16T07:19:33.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.708+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe788005120 con 0x7fe7a8154040 2023-05-16T07:19:33.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.709+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fe7a001b030 con 0x7fe7a8154040 2023-05-16T07:19:33.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.709+0000 7fe78effd700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 0x7fe7940612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:33.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.709+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe7a0079b60 con 0x7fe7a8154040 2023-05-16T07:19:33.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.709+0000 7fe7a6ffd700 1 -- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 msgr2=0x7fe7940612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:33.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.710+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 0x7fe7940612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:33.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.714+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe7a004add0 con 0x7fe7a8154040 2023-05-16T07:19:33.868 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:33.864+0000 7fa92f2e3680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:33.885 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:33.881+0000 7f2fb3812680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:33.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.911+0000 7fe7a6ffd700 1 -- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 msgr2=0x7fe7940612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:33.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.911+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 0x7fe7940612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.928+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe788005d70 con 0x7fe7a8154040 2023-05-16T07:19:33.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.928+0000 7fe78effd700 1 -- 172.21.15.92:0/933720105 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fe7a004e3f0 con 0x7fe7a8154040 2023-05-16T07:19:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:33.932 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:19.134223+0000","modified":"2023-05-16T07:19:21.140319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7461":{"gid":7461,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1482305851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1482305851},{"type":"v1","addr":"172.21.15.92:6835","nonce":1482305851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-05-16T07:19:33.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 msgr2=0x7fe7940612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:33.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 0x7fe7940612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:33.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 msgr2=0x7fe7a80c09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 0x7fe7a80c09b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fe7a0009f10 tx=0x7fe7a000b040).stop 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 shutdown_connections 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fe79405ede0 0x7fe7940612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7a8063e30 0x7fe7a814f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8154040 0x7fe7a80c09b0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 --2- 172.21.15.92:0/933720105 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7a80bf4f0 0x7fe7a80aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.930+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 >> 172.21.15.92:0/933720105 conn(0x7fe7a80c8aa0 msgr2=0x7fe7a80c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:33.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.931+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 shutdown_connections 2023-05-16T07:19:33.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:33.931+0000 7fe7ad94b700 1 -- 172.21.15.92:0/933720105 wait complete. 2023-05-16T07:19:33.936 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-05-16T07:19:33.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.097+0000 7ff6ba3bb700 1 Processor -- start 2023-05-16T07:19:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.097+0000 7ff6ba3bb700 1 -- start start 2023-05-16T07:19:34.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6ba3bb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4066420 0x7ff6b40c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6ba3bb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b4066e50 con 0x7ff6b4150610 2023-05-16T07:19:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6ba3bb700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b40676f0 con 0x7ff6b4150230 2023-05-16T07:19:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6ba3bb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40668d0 con 0x7ff6b4066420 2023-05-16T07:19:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6b8bb8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff6b4150610 0x7ff6b40bc3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60230/0 (socket says 172.21.15.92:60230) 2023-05-16T07:19:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6b8bb8700 1 -- 172.21.15.92:0/1734349933 learned_addr learned my addr 172.21.15.92:0/1734349933 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6b9bba700 1 --2- 172.21.15.92:0/1734349933 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4066420 0x7ff6b40c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124050561 0 0) 0x7ff6b4066e50 con 0x7ff6b4150610 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a00033f0 con 0x7ff6b4150610 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366049313 0 0) 0x7ff6b40676f0 con 0x7ff6b4150230 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b4066e50 con 0x7ff6b4150230 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6b9bba700 1 -- 172.21.15.92:0/1734349933 >> v1:172.21.15.158:6789/0 conn(0x7ff6b4150230 legacy=0x7ff6b4065430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383871758 0 0) 0x7ff6a00033f0 con 0x7ff6b4150610 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6b9bba700 1 -- 172.21.15.92:0/1734349933 >> v1:172.21.15.92:6789/0 conn(0x7ff6b4150610 legacy=0x7ff6b40bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.098+0000 7ff6b9bba700 1 -- 172.21.15.92:0/1734349933 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b4065be0 con 0x7ff6b4066420 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6b9bba700 1 --2- 172.21.15.92:0/1734349933 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4066420 0x7ff6b40c1e90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff6ac00afa0 tx=0x7ff6ac00a890).ready entity=mon.2 client_cookie=cd1628051ee5834d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00f070 con 0x7ff6b4066420 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac00a2e0 con 0x7ff6b4066420 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6abfff700 1 -- 172.21.15.92:0/1734349933 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00cbb0 con 0x7ff6b4066420 2023-05-16T07:19:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/1734349933 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4066420 msgr2=0x7ff6b40c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/1734349933 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4066420 0x7ff6b40c1e90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff6ac00afa0 tx=0x7ff6ac00a890).stop 2023-05-16T07:19:34.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/1734349933 shutdown_connections 2023-05-16T07:19:34.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/1734349933 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4066420 0x7ff6b40c1e90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.099+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/1734349933 >> 172.21.15.92:0/1734349933 conn(0x7ff6b4069940 msgr2=0x7ff6b40c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:34.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.100+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/1734349933 shutdown_connections 2023-05-16T07:19:34.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.100+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/1734349933 wait complete. 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.101+0000 7ff6ba3bb700 1 Processor -- start 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.101+0000 7ff6ba3bb700 1 -- start start 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.101+0000 7ff6ba3bb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6b4066420 0x7ff6b4152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.101+0000 7ff6ba3bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 0x7ff6b40c0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.101+0000 7ff6ba3bb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4150610 0x7ff6b40ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6ba3bb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b41532f0 con 0x7ff6b4150230 2023-05-16T07:19:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6ba3bb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40befc0 con 0x7ff6b4066420 2023-05-16T07:19:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6ba3bb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4148290 con 0x7ff6b4150610 2023-05-16T07:19:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 0x7ff6b40c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 0x7ff6b40c0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49488/0 (socket says 172.21.15.92:49488) 2023-05-16T07:19:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 -- 172.21.15.92:0/673853218 learned_addr learned my addr 172.21.15.92:0/673853218 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b93b9700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6b4066420 0x7ff6b4152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b9bba700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4150610 0x7ff6b40ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 -- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4150610 msgr2=0x7ff6b40ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4150610 0x7ff6b40ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 -- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6b4066420 msgr2=0x7ff6b4152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6b4066420 0x7ff6b4152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 -- 172.21.15.92:0/673853218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac00ab30 con 0x7ff6b4150230 2023-05-16T07:19:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6b8bb8700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 0x7ff6b40c0700 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7ff6a4007fd0 tx=0x7ff6a4007a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a400ce20 con 0x7ff6b4150230 2023-05-16T07:19:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a4007170 con 0x7ff6b4150230 2023-05-16T07:19:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4007530 con 0x7ff6b4150230 2023-05-16T07:19:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b40bf210 con 0x7ff6b4150230 2023-05-16T07:19:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.102+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b40bf7d0 con 0x7ff6b4150230 2023-05-16T07:19:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.103+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b4047150 con 0x7ff6b4150230 2023-05-16T07:19:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.104+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7ff6a4004ad0 con 0x7ff6b4150230 2023-05-16T07:19:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.104+0000 7ff6a97fa700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff69805edb0 0x7ff698061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.104+0000 7ff6b93b9700 1 -- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff69805edb0 msgr2=0x7ff698061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.104+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ff6a40798a0 con 0x7ff6b4150230 2023-05-16T07:19:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.104+0000 7ff6b93b9700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff69805edb0 0x7ff698061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.108+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff6a404aa00 con 0x7ff6b4150230 2023-05-16T07:19:34.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.280+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6b405ede0 con 0x7ff6b4150230 2023-05-16T07:19:34.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.281+0000 7ff6a97fa700 1 -- 172.21.15.92:0/673853218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18164 (secure 0 0 0) 0x7ff6a404e020 con 0x7ff6b4150230 2023-05-16T07:19:34.285 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:34.285 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":122,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:30.307193+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:17.674878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:18.097566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/931936336":"2023-05-17T07:19:30.307167+0000","172.21.15.158:6833/86301":"2023-05-17T07:19:30.307167+0000","172.21.15.158:6832/86301":"2023-05-17T07:19:30.307167+0000","172.21.15.158:0/2541040642":"2023-05-17T07:19:30.307167+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff69805edb0 msgr2=0x7ff698061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff69805edb0 0x7ff698061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 msgr2=0x7ff6b40c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 0x7ff6b40c0700 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7ff6a4007fd0 tx=0x7ff6a4007a10).stop 2023-05-16T07:19:34.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 shutdown_connections 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff69805edb0 0x7ff698061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff6b4150610 0x7ff6b40ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6b4066420 0x7ff6b4152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 --2- 172.21.15.92:0/673853218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6b4150230 0x7ff6b40c0700 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 >> 172.21.15.92:0/673853218 conn(0x7ff6b4069940 msgr2=0x7ff6b414b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 shutdown_connections 2023-05-16T07:19:34.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.283+0000 7ff6ba3bb700 1 -- 172.21.15.92:0/673853218 wait complete. 2023-05-16T07:19:34.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:19:17.674878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:19:18.097566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:19:34.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:19:34.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 Processor -- start 2023-05-16T07:19:34.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 -- start start 2023-05-16T07:19:34.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b54153ee0 0x7f9b541542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9b54062cc0 0x7f9b54066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b54155870 con 0x7f9b540bdb70 2023-05-16T07:19:34.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b54067340 con 0x7f9b54153ee0 2023-05-16T07:19:34.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.416+0000 7f9b59b0b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b54155670 con 0x7f9b54062cc0 2023-05-16T07:19:34.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b54153ee0 0x7f9b541542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b54153ee0 0x7f9b541542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49494/0 (socket says 172.21.15.92:49494) 2023-05-16T07:19:34.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 -- 172.21.15.92:0/675771652 learned_addr learned my addr 172.21.15.92:0/675771652 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:34.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b52ffd700 1 --2- 172.21.15.92:0/675771652 >> v2:172.21.15.158:3300/0 conn(0x7f9b54062cc0 0x7f9b54066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 -- 172.21.15.92:0/675771652 >> v2:172.21.15.158:3300/0 conn(0x7f9b54062cc0 msgr2=0x7f9b54066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:34.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 --2- 172.21.15.92:0/675771652 >> v2:172.21.15.158:3300/0 conn(0x7f9b54062cc0 0x7f9b54066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 -- 172.21.15.92:0/675771652 >> v1:172.21.15.158:6789/0 conn(0x7f9b540bdb70 legacy=0x7f9b540bc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:34.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 -- 172.21.15.92:0/675771652 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54063120 con 0x7f9b54153ee0 2023-05-16T07:19:34.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b537fe700 1 --2- 172.21.15.92:0/675771652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b541542c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f9b54064a40 tx=0x7f9b4c0090f0).ready entity=mon.0 client_cookie=78c1319b443cad8f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:34.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b527fc700 1 -- 172.21.15.92:0/675771652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00b5a0 con 0x7f9b54153ee0 2023-05-16T07:19:34.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b527fc700 1 -- 172.21.15.92:0/675771652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c00b700 con 0x7f9b54153ee0 2023-05-16T07:19:34.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.417+0000 7f9b527fc700 1 -- 172.21.15.92:0/675771652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00ba80 con 0x7f9b54153ee0 2023-05-16T07:19:34.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 -- 172.21.15.92:0/675771652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 msgr2=0x7f9b541542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/675771652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b541542c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f9b54064a40 tx=0x7f9b4c0090f0).stop 2023-05-16T07:19:34.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 -- 172.21.15.92:0/675771652 shutdown_connections 2023-05-16T07:19:34.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/675771652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b541542c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/675771652 >> v2:172.21.15.158:3300/0 conn(0x7f9b54062cc0 0x7f9b54066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 -- 172.21.15.92:0/675771652 >> 172.21.15.92:0/675771652 conn(0x7f9b540c8420 msgr2=0x7f9b540c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:34.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 -- 172.21.15.92:0/675771652 shutdown_connections 2023-05-16T07:19:34.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.418+0000 7f9b59b0b700 1 -- 172.21.15.92:0/675771652 wait complete. 2023-05-16T07:19:34.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.419+0000 7f9b59b0b700 1 Processor -- start 2023-05-16T07:19:34.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.419+0000 7f9b59b0b700 1 -- start start 2023-05-16T07:19:34.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b59b0b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b54062cc0 0x7f9b540c1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b59b0b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b540bdb70 0x7f9b5414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b59b0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b540bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b59b0b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b540ad910 con 0x7f9b54153ee0 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b59b0b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b540ab2e0 con 0x7f9b54062cc0 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b59b0b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541676f0 con 0x7f9b540bdb70 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b540bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b540bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49504/0 (socket says 172.21.15.92:49504) 2023-05-16T07:19:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 -- 172.21.15.92:0/657340327 learned_addr learned my addr 172.21.15.92:0/657340327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b537fe700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b54062cc0 0x7f9b540c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b52ffd700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b540bdb70 0x7f9b5414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 -- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b540bdb70 msgr2=0x7f9b5414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b540bdb70 0x7f9b5414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 -- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b54062cc0 msgr2=0x7f9b540c1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b54062cc0 0x7f9b540c1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 -- 172.21.15.92:0/657340327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c009370 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.420+0000 7f9b53fff700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b540bfdc0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f9b4400c490 tx=0x7f9b44007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.421+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b44002ea0 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.421+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b440123f0 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.421+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b44012790 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.421+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b540ab530 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.421+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b540aad10 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.422+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b54167880 con 0x7f9b54153ee0 2023-05-16T07:19:34.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.422+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f9b44010070 con 0x7f9b54153ee0 2023-05-16T07:19:34.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.422+0000 7f9b3b7fe700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 0x7f9b400612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.422+0000 7f9b537fe700 1 -- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 msgr2=0x7f9b400612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:34.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.422+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9b44079360 con 0x7f9b54153ee0 2023-05-16T07:19:34.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.422+0000 7f9b537fe700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 0x7f9b400612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:34.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.426+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9b440478f0 con 0x7f9b54153ee0 2023-05-16T07:19:34.472 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:34.468+0000 7fa92f2e3680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:34.520 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:34.517+0000 7f2fb3812680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:34.543 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:34.539+0000 7fa92f2e3680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:34.591 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:34.588+0000 7f2fb3812680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:34.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.623+0000 7f9b537fe700 1 -- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 msgr2=0x7f9b400612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.623+0000 7f9b537fe700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 0x7f9b400612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:34.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.628+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9b540aaea0 con 0x7f9b54153ee0 2023-05-16T07:19:34.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.628+0000 7f9b3b7fe700 1 -- 172.21.15.92:0/657340327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v122) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9b4404a520 con 0x7f9b54153ee0 2023-05-16T07:19:34.632 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:19:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 msgr2=0x7f9b400612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 0x7f9b400612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 msgr2=0x7f9b540bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b540bfdc0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f9b4400c490 tx=0x7f9b44007de0).stop 2023-05-16T07:19:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 shutdown_connections 2023-05-16T07:19:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4005ede0 0x7f9b400612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b54153ee0 0x7f9b540bfdc0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b54062cc0 0x7f9b540c1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 --2- 172.21.15.92:0/657340327 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b540bdb70 0x7f9b5414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 >> 172.21.15.92:0/657340327 conn(0x7f9b540c8420 msgr2=0x7f9b540c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 shutdown_connections 2023-05-16T07:19:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.630+0000 7f9b59b0b700 1 -- 172.21.15.92:0/657340327 wait complete. 2023-05-16T07:19:34.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:19:34.727 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:34.722+0000 7fa92f2e3680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:34.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.761+0000 7f5525e6d700 1 Processor -- start 2023-05-16T07:19:34.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.761+0000 7f5525e6d700 1 -- start start 2023-05-16T07:19:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.762+0000 7f5525e6d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55201540b0 0x7f55200a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.762+0000 7f5525e6d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5520066e50 0x7f55200672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.762+0000 7f5525e6d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55200677b0 con 0x7f5520153cd0 2023-05-16T07:19:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.762+0000 7f5525e6d700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55200a7370 con 0x7f5520066e50 2023-05-16T07:19:34.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.762+0000 7f5525e6d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200a6760 con 0x7f55201540b0 2023-05-16T07:19:34.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f5524e6b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5520153cd0 0x7f5520064f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60258/0 (socket says 172.21.15.92:60258) 2023-05-16T07:19:34.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f5524e6b700 1 -- 172.21.15.92:0/3914097921 learned_addr learned my addr 172.21.15.92:0/3914097921 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:34.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551ffff700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55201540b0 0x7f55200a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f552566c700 1 --2- 172.21.15.92:0/3914097921 >> v2:172.21.15.158:3300/0 conn(0x7f5520066e50 0x7f55200672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551f7fe700 1 -- 172.21.15.92:0/3914097921 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367548582 0 0) 0x7f55200677b0 con 0x7f5520153cd0 2023-05-16T07:19:34.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551f7fe700 1 -- 172.21.15.92:0/3914097921 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55100033f0 con 0x7f5520153cd0 2023-05-16T07:19:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f552566c700 1 -- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55201540b0 msgr2=0x7f55200a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f552566c700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55201540b0 0x7f55200a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f552566c700 1 -- 172.21.15.92:0/3914097921 >> v1:172.21.15.92:6789/0 conn(0x7f5520153cd0 legacy=0x7f5520064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f552566c700 1 -- 172.21.15.92:0/3914097921 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200a6a40 con 0x7f5520066e50 2023-05-16T07:19:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551ffff700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55201540b0 0x7f55200a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f552566c700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200672c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f5518009e60 tx=0x7f5518009780).ready entity=mon.1 client_cookie=36e5dc56fe32f1c2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:34.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551f7fe700 1 -- 172.21.15.92:0/3914097921 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800e070 con 0x7f5520066e50 2023-05-16T07:19:34.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551f7fe700 1 -- 172.21.15.92:0/3914097921 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551800bab0 con 0x7f5520066e50 2023-05-16T07:19:34.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.763+0000 7f551f7fe700 1 -- 172.21.15.92:0/3914097921 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800be30 con 0x7f5520066e50 2023-05-16T07:19:34.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 -- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 msgr2=0x7f55200672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200672c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f5518009e60 tx=0x7f5518009780).stop 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 -- 172.21.15.92:0/3914097921 shutdown_connections 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200672c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 --2- 172.21.15.92:0/3914097921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55201540b0 0x7f55200a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 -- 172.21.15.92:0/3914097921 >> 172.21.15.92:0/3914097921 conn(0x7f5520068a30 msgr2=0x7f55200c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 -- 172.21.15.92:0/3914097921 shutdown_connections 2023-05-16T07:19:34.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.764+0000 7f5525e6d700 1 -- 172.21.15.92:0/3914097921 wait complete. 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.765+0000 7f5525e6d700 1 Processor -- start 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.765+0000 7f5525e6d700 1 -- start start 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5525e6d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200c3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5525e6d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5520153cd0 0x7f55200abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5525e6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 0x7f55200ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5525e6d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5520151ec0 con 0x7f55201540b0 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5525e6d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200acbd0 con 0x7f5520066e50 2023-05-16T07:19:34.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5525e6d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200aceb0 con 0x7f5520153cd0 2023-05-16T07:19:34.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f552566c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 0x7f55200ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f552566c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 0x7f55200ac400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49520/0 (socket says 172.21.15.92:49520) 2023-05-16T07:19:34.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f552566c700 1 -- 172.21.15.92:0/882352619 learned_addr learned my addr 172.21.15.92:0/882352619 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:34.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f551ffff700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5520153cd0 0x7f55200abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.766+0000 7f5524e6b700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:34.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f552566c700 1 -- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5520153cd0 msgr2=0x7f55200abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f552566c700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5520153cd0 0x7f55200abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f552566c700 1 -- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 msgr2=0x7f55200c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f552566c700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f552566c700 1 -- 172.21.15.92:0/882352619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0092a0 con 0x7f55201540b0 2023-05-16T07:19:34.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f552566c700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 0x7f55200ac400 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f551800b610 tx=0x7f5518009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551801d630 con 0x7f55201540b0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551801d790 con 0x7f55201540b0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5518009a20 con 0x7f55201540b0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551801db30 con 0x7f55201540b0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.767+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55200ad470 con 0x7f55201540b0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.768+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5520047150 con 0x7f55201540b0 2023-05-16T07:19:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.768+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f5518030430 con 0x7f55201540b0 2023-05-16T07:19:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.769+0000 7f551cff9700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f550405ede0 0x7f55040612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.769+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f551807c8e0 con 0x7f55201540b0 2023-05-16T07:19:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.769+0000 7f5524e6b700 1 -- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f550405ede0 msgr2=0x7f55040612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.769+0000 7f5524e6b700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f550405ede0 0x7f55040612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.772+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f551804dac0 con 0x7f55201540b0 2023-05-16T07:19:34.788 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:34.784+0000 7f2fb3812680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.945+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f55200c6980 con 0x7f55201540b0 2023-05-16T07:19:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.946+0000 7f551cff9700 1 -- 172.21.15.92:0/882352619 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v122) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55180510e0 con 0x7f55201540b0 2023-05-16T07:19:34.950 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:19:34.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f550405ede0 msgr2=0x7f55040612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f550405ede0 0x7f55040612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 msgr2=0x7f55200ac400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 0x7f55200ac400 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f551800b610 tx=0x7f5518009780).stop 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 shutdown_connections 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f550405ede0 0x7f55040612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55201540b0 0x7f55200ac400 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5520066e50 0x7f55200c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 --2- 172.21.15.92:0/882352619 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5520153cd0 0x7f55200abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 >> 172.21.15.92:0/882352619 conn(0x7f5520068a30 msgr2=0x7f55200c5630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 shutdown_connections 2023-05-16T07:19:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:34.948+0000 7f5525e6d700 1 -- 172.21.15.92:0/882352619 wait complete. 2023-05-16T07:19:34.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:19:35.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.078+0000 7f7002dd2700 1 Processor -- start 2023-05-16T07:19:35.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.078+0000 7f7002dd2700 1 -- start start 2023-05-16T07:19:35.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.079+0000 7f7002dd2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc0bdac0 0x7f6ffc155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.079+0000 7f7002dd2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc151c70 con 0x7f6ffc154f70 2023-05-16T07:19:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.079+0000 7f7002dd2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc151340 con 0x7f6ffc0699c0 2023-05-16T07:19:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.079+0000 7f7002dd2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc152100 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.079+0000 7f7000b6e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ffc154f70 0x7f6ffc0692b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60306/0 (socket says 172.21.15.92:60306) 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.079+0000 7f7000b6e700 1 -- 172.21.15.92:0/1114080606 learned_addr learned my addr 172.21.15.92:0/1114080606 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f700136f700 1 --2- 172.21.15.92:0/1114080606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc0bdac0 0x7f6ffc155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83461374 0 0) 0x7f6ffc151c70 con 0x7f6ffc154f70 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec0033f0 con 0x7f6ffc154f70 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196024493 0 0) 0x7f6ffc151340 con 0x7f6ffc0699c0 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ffc151c70 con 0x7f6ffc0699c0 2023-05-16T07:19:35.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f700136f700 1 -- 172.21.15.92:0/1114080606 >> v1:172.21.15.158:6789/0 conn(0x7f6ffc0699c0 legacy=0x7f6ffc0bd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f700136f700 1 -- 172.21.15.92:0/1114080606 >> v1:172.21.15.92:6789/0 conn(0x7f6ffc154f70 legacy=0x7f6ffc0692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f700136f700 1 -- 172.21.15.92:0/1114080606 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc151620 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f700136f700 1 --2- 172.21.15.92:0/1114080606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc0bdac0 0x7f6ffc155350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f6ff4009e90 tx=0x7f6ff4009780).ready entity=mon.2 client_cookie=3ef1a27f5800fb38 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400e070 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff400bab0 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.080+0000 7f6ffb7fe700 1 -- 172.21.15.92:0/1114080606 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400be30 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 -- 172.21.15.92:0/1114080606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc0bdac0 msgr2=0x7f6ffc155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1114080606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc0bdac0 0x7f6ffc155350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f6ff4009e90 tx=0x7f6ff4009780).stop 2023-05-16T07:19:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 -- 172.21.15.92:0/1114080606 shutdown_connections 2023-05-16T07:19:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1114080606 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc0bdac0 0x7f6ffc155350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 -- 172.21.15.92:0/1114080606 >> 172.21.15.92:0/1114080606 conn(0x7f6ffc0c8470 msgr2=0x7f6ffc0c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 -- 172.21.15.92:0/1114080606 shutdown_connections 2023-05-16T07:19:35.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.081+0000 7f7002dd2700 1 -- 172.21.15.92:0/1114080606 wait complete. 2023-05-16T07:19:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.082+0000 7f7002dd2700 1 Processor -- start 2023-05-16T07:19:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.082+0000 7f7002dd2700 1 -- start start 2023-05-16T07:19:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f7002dd2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ffc0699c0 0x7f6ffc161b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f7002dd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 0x7f6ffc0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f7002dd2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc154f70 0x7f6ffc0ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f7002dd2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0ada10 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f7002dd2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0bf680 con 0x7f6ffc0699c0 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f7002dd2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc161ff0 con 0x7f6ffc154f70 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 0x7f6ffc0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 0x7f6ffc0acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49566/0 (socket says 172.21.15.92:49566) 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 -- 172.21.15.92:0/1581959356 learned_addr learned my addr 172.21.15.92:0/1581959356 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:35.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 -- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc154f70 msgr2=0x7f6ffc0ad440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-16T07:19:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc154f70 0x7f6ffc0ad440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 -- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ffc0699c0 msgr2=0x7f6ffc161b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-16T07:19:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ffc0699c0 0x7f6ffc161b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.083+0000 7f6ffbfff700 1 -- 172.21.15.92:0/1581959356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4009a20 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.084+0000 7f6ffbfff700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 0x7f6ffc0acf50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f6fe800c450 tx=0x7f6fe8007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:35.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.084+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800f470 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.084+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe800f5d0 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.084+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800f780 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.084+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc0bed50 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.084+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc0bf310 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.085+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc06c350 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.085+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f6fe8010070 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.086+0000 7f6ff8ff9700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fe005ede0 0x7f6fe00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.086+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f6fe80792c0 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.086+0000 7f7000b6e700 1 -- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fe005ede0 msgr2=0x7f6fe00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.086+0000 7f7000b6e700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fe005ede0 0x7f6fe00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:35.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.090+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6fe80478f0 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.142 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.138+0000 7fa92f2e3680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:35.223 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:35.219+0000 7f2fb3812680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:35.246 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.242+0000 7fa92f2e3680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.262+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ffc0c8880 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.263+0000 7f6ff8ff9700 1 -- 172.21.15.92:0/1581959356 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fe804a370 con 0x7f6ffc0bdac0 2023-05-16T07:19:35.267 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:19:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fe005ede0 msgr2=0x7f6fe00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fe005ede0 0x7f6fe00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 msgr2=0x7f6ffc0acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 0x7f6ffc0acf50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f6fe800c450 tx=0x7f6fe8007be0).stop 2023-05-16T07:19:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 shutdown_connections 2023-05-16T07:19:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f6fe005ede0 0x7f6fe00612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6ffc154f70 0x7f6ffc0ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6ffc0699c0 0x7f6ffc161b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 --2- 172.21.15.92:0/1581959356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ffc0bdac0 0x7f6ffc0acf50 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 >> 172.21.15.92:0/1581959356 conn(0x7f6ffc0c8470 msgr2=0x7f6ffc14cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 shutdown_connections 2023-05-16T07:19:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.265+0000 7f7002dd2700 1 -- 172.21.15.92:0/1581959356 wait complete. 2023-05-16T07:19:35.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:19:35.331 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:35.328+0000 7f2fb3812680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:35.399 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.394+0000 7fa92f2e3680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:35.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.395+0000 7f3d6f100700 1 Processor -- start 2023-05-16T07:19:35.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.395+0000 7f3d6f100700 1 -- start start 2023-05-16T07:19:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6f100700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3d68153f90 0x7f3d680a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6f100700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d680c5d00 0x7f3d680c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6f100700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d680c66a0 con 0x7f3d680c0150 2023-05-16T07:19:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6f100700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d68069f60 con 0x7f3d680c5d00 2023-05-16T07:19:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6f100700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6806a9e0 con 0x7f3d68153f90 2023-05-16T07:19:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6d8fd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d680c5d00 0x7f3d680c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6d8fd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d680c5d00 0x7f3d680c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49574/0 (socket says 172.21.15.92:49574) 2023-05-16T07:19:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6d8fd700 1 -- 172.21.15.92:0/2641438072 learned_addr learned my addr 172.21.15.92:0/2641438072 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6e0fe700 1 --2- 172.21.15.92:0/2641438072 >> v2:172.21.15.158:3300/0 conn(0x7f3d68153f90 0x7f3d680a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6e0fe700 1 -- 172.21.15.92:0/2641438072 >> v2:172.21.15.92:3300/0 conn(0x7f3d680c5d00 msgr2=0x7f3d680c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:35.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6e0fe700 1 --2- 172.21.15.92:0/2641438072 >> v2:172.21.15.92:3300/0 conn(0x7f3d680c5d00 0x7f3d680c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6e0fe700 1 -- 172.21.15.92:0/2641438072 >> v1:172.21.15.158:6789/0 conn(0x7f3d680c0150 legacy=0x7f3d680c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.396+0000 7f3d6e0fe700 1 -- 172.21.15.92:0/2641438072 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6806a140 con 0x7f3d68153f90 2023-05-16T07:19:35.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.397+0000 7f3d6e0fe700 1 --2- 172.21.15.92:0/2641438072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d68153f90 0x7f3d680a9d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f3d680caa40 tx=0x7f3d64009060).ready entity=mon.1 client_cookie=f655b63c910b71cd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.398+0000 7f3d6d0fc700 1 -- 172.21.15.92:0/2641438072 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400f510 con 0x7f3d68153f90 2023-05-16T07:19:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.398+0000 7f3d6d0fc700 1 -- 172.21.15.92:0/2641438072 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400f670 con 0x7f3d68153f90 2023-05-16T07:19:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.398+0000 7f3d6d0fc700 1 -- 172.21.15.92:0/2641438072 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400f9f0 con 0x7f3d68153f90 2023-05-16T07:19:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 -- 172.21.15.92:0/2641438072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d68153f90 msgr2=0x7f3d680a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 --2- 172.21.15.92:0/2641438072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d68153f90 0x7f3d680a9d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f3d680caa40 tx=0x7f3d64009060).stop 2023-05-16T07:19:35.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 -- 172.21.15.92:0/2641438072 shutdown_connections 2023-05-16T07:19:35.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 --2- 172.21.15.92:0/2641438072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d68153f90 0x7f3d680a9d20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 --2- 172.21.15.92:0/2641438072 >> v2:172.21.15.92:3300/0 conn(0x7f3d680c5d00 0x7f3d680c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 -- 172.21.15.92:0/2641438072 >> 172.21.15.92:0/2641438072 conn(0x7f3d680cb600 msgr2=0x7f3d680c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:35.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 -- 172.21.15.92:0/2641438072 shutdown_connections 2023-05-16T07:19:35.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.399+0000 7f3d6f100700 1 -- 172.21.15.92:0/2641438072 wait complete. 2023-05-16T07:19:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.400+0000 7f3d6f100700 1 Processor -- start 2023-05-16T07:19:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.400+0000 7f3d6f100700 1 -- start start 2023-05-16T07:19:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6f100700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d680c0150 0x7f3d68150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6f100700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d680c5d00 0x7f3d680adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6f100700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 0x7f3d68155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6f100700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d68165230 con 0x7f3d68153f90 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6f100700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d68154b60 con 0x7f3d680c5d00 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6f100700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d68164f50 con 0x7f3d680c0150 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6e8ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 0x7f3d68155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6e8ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 0x7f3d68155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49582/0 (socket says 172.21.15.92:49582) 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6e8ff700 1 -- 172.21.15.92:0/4167356812 learned_addr learned my addr 172.21.15.92:0/4167356812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:35.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6e0fe700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d680c0150 0x7f3d68150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.401+0000 7f3d6d8fd700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d680c5d00 0x7f3d680adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6e8ff700 1 -- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d680c0150 msgr2=0x7f3d68150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6e8ff700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d680c0150 0x7f3d68150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6e8ff700 1 -- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d680c5d00 msgr2=0x7f3d680adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6e8ff700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d680c5d00 0x7f3d680adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6e8ff700 1 -- 172.21.15.92:0/4167356812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640092a0 con 0x7f3d68153f90 2023-05-16T07:19:35.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6e8ff700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 0x7f3d68155cf0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f3d60002230 tx=0x7f3d600015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000e4a0 con 0x7f3d68153f90 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6000e600 con 0x7f3d68153f90 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d68154510 con 0x7f3d68153f90 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000e980 con 0x7f3d68153f90 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.402+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d68150fb0 con 0x7f3d68153f90 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.403+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d68154370 con 0x7f3d68153f90 2023-05-16T07:19:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.404+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f3d60019030 con 0x7f3d68153f90 2023-05-16T07:19:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.404+0000 7f3d5e7fc700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d4c05ede0 0x7f3d4c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.404+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f3d600795e0 con 0x7f3d68153f90 2023-05-16T07:19:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.404+0000 7f3d6e0fe700 1 -- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d4c05ede0 msgr2=0x7f3d4c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.404+0000 7f3d6e0fe700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d4c05ede0 0x7f3d4c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.408+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3d6004a8d0 con 0x7f3d68153f90 2023-05-16T07:19:35.472 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.468+0000 7fa92f2e3680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:35.489 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:35.486+0000 7f2fb3812680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:35.537 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.533+0000 7fa92f2e3680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:35.564 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:35.560+0000 7f2fb3812680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:35.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.578+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d680c66a0 con 0x7f3d68153f90 2023-05-16T07:19:35.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.578+0000 7f3d5e7fc700 1 -- 172.21.15.92:0/4167356812 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d680c66a0 con 0x7f3d68153f90 2023-05-16T07:19:35.583 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:19:35.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d4c05ede0 msgr2=0x7f3d4c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:35.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d4c05ede0 0x7f3d4c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 msgr2=0x7f3d68155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 0x7f3d68155cf0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f3d60002230 tx=0x7f3d600015e0).stop 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 shutdown_connections 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3d4c05ede0 0x7f3d4c0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3d680c0150 0x7f3d68150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d68153f90 0x7f3d68155cf0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 --2- 172.21.15.92:0/4167356812 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d680c5d00 0x7f3d680adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 >> 172.21.15.92:0/4167356812 conn(0x7f3d680cb600 msgr2=0x7f3d680c0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:35.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 shutdown_connections 2023-05-16T07:19:35.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.580+0000 7f3d6f100700 1 -- 172.21.15.92:0/4167356812 wait complete. 2023-05-16T07:19:35.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:35.635 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:35.632+0000 7f2fb3812680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.707+0000 7f3bf2f28700 1 Processor -- start 2023-05-16T07:19:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.707+0000 7f3bf2f28700 1 -- start start 2023-05-16T07:19:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf2f28700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e5e0 0x7f3bec157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf2f28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bec157f40 0x7f3bec15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf2f28700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0683d0 con 0x7f3bec14e200 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf2f28700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec137470 con 0x7f3bec157f40 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf2f28700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0c3520 con 0x7f3bec14e5e0 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf14c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bec157f40 0x7f3bec15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf0cc4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3bec14e200 0x7f3bec066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60334/0 (socket says 172.21.15.92:60334) 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf0cc4700 1 -- 172.21.15.92:0/655388790 learned_addr learned my addr 172.21.15.92:0/655388790 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bebfff700 1 --2- 172.21.15.92:0/655388790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e5e0 0x7f3bec157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3beb7fe700 1 -- 172.21.15.92:0/655388790 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057504700 0 0) 0x7f3bec0683d0 con 0x7f3bec14e200 2023-05-16T07:19:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3beb7fe700 1 -- 172.21.15.92:0/655388790 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc0033f0 con 0x7f3bec14e200 2023-05-16T07:19:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf14c5700 1 -- 172.21.15.92:0/655388790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e5e0 msgr2=0x7f3bec157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf14c5700 1 --2- 172.21.15.92:0/655388790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e5e0 0x7f3bec157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf14c5700 1 -- 172.21.15.92:0/655388790 >> v1:172.21.15.92:6789/0 conn(0x7f3bec14e200 legacy=0x7f3bec066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf14c5700 1 -- 172.21.15.92:0/655388790 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0be060 con 0x7f3bec157f40 2023-05-16T07:19:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.708+0000 7f3bf14c5700 1 --2- 172.21.15.92:0/655388790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec157f40 0x7f3bec15ddf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f3be4009e90 tx=0x7f3be4009780).ready entity=mon.0 client_cookie=82824773e43b134c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3beb7fe700 1 -- 172.21.15.92:0/655388790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400e070 con 0x7f3bec157f40 2023-05-16T07:19:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3beb7fe700 1 -- 172.21.15.92:0/655388790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400ba70 con 0x7f3bec157f40 2023-05-16T07:19:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3beb7fe700 1 -- 172.21.15.92:0/655388790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400bdf0 con 0x7f3bec157f40 2023-05-16T07:19:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 -- 172.21.15.92:0/655388790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec157f40 msgr2=0x7f3bec15ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/655388790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec157f40 0x7f3bec15ddf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f3be4009e90 tx=0x7f3be4009780).stop 2023-05-16T07:19:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 -- 172.21.15.92:0/655388790 shutdown_connections 2023-05-16T07:19:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/655388790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec157f40 0x7f3bec15ddf0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/655388790 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e5e0 0x7f3bec157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 -- 172.21.15.92:0/655388790 >> 172.21.15.92:0/655388790 conn(0x7f3bec0c7bc0 msgr2=0x7f3bec0c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 -- 172.21.15.92:0/655388790 shutdown_connections 2023-05-16T07:19:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.709+0000 7f3bf2f28700 1 -- 172.21.15.92:0/655388790 wait complete. 2023-05-16T07:19:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.710+0000 7f3bf2f28700 1 Processor -- start 2023-05-16T07:19:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 -- start start 2023-05-16T07:19:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e200 0x7f3bec135c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 0x7f3bec0c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bec157f40 0x7f3bec159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec136160 con 0x7f3bec14e5e0 2023-05-16T07:19:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0c6d90 con 0x7f3bec157f40 2023-05-16T07:19:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf2f28700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1555b0 con 0x7f3bec14e200 2023-05-16T07:19:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 0x7f3bec0c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 0x7f3bec0c68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49592/0 (socket says 172.21.15.92:49592) 2023-05-16T07:19:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bebfff700 1 -- 172.21.15.92:0/840003703 learned_addr learned my addr 172.21.15.92:0/840003703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf0cc4700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e200 0x7f3bec135c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.711+0000 7f3bf14c5700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bec157f40 0x7f3bec159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bebfff700 1 -- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e200 msgr2=0x7f3bec135c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bebfff700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e200 0x7f3bec135c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bebfff700 1 -- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bec157f40 msgr2=0x7f3bec159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bebfff700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bec157f40 0x7f3bec159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bebfff700 1 -- 172.21.15.92:0/840003703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4009a20 con 0x7f3bec14e5e0 2023-05-16T07:19:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bf14c5700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bec157f40 0x7f3bec159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bebfff700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 0x7f3bec0c68a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f3bd800e960 tx=0x7f3bd8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:35.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8012600 con 0x7f3bec14e5e0 2023-05-16T07:19:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8012760 con 0x7f3bec14e5e0 2023-05-16T07:19:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec155e50 con 0x7f3bec14e5e0 2023-05-16T07:19:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8012ae0 con 0x7f3bec14e5e0 2023-05-16T07:19:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.712+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec155800 con 0x7f3bec14e5e0 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.713+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec06c350 con 0x7f3bec14e5e0 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.713+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f3bd8022070 con 0x7f3bec14e5e0 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.714+0000 7f3be8ff9700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 0x7f3bd00612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.714+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f3bd807a2c0 con 0x7f3bec14e5e0 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.714+0000 7f3bf0cc4700 1 -- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 msgr2=0x7f3bd00612d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.714+0000 7f3bf0cc4700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 0x7f3bd00612d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.718+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3bd804b480 con 0x7f3bec14e5e0 2023-05-16T07:19:35.789 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.785+0000 7fa92f2e3680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:35.898 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:35.895+0000 7f2fb3812680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.915+0000 7f3bf0cc4700 1 -- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 msgr2=0x7f3bd00612d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.915+0000 7f3bf0cc4700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 0x7f3bd00612d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:35.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.927+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bec0c43e0 con 0x7f3bec14e5e0 2023-05-16T07:19:35.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.927+0000 7f3be8ff9700 1 -- 172.21.15.92:0/840003703 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f3bd804eaa0 con 0x7f3bec14e5e0 2023-05-16T07:19:35.931 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:35.931 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:19.134223+0000","modified":"2023-05-16T07:19:21.140319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7461":{"gid":7461,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1482305851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1482305851},{"type":"v1","addr":"172.21.15.92:6835","nonce":1482305851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-05-16T07:19:35.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 msgr2=0x7f3bd00612d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 0x7f3bd00612d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 msgr2=0x7f3bec0c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 0x7f3bec0c68a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f3bd800e960 tx=0x7f3bd8009060).stop 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 shutdown_connections 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f3bd005ee10 0x7f3bd00612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3bec14e200 0x7f3bec135c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3bec157f40 0x7f3bec159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 --2- 172.21.15.92:0/840003703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bec14e5e0 0x7f3bec0c68a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 >> 172.21.15.92:0/840003703 conn(0x7f3bec0c7bc0 msgr2=0x7f3bec1362a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 shutdown_connections 2023-05-16T07:19:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:35.930+0000 7f3bf2f28700 1 -- 172.21.15.92:0/840003703 wait complete. 2023-05-16T07:19:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-05-16T07:19:35.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:35.970 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:35.966+0000 7fa92f2e3680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:36.086 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:36.083+0000 7f2fb3812680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:36.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.099+0000 7fc6ab33f700 1 Processor -- start 2023-05-16T07:19:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.099+0000 7fc6ab33f700 1 -- start start 2023-05-16T07:19:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6ab33f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc6a40c76e0 0x7fc6a40c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6ab33f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6a40a9b70 0x7fc6a40aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6ab33f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6ab33f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a4158900 con 0x7fc6a40a9b70 2023-05-16T07:19:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6ab33f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a4157cf0 con 0x7fc6a40c76e0 2023-05-16T07:19:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6ab33f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4157fd0 con 0x7fc6a4153820 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6a40a9b70 0x7fc6a40aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6a40a9b70 0x7fc6a40aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49602/0 (socket says 172.21.15.92:49602) 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 -- 172.21.15.92:0/3280700087 learned_addr learned my addr 172.21.15.92:0/3280700087 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6aab3e700 1 --2- 172.21.15.92:0/3280700087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6aa33d700 1 --2- 172.21.15.92:0/3280700087 >> v2:172.21.15.158:3300/0 conn(0x7fc6a40c76e0 0x7fc6a40c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 -- 172.21.15.92:0/3280700087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 msgr2=0x7fc6a4153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 --2- 172.21.15.92:0/3280700087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a4153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 -- 172.21.15.92:0/3280700087 >> v2:172.21.15.158:3300/0 conn(0x7fc6a40c76e0 msgr2=0x7fc6a40c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 --2- 172.21.15.92:0/3280700087 >> v2:172.21.15.158:3300/0 conn(0x7fc6a40c76e0 0x7fc6a40c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 -- 172.21.15.92:0/3280700087 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a41582b0 con 0x7fc6a40a9b70 2023-05-16T07:19:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6aa33d700 1 --2- 172.21.15.92:0/3280700087 >> v2:172.21.15.158:3300/0 conn(0x7fc6a40c76e0 0x7fc6a40c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a9b3c700 1 --2- 172.21.15.92:0/3280700087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40aa000 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fc6a40841a0 tx=0x7fc6940090f0).ready entity=mon.0 client_cookie=743704c43f59adc1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:36.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a933b700 1 -- 172.21.15.92:0/3280700087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400b600 con 0x7fc6a40a9b70 2023-05-16T07:19:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a933b700 1 -- 172.21.15.92:0/3280700087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400b760 con 0x7fc6a40a9b70 2023-05-16T07:19:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.100+0000 7fc6a933b700 1 -- 172.21.15.92:0/3280700087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400bae0 con 0x7fc6a40a9b70 2023-05-16T07:19:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3280700087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 msgr2=0x7fc6a40aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3280700087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40aa000 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fc6a40841a0 tx=0x7fc6940090f0).stop 2023-05-16T07:19:36.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3280700087 shutdown_connections 2023-05-16T07:19:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3280700087 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3280700087 >> v2:172.21.15.158:3300/0 conn(0x7fc6a40c76e0 0x7fc6a40c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3280700087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40aa000 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3280700087 >> 172.21.15.92:0/3280700087 conn(0x7fc6a4068330 msgr2=0x7fc6a4068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3280700087 shutdown_connections 2023-05-16T07:19:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.101+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3280700087 wait complete. 2023-05-16T07:19:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.102+0000 7fc6ab33f700 1 Processor -- start 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.102+0000 7fc6ab33f700 1 -- start start 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6ab33f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6ab33f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6a40c76e0 0x7fc6a4153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6ab33f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6ab33f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40adf90 con 0x7fc6a40a9b70 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6ab33f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a41544a0 con 0x7fc6a40c76e0 2023-05-16T07:19:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6ab33f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4156690 con 0x7fc6a4153820 2023-05-16T07:19:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49606/0 (socket says 172.21.15.92:49606) 2023-05-16T07:19:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 -- 172.21.15.92:0/3665443889 learned_addr learned my addr 172.21.15.92:0/3665443889 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6a9b3c700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6a40c76e0 0x7fc6a4153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aab3e700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 -- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 msgr2=0x7fc6a414c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a414c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 -- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6a40c76e0 msgr2=0x7fc6a4153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6a40c76e0 0x7fc6a4153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 -- 172.21.15.92:0/3665443889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc694009370 con 0x7fc6a40a9b70 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.103+0000 7fc6aa33d700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40adaa0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fc6a000d890 tx=0x7fc6a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.104+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011590 con 0x7fc6a40a9b70 2023-05-16T07:19:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.104+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a00116f0 con 0x7fc6a40a9b70 2023-05-16T07:19:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.104+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a40aea40 con 0x7fc6a40a9b70 2023-05-16T07:19:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.104+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011a70 con 0x7fc6a40a9b70 2023-05-16T07:19:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.104+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a40ae480 con 0x7fc6a40a9b70 2023-05-16T07:19:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.105+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a4156c30 con 0x7fc6a40a9b70 2023-05-16T07:19:36.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.105+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fc6a0012420 con 0x7fc6a40a9b70 2023-05-16T07:19:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.105+0000 7fc69a7fc700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc68805ede0 0x7fc6880612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.105+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc6a0079b00 con 0x7fc6a40a9b70 2023-05-16T07:19:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.105+0000 7fc6a9b3c700 1 -- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc68805ede0 msgr2=0x7fc6880612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.105+0000 7fc6a9b3c700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc68805ede0 0x7fc6880612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.109+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc6a004ac60 con 0x7fc6a40a9b70 2023-05-16T07:19:36.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.290+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6a4047150 con 0x7fc6a40a9b70 2023-05-16T07:19:36.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.291+0000 7fc69a7fc700 1 -- 172.21.15.92:0/3665443889 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18164 (secure 0 0 0) 0x7fc6a004e280 con 0x7fc6a40a9b70 2023-05-16T07:19:36.295 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:36.295 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":122,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:30.307193+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:17.674878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:18.097566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/931936336":"2023-05-17T07:19:30.307167+0000","172.21.15.158:6833/86301":"2023-05-17T07:19:30.307167+0000","172.21.15.158:6832/86301":"2023-05-17T07:19:30.307167+0000","172.21.15.158:0/2541040642":"2023-05-17T07:19:30.307167+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc68805ede0 msgr2=0x7fc6880612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc68805ede0 0x7fc6880612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 msgr2=0x7fc6a40adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40adaa0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fc6a000d890 tx=0x7fc6a0008040).stop 2023-05-16T07:19:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 shutdown_connections 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc68805ede0 0x7fc6880612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6a4153820 0x7fc6a414c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a40a9b70 0x7fc6a40adaa0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 --2- 172.21.15.92:0/3665443889 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6a40c76e0 0x7fc6a4153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 >> 172.21.15.92:0/3665443889 conn(0x7fc6a4068330 msgr2=0x7fc6a40c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 shutdown_connections 2023-05-16T07:19:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.293+0000 7fc6ab33f700 1 -- 172.21.15.92:0/3665443889 wait complete. 2023-05-16T07:19:36.309 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:19:36.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.419+0000 7f9133618700 1 Processor -- start 2023-05-16T07:19:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.420+0000 7f9133618700 1 -- start start 2023-05-16T07:19:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9133618700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 0x7f912c0663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9133618700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f912c0bc760 0x7f912c155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9133618700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0a6ce0 con 0x7f912c151de0 2023-05-16T07:19:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9133618700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c152260 con 0x7f912c0bc760 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9133618700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0c4b20 con 0x7f912c151a00 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 0x7f912c0663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9131bb5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f912c0bc760 0x7f912c155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 0x7f912c0663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:42200/0 (socket says 172.21.15.92:42200) 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 -- 172.21.15.92:0/2098756046 learned_addr learned my addr 172.21.15.92:0/2098756046 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 -- 172.21.15.92:0/2098756046 >> v2:172.21.15.158:3300/0 conn(0x7f912c0bc760 msgr2=0x7f912c155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 --2- 172.21.15.92:0/2098756046 >> v2:172.21.15.158:3300/0 conn(0x7f912c0bc760 0x7f912c155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 -- 172.21.15.92:0/2098756046 >> v1:172.21.15.158:6789/0 conn(0x7f912c151de0 legacy=0x7f912c0bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 -- 172.21.15.92:0/2098756046 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c066a10 con 0x7f912c151a00 2023-05-16T07:19:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f91313b4700 1 --2- 172.21.15.92:0/2098756046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 0x7f912c0663a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f911c006fd0 tx=0x7f911c0090f0).ready entity=mon.2 client_cookie=931071ce6cfff396 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9123fff700 1 -- 172.21.15.92:0/2098756046 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00b580 con 0x7f912c151a00 2023-05-16T07:19:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9123fff700 1 -- 172.21.15.92:0/2098756046 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c00b6e0 con 0x7f912c151a00 2023-05-16T07:19:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.421+0000 7f9123fff700 1 -- 172.21.15.92:0/2098756046 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00ba60 con 0x7f912c151a00 2023-05-16T07:19:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 -- 172.21.15.92:0/2098756046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 msgr2=0x7f912c0663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 --2- 172.21.15.92:0/2098756046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 0x7f912c0663a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f911c006fd0 tx=0x7f911c0090f0).stop 2023-05-16T07:19:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 -- 172.21.15.92:0/2098756046 shutdown_connections 2023-05-16T07:19:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 --2- 172.21.15.92:0/2098756046 >> v2:172.21.15.158:3300/0 conn(0x7f912c0bc760 0x7f912c155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 --2- 172.21.15.92:0/2098756046 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c151a00 0x7f912c0663a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 -- 172.21.15.92:0/2098756046 >> 172.21.15.92:0/2098756046 conn(0x7f912c0c7bc0 msgr2=0x7f912c0c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 -- 172.21.15.92:0/2098756046 shutdown_connections 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.422+0000 7f9133618700 1 -- 172.21.15.92:0/2098756046 wait complete. 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.423+0000 7f9133618700 1 Processor -- start 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.423+0000 7f9133618700 1 -- start start 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9133618700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c0bc760 0x7f912c0aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9133618700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 0x7f912c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9133618700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c151de0 0x7f912c150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9133618700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c157460 con 0x7f912c151a00 2023-05-16T07:19:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9133618700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c157650 con 0x7f912c151de0 2023-05-16T07:19:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9133618700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c150620 con 0x7f912c0bc760 2023-05-16T07:19:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 0x7f912c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 0x7f912c156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49636/0 (socket says 172.21.15.92:49636) 2023-05-16T07:19:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 -- 172.21.15.92:0/2222528398 learned_addr learned my addr 172.21.15.92:0/2222528398 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:36.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f91313b4700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c0bc760 0x7f912c0aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9131bb5700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c151de0 0x7f912c150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 -- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c0bc760 msgr2=0x7f912c0aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c0bc760 0x7f912c0aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 -- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c151de0 msgr2=0x7f912c150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c151de0 0x7f912c150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.424+0000 7f9130bb3700 1 -- 172.21.15.92:0/2222528398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c009370 con 0x7f912c151a00 2023-05-16T07:19:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f91313b4700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c0bc760 0x7f912c0aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f9130bb3700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 0x7f912c156f70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f912800c470 tx=0x7f9128007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128007020 con 0x7f912c151a00 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9128007180 con 0x7f912c151a00 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128007520 con 0x7f912c151a00 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c0aab30 con 0x7f912c151a00 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.425+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c0aa4e0 con 0x7f912c151a00 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.426+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c06c350 con 0x7f912c151a00 2023-05-16T07:19:36.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.426+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f9128010070 con 0x7f912c151a00 2023-05-16T07:19:36.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.427+0000 7f91217fa700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 0x7f9110061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.427+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9128079830 con 0x7f912c151a00 2023-05-16T07:19:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.427+0000 7f91313b4700 1 -- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 msgr2=0x7f9110061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.427+0000 7f91313b4700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 0x7f9110061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:36.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.431+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f912804a910 con 0x7f912c151a00 2023-05-16T07:19:36.609 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:36.605+0000 7fa92f2e3680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.628+0000 7f91313b4700 1 -- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 msgr2=0x7f9110061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.628+0000 7f91313b4700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 0x7f9110061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:36.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.640+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f912c0c4560 con 0x7f912c151a00 2023-05-16T07:19:36.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.641+0000 7f91217fa700 1 -- 172.21.15.92:0/2222528398 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f912804df30 con 0x7f912c151a00 2023-05-16T07:19:36.645 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:36.645 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:19.134223+0000","modified":"2023-05-16T07:19:21.140319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7461":{"gid":7461,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1482305851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1482305851},{"type":"v1","addr":"172.21.15.92:6835","nonce":1482305851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-05-16T07:19:36.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 msgr2=0x7f9110061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:36.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 0x7f9110061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 msgr2=0x7f912c156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 0x7f912c156f70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f912800c470 tx=0x7f9128007dd0).stop 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 shutdown_connections 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f911005edb0 0x7f9110061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c0bc760 0x7f912c0aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c151de0 0x7f912c150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 --2- 172.21.15.92:0/2222528398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c151a00 0x7f912c156f70 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 >> 172.21.15.92:0/2222528398 conn(0x7f912c0c7bc0 msgr2=0x7f912c0a7380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 shutdown_connections 2023-05-16T07:19:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.643+0000 7f9133618700 1 -- 172.21.15.92:0/2222528398 wait complete. 2023-05-16T07:19:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-05-16T07:19:36.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:36.675 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:36.671+0000 7fa92f2e3680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:36.748 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:36.744+0000 7fa92f2e3680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:36.760 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:36.756+0000 7f2fb3812680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.772+0000 7f12c20da700 1 Processor -- start 2023-05-16T07:19:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.772+0000 7f12c20da700 1 -- start start 2023-05-16T07:19:36.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c20da700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f12bc154d50 0x7f12bc0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c20da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12bc067560 0x7f12bc0a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c20da700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12bc068750 con 0x7f12bc155130 2023-05-16T07:19:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c20da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12bc0a5260 con 0x7f12bc067560 2023-05-16T07:19:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c20da700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12bc067b40 con 0x7f12bc154d50 2023-05-16T07:19:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12bc067560 0x7f12bc0a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12bc067560 0x7f12bc0a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49648/0 (socket says 172.21.15.92:49648) 2023-05-16T07:19:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 -- 172.21.15.92:0/999466349 learned_addr learned my addr 172.21.15.92:0/999466349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c10d8700 1 --2- 172.21.15.92:0/999466349 >> v2:172.21.15.158:3300/0 conn(0x7f12bc154d50 0x7f12bc0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 -- 172.21.15.92:0/999466349 >> v2:172.21.15.158:3300/0 conn(0x7f12bc154d50 msgr2=0x7f12bc0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 --2- 172.21.15.92:0/999466349 >> v2:172.21.15.158:3300/0 conn(0x7f12bc154d50 0x7f12bc0a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 -- 172.21.15.92:0/999466349 >> v1:172.21.15.158:6789/0 conn(0x7f12bc155130 legacy=0x7f12bc066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 -- 172.21.15.92:0/999466349 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc067e20 con 0x7f12bc067560 2023-05-16T07:19:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.773+0000 7f12c18d9700 1 --2- 172.21.15.92:0/999466349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc067560 0x7f12bc0a4d70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f12bc066bf0 tx=0x7f12b4009840).ready entity=mon.0 client_cookie=f997bebf4c7cf403 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12b3fff700 1 -- 172.21.15.92:0/999466349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b400e070 con 0x7f12bc067560 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12b3fff700 1 -- 172.21.15.92:0/999466349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b400ba20 con 0x7f12bc067560 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12b3fff700 1 -- 172.21.15.92:0/999466349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b400bda0 con 0x7f12bc067560 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 -- 172.21.15.92:0/999466349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc067560 msgr2=0x7f12bc0a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 --2- 172.21.15.92:0/999466349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc067560 0x7f12bc0a4d70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f12bc066bf0 tx=0x7f12b4009840).stop 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 -- 172.21.15.92:0/999466349 shutdown_connections 2023-05-16T07:19:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 --2- 172.21.15.92:0/999466349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc067560 0x7f12bc0a4d70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 --2- 172.21.15.92:0/999466349 >> v2:172.21.15.158:3300/0 conn(0x7f12bc154d50 0x7f12bc0a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 -- 172.21.15.92:0/999466349 >> 172.21.15.92:0/999466349 conn(0x7f12bc069940 msgr2=0x7f12bc0c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 -- 172.21.15.92:0/999466349 shutdown_connections 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.774+0000 7f12c20da700 1 -- 172.21.15.92:0/999466349 wait complete. 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.775+0000 7f12c20da700 1 Processor -- start 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.775+0000 7f12c20da700 1 -- start start 2023-05-16T07:19:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c20da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12bc067560 0x7f12bc151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c20da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 0x7f12bc0a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c20da700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12bc155130 0x7f12bc137750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c20da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc138cd0 con 0x7f12bc154d50 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c20da700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1380c0 con 0x7f12bc155130 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c20da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1383a0 con 0x7f12bc067560 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c08d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 0x7f12bc0a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c08d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 0x7f12bc0a9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49652/0 (socket says 172.21.15.92:49652) 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c08d7700 1 -- 172.21.15.92:0/4088021178 learned_addr learned my addr 172.21.15.92:0/4088021178 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c10d8700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12bc067560 0x7f12bc151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.776+0000 7f12c18d9700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12bc155130 0x7f12bc137750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c08d7700 1 -- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12bc067560 msgr2=0x7f12bc151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c08d7700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12bc067560 0x7f12bc151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c08d7700 1 -- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12bc155130 msgr2=0x7f12bc137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c08d7700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12bc155130 0x7f12bc137750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c08d7700 1 -- 172.21.15.92:0/4088021178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4009ae0 con 0x7f12bc154d50 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c10d8700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12bc067560 0x7f12bc151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c08d7700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 0x7f12bc0a9f00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f12ac00c440 tx=0x7f12ac007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac007020 con 0x7f12bc154d50 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac007180 con 0x7f12bc154d50 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc138680 con 0x7f12bc154d50 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac007520 con 0x7f12bc154d50 2023-05-16T07:19:36.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.777+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc0aa3f0 con 0x7f12bc154d50 2023-05-16T07:19:36.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.778+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12bc047150 con 0x7f12bc154d50 2023-05-16T07:19:36.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.778+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f12ac010070 con 0x7f12bc154d50 2023-05-16T07:19:36.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.779+0000 7f12b17fa700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 0x7f12a00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:36.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.779+0000 7f12c10d8700 1 -- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 msgr2=0x7f12a00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:36.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.779+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f12ac079720 con 0x7f12bc154d50 2023-05-16T07:19:36.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.779+0000 7f12c10d8700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 0x7f12a00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.783+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f12ac04a8e0 con 0x7f12bc154d50 2023-05-16T07:19:36.832 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:36.829+0000 7f2fb3812680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:36.890 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:36.886+0000 7fa92f2e3680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:36.909 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:36.905+0000 7f2fb3812680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:36.959 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:36.955+0000 7fa92f2e3680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:36.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.980+0000 7f12c10d8700 1 -- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 msgr2=0x7f12a00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:36.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.980+0000 7f12c10d8700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 0x7f12a00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.994+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12bc068100 con 0x7f12bc154d50 2023-05-16T07:19:36.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.994+0000 7f12b17fa700 1 -- 172.21.15.92:0/4088021178 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f12ac04df00 con 0x7f12bc154d50 2023-05-16T07:19:36.998 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:36.999 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:19.134223+0000","modified":"2023-05-16T07:19:21.140319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7461":{"gid":7461,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/1482305851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1482305851},{"type":"v1","addr":"172.21.15.92:6835","nonce":1482305851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-05-16T07:19:37.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 msgr2=0x7f12a00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:37.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 0x7f12a00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:37.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 msgr2=0x7f12bc0a9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 0x7f12bc0a9f00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f12ac00c440 tx=0x7f12ac007dd0).stop 2023-05-16T07:19:37.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 shutdown_connections 2023-05-16T07:19:37.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f12a005ede0 0x7f12a00612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:37.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f12bc067560 0x7f12bc151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12bc155130 0x7f12bc137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 --2- 172.21.15.92:0/4088021178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12bc154d50 0x7f12bc0a9f00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 >> 172.21.15.92:0/4088021178 conn(0x7f12bc069940 msgr2=0x7f12bc155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:37.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 shutdown_connections 2023-05-16T07:19:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:36.996+0000 7f12c20da700 1 -- 172.21.15.92:0/4088021178 wait complete. 2023-05-16T07:19:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-05-16T07:19:37.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:37.060 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:37.057+0000 7f2fb3812680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:37.070 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:37.066+0000 7fa92f2e3680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:37.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.125+0000 7f8812830700 1 Processor -- start 2023-05-16T07:19:37.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.125+0000 7f8812830700 1 -- start start 2023-05-16T07:19:37.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8812830700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8812830700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f880c154550 0x7f880c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8812830700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f880c0696a0 0x7f880c069b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8812830700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880c0c0660 con 0x7f880c0696a0 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8812830700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880c06a000 con 0x7f880c154550 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8812830700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0bfd30 con 0x7f880c154170 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f880c0696a0 0x7f880c069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f880c0696a0 0x7f880c069b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49662/0 (socket says 172.21.15.92:49662) 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 -- 172.21.15.92:0/1277889036 learned_addr learned my addr 172.21.15.92:0/1277889036 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f880bfff700 1 --2- 172.21.15.92:0/1277889036 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f880b7fe700 1 --2- 172.21.15.92:0/1277889036 >> v2:172.21.15.158:3300/0 conn(0x7f880c154550 0x7f880c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 -- 172.21.15.92:0/1277889036 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 msgr2=0x7f880c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 --2- 172.21.15.92:0/1277889036 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 -- 172.21.15.92:0/1277889036 >> v2:172.21.15.158:3300/0 conn(0x7f880c154550 msgr2=0x7f880c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 --2- 172.21.15.92:0/1277889036 >> v2:172.21.15.158:3300/0 conn(0x7f880c154550 0x7f880c0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 -- 172.21.15.92:0/1277889036 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0c0010 con 0x7f880c0696a0 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f880b7fe700 1 --2- 172.21.15.92:0/1277889036 >> v2:172.21.15.158:3300/0 conn(0x7f880c154550 0x7f880c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f8810dcd700 1 --2- 172.21.15.92:0/1277889036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c069b10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f880c083fe0 tx=0x7f8804009840).ready entity=mon.0 client_cookie=58dd53af71ce60fb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.126+0000 7f880affd700 1 -- 172.21.15.92:0/1277889036 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880400e070 con 0x7f880c0696a0 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f880affd700 1 -- 172.21.15.92:0/1277889036 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880400b970 con 0x7f880c0696a0 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f880affd700 1 -- 172.21.15.92:0/1277889036 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880400bcf0 con 0x7f880c0696a0 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 -- 172.21.15.92:0/1277889036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 msgr2=0x7f880c069b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 --2- 172.21.15.92:0/1277889036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c069b10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f880c083fe0 tx=0x7f8804009840).stop 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 -- 172.21.15.92:0/1277889036 shutdown_connections 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 --2- 172.21.15.92:0/1277889036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c069b10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 --2- 172.21.15.92:0/1277889036 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 --2- 172.21.15.92:0/1277889036 >> v2:172.21.15.158:3300/0 conn(0x7f880c154550 0x7f880c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 -- 172.21.15.92:0/1277889036 >> 172.21.15.92:0/1277889036 conn(0x7f880c0cb750 msgr2=0x7f880c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 -- 172.21.15.92:0/1277889036 shutdown_connections 2023-05-16T07:19:37.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.127+0000 7f8812830700 1 -- 172.21.15.92:0/1277889036 wait complete. 2023-05-16T07:19:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.129+0000 7f8812830700 1 Processor -- start 2023-05-16T07:19:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.129+0000 7f8812830700 1 -- start start 2023-05-16T07:19:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.130+0000 7f8812830700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c1447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.130+0000 7f8812830700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.130+0000 7f8812830700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f880c154550 0x7f880c0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.130+0000 7f8812830700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0c1d40 con 0x7f880c0696a0 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.130+0000 7f8812830700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c151060 con 0x7f880c154550 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.130+0000 7f8812830700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0ae7c0 con 0x7f880c154170 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c1447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c1447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49664/0 (socket says 172.21.15.92:49664) 2023-05-16T07:19:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 -- 172.21.15.92:0/2716716658 learned_addr learned my addr 172.21.15.92:0/2716716658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880b7fe700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f8810dcd700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f880c154550 0x7f880c0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 -- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 msgr2=0x7f880c0adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 -- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f880c154550 msgr2=0x7f880c0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f880c154550 0x7f880c0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 -- 172.21.15.92:0/2716716658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0092a0 con 0x7f880c0696a0 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f880bfff700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c1447c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f880000e930 tx=0x7f8800009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800012600 con 0x7f880c0696a0 2023-05-16T07:19:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8800012760 con 0x7f880c0696a0 2023-05-16T07:19:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8804009ae0 con 0x7f880c0696a0 2023-05-16T07:19:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800012ae0 con 0x7f880c0696a0 2023-05-16T07:19:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.131+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c0aed80 con 0x7f880c0696a0 2023-05-16T07:19:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.132+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c0666c0 con 0x7f880c0696a0 2023-05-16T07:19:37.140 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:37.133+0000 7f2fb3812680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:37.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.134+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f8800022070 con 0x7f880c0696a0 2023-05-16T07:19:37.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.134+0000 7f87f3fff700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87f805ede0 0x7f87f80612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.134+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f880007a840 con 0x7f880c0696a0 2023-05-16T07:19:37.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.134+0000 7f880b7fe700 1 -- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87f805ede0 msgr2=0x7f87f80612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:37.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.134+0000 7f880b7fe700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87f805ede0 0x7f87f80612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:37.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.141+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f880004ba80 con 0x7f880c0696a0 2023-05-16T07:19:37.176 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:37.172+0000 7fa92f2e3680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:37.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:37.252+0000 7f2fb3812680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:37.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.310+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f880c06c350 con 0x7f880c0696a0 2023-05-16T07:19:37.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.311+0000 7f87f3fff700 1 -- 172.21.15.92:0/2716716658 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18164 (secure 0 0 0) 0x7f880004f0a0 con 0x7f880c0696a0 2023-05-16T07:19:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:37.315 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":122,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:30.307193+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:17.674878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:18.097566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/931936336":"2023-05-17T07:19:30.307167+0000","172.21.15.158:6833/86301":"2023-05-17T07:19:30.307167+0000","172.21.15.158:6832/86301":"2023-05-17T07:19:30.307167+0000","172.21.15.158:0/2541040642":"2023-05-17T07:19:30.307167+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87f805ede0 msgr2=0x7f87f80612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87f805ede0 0x7f87f80612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 msgr2=0x7f880c1447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c1447c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f880000e930 tx=0x7f8800009060).stop 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 shutdown_connections 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f87f805ede0 0x7f87f80612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:37.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f880c154550 0x7f880c0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f880c0696a0 0x7f880c1447c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 --2- 172.21.15.92:0/2716716658 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f880c154170 0x7f880c0adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 >> 172.21.15.92:0/2716716658 conn(0x7f880c0cb750 msgr2=0x7f880c0aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:37.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 shutdown_connections 2023-05-16T07:19:37.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.313+0000 7f8812830700 1 -- 172.21.15.92:0/2716716658 wait complete. 2023-05-16T07:19:37.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:19:37.368 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:37.364+0000 7f2fb3812680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:37.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:19:37.415 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:19:37.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.449+0000 7f4ff56be700 1 Processor -- start 2023-05-16T07:19:37.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.449+0000 7f4ff56be700 1 -- start start 2023-05-16T07:19:37.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4ff56be700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4ff56be700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ff0069b90 0x7f4ff00c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4ff56be700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff00c0660 con 0x7f4ff0154170 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4ff56be700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff006a020 con 0x7f4ff0069b90 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4ff56be700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00bfd30 con 0x7f4ff0154550 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4fef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ff0069b90 0x7f4ff00c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4fef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ff0069b90 0x7f4ff00c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49670/0 (socket says 172.21.15.92:49670) 2023-05-16T07:19:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4fef7fe700 1 -- 172.21.15.92:0/1516178628 learned_addr learned my addr 172.21.15.92:0/1516178628 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.450+0000 7f4fee7fc700 1 --2- 172.21.15.92:0/1516178628 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fef7fe700 1 -- 172.21.15.92:0/1516178628 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 msgr2=0x7f4ff00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fef7fe700 1 --2- 172.21.15.92:0/1516178628 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff00696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fef7fe700 1 -- 172.21.15.92:0/1516178628 >> v1:172.21.15.158:6789/0 conn(0x7f4ff0154170 legacy=0x7f4ff00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fef7fe700 1 -- 172.21.15.92:0/1516178628 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff00c0010 con 0x7f4ff0069b90 2023-05-16T07:19:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fef7fe700 1 --2- 172.21.15.92:0/1516178628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00c0af0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f4fe8009f20 tx=0x7f4fe8009840).ready entity=mon.0 client_cookie=30bff5c35240ca51 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fedffb700 1 -- 172.21.15.92:0/1516178628 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800e070 con 0x7f4ff0069b90 2023-05-16T07:19:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fedffb700 1 -- 172.21.15.92:0/1516178628 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800b9d0 con 0x7f4ff0069b90 2023-05-16T07:19:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4fedffb700 1 -- 172.21.15.92:0/1516178628 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800bd50 con 0x7f4ff0069b90 2023-05-16T07:19:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4ff56be700 1 -- 172.21.15.92:0/1516178628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 msgr2=0x7f4ff00c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.451+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1516178628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00c0af0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f4fe8009f20 tx=0x7f4fe8009840).stop 2023-05-16T07:19:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.452+0000 7f4ff56be700 1 -- 172.21.15.92:0/1516178628 shutdown_connections 2023-05-16T07:19:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.452+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1516178628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00c0af0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.452+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1516178628 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff00696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.452+0000 7f4ff56be700 1 -- 172.21.15.92:0/1516178628 >> 172.21.15.92:0/1516178628 conn(0x7f4ff00cb700 msgr2=0x7f4ff00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.452+0000 7f4ff56be700 1 -- 172.21.15.92:0/1516178628 shutdown_connections 2023-05-16T07:19:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.452+0000 7f4ff56be700 1 -- 172.21.15.92:0/1516178628 wait complete. 2023-05-16T07:19:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.453+0000 7f4ff56be700 1 Processor -- start 2023-05-16T07:19:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.453+0000 7f4ff56be700 1 -- start start 2023-05-16T07:19:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.454+0000 7f4ff56be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.454+0000 7f4ff56be700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ff0154170 0x7f4ff0156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.454+0000 7f4ff56be700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff014d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.454+0000 7f4ff56be700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00afa10 con 0x7f4ff0069b90 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.454+0000 7f4ff56be700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff0157150 con 0x7f4ff0154170 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.454+0000 7f4ff56be700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff0157430 con 0x7f4ff0154550 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49674/0 (socket says 172.21.15.92:49674) 2023-05-16T07:19:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 -- 172.21.15.92:0/1964336650 learned_addr learned my addr 172.21.15.92:0/1964336650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4fee7fc700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ff0154170 0x7f4ff0156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4fef7fe700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff014d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 msgr2=0x7f4ff014d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff014d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ff0154170 msgr2=0x7f4ff0156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ff0154170 0x7f4ff0156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 -- 172.21.15.92:0/1964336650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8009ae0 con 0x7f4ff0069b90 2023-05-16T07:19:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4fef7fe700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff014d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4feeffd700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00afc50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f4fe400c4c0 tx=0x7f4fe4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4007020 con 0x7f4ff0069b90 2023-05-16T07:19:37.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4001e90 con 0x7f4ff0069b90 2023-05-16T07:19:37.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40073b0 con 0x7f4ff0069b90 2023-05-16T07:19:37.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff0157710 con 0x7f4ff0069b90 2023-05-16T07:19:37.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.455+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff00c2a30 con 0x7f4ff0069b90 2023-05-16T07:19:37.469 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:37.465+0000 7fa92f2e3680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:19:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.456+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff00666c0 con 0x7f4ff0069b90 2023-05-16T07:19:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.457+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 52) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f4fe4010070 con 0x7f4ff0069b90 2023-05-16T07:19:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.457+0000 7f4fd6ffd700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 0x7f4fdc0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:37.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.457+0000 7f4fee7fc700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 msgr2=0x7f4fdc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:37.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.457+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f4fe40797e0 con 0x7f4ff0069b90 2023-05-16T07:19:37.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.457+0000 7f4fee7fc700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 0x7f4fdc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:37.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.461+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4fe404a9a0 con 0x7f4ff0069b90 2023-05-16T07:19:37.539 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:37.535+0000 7fa92f2e3680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:19:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.658+0000 7f4fee7fc700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 msgr2=0x7f4fdc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:37.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.658+0000 7f4fee7fc700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 0x7f4fdc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:37.666 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:37.663+0000 7f2fb3812680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:19:37.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:37.675+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4ff006c350 con 0x7f4ff0069b90 2023-05-16T07:19:37.738 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:37.734+0000 7f2fb3812680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:19:38.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.059+0000 7f4fee7fc700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 msgr2=0x7f4fdc0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:38.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.059+0000 7f4fee7fc700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 0x7f4fdc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:19:38.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.407+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 53) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f4fe4077300 con 0x7f4ff0069b90 2023-05-16T07:19:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.407+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 msgr2=0x7f4fdc0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.407+0000 7f4fd6ffd700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 0x7f4fdc0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:38.463 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8b22ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.464 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8b22ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.464 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8b22ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.464 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8b22ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.464 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8b22ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.464 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8af2c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.464 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8af2c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.465 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8af2c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.465 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8af2c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.465 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:38.459+0000 7fa8af2c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:19:38.468 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-05-16T07:19:38.464+0000 7fb389ad4700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:19:38.468 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-05-16T07:19:38.464+0000 7fb389ad4700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:19:38.468 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-05-16T07:19:38.464+0000 7fb389ad4700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:19:38.471 INFO:tasks.ceph.mds.c.smithi092.stderr: -2141> 2023-05-16T07:19:25.060+0000 7fb38d2db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:19:38.474 INFO:tasks.ceph.mds.c.smithi092.stderr: -4> 2023-05-16T07:19:38.464+0000 7fb389ad4700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:19:38.474 INFO:tasks.ceph.mds.c.smithi092.stderr: -3> 2023-05-16T07:19:38.464+0000 7fb389ad4700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:19:38.474 INFO:tasks.ceph.mds.c.smithi092.stderr: -1> 2023-05-16T07:19:38.464+0000 7fb389ad4700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:19:38.493 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:19:38.556 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:19:38.552+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:19:38.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.557+0000 7f4fd6ffd700 1 -- 172.21.15.92:0/1964336650 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4fe406ebb0 con 0x7f4ff0069b90 2023-05-16T07:19:38.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 msgr2=0x7f4ff00afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00afc50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f4fe400c4c0 tx=0x7f4fe4007e00).stop 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 shutdown_connections 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f4fdc05ede0 0x7f4fdc0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4ff0154550 0x7f4ff014d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ff0069b90 0x7f4ff00afc50 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 --2- 172.21.15.92:0/1964336650 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4ff0154170 0x7f4ff0156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.559+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 >> 172.21.15.92:0/1964336650 conn(0x7f4ff00cb700 msgr2=0x7f4ff00c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.560+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 shutdown_connections 2023-05-16T07:19:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.560+0000 7f4ff56be700 1 -- 172.21.15.92:0/1964336650 wait complete. 2023-05-16T07:19:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:19:38.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:19:38.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.686+0000 7f0745448700 1 Processor -- start 2023-05-16T07:19:38.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.686+0000 7f0745448700 1 -- start start 2023-05-16T07:19:38.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f0745448700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0740154470 0x7f07400aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:38.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f0745448700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0740068160 0x7f07400685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:38.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f0745448700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400c94d0 con 0x7f0740154090 2023-05-16T07:19:38.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f0745448700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07400cb7e0 con 0x7f0740068160 2023-05-16T07:19:38.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f0745448700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0740068aa0 con 0x7f0740154470 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0740068160 0x7f07400685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0740068160 0x7f07400685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49706/0 (socket says 172.21.15.92:49706) 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0740154090 0x7f07400a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60452/0 (socket says 172.21.15.92:60452) 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073f7fe700 1 -- 172.21.15.92:0/548885523 learned_addr learned my addr 172.21.15.92:0/548885523 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073e7fc700 1 --2- 172.21.15.92:0/548885523 >> v2:172.21.15.158:3300/0 conn(0x7f0740154470 0x7f07400aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073dffb700 1 -- 172.21.15.92:0/548885523 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345701029 0 0) 0x7f07400c94d0 con 0x7f0740154090 2023-05-16T07:19:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073dffb700 1 -- 172.21.15.92:0/548885523 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f072c0033f0 con 0x7f0740154090 2023-05-16T07:19:38.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073f7fe700 1 -- 172.21.15.92:0/548885523 >> v2:172.21.15.158:3300/0 conn(0x7f0740154470 msgr2=0x7f07400aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:38.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.687+0000 7f073f7fe700 1 --2- 172.21.15.92:0/548885523 >> v2:172.21.15.158:3300/0 conn(0x7f0740154470 0x7f07400aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f073f7fe700 1 -- 172.21.15.92:0/548885523 >> v1:172.21.15.92:6789/0 conn(0x7f0740154090 legacy=0x7f07400a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:38.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f073f7fe700 1 -- 172.21.15.92:0/548885523 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400c88c0 con 0x7f0740068160 2023-05-16T07:19:38.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f073f7fe700 1 --2- 172.21.15.92:0/548885523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740068160 0x7f07400685b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f0740084320 tx=0x7f07380090f0).ready entity=mon.0 client_cookie=e432b5136954ccb3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f073dffb700 1 -- 172.21.15.92:0/548885523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073800f510 con 0x7f0740068160 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f073dffb700 1 -- 172.21.15.92:0/548885523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073800f670 con 0x7f0740068160 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f073dffb700 1 -- 172.21.15.92:0/548885523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073800f9f0 con 0x7f0740068160 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 -- 172.21.15.92:0/548885523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740068160 msgr2=0x7f07400685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 --2- 172.21.15.92:0/548885523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740068160 0x7f07400685b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f0740084320 tx=0x7f07380090f0).stop 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 -- 172.21.15.92:0/548885523 shutdown_connections 2023-05-16T07:19:38.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 --2- 172.21.15.92:0/548885523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740068160 0x7f07400685b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 --2- 172.21.15.92:0/548885523 >> v2:172.21.15.158:3300/0 conn(0x7f0740154470 0x7f07400aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 -- 172.21.15.92:0/548885523 >> 172.21.15.92:0/548885523 conn(0x7f0740067920 msgr2=0x7f07400cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:38.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 -- 172.21.15.92:0/548885523 shutdown_connections 2023-05-16T07:19:38.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.688+0000 7f0745448700 1 -- 172.21.15.92:0/548885523 wait complete. 2023-05-16T07:19:38.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 Processor -- start 2023-05-16T07:19:38.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 -- start start 2023-05-16T07:19:38.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0740068160 0x7f074014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:38.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 0x7f07401586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:38.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0740154470 0x7f074015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400aedd0 con 0x7f0740154090 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400af0b0 con 0x7f0740068160 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f0745448700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400afe30 con 0x7f0740154470 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f073e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 0x7f07401586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.690+0000 7f073e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 0x7f07401586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49714/0 (socket says 172.21.15.92:49714) 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 -- 172.21.15.92:0/355123071 learned_addr learned my addr 172.21.15.92:0/355123071 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073effd700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0740068160 0x7f074014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073f7fe700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0740154470 0x7f074015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 -- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0740154470 msgr2=0x7f074015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0740154470 0x7f074015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 -- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0740068160 msgr2=0x7f074014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0740068160 0x7f074014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 -- 172.21.15.92:0/355123071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0738009370 con 0x7f0740154090 2023-05-16T07:19:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073f7fe700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0740154470 0x7f074015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f073e7fc700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 0x7f07401586a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f072800e900 tx=0x7f0728009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728012600 con 0x7f0740154090 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728012760 con 0x7f0740154090 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07400b0110 con 0x7f0740154090 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728012ae0 con 0x7f0740154090 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.691+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07400ae9f0 con 0x7f0740154090 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.692+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07400b0630 con 0x7f0740154090 2023-05-16T07:19:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.692+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 53) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f0728022070 con 0x7f0740154090 2023-05-16T07:19:38.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.693+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f072807a090 con 0x7f0740154090 2023-05-16T07:19:38.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.697+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f07400b0630 con 0x7f0740154090 2023-05-16T07:19:38.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:38.899+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07400b0630 con 0x7f0740154090 2023-05-16T07:19:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.410+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 54) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f0728077bf0 con 0x7f0740154090 2023-05-16T07:19:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.410+0000 7f0736ffd700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f07200301b0 0x7f0720067d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.414+0000 7f073effd700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f07200301b0 0x7f0720067d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.415+0000 7f073effd700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f07200301b0 0x7f0720067d60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f0730007fd0 tx=0x7f0730000f40).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.662+0000 7f0736ffd700 1 -- 172.21.15.92:0/355123071 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f072806ceb0 con 0x7f0740154090 2023-05-16T07:19:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f07200301b0 msgr2=0x7f0720067d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f07200301b0 0x7f0720067d60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f0730007fd0 tx=0x7f0730000f40).stop 2023-05-16T07:19:39.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 msgr2=0x7f07401586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:39.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 0x7f07401586a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f072800e900 tx=0x7f0728009060).stop 2023-05-16T07:19:39.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 shutdown_connections 2023-05-16T07:19:39.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f07200301b0 0x7f0720067d60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0740154470 0x7f074015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0740068160 0x7f074014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 --2- 172.21.15.92:0/355123071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0740154090 0x7f07401586a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 >> 172.21.15.92:0/355123071 conn(0x7f0740067920 msgr2=0x7f07400c1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 shutdown_connections 2023-05-16T07:19:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.664+0000 7f0745448700 1 -- 172.21.15.92:0/355123071 wait complete. 2023-05-16T07:19:39.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:19:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.794+0000 7faf5359e700 1 Processor -- start 2023-05-16T07:19:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.794+0000 7faf5359e700 1 -- start start 2023-05-16T07:19:39.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.795+0000 7faf5359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf540a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.795+0000 7faf5359e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7faf54155380 0x7faf540c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.795+0000 7faf5359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf540c6670 con 0x7faf540c5d00 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.795+0000 7faf5359e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf54069f60 con 0x7faf54155380 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.795+0000 7faf5359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5406a9e0 con 0x7faf54153f90 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf51d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faf540c5d00 0x7faf540c0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60482/0 (socket says 172.21.15.92:60482) 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf51d9b700 1 -- 172.21.15.92:0/1014771179 learned_addr learned my addr 172.21.15.92:0/1014771179 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767593729 0 0) 0x7faf540c6670 con 0x7faf540c5d00 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5259c700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf540a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf52d9d700 1 --2- 172.21.15.92:0/1014771179 >> v2:172.21.15.158:3300/0 conn(0x7faf54155380 0x7faf540c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf400033f0 con 0x7faf540c5d00 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3898772281 0 0) 0x7faf400033f0 con 0x7faf540c5d00 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 msgr2=0x7faf540a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf540a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 >> v2:172.21.15.158:3300/0 conn(0x7faf54155380 msgr2=0x7faf540c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 --2- 172.21.15.92:0/1014771179 >> v2:172.21.15.158:3300/0 conn(0x7faf54155380 0x7faf540c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5406a140 con 0x7faf540c5d00 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7faf3c001470 con 0x7faf540c5d00 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 >> v1:172.21.15.92:6789/0 conn(0x7faf540c5d00 legacy=0x7faf540c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf40003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf40004690 0x7faf40004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf40005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf54069f60 con 0x7faf400038e0 2023-05-16T07:19:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400044b0 con 0x7faf40004690 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4000b910 con 0x7faf54153f90 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf5259c700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf40003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf51d9b700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf40004690 0x7faf40004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.796+0000 7faf52d9d700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf40005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5259c700 1 -- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 msgr2=0x7faf40005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5259c700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf40005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5259c700 1 -- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf40004690 msgr2=0x7faf40004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5259c700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf40004690 0x7faf40004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5259c700 1 -- 172.21.15.92:0/1014771179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400033f0 con 0x7faf400038e0 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf52d9d700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf54153f90 0x7faf40005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf51d9b700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf40004690 0x7faf40004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5259c700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf40003d00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7faf4400e210 tx=0x7faf4400b040).ready entity=mon.0 client_cookie=451d31f0eaffce82 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44009ac0 con 0x7faf400038e0 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4400f040 con 0x7faf400038e0 2023-05-16T07:19:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5159a700 1 -- 172.21.15.92:0/1014771179 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400ecd0 con 0x7faf400038e0 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5359e700 1 -- 172.21.15.92:0/1014771179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 msgr2=0x7faf40003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5359e700 1 --2- 172.21.15.92:0/1014771179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf40003d00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7faf4400e210 tx=0x7faf4400b040).stop 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf52d9d700 1 -- 172.21.15.92:0/1014771179 reap_dead start 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5359e700 1 -- 172.21.15.92:0/1014771179 shutdown_connections 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5359e700 1 -- 172.21.15.92:0/1014771179 >> 172.21.15.92:0/1014771179 conn(0x7faf540cb600 msgr2=0x7faf540c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5359e700 1 -- 172.21.15.92:0/1014771179 shutdown_connections 2023-05-16T07:19:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.797+0000 7faf5359e700 1 -- 172.21.15.92:0/1014771179 wait complete. 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.798+0000 7faf5359e700 1 Processor -- start 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 -- start start 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf5414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf54156980 0x7faf54156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf540c3300 0x7faf540c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf540c3c50 con 0x7faf400038e0 2023-05-16T07:19:39.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf540c2e70 con 0x7faf54156980 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf540c2540 con 0x7faf540c3300 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf5414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf5414f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49744/0 (socket says 172.21.15.92:49744) 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf5259c700 1 -- 172.21.15.92:0/1803155158 learned_addr learned my addr 172.21.15.92:0/1803155158 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf52d9d700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf540c3300 0x7faf540c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.799+0000 7faf51d9b700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf54156980 0x7faf54156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5259c700 1 -- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf540c3300 msgr2=0x7faf540c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5259c700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf540c3300 0x7faf540c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5259c700 1 -- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf54156980 msgr2=0x7faf54156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5259c700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf54156980 0x7faf54156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5259c700 1 -- 172.21.15.92:0/1803155158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540c2820 con 0x7faf400038e0 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5259c700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf5414f730 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7faf4400e800 tx=0x7faf4401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44013cb0 con 0x7faf400038e0 2023-05-16T07:19:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44013e10 con 0x7faf400038e0 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf540c2b00 con 0x7faf400038e0 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44027420 con 0x7faf400038e0 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.800+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5414fe60 con 0x7faf400038e0 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.801+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf540c3150 con 0x7faf400038e0 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.801+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 54) v1 ==== 75162+0+0 (secure 0 0 0) 0x7faf4402e070 con 0x7faf400038e0 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.801+0000 7faf4a7fc700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7faf3405ecf0 0x7faf340611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:39.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.802+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7206+0+0 (secure 0 0 0) 0x7faf4407c010 con 0x7faf400038e0 2023-05-16T07:19:39.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.802+0000 7faf51d9b700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7faf3405ecf0 0x7faf340611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:39.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.802+0000 7faf51d9b700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7faf3405ecf0 0x7faf340611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7faf3c000fd0 tx=0x7faf3c009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:39.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.806+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7faf4404d100 con 0x7faf400038e0 2023-05-16T07:19:39.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:39.968+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf54083120 con 0x7faf400038e0 2023-05-16T07:19:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.422+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7faf44079a60 con 0x7faf400038e0 2023-05-16T07:19:40.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.665+0000 7faf4a7fc700 1 -- 172.21.15.92:0/1803155158 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7faf440561a0 con 0x7faf400038e0 2023-05-16T07:19:40.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7faf3405ecf0 msgr2=0x7faf340611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:40.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7faf3405ecf0 0x7faf340611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7faf3c000fd0 tx=0x7faf3c009040).stop 2023-05-16T07:19:40.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 msgr2=0x7faf5414f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:40.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf5414f730 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7faf4400e800 tx=0x7faf4401a000).stop 2023-05-16T07:19:40.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 shutdown_connections 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7faf3405ecf0 0x7faf340611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7faf540c3300 0x7faf540c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf400038e0 0x7faf5414f730 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 --2- 172.21.15.92:0/1803155158 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faf54156980 0x7faf54156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 >> 172.21.15.92:0/1803155158 conn(0x7faf540cb600 msgr2=0x7faf540c98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 shutdown_connections 2023-05-16T07:19:40.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.669+0000 7faf5359e700 1 -- 172.21.15.92:0/1803155158 wait complete. 2023-05-16T07:19:40.675 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:19:40.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:19:40.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.804+0000 7fbb42484700 1 Processor -- start 2023-05-16T07:19:40.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.804+0000 7fbb42484700 1 -- start start 2023-05-16T07:19:40.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb42484700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbb3c064a50 0x7fbb3c068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:40.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb42484700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb3c06a0f0 0x7fbb3c066490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:40.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb42484700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c069c60 con 0x7fbb3c155840 2023-05-16T07:19:40.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb42484700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c135380 con 0x7fbb3c06a0f0 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb42484700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c069050 con 0x7fbb3c064a50 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb3c06a0f0 0x7fbb3c066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb3c06a0f0 0x7fbb3c066490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49752/0 (socket says 172.21.15.92:49752) 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/1714015987 learned_addr learned my addr 172.21.15.92:0/1714015987 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/1714015987 >> v2:172.21.15.158:3300/0 conn(0x7fbb3c064a50 0x7fbb3c068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/1714015987 >> v2:172.21.15.158:3300/0 conn(0x7fbb3c064a50 msgr2=0x7fbb3c068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:40.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 --2- 172.21.15.92:0/1714015987 >> v2:172.21.15.158:3300/0 conn(0x7fbb3c064a50 0x7fbb3c068a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/1714015987 >> v1:172.21.15.158:6789/0 conn(0x7fbb3c155840 legacy=0x7fbb3c0bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:40.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 -- 172.21.15.92:0/1714015987 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c069330 con 0x7fbb3c06a0f0 2023-05-16T07:19:40.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/1714015987 >> v2:172.21.15.158:3300/0 conn(0x7fbb3c064a50 0x7fbb3c068a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:40.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3b7fe700 1 --2- 172.21.15.92:0/1714015987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c06a0f0 0x7fbb3c066490 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7fbb3c0c3a00 tx=0x7fbb300090f0).ready entity=mon.0 client_cookie=56fdd0b1072a45fe server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:40.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3affd700 1 -- 172.21.15.92:0/1714015987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000b600 con 0x7fbb3c06a0f0 2023-05-16T07:19:40.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3affd700 1 -- 172.21.15.92:0/1714015987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3000b760 con 0x7fbb3c06a0f0 2023-05-16T07:19:40.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.805+0000 7fbb3affd700 1 -- 172.21.15.92:0/1714015987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000bae0 con 0x7fbb3c06a0f0 2023-05-16T07:19:40.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 -- 172.21.15.92:0/1714015987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c06a0f0 msgr2=0x7fbb3c066490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:40.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 --2- 172.21.15.92:0/1714015987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c06a0f0 0x7fbb3c066490 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7fbb3c0c3a00 tx=0x7fbb300090f0).stop 2023-05-16T07:19:40.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 -- 172.21.15.92:0/1714015987 shutdown_connections 2023-05-16T07:19:40.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 --2- 172.21.15.92:0/1714015987 >> v2:172.21.15.158:3300/0 conn(0x7fbb3c064a50 0x7fbb3c068a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 --2- 172.21.15.92:0/1714015987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c06a0f0 0x7fbb3c066490 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 -- 172.21.15.92:0/1714015987 >> 172.21.15.92:0/1714015987 conn(0x7fbb3c0c4620 msgr2=0x7fbb3c066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:40.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 -- 172.21.15.92:0/1714015987 shutdown_connections 2023-05-16T07:19:40.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.806+0000 7fbb42484700 1 -- 172.21.15.92:0/1714015987 wait complete. 2023-05-16T07:19:40.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.807+0000 7fbb42484700 1 Processor -- start 2023-05-16T07:19:40.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.807+0000 7fbb42484700 1 -- start start 2023-05-16T07:19:40.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb42484700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c064a50 0x7fbb3c134b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb42484700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c06a0f0 0x7fbb3c0be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb42484700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 0x7fbb3c0beac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb42484700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0aa9a0 con 0x7fbb3c155840 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb42484700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c135080 con 0x7fbb3c064a50 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb42484700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c1383c0 con 0x7fbb3c06a0f0 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 0x7fbb3c0beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:40.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 0x7fbb3c0beac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49758/0 (socket says 172.21.15.92:49758) 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 -- 172.21.15.92:0/607252535 learned_addr learned my addr 172.21.15.92:0/607252535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c064a50 0x7fbb3c134b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb3b7fe700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c06a0f0 0x7fbb3c0be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 -- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c06a0f0 msgr2=0x7fbb3c0be5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c06a0f0 0x7fbb3c0be5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 -- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c064a50 msgr2=0x7fbb3c134b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:40.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c064a50 0x7fbb3c134b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 -- 172.21.15.92:0/607252535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30009370 con 0x7fbb3c155840 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.808+0000 7fbb40a21700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 0x7fbb3c0beac0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7fbb34002230 tx=0x7fbb340015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.809+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34010510 con 0x7fbb3c155840 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.809+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34010670 con 0x7fbb3c155840 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.809+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c0aa350 con 0x7fbb3c155840 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.809+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340109f0 con 0x7fbb3c155840 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.809+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0acc40 con 0x7fbb3c155840 2023-05-16T07:19:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.810+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c1386a0 con 0x7fbb3c155840 2023-05-16T07:19:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.810+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbb34010070 con 0x7fbb3c155840 2023-05-16T07:19:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.811+0000 7fbb23fff700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb2805ed90 0x7fbb28061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.811+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6768+0+0 (secure 0 0 0) 0x7fbb3407a4c0 con 0x7fbb3c155840 2023-05-16T07:19:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.811+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb2805ed90 0x7fbb28061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.811+0000 7fbb3bfff700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb2805ed90 0x7fbb28061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbb2c006fd0 tx=0x7fbb2c008040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.815+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbb3404b880 con 0x7fbb3c155840 2023-05-16T07:19:40.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:40.984+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb3c06c350 con 0x7fbb3c155840 2023-05-16T07:19:41.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.670+0000 7fbb23fff700 1 -- 172.21.15.92:0/607252535 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbb3404eea0 con 0x7fbb3c155840 2023-05-16T07:19:41.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb2805ed90 msgr2=0x7fbb28061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb2805ed90 0x7fbb28061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbb2c006fd0 tx=0x7fbb2c008040).stop 2023-05-16T07:19:41.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 msgr2=0x7fbb3c0beac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 0x7fbb3c0beac0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7fbb34002230 tx=0x7fbb340015e0).stop 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 shutdown_connections 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbb2805ed90 0x7fbb28061250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c155840 0x7fbb3c0beac0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbb3c064a50 0x7fbb3c134b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 --2- 172.21.15.92:0/607252535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbb3c06a0f0 0x7fbb3c0be5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 >> 172.21.15.92:0/607252535 conn(0x7fbb3c0c4620 msgr2=0x7fbb3c067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 shutdown_connections 2023-05-16T07:19:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.673+0000 7fbb42484700 1 -- 172.21.15.92:0/607252535 wait complete. 2023-05-16T07:19:41.679 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:19:41.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:19:41.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.804+0000 7f9e928a3700 1 Processor -- start 2023-05-16T07:19:41.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.804+0000 7f9e928a3700 1 -- start start 2023-05-16T07:19:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e928a3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9e8c14ea00 0x7f9e8c154760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e928a3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c155100 con 0x7f9e8c14e620 2023-05-16T07:19:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e928a3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c0c4830 con 0x7f9e8c154c50 2023-05-16T07:19:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e928a3700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8c065bf0 con 0x7f9e8c14ea00 2023-05-16T07:19:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e918a1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9e8c14e620 0x7f9e8c0c4120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60512/0 (socket says 172.21.15.92:60512) 2023-05-16T07:19:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e918a1700 1 -- 172.21.15.92:0/309301133 learned_addr learned my addr 172.21.15.92:0/309301133 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e910a0700 1 --2- 172.21.15.92:0/309301133 >> v2:172.21.15.158:3300/0 conn(0x7f9e8c14ea00 0x7f9e8c154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185286519 0 0) 0x7f9e88001110 con 0x7f9e8c14e620 2023-05-16T07:19:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e780033f0 con 0x7f9e8c14e620 2023-05-16T07:19:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3039434081 0 0) 0x7f9e780033f0 con 0x7f9e8c14e620 2023-05-16T07:19:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 >> v2:172.21.15.158:3300/0 conn(0x7f9e8c14ea00 msgr2=0x7f9e8c154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 --2- 172.21.15.92:0/309301133 >> v2:172.21.15.158:3300/0 conn(0x7f9e8c14ea00 0x7f9e8c154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 >> v1:172.21.15.158:6789/0 conn(0x7f9e8c154c50 legacy=0x7f9e8c0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c065ed0 con 0x7f9e8c14e620 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f9e88001600 con 0x7f9e8c14e620 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 >> v1:172.21.15.92:6789/0 conn(0x7f9e8c14e620 legacy=0x7f9e8c0c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780038e0 0x7f9e78003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e78004690 0x7f9e78004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e78005000 0x7f9e7800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c065bf0 con 0x7f9e780038e0 2023-05-16T07:19:41.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7800be00 con 0x7f9e78005000 2023-05-16T07:19:41.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7800bfc0 con 0x7f9e78004690 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.805+0000 7f9e918a1700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780038e0 0x7f9e78003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e920a2700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e78005000 0x7f9e7800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e910a0700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e78004690 0x7f9e78004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e918a1700 1 -- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e78004690 msgr2=0x7f9e78004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e918a1700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e78004690 0x7f9e78004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e918a1700 1 -- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e78005000 msgr2=0x7f9e7800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e918a1700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e78005000 0x7f9e7800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e918a1700 1 -- 172.21.15.92:0/309301133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780033f0 con 0x7f9e780038e0 2023-05-16T07:19:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e920a2700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e78005000 0x7f9e7800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e920a2700 1 -- 172.21.15.92:0/309301133 reap_dead start 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e918a1700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780038e0 0x7f9e78003cc0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f9e88007fd0 tx=0x7f9e8c064690).ready entity=mon.0 client_cookie=b6eebc010b3b2e1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800cba0 con 0x7f9e780038e0 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8800cd00 con 0x7f9e780038e0 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e9089f700 1 -- 172.21.15.92:0/309301133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e880072e0 con 0x7f9e780038e0 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e928a3700 1 -- 172.21.15.92:0/309301133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780038e0 msgr2=0x7f9e78003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.806+0000 7f9e928a3700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780038e0 0x7f9e78003cc0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f9e88007fd0 tx=0x7f9e8c064690).stop 2023-05-16T07:19:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.807+0000 7f9e928a3700 1 -- 172.21.15.92:0/309301133 shutdown_connections 2023-05-16T07:19:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.807+0000 7f9e928a3700 1 --2- 172.21.15.92:0/309301133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780038e0 0x7f9e78003cc0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.807+0000 7f9e928a3700 1 -- 172.21.15.92:0/309301133 >> 172.21.15.92:0/309301133 conn(0x7f9e8c0c7a80 msgr2=0x7f9e8c0c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.807+0000 7f9e928a3700 1 -- 172.21.15.92:0/309301133 shutdown_connections 2023-05-16T07:19:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.807+0000 7f9e928a3700 1 -- 172.21.15.92:0/309301133 wait complete. 2023-05-16T07:19:41.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.808+0000 7f9e928a3700 1 Processor -- start 2023-05-16T07:19:41.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.808+0000 7f9e928a3700 1 -- start start 2023-05-16T07:19:41.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e780038e0 0x7f9e8c14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 0x7f9e8c069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e8c0aab10 0x7f9e8c0aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0a7250 con 0x7f9e8c069190 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c069b00 con 0x7f9e780038e0 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0a6f20 con 0x7f9e8c0aab10 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 0x7f9e8c069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 0x7f9e8c069610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49778/0 (socket says 172.21.15.92:49778) 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 -- 172.21.15.92:0/2502581604 learned_addr learned my addr 172.21.15.92:0/2502581604 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:41.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e918a1700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e780038e0 0x7f9e8c14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e920a2700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e8c0aab10 0x7f9e8c0aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 -- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e8c0aab10 msgr2=0x7f9e8c0aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e8c0aab10 0x7f9e8c0aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 -- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e780038e0 msgr2=0x7f9e8c14bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e780038e0 0x7f9e8c14bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 -- 172.21.15.92:0/2502581604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c0ab840 con 0x7f9e8c069190 2023-05-16T07:19:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e910a0700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 0x7f9e8c069610 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9e7c011690 tx=0x7f9e7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c013620 con 0x7f9e8c069190 2023-05-16T07:19:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c013780 con 0x7f9e8c069190 2023-05-16T07:19:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c0abb20 con 0x7f9e8c069190 2023-05-16T07:19:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.809+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c013b40 con 0x7f9e8c069190 2023-05-16T07:19:41.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.810+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c0ac0e0 con 0x7f9e8c069190 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.810+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c0a7090 con 0x7f9e8c069190 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.811+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9e7c012040 con 0x7f9e8c069190 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.811+0000 7f9e81ffb700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e7005ed90 0x7f9e70061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.811+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f9e7c07a180 con 0x7f9e8c069190 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.811+0000 7f9e918a1700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e7005ed90 0x7f9e70061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.812+0000 7f9e918a1700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e7005ed90 0x7f9e70061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9e88007970 tx=0x7f9e88015000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:41.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.815+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9e8c0a7090 con 0x7f9e8c069190 2023-05-16T07:19:41.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.985+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9e8c0a7090 con 0x7f9e8c069190 2023-05-16T07:19:41.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.988+0000 7f9e81ffb700 1 -- 172.21.15.92:0/2502581604 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9e8c0a7090 con 0x7f9e8c069190 2023-05-16T07:19:41.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e7005ed90 msgr2=0x7f9e70061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e7005ed90 0x7f9e70061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9e88007970 tx=0x7f9e88015000).stop 2023-05-16T07:19:41.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 msgr2=0x7f9e8c069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:41.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 0x7f9e8c069610 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9e7c011690 tx=0x7f9e7c008040).stop 2023-05-16T07:19:41.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 shutdown_connections 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e7005ed90 0x7f9e70061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9e8c0aab10 0x7f9e8c0aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e780038e0 0x7f9e8c14bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 --2- 172.21.15.92:0/2502581604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e8c069190 0x7f9e8c069610 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 >> 172.21.15.92:0/2502581604 conn(0x7f9e8c0c7a80 msgr2=0x7f9e8c066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 shutdown_connections 2023-05-16T07:19:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:41.990+0000 7f9e928a3700 1 -- 172.21.15.92:0/2502581604 wait complete. 2023-05-16T07:19:42.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:19:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.121+0000 7f1fcc731700 1 Processor -- start 2023-05-16T07:19:42.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.121+0000 7f1fcc731700 1 -- start start 2023-05-16T07:19:42.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcc731700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 0x7f1fc40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcc731700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fc4068160 0x7f1fc40685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcc731700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc40c94d0 con 0x7f1fc4154090 2023-05-16T07:19:42.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcc731700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc40cb7e0 con 0x7f1fc4068160 2023-05-16T07:19:42.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcc731700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4068ac0 con 0x7f1fc4154470 2023-05-16T07:19:42.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcacce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fc4068160 0x7f1fc40685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fca4cd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1fc4154090 0x7f1fc40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60532/0 (socket says 172.21.15.92:60532) 2023-05-16T07:19:42.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fca4cd700 1 -- 172.21.15.92:0/1947414650 learned_addr learned my addr 172.21.15.92:0/1947414650 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:42.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcacce700 1 -- 172.21.15.92:0/1947414650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 msgr2=0x7f1fc40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:42.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcacce700 1 --2- 172.21.15.92:0/1947414650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 0x7f1fc40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:42.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcacce700 1 -- 172.21.15.92:0/1947414650 >> v1:172.21.15.92:6789/0 conn(0x7f1fc4154090 legacy=0x7f1fc40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fc94cb700 1 -- 172.21.15.92:0/1947414650 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951162343 0 0) 0x7f1fc40c94d0 con 0x7f1fc4154090 2023-05-16T07:19:42.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcacce700 1 -- 172.21.15.92:0/1947414650 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc40c88c0 con 0x7f1fc4068160 2023-05-16T07:19:42.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fcacce700 1 --2- 172.21.15.92:0/1947414650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4068160 0x7f1fc40685d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f1fc0009760 tx=0x7f1fc0009060).ready entity=mon.0 client_cookie=642e6e4f37dffed7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fc94cb700 1 -- 172.21.15.92:0/1947414650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000b580 con 0x7f1fc4068160 2023-05-16T07:19:42.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fc94cb700 1 -- 172.21.15.92:0/1947414650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc000b6e0 con 0x7f1fc4068160 2023-05-16T07:19:42.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.122+0000 7f1fc94cb700 1 -- 172.21.15.92:0/1947414650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000ba60 con 0x7f1fc4068160 2023-05-16T07:19:42.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 -- 172.21.15.92:0/1947414650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4068160 msgr2=0x7f1fc40685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1947414650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4068160 0x7f1fc40685d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f1fc0009760 tx=0x7f1fc0009060).stop 2023-05-16T07:19:42.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 -- 172.21.15.92:0/1947414650 shutdown_connections 2023-05-16T07:19:42.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1947414650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4068160 0x7f1fc40685d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1947414650 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 0x7f1fc40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:42.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 -- 172.21.15.92:0/1947414650 >> 172.21.15.92:0/1947414650 conn(0x7f1fc4067920 msgr2=0x7f1fc40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:42.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 -- 172.21.15.92:0/1947414650 shutdown_connections 2023-05-16T07:19:42.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.123+0000 7f1fcc731700 1 -- 172.21.15.92:0/1947414650 wait complete. 2023-05-16T07:19:42.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.124+0000 7f1fcc731700 1 Processor -- start 2023-05-16T07:19:42.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.124+0000 7f1fcc731700 1 -- start start 2023-05-16T07:19:42.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fc4068160 0x7f1fc40ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 0x7f1fc40ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 0x7f1fc4164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40ace40 con 0x7f1fc4154090 2023-05-16T07:19:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41568c0 con 0x7f1fc4068160 2023-05-16T07:19:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4152e00 con 0x7f1fc4154470 2023-05-16T07:19:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 0x7f1fc40ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 0x7f1fc40ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49792/0 (socket says 172.21.15.92:49792) 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 -- 172.21.15.92:0/1457248001 learned_addr learned my addr 172.21.15.92:0/1457248001 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 -- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 msgr2=0x7f1fc4164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 0x7f1fc4164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fca4cd700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fc4068160 0x7f1fc40ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 -- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fc4068160 msgr2=0x7f1fc40ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fc4068160 0x7f1fc40ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 -- 172.21.15.92:0/1457248001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00092a0 con 0x7f1fc4154090 2023-05-16T07:19:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fc9ccc700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 0x7f1fc40ac950 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f1fb400b490 tx=0x7f1fb4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4002320 con 0x7f1fc4154090 2023-05-16T07:19:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4002480 con 0x7f1fc4154090 2023-05-16T07:19:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc4155cb0 con 0x7f1fc4154090 2023-05-16T07:19:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4002840 con 0x7f1fc4154090 2023-05-16T07:19:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.125+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc4156270 con 0x7f1fc4154090 2023-05-16T07:19:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.126+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc4066730 con 0x7f1fc4154090 2023-05-16T07:19:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.127+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1fb400f030 con 0x7f1fc4154090 2023-05-16T07:19:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.127+0000 7f1fba7fc700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fa805ed60 0x7f1fa8061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.127+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f1fb40798d0 con 0x7f1fc4154090 2023-05-16T07:19:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.127+0000 7f1fca4cd700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fa805ed60 0x7f1fa8061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.128+0000 7f1fca4cd700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fa805ed60 0x7f1fa8061220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1fc4157110 tx=0x7f1fbc009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.131+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1fb404ad80 con 0x7f1fc4154090 2023-05-16T07:19:42.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.295+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f1fc40c8ba0 con 0x7f1fc4154090 2023-05-16T07:19:42.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.298+0000 7f1fba7fc700 1 -- 172.21.15.92:0/1457248001 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v25) v1 ==== 120+0+0 (secure 0 0 0) 0x7f1fb404e3a0 con 0x7f1fc4154090 2023-05-16T07:19:42.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fa805ed60 msgr2=0x7f1fa8061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fa805ed60 0x7f1fa8061220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1fc4157110 tx=0x7f1fbc009040).stop 2023-05-16T07:19:42.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 msgr2=0x7f1fc40ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 0x7f1fc40ac950 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f1fb400b490 tx=0x7f1fb4009060).stop 2023-05-16T07:19:42.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 shutdown_connections 2023-05-16T07:19:42.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1fa805ed60 0x7f1fa8061220 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fc4154470 0x7f1fc4164060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:42.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fc4068160 0x7f1fc40ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 --2- 172.21.15.92:0/1457248001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc4154090 0x7f1fc40ac950 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 >> 172.21.15.92:0/1457248001 conn(0x7f1fc4067920 msgr2=0x7f1fc40bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 shutdown_connections 2023-05-16T07:19:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.300+0000 7f1fcc731700 1 -- 172.21.15.92:0/1457248001 wait complete. 2023-05-16T07:19:42.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-05-16T07:19:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.422+0000 7f57f8bf9700 1 Processor -- start 2023-05-16T07:19:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.422+0000 7f57f8bf9700 1 -- start start 2023-05-16T07:19:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f8bf9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f57f40c7380 0x7f57f4154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f8bf9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7760 0x7f57f40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f8bf9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57f40ad040 0x7f57f414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f8bf9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f4158970 con 0x7f57f40ad040 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f8bf9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f4155340 con 0x7f57f40c7380 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f8bf9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4157d60 con 0x7f57f40c7760 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57f40ad040 0x7f57f414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57f40ad040 0x7f57f414dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49800/0 (socket says 172.21.15.92:49800) 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f3fff700 1 -- 172.21.15.92:0/952220690 learned_addr learned my addr 172.21.15.92:0/952220690 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f37fe700 1 --2- 172.21.15.92:0/952220690 >> v2:172.21.15.158:3300/0 conn(0x7f57f40c7380 0x7f57f4154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f2ffd700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7760 0x7f57f40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.423+0000 7f57f3fff700 1 -- 172.21.15.92:0/952220690 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7760 msgr2=0x7f57f40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f3fff700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7760 0x7f57f40acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f3fff700 1 -- 172.21.15.92:0/952220690 >> v2:172.21.15.158:3300/0 conn(0x7f57f40c7380 msgr2=0x7f57f4154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f3fff700 1 --2- 172.21.15.92:0/952220690 >> v2:172.21.15.158:3300/0 conn(0x7f57f40c7380 0x7f57f4154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f3fff700 1 -- 172.21.15.92:0/952220690 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4158040 con 0x7f57f40ad040 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f37fe700 1 --2- 172.21.15.92:0/952220690 >> v2:172.21.15.158:3300/0 conn(0x7f57f40c7380 0x7f57f4154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f2ffd700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7760 0x7f57f40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f3fff700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40ad040 0x7f57f414dd70 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f57f4083ba0 tx=0x7f57ec0090f0).ready entity=mon.0 client_cookie=3ad7243299b74f66 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f27fc700 1 -- 172.21.15.92:0/952220690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00b600 con 0x7f57f40ad040 2023-05-16T07:19:42.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f27fc700 1 -- 172.21.15.92:0/952220690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec00b760 con 0x7f57f40ad040 2023-05-16T07:19:42.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f27fc700 1 -- 172.21.15.92:0/952220690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00bae0 con 0x7f57f40ad040 2023-05-16T07:19:42.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 -- 172.21.15.92:0/952220690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40ad040 msgr2=0x7f57f414dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40ad040 0x7f57f414dd70 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f57f4083ba0 tx=0x7f57ec0090f0).stop 2023-05-16T07:19:42.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 -- 172.21.15.92:0/952220690 shutdown_connections 2023-05-16T07:19:42.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40ad040 0x7f57f414dd70 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/952220690 >> v2:172.21.15.158:3300/0 conn(0x7f57f40c7380 0x7f57f4154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/952220690 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7760 0x7f57f40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 -- 172.21.15.92:0/952220690 >> 172.21.15.92:0/952220690 conn(0x7f57f4068330 msgr2=0x7f57f4068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 -- 172.21.15.92:0/952220690 shutdown_connections 2023-05-16T07:19:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.424+0000 7f57f8bf9700 1 -- 172.21.15.92:0/952220690 wait complete. 2023-05-16T07:19:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 Processor -- start 2023-05-16T07:19:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 -- start start 2023-05-16T07:19:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57f40ad040 0x7f57f40c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7380 0x7f57f4163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 0x7f57f40c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ca1f0 con 0x7f57f40c7760 2023-05-16T07:19:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4164fe0 con 0x7f57f40ad040 2023-05-16T07:19:42.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f8bf9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4164a20 con 0x7f57f40c7380 2023-05-16T07:19:42.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.426+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 0x7f57f40c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 0x7f57f40c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49808/0 (socket says 172.21.15.92:49808) 2023-05-16T07:19:42.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 -- 172.21.15.92:0/2341481351 learned_addr learned my addr 172.21.15.92:0/2341481351 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:42.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f2ffd700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7380 0x7f57f4163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f37fe700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57f40ad040 0x7f57f40c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 -- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7380 msgr2=0x7f57f4163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7380 0x7f57f4163ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 -- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57f40ad040 msgr2=0x7f57f40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57f40ad040 0x7f57f40c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 -- 172.21.15.92:0/2341481351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009370 con 0x7f57f40c7760 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f2ffd700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7380 0x7f57f4163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f3fff700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 0x7f57f40c4070 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f57f4083bd0 tx=0x7f57ec017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00bae0 con 0x7f57f40c7760 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec01c420 con 0x7f57f40c7760 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f4164d00 con 0x7f57f40c7760 2023-05-16T07:19:42.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec01c7a0 con 0x7f57f40c7760 2023-05-16T07:19:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.427+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f40c2000 con 0x7f57f40c7760 2023-05-16T07:19:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.428+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f40c2220 con 0x7f57f40c7760 2023-05-16T07:19:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.428+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f57ec00f070 con 0x7f57f40c7760 2023-05-16T07:19:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.429+0000 7f57db7fe700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57e005ed90 0x7f57e0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.429+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f57ec07b540 con 0x7f57f40c7760 2023-05-16T07:19:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.429+0000 7f57f37fe700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57e005ed90 0x7f57e0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.429+0000 7f57f37fe700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57e005ed90 0x7f57e0061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f57f4167570 tx=0x7f57e4007850).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.433+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f57ec04c9f0 con 0x7f57f40c7760 2023-05-16T07:19:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.661+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f57f4047150 con 0x7f57f40c7760 2023-05-16T07:19:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.671+0000 7f57db7fe700 1 -- 172.21.15.92:0/2341481351 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v646) v1 ==== 174+0+0 (secure 0 0 0) 0x7f57ec050010 con 0x7f57f40c7760 2023-05-16T07:19:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57e005ed90 msgr2=0x7f57e0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57e005ed90 0x7f57e0061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f57f4167570 tx=0x7f57e4007850).stop 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 msgr2=0x7f57f40c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 0x7f57f40c4070 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f57f4083bd0 tx=0x7f57ec017000).stop 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 shutdown_connections 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f57e005ed90 0x7f57e0061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57f40c7760 0x7f57f40c4070 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f57f40ad040 0x7f57f40c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 --2- 172.21.15.92:0/2341481351 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f57f40c7380 0x7f57f4163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 >> 172.21.15.92:0/2341481351 conn(0x7f57f4068330 msgr2=0x7f57f40c1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:42.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 shutdown_connections 2023-05-16T07:19:42.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.673+0000 7f57f8bf9700 1 -- 172.21.15.92:0/2341481351 wait complete. 2023-05-16T07:19:42.688 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-05-16T07:19:42.688 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-05-16T07:19:42.689 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:19:42.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-05-16T07:19:42.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.797+0000 7f19f2e1a700 1 Processor -- start 2023-05-16T07:19:42.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.797+0000 7f19f2e1a700 1 -- start start 2023-05-16T07:19:42.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.798+0000 7f19f2e1a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19ec0c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.798+0000 7f19f2e1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0bc5f0 con 0x7f19ec151160 2023-05-16T07:19:42.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.798+0000 7f19f2e1a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec066670 con 0x7f19ec150d80 2023-05-16T07:19:42.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.798+0000 7f19f2e1a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0bbfe0 con 0x7f19ec0c1550 2023-05-16T07:19:42.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.798+0000 7f19f2619700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f19ec151160 0x7f19ec1515c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60564/0 (socket says 172.21.15.92:60564) 2023-05-16T07:19:42.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.798+0000 7f19f2619700 1 -- 172.21.15.92:0/1727219939 learned_addr learned my addr 172.21.15.92:0/1727219939 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:42.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845931589 0 0) 0x7f19ec0bc5f0 con 0x7f19ec151160 2023-05-16T07:19:42.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19ec0c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d40033f0 con 0x7f19ec151160 2023-05-16T07:19:42.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260149240 0 0) 0x7f19d40033f0 con 0x7f19ec151160 2023-05-16T07:19:42.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 msgr2=0x7f19ec0c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:42.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19ec0c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 >> v1:172.21.15.158:6789/0 conn(0x7f19ec150d80 legacy=0x7f19ec065bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec066390 con 0x7f19ec151160 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f19e4001470 con 0x7f19ec151160 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 >> v1:172.21.15.92:6789/0 conn(0x7f19ec151160 legacy=0x7f19ec1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d4003910 0x7f19d4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19d4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19d4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0bbfe0 con 0x7f19d4004690 2023-05-16T07:19:42.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d400b910 con 0x7f19d4003910 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d400bad0 con 0x7f19ec0c1550 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19d4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1e18700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d4003910 0x7f19d4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f2619700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19d4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 -- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 msgr2=0x7f19d4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19d4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 -- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d4003910 msgr2=0x7f19d4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19d4003910 0x7f19d4003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 -- 172.21.15.92:0/1727219939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40033f0 con 0x7f19d4004690 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f2619700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec0c1550 0x7f19d4005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f1617700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19d4004b10 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f19ec0c1970 tx=0x7f19e0009060).ready entity=mon.0 client_cookie=f08736feb83de9ec server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.799+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0009e90 con 0x7f19d4004690 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e0009430 con 0x7f19d4004690 2023-05-16T07:19:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f0e16700 1 -- 172.21.15.92:0/1727219939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000e850 con 0x7f19d4004690 2023-05-16T07:19:42.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1727219939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 msgr2=0x7f19d4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1727219939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19d4004b10 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f19ec0c1970 tx=0x7f19e0009060).stop 2023-05-16T07:19:42.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2619700 1 -- 172.21.15.92:0/1727219939 reap_dead start 2023-05-16T07:19:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1727219939 shutdown_connections 2023-05-16T07:19:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1727219939 >> 172.21.15.92:0/1727219939 conn(0x7f19ec069940 msgr2=0x7f19ec0c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1727219939 shutdown_connections 2023-05-16T07:19:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.800+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1727219939 wait complete. 2023-05-16T07:19:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.801+0000 7f19f2e1a700 1 Processor -- start 2023-05-16T07:19:42.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.801+0000 7f19f2e1a700 1 -- start start 2023-05-16T07:19:42.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f2e1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19ec164df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f2e1a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec14b9a0 0x7f19ec0ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f2e1a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19ec14d290 0x7f19ec14d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f2e1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0a9fa0 con 0x7f19d4004690 2023-05-16T07:19:42.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f2e1a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0aa210 con 0x7f19ec14d290 2023-05-16T07:19:42.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f2e1a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0aa4f0 con 0x7f19ec14b9a0 2023-05-16T07:19:42.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19ec164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19ec164df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49824/0 (socket says 172.21.15.92:49824) 2023-05-16T07:19:42.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 -- 172.21.15.92:0/1234534186 learned_addr learned my addr 172.21.15.92:0/1234534186 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1617700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec14b9a0 0x7f19ec0ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 -- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec14b9a0 msgr2=0x7f19ec0ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec14b9a0 0x7f19ec0ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 -- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19ec14d290 msgr2=0x7f19ec14d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19ec14d290 0x7f19ec14d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 -- 172.21.15.92:0/1234534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0aa7d0 con 0x7f19d4004690 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1617700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec14b9a0 0x7f19ec0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19f1e18700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19ec164df0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f19dc00e940 tx=0x7f19dc007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.802+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00eee0 con 0x7f19d4004690 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.803+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00c7e0 con 0x7f19d4004690 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.803+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0aaab0 con 0x7f19d4004690 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.803+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00cb60 con 0x7f19d4004690 2023-05-16T07:19:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.803+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec14be20 con 0x7f19d4004690 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.804+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec047150 con 0x7f19d4004690 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.804+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f19dc01d460 con 0x7f19d4004690 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.804+0000 7f19e9ffb700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19d805ed90 0x7f19d8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.804+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f19dc079fd0 con 0x7f19d4004690 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.805+0000 7f19f1617700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19d805ed90 0x7f19d8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.805+0000 7f19f1617700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19d805ed90 0x7f19d8061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f19ec14c220 tx=0x7f19e001a000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:42.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:42.808+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f19dc04b420 con 0x7f19d4004690 2023-05-16T07:19:43.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.036+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f19ec0aae20 con 0x7f19d4004690 2023-05-16T07:19:43.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.671+0000 7f19e9ffb700 1 -- 172.21.15.92:0/1234534186 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v647) v1 ==== 168+0+0 (secure 0 0 0) 0x7f19dc04ea40 con 0x7f19d4004690 2023-05-16T07:19:43.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.673+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19d805ed90 msgr2=0x7f19d8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.673+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19d805ed90 0x7f19d8061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f19ec14c220 tx=0x7f19e001a000).stop 2023-05-16T07:19:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.673+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 msgr2=0x7f19ec164df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.673+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19ec164df0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f19dc00e940 tx=0x7f19dc007f30).stop 2023-05-16T07:19:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 shutdown_connections 2023-05-16T07:19:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f19d805ed90 0x7f19d8061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f19ec14d290 0x7f19ec14d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d4004690 0x7f19ec164df0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 --2- 172.21.15.92:0/1234534186 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f19ec14b9a0 0x7f19ec0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 >> 172.21.15.92:0/1234534186 conn(0x7f19ec069940 msgr2=0x7f19ec0c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 shutdown_connections 2023-05-16T07:19:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.674+0000 7f19f2e1a700 1 -- 172.21.15.92:0/1234534186 wait complete. 2023-05-16T07:19:43.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:19:43.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.802+0000 7f8dbb0d7700 1 Processor -- start 2023-05-16T07:19:43.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.802+0000 7f8dbb0d7700 1 -- start start 2023-05-16T07:19:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8dbb0d7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8dbb0d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8db4068160 0x7f8db40685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:43.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8dbb0d7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40c94d0 con 0x7f8db4154090 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8dbb0d7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40cb7e0 con 0x7f8db4068160 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8dbb0d7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4068ac0 con 0x7f8db4154470 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8db4068160 0x7f8db40685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8db4068160 0x7f8db40685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49836/0 (socket says 172.21.15.92:49836) 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 -- 172.21.15.92:0/3298668266 learned_addr learned my addr 172.21.15.92:0/3298668266 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 -- 172.21.15.92:0/3298668266 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 msgr2=0x7f8db40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:43.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 --2- 172.21.15.92:0/3298668266 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 -- 172.21.15.92:0/3298668266 >> v1:172.21.15.158:6789/0 conn(0x7f8db4154090 legacy=0x7f8db40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.803+0000 7f8db9674700 1 -- 172.21.15.92:0/3298668266 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40c88c0 con 0x7f8db4068160 2023-05-16T07:19:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8db9674700 1 --2- 172.21.15.92:0/3298668266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db40685d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f8dac009f20 tx=0x7f8dac009840).ready entity=mon.0 client_cookie=2590e40a0093b2b9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8db37fe700 1 -- 172.21.15.92:0/3298668266 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00e070 con 0x7f8db4068160 2023-05-16T07:19:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8db37fe700 1 -- 172.21.15.92:0/3298668266 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00ba40 con 0x7f8db4068160 2023-05-16T07:19:43.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8db37fe700 1 -- 172.21.15.92:0/3298668266 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00bdc0 con 0x7f8db4068160 2023-05-16T07:19:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/3298668266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 msgr2=0x7f8db40685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/3298668266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db40685d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f8dac009f20 tx=0x7f8dac009840).stop 2023-05-16T07:19:43.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/3298668266 shutdown_connections 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/3298668266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db40685d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/3298668266 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/3298668266 >> 172.21.15.92:0/3298668266 conn(0x7f8db4067920 msgr2=0x7f8db40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/3298668266 shutdown_connections 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.804+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/3298668266 wait complete. 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.805+0000 7f8dbb0d7700 1 Processor -- start 2023-05-16T07:19:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.805+0000 7f8dbb0d7700 1 -- start start 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8dbb0d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8dbb0d7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8db4154090 0x7f8db4157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8dbb0d7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db4158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8dbb0d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4158720 con 0x7f8db4068160 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8dbb0d7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c5150 con 0x7f8db4154090 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8dbb0d7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c4540 con 0x7f8db4154470 2023-05-16T07:19:43.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8db8e73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8db8e73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db414f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49838/0 (socket says 172.21.15.92:49838) 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8db8e73700 1 -- 172.21.15.92:0/1934064572 learned_addr learned my addr 172.21.15.92:0/1934064572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.806+0000 7f8db3fff700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8db4154090 0x7f8db4157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db9674700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db4158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db8e73700 1 -- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 msgr2=0x7f8db4158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db8e73700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db4158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db8e73700 1 -- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8db4154090 msgr2=0x7f8db4157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db8e73700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8db4154090 0x7f8db4157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db8e73700 1 -- 172.21.15.92:0/1934064572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac009ae0 con 0x7f8db4068160 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db9674700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db4158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db8e73700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db414f860 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f8da400c440 tx=0x7f8da4007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400cea0 con 0x7f8db4068160 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400f470 con 0x7f8db4068160 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db40c4820 con 0x7f8db4068160 2023-05-16T07:19:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400f810 con 0x7f8db4068160 2023-05-16T07:19:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.807+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db40c4de0 con 0x7f8db4068160 2023-05-16T07:19:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.808+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db40c5430 con 0x7f8db4068160 2023-05-16T07:19:43.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.808+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8da4019030 con 0x7f8db4068160 2023-05-16T07:19:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.808+0000 7f8db0ff9700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8da005ed90 0x7f8da0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.809+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f8da4078f20 con 0x7f8db4068160 2023-05-16T07:19:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.809+0000 7f8db3fff700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8da005ed90 0x7f8da0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.809+0000 7f8db3fff700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8da005ed90 0x7f8da0061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8db40ad940 tx=0x7f8da8006f10).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.813+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8da40488f0 con 0x7f8db4068160 2023-05-16T07:19:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.976+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8db4066730 con 0x7f8db4068160 2023-05-16T07:19:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.979+0000 7f8db0ff9700 1 -- 172.21.15.92:0/1934064572 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8da404a480 con 0x7f8db4068160 2023-05-16T07:19:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.981+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8da005ed90 msgr2=0x7f8da0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.981+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8da005ed90 0x7f8da0061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8db40ad940 tx=0x7f8da8006f10).stop 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.981+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 msgr2=0x7f8db414f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.981+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db414f860 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f8da400c440 tx=0x7f8da4007de0).stop 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 shutdown_connections 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f8da005ed90 0x7f8da0061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8db4154470 0x7f8db4158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db4068160 0x7f8db414f860 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 --2- 172.21.15.92:0/1934064572 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8db4154090 0x7f8db4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 >> 172.21.15.92:0/1934064572 conn(0x7f8db4067920 msgr2=0x7f8db40c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 shutdown_connections 2023-05-16T07:19:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:43.982+0000 7f8dbb0d7700 1 -- 172.21.15.92:0/1934064572 wait complete. 2023-05-16T07:19:43.997 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:19:43.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:44.024 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:19:44.057 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:19:44.057 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:19:44.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:44.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.171+0000 7f0615a65700 1 Processor -- start 2023-05-16T07:19:44.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.171+0000 7f0615a65700 1 -- start start 2023-05-16T07:19:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f0615a65700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f06100c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f0615a65700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f061013b930 0x7f06100abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f0615a65700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0610062c80 0x7f0610063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f0615a65700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100c5400 con 0x7f061013b930 2023-05-16T07:19:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f0615a65700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100bdc30 con 0x7f0610062c80 2023-05-16T07:19:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f0615a65700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610063130 con 0x7f061013b550 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f061013b930 0x7f06100abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f061013b930 0x7f06100abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49848/0 (socket says 172.21.15.92:49848) 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 -- 172.21.15.92:0/703071232 learned_addr learned my addr 172.21.15.92:0/703071232 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060ffff700 1 --2- 172.21.15.92:0/703071232 >> v2:172.21.15.158:3300/0 conn(0x7f0610062c80 0x7f0610063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060f7fe700 1 --2- 172.21.15.92:0/703071232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f06100c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 -- 172.21.15.92:0/703071232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 msgr2=0x7f06100c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 --2- 172.21.15.92:0/703071232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f06100c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 -- 172.21.15.92:0/703071232 >> v2:172.21.15.158:3300/0 conn(0x7f0610062c80 msgr2=0x7f0610063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 --2- 172.21.15.92:0/703071232 >> v2:172.21.15.158:3300/0 conn(0x7f0610062c80 0x7f0610063720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.172+0000 7f060effd700 1 -- 172.21.15.92:0/703071232 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100c4e40 con 0x7f061013b930 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f060ffff700 1 --2- 172.21.15.92:0/703071232 >> v2:172.21.15.158:3300/0 conn(0x7f0610062c80 0x7f0610063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f060effd700 1 --2- 172.21.15.92:0/703071232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f061013b930 0x7f06100abb20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f06100ab7a0 tx=0x7f06040090f0).ready entity=mon.0 client_cookie=66897ed27fa611c5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f060e7fc700 1 -- 172.21.15.92:0/703071232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400b600 con 0x7f061013b930 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f060e7fc700 1 -- 172.21.15.92:0/703071232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060400b760 con 0x7f061013b930 2023-05-16T07:19:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f060e7fc700 1 -- 172.21.15.92:0/703071232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400bae0 con 0x7f061013b930 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 -- 172.21.15.92:0/703071232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f061013b930 msgr2=0x7f06100abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 --2- 172.21.15.92:0/703071232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f061013b930 0x7f06100abb20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f06100ab7a0 tx=0x7f06040090f0).stop 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 -- 172.21.15.92:0/703071232 shutdown_connections 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 --2- 172.21.15.92:0/703071232 >> v2:172.21.15.158:3300/0 conn(0x7f0610062c80 0x7f0610063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 --2- 172.21.15.92:0/703071232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f06100c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 --2- 172.21.15.92:0/703071232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f061013b930 0x7f06100abb20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 -- 172.21.15.92:0/703071232 >> 172.21.15.92:0/703071232 conn(0x7f06100c8aa0 msgr2=0x7f06100c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 -- 172.21.15.92:0/703071232 shutdown_connections 2023-05-16T07:19:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.173+0000 7f0615a65700 1 -- 172.21.15.92:0/703071232 wait complete. 2023-05-16T07:19:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 Processor -- start 2023-05-16T07:19:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 -- start start 2023-05-16T07:19:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 0x7f061014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f0610161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061013b930 0x7f06101622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610157a30 con 0x7f0610062c80 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610157d10 con 0x7f061013b930 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f0615a65700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610157ff0 con 0x7f061013b550 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f060f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 0x7f061014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f060f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 0x7f061014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49852/0 (socket says 172.21.15.92:49852) 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.175+0000 7f060f7fe700 1 -- 172.21.15.92:0/532118591 learned_addr learned my addr 172.21.15.92:0/532118591 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060f7fe700 1 -- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 msgr2=0x7f0610161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060f7fe700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f0610161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060f7fe700 1 -- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061013b930 msgr2=0x7f06101622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060ffff700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061013b930 0x7f06101622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060f7fe700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061013b930 0x7f06101622d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060f7fe700 1 -- 172.21.15.92:0/532118591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604009370 con 0x7f0610062c80 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f060f7fe700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 0x7f061014ab60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f060000c880 tx=0x7f0600008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600010580 con 0x7f0610062c80 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06000106e0 con 0x7f0610062c80 2023-05-16T07:19:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600010a60 con 0x7f0610062c80 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06101582d0 con 0x7f0610062c80 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.176+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0610157630 con 0x7f0610062c80 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.177+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061006c350 con 0x7f0610062c80 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.177+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f060001b460 con 0x7f0610062c80 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.178+0000 7f05f77fe700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05fc05ed90 0x7f05fc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.178+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f0600079d30 con 0x7f0610062c80 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.178+0000 7f060effd700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05fc05ed90 0x7f05fc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.178+0000 7f060effd700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05fc05ed90 0x7f05fc061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f06100ab7a0 tx=0x7f0604017000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:44.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.182+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f060004b180 con 0x7f0610062c80 2023-05-16T07:19:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.383+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06100c5120 con 0x7f0610062c80 2023-05-16T07:19:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.383+0000 7f05f77fe700 1 -- 172.21.15.92:0/532118591 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f060004e7a0 con 0x7f0610062c80 2023-05-16T07:19:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:44.388 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/100916294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":100916294},{"type":"v1","addr":"172.21.15.158:6837","nonce":100916294}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6952,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/3950936830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":3950936830},{"type":"v1","addr":"172.21.15.158:6836","nonce":3950936830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1785543098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1785543098},{"type":"v1","addr":"172.21.15.92:6833","nonce":1785543098}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3177054946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3177054946},{"type":"v1","addr":"172.21.15.92:6835","nonce":3177054946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[]} 2023-05-16T07:19:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05fc05ed90 msgr2=0x7f05fc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05fc05ed90 0x7f05fc061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f06100ab7a0 tx=0x7f0604017000).stop 2023-05-16T07:19:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 msgr2=0x7f061014ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 0x7f061014ab60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f060000c880 tx=0x7f0600008040).stop 2023-05-16T07:19:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 shutdown_connections 2023-05-16T07:19:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f05fc05ed90 0x7f05fc061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061013b930 0x7f06101622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0610062c80 0x7f061014ab60 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 --2- 172.21.15.92:0/532118591 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f061013b550 0x7f0610161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:44.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 >> 172.21.15.92:0/532118591 conn(0x7f06100c8aa0 msgr2=0x7f06100c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:44.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 shutdown_connections 2023-05-16T07:19:44.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.385+0000 7f0615a65700 1 -- 172.21.15.92:0/532118591 wait complete. 2023-05-16T07:19:44.392 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 45 2023-05-16T07:19:44.401 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:19:44.401 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:19:44.402 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:19:44.402 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:44.402 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:19:44.402 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:19:44.402 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:19:44.403 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:44.403 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:19:44.403 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:19:44.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:19:44.404 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:44.404 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:19:44.404 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:19:44.404 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:19:44.405 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:19:44.410 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:19:44.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:19:44.411 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:19:44.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:19:44.414 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:19:44.414 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:19:44.415 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:19:44.415 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:19:44.452 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:19:44.453 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:44.454 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:19:44.454 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:44.456 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:19:44.457 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:44.457 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:19:44.458 DEBUG:teuthology.parallel:result is None 2023-05-16T07:19:44.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:19:44.523 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:19:44.524 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:19:44.524 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:19:44.520+0000 7f205faa5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-05-16T07:19:44.535 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:19:44.535 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:19:44.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.574+0000 7fc60affd700 1 Processor -- start 2023-05-16T07:19:44.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.574+0000 7fc60affd700 1 -- start start 2023-05-16T07:19:44.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc60affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604069750 0x7fc604069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc60affd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60406a920 con 0x7fc604153b00 2023-05-16T07:19:44.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc60affd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6040bfa30 con 0x7fc6040aab40 2023-05-16T07:19:44.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc60affd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040c65f0 con 0x7fc604069750 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc60a7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc604153b00 0x7fc604153f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60636/0 (socket says 172.21.15.92:60636) 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc60a7fc700 1 -- 172.21.15.92:0/1305423089 learned_addr learned my addr 172.21.15.92:0/1305423089 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc609ffb700 1 --2- 172.21.15.92:0/1305423089 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604069750 0x7fc604069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188304527 0 0) 0x7fc60406a920 con 0x7fc604153b00 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f00033f0 con 0x7fc604153b00 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037406798 0 0) 0x7fc6040bfa30 con 0x7fc6040aab40 2023-05-16T07:19:44.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc60406a920 con 0x7fc6040aab40 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc609ffb700 1 -- 172.21.15.92:0/1305423089 >> v1:172.21.15.158:6789/0 conn(0x7fc6040aab40 legacy=0x7fc6040c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222842649 0 0) 0x7fc5f00033f0 con 0x7fc604153b00 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc609ffb700 1 -- 172.21.15.92:0/1305423089 >> v1:172.21.15.92:6789/0 conn(0x7fc604153b00 legacy=0x7fc604153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.575+0000 7fc609ffb700 1 -- 172.21.15.92:0/1305423089 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6040bfc80 con 0x7fc604069750 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc609ffb700 1 --2- 172.21.15.92:0/1305423089 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604069750 0x7fc604069b30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc600009790 tx=0x7fc600009060).ready entity=mon.2 client_cookie=4a3ef8712f56f66b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0017f0 con 0x7fc604069750 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600011400 con 0x7fc604069750 2023-05-16T07:19:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc608ff9700 1 -- 172.21.15.92:0/1305423089 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600011780 con 0x7fc604069750 2023-05-16T07:19:44.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 -- 172.21.15.92:0/1305423089 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604069750 msgr2=0x7fc604069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 --2- 172.21.15.92:0/1305423089 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604069750 0x7fc604069b30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc600009790 tx=0x7fc600009060).stop 2023-05-16T07:19:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 -- 172.21.15.92:0/1305423089 shutdown_connections 2023-05-16T07:19:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 --2- 172.21.15.92:0/1305423089 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604069750 0x7fc604069b30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 -- 172.21.15.92:0/1305423089 >> 172.21.15.92:0/1305423089 conn(0x7fc6040cb550 msgr2=0x7fc6040c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 -- 172.21.15.92:0/1305423089 shutdown_connections 2023-05-16T07:19:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.576+0000 7fc60affd700 1 -- 172.21.15.92:0/1305423089 wait complete. 2023-05-16T07:19:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.577+0000 7fc60affd700 1 Processor -- start 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.577+0000 7fc60affd700 1 -- start start 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 0x7fc604156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60affd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6040aab40 0x7fc60414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604153b00 0x7fc6040ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60affd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc604163f10 con 0x7fc604069750 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60affd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc604157bb0 con 0x7fc6040aab40 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60affd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc604164740 con 0x7fc604153b00 2023-05-16T07:19:44.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 0x7fc604156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 0x7fc604156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49894/0 (socket says 172.21.15.92:49894) 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 -- 172.21.15.92:0/399874395 learned_addr learned my addr 172.21.15.92:0/399874395 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc60a7fc700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604153b00 0x7fc6040ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc6097fa700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6040aab40 0x7fc60414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 -- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604153b00 msgr2=0x7fc6040ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604153b00 0x7fc6040ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 -- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6040aab40 msgr2=0x7fc60414f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6040aab40 0x7fc60414f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.578+0000 7fc609ffb700 1 -- 172.21.15.92:0/399874395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000092a0 con 0x7fc604069750 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc60a7fc700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604153b00 0x7fc6040ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc6097fa700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6040aab40 0x7fc60414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc609ffb700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 0x7fc604156a60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fc600009d80 tx=0x7fc600016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600010920 con 0x7fc604069750 2023-05-16T07:19:44.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6040aa240 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600010a80 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600010e00 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.579+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6041640f0 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.580+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc604083040 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.580+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc60000f070 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.580+0000 7fc5f9ffb700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5e805edc0 0x7fc5e8061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.580+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fc60007b1e0 con 0x7fc604069750 2023-05-16T07:19:44.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.581+0000 7fc6097fa700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5e805edc0 0x7fc5e8061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:44.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.581+0000 7fc6097fa700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5e805edc0 0x7fc5e8061280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc5f4007fd0 tx=0x7fc5f4009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:44.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.584+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc60004c690 con 0x7fc604069750 2023-05-16T07:19:44.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-05-16T07:19:44.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-05-16T07:19:44.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:44.765+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc604047150 con 0x7fc604069750 2023-05-16T07:19:45.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.678+0000 7fc5f9ffb700 1 -- 172.21.15.92:0/399874395 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc60004fcb0 con 0x7fc604069750 2023-05-16T07:19:45.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5e805edc0 msgr2=0x7fc5e8061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:45.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5e805edc0 0x7fc5e8061280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc5f4007fd0 tx=0x7fc5f4009040).stop 2023-05-16T07:19:45.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 msgr2=0x7fc604156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:45.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 0x7fc604156a60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fc600009d80 tx=0x7fc600016000).stop 2023-05-16T07:19:45.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 shutdown_connections 2023-05-16T07:19:45.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc5e805edc0 0x7fc5e8061280 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc604153b00 0x7fc6040ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc604069750 0x7fc604156a60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 --2- 172.21.15.92:0/399874395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6040aab40 0x7fc60414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 >> 172.21.15.92:0/399874395 conn(0x7fc6040cb550 msgr2=0x7fc604159f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:45.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 shutdown_connections 2023-05-16T07:19:45.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.681+0000 7fc60affd700 1 -- 172.21.15.92:0/399874395 wait complete. 2023-05-16T07:19:45.689 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:19:45.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:19:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.813+0000 7f9ebce98700 1 Processor -- start 2023-05-16T07:19:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.813+0000 7f9ebce98700 1 -- start start 2023-05-16T07:19:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9ebce98700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9eb81539d0 0x7f9eb80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9ebce98700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb80c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9ebce98700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb8069fa0 con 0x7f9eb8153db0 2023-05-16T07:19:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9ebce98700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb8063fe0 con 0x7f9eb81539d0 2023-05-16T07:19:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9ebce98700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8069220 con 0x7f9eb8069b10 2023-05-16T07:19:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9eb7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb80c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9eb77fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9eb81539d0 0x7f9eb80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9eb7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb80c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:42478/0 (socket says 172.21.15.92:42478) 2023-05-16T07:19:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9eb77fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9eb81539d0 0x7f9eb80c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:53538/0 (socket says 172.21.15.92:53538) 2023-05-16T07:19:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9eb7fff700 1 -- 172.21.15.92:0/2018902051 learned_addr learned my addr 172.21.15.92:0/2018902051 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.814+0000 7f9eaffff700 1 --1- 172.21.15.92:0/2018902051 >> v1:172.21.15.158:6789/0 conn(0x7f9eb8153db0 0x7f9eb80c2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:36018/0 (socket says 172.21.15.92:36018) 2023-05-16T07:19:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb77fe700 1 -- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 msgr2=0x7f9eb80c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb77fe700 1 --2- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb80c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb77fe700 1 -- 172.21.15.92:0/2018902051 >> v1:172.21.15.158:6789/0 conn(0x7f9eb8153db0 legacy=0x7f9eb80c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb6ffd700 1 -- 172.21.15.92:0/2018902051 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380892970 0 0) 0x7f9eb8069fa0 con 0x7f9eb8153db0 2023-05-16T07:19:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb77fe700 1 -- 172.21.15.92:0/2018902051 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8064260 con 0x7f9eb81539d0 2023-05-16T07:19:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb7fff700 1 --2- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb80c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:45.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb77fe700 1 --2- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80c6500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9ea8006fd0 tx=0x7f9ea8006f40).ready entity=mon.1 client_cookie=ef839b6914956f2b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:45.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb6ffd700 1 -- 172.21.15.92:0/2018902051 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8005880 con 0x7f9eb81539d0 2023-05-16T07:19:45.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb6ffd700 1 -- 172.21.15.92:0/2018902051 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea80059e0 con 0x7f9eb81539d0 2023-05-16T07:19:45.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9eb6ffd700 1 -- 172.21.15.92:0/2018902051 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8005d60 con 0x7f9eb81539d0 2023-05-16T07:19:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 -- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 msgr2=0x7f9eb80c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 --2- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80c6500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9ea8006fd0 tx=0x7f9ea8006f40).stop 2023-05-16T07:19:45.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 -- 172.21.15.92:0/2018902051 shutdown_connections 2023-05-16T07:19:45.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 --2- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80c6500 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 --2- 172.21.15.92:0/2018902051 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb80c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 -- 172.21.15.92:0/2018902051 >> 172.21.15.92:0/2018902051 conn(0x7f9eb805e820 msgr2=0x7f9eb80622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:45.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 -- 172.21.15.92:0/2018902051 shutdown_connections 2023-05-16T07:19:45.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.815+0000 7f9ebce98700 1 -- 172.21.15.92:0/2018902051 wait complete. 2023-05-16T07:19:45.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.816+0000 7f9ebce98700 1 Processor -- start 2023-05-16T07:19:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 -- start start 2023-05-16T07:19:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 0x7f9eb81558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb814da40 con 0x7f9eb8153db0 2023-05-16T07:19:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80a4c60 con 0x7f9eb81539d0 2023-05-16T07:19:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9ebce98700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80a4f40 con 0x7f9eb8069b10 2023-05-16T07:19:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9eb7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 0x7f9eb81558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9eb7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 0x7f9eb81558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49912/0 (socket says 172.21.15.92:49912) 2023-05-16T07:19:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.817+0000 7f9eb7fff700 1 -- 172.21.15.92:0/3085571412 learned_addr learned my addr 172.21.15.92:0/3085571412 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb77fe700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:45.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eaffff700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:45.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb7fff700 1 -- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 msgr2=0x7f9eb814d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:45.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb7fff700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb814d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb7fff700 1 -- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 msgr2=0x7f9eb80a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb7fff700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb7fff700 1 -- 172.21.15.92:0/3085571412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0009ae0 con 0x7f9eb8153db0 2023-05-16T07:19:45.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eb7fff700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 0x7f9eb81558c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f9eb0009f30 tx=0x7f9eb000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00092e0 con 0x7f9eb8153db0 2023-05-16T07:19:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb000f040 con 0x7f9eb8153db0 2023-05-16T07:19:45.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea8009250 con 0x7f9eb8153db0 2023-05-16T07:19:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000cb70 con 0x7f9eb8153db0 2023-05-16T07:19:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.818+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb80a5500 con 0x7f9eb8153db0 2023-05-16T07:19:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.819+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb8062c50 con 0x7f9eb8153db0 2023-05-16T07:19:45.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.819+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9eb001b030 con 0x7f9eb8153db0 2023-05-16T07:19:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.820+0000 7f9eaf7fe700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e9805ed90 0x7f9e98061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.820+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9eb0079770 con 0x7f9eb8153db0 2023-05-16T07:19:45.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.820+0000 7f9eb77fe700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e9805ed90 0x7f9e98061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:45.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.821+0000 7f9eb77fe700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e9805ed90 0x7f9e98061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9ea8006fa0 tx=0x7f9ea8016000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:45.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:45.824+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9eb004aeb0 con 0x7f9eb8153db0 2023-05-16T07:19:46.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.049+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9eb8047150 con 0x7f9eb8153db0 2023-05-16T07:19:46.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.050+0000 7f9eaf7fe700 1 -- 172.21.15.92:0/3085571412 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f9eb004e4d0 con 0x7f9eb8153db0 2023-05-16T07:19:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:46.054 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:19:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:19:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:19:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:19:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:19:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:19:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:19:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:19:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:19:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:19:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:19:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:19:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:19:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:19:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:19:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:19:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:19:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:19:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:19:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:19:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:19:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:19:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:19:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:19:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:19:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:19:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:19:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:19:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:19:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:19:46.082 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:19:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e9805ed90 msgr2=0x7f9e98061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e9805ed90 0x7f9e98061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9ea8006fa0 tx=0x7f9ea8016000).stop 2023-05-16T07:19:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 msgr2=0x7f9eb81558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 0x7f9eb81558c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f9eb0009f30 tx=0x7f9eb000b040).stop 2023-05-16T07:19:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 shutdown_connections 2023-05-16T07:19:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9e9805ed90 0x7f9e98061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9eb8069b10 0x7f9eb814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb8153db0 0x7f9eb81558c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 --2- 172.21.15.92:0/3085571412 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9eb81539d0 0x7f9eb80a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 >> 172.21.15.92:0/3085571412 conn(0x7f9eb805e820 msgr2=0x7f9eb80abc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 shutdown_connections 2023-05-16T07:19:46.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.051+0000 7f9ebce98700 1 -- 172.21.15.92:0/3085571412 wait complete. 2023-05-16T07:19:46.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:46.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.193+0000 7f42d2e74700 1 Processor -- start 2023-05-16T07:19:46.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.193+0000 7f42d2e74700 1 -- start start 2023-05-16T07:19:46.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42d2e74700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154170 0x7f42cc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42d2e74700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42cc069db0 0x7f42cc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42d2e74700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42cc0c0660 con 0x7f42cc154550 2023-05-16T07:19:46.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42d2e74700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0c1b00 con 0x7f42cc069db0 2023-05-16T07:19:46.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42d2e74700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0bfd30 con 0x7f42cc154170 2023-05-16T07:19:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42d1411700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42cc069db0 0x7f42cc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42cbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42cc154550 0x7f42cc0696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60666/0 (socket says 172.21.15.92:60666) 2023-05-16T07:19:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.194+0000 7f42cbfff700 1 -- 172.21.15.92:0/2443369293 learned_addr learned my addr 172.21.15.92:0/2443369293 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d0c10700 1 --2- 172.21.15.92:0/2443369293 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154170 0x7f42cc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d1411700 1 -- 172.21.15.92:0/2443369293 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154170 msgr2=0x7f42cc0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d1411700 1 --2- 172.21.15.92:0/2443369293 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154170 0x7f42cc0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d1411700 1 -- 172.21.15.92:0/2443369293 >> v1:172.21.15.92:6789/0 conn(0x7f42cc154550 legacy=0x7f42cc0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42cb7fe700 1 -- 172.21.15.92:0/2443369293 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047039270 0 0) 0x7f42cc0c0660 con 0x7f42cc154550 2023-05-16T07:19:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d1411700 1 -- 172.21.15.92:0/2443369293 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0c0010 con 0x7f42cc069db0 2023-05-16T07:19:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d1411700 1 --2- 172.21.15.92:0/2443369293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc0c6160 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f42c4009760 tx=0x7f42c4009060).ready entity=mon.0 client_cookie=60a9821a2e9494e9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42cb7fe700 1 -- 172.21.15.92:0/2443369293 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40090d0 con 0x7f42cc069db0 2023-05-16T07:19:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42cb7fe700 1 -- 172.21.15.92:0/2443369293 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c400b580 con 0x7f42cc069db0 2023-05-16T07:19:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42cb7fe700 1 -- 172.21.15.92:0/2443369293 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400b900 con 0x7f42cc069db0 2023-05-16T07:19:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d2e74700 1 -- 172.21.15.92:0/2443369293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 msgr2=0x7f42cc0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.195+0000 7f42d2e74700 1 --2- 172.21.15.92:0/2443369293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc0c6160 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f42c4009760 tx=0x7f42c4009060).stop 2023-05-16T07:19:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.196+0000 7f42d2e74700 1 -- 172.21.15.92:0/2443369293 shutdown_connections 2023-05-16T07:19:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.196+0000 7f42d2e74700 1 --2- 172.21.15.92:0/2443369293 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154170 0x7f42cc0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.196+0000 7f42d2e74700 1 --2- 172.21.15.92:0/2443369293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc0c6160 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.196+0000 7f42d2e74700 1 -- 172.21.15.92:0/2443369293 >> 172.21.15.92:0/2443369293 conn(0x7f42cc0cb700 msgr2=0x7f42cc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.196+0000 7f42d2e74700 1 -- 172.21.15.92:0/2443369293 shutdown_connections 2023-05-16T07:19:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.196+0000 7f42d2e74700 1 -- 172.21.15.92:0/2443369293 wait complete. 2023-05-16T07:19:46.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.197+0000 7f42d2e74700 1 Processor -- start 2023-05-16T07:19:46.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.197+0000 7f42d2e74700 1 -- start start 2023-05-16T07:19:46.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42cc154170 0x7f42cc1641c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154550 0x7f42cc1646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0c39a0 con 0x7f42cc069db0 2023-05-16T07:19:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc158630 con 0x7f42cc154170 2023-05-16T07:19:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0c3bf0 con 0x7f42cc154550 2023-05-16T07:19:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc14d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49922/0 (socket says 172.21.15.92:49922) 2023-05-16T07:19:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 -- 172.21.15.92:0/3459667775 learned_addr learned my addr 172.21.15.92:0/3459667775 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42cbfff700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42cc154170 0x7f42cc1641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d1411700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154550 0x7f42cc1646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 -- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154550 msgr2=0x7f42cc1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154550 0x7f42cc1646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 -- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42cc154170 msgr2=0x7f42cc1641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42cc154170 0x7f42cc1641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 -- 172.21.15.92:0/3459667775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40092a0 con 0x7f42cc069db0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d0c10700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc14d8f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f42bc00d890 tx=0x7f42bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0125f0 con 0x7f42cc069db0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc012750 con 0x7f42cc069db0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc0c3ed0 con 0x7f42cc069db0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc012ad0 con 0x7f42cc069db0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.198+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc14dde0 con 0x7f42cc069db0 2023-05-16T07:19:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.199+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc0666c0 con 0x7f42cc069db0 2023-05-16T07:19:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.199+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f42bc013480 con 0x7f42cc069db0 2023-05-16T07:19:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.200+0000 7f42c8ff9700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42ac05ed90 0x7f42ac061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.200+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f42bc079900 con 0x7f42cc069db0 2023-05-16T07:19:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.200+0000 7f42cbfff700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42ac05ed90 0x7f42ac061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.200+0000 7f42cbfff700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42ac05ed90 0x7f42ac061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f42cc0abb70 tx=0x7f42b4009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.204+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f42bc04af30 con 0x7f42cc069db0 2023-05-16T07:19:46.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.369+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42cc06c350 con 0x7f42cc069db0 2023-05-16T07:19:46.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.369+0000 7f42c8ff9700 1 -- 172.21.15.92:0/3459667775 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+14698 (secure 0 0 0) 0x7f42bc04e550 con 0x7f42cc069db0 2023-05-16T07:19:46.374 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:46.374 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":131,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:45.672710+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:46.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42ac05ed90 msgr2=0x7f42ac061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42ac05ed90 0x7f42ac061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f42cc0abb70 tx=0x7f42b4009040).stop 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 msgr2=0x7f42cc14d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc14d8f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f42bc00d890 tx=0x7f42bc008040).stop 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 shutdown_connections 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f42ac05ed90 0x7f42ac061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42cc154550 0x7f42cc1646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42cc069db0 0x7f42cc14d8f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 --2- 172.21.15.92:0/3459667775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42cc154170 0x7f42cc1641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 >> 172.21.15.92:0/3459667775 conn(0x7f42cc0cb700 msgr2=0x7f42cc068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 shutdown_connections 2023-05-16T07:19:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.371+0000 7f42d2e74700 1 -- 172.21.15.92:0/3459667775 wait complete. 2023-05-16T07:19:46.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:19:46.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:19:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.497+0000 7f0060f6d700 1 Processor -- start 2023-05-16T07:19:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.497+0000 7f0060f6d700 1 -- start start 2023-05-16T07:19:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f0060f6d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f005c151ff0 0x7f005c0bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f0060f6d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c0a7140 con 0x7f005c0a6ce0 2023-05-16T07:19:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f0060f6d700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c0bdf60 con 0x7f005c069fa0 2023-05-16T07:19:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f0060f6d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c0c3410 con 0x7f005c151ff0 2023-05-16T07:19:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f005a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f005c151ff0 0x7f005c0bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f0059d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f005c0a6ce0 0x7f005c069890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60680/0 (socket says 172.21.15.92:60680) 2023-05-16T07:19:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.498+0000 7f0059d9b700 1 -- 172.21.15.92:0/692370030 learned_addr learned my addr 172.21.15.92:0/692370030 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005a59c700 1 -- 172.21.15.92:0/692370030 >> v1:172.21.15.158:6789/0 conn(0x7f005c069fa0 legacy=0x7f005c0c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005a59c700 1 -- 172.21.15.92:0/692370030 >> v1:172.21.15.92:6789/0 conn(0x7f005c0a6ce0 legacy=0x7f005c069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005a59c700 1 -- 172.21.15.92:0/692370030 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0c36b0 con 0x7f005c151ff0 2023-05-16T07:19:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005959a700 1 -- 172.21.15.92:0/692370030 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499625700 0 0) 0x7f005c0a7140 con 0x7f005c0a6ce0 2023-05-16T07:19:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005a59c700 1 --2- 172.21.15.92:0/692370030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c151ff0 0x7f005c0bbff0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f0054009e60 tx=0x7f0054009780).ready entity=mon.0 client_cookie=497b404c4f88cbf0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005959a700 1 -- 172.21.15.92:0/692370030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400e070 con 0x7f005c151ff0 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005959a700 1 -- 172.21.15.92:0/692370030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005400bc30 con 0x7f005c151ff0 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f005959a700 1 -- 172.21.15.92:0/692370030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054012410 con 0x7f005c151ff0 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 -- 172.21.15.92:0/692370030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c151ff0 msgr2=0x7f005c0bbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 --2- 172.21.15.92:0/692370030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c151ff0 0x7f005c0bbff0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f0054009e60 tx=0x7f0054009780).stop 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 -- 172.21.15.92:0/692370030 shutdown_connections 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 --2- 172.21.15.92:0/692370030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c151ff0 0x7f005c0bbff0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 -- 172.21.15.92:0/692370030 >> 172.21.15.92:0/692370030 conn(0x7f005c0c4600 msgr2=0x7f005c066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 -- 172.21.15.92:0/692370030 shutdown_connections 2023-05-16T07:19:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.499+0000 7f0060f6d700 1 -- 172.21.15.92:0/692370030 wait complete. 2023-05-16T07:19:46.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.500+0000 7f0060f6d700 1 Processor -- start 2023-05-16T07:19:46.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 -- start start 2023-05-16T07:19:46.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f005c069fa0 0x7f005c14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f005c0a6ce0 0x7f005c13eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 0x7f005c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c13f190 con 0x7f005c165820 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c165cb0 con 0x7f005c069fa0 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0060f6d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c165f90 con 0x7f005c0a6ce0 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f005ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 0x7f005c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f005ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 0x7f005c0c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49938/0 (socket says 172.21.15.92:49938) 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f005ad9d700 1 -- 172.21.15.92:0/2185838387 learned_addr learned my addr 172.21.15.92:0/2185838387 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f005a59c700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f005c069fa0 0x7f005c14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.501+0000 7f0059d9b700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f005c0a6ce0 0x7f005c13eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f005ad9d700 1 -- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f005c0a6ce0 msgr2=0x7f005c13eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f005ad9d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f005c0a6ce0 0x7f005c13eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f005ad9d700 1 -- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f005c069fa0 msgr2=0x7f005c14c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f005ad9d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f005c069fa0 0x7f005c14c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f005ad9d700 1 -- 172.21.15.92:0/2185838387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054009a20 con 0x7f005c165820 2023-05-16T07:19:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f0059d9b700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f005c0a6ce0 0x7f005c13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f005ad9d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 0x7f005c0c4ec0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f004c00c440 tx=0x7f004c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00cea0 con 0x7f005c165820 2023-05-16T07:19:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c001e90 con 0x7f005c165820 2023-05-16T07:19:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c166270 con 0x7f005c165820 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00f680 con 0x7f005c165820 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.502+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c0be290 con 0x7f005c165820 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.503+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c166710 con 0x7f005c165820 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.503+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f004c010070 con 0x7f005c165820 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.503+0000 7f004a7fc700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f003c05ed60 0x7f003c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.504+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f004c078ce0 con 0x7f005c165820 2023-05-16T07:19:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.504+0000 7f005a59c700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f003c05ed60 0x7f003c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.504+0000 7f005a59c700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f003c05ed60 0x7f003c061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0054009e60 tx=0x7f0054009780).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.507+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f004c0478f0 con 0x7f005c165820 2023-05-16T07:19:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.672+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f005c06c350 con 0x7f005c165820 2023-05-16T07:19:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.672+0000 7f004a7fc700 1 -- 172.21.15.92:0/2185838387 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v131) v1 ==== 110+0+10 (secure 0 0 0) 0x7f004c04a370 con 0x7f005c165820 2023-05-16T07:19:46.677 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:19:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f003c05ed60 msgr2=0x7f003c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f003c05ed60 0x7f003c061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0054009e60 tx=0x7f0054009780).stop 2023-05-16T07:19:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 msgr2=0x7f005c0c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 0x7f005c0c4ec0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f004c00c440 tx=0x7f004c007d70).stop 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 shutdown_connections 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f003c05ed60 0x7f003c061220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f005c165820 0x7f005c0c4ec0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f005c069fa0 0x7f005c14c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 --2- 172.21.15.92:0/2185838387 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f005c0a6ce0 0x7f005c13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 >> 172.21.15.92:0/2185838387 conn(0x7f005c0c4600 msgr2=0x7f005c14ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 shutdown_connections 2023-05-16T07:19:46.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.674+0000 7f0060f6d700 1 -- 172.21.15.92:0/2185838387 wait complete. 2023-05-16T07:19:46.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:19:46.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.800+0000 7fae1c812700 1 Processor -- start 2023-05-16T07:19:46.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.800+0000 7fae1c812700 1 -- start start 2023-05-16T07:19:46.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1c812700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae18154090 0x7fae180aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1c812700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fae18154470 0x7fae180a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1c812700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae180aa150 0x7fae18068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1c812700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae180aa5a0 con 0x7fae18154090 2023-05-16T07:19:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1c812700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae180c94d0 con 0x7fae18154470 2023-05-16T07:19:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1c812700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180cb7e0 con 0x7fae180aa150 2023-05-16T07:19:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae18154090 0x7fae180aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae18154090 0x7fae180aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49944/0 (socket says 172.21.15.92:49944) 2023-05-16T07:19:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 -- 172.21.15.92:0/2347019161 learned_addr learned my addr 172.21.15.92:0/2347019161 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:46.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae16d9d700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae180aa150 0x7fae18068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae15d9b700 1 --2- 172.21.15.92:0/2347019161 >> v2:172.21.15.158:3300/0 conn(0x7fae18154470 0x7fae180a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 -- 172.21.15.92:0/2347019161 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae180aa150 msgr2=0x7fae18068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:46.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae180aa150 0x7fae18068160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 -- 172.21.15.92:0/2347019161 >> v2:172.21.15.158:3300/0 conn(0x7fae18154470 msgr2=0x7fae180a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:46.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 --2- 172.21.15.92:0/2347019161 >> v2:172.21.15.158:3300/0 conn(0x7fae18154470 0x7fae180a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae1659c700 1 -- 172.21.15.92:0/2347019161 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180c88c0 con 0x7fae18154090 2023-05-16T07:19:46.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae16d9d700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae180aa150 0x7fae18068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:46.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.801+0000 7fae15d9b700 1 --2- 172.21.15.92:0/2347019161 >> v2:172.21.15.158:3300/0 conn(0x7fae18154470 0x7fae180a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:46.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1659c700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae18154090 0x7fae180aaca0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fae18084320 tx=0x7fae08009840).ready entity=mon.0 client_cookie=b6252ce8acbcfe61 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1559a700 1 -- 172.21.15.92:0/2347019161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800e070 con 0x7fae18154090 2023-05-16T07:19:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1559a700 1 -- 172.21.15.92:0/2347019161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800ba50 con 0x7fae18154090 2023-05-16T07:19:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1559a700 1 -- 172.21.15.92:0/2347019161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bdd0 con 0x7fae18154090 2023-05-16T07:19:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 -- 172.21.15.92:0/2347019161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae18154090 msgr2=0x7fae180aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae18154090 0x7fae180aaca0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fae18084320 tx=0x7fae08009840).stop 2023-05-16T07:19:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 -- 172.21.15.92:0/2347019161 shutdown_connections 2023-05-16T07:19:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae180aa150 0x7fae18068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 --2- 172.21.15.92:0/2347019161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae18154090 0x7fae180aaca0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 --2- 172.21.15.92:0/2347019161 >> v2:172.21.15.158:3300/0 conn(0x7fae18154470 0x7fae180a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 -- 172.21.15.92:0/2347019161 >> 172.21.15.92:0/2347019161 conn(0x7fae18067920 msgr2=0x7fae180cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 -- 172.21.15.92:0/2347019161 shutdown_connections 2023-05-16T07:19:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.802+0000 7fae1c812700 1 -- 172.21.15.92:0/2347019161 wait complete. 2023-05-16T07:19:46.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.803+0000 7fae1c812700 1 Processor -- start 2023-05-16T07:19:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.803+0000 7fae1c812700 1 -- start start 2023-05-16T07:19:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1c812700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 0x7fae181564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1c812700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae18154090 0x7fae18155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1c812700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae18154470 0x7fae1813bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1c812700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181577c0 con 0x7fae180aa150 2023-05-16T07:19:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1c812700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18156bb0 con 0x7fae18154090 2023-05-16T07:19:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1c812700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae18156e90 con 0x7fae18154470 2023-05-16T07:19:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 0x7fae181564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 0x7fae181564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49950/0 (socket says 172.21.15.92:49950) 2023-05-16T07:19:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae1659c700 1 -- 172.21.15.92:0/1694940870 learned_addr learned my addr 172.21.15.92:0/1694940870 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae15d9b700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae18154090 0x7fae18155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.804+0000 7fae16d9d700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae18154470 0x7fae1813bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1659c700 1 -- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae18154470 msgr2=0x7fae1813bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1659c700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae18154470 0x7fae1813bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1659c700 1 -- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae18154090 msgr2=0x7fae18155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1659c700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae18154090 0x7fae18155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1659c700 1 -- 172.21.15.92:0/1694940870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08009ae0 con 0x7fae180aa150 2023-05-16T07:19:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae15d9b700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae18154090 0x7fae18155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae16d9d700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae18154470 0x7fae1813bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1659c700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 0x7fae181564f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fae08009fd0 tx=0x7fae08009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bdd0 con 0x7fae180aa150 2023-05-16T07:19:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae08001010 con 0x7fae180aa150 2023-05-16T07:19:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae18157170 con 0x7fae180aa150 2023-05-16T07:19:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0801d670 con 0x7fae180aa150 2023-05-16T07:19:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.805+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae181558f0 con 0x7fae180aa150 2023-05-16T07:19:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.806+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae18066730 con 0x7fae180aa150 2023-05-16T07:19:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.806+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fae08024030 con 0x7fae180aa150 2023-05-16T07:19:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.806+0000 7fae0e7fc700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fadf805ed90 0x7fadf8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.807+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fae0807ae40 con 0x7fae180aa150 2023-05-16T07:19:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.807+0000 7fae15d9b700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fadf805ed90 0x7fadf8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.807+0000 7fae15d9b700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fadf805ed90 0x7fadf8061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fae18155f40 tx=0x7fae00008040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.811+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fae0804c580 con 0x7fae180aa150 2023-05-16T07:19:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.973+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae1806c350 con 0x7fae180aa150 2023-05-16T07:19:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.974+0000 7fae0e7fc700 1 -- 172.21.15.92:0/1694940870 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v131) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae08022030 con 0x7fae180aa150 2023-05-16T07:19:46.978 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:19:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.975+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fadf805ed90 msgr2=0x7fadf8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.975+0000 7fae1c812700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fadf805ed90 0x7fadf8061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fae18155f40 tx=0x7fae00008040).stop 2023-05-16T07:19:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.975+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 msgr2=0x7fae181564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.975+0000 7fae1c812700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 0x7fae181564f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fae08009fd0 tx=0x7fae08009840).stop 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 shutdown_connections 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fadf805ed90 0x7fadf8061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fae18154470 0x7fae1813bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae180aa150 0x7fae181564f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 --2- 172.21.15.92:0/1694940870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fae18154090 0x7fae18155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 >> 172.21.15.92:0/1694940870 conn(0x7fae18067920 msgr2=0x7fae180c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 shutdown_connections 2023-05-16T07:19:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:46.976+0000 7fae1c812700 1 -- 172.21.15.92:0/1694940870 wait complete. 2023-05-16T07:19:46.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:19:47.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 Processor -- start 2023-05-16T07:19:47.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 -- start start 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6184151bd0 0x7f6184156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f61840a76f0 0x7f61840a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61840c8750 con 0x7f61841517f0 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6184155d70 con 0x7f6184151bd0 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.098+0000 7f6183fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61840a8030 con 0x7f61840a76f0 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61827fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6184151bd0 0x7f6184156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f6182ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f61841517f0 0x7f6184152a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60706/0 (socket says 172.21.15.92:60706) 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f6182ffd700 1 -- 172.21.15.92:0/1032985042 learned_addr learned my addr 172.21.15.92:0/1032985042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61837fe700 1 --2- 172.21.15.92:0/1032985042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f61840a76f0 0x7f61840a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61827fc700 1 -- 172.21.15.92:0/1032985042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f61840a76f0 msgr2=0x7f61840a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61827fc700 1 --2- 172.21.15.92:0/1032985042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f61840a76f0 0x7f61840a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61827fc700 1 -- 172.21.15.92:0/1032985042 >> v1:172.21.15.92:6789/0 conn(0x7f61841517f0 legacy=0x7f6184152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f6181ffb700 1 -- 172.21.15.92:0/1032985042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454254771 0 0) 0x7f61840c8750 con 0x7f61841517f0 2023-05-16T07:19:47.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61827fc700 1 -- 172.21.15.92:0/1032985042 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840c8470 con 0x7f6184151bd0 2023-05-16T07:19:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f61827fc700 1 --2- 172.21.15.92:0/1032985042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6184151bd0 0x7f6184156200 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f6178009760 tx=0x7f6178009060).ready entity=mon.0 client_cookie=85766ebdaee94fb8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f6181ffb700 1 -- 172.21.15.92:0/1032985042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61780090d0 con 0x7f6184151bd0 2023-05-16T07:19:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f6181ffb700 1 -- 172.21.15.92:0/1032985042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617800b580 con 0x7f6184151bd0 2023-05-16T07:19:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.099+0000 7f6181ffb700 1 -- 172.21.15.92:0/1032985042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800b900 con 0x7f6184151bd0 2023-05-16T07:19:47.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 -- 172.21.15.92:0/1032985042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6184151bd0 msgr2=0x7f6184156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 --2- 172.21.15.92:0/1032985042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6184151bd0 0x7f6184156200 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f6178009760 tx=0x7f6178009060).stop 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 -- 172.21.15.92:0/1032985042 shutdown_connections 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 --2- 172.21.15.92:0/1032985042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f61840a76f0 0x7f61840a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 --2- 172.21.15.92:0/1032985042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6184151bd0 0x7f6184156200 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 -- 172.21.15.92:0/1032985042 >> 172.21.15.92:0/1032985042 conn(0x7f618405e820 msgr2=0x7f61840622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 -- 172.21.15.92:0/1032985042 shutdown_connections 2023-05-16T07:19:47.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.100+0000 7f6183fff700 1 -- 172.21.15.92:0/1032985042 wait complete. 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.101+0000 7f6183fff700 1 Processor -- start 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.101+0000 7f6183fff700 1 -- start start 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 0x7f61841629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61841517f0 0x7f618414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6184151bd0 0x7f618414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61840ae9b0 con 0x7f61840a76f0 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618414dd00 con 0x7f61841517f0 2023-05-16T07:19:47.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6184162eb0 con 0x7f6184151bd0 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 0x7f61841629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 0x7f61841629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49964/0 (socket says 172.21.15.92:49964) 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 -- 172.21.15.92:0/2087030459 learned_addr learned my addr 172.21.15.92:0/2087030459 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 -- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6184151bd0 msgr2=0x7f618414e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6184151bd0 0x7f618414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 -- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61841517f0 msgr2=0x7f618414b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61841517f0 0x7f618414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:47.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 -- 172.21.15.92:0/2087030459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780092a0 con 0x7f61840a76f0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6182ffd700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 0x7f61841629c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f617c00c7f0 tx=0x7f617c007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c005c00 con 0x7f61840a76f0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c005d60 con 0x7f61840a76f0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c007360 con 0x7f61840a76f0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61840ae6d0 con 0x7f61840a76f0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.102+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618414d990 con 0x7f61840a76f0 2023-05-16T07:19:47.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.103+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6184047150 con 0x7f61840a76f0 2023-05-16T07:19:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.104+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f617c010030 con 0x7f61840a76f0 2023-05-16T07:19:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.104+0000 7f616affd700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617005ed90 0x7f6170061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.104+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f617c079990 con 0x7f61840a76f0 2023-05-16T07:19:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.104+0000 7f61827fc700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617005ed90 0x7f6170061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.104+0000 7f61827fc700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617005ed90 0x7f6170061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f618414fbf0 tx=0x7f6178017000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.108+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f617c04b020 con 0x7f61840a76f0 2023-05-16T07:19:47.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.273+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6184155720 con 0x7f61840a76f0 2023-05-16T07:19:47.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.273+0000 7f616affd700 1 -- 172.21.15.92:0/2087030459 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+143 (secure 0 0 0) 0x7f617c04e640 con 0x7f61840a76f0 2023-05-16T07:19:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:47.277 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:19:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:19:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:47.279 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:19:47.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617005ed90 msgr2=0x7f6170061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617005ed90 0x7f6170061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f618414fbf0 tx=0x7f6178017000).stop 2023-05-16T07:19:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 msgr2=0x7f61841629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 0x7f61841629c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f617c00c7f0 tx=0x7f617c007e80).stop 2023-05-16T07:19:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 shutdown_connections 2023-05-16T07:19:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f617005ed90 0x7f6170061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6184151bd0 0x7f618414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61840a76f0 0x7f61841629c0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 --2- 172.21.15.92:0/2087030459 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f61841517f0 0x7f618414b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 >> 172.21.15.92:0/2087030459 conn(0x7f618405e820 msgr2=0x7f61840c2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 shutdown_connections 2023-05-16T07:19:47.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.275+0000 7f6183fff700 1 -- 172.21.15.92:0/2087030459 wait complete. 2023-05-16T07:19:47.290 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:19:47.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:19:47.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.402+0000 7fd665335700 1 Processor -- start 2023-05-16T07:19:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.402+0000 7fd665335700 1 -- start start 2023-05-16T07:19:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.403+0000 7fd665335700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd6600698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.403+0000 7fd665335700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66005e760 con 0x7fd660153bb0 2023-05-16T07:19:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.403+0000 7fd665335700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd660069ef0 con 0x7fd660153f90 2023-05-16T07:19:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.403+0000 7fd665335700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600bf540 con 0x7fd6600bdf20 2023-05-16T07:19:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.403+0000 7fd65ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd660153bb0 0x7fd6600bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60720/0 (socket says 172.21.15.92:60720) 2023-05-16T07:19:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.403+0000 7fd65ffff700 1 -- 172.21.15.92:0/2723128370 learned_addr learned my addr 172.21.15.92:0/2723128370 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377839009 0 0) 0x7fd66005e760 con 0x7fd660153bb0 2023-05-16T07:19:47.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6480033f0 con 0x7fd660153bb0 2023-05-16T07:19:47.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd6600698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1289539361 0 0) 0x7fd6480033f0 con 0x7fd660153bb0 2023-05-16T07:19:47.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 msgr2=0x7fd6600698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd6600698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 >> v1:172.21.15.158:6789/0 conn(0x7fd660153f90 legacy=0x7fd6600c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660158460 con 0x7fd660153bb0 2023-05-16T07:19:47.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd650001470 con 0x7fd660153bb0 2023-05-16T07:19:47.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd6600698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:47.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 >> v1:172.21.15.92:6789/0 conn(0x7fd660153bb0 legacy=0x7fd6600bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd648003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd648003eb0 0x7fd648004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648004b80 0x7fd648005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600bf540 con 0x7fd648003eb0 2023-05-16T07:19:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64800b9f0 con 0x7fd648004b80 2023-05-16T07:19:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64800bbb0 con 0x7fd6600bdf20 2023-05-16T07:19:47.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd648003eb0 0x7fd648004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65ffff700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd648003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd664b34700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648004b80 0x7fd648005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 -- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 msgr2=0x7fd648003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6600bdf20 0x7fd648003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 -- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648004b80 msgr2=0x7fd648005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648004b80 0x7fd648005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd65f7fe700 1 -- 172.21.15.92:0/2723128370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480033f0 con 0x7fd648003eb0 2023-05-16T07:19:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.404+0000 7fd664b34700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648004b80 0x7fd648005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd648003eb0 0x7fd648004690 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7fd6600be320 tx=0x7fd654008040).ready entity=mon.0 client_cookie=b4037f93534e20fa server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400b410 con 0x7fd648003eb0 2023-05-16T07:19:47.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65400be50 con 0x7fd648003eb0 2023-05-16T07:19:47.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd65effd700 1 -- 172.21.15.92:0/2723128370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400d820 con 0x7fd648003eb0 2023-05-16T07:19:47.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd665335700 1 -- 172.21.15.92:0/2723128370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd648003eb0 msgr2=0x7fd648004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd665335700 1 --2- 172.21.15.92:0/2723128370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd648003eb0 0x7fd648004690 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7fd6600be320 tx=0x7fd654008040).stop 2023-05-16T07:19:47.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd664b34700 1 -- 172.21.15.92:0/2723128370 reap_dead start 2023-05-16T07:19:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd665335700 1 -- 172.21.15.92:0/2723128370 shutdown_connections 2023-05-16T07:19:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd665335700 1 -- 172.21.15.92:0/2723128370 >> 172.21.15.92:0/2723128370 conn(0x7fd660062d80 msgr2=0x7fd66005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:47.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd665335700 1 -- 172.21.15.92:0/2723128370 shutdown_connections 2023-05-16T07:19:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.405+0000 7fd665335700 1 -- 172.21.15.92:0/2723128370 wait complete. 2023-05-16T07:19:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.406+0000 7fd665335700 1 Processor -- start 2023-05-16T07:19:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 -- start start 2023-05-16T07:19:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648003eb0 0x7fd660141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 0x7fd6601543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6601548b0 0x7fd660154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd660155220 con 0x7fd6601418f0 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd660141e50 con 0x7fd648003eb0 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd665335700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd660142130 con 0x7fd6601548b0 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd65f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 0x7fd6601543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd65f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 0x7fd6601543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49984/0 (socket says 172.21.15.92:49984) 2023-05-16T07:19:47.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd65f7fe700 1 -- 172.21.15.92:0/3991428473 learned_addr learned my addr 172.21.15.92:0/3991428473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd664b34700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6601548b0 0x7fd660154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.407+0000 7fd65ffff700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648003eb0 0x7fd660141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd65f7fe700 1 -- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6601548b0 msgr2=0x7fd660154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6601548b0 0x7fd660154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd65f7fe700 1 -- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648003eb0 msgr2=0x7fd660141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648003eb0 0x7fd660141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd65f7fe700 1 -- 172.21.15.92:0/3991428473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660142410 con 0x7fd6601418f0 2023-05-16T07:19:47.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd664b34700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6601548b0 0x7fd660154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:47.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd65f7fe700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 0x7fd6601543c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7fd654008010 tx=0x7fd654008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400dec0 con 0x7fd6601418f0 2023-05-16T07:19:47.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654012630 con 0x7fd6601418f0 2023-05-16T07:19:47.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6601426f0 con 0x7fd6601418f0 2023-05-16T07:19:47.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540129f0 con 0x7fd6601418f0 2023-05-16T07:19:47.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.408+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd660141010 con 0x7fd6601418f0 2023-05-16T07:19:47.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.409+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd660142d40 con 0x7fd6601418f0 2023-05-16T07:19:47.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.409+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd654025470 con 0x7fd6601418f0 2023-05-16T07:19:47.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.410+0000 7fd647fff700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd64c05ed90 0x7fd64c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.410+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd65407b390 con 0x7fd6601418f0 2023-05-16T07:19:47.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.410+0000 7fd65ffff700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd64c05ed90 0x7fd64c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.410+0000 7fd65ffff700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd64c05ed90 0x7fd64c061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd650001120 tx=0x7fd650009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.414+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd65404ca00 con 0x7fd6601418f0 2023-05-16T07:19:47.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.577+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fd660047150 con 0x7fd6601418f0 2023-05-16T07:19:47.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.688+0000 7fd647fff700 1 -- 172.21.15.92:0/3991428473 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd65404ffe0 con 0x7fd6601418f0 2023-05-16T07:19:47.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.691+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd64c05ed90 msgr2=0x7fd64c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.691+0000 7fd665335700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd64c05ed90 0x7fd64c061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd650001120 tx=0x7fd650009040).stop 2023-05-16T07:19:47.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.691+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 msgr2=0x7fd6601543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.691+0000 7fd665335700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 0x7fd6601543c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7fd654008010 tx=0x7fd654008040).stop 2023-05-16T07:19:47.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.693+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 shutdown_connections 2023-05-16T07:19:47.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.693+0000 7fd665335700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd64c05ed90 0x7fd64c061250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.693+0000 7fd665335700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd6601548b0 0x7fd660154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.693+0000 7fd665335700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd648003eb0 0x7fd660141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.693+0000 7fd665335700 1 --2- 172.21.15.92:0/3991428473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6601418f0 0x7fd6601543c0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.693+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 >> 172.21.15.92:0/3991428473 conn(0x7fd660062d80 msgr2=0x7fd6601534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:47.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.694+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 shutdown_connections 2023-05-16T07:19:47.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.695+0000 7fd665335700 1 -- 172.21.15.92:0/3991428473 wait complete. 2023-05-16T07:19:47.700 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:19:47.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:19:47.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.825+0000 7fd2d3daa700 1 Processor -- start 2023-05-16T07:19:47.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.825+0000 7fd2d3daa700 1 -- start start 2023-05-16T07:19:47.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d3daa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 0x7fd2cc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d3daa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0c94d0 con 0x7fd2cc154470 2023-05-16T07:19:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d3daa700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0cb7e0 con 0x7fd2cc154090 2023-05-16T07:19:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d3daa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c88c0 con 0x7fd2cc068870 2023-05-16T07:19:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d1345700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd2cc154470 0x7fd2cc068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60742/0 (socket says 172.21.15.92:60742) 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d1345700 1 -- 172.21.15.92:0/3947305203 learned_addr learned my addr 172.21.15.92:0/3947305203 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625388136 0 0) 0x7fd2cc0c94d0 con 0x7fd2cc154470 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d2347700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 0x7fd2cc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b80033f0 con 0x7fd2cc154470 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795259683 0 0) 0x7fd2b80033f0 con 0x7fd2cc154470 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 msgr2=0x7fd2cc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 0x7fd2cc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 >> v1:172.21.15.158:6789/0 conn(0x7fd2cc154090 legacy=0x7fd2cc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0c8ba0 con 0x7fd2cc154470 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd2bc001470 con 0x7fd2cc154470 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 >> v1:172.21.15.92:6789/0 conn(0x7fd2cc154470 legacy=0x7fd2cc068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2b8003910 0x7fd2b8003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2b8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 0x7fd2b8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c88c0 con 0x7fd2b8004690 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b800b910 con 0x7fd2b8003910 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.826+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b800bad0 con 0x7fd2cc068870 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2b8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1b46700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2b8003910 0x7fd2b8003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d2347700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 0x7fd2b8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 -- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 msgr2=0x7fd2b8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc068870 0x7fd2b8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 -- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2b8003910 msgr2=0x7fd2b8003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2b8003910 0x7fd2b8003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 -- 172.21.15.92:0/3947305203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80033f0 con 0x7fd2b8004690 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d1345700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2b8004b10 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fd2bc00c190 tx=0x7fd2bc001e50).ready entity=mon.0 client_cookie=4dd3016b78b9c200 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007a20 con 0x7fd2b8004690 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc00cac0 con 0x7fd2b8004690 2023-05-16T07:19:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d0b44700 1 -- 172.21.15.92:0/3947305203 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00ce40 con 0x7fd2b8004690 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d3daa700 1 -- 172.21.15.92:0/3947305203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 msgr2=0x7fd2b8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.827+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/3947305203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2b8004b10 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fd2bc00c190 tx=0x7fd2bc001e50).stop 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.828+0000 7fd2d2347700 1 -- 172.21.15.92:0/3947305203 reap_dead start 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.828+0000 7fd2d3daa700 1 -- 172.21.15.92:0/3947305203 shutdown_connections 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.828+0000 7fd2d3daa700 1 -- 172.21.15.92:0/3947305203 >> 172.21.15.92:0/3947305203 conn(0x7fd2cc067920 msgr2=0x7fd2cc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.828+0000 7fd2d3daa700 1 -- 172.21.15.92:0/3947305203 shutdown_connections 2023-05-16T07:19:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.828+0000 7fd2d3daa700 1 -- 172.21.15.92:0/3947305203 wait complete. 2023-05-16T07:19:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 Processor -- start 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 -- start start 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2cc142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc14ec60 0x7fd2cc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2cc0c5a80 0x7fd2cc0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c3530 con 0x7fd2b8004690 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c63d0 con 0x7fd2cc0c5a80 2023-05-16T07:19:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.829+0000 7fd2d3daa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc142e10 con 0x7fd2cc14ec60 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2cc142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2cc142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50004/0 (socket says 172.21.15.92:50004) 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 -- 172.21.15.92:0/215112556 learned_addr learned my addr 172.21.15.92:0/215112556 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 -- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc14ec60 msgr2=0x7fd2cc155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc14ec60 0x7fd2cc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 -- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2cc0c5a80 msgr2=0x7fd2cc0c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2cc0c5a80 0x7fd2cc0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 -- 172.21.15.92:0/215112556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc156470 con 0x7fd2b8004690 2023-05-16T07:19:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d1b46700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2cc142920 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7fd2c80028f0 tx=0x7fd2c8007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800f6c0 con 0x7fd2b8004690 2023-05-16T07:19:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c800f820 con 0x7fd2b8004690 2023-05-16T07:19:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800fba0 con 0x7fd2b8004690 2023-05-16T07:19:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc156750 con 0x7fd2b8004690 2023-05-16T07:19:47.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.830+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc156d10 con 0x7fd2b8004690 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.831+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc0cb210 con 0x7fd2b8004690 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.831+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd2c801b030 con 0x7fd2b8004690 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.832+0000 7fd2c1ffb700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2b005ed90 0x7fd2b0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.832+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6251+0+0 (secure 0 0 0) 0x7fd2c8079730 con 0x7fd2b8004690 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.832+0000 7fd2d1345700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2b005ed90 0x7fd2b0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.833+0000 7fd2d1345700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2b005ed90 0x7fd2b0061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd2cc14f0a0 tx=0x7fd2bc015000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:47.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.836+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd2c804ad00 con 0x7fd2b8004690 2023-05-16T07:19:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:47.999+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd2cc066730 con 0x7fd2b8004690 2023-05-16T07:19:48.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.696+0000 7fd2c1ffb700 1 -- 172.21.15.92:0/215112556 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd2c804e320 con 0x7fd2b8004690 2023-05-16T07:19:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2b005ed90 msgr2=0x7fd2b0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2b005ed90 0x7fd2b0061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd2cc14f0a0 tx=0x7fd2bc015000).stop 2023-05-16T07:19:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 msgr2=0x7fd2cc142920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2cc142920 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7fd2c80028f0 tx=0x7fd2c8007bb0).stop 2023-05-16T07:19:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 shutdown_connections 2023-05-16T07:19:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fd2b005ed90 0x7fd2b0061250 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0x7fd2cc14f0a0 tx=0x7fd2bc015000).stop 2023-05-16T07:19:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2cc0c5a80 0x7fd2cc0c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b8004690 0x7fd2cc142920 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 --2- 172.21.15.92:0/215112556 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd2cc14ec60 0x7fd2cc155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.703+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 >> 172.21.15.92:0/215112556 conn(0x7fd2cc067920 msgr2=0x7fd2cc0acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.704+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 shutdown_connections 2023-05-16T07:19:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.705+0000 7fd2d3daa700 1 -- 172.21.15.92:0/215112556 wait complete. 2023-05-16T07:19:48.716 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:19:48.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:19:48.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.844+0000 7f70ba0d7700 1 Processor -- start 2023-05-16T07:19:48.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.844+0000 7f70ba0d7700 1 -- start start 2023-05-16T07:19:48.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70ba0d7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 0x7f70b40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70ba0d7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40c94d0 con 0x7f70b4068160 2023-05-16T07:19:48.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70ba0d7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40cb7e0 con 0x7f70b4154470 2023-05-16T07:19:48.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70ba0d7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c88c0 con 0x7f70b4154090 2023-05-16T07:19:48.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70b3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f70b4068160 0x7f70b4068580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60764/0 (socket says 172.21.15.92:60764) 2023-05-16T07:19:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70b3fff700 1 -- 172.21.15.92:0/1687937325 learned_addr learned my addr 172.21.15.92:0/1687937325 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576700543 0 0) 0x7f70b40c94d0 con 0x7f70b4068160 2023-05-16T07:19:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f709c0033f0 con 0x7f70b4068160 2023-05-16T07:19:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.845+0000 7f70b37fe700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 0x7f70b40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3581682053 0 0) 0x7f709c0033f0 con 0x7f70b4068160 2023-05-16T07:19:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 msgr2=0x7f70b40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 0x7f70b40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 >> v1:172.21.15.158:6789/0 conn(0x7f70b4154470 legacy=0x7f70b40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40c8ba0 con 0x7f70b4068160 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f70ac001470 con 0x7f70b4068160 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 >> v1:172.21.15.92:6789/0 conn(0x7f70b4068160 legacy=0x7f70b4068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f709c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 0x7f709c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f709c004b80 0x7f709c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c88c0 con 0x7f709c003910 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c00b910 con 0x7f709c004b80 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c00bad0 con 0x7f70b4154090 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f709c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b3fff700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f709c004b80 0x7f709c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b2ffd700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 0x7f709c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 -- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 msgr2=0x7f709c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b4154090 0x7f709c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 -- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f709c004b80 msgr2=0x7f709c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f709c004b80 0x7f709c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 -- 172.21.15.92:0/1687937325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0033f0 con 0x7f709c003910 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b37fe700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f709c003d30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f70a400e210 tx=0x7f70a400b040).ready entity=mon.0 client_cookie=72ec5372a70574b9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4009ac0 con 0x7f709c003910 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a400f040 con 0x7f709c003910 2023-05-16T07:19:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.846+0000 7f70b27fc700 1 -- 172.21.15.92:0/1687937325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400ecd0 con 0x7f709c003910 2023-05-16T07:19:48.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70ba0d7700 1 -- 172.21.15.92:0/1687937325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 msgr2=0x7f709c003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/1687937325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f709c003d30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f70a400e210 tx=0x7f70a400b040).stop 2023-05-16T07:19:48.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70b3fff700 1 -- 172.21.15.92:0/1687937325 reap_dead start 2023-05-16T07:19:48.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70ba0d7700 1 -- 172.21.15.92:0/1687937325 shutdown_connections 2023-05-16T07:19:48.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70ba0d7700 1 -- 172.21.15.92:0/1687937325 >> 172.21.15.92:0/1687937325 conn(0x7f70b4067920 msgr2=0x7f70b40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:48.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70ba0d7700 1 -- 172.21.15.92:0/1687937325 shutdown_connections 2023-05-16T07:19:48.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.847+0000 7f70ba0d7700 1 -- 172.21.15.92:0/1687937325 wait complete. 2023-05-16T07:19:48.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.848+0000 7f70ba0d7700 1 Processor -- start 2023-05-16T07:19:48.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.848+0000 7f70ba0d7700 1 -- start start 2023-05-16T07:19:48.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70ba0d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f70b413b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70ba0d7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b41564c0 0x7f70b4156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70ba0d7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f70b40c9ce0 0x7f70b40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70ba0d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b413b980 con 0x7f709c003910 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70ba0d7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40ca630 con 0x7f70b40c9ce0 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70ba0d7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4156fa0 con 0x7f70b41564c0 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f70b413b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f70b413b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50026/0 (socket says 172.21.15.92:50026) 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 -- 172.21.15.92:0/74800188 learned_addr learned my addr 172.21.15.92:0/74800188 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b3fff700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f70b40c9ce0 0x7f70b40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b2ffd700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b41564c0 0x7f70b4156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 -- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b41564c0 msgr2=0x7f70b4156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b41564c0 0x7f70b4156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 -- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f70b40c9ce0 msgr2=0x7f70b40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f70b40c9ce0 0x7f70b40ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 -- 172.21.15.92:0/74800188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4157280 con 0x7f709c003910 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.849+0000 7f70b37fe700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f70b413b490 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f709c00b8e0 tx=0x7f70a401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.850+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4013c10 con 0x7f709c003910 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.850+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4013d70 con 0x7f709c003910 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.850+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b4157560 con 0x7f709c003910 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.850+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4027420 con 0x7f709c003910 2023-05-16T07:19:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.850+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b4165980 con 0x7f709c003910 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.851+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4066730 con 0x7f709c003910 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.851+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f70a402e070 con 0x7f709c003910 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.851+0000 7f709b7fe700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f70a005ed90 0x7f70a0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.852+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6633+0+0 (secure 0 0 0) 0x7f70a407bbc0 con 0x7f709c003910 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.852+0000 7f70b2ffd700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f70a005ed90 0x7f70a0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.852+0000 7f70b2ffd700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f70a005ed90 0x7f70a0061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f70a8007fd0 tx=0x7f70a8007bb0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:48.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:48.856+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f70a404cf60 con 0x7f709c003910 2023-05-16T07:19:49.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.063+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f70b406c350 con 0x7f709c003910 2023-05-16T07:19:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.069+0000 7f709b7fe700 1 -- 172.21.15.92:0/74800188 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v47) v1 ==== 174+0+0 (secure 0 0 0) 0x7f70a402c030 con 0x7f709c003910 2023-05-16T07:19:49.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f70a005ed90 msgr2=0x7f70a0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f70a005ed90 0x7f70a0061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f70a8007fd0 tx=0x7f70a8007bb0).stop 2023-05-16T07:19:49.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 msgr2=0x7f70b413b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f70b413b490 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f709c00b8e0 tx=0x7f70a401a000).stop 2023-05-16T07:19:49.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 shutdown_connections 2023-05-16T07:19:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f70a005ed90 0x7f70a0061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f70b40c9ce0 0x7f70b40ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f709c003910 0x7f70b413b490 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 --2- 172.21.15.92:0/74800188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f70b41564c0 0x7f70b4156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.076+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 >> 172.21.15.92:0/74800188 conn(0x7f70b4067920 msgr2=0x7f70b414d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.077+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 shutdown_connections 2023-05-16T07:19:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.077+0000 7f70ba0d7700 1 -- 172.21.15.92:0/74800188 wait complete. 2023-05-16T07:19:49.083 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 17 and data pool 18 2023-05-16T07:19:49.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:49.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.208+0000 7f0ec359e700 1 Processor -- start 2023-05-16T07:19:49.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.208+0000 7f0ec359e700 1 -- start start 2023-05-16T07:19:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ec40c7380 0x7f0ec4154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec359e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ec40c7760 0x7f0ec40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec4158970 con 0x7f0ec40ad040 2023-05-16T07:19:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec4155340 con 0x7f0ec40c7380 2023-05-16T07:19:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec359e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec4157d60 con 0x7f0ec40c7760 2023-05-16T07:19:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ec40c7380 0x7f0ec4154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ec40c7380 0x7f0ec4154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50048/0 (socket says 172.21.15.92:50048) 2023-05-16T07:19:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 -- 172.21.15.92:0/2817292088 learned_addr learned my addr 172.21.15.92:0/2817292088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec1d9b700 1 --2- 172.21.15.92:0/2817292088 >> v2:172.21.15.158:3300/0 conn(0x7f0ec40c7760 0x7f0ec40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 -- 172.21.15.92:0/2817292088 >> v2:172.21.15.158:3300/0 conn(0x7f0ec40c7760 msgr2=0x7f0ec40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 --2- 172.21.15.92:0/2817292088 >> v2:172.21.15.158:3300/0 conn(0x7f0ec40c7760 0x7f0ec40acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 -- 172.21.15.92:0/2817292088 >> v1:172.21.15.158:6789/0 conn(0x7f0ec40ad040 legacy=0x7f0ec414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 -- 172.21.15.92:0/2817292088 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4158040 con 0x7f0ec40c7380 2023-05-16T07:19:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec259c700 1 --2- 172.21.15.92:0/2817292088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7380 0x7f0ec4154e50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f0ec4084350 tx=0x7f0eb40090f0).ready entity=mon.0 client_cookie=83a36cac2ddbc04a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec159a700 1 -- 172.21.15.92:0/2817292088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400b5a0 con 0x7f0ec40c7380 2023-05-16T07:19:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec159a700 1 -- 172.21.15.92:0/2817292088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400b700 con 0x7f0ec40c7380 2023-05-16T07:19:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.209+0000 7f0ec159a700 1 -- 172.21.15.92:0/2817292088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400ba80 con 0x7f0ec40c7380 2023-05-16T07:19:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 -- 172.21.15.92:0/2817292088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7380 msgr2=0x7f0ec4154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 --2- 172.21.15.92:0/2817292088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7380 0x7f0ec4154e50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f0ec4084350 tx=0x7f0eb40090f0).stop 2023-05-16T07:19:49.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 -- 172.21.15.92:0/2817292088 shutdown_connections 2023-05-16T07:19:49.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 --2- 172.21.15.92:0/2817292088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7380 0x7f0ec4154e50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 --2- 172.21.15.92:0/2817292088 >> v2:172.21.15.158:3300/0 conn(0x7f0ec40c7760 0x7f0ec40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 -- 172.21.15.92:0/2817292088 >> 172.21.15.92:0/2817292088 conn(0x7f0ec4068330 msgr2=0x7f0ec4068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 -- 172.21.15.92:0/2817292088 shutdown_connections 2023-05-16T07:19:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.210+0000 7f0ec359e700 1 -- 172.21.15.92:0/2817292088 wait complete. 2023-05-16T07:19:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.211+0000 7f0ec359e700 1 Processor -- start 2023-05-16T07:19:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.211+0000 7f0ec359e700 1 -- start start 2023-05-16T07:19:49.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ec40ad040 0x7f0ec4168780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ec40c7380 0x7f0ec414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 0x7f0ec40adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40c97d0 con 0x7f0ec40c7760 2023-05-16T07:19:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4168c70 con 0x7f0ec40ad040 2023-05-16T07:19:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40c88e0 con 0x7f0ec40c7380 2023-05-16T07:19:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 0x7f0ec40adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 0x7f0ec40adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50058/0 (socket says 172.21.15.92:50058) 2023-05-16T07:19:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 -- 172.21.15.92:0/1296955436 learned_addr learned my addr 172.21.15.92:0/1296955436 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec259c700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ec40ad040 0x7f0ec4168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec1d9b700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ec40c7380 0x7f0ec414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 -- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ec40c7380 msgr2=0x7f0ec414e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ec40c7380 0x7f0ec414e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 -- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ec40ad040 msgr2=0x7f0ec4168780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ec40ad040 0x7f0ec4168780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 -- 172.21.15.92:0/1296955436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4009370 con 0x7f0ec40c7760 2023-05-16T07:19:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0ec2d9d700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 0x7f0ec40adbc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f0ebc00c810 tx=0x7f0ebc002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.212+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc010510 con 0x7f0ec40c7760 2023-05-16T07:19:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.213+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc010690 con 0x7f0ec40c7760 2023-05-16T07:19:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.213+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec40c8bc0 con 0x7f0ec40c7760 2023-05-16T07:19:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.213+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc010a50 con 0x7f0ec40c7760 2023-05-16T07:19:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.213+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec40c9180 con 0x7f0ec40c7760 2023-05-16T07:19:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.214+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec40c9980 con 0x7f0ec40c7760 2023-05-16T07:19:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.214+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0ebc019030 con 0x7f0ec40c7760 2023-05-16T07:19:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.214+0000 7f0eb27fc700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0ea405ed90 0x7f0ea4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.214+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f0ebc079550 con 0x7f0ec40c7760 2023-05-16T07:19:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.214+0000 7f0ec259c700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0ea405ed90 0x7f0ea4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.215+0000 7f0ec259c700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0ea405ed90 0x7f0ea4061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0eb4009130 tx=0x7f0eb4015000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.218+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0ebc0488f0 con 0x7f0ec40c7760 2023-05-16T07:19:49.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.381+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ec4047150 con 0x7f0ec40c7760 2023-05-16T07:19:49.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.382+0000 7f0eb27fc700 1 -- 172.21.15.92:0/1296955436 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17931 (secure 0 0 0) 0x7f0ec40c9980 con 0x7f0ec40c7760 2023-05-16T07:19:49.386 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:49.386 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":134,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:49.065058+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:47.601645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:48.023978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:49.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0ea405ed90 msgr2=0x7f0ea4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0ea405ed90 0x7f0ea4061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0eb4009130 tx=0x7f0eb4015000).stop 2023-05-16T07:19:49.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 msgr2=0x7f0ec40adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 0x7f0ec40adbc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f0ebc00c810 tx=0x7f0ebc002560).stop 2023-05-16T07:19:49.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 shutdown_connections 2023-05-16T07:19:49.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f0ea405ed90 0x7f0ea4061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec40c7760 0x7f0ec40adbc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ec40ad040 0x7f0ec4168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 --2- 172.21.15.92:0/1296955436 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ec40c7380 0x7f0ec414e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 >> 172.21.15.92:0/1296955436 conn(0x7f0ec4068330 msgr2=0x7f0ec414f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:49.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 shutdown_connections 2023-05-16T07:19:49.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.384+0000 7f0ec359e700 1 -- 172.21.15.92:0/1296955436 wait complete. 2023-05-16T07:19:49.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.507+0000 7fb25b59e700 1 Processor -- start 2023-05-16T07:19:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.507+0000 7fb25b59e700 1 -- start start 2023-05-16T07:19:49.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb25b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb25c154e50 0x7fb25c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb25b59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb25c0ad040 0x7fb25c14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb25b59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c158970 con 0x7fb25c0c7380 2023-05-16T07:19:49.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb25b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c155300 con 0x7fb25c154e50 2023-05-16T07:19:49.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb25b59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c157d60 con 0x7fb25c0ad040 2023-05-16T07:19:49.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb25c154e50 0x7fb25c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb25c154e50 0x7fb25c0acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50064/0 (socket says 172.21.15.92:50064) 2023-05-16T07:19:49.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 -- 172.21.15.92:0/3195925630 learned_addr learned my addr 172.21.15.92:0/3195925630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:49.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb25ad9d700 1 --2- 172.21.15.92:0/3195925630 >> v2:172.21.15.158:3300/0 conn(0x7fb25c0ad040 0x7fb25c14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 -- 172.21.15.92:0/3195925630 >> v2:172.21.15.158:3300/0 conn(0x7fb25c0ad040 msgr2=0x7fb25c14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:49.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 --2- 172.21.15.92:0/3195925630 >> v2:172.21.15.158:3300/0 conn(0x7fb25c0ad040 0x7fb25c14dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 -- 172.21.15.92:0/3195925630 >> v1:172.21.15.158:6789/0 conn(0x7fb25c0c7380 legacy=0x7fb25c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 -- 172.21.15.92:0/3195925630 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c158040 con 0x7fb25c154e50 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb253fff700 1 --2- 172.21.15.92:0/3195925630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c154e50 0x7fb25c0acb50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fb25c05eea0 tx=0x7fb2440090f0).ready entity=mon.0 client_cookie=939a82e6bba86eb9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb259d9b700 1 -- 172.21.15.92:0/3195925630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24400b600 con 0x7fb25c154e50 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb259d9b700 1 -- 172.21.15.92:0/3195925630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24400b760 con 0x7fb25c154e50 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.508+0000 7fb259d9b700 1 -- 172.21.15.92:0/3195925630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24400bae0 con 0x7fb25c154e50 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 -- 172.21.15.92:0/3195925630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c154e50 msgr2=0x7fb25c0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 --2- 172.21.15.92:0/3195925630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c154e50 0x7fb25c0acb50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fb25c05eea0 tx=0x7fb2440090f0).stop 2023-05-16T07:19:49.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 -- 172.21.15.92:0/3195925630 shutdown_connections 2023-05-16T07:19:49.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 --2- 172.21.15.92:0/3195925630 >> v2:172.21.15.158:3300/0 conn(0x7fb25c0ad040 0x7fb25c14dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 --2- 172.21.15.92:0/3195925630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c154e50 0x7fb25c0acb50 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 -- 172.21.15.92:0/3195925630 >> 172.21.15.92:0/3195925630 conn(0x7fb25c068330 msgr2=0x7fb25c068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:49.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 -- 172.21.15.92:0/3195925630 shutdown_connections 2023-05-16T07:19:49.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.509+0000 7fb25b59e700 1 -- 172.21.15.92:0/3195925630 wait complete. 2023-05-16T07:19:49.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.510+0000 7fb25b59e700 1 Processor -- start 2023-05-16T07:19:49.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.510+0000 7fb25b59e700 1 -- start start 2023-05-16T07:19:49.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 0x7fb25c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb25c0c7380 0x7fb25c0c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb25c154e50 0x7fb25c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0c8cf0 con 0x7fb25c0ad040 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0c39a0 con 0x7fb25c0c7380 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0c3c80 con 0x7fb25c154e50 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 0x7fb25c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 0x7fb25c14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50068/0 (socket says 172.21.15.92:50068) 2023-05-16T07:19:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 -- 172.21.15.92:0/583137963 learned_addr learned my addr 172.21.15.92:0/583137963 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb253fff700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb25c0c7380 0x7fb25c0c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25ad9d700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb25c154e50 0x7fb25c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 -- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb25c154e50 msgr2=0x7fb25c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb25c154e50 0x7fb25c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 -- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb25c0c7380 msgr2=0x7fb25c0c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb25c0c7380 0x7fb25c0c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 -- 172.21.15.92:0/583137963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb244009370 con 0x7fb25c0ad040 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25ad9d700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb25c154e50 0x7fb25c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb253fff700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb25c0c7380 0x7fb25c0c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25a59c700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 0x7fb25c14e160 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fb24c00c4c0 tx=0x7fb24c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c007020 con 0x7fb25c0ad040 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c001e90 con 0x7fb25c0ad040 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c0c3f60 con 0x7fb25c0ad040 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0073b0 con 0x7fb25c0ad040 2023-05-16T07:19:49.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.511+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c14eb60 con 0x7fb25c0ad040 2023-05-16T07:19:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.512+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c047150 con 0x7fb25c0ad040 2023-05-16T07:19:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.513+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb24c010070 con 0x7fb25c0ad040 2023-05-16T07:19:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.513+0000 7fb2527fc700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb23c05ed90 0x7fb23c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.513+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fb24c0795f0 con 0x7fb25c0ad040 2023-05-16T07:19:49.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.513+0000 7fb253fff700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb23c05ed90 0x7fb23c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.514+0000 7fb253fff700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb23c05ed90 0x7fb23c061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb244009130 tx=0x7fb244017000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.517+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb24c04a930 con 0x7fb25c0ad040 2023-05-16T07:19:49.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.682+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb25c0bfb90 con 0x7fb25c0ad040 2023-05-16T07:19:49.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.683+0000 7fb2527fc700 1 -- 172.21.15.92:0/583137963 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17931 (secure 0 0 0) 0x7fb24c04df50 con 0x7fb25c0ad040 2023-05-16T07:19:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:49.687 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":134,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:49.065058+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:47.601645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:48.023978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb23c05ed90 msgr2=0x7fb23c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb23c05ed90 0x7fb23c061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb244009130 tx=0x7fb244017000).stop 2023-05-16T07:19:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 msgr2=0x7fb25c14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 0x7fb25c14e160 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fb24c00c4c0 tx=0x7fb24c007e00).stop 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 shutdown_connections 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb23c05ed90 0x7fb23c061250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb25c154e50 0x7fb25c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb25c0ad040 0x7fb25c14e160 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 --2- 172.21.15.92:0/583137963 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb25c0c7380 0x7fb25c0c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 >> 172.21.15.92:0/583137963 conn(0x7fb25c068330 msgr2=0x7fb25c14f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 shutdown_connections 2023-05-16T07:19:49.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.685+0000 7fb25b59e700 1 -- 172.21.15.92:0/583137963 wait complete. 2023-05-16T07:19:49.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:19:49.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.813+0000 7fb0908e1700 1 Processor -- start 2023-05-16T07:19:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.813+0000 7fb0908e1700 1 -- start start 2023-05-16T07:19:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb0908e1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb08c153f90 0x7fb08c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb0908e1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb08c0c5d00 0x7fb08c0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb0908e1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0c66a0 con 0x7fb08c0c0150 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb0908e1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c069f60 con 0x7fb08c0c5d00 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb0908e1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c06a9e0 con 0x7fb08c153f90 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb08c0c5d00 0x7fb08c0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb08c0c5d00 0x7fb08c0c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50078/0 (socket says 172.21.15.92:50078) 2023-05-16T07:19:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 -- 172.21.15.92:0/3195274879 learned_addr learned my addr 172.21.15.92:0/3195274879 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08ad9d700 1 --2- 172.21.15.92:0/3195274879 >> v2:172.21.15.158:3300/0 conn(0x7fb08c153f90 0x7fb08c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 -- 172.21.15.92:0/3195274879 >> v2:172.21.15.158:3300/0 conn(0x7fb08c153f90 msgr2=0x7fb08c0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 --2- 172.21.15.92:0/3195274879 >> v2:172.21.15.158:3300/0 conn(0x7fb08c153f90 0x7fb08c0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 -- 172.21.15.92:0/3195274879 >> v1:172.21.15.158:6789/0 conn(0x7fb08c0c0150 legacy=0x7fb08c0c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 -- 172.21.15.92:0/3195274879 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c06a140 con 0x7fb08c0c5d00 2023-05-16T07:19:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08ad9d700 1 --2- 172.21.15.92:0/3195274879 >> v2:172.21.15.158:3300/0 conn(0x7fb08c153f90 0x7fb08c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.814+0000 7fb08259c700 1 --2- 172.21.15.92:0/3195274879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c5d00 0x7fb08c0c61b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7fb08c0caa40 tx=0x7fb0740090f0).ready entity=mon.0 client_cookie=453422f62ef74ab1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb08a59c700 1 -- 172.21.15.92:0/3195274879 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07400b5a0 con 0x7fb08c0c5d00 2023-05-16T07:19:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb08a59c700 1 -- 172.21.15.92:0/3195274879 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07400b700 con 0x7fb08c0c5d00 2023-05-16T07:19:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb08a59c700 1 -- 172.21.15.92:0/3195274879 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07400ba80 con 0x7fb08c0c5d00 2023-05-16T07:19:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 -- 172.21.15.92:0/3195274879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c5d00 msgr2=0x7fb08c0c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 --2- 172.21.15.92:0/3195274879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c5d00 0x7fb08c0c61b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7fb08c0caa40 tx=0x7fb0740090f0).stop 2023-05-16T07:19:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 -- 172.21.15.92:0/3195274879 shutdown_connections 2023-05-16T07:19:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 --2- 172.21.15.92:0/3195274879 >> v2:172.21.15.158:3300/0 conn(0x7fb08c153f90 0x7fb08c0a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 --2- 172.21.15.92:0/3195274879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c5d00 0x7fb08c0c61b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 -- 172.21.15.92:0/3195274879 >> 172.21.15.92:0/3195274879 conn(0x7fb08c0cb600 msgr2=0x7fb08c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 -- 172.21.15.92:0/3195274879 shutdown_connections 2023-05-16T07:19:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.815+0000 7fb0908e1700 1 -- 172.21.15.92:0/3195274879 wait complete. 2023-05-16T07:19:49.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.816+0000 7fb0908e1700 1 Processor -- start 2023-05-16T07:19:49.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.816+0000 7fb0908e1700 1 -- start start 2023-05-16T07:19:49.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb0908e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 0x7fb08c14b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb0908e1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb08c0c5d00 0x7fb08c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb0908e1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb08c153f90 0x7fb08c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb0908e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0ae900 con 0x7fb08c0c0150 2023-05-16T07:19:49.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb0908e1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0ac390 con 0x7fb08c153f90 2023-05-16T07:19:49.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb0908e1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0aff30 con 0x7fb08c0c5d00 2023-05-16T07:19:49.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 0x7fb08c14b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 0x7fb08c14b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50082/0 (socket says 172.21.15.92:50082) 2023-05-16T07:19:49.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 -- 172.21.15.92:0/2489478057 learned_addr learned my addr 172.21.15.92:0/2489478057 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08b59e700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb08c153f90 0x7fb08c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08259c700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb08c0c5d00 0x7fb08c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 -- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb08c0c5d00 msgr2=0x7fb08c0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb08c0c5d00 0x7fb08c0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 -- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb08c153f90 msgr2=0x7fb08c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb08c153f90 0x7fb08c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08ad9d700 1 -- 172.21.15.92:0/2489478057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009370 con 0x7fb08c0c0150 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.817+0000 7fb08b59e700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb08c153f90 0x7fb08c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.818+0000 7fb08ad9d700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 0x7fb08c14b770 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7fb07c00c880 tx=0x7fb07c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.818+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c010580 con 0x7fb08c0c0150 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.818+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c0106e0 con 0x7fb08c0c0150 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.818+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c0b0160 con 0x7fb08c0c0150 2023-05-16T07:19:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.818+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c010a80 con 0x7fb08c0c0150 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.818+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c0b0720 con 0x7fb08c0c0150 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.819+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c150320 con 0x7fb08c0c0150 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.819+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb07c01b460 con 0x7fb08c0c0150 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.819+0000 7fb0837fe700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb06c05ed90 0x7fb06c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.820+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fb07c079dd0 con 0x7fb08c0c0150 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.820+0000 7fb08259c700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb06c05ed90 0x7fb06c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.820+0000 7fb08259c700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb06c05ed90 0x7fb06c061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb074009130 tx=0x7fb074015000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:49.824+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb07c04b110 con 0x7fb08c0c0150 2023-05-16T07:19:50.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.028+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb08c083120 con 0x7fb08c0c0150 2023-05-16T07:19:50.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.071+0000 7fb0837fe700 1 -- 172.21.15.92:0/2489478057 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb07c04e730 con 0x7fb08c0c0150 2023-05-16T07:19:50.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb06c05ed90 msgr2=0x7fb06c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb06c05ed90 0x7fb06c061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb074009130 tx=0x7fb074015000).stop 2023-05-16T07:19:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 msgr2=0x7fb08c14b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 0x7fb08c14b770 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7fb07c00c880 tx=0x7fb07c008040).stop 2023-05-16T07:19:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 shutdown_connections 2023-05-16T07:19:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb06c05ed90 0x7fb06c061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb08c153f90 0x7fb08c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb08c0c0150 0x7fb08c14b770 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 --2- 172.21.15.92:0/2489478057 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb08c0c5d00 0x7fb08c0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 >> 172.21.15.92:0/2489478057 conn(0x7fb08c0cb600 msgr2=0x7fb08c0c11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:50.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 shutdown_connections 2023-05-16T07:19:50.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.073+0000 7fb0908e1700 1 -- 172.21.15.92:0/2489478057 wait complete. 2023-05-16T07:19:50.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:19:50.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.199+0000 7f17988f5700 1 Processor -- start 2023-05-16T07:19:50.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.199+0000 7f17988f5700 1 -- start start 2023-05-16T07:19:50.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f17988f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f179414e6f0 0x7f1794155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:50.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f17988f5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f17940629c0 0x7f1794063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:50.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f17988f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179414ead0 con 0x7f17940c4d40 2023-05-16T07:19:50.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f17988f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1794154cb0 con 0x7f179414e6f0 2023-05-16T07:19:50.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f17988f5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17940c6020 con 0x7f17940629c0 2023-05-16T07:19:50.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f179259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f179414e6f0 0x7f1794155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:50.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f1791d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f17940c4d40 0x7f17940622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60836/0 (socket says 172.21.15.92:60836) 2023-05-16T07:19:50.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.200+0000 7f1791d9b700 1 -- 172.21.15.92:0/2646255904 learned_addr learned my addr 172.21.15.92:0/2646255904 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f1792d9d700 1 --2- 172.21.15.92:0/2646255904 >> v2:172.21.15.158:3300/0 conn(0x7f17940629c0 0x7f1794063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179259c700 1 -- 172.21.15.92:0/2646255904 >> v2:172.21.15.158:3300/0 conn(0x7f17940629c0 msgr2=0x7f1794063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179259c700 1 --2- 172.21.15.92:0/2646255904 >> v2:172.21.15.158:3300/0 conn(0x7f17940629c0 0x7f1794063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179259c700 1 -- 172.21.15.92:0/2646255904 >> v1:172.21.15.92:6789/0 conn(0x7f17940c4d40 legacy=0x7f17940622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179159a700 1 -- 172.21.15.92:0/2646255904 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292824973 0 0) 0x7f179414ead0 con 0x7f17940c4d40 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179259c700 1 -- 172.21.15.92:0/2646255904 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794154660 con 0x7f179414e6f0 2023-05-16T07:19:50.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179259c700 1 --2- 172.21.15.92:0/2646255904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179414e6f0 0x7f1794155140 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f17940647e0 tx=0x7f1784009780).ready entity=mon.0 client_cookie=7857f90fcde2f250 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:50.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179159a700 1 -- 172.21.15.92:0/2646255904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400e070 con 0x7f179414e6f0 2023-05-16T07:19:50.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179159a700 1 -- 172.21.15.92:0/2646255904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400bc30 con 0x7f179414e6f0 2023-05-16T07:19:50.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f179159a700 1 -- 172.21.15.92:0/2646255904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784012410 con 0x7f179414e6f0 2023-05-16T07:19:50.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f17988f5700 1 -- 172.21.15.92:0/2646255904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179414e6f0 msgr2=0x7f1794155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:50.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f17988f5700 1 --2- 172.21.15.92:0/2646255904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179414e6f0 0x7f1794155140 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f17940647e0 tx=0x7f1784009780).stop 2023-05-16T07:19:50.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f17988f5700 1 -- 172.21.15.92:0/2646255904 shutdown_connections 2023-05-16T07:19:50.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.201+0000 7f17988f5700 1 --2- 172.21.15.92:0/2646255904 >> v2:172.21.15.158:3300/0 conn(0x7f17940629c0 0x7f1794063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.202+0000 7f17988f5700 1 --2- 172.21.15.92:0/2646255904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179414e6f0 0x7f1794155140 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.202+0000 7f17988f5700 1 -- 172.21.15.92:0/2646255904 >> 172.21.15.92:0/2646255904 conn(0x7f179405e820 msgr2=0x7f17940c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:50.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.202+0000 7f17988f5700 1 -- 172.21.15.92:0/2646255904 shutdown_connections 2023-05-16T07:19:50.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.202+0000 7f17988f5700 1 -- 172.21.15.92:0/2646255904 wait complete. 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 Processor -- start 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 -- start start 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f17940629c0 0x7f17941515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 0x7f179414eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f179414e6f0 0x7f179414f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17940aea20 con 0x7f17940c4d40 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17940adda0 con 0x7f179414e6f0 2023-05-16T07:19:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f17988f5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17940ae080 con 0x7f17940629c0 2023-05-16T07:19:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f1791d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 0x7f179414eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f1791d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 0x7f179414eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50094/0 (socket says 172.21.15.92:50094) 2023-05-16T07:19:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f1791d9b700 1 -- 172.21.15.92:0/851210337 learned_addr learned my addr 172.21.15.92:0/851210337 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.203+0000 7f1792d9d700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f179414e6f0 0x7f179414f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f179259c700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f17940629c0 0x7f17941515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f1791d9b700 1 -- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f17940629c0 msgr2=0x7f17941515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f1791d9b700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f17940629c0 0x7f17941515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f1791d9b700 1 -- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f179414e6f0 msgr2=0x7f179414f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f1791d9b700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f179414e6f0 0x7f179414f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f1791d9b700 1 -- 172.21.15.92:0/851210337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009a20 con 0x7f17940c4d40 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f1791d9b700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 0x7f179414eb80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f177c00c4e0 tx=0x7f177c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0072e0 con 0x7f17940c4d40 2023-05-16T07:19:50.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c007440 con 0x7f17940c4d40 2023-05-16T07:19:50.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17940ae360 con 0x7f17940c4d40 2023-05-16T07:19:50.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0057a0 con 0x7f17940c4d40 2023-05-16T07:19:50.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.204+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179414f070 con 0x7f17940c4d40 2023-05-16T07:19:50.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.205+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179406c350 con 0x7f17940c4d40 2023-05-16T07:19:50.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.205+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f177c01a060 con 0x7f17940c4d40 2023-05-16T07:19:50.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.206+0000 7f178a7fc700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f177405ed90 0x7f1774061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:50.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.206+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f177c0794e0 con 0x7f17940c4d40 2023-05-16T07:19:50.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.206+0000 7f179259c700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f177405ed90 0x7f1774061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:50.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.206+0000 7f179259c700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f177405ed90 0x7f1774061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1784009f90 tx=0x7f1784009780).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:50.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.210+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f177c04a820 con 0x7f17940c4d40 2023-05-16T07:19:50.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:50.412+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f17940c66a0 con 0x7f17940c4d40 2023-05-16T07:19:51.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.080+0000 7f178a7fc700 1 -- 172.21.15.92:0/851210337 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7f177c018020 con 0x7f17940c4d40 2023-05-16T07:19:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f177405ed90 msgr2=0x7f1774061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f177405ed90 0x7f1774061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1784009f90 tx=0x7f1784009780).stop 2023-05-16T07:19:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 msgr2=0x7f179414eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 0x7f179414eb80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f177c00c4e0 tx=0x7f177c007c70).stop 2023-05-16T07:19:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 shutdown_connections 2023-05-16T07:19:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f177405ed90 0x7f1774061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f17940629c0 0x7f17941515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f179414e6f0 0x7f179414f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 --2- 172.21.15.92:0/851210337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17940c4d40 0x7f179414eb80 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 >> 172.21.15.92:0/851210337 conn(0x7f179405e820 msgr2=0x7f1794068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 shutdown_connections 2023-05-16T07:19:51.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.082+0000 7f17988f5700 1 -- 172.21.15.92:0/851210337 wait complete. 2023-05-16T07:19:51.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.209+0000 7f1b56883700 1 Processor -- start 2023-05-16T07:19:51.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.209+0000 7f1b56883700 1 -- start start 2023-05-16T07:19:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b56883700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1b50154170 0x7f1b500c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b56883700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b500c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b56883700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b500c0660 con 0x7f1b50154550 2023-05-16T07:19:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b56883700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500c1b00 con 0x7f1b50154170 2023-05-16T07:19:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b56883700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500bfd30 con 0x7f1b50069db0 2023-05-16T07:19:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b47fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b50154550 0x7f1b500696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60850/0 (socket says 172.21.15.92:60850) 2023-05-16T07:19:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b47fff700 1 -- 172.21.15.92:0/4009030722 learned_addr learned my addr 172.21.15.92:0/4009030722 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4ffff700 1 --2- 172.21.15.92:0/4009030722 >> v2:172.21.15.158:3300/0 conn(0x7f1b50154170 0x7f1b500c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495384518 0 0) 0x7f1b500c0660 con 0x7f1b50154550 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b3c0033f0 con 0x7f1b50154550 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b54e20700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b500c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482466130 0 0) 0x7f1b3c0033f0 con 0x7f1b50154550 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 msgr2=0x7f1b500c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b500c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 >> v2:172.21.15.158:3300/0 conn(0x7f1b50154170 msgr2=0x7f1b500c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 --2- 172.21.15.92:0/4009030722 >> v2:172.21.15.158:3300/0 conn(0x7f1b50154170 0x7f1b500c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b500c0010 con 0x7f1b50154550 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f1b38001470 con 0x7f1b50154550 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 >> v1:172.21.15.92:6789/0 conn(0x7f1b50154550 legacy=0x7f1b500696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b3c0038e0 0x7f1b3c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b3c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b3c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500c1b00 con 0x7f1b3c004690 2023-05-16T07:19:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500bfd30 con 0x7f1b3c0038e0 2023-05-16T07:19:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c00b8e0 con 0x7f1b50069db0 2023-05-16T07:19:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.210+0000 7f1b47fff700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b3c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b54e20700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b3c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b4ffff700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b3c0038e0 0x7f1b3c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b47fff700 1 -- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 msgr2=0x7f1b3c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b47fff700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b3c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b47fff700 1 -- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b3c0038e0 msgr2=0x7f1b3c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b47fff700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b3c0038e0 0x7f1b3c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b47fff700 1 -- 172.21.15.92:0/4009030722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c0033f0 con 0x7f1b3c004690 2023-05-16T07:19:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b54e20700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50069db0 0x7f1b3c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b4ffff700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b3c0038e0 0x7f1b3c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b47fff700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b3c004b10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f1b3800c1e0 tx=0x7f1b38001e90).ready entity=mon.0 client_cookie=f42d341e36c078eb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007a60 con 0x7f1b3c004690 2023-05-16T07:19:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3800cab0 con 0x7f1b3c004690 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b4f7fe700 1 -- 172.21.15.92:0/4009030722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800ce30 con 0x7f1b3c004690 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b56883700 1 -- 172.21.15.92:0/4009030722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 msgr2=0x7f1b3c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b56883700 1 --2- 172.21.15.92:0/4009030722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b3c004b10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f1b3800c1e0 tx=0x7f1b38001e90).stop 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.211+0000 7f1b54e20700 1 -- 172.21.15.92:0/4009030722 reap_dead start 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.212+0000 7f1b56883700 1 -- 172.21.15.92:0/4009030722 shutdown_connections 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.212+0000 7f1b56883700 1 -- 172.21.15.92:0/4009030722 >> 172.21.15.92:0/4009030722 conn(0x7f1b500cb700 msgr2=0x7f1b500c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:51.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.212+0000 7f1b56883700 1 -- 172.21.15.92:0/4009030722 shutdown_connections 2023-05-16T07:19:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.212+0000 7f1b56883700 1 -- 172.21.15.92:0/4009030722 wait complete. 2023-05-16T07:19:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 Processor -- start 2023-05-16T07:19:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 -- start start 2023-05-16T07:19:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b50152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50154960 0x7f1b500c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b50156270 0x7f1b501566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50154de0 con 0x7f1b3c004690 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50155c00 con 0x7f1b50156270 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b56883700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500c5020 con 0x7f1b50154960 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.213+0000 7f1b4ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b50152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b50152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50114/0 (socket says 172.21.15.92:50114) 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 -- 172.21.15.92:0/1851458332 learned_addr learned my addr 172.21.15.92:0/1851458332 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b47fff700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50154960 0x7f1b500c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b54e20700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b50156270 0x7f1b501566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 -- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50154960 msgr2=0x7f1b500c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50154960 0x7f1b500c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 -- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b50156270 msgr2=0x7f1b501566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b50156270 0x7f1b501566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 -- 172.21.15.92:0/1851458332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50156be0 con 0x7f1b3c004690 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4ffff700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b50152890 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f1b40010ad0 tx=0x7f1b40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000e8c0 con 0x7f1b3c004690 2023-05-16T07:19:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000ea20 con 0x7f1b3c004690 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b50154ff0 con 0x7f1b3c004690 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000eda0 con 0x7f1b3c004690 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.214+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b501555b0 con 0x7f1b3c004690 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.215+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b500666c0 con 0x7f1b3c004690 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.215+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1b40024070 con 0x7f1b3c004690 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.216+0000 7f1b4cff9700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b3005ed90 0x7f1b30061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.216+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f1b4007a270 con 0x7f1b3c004690 2023-05-16T07:19:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.216+0000 7f1b47fff700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b3005ed90 0x7f1b30061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.217+0000 7f1b47fff700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b3005ed90 0x7f1b30061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1b500ad920 tx=0x7f1b38015000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.220+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1b4004b5b0 con 0x7f1b3c004690 2023-05-16T07:19:51.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-05-16T07:19:51.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-05-16T07:19:51.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.421+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b5006c350 con 0x7f1b3c004690 2023-05-16T07:19:51.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.422+0000 7f1b4cff9700 1 -- 172.21.15.92:0/1851458332 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f1b4004ebd0 con 0x7f1b3c004690 2023-05-16T07:19:51.426 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:51.426 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:49.065391+0000","modified":"2023-05-16T07:19:51.075477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2186588149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2186588149},{"type":"v1","addr":"172.21.15.92:6835","nonce":2186588149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-05-16T07:19:51.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b3005ed90 msgr2=0x7f1b30061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b3005ed90 0x7f1b30061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1b500ad920 tx=0x7f1b38015000).stop 2023-05-16T07:19:51.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 msgr2=0x7f1b50152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b50152890 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f1b40010ad0 tx=0x7f1b40008040).stop 2023-05-16T07:19:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 shutdown_connections 2023-05-16T07:19:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f1b3005ed90 0x7f1b30061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b50156270 0x7f1b501566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b3c004690 0x7f1b50152890 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 --2- 172.21.15.92:0/1851458332 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b50154960 0x7f1b500c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 >> 172.21.15.92:0/1851458332 conn(0x7f1b500cb700 msgr2=0x7f1b500abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 shutdown_connections 2023-05-16T07:19:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.423+0000 7f1b56883700 1 -- 172.21.15.92:0/1851458332 wait complete. 2023-05-16T07:19:51.430 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 49 2023-05-16T07:19:51.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:19:51.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.546+0000 7fafe5d9f700 1 Processor -- start 2023-05-16T07:19:51.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.546+0000 7fafe5d9f700 1 -- start start 2023-05-16T07:19:51.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe5d9f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fafe00a7e00 0x7fafe0156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe5d9f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe00c8750 con 0x7fafe01517f0 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe5d9f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe0155d70 con 0x7fafe0151bd0 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe5d9f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe00c8470 con 0x7fafe00a7e00 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe559e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fafe00a7e00 0x7fafe0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe4d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fafe01517f0 0x7fafe0152a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60870/0 (socket says 172.21.15.92:60870) 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe4d9d700 1 -- 172.21.15.92:0/3217444790 learned_addr learned my addr 172.21.15.92:0/3217444790 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe559e700 1 -- 172.21.15.92:0/3217444790 >> v1:172.21.15.158:6789/0 conn(0x7fafe0151bd0 legacy=0x7fafe00a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe559e700 1 -- 172.21.15.92:0/3217444790 >> v1:172.21.15.92:6789/0 conn(0x7fafe01517f0 legacy=0x7fafe0152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafdf7fe700 1 -- 172.21.15.92:0/3217444790 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782841201 0 0) 0x7fafe00c8750 con 0x7fafe01517f0 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe559e700 1 -- 172.21.15.92:0/3217444790 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe0155720 con 0x7fafe00a7e00 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafe559e700 1 --2- 172.21.15.92:0/3217444790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe0156200 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fafd8009e60 tx=0x7fafd8009780).ready entity=mon.0 client_cookie=f056da9f6afe9945 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafdf7fe700 1 -- 172.21.15.92:0/3217444790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd800e070 con 0x7fafe00a7e00 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafdf7fe700 1 -- 172.21.15.92:0/3217444790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd800bbf0 con 0x7fafe00a7e00 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.547+0000 7fafdf7fe700 1 -- 172.21.15.92:0/3217444790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8012410 con 0x7fafe00a7e00 2023-05-16T07:19:51.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 -- 172.21.15.92:0/3217444790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 msgr2=0x7fafe0156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/3217444790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe0156200 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fafd8009e60 tx=0x7fafd8009780).stop 2023-05-16T07:19:51.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 -- 172.21.15.92:0/3217444790 shutdown_connections 2023-05-16T07:19:51.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/3217444790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe0156200 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 -- 172.21.15.92:0/3217444790 >> 172.21.15.92:0/3217444790 conn(0x7fafe005e820 msgr2=0x7fafe00622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:51.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 -- 172.21.15.92:0/3217444790 shutdown_connections 2023-05-16T07:19:51.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.548+0000 7fafe5d9f700 1 -- 172.21.15.92:0/3217444790 wait complete. 2023-05-16T07:19:51.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.549+0000 7fafe5d9f700 1 Processor -- start 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.549+0000 7fafe5d9f700 1 -- start start 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe5d9f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe5d9f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe01517f0 0x7fafe0152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe5d9f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe0151bd0 0x7fafe0161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe5d9f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe014d980 con 0x7fafe00a7e00 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe5d9f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe014dc60 con 0x7fafe0151bd0 2023-05-16T07:19:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe5d9f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe014df40 con 0x7fafe01517f0 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe014b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50130/0 (socket says 172.21.15.92:50130) 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 -- 172.21.15.92:0/317480645 learned_addr learned my addr 172.21.15.92:0/317480645 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe559e700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe0151bd0 0x7fafe0161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafdffff700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe01517f0 0x7fafe0152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 -- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe01517f0 msgr2=0x7fafe0152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe01517f0 0x7fafe0152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 -- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe0151bd0 msgr2=0x7fafe0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe0151bd0 0x7fafe0161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 -- 172.21.15.92:0/317480645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd8009a20 con 0x7fafe00a7e00 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe559e700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe0151bd0 0x7fafe0161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.550+0000 7fafe4d9d700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe014b400 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fafd4001360 tx=0x7fafd4007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.551+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40072e0 con 0x7fafe00a7e00 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.551+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd4007440 con 0x7fafe00a7e00 2023-05-16T07:19:51.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.551+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe014e220 con 0x7fafe00a7e00 2023-05-16T07:19:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.551+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40058a0 con 0x7fafe00a7e00 2023-05-16T07:19:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.551+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe014b8f0 con 0x7fafe00a7e00 2023-05-16T07:19:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.552+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe014e740 con 0x7fafe00a7e00 2023-05-16T07:19:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.552+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fafd401a060 con 0x7fafe00a7e00 2023-05-16T07:19:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.552+0000 7fafdcff9700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafc405ed90 0x7fafc4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.552+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fafd4079570 con 0x7fafe00a7e00 2023-05-16T07:19:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.553+0000 7fafdffff700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafc405ed90 0x7fafc4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.553+0000 7fafdffff700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafc405ed90 0x7fafc4061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fafe00aac80 tx=0x7fafcc008040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.556+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fafd404a8b0 con 0x7fafe00a7e00 2023-05-16T07:19:51.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.720+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafe0047150 con 0x7fafe00a7e00 2023-05-16T07:19:51.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.721+0000 7fafdcff9700 1 -- 172.21.15.92:0/317480645 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17918 (secure 0 0 0) 0x7fafd4018020 con 0x7fafe00a7e00 2023-05-16T07:19:51.726 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:51.726 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":135,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:19:50.067047+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:47.601645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:48.023978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:19:51.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafc405ed90 msgr2=0x7fafc4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafc405ed90 0x7fafc4061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fafe00aac80 tx=0x7fafcc008040).stop 2023-05-16T07:19:51.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 msgr2=0x7fafe014b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe014b400 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fafd4001360 tx=0x7fafd4007b00).stop 2023-05-16T07:19:51.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 shutdown_connections 2023-05-16T07:19:51.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fafc405ed90 0x7fafc4061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafe0151bd0 0x7fafe0161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafe00a7e00 0x7fafe014b400 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 --2- 172.21.15.92:0/317480645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fafe01517f0 0x7fafe0152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.722+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 >> 172.21.15.92:0/317480645 conn(0x7fafe005e820 msgr2=0x7fafe00ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:51.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.723+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 shutdown_connections 2023-05-16T07:19:51.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.723+0000 7fafe5d9f700 1 -- 172.21.15.92:0/317480645 wait complete. 2023-05-16T07:19:51.739 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:19:51.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:19:51.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.849+0000 7f2afaa5a700 1 Processor -- start 2023-05-16T07:19:51.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.849+0000 7f2afaa5a700 1 -- start start 2023-05-16T07:19:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2afaa5a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2af4069db0 0x7f2af415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2afaa5a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40c0660 con 0x7f2af4154550 2023-05-16T07:19:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2afaa5a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40c1b00 con 0x7f2af4154170 2023-05-16T07:19:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2afaa5a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40bfd30 con 0x7f2af4069db0 2023-05-16T07:19:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2aebfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2af4154550 0x7f2af40696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60886/0 (socket says 172.21.15.92:60886) 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2aebfff700 1 -- 172.21.15.92:0/2648984243 learned_addr learned my addr 172.21.15.92:0/2648984243 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2af8ff7700 1 --2- 172.21.15.92:0/2648984243 >> v2:172.21.15.92:3300/0 conn(0x7f2af4069db0 0x7f2af415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.850+0000 7f2af37fe700 1 -- 172.21.15.92:0/2648984243 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400024333 0 0) 0x7f2af40c0660 con 0x7f2af4154550 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af37fe700 1 -- 172.21.15.92:0/2648984243 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae00033f0 con 0x7f2af4154550 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af8ff7700 1 -- 172.21.15.92:0/2648984243 >> v1:172.21.15.158:6789/0 conn(0x7f2af4154170 legacy=0x7f2af40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af8ff7700 1 -- 172.21.15.92:0/2648984243 >> v1:172.21.15.92:6789/0 conn(0x7f2af4154550 legacy=0x7f2af40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af8ff7700 1 -- 172.21.15.92:0/2648984243 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40c0010 con 0x7f2af4069db0 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af8ff7700 1 --2- 172.21.15.92:0/2648984243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4069db0 0x7f2af415f200 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f2aec009e60 tx=0x7f2aec009780).ready entity=mon.0 client_cookie=d4a304c0e2918d64 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af37fe700 1 -- 172.21.15.92:0/2648984243 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00e070 con 0x7f2af4069db0 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af37fe700 1 -- 172.21.15.92:0/2648984243 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec00bb60 con 0x7f2af4069db0 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2af37fe700 1 -- 172.21.15.92:0/2648984243 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec012410 con 0x7f2af4069db0 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2648984243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4069db0 msgr2=0x7f2af415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2648984243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4069db0 0x7f2af415f200 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f2aec009e60 tx=0x7f2aec009780).stop 2023-05-16T07:19:51.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2648984243 shutdown_connections 2023-05-16T07:19:51.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2648984243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4069db0 0x7f2af415f200 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2648984243 >> 172.21.15.92:0/2648984243 conn(0x7f2af40cb700 msgr2=0x7f2af40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:51.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2648984243 shutdown_connections 2023-05-16T07:19:51.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.851+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2648984243 wait complete. 2023-05-16T07:19:51.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 Processor -- start 2023-05-16T07:19:51.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 -- start start 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2af4069db0 0x7f2af41448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 0x7f2af414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2af4154550 0x7f2af414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41451b0 con 0x7f2af4154170 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af414ae70 con 0x7f2af4069db0 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2afaa5a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af414b150 con 0x7f2af4154550 2023-05-16T07:19:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2aebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 0x7f2af414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2aebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 0x7f2af414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50144/0 (socket says 172.21.15.92:50144) 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2aebfff700 1 -- 172.21.15.92:0/2383540749 learned_addr learned my addr 172.21.15.92:0/2383540749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2af3fff700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2af4069db0 0x7f2af41448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2aebfff700 1 -- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2af4154550 msgr2=0x7f2af414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.853+0000 7f2af8ff7700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2af4154550 0x7f2af414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2aebfff700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2af4154550 0x7f2af414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2aebfff700 1 -- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2af4069db0 msgr2=0x7f2af41448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2aebfff700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2af4069db0 0x7f2af41448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:51.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2aebfff700 1 -- 172.21.15.92:0/2383540749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec009a20 con 0x7f2af4154170 2023-05-16T07:19:51.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2af8ff7700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2af4154550 0x7f2af414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:51.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2aebfff700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 0x7f2af414a140 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f2adc00c420 tx=0x7f2adc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0072e0 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc007440 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af414b430 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0057a0 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.854+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af414ab20 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.855+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af40666c0 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.855+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2adc010070 con 0x7f2af4154170 2023-05-16T07:19:51.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.856+0000 7f2af0ff9700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2ad405ed60 0x7f2ad4061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:51.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.856+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f2adc079400 con 0x7f2af4154170 2023-05-16T07:19:51.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.856+0000 7f2af3fff700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2ad405ed60 0x7f2ad4061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:51.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.856+0000 7f2af3fff700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2ad405ed60 0x7f2ad4061220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2ae4003750 tx=0x7f2ae4009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:51.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:51.860+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2adc04a740 con 0x7f2af4154170 2023-05-16T07:19:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.019+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2af406c350 con 0x7f2ad405ed60 2023-05-16T07:19:52.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.030+0000 7f2af0ff9700 1 -- 172.21.15.92:0/2383540749 <== mgr.7159 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361468 (secure 0 0 0) 0x7f2af406c350 con 0x7f2ad405ed60 2023-05-16T07:19:52.037 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:52.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2ad405ed60 msgr2=0x7f2ad4061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2ad405ed60 0x7f2ad4061220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2ae4003750 tx=0x7f2ae4009040).stop 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 msgr2=0x7f2af414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 0x7f2af414a140 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f2adc00c420 tx=0x7f2adc007c50).stop 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 shutdown_connections 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f2ad405ed60 0x7f2ad4061220 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2af4154550 0x7f2af414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2af4069db0 0x7f2af41448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 --2- 172.21.15.92:0/2383540749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af4154170 0x7f2af414a140 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 >> 172.21.15.92:0/2383540749 conn(0x7f2af40cb700 msgr2=0x7f2af40acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 shutdown_connections 2023-05-16T07:19:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.034+0000 7f2afaa5a700 1 -- 172.21.15.92:0/2383540749 wait complete. 2023-05-16T07:19:52.041 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:19:52.049 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:19:50.416764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":173384,"kb_used_data":12936,"kb_used_omap":0,"kb_used_meta":160384,"kb_avail":754801336,"statfs":{"total":773094113280,"available":772916568064,"internally_reserved":0,"allocated":13246464,"data_stored":6949522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":164233216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11159,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.585454"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093174+0000","last_change":"2023-05-16T07:19:49.093174+0000","last_active":"2023-05-16T07:19:49.093174+0000","last_peered":"2023-05-16T07:19:49.093174+0000","last_clean":"2023-05-16T07:19:49.093174+0000","last_became_active":"2023-05-16T07:19:49.092883+0000","last_became_peered":"2023-05-16T07:19:49.092883+0000","last_unstale":"2023-05-16T07:19:49.093174+0000","last_undegraded":"2023-05-16T07:19:49.093174+0000","last_fullsized":"2023-05-16T07:19:49.093174+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"134'3","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089809+0000","last_change":"2023-05-16T07:19:48.711876+0000","last_active":"2023-05-16T07:19:49.089809+0000","last_peered":"2023-05-16T07:19:49.089809+0000","last_clean":"2023-05-16T07:19:49.089809+0000","last_became_active":"2023-05-16T07:19:48.711598+0000","last_became_peered":"2023-05-16T07:19:48.711598+0000","last_unstale":"2023-05-16T07:19:49.089809+0000","last_undegraded":"2023-05-16T07:19:49.089809+0000","last_fullsized":"2023-05-16T07:19:49.089809+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.082353+0000","last_change":"2023-05-16T07:19:49.082353+0000","last_active":"2023-05-16T07:19:49.082353+0000","last_peered":"2023-05-16T07:19:49.082353+0000","last_clean":"2023-05-16T07:19:49.082353+0000","last_became_active":"2023-05-16T07:19:49.082075+0000","last_became_peered":"2023-05-16T07:19:49.082075+0000","last_unstale":"2023-05-16T07:19:49.082353+0000","last_undegraded":"2023-05-16T07:19:49.082353+0000","last_fullsized":"2023-05-16T07:19:49.082353+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719395+0000","last_change":"2023-05-16T07:19:48.714290+0000","last_active":"2023-05-16T07:19:48.719395+0000","last_peered":"2023-05-16T07:19:48.719395+0000","last_clean":"2023-05-16T07:19:48.719395+0000","last_became_active":"2023-05-16T07:19:48.714026+0000","last_became_peered":"2023-05-16T07:19:48.714026+0000","last_unstale":"2023-05-16T07:19:48.719395+0000","last_undegraded":"2023-05-16T07:19:48.719395+0000","last_fullsized":"2023-05-16T07:19:48.719395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.080691+0000","last_change":"2023-05-16T07:19:49.080691+0000","last_active":"2023-05-16T07:19:49.080691+0000","last_peered":"2023-05-16T07:19:49.080691+0000","last_clean":"2023-05-16T07:19:49.080691+0000","last_became_active":"2023-05-16T07:19:49.080486+0000","last_became_peered":"2023-05-16T07:19:49.080486+0000","last_unstale":"2023-05-16T07:19:49.080691+0000","last_undegraded":"2023-05-16T07:19:49.080691+0000","last_fullsized":"2023-05-16T07:19:49.080691+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720955+0000","last_change":"2023-05-16T07:19:48.719963+0000","last_active":"2023-05-16T07:19:48.720955+0000","last_peered":"2023-05-16T07:19:48.720955+0000","last_clean":"2023-05-16T07:19:48.720955+0000","last_became_active":"2023-05-16T07:19:48.719680+0000","last_became_peered":"2023-05-16T07:19:48.719680+0000","last_unstale":"2023-05-16T07:19:48.720955+0000","last_undegraded":"2023-05-16T07:19:48.720955+0000","last_fullsized":"2023-05-16T07:19:48.720955+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086571+0000","last_change":"2023-05-16T07:19:49.086571+0000","last_active":"2023-05-16T07:19:49.086571+0000","last_peered":"2023-05-16T07:19:49.086571+0000","last_clean":"2023-05-16T07:19:49.086571+0000","last_became_active":"2023-05-16T07:19:49.086297+0000","last_became_peered":"2023-05-16T07:19:49.086297+0000","last_unstale":"2023-05-16T07:19:49.086571+0000","last_undegraded":"2023-05-16T07:19:49.086571+0000","last_fullsized":"2023-05-16T07:19:49.086571+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099097+0000","last_change":"2023-05-16T07:19:48.720083+0000","last_active":"2023-05-16T07:19:49.099097+0000","last_peered":"2023-05-16T07:19:49.099097+0000","last_clean":"2023-05-16T07:19:49.099097+0000","last_became_active":"2023-05-16T07:19:48.719666+0000","last_became_peered":"2023-05-16T07:19:48.719666+0000","last_unstale":"2023-05-16T07:19:49.099097+0000","last_undegraded":"2023-05-16T07:19:49.099097+0000","last_fullsized":"2023-05-16T07:19:49.099097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.075151+0000","last_change":"2023-05-16T07:19:49.075151+0000","last_active":"2023-05-16T07:19:49.075151+0000","last_peered":"2023-05-16T07:19:49.075151+0000","last_clean":"2023-05-16T07:19:49.075151+0000","last_became_active":"2023-05-16T07:19:49.074891+0000","last_became_peered":"2023-05-16T07:19:49.074891+0000","last_unstale":"2023-05-16T07:19:49.075151+0000","last_undegraded":"2023-05-16T07:19:49.075151+0000","last_fullsized":"2023-05-16T07:19:49.075151+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.075042+0000","last_change":"2023-05-16T07:19:48.715541+0000","last_active":"2023-05-16T07:19:49.075042+0000","last_peered":"2023-05-16T07:19:49.075042+0000","last_clean":"2023-05-16T07:19:49.075042+0000","last_became_active":"2023-05-16T07:19:48.715256+0000","last_became_peered":"2023-05-16T07:19:48.715256+0000","last_unstale":"2023-05-16T07:19:49.075042+0000","last_undegraded":"2023-05-16T07:19:49.075042+0000","last_fullsized":"2023-05-16T07:19:49.075042+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093115+0000","last_change":"2023-05-16T07:19:49.093115+0000","last_active":"2023-05-16T07:19:49.093115+0000","last_peered":"2023-05-16T07:19:49.093115+0000","last_clean":"2023-05-16T07:19:49.093115+0000","last_became_active":"2023-05-16T07:19:49.092862+0000","last_became_peered":"2023-05-16T07:19:49.092862+0000","last_unstale":"2023-05-16T07:19:49.093115+0000","last_undegraded":"2023-05-16T07:19:49.093115+0000","last_fullsized":"2023-05-16T07:19:49.093115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.078112+0000","last_change":"2023-05-16T07:19:48.707523+0000","last_active":"2023-05-16T07:19:49.078112+0000","last_peered":"2023-05-16T07:19:49.078112+0000","last_clean":"2023-05-16T07:19:49.078112+0000","last_became_active":"2023-05-16T07:19:48.706874+0000","last_became_peered":"2023-05-16T07:19:48.706874+0000","last_unstale":"2023-05-16T07:19:49.078112+0000","last_undegraded":"2023-05-16T07:19:49.078112+0000","last_fullsized":"2023-05-16T07:19:49.078112+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089012+0000","last_change":"2023-05-16T07:19:49.089012+0000","last_active":"2023-05-16T07:19:49.089012+0000","last_peered":"2023-05-16T07:19:49.089012+0000","last_clean":"2023-05-16T07:19:49.089012+0000","last_became_active":"2023-05-16T07:19:49.088777+0000","last_became_peered":"2023-05-16T07:19:49.088777+0000","last_unstale":"2023-05-16T07:19:49.089012+0000","last_undegraded":"2023-05-16T07:19:49.089012+0000","last_fullsized":"2023-05-16T07:19:49.089012+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714356+0000","last_change":"2023-05-16T07:19:48.711407+0000","last_active":"2023-05-16T07:19:48.714356+0000","last_peered":"2023-05-16T07:19:48.714356+0000","last_clean":"2023-05-16T07:19:48.714356+0000","last_became_active":"2023-05-16T07:19:48.710678+0000","last_became_peered":"2023-05-16T07:19:48.710678+0000","last_unstale":"2023-05-16T07:19:48.714356+0000","last_undegraded":"2023-05-16T07:19:48.714356+0000","last_fullsized":"2023-05-16T07:19:48.714356+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.080756+0000","last_change":"2023-05-16T07:19:49.080756+0000","last_active":"2023-05-16T07:19:49.080756+0000","last_peered":"2023-05-16T07:19:49.080756+0000","last_clean":"2023-05-16T07:19:49.080756+0000","last_became_active":"2023-05-16T07:19:49.080485+0000","last_became_peered":"2023-05-16T07:19:49.080485+0000","last_unstale":"2023-05-16T07:19:49.080756+0000","last_undegraded":"2023-05-16T07:19:49.080756+0000","last_fullsized":"2023-05-16T07:19:49.080756+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719618+0000","last_change":"2023-05-16T07:19:48.714875+0000","last_active":"2023-05-16T07:19:48.719618+0000","last_peered":"2023-05-16T07:19:48.719618+0000","last_clean":"2023-05-16T07:19:48.719618+0000","last_became_active":"2023-05-16T07:19:48.714526+0000","last_became_peered":"2023-05-16T07:19:48.714526+0000","last_unstale":"2023-05-16T07:19:48.719618+0000","last_undegraded":"2023-05-16T07:19:48.719618+0000","last_fullsized":"2023-05-16T07:19:48.719618+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090049+0000","last_change":"2023-05-16T07:19:49.090049+0000","last_active":"2023-05-16T07:19:49.090049+0000","last_peered":"2023-05-16T07:19:49.090049+0000","last_clean":"2023-05-16T07:19:49.090049+0000","last_became_active":"2023-05-16T07:19:49.089209+0000","last_became_peered":"2023-05-16T07:19:49.089209+0000","last_unstale":"2023-05-16T07:19:49.090049+0000","last_undegraded":"2023-05-16T07:19:49.090049+0000","last_fullsized":"2023-05-16T07:19:49.090049+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723402+0000","last_change":"2023-05-16T07:19:48.720485+0000","last_active":"2023-05-16T07:19:48.723402+0000","last_peered":"2023-05-16T07:19:48.723402+0000","last_clean":"2023-05-16T07:19:48.723402+0000","last_became_active":"2023-05-16T07:19:48.720202+0000","last_became_peered":"2023-05-16T07:19:48.720202+0000","last_unstale":"2023-05-16T07:19:48.723402+0000","last_undegraded":"2023-05-16T07:19:48.723402+0000","last_fullsized":"2023-05-16T07:19:48.723402+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086671+0000","last_change":"2023-05-16T07:19:49.086671+0000","last_active":"2023-05-16T07:19:49.086671+0000","last_peered":"2023-05-16T07:19:49.086671+0000","last_clean":"2023-05-16T07:19:49.086671+0000","last_became_active":"2023-05-16T07:19:49.084485+0000","last_became_peered":"2023-05-16T07:19:49.084485+0000","last_unstale":"2023-05-16T07:19:49.086671+0000","last_undegraded":"2023-05-16T07:19:49.086671+0000","last_fullsized":"2023-05-16T07:19:49.086671+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721339+0000","last_change":"2023-05-16T07:19:48.712449+0000","last_active":"2023-05-16T07:19:48.721339+0000","last_peered":"2023-05-16T07:19:48.721339+0000","last_clean":"2023-05-16T07:19:48.721339+0000","last_became_active":"2023-05-16T07:19:48.712111+0000","last_became_peered":"2023-05-16T07:19:48.712111+0000","last_unstale":"2023-05-16T07:19:48.721339+0000","last_undegraded":"2023-05-16T07:19:48.721339+0000","last_fullsized":"2023-05-16T07:19:48.721339+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.100072+0000","last_change":"2023-05-16T07:19:49.100072+0000","last_active":"2023-05-16T07:19:49.100072+0000","last_peered":"2023-05-16T07:19:49.100072+0000","last_clean":"2023-05-16T07:19:49.100072+0000","last_became_active":"2023-05-16T07:19:49.099855+0000","last_became_peered":"2023-05-16T07:19:49.099855+0000","last_unstale":"2023-05-16T07:19:49.100072+0000","last_undegraded":"2023-05-16T07:19:49.100072+0000","last_fullsized":"2023-05-16T07:19:49.100072+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719533+0000","last_change":"2023-05-16T07:19:48.714615+0000","last_active":"2023-05-16T07:19:48.719533+0000","last_peered":"2023-05-16T07:19:48.719533+0000","last_clean":"2023-05-16T07:19:48.719533+0000","last_became_active":"2023-05-16T07:19:48.714114+0000","last_became_peered":"2023-05-16T07:19:48.714114+0000","last_unstale":"2023-05-16T07:19:48.719533+0000","last_undegraded":"2023-05-16T07:19:48.719533+0000","last_fullsized":"2023-05-16T07:19:48.719533+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.092508+0000","last_change":"2023-05-16T07:19:49.092508+0000","last_active":"2023-05-16T07:19:49.092508+0000","last_peered":"2023-05-16T07:19:49.092508+0000","last_clean":"2023-05-16T07:19:49.092508+0000","last_became_active":"2023-05-16T07:19:49.092022+0000","last_became_peered":"2023-05-16T07:19:49.092022+0000","last_unstale":"2023-05-16T07:19:49.092508+0000","last_undegraded":"2023-05-16T07:19:49.092508+0000","last_fullsized":"2023-05-16T07:19:49.092508+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099628+0000","last_change":"2023-05-16T07:19:48.715882+0000","last_active":"2023-05-16T07:19:49.099628+0000","last_peered":"2023-05-16T07:19:49.099628+0000","last_clean":"2023-05-16T07:19:49.099628+0000","last_became_active":"2023-05-16T07:19:48.715595+0000","last_became_peered":"2023-05-16T07:19:48.715595+0000","last_unstale":"2023-05-16T07:19:49.099628+0000","last_undegraded":"2023-05-16T07:19:49.099628+0000","last_fullsized":"2023-05-16T07:19:49.099628+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.096249+0000","last_change":"2023-05-16T07:19:49.096249+0000","last_active":"2023-05-16T07:19:49.096249+0000","last_peered":"2023-05-16T07:19:49.096249+0000","last_clean":"2023-05-16T07:19:49.096249+0000","last_became_active":"2023-05-16T07:19:49.095791+0000","last_became_peered":"2023-05-16T07:19:49.095791+0000","last_unstale":"2023-05-16T07:19:49.096249+0000","last_undegraded":"2023-05-16T07:19:49.096249+0000","last_fullsized":"2023-05-16T07:19:49.096249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.072369+0000","last_change":"2023-05-16T07:19:48.719099+0000","last_active":"2023-05-16T07:19:49.072369+0000","last_peered":"2023-05-16T07:19:49.072369+0000","last_clean":"2023-05-16T07:19:49.072369+0000","last_became_active":"2023-05-16T07:19:48.718788+0000","last_became_peered":"2023-05-16T07:19:48.718788+0000","last_unstale":"2023-05-16T07:19:49.072369+0000","last_undegraded":"2023-05-16T07:19:49.072369+0000","last_fullsized":"2023-05-16T07:19:49.072369+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085323+0000","last_change":"2023-05-16T07:19:49.085323+0000","last_active":"2023-05-16T07:19:49.085323+0000","last_peered":"2023-05-16T07:19:49.085323+0000","last_clean":"2023-05-16T07:19:49.085323+0000","last_became_active":"2023-05-16T07:19:49.084896+0000","last_became_peered":"2023-05-16T07:19:49.084896+0000","last_unstale":"2023-05-16T07:19:49.085323+0000","last_undegraded":"2023-05-16T07:19:49.085323+0000","last_fullsized":"2023-05-16T07:19:49.085323+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715367+0000","last_change":"2023-05-16T07:19:48.713283+0000","last_active":"2023-05-16T07:19:48.715367+0000","last_peered":"2023-05-16T07:19:48.715367+0000","last_clean":"2023-05-16T07:19:48.715367+0000","last_became_active":"2023-05-16T07:19:48.713015+0000","last_became_peered":"2023-05-16T07:19:48.713015+0000","last_unstale":"2023-05-16T07:19:48.715367+0000","last_undegraded":"2023-05-16T07:19:48.715367+0000","last_fullsized":"2023-05-16T07:19:48.715367+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.075003+0000","last_change":"2023-05-16T07:19:49.075003+0000","last_active":"2023-05-16T07:19:49.075003+0000","last_peered":"2023-05-16T07:19:49.075003+0000","last_clean":"2023-05-16T07:19:49.075003+0000","last_became_active":"2023-05-16T07:19:49.074782+0000","last_became_peered":"2023-05-16T07:19:49.074782+0000","last_unstale":"2023-05-16T07:19:49.075003+0000","last_undegraded":"2023-05-16T07:19:49.075003+0000","last_fullsized":"2023-05-16T07:19:49.075003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723077+0000","last_change":"2023-05-16T07:19:48.718570+0000","last_active":"2023-05-16T07:19:48.723077+0000","last_peered":"2023-05-16T07:19:48.723077+0000","last_clean":"2023-05-16T07:19:48.723077+0000","last_became_active":"2023-05-16T07:19:48.718292+0000","last_became_peered":"2023-05-16T07:19:48.718292+0000","last_unstale":"2023-05-16T07:19:48.723077+0000","last_undegraded":"2023-05-16T07:19:48.723077+0000","last_fullsized":"2023-05-16T07:19:48.723077+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.091333+0000","last_change":"2023-05-16T07:19:49.091333+0000","last_active":"2023-05-16T07:19:49.091333+0000","last_peered":"2023-05-16T07:19:49.091333+0000","last_clean":"2023-05-16T07:19:49.091333+0000","last_became_active":"2023-05-16T07:19:49.091101+0000","last_became_peered":"2023-05-16T07:19:49.091101+0000","last_unstale":"2023-05-16T07:19:49.091333+0000","last_undegraded":"2023-05-16T07:19:49.091333+0000","last_fullsized":"2023-05-16T07:19:49.091333+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099091+0000","last_change":"2023-05-16T07:19:48.716129+0000","last_active":"2023-05-16T07:19:49.099091+0000","last_peered":"2023-05-16T07:19:49.099091+0000","last_clean":"2023-05-16T07:19:49.099091+0000","last_became_active":"2023-05-16T07:19:48.715860+0000","last_became_peered":"2023-05-16T07:19:48.715860+0000","last_unstale":"2023-05-16T07:19:49.099091+0000","last_undegraded":"2023-05-16T07:19:49.099091+0000","last_fullsized":"2023-05-16T07:19:49.099091+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099291+0000","last_change":"2023-05-16T07:19:49.099291+0000","last_active":"2023-05-16T07:19:49.099291+0000","last_peered":"2023-05-16T07:19:49.099291+0000","last_clean":"2023-05-16T07:19:49.099291+0000","last_became_active":"2023-05-16T07:19:49.098985+0000","last_became_peered":"2023-05-16T07:19:49.098985+0000","last_unstale":"2023-05-16T07:19:49.099291+0000","last_undegraded":"2023-05-16T07:19:49.099291+0000","last_fullsized":"2023-05-16T07:19:49.099291+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719487+0000","last_change":"2023-05-16T07:19:48.717134+0000","last_active":"2023-05-16T07:19:48.719487+0000","last_peered":"2023-05-16T07:19:48.719487+0000","last_clean":"2023-05-16T07:19:48.719487+0000","last_became_active":"2023-05-16T07:19:48.716805+0000","last_became_peered":"2023-05-16T07:19:48.716805+0000","last_unstale":"2023-05-16T07:19:48.719487+0000","last_undegraded":"2023-05-16T07:19:48.719487+0000","last_fullsized":"2023-05-16T07:19:48.719487+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088827+0000","last_change":"2023-05-16T07:19:49.088827+0000","last_active":"2023-05-16T07:19:49.088827+0000","last_peered":"2023-05-16T07:19:49.088827+0000","last_clean":"2023-05-16T07:19:49.088827+0000","last_became_active":"2023-05-16T07:19:49.088537+0000","last_became_peered":"2023-05-16T07:19:49.088537+0000","last_unstale":"2023-05-16T07:19:49.088827+0000","last_undegraded":"2023-05-16T07:19:49.088827+0000","last_fullsized":"2023-05-16T07:19:49.088827+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.718345+0000","last_change":"2023-05-16T07:19:48.714398+0000","last_active":"2023-05-16T07:19:48.718345+0000","last_peered":"2023-05-16T07:19:48.718345+0000","last_clean":"2023-05-16T07:19:48.718345+0000","last_became_active":"2023-05-16T07:19:48.714098+0000","last_became_peered":"2023-05-16T07:19:48.714098+0000","last_unstale":"2023-05-16T07:19:48.718345+0000","last_undegraded":"2023-05-16T07:19:48.718345+0000","last_fullsized":"2023-05-16T07:19:48.718345+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093197+0000","last_change":"2023-05-16T07:19:49.093197+0000","last_active":"2023-05-16T07:19:49.093197+0000","last_peered":"2023-05-16T07:19:49.093197+0000","last_clean":"2023-05-16T07:19:49.093197+0000","last_became_active":"2023-05-16T07:19:49.092927+0000","last_became_peered":"2023-05-16T07:19:49.092927+0000","last_unstale":"2023-05-16T07:19:49.093197+0000","last_undegraded":"2023-05-16T07:19:49.093197+0000","last_fullsized":"2023-05-16T07:19:49.093197+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.071309+0000","last_change":"2023-05-16T07:19:48.716306+0000","last_active":"2023-05-16T07:19:49.071309+0000","last_peered":"2023-05-16T07:19:49.071309+0000","last_clean":"2023-05-16T07:19:49.071309+0000","last_became_active":"2023-05-16T07:19:48.715984+0000","last_became_peered":"2023-05-16T07:19:48.715984+0000","last_unstale":"2023-05-16T07:19:49.071309+0000","last_undegraded":"2023-05-16T07:19:49.071309+0000","last_fullsized":"2023-05-16T07:19:49.071309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088701+0000","last_change":"2023-05-16T07:19:49.088701+0000","last_active":"2023-05-16T07:19:49.088701+0000","last_peered":"2023-05-16T07:19:49.088701+0000","last_clean":"2023-05-16T07:19:49.088701+0000","last_became_active":"2023-05-16T07:19:49.088435+0000","last_became_peered":"2023-05-16T07:19:49.088435+0000","last_unstale":"2023-05-16T07:19:49.088701+0000","last_undegraded":"2023-05-16T07:19:49.088701+0000","last_fullsized":"2023-05-16T07:19:49.088701+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.718907+0000","last_change":"2023-05-16T07:19:48.711741+0000","last_active":"2023-05-16T07:19:48.718907+0000","last_peered":"2023-05-16T07:19:48.718907+0000","last_clean":"2023-05-16T07:19:48.718907+0000","last_became_active":"2023-05-16T07:19:48.711508+0000","last_became_peered":"2023-05-16T07:19:48.711508+0000","last_unstale":"2023-05-16T07:19:48.718907+0000","last_undegraded":"2023-05-16T07:19:48.718907+0000","last_fullsized":"2023-05-16T07:19:48.718907+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085844+0000","last_change":"2023-05-16T07:19:49.085844+0000","last_active":"2023-05-16T07:19:49.085844+0000","last_peered":"2023-05-16T07:19:49.085844+0000","last_clean":"2023-05-16T07:19:49.085844+0000","last_became_active":"2023-05-16T07:19:49.085554+0000","last_became_peered":"2023-05-16T07:19:49.085554+0000","last_unstale":"2023-05-16T07:19:49.085844+0000","last_undegraded":"2023-05-16T07:19:49.085844+0000","last_fullsized":"2023-05-16T07:19:49.085844+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.076491+0000","last_change":"2023-05-16T07:19:48.718393+0000","last_active":"2023-05-16T07:19:49.076491+0000","last_peered":"2023-05-16T07:19:49.076491+0000","last_clean":"2023-05-16T07:19:49.076491+0000","last_became_active":"2023-05-16T07:19:48.718151+0000","last_became_peered":"2023-05-16T07:19:48.718151+0000","last_unstale":"2023-05-16T07:19:49.076491+0000","last_undegraded":"2023-05-16T07:19:49.076491+0000","last_fullsized":"2023-05-16T07:19:49.076491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087091+0000","last_change":"2023-05-16T07:19:49.087091+0000","last_active":"2023-05-16T07:19:49.087091+0000","last_peered":"2023-05-16T07:19:49.087091+0000","last_clean":"2023-05-16T07:19:49.087091+0000","last_became_active":"2023-05-16T07:19:49.086778+0000","last_became_peered":"2023-05-16T07:19:49.086778+0000","last_unstale":"2023-05-16T07:19:49.087091+0000","last_undegraded":"2023-05-16T07:19:49.087091+0000","last_fullsized":"2023-05-16T07:19:49.087091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721818+0000","last_change":"2023-05-16T07:19:48.717012+0000","last_active":"2023-05-16T07:19:48.721818+0000","last_peered":"2023-05-16T07:19:48.721818+0000","last_clean":"2023-05-16T07:19:48.721818+0000","last_became_active":"2023-05-16T07:19:48.716750+0000","last_became_peered":"2023-05-16T07:19:48.716750+0000","last_unstale":"2023-05-16T07:19:48.721818+0000","last_undegraded":"2023-05-16T07:19:48.721818+0000","last_fullsized":"2023-05-16T07:19:48.721818+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087451+0000","last_change":"2023-05-16T07:19:49.087451+0000","last_active":"2023-05-16T07:19:49.087451+0000","last_peered":"2023-05-16T07:19:49.087451+0000","last_clean":"2023-05-16T07:19:49.087451+0000","last_became_active":"2023-05-16T07:19:49.087066+0000","last_became_peered":"2023-05-16T07:19:49.087066+0000","last_unstale":"2023-05-16T07:19:49.087451+0000","last_undegraded":"2023-05-16T07:19:49.087451+0000","last_fullsized":"2023-05-16T07:19:49.087451+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719428+0000","last_change":"2023-05-16T07:19:48.715679+0000","last_active":"2023-05-16T07:19:48.719428+0000","last_peered":"2023-05-16T07:19:48.719428+0000","last_clean":"2023-05-16T07:19:48.719428+0000","last_became_active":"2023-05-16T07:19:48.715393+0000","last_became_peered":"2023-05-16T07:19:48.715393+0000","last_unstale":"2023-05-16T07:19:48.719428+0000","last_undegraded":"2023-05-16T07:19:48.719428+0000","last_fullsized":"2023-05-16T07:19:48.719428+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088080+0000","last_change":"2023-05-16T07:19:49.088080+0000","last_active":"2023-05-16T07:19:49.088080+0000","last_peered":"2023-05-16T07:19:49.088080+0000","last_clean":"2023-05-16T07:19:49.088080+0000","last_became_active":"2023-05-16T07:19:49.087793+0000","last_became_peered":"2023-05-16T07:19:49.087793+0000","last_unstale":"2023-05-16T07:19:49.088080+0000","last_undegraded":"2023-05-16T07:19:49.088080+0000","last_fullsized":"2023-05-16T07:19:49.088080+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.717698+0000","last_change":"2023-05-16T07:19:48.710180+0000","last_active":"2023-05-16T07:19:48.717698+0000","last_peered":"2023-05-16T07:19:48.717698+0000","last_clean":"2023-05-16T07:19:48.717698+0000","last_became_active":"2023-05-16T07:19:48.709728+0000","last_became_peered":"2023-05-16T07:19:48.709728+0000","last_unstale":"2023-05-16T07:19:48.717698+0000","last_undegraded":"2023-05-16T07:19:48.717698+0000","last_fullsized":"2023-05-16T07:19:48.717698+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.092263+0000","last_change":"2023-05-16T07:19:49.092263+0000","last_active":"2023-05-16T07:19:49.092263+0000","last_peered":"2023-05-16T07:19:49.092263+0000","last_clean":"2023-05-16T07:19:49.092263+0000","last_became_active":"2023-05-16T07:19:49.089550+0000","last_became_peered":"2023-05-16T07:19:49.089550+0000","last_unstale":"2023-05-16T07:19:49.092263+0000","last_undegraded":"2023-05-16T07:19:49.092263+0000","last_fullsized":"2023-05-16T07:19:49.092263+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719506+0000","last_change":"2023-05-16T07:19:48.717542+0000","last_active":"2023-05-16T07:19:48.719506+0000","last_peered":"2023-05-16T07:19:48.719506+0000","last_clean":"2023-05-16T07:19:48.719506+0000","last_became_active":"2023-05-16T07:19:48.717270+0000","last_became_peered":"2023-05-16T07:19:48.717270+0000","last_unstale":"2023-05-16T07:19:48.719506+0000","last_undegraded":"2023-05-16T07:19:48.719506+0000","last_fullsized":"2023-05-16T07:19:48.719506+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088736+0000","last_change":"2023-05-16T07:19:49.088736+0000","last_active":"2023-05-16T07:19:49.088736+0000","last_peered":"2023-05-16T07:19:49.088736+0000","last_clean":"2023-05-16T07:19:49.088736+0000","last_became_active":"2023-05-16T07:19:49.088429+0000","last_became_peered":"2023-05-16T07:19:49.088429+0000","last_unstale":"2023-05-16T07:19:49.088736+0000","last_undegraded":"2023-05-16T07:19:49.088736+0000","last_fullsized":"2023-05-16T07:19:49.088736+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721318+0000","last_change":"2023-05-16T07:19:48.712342+0000","last_active":"2023-05-16T07:19:48.721318+0000","last_peered":"2023-05-16T07:19:48.721318+0000","last_clean":"2023-05-16T07:19:48.721318+0000","last_became_active":"2023-05-16T07:19:48.711861+0000","last_became_peered":"2023-05-16T07:19:48.711861+0000","last_unstale":"2023-05-16T07:19:48.721318+0000","last_undegraded":"2023-05-16T07:19:48.721318+0000","last_fullsized":"2023-05-16T07:19:48.721318+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086365+0000","last_change":"2023-05-16T07:19:49.086365+0000","last_active":"2023-05-16T07:19:49.086365+0000","last_peered":"2023-05-16T07:19:49.086365+0000","last_clean":"2023-05-16T07:19:49.086365+0000","last_became_active":"2023-05-16T07:19:49.086098+0000","last_became_peered":"2023-05-16T07:19:49.086098+0000","last_unstale":"2023-05-16T07:19:49.086365+0000","last_undegraded":"2023-05-16T07:19:49.086365+0000","last_fullsized":"2023-05-16T07:19:49.086365+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.717700+0000","last_change":"2023-05-16T07:19:48.711741+0000","last_active":"2023-05-16T07:19:48.717700+0000","last_peered":"2023-05-16T07:19:48.717700+0000","last_clean":"2023-05-16T07:19:48.717700+0000","last_became_active":"2023-05-16T07:19:48.711454+0000","last_became_peered":"2023-05-16T07:19:48.711454+0000","last_unstale":"2023-05-16T07:19:48.717700+0000","last_undegraded":"2023-05-16T07:19:48.717700+0000","last_fullsized":"2023-05-16T07:19:48.717700+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083717+0000","last_change":"2023-05-16T07:19:49.083717+0000","last_active":"2023-05-16T07:19:49.083717+0000","last_peered":"2023-05-16T07:19:49.083717+0000","last_clean":"2023-05-16T07:19:49.083717+0000","last_became_active":"2023-05-16T07:19:49.083449+0000","last_became_peered":"2023-05-16T07:19:49.083449+0000","last_unstale":"2023-05-16T07:19:49.083717+0000","last_undegraded":"2023-05-16T07:19:49.083717+0000","last_fullsized":"2023-05-16T07:19:49.083717+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721677+0000","last_change":"2023-05-16T07:19:48.716559+0000","last_active":"2023-05-16T07:19:48.721677+0000","last_peered":"2023-05-16T07:19:48.721677+0000","last_clean":"2023-05-16T07:19:48.721677+0000","last_became_active":"2023-05-16T07:19:48.716264+0000","last_became_peered":"2023-05-16T07:19:48.716264+0000","last_unstale":"2023-05-16T07:19:48.721677+0000","last_undegraded":"2023-05-16T07:19:48.721677+0000","last_fullsized":"2023-05-16T07:19:48.721677+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083881+0000","last_change":"2023-05-16T07:19:49.083881+0000","last_active":"2023-05-16T07:19:49.083881+0000","last_peered":"2023-05-16T07:19:49.083881+0000","last_clean":"2023-05-16T07:19:49.083881+0000","last_became_active":"2023-05-16T07:19:49.083551+0000","last_became_peered":"2023-05-16T07:19:49.083551+0000","last_unstale":"2023-05-16T07:19:49.083881+0000","last_undegraded":"2023-05-16T07:19:49.083881+0000","last_fullsized":"2023-05-16T07:19:49.083881+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719626+0000","last_change":"2023-05-16T07:19:48.713299+0000","last_active":"2023-05-16T07:19:48.719626+0000","last_peered":"2023-05-16T07:19:48.719626+0000","last_clean":"2023-05-16T07:19:48.719626+0000","last_became_active":"2023-05-16T07:19:48.713036+0000","last_became_peered":"2023-05-16T07:19:48.713036+0000","last_unstale":"2023-05-16T07:19:48.719626+0000","last_undegraded":"2023-05-16T07:19:48.719626+0000","last_fullsized":"2023-05-16T07:19:48.719626+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086498+0000","last_change":"2023-05-16T07:19:49.086498+0000","last_active":"2023-05-16T07:19:49.086498+0000","last_peered":"2023-05-16T07:19:49.086498+0000","last_clean":"2023-05-16T07:19:49.086498+0000","last_became_active":"2023-05-16T07:19:49.085990+0000","last_became_peered":"2023-05-16T07:19:49.085990+0000","last_unstale":"2023-05-16T07:19:49.086498+0000","last_undegraded":"2023-05-16T07:19:49.086498+0000","last_fullsized":"2023-05-16T07:19:49.086498+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.722723+0000","last_change":"2023-05-16T07:19:48.722141+0000","last_active":"2023-05-16T07:19:48.722723+0000","last_peered":"2023-05-16T07:19:48.722723+0000","last_clean":"2023-05-16T07:19:48.722723+0000","last_became_active":"2023-05-16T07:19:48.721868+0000","last_became_peered":"2023-05-16T07:19:48.721868+0000","last_unstale":"2023-05-16T07:19:48.722723+0000","last_undegraded":"2023-05-16T07:19:48.722723+0000","last_fullsized":"2023-05-16T07:19:48.722723+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.084605+0000","last_change":"2023-05-16T07:19:49.084605+0000","last_active":"2023-05-16T07:19:49.084605+0000","last_peered":"2023-05-16T07:19:49.084605+0000","last_clean":"2023-05-16T07:19:49.084605+0000","last_became_active":"2023-05-16T07:19:49.084088+0000","last_became_peered":"2023-05-16T07:19:49.084088+0000","last_unstale":"2023-05-16T07:19:49.084605+0000","last_undegraded":"2023-05-16T07:19:49.084605+0000","last_fullsized":"2023-05-16T07:19:49.084605+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721153+0000","last_change":"2023-05-16T07:19:48.711305+0000","last_active":"2023-05-16T07:19:48.721153+0000","last_peered":"2023-05-16T07:19:48.721153+0000","last_clean":"2023-05-16T07:19:48.721153+0000","last_became_active":"2023-05-16T07:19:48.710749+0000","last_became_peered":"2023-05-16T07:19:48.710749+0000","last_unstale":"2023-05-16T07:19:48.721153+0000","last_undegraded":"2023-05-16T07:19:48.721153+0000","last_fullsized":"2023-05-16T07:19:48.721153+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086417+0000","last_change":"2023-05-16T07:19:49.086417+0000","last_active":"2023-05-16T07:19:49.086417+0000","last_peered":"2023-05-16T07:19:49.086417+0000","last_clean":"2023-05-16T07:19:49.086417+0000","last_became_active":"2023-05-16T07:19:49.086138+0000","last_became_peered":"2023-05-16T07:19:49.086138+0000","last_unstale":"2023-05-16T07:19:49.086417+0000","last_undegraded":"2023-05-16T07:19:49.086417+0000","last_fullsized":"2023-05-16T07:19:49.086417+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714338+0000","last_change":"2023-05-16T07:19:48.708850+0000","last_active":"2023-05-16T07:19:48.714338+0000","last_peered":"2023-05-16T07:19:48.714338+0000","last_clean":"2023-05-16T07:19:48.714338+0000","last_became_active":"2023-05-16T07:19:48.708211+0000","last_became_peered":"2023-05-16T07:19:48.708211+0000","last_unstale":"2023-05-16T07:19:48.714338+0000","last_undegraded":"2023-05-16T07:19:48.714338+0000","last_fullsized":"2023-05-16T07:19:48.714338+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099670+0000","last_change":"2023-05-16T07:19:49.099670+0000","last_active":"2023-05-16T07:19:49.099670+0000","last_peered":"2023-05-16T07:19:49.099670+0000","last_clean":"2023-05-16T07:19:49.099670+0000","last_became_active":"2023-05-16T07:19:49.099413+0000","last_became_peered":"2023-05-16T07:19:49.099413+0000","last_unstale":"2023-05-16T07:19:49.099670+0000","last_undegraded":"2023-05-16T07:19:49.099670+0000","last_fullsized":"2023-05-16T07:19:49.099670+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.074935+0000","last_change":"2023-05-16T07:19:48.710175+0000","last_active":"2023-05-16T07:19:49.074935+0000","last_peered":"2023-05-16T07:19:49.074935+0000","last_clean":"2023-05-16T07:19:49.074935+0000","last_became_active":"2023-05-16T07:19:48.709900+0000","last_became_peered":"2023-05-16T07:19:48.709900+0000","last_unstale":"2023-05-16T07:19:49.074935+0000","last_undegraded":"2023-05-16T07:19:49.074935+0000","last_fullsized":"2023-05-16T07:19:49.074935+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089056+0000","last_change":"2023-05-16T07:19:49.089056+0000","last_active":"2023-05-16T07:19:49.089056+0000","last_peered":"2023-05-16T07:19:49.089056+0000","last_clean":"2023-05-16T07:19:49.089056+0000","last_became_active":"2023-05-16T07:19:49.088721+0000","last_became_peered":"2023-05-16T07:19:49.088721+0000","last_unstale":"2023-05-16T07:19:49.089056+0000","last_undegraded":"2023-05-16T07:19:49.089056+0000","last_fullsized":"2023-05-16T07:19:49.089056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.082209+0000","last_change":"2023-05-16T07:19:48.712805+0000","last_active":"2023-05-16T07:19:49.082209+0000","last_peered":"2023-05-16T07:19:49.082209+0000","last_clean":"2023-05-16T07:19:49.082209+0000","last_became_active":"2023-05-16T07:19:48.712530+0000","last_became_peered":"2023-05-16T07:19:48.712530+0000","last_unstale":"2023-05-16T07:19:49.082209+0000","last_undegraded":"2023-05-16T07:19:49.082209+0000","last_fullsized":"2023-05-16T07:19:49.082209+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083826+0000","last_change":"2023-05-16T07:19:49.083826+0000","last_active":"2023-05-16T07:19:49.083826+0000","last_peered":"2023-05-16T07:19:49.083826+0000","last_clean":"2023-05-16T07:19:49.083826+0000","last_became_active":"2023-05-16T07:19:49.083469+0000","last_became_peered":"2023-05-16T07:19:49.083469+0000","last_unstale":"2023-05-16T07:19:49.083826+0000","last_undegraded":"2023-05-16T07:19:49.083826+0000","last_fullsized":"2023-05-16T07:19:49.083826+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090850+0000","last_change":"2023-05-16T07:19:48.716287+0000","last_active":"2023-05-16T07:19:49.090850+0000","last_peered":"2023-05-16T07:19:49.090850+0000","last_clean":"2023-05-16T07:19:49.090850+0000","last_became_active":"2023-05-16T07:19:48.715656+0000","last_became_peered":"2023-05-16T07:19:48.715656+0000","last_unstale":"2023-05-16T07:19:49.090850+0000","last_undegraded":"2023-05-16T07:19:49.090850+0000","last_fullsized":"2023-05-16T07:19:49.090850+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086397+0000","last_change":"2023-05-16T07:19:49.086397+0000","last_active":"2023-05-16T07:19:49.086397+0000","last_peered":"2023-05-16T07:19:49.086397+0000","last_clean":"2023-05-16T07:19:49.086397+0000","last_became_active":"2023-05-16T07:19:49.086143+0000","last_became_peered":"2023-05-16T07:19:49.086143+0000","last_unstale":"2023-05-16T07:19:49.086397+0000","last_undegraded":"2023-05-16T07:19:49.086397+0000","last_fullsized":"2023-05-16T07:19:49.086397+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.713088+0000","last_change":"2023-05-16T07:19:48.707314+0000","last_active":"2023-05-16T07:19:48.713088+0000","last_peered":"2023-05-16T07:19:48.713088+0000","last_clean":"2023-05-16T07:19:48.713088+0000","last_became_active":"2023-05-16T07:19:48.707009+0000","last_became_peered":"2023-05-16T07:19:48.707009+0000","last_unstale":"2023-05-16T07:19:48.713088+0000","last_undegraded":"2023-05-16T07:19:48.713088+0000","last_fullsized":"2023-05-16T07:19:48.713088+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090568+0000","last_change":"2023-05-16T07:19:49.090568+0000","last_active":"2023-05-16T07:19:49.090568+0000","last_peered":"2023-05-16T07:19:49.090568+0000","last_clean":"2023-05-16T07:19:49.090568+0000","last_became_active":"2023-05-16T07:19:49.090351+0000","last_became_peered":"2023-05-16T07:19:49.090351+0000","last_unstale":"2023-05-16T07:19:49.090568+0000","last_undegraded":"2023-05-16T07:19:49.090568+0000","last_fullsized":"2023-05-16T07:19:49.090568+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719643+0000","last_change":"2023-05-16T07:19:48.714373+0000","last_active":"2023-05-16T07:19:48.719643+0000","last_peered":"2023-05-16T07:19:48.719643+0000","last_clean":"2023-05-16T07:19:48.719643+0000","last_became_active":"2023-05-16T07:19:48.713710+0000","last_became_peered":"2023-05-16T07:19:48.713710+0000","last_unstale":"2023-05-16T07:19:48.719643+0000","last_undegraded":"2023-05-16T07:19:48.719643+0000","last_fullsized":"2023-05-16T07:19:48.719643+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.094157+0000","last_change":"2023-05-16T07:19:49.094157+0000","last_active":"2023-05-16T07:19:49.094157+0000","last_peered":"2023-05-16T07:19:49.094157+0000","last_clean":"2023-05-16T07:19:49.094157+0000","last_became_active":"2023-05-16T07:19:49.093843+0000","last_became_peered":"2023-05-16T07:19:49.093843+0000","last_unstale":"2023-05-16T07:19:49.094157+0000","last_undegraded":"2023-05-16T07:19:49.094157+0000","last_fullsized":"2023-05-16T07:19:49.094157+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723228+0000","last_change":"2023-05-16T07:19:48.718660+0000","last_active":"2023-05-16T07:19:48.723228+0000","last_peered":"2023-05-16T07:19:48.723228+0000","last_clean":"2023-05-16T07:19:48.723228+0000","last_became_active":"2023-05-16T07:19:48.718386+0000","last_became_peered":"2023-05-16T07:19:48.718386+0000","last_unstale":"2023-05-16T07:19:48.723228+0000","last_undegraded":"2023-05-16T07:19:48.723228+0000","last_fullsized":"2023-05-16T07:19:48.723228+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.080166+0000","last_change":"2023-05-16T07:19:49.080166+0000","last_active":"2023-05-16T07:19:49.080166+0000","last_peered":"2023-05-16T07:19:49.080166+0000","last_clean":"2023-05-16T07:19:49.080166+0000","last_became_active":"2023-05-16T07:19:49.079941+0000","last_became_peered":"2023-05-16T07:19:49.079941+0000","last_unstale":"2023-05-16T07:19:49.080166+0000","last_undegraded":"2023-05-16T07:19:49.080166+0000","last_fullsized":"2023-05-16T07:19:49.080166+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714335+0000","last_change":"2023-05-16T07:19:48.710777+0000","last_active":"2023-05-16T07:19:48.714335+0000","last_peered":"2023-05-16T07:19:48.714335+0000","last_clean":"2023-05-16T07:19:48.714335+0000","last_became_active":"2023-05-16T07:19:48.710476+0000","last_became_peered":"2023-05-16T07:19:48.710476+0000","last_unstale":"2023-05-16T07:19:48.714335+0000","last_undegraded":"2023-05-16T07:19:48.714335+0000","last_fullsized":"2023-05-16T07:19:48.714335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093403+0000","last_change":"2023-05-16T07:19:49.093403+0000","last_active":"2023-05-16T07:19:49.093403+0000","last_peered":"2023-05-16T07:19:49.093403+0000","last_clean":"2023-05-16T07:19:49.093403+0000","last_became_active":"2023-05-16T07:19:49.093143+0000","last_became_peered":"2023-05-16T07:19:49.093143+0000","last_unstale":"2023-05-16T07:19:49.093403+0000","last_undegraded":"2023-05-16T07:19:49.093403+0000","last_fullsized":"2023-05-16T07:19:49.093403+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723272+0000","last_change":"2023-05-16T07:19:48.718851+0000","last_active":"2023-05-16T07:19:48.723272+0000","last_peered":"2023-05-16T07:19:48.723272+0000","last_clean":"2023-05-16T07:19:48.723272+0000","last_became_active":"2023-05-16T07:19:48.718573+0000","last_became_peered":"2023-05-16T07:19:48.718573+0000","last_unstale":"2023-05-16T07:19:48.723272+0000","last_undegraded":"2023-05-16T07:19:48.723272+0000","last_fullsized":"2023-05-16T07:19:48.723272+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099763+0000","last_change":"2023-05-16T07:19:49.099763+0000","last_active":"2023-05-16T07:19:49.099763+0000","last_peered":"2023-05-16T07:19:49.099763+0000","last_clean":"2023-05-16T07:19:49.099763+0000","last_became_active":"2023-05-16T07:19:49.099478+0000","last_became_peered":"2023-05-16T07:19:49.099478+0000","last_unstale":"2023-05-16T07:19:49.099763+0000","last_undegraded":"2023-05-16T07:19:49.099763+0000","last_fullsized":"2023-05-16T07:19:49.099763+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083502+0000","last_change":"2023-05-16T07:19:48.710485+0000","last_active":"2023-05-16T07:19:49.083502+0000","last_peered":"2023-05-16T07:19:49.083502+0000","last_clean":"2023-05-16T07:19:49.083502+0000","last_became_active":"2023-05-16T07:19:48.708387+0000","last_became_peered":"2023-05-16T07:19:48.708387+0000","last_unstale":"2023-05-16T07:19:49.083502+0000","last_undegraded":"2023-05-16T07:19:49.083502+0000","last_fullsized":"2023-05-16T07:19:49.083502+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085869+0000","last_change":"2023-05-16T07:19:49.085869+0000","last_active":"2023-05-16T07:19:49.085869+0000","last_peered":"2023-05-16T07:19:49.085869+0000","last_clean":"2023-05-16T07:19:49.085869+0000","last_became_active":"2023-05-16T07:19:49.085611+0000","last_became_peered":"2023-05-16T07:19:49.085611+0000","last_unstale":"2023-05-16T07:19:49.085869+0000","last_undegraded":"2023-05-16T07:19:49.085869+0000","last_fullsized":"2023-05-16T07:19:49.085869+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.098907+0000","last_change":"2023-05-16T07:19:48.715081+0000","last_active":"2023-05-16T07:19:49.098907+0000","last_peered":"2023-05-16T07:19:49.098907+0000","last_clean":"2023-05-16T07:19:49.098907+0000","last_became_active":"2023-05-16T07:19:48.714412+0000","last_became_peered":"2023-05-16T07:19:48.714412+0000","last_unstale":"2023-05-16T07:19:49.098907+0000","last_undegraded":"2023-05-16T07:19:49.098907+0000","last_fullsized":"2023-05-16T07:19:49.098907+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.091808+0000","last_change":"2023-05-16T07:19:49.091808+0000","last_active":"2023-05-16T07:19:49.091808+0000","last_peered":"2023-05-16T07:19:49.091808+0000","last_clean":"2023-05-16T07:19:49.091808+0000","last_became_active":"2023-05-16T07:19:49.089497+0000","last_became_peered":"2023-05-16T07:19:49.089497+0000","last_unstale":"2023-05-16T07:19:49.091808+0000","last_undegraded":"2023-05-16T07:19:49.091808+0000","last_fullsized":"2023-05-16T07:19:49.091808+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720555+0000","last_change":"2023-05-16T07:19:48.718161+0000","last_active":"2023-05-16T07:19:48.720555+0000","last_peered":"2023-05-16T07:19:48.720555+0000","last_clean":"2023-05-16T07:19:48.720555+0000","last_became_active":"2023-05-16T07:19:48.716022+0000","last_became_peered":"2023-05-16T07:19:48.716022+0000","last_unstale":"2023-05-16T07:19:48.720555+0000","last_undegraded":"2023-05-16T07:19:48.720555+0000","last_fullsized":"2023-05-16T07:19:48.720555+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093169+0000","last_change":"2023-05-16T07:19:49.093169+0000","last_active":"2023-05-16T07:19:49.093169+0000","last_peered":"2023-05-16T07:19:49.093169+0000","last_clean":"2023-05-16T07:19:49.093169+0000","last_became_active":"2023-05-16T07:19:49.092907+0000","last_became_peered":"2023-05-16T07:19:49.092907+0000","last_unstale":"2023-05-16T07:19:49.093169+0000","last_undegraded":"2023-05-16T07:19:49.093169+0000","last_fullsized":"2023-05-16T07:19:49.093169+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.722504+0000","last_change":"2023-05-16T07:19:48.721859+0000","last_active":"2023-05-16T07:19:48.722504+0000","last_peered":"2023-05-16T07:19:48.722504+0000","last_clean":"2023-05-16T07:19:48.722504+0000","last_became_active":"2023-05-16T07:19:48.721641+0000","last_became_peered":"2023-05-16T07:19:48.721641+0000","last_unstale":"2023-05-16T07:19:48.722504+0000","last_undegraded":"2023-05-16T07:19:48.722504+0000","last_fullsized":"2023-05-16T07:19:48.722504+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099114+0000","last_change":"2023-05-16T07:19:49.099114+0000","last_active":"2023-05-16T07:19:49.099114+0000","last_peered":"2023-05-16T07:19:49.099114+0000","last_clean":"2023-05-16T07:19:49.099114+0000","last_became_active":"2023-05-16T07:19:49.098842+0000","last_became_peered":"2023-05-16T07:19:49.098842+0000","last_unstale":"2023-05-16T07:19:49.099114+0000","last_undegraded":"2023-05-16T07:19:49.099114+0000","last_fullsized":"2023-05-16T07:19:49.099114+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.072739+0000","last_change":"2023-05-16T07:19:48.719930+0000","last_active":"2023-05-16T07:19:49.072739+0000","last_peered":"2023-05-16T07:19:49.072739+0000","last_clean":"2023-05-16T07:19:49.072739+0000","last_became_active":"2023-05-16T07:19:48.719621+0000","last_became_peered":"2023-05-16T07:19:48.719621+0000","last_unstale":"2023-05-16T07:19:49.072739+0000","last_undegraded":"2023-05-16T07:19:49.072739+0000","last_fullsized":"2023-05-16T07:19:49.072739+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085507+0000","last_change":"2023-05-16T07:19:49.085507+0000","last_active":"2023-05-16T07:19:49.085507+0000","last_peered":"2023-05-16T07:19:49.085507+0000","last_clean":"2023-05-16T07:19:49.085507+0000","last_became_active":"2023-05-16T07:19:49.085049+0000","last_became_peered":"2023-05-16T07:19:49.085049+0000","last_unstale":"2023-05-16T07:19:49.085507+0000","last_undegraded":"2023-05-16T07:19:49.085507+0000","last_fullsized":"2023-05-16T07:19:49.085507+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720970+0000","last_change":"2023-05-16T07:19:48.720029+0000","last_active":"2023-05-16T07:19:48.720970+0000","last_peered":"2023-05-16T07:19:48.720970+0000","last_clean":"2023-05-16T07:19:48.720970+0000","last_became_active":"2023-05-16T07:19:48.719761+0000","last_became_peered":"2023-05-16T07:19:48.719761+0000","last_unstale":"2023-05-16T07:19:48.720970+0000","last_undegraded":"2023-05-16T07:19:48.720970+0000","last_fullsized":"2023-05-16T07:19:48.720970+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.084371+0000","last_change":"2023-05-16T07:19:49.084371+0000","last_active":"2023-05-16T07:19:49.084371+0000","last_peered":"2023-05-16T07:19:49.084371+0000","last_clean":"2023-05-16T07:19:49.084371+0000","last_became_active":"2023-05-16T07:19:49.083649+0000","last_became_peered":"2023-05-16T07:19:49.083649+0000","last_unstale":"2023-05-16T07:19:49.084371+0000","last_undegraded":"2023-05-16T07:19:49.084371+0000","last_fullsized":"2023-05-16T07:19:49.084371+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721734+0000","last_change":"2023-05-16T07:19:48.716582+0000","last_active":"2023-05-16T07:19:48.721734+0000","last_peered":"2023-05-16T07:19:48.721734+0000","last_clean":"2023-05-16T07:19:48.721734+0000","last_became_active":"2023-05-16T07:19:48.716297+0000","last_became_peered":"2023-05-16T07:19:48.716297+0000","last_unstale":"2023-05-16T07:19:48.721734+0000","last_undegraded":"2023-05-16T07:19:48.721734+0000","last_fullsized":"2023-05-16T07:19:48.721734+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.091787+0000","last_change":"2023-05-16T07:19:49.091787+0000","last_active":"2023-05-16T07:19:49.091787+0000","last_peered":"2023-05-16T07:19:49.091787+0000","last_clean":"2023-05-16T07:19:49.091787+0000","last_became_active":"2023-05-16T07:19:49.091491+0000","last_became_peered":"2023-05-16T07:19:49.091491+0000","last_unstale":"2023-05-16T07:19:49.091787+0000","last_undegraded":"2023-05-16T07:19:49.091787+0000","last_fullsized":"2023-05-16T07:19:49.091787+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720917+0000","last_change":"2023-05-16T07:19:48.719269+0000","last_active":"2023-05-16T07:19:48.720917+0000","last_peered":"2023-05-16T07:19:48.720917+0000","last_clean":"2023-05-16T07:19:48.720917+0000","last_became_active":"2023-05-16T07:19:48.718794+0000","last_became_peered":"2023-05-16T07:19:48.718794+0000","last_unstale":"2023-05-16T07:19:48.720917+0000","last_undegraded":"2023-05-16T07:19:48.720917+0000","last_fullsized":"2023-05-16T07:19:48.720917+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.096108+0000","last_change":"2023-05-16T07:19:49.096108+0000","last_active":"2023-05-16T07:19:49.096108+0000","last_peered":"2023-05-16T07:19:49.096108+0000","last_clean":"2023-05-16T07:19:49.096108+0000","last_became_active":"2023-05-16T07:19:49.095835+0000","last_became_peered":"2023-05-16T07:19:49.095835+0000","last_unstale":"2023-05-16T07:19:49.096108+0000","last_undegraded":"2023-05-16T07:19:49.096108+0000","last_fullsized":"2023-05-16T07:19:49.096108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088488+0000","last_change":"2023-05-16T07:19:48.718846+0000","last_active":"2023-05-16T07:19:49.088488+0000","last_peered":"2023-05-16T07:19:49.088488+0000","last_clean":"2023-05-16T07:19:49.088488+0000","last_became_active":"2023-05-16T07:19:48.718043+0000","last_became_peered":"2023-05-16T07:19:48.718043+0000","last_unstale":"2023-05-16T07:19:49.088488+0000","last_undegraded":"2023-05-16T07:19:49.088488+0000","last_fullsized":"2023-05-16T07:19:49.088488+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.084868+0000","last_change":"2023-05-16T07:19:49.084868+0000","last_active":"2023-05-16T07:19:49.084868+0000","last_peered":"2023-05-16T07:19:49.084868+0000","last_clean":"2023-05-16T07:19:49.084868+0000","last_became_active":"2023-05-16T07:19:49.084601+0000","last_became_peered":"2023-05-16T07:19:49.084601+0000","last_unstale":"2023-05-16T07:19:49.084868+0000","last_undegraded":"2023-05-16T07:19:49.084868+0000","last_fullsized":"2023-05-16T07:19:49.084868+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715406+0000","last_change":"2023-05-16T07:19:48.711247+0000","last_active":"2023-05-16T07:19:48.715406+0000","last_peered":"2023-05-16T07:19:48.715406+0000","last_clean":"2023-05-16T07:19:48.715406+0000","last_became_active":"2023-05-16T07:19:48.710944+0000","last_became_peered":"2023-05-16T07:19:48.710944+0000","last_unstale":"2023-05-16T07:19:48.715406+0000","last_undegraded":"2023-05-16T07:19:48.715406+0000","last_fullsized":"2023-05-16T07:19:48.715406+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093421+0000","last_change":"2023-05-16T07:19:49.093421+0000","last_active":"2023-05-16T07:19:49.093421+0000","last_peered":"2023-05-16T07:19:49.093421+0000","last_clean":"2023-05-16T07:19:49.093421+0000","last_became_active":"2023-05-16T07:19:49.093130+0000","last_became_peered":"2023-05-16T07:19:49.093130+0000","last_unstale":"2023-05-16T07:19:49.093421+0000","last_undegraded":"2023-05-16T07:19:49.093421+0000","last_fullsized":"2023-05-16T07:19:49.093421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715402+0000","last_change":"2023-05-16T07:19:48.711117+0000","last_active":"2023-05-16T07:19:48.715402+0000","last_peered":"2023-05-16T07:19:48.715402+0000","last_clean":"2023-05-16T07:19:48.715402+0000","last_became_active":"2023-05-16T07:19:48.710839+0000","last_became_peered":"2023-05-16T07:19:48.710839+0000","last_unstale":"2023-05-16T07:19:48.715402+0000","last_undegraded":"2023-05-16T07:19:48.715402+0000","last_fullsized":"2023-05-16T07:19:48.715402+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085287+0000","last_change":"2023-05-16T07:19:49.085287+0000","last_active":"2023-05-16T07:19:49.085287+0000","last_peered":"2023-05-16T07:19:49.085287+0000","last_clean":"2023-05-16T07:19:49.085287+0000","last_became_active":"2023-05-16T07:19:49.085011+0000","last_became_peered":"2023-05-16T07:19:49.085011+0000","last_unstale":"2023-05-16T07:19:49.085287+0000","last_undegraded":"2023-05-16T07:19:49.085287+0000","last_fullsized":"2023-05-16T07:19:49.085287+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723328+0000","last_change":"2023-05-16T07:19:48.719669+0000","last_active":"2023-05-16T07:19:48.723328+0000","last_peered":"2023-05-16T07:19:48.723328+0000","last_clean":"2023-05-16T07:19:48.723328+0000","last_became_active":"2023-05-16T07:19:48.719390+0000","last_became_peered":"2023-05-16T07:19:48.719390+0000","last_unstale":"2023-05-16T07:19:48.723328+0000","last_undegraded":"2023-05-16T07:19:48.723328+0000","last_fullsized":"2023-05-16T07:19:48.723328+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083465+0000","last_change":"2023-05-16T07:19:49.083465+0000","last_active":"2023-05-16T07:19:49.083465+0000","last_peered":"2023-05-16T07:19:49.083465+0000","last_clean":"2023-05-16T07:19:49.083465+0000","last_became_active":"2023-05-16T07:19:49.083224+0000","last_became_peered":"2023-05-16T07:19:49.083224+0000","last_unstale":"2023-05-16T07:19:49.083465+0000","last_undegraded":"2023-05-16T07:19:49.083465+0000","last_fullsized":"2023-05-16T07:19:49.083465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.713001+0000","last_change":"2023-05-16T07:19:48.706422+0000","last_active":"2023-05-16T07:19:48.713001+0000","last_peered":"2023-05-16T07:19:48.713001+0000","last_clean":"2023-05-16T07:19:48.713001+0000","last_became_active":"2023-05-16T07:19:48.706135+0000","last_became_peered":"2023-05-16T07:19:48.706135+0000","last_unstale":"2023-05-16T07:19:48.713001+0000","last_undegraded":"2023-05-16T07:19:48.713001+0000","last_fullsized":"2023-05-16T07:19:48.713001+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.074133+0000","last_change":"2023-05-16T07:19:49.074133+0000","last_active":"2023-05-16T07:19:49.074133+0000","last_peered":"2023-05-16T07:19:49.074133+0000","last_clean":"2023-05-16T07:19:49.074133+0000","last_became_active":"2023-05-16T07:19:49.073881+0000","last_became_peered":"2023-05-16T07:19:49.073881+0000","last_unstale":"2023-05-16T07:19:49.074133+0000","last_undegraded":"2023-05-16T07:19:49.074133+0000","last_fullsized":"2023-05-16T07:19:49.074133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719041+0000","last_change":"2023-05-16T07:19:48.712776+0000","last_active":"2023-05-16T07:19:48.719041+0000","last_peered":"2023-05-16T07:19:48.719041+0000","last_clean":"2023-05-16T07:19:48.719041+0000","last_became_active":"2023-05-16T07:19:48.709470+0000","last_became_peered":"2023-05-16T07:19:48.709470+0000","last_unstale":"2023-05-16T07:19:48.719041+0000","last_undegraded":"2023-05-16T07:19:48.719041+0000","last_fullsized":"2023-05-16T07:19:48.719041+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099077+0000","last_change":"2023-05-16T07:19:49.099077+0000","last_active":"2023-05-16T07:19:49.099077+0000","last_peered":"2023-05-16T07:19:49.099077+0000","last_clean":"2023-05-16T07:19:49.099077+0000","last_became_active":"2023-05-16T07:19:49.098877+0000","last_became_peered":"2023-05-16T07:19:49.098877+0000","last_unstale":"2023-05-16T07:19:49.099077+0000","last_undegraded":"2023-05-16T07:19:49.099077+0000","last_fullsized":"2023-05-16T07:19:49.099077+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723312+0000","last_change":"2023-05-16T07:19:48.719129+0000","last_active":"2023-05-16T07:19:48.723312+0000","last_peered":"2023-05-16T07:19:48.723312+0000","last_clean":"2023-05-16T07:19:48.723312+0000","last_became_active":"2023-05-16T07:19:48.718857+0000","last_became_peered":"2023-05-16T07:19:48.718857+0000","last_unstale":"2023-05-16T07:19:48.723312+0000","last_undegraded":"2023-05-16T07:19:48.723312+0000","last_fullsized":"2023-05-16T07:19:48.723312+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090881+0000","last_change":"2023-05-16T07:19:49.090881+0000","last_active":"2023-05-16T07:19:49.090881+0000","last_peered":"2023-05-16T07:19:49.090881+0000","last_clean":"2023-05-16T07:19:49.090881+0000","last_became_active":"2023-05-16T07:19:49.090622+0000","last_became_peered":"2023-05-16T07:19:49.090622+0000","last_unstale":"2023-05-16T07:19:49.090881+0000","last_undegraded":"2023-05-16T07:19:49.090881+0000","last_fullsized":"2023-05-16T07:19:49.090881+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715425+0000","last_change":"2023-05-16T07:19:48.711822+0000","last_active":"2023-05-16T07:19:48.715425+0000","last_peered":"2023-05-16T07:19:48.715425+0000","last_clean":"2023-05-16T07:19:48.715425+0000","last_became_active":"2023-05-16T07:19:48.711537+0000","last_became_peered":"2023-05-16T07:19:48.711537+0000","last_unstale":"2023-05-16T07:19:48.715425+0000","last_undegraded":"2023-05-16T07:19:48.715425+0000","last_fullsized":"2023-05-16T07:19:48.715425+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088371+0000","last_change":"2023-05-16T07:19:49.088371+0000","last_active":"2023-05-16T07:19:49.088371+0000","last_peered":"2023-05-16T07:19:49.088371+0000","last_clean":"2023-05-16T07:19:49.088371+0000","last_became_active":"2023-05-16T07:19:49.088075+0000","last_became_peered":"2023-05-16T07:19:49.088075+0000","last_unstale":"2023-05-16T07:19:49.088371+0000","last_undegraded":"2023-05-16T07:19:49.088371+0000","last_fullsized":"2023-05-16T07:19:49.088371+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.716004+0000","last_change":"2023-05-16T07:19:48.710746+0000","last_active":"2023-05-16T07:19:48.716004+0000","last_peered":"2023-05-16T07:19:48.716004+0000","last_clean":"2023-05-16T07:19:48.716004+0000","last_became_active":"2023-05-16T07:19:48.710437+0000","last_became_peered":"2023-05-16T07:19:48.710437+0000","last_unstale":"2023-05-16T07:19:48.716004+0000","last_undegraded":"2023-05-16T07:19:48.716004+0000","last_fullsized":"2023-05-16T07:19:48.716004+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.696071+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:19:48.696071+0000","last_peered":"2023-05-16T07:19:48.696071+0000","last_clean":"2023-05-16T07:19:48.696071+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:19:48.696071+0000","last_undegraded":"2023-05-16T07:19:48.696071+0000","last_fullsized":"2023-05-16T07:19:48.696071+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083154+0000","last_change":"2023-05-16T07:19:49.083154+0000","last_active":"2023-05-16T07:19:49.083154+0000","last_peered":"2023-05-16T07:19:49.083154+0000","last_clean":"2023-05-16T07:19:49.083154+0000","last_became_active":"2023-05-16T07:19:49.082866+0000","last_became_peered":"2023-05-16T07:19:49.082866+0000","last_unstale":"2023-05-16T07:19:49.083154+0000","last_undegraded":"2023-05-16T07:19:49.083154+0000","last_fullsized":"2023-05-16T07:19:49.083154+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721778+0000","last_change":"2023-05-16T07:19:48.716933+0000","last_active":"2023-05-16T07:19:48.721778+0000","last_peered":"2023-05-16T07:19:48.721778+0000","last_clean":"2023-05-16T07:19:48.721778+0000","last_became_active":"2023-05-16T07:19:48.716393+0000","last_became_peered":"2023-05-16T07:19:48.716393+0000","last_unstale":"2023-05-16T07:19:48.721778+0000","last_undegraded":"2023-05-16T07:19:48.721778+0000","last_fullsized":"2023-05-16T07:19:48.721778+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":101,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.705673+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:19:48.705673+0000","last_peered":"2023-05-16T07:19:48.705673+0000","last_clean":"2023-05-16T07:19:48.705673+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:19:48.705673+0000","last_undegraded":"2023-05-16T07:19:48.705673+0000","last_fullsized":"2023-05-16T07:19:48.705673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089115+0000","last_change":"2023-05-16T07:19:49.089115+0000","last_active":"2023-05-16T07:19:49.089115+0000","last_peered":"2023-05-16T07:19:49.089115+0000","last_clean":"2023-05-16T07:19:49.089115+0000","last_became_active":"2023-05-16T07:19:49.088833+0000","last_became_peered":"2023-05-16T07:19:49.088833+0000","last_unstale":"2023-05-16T07:19:49.089115+0000","last_undegraded":"2023-05-16T07:19:49.089115+0000","last_fullsized":"2023-05-16T07:19:49.089115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721798+0000","last_change":"2023-05-16T07:19:48.716929+0000","last_active":"2023-05-16T07:19:48.721798+0000","last_peered":"2023-05-16T07:19:48.721798+0000","last_clean":"2023-05-16T07:19:48.721798+0000","last_became_active":"2023-05-16T07:19:48.716406+0000","last_became_peered":"2023-05-16T07:19:48.716406+0000","last_unstale":"2023-05-16T07:19:48.721798+0000","last_undegraded":"2023-05-16T07:19:48.721798+0000","last_fullsized":"2023-05-16T07:19:48.721798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.703576+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:19:48.703576+0000","last_peered":"2023-05-16T07:19:48.703576+0000","last_clean":"2023-05-16T07:19:48.703576+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:19:48.703576+0000","last_undegraded":"2023-05-16T07:19:48.703576+0000","last_fullsized":"2023-05-16T07:19:48.703576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087814+0000","last_change":"2023-05-16T07:19:49.087814+0000","last_active":"2023-05-16T07:19:49.087814+0000","last_peered":"2023-05-16T07:19:49.087814+0000","last_clean":"2023-05-16T07:19:49.087814+0000","last_became_active":"2023-05-16T07:19:49.087536+0000","last_became_peered":"2023-05-16T07:19:49.087536+0000","last_unstale":"2023-05-16T07:19:49.087814+0000","last_undegraded":"2023-05-16T07:19:49.087814+0000","last_fullsized":"2023-05-16T07:19:49.087814+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.094125+0000","last_change":"2023-05-16T07:19:48.710011+0000","last_active":"2023-05-16T07:19:49.094125+0000","last_peered":"2023-05-16T07:19:49.094125+0000","last_clean":"2023-05-16T07:19:49.094125+0000","last_became_active":"2023-05-16T07:19:48.709388+0000","last_became_peered":"2023-05-16T07:19:48.709388+0000","last_unstale":"2023-05-16T07:19:49.094125+0000","last_undegraded":"2023-05-16T07:19:49.094125+0000","last_fullsized":"2023-05-16T07:19:49.094125+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":101,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.705613+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:19:48.705613+0000","last_peered":"2023-05-16T07:19:48.705613+0000","last_clean":"2023-05-16T07:19:48.705613+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:19:48.705613+0000","last_undegraded":"2023-05-16T07:19:48.705613+0000","last_fullsized":"2023-05-16T07:19:48.705613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.706285+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:19:48.706285+0000","last_peered":"2023-05-16T07:19:48.706285+0000","last_clean":"2023-05-16T07:19:48.706285+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:19:48.706285+0000","last_undegraded":"2023-05-16T07:19:48.706285+0000","last_fullsized":"2023-05-16T07:19:48.706285+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086658+0000","last_change":"2023-05-16T07:19:49.086658+0000","last_active":"2023-05-16T07:19:49.086658+0000","last_peered":"2023-05-16T07:19:49.086658+0000","last_clean":"2023-05-16T07:19:49.086658+0000","last_became_active":"2023-05-16T07:19:49.086348+0000","last_became_peered":"2023-05-16T07:19:49.086348+0000","last_unstale":"2023-05-16T07:19:49.086658+0000","last_undegraded":"2023-05-16T07:19:49.086658+0000","last_fullsized":"2023-05-16T07:19:49.086658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.069473+0000","last_change":"2023-05-16T07:19:48.709921+0000","last_active":"2023-05-16T07:19:49.069473+0000","last_peered":"2023-05-16T07:19:49.069473+0000","last_clean":"2023-05-16T07:19:49.069473+0000","last_became_active":"2023-05-16T07:19:48.709639+0000","last_became_peered":"2023-05-16T07:19:48.709639+0000","last_unstale":"2023-05-16T07:19:49.069473+0000","last_undegraded":"2023-05-16T07:19:49.069473+0000","last_fullsized":"2023-05-16T07:19:49.069473+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.706367+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:19:48.706367+0000","last_peered":"2023-05-16T07:19:48.706367+0000","last_clean":"2023-05-16T07:19:48.706367+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:19:48.706367+0000","last_undegraded":"2023-05-16T07:19:48.706367+0000","last_fullsized":"2023-05-16T07:19:48.706367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087160+0000","last_change":"2023-05-16T07:19:49.087160+0000","last_active":"2023-05-16T07:19:49.087160+0000","last_peered":"2023-05-16T07:19:49.087160+0000","last_clean":"2023-05-16T07:19:49.087160+0000","last_became_active":"2023-05-16T07:19:49.086865+0000","last_became_peered":"2023-05-16T07:19:49.086865+0000","last_unstale":"2023-05-16T07:19:49.087160+0000","last_undegraded":"2023-05-16T07:19:49.087160+0000","last_fullsized":"2023-05-16T07:19:49.087160+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714803+0000","last_change":"2023-05-16T07:19:48.712443+0000","last_active":"2023-05-16T07:19:48.714803+0000","last_peered":"2023-05-16T07:19:48.714803+0000","last_clean":"2023-05-16T07:19:48.714803+0000","last_became_active":"2023-05-16T07:19:48.712066+0000","last_became_peered":"2023-05-16T07:19:48.712066+0000","last_unstale":"2023-05-16T07:19:48.714803+0000","last_undegraded":"2023-05-16T07:19:48.714803+0000","last_fullsized":"2023-05-16T07:19:48.714803+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":102,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.709337+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:19:48.709337+0000","last_peered":"2023-05-16T07:19:48.709337+0000","last_clean":"2023-05-16T07:19:48.709337+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:19:48.709337+0000","last_undegraded":"2023-05-16T07:19:48.709337+0000","last_fullsized":"2023-05-16T07:19:48.709337+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083108+0000","last_change":"2023-05-16T07:19:49.083108+0000","last_active":"2023-05-16T07:19:49.083108+0000","last_peered":"2023-05-16T07:19:49.083108+0000","last_clean":"2023-05-16T07:19:49.083108+0000","last_became_active":"2023-05-16T07:19:49.082702+0000","last_became_peered":"2023-05-16T07:19:49.082702+0000","last_unstale":"2023-05-16T07:19:49.083108+0000","last_undegraded":"2023-05-16T07:19:49.083108+0000","last_fullsized":"2023-05-16T07:19:49.083108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"134'3","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.095505+0000","last_change":"2023-05-16T07:19:48.716009+0000","last_active":"2023-05-16T07:19:49.095505+0000","last_peered":"2023-05-16T07:19:49.095505+0000","last_clean":"2023-05-16T07:19:49.095505+0000","last_became_active":"2023-05-16T07:19:48.715769+0000","last_became_peered":"2023-05-16T07:19:48.715769+0000","last_unstale":"2023-05-16T07:19:49.095505+0000","last_undegraded":"2023-05-16T07:19:49.095505+0000","last_fullsized":"2023-05-16T07:19:49.095505+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":107,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.696318+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:19:48.696318+0000","last_peered":"2023-05-16T07:19:48.696318+0000","last_clean":"2023-05-16T07:19:48.696318+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:19:48.696318+0000","last_undegraded":"2023-05-16T07:19:48.696318+0000","last_fullsized":"2023-05-16T07:19:48.696318+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087217+0000","last_change":"2023-05-16T07:19:49.087217+0000","last_active":"2023-05-16T07:19:49.087217+0000","last_peered":"2023-05-16T07:19:49.087217+0000","last_clean":"2023-05-16T07:19:49.087217+0000","last_became_active":"2023-05-16T07:19:49.086919+0000","last_became_peered":"2023-05-16T07:19:49.086919+0000","last_unstale":"2023-05-16T07:19:49.087217+0000","last_undegraded":"2023-05-16T07:19:49.087217+0000","last_fullsized":"2023-05-16T07:19:49.087217+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083485+0000","last_change":"2023-05-16T07:19:48.707143+0000","last_active":"2023-05-16T07:19:49.083485+0000","last_peered":"2023-05-16T07:19:49.083485+0000","last_clean":"2023-05-16T07:19:49.083485+0000","last_became_active":"2023-05-16T07:19:48.706879+0000","last_became_peered":"2023-05-16T07:19:48.706879+0000","last_unstale":"2023-05-16T07:19:49.083485+0000","last_undegraded":"2023-05-16T07:19:49.083485+0000","last_fullsized":"2023-05-16T07:19:49.083485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.700854+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:19:48.700854+0000","last_peered":"2023-05-16T07:19:48.700854+0000","last_clean":"2023-05-16T07:19:48.700854+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:19:48.700854+0000","last_undegraded":"2023-05-16T07:19:48.700854+0000","last_fullsized":"2023-05-16T07:19:48.700854+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575054,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23212,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":21568,"kb_avail":94348628,"statfs":{"total":96636764160,"available":96612995072,"internally_reserved":0,"allocated":1675264,"data_stored":874634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22085632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.56200000000000006},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.77700000000000002,"15min":0.84199999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.54200000000000004,"15min":0.56499999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73299999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.623}]},{"osd":1,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.55800000000000005},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.54400000000000004,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.749,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56499999999999995},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.79300000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54500000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65000000000000002,"5min":0.752,"15min":0.81999999999999995},"last":0.47799999999999998}]},{"osd":5,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.499},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68200000000000005,"5min":0.748,"15min":0.751},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65700000000000003,"5min":0.879,"15min":0.879},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.53100000000000003,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.54900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575054,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22884,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":94348956,"statfs":{"total":96636764160,"available":96613330944,"internally_reserved":0,"allocated":1667072,"data_stored":874994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53600000000000003,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.85099999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71499999999999997,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78400000000000003,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38},"max":{"1min":0.73999999999999999,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.42999999999999999}]},{"osd":2,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70999999999999996,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.377,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.73899999999999999,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.495}]},{"osd":4,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57699999999999996,"5min":0.749,"15min":0.89400000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":4.048,"15min":4.048},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.92300000000000004},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70699999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575055,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18772,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":17152,"kb_avail":94353068,"statfs":{"total":96636764160,"available":96617541632,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17563648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.69899999999999995},"last":0.316},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.39800000000000002,"15min":0.42099999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.5,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35699999999999998,"15min":0.379},"min":{"1min":0.27200000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41099999999999998,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38400000000000001,"15min":0.40500000000000003},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52100000000000002,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.377,"15min":0.40000000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46300000000000002,"5min":0.622,"15min":0.64000000000000001},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.80000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.623},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.45700000000000002}]},{"osd":5,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61099999999999999,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77400000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.502},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.63500000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85599999999999998,"5min":0.94699999999999995,"15min":1.087},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.93700000000000006,"15min":1.204},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.47399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575055,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17748,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":16128,"kb_avail":94354092,"statfs":{"total":96636764160,"available":96618590208,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16515072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.36299999999999999,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.34499999999999997}]},{"osd":2,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34599999999999997,"15min":0.372},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.437,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.40500000000000003},"min":{"1min":0.251,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.311}]},{"osd":3,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.40799999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.49399999999999999,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65500000000000003,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0669999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575055,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19476,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":17856,"kb_avail":94352364,"statfs":{"total":96636764160,"available":96616820736,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18284544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33700000000000002,"15min":0.36099999999999999},"min":{"1min":0.23499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.59699999999999998,"15min":0.64900000000000002},"last":0.251},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.373},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.58199999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38600000000000001,"15min":0.40699999999999997},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.71199999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40600000000000003,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.23200000000000001},"max":{"1min":0.621,"5min":0.626,"15min":0.72899999999999998},"last":0.379}]},{"osd":3,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36299999999999999,"15min":0.39100000000000001},"min":{"1min":0.245,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.64700000000000002},"last":0.316},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39700000000000002,"15min":0.434},"min":{"1min":0.29799999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56399999999999995,"5min":0.64900000000000002,"15min":0.72899999999999998},"last":0.28999999999999998}]},{"osd":4,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.79200000000000004,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61099999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.113},"last":0.45400000000000001}]},{"osd":5,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.48299999999999998,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.63},"min":{"1min":0.432,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.5609999999999999},"last":0.435},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63200000000000001,"15min":0.66300000000000003},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.47},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.443,"5min":0.39200000000000002,"15min":0.36399999999999999},"max":{"1min":0.88400000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.41199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575054,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26516,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":24896,"kb_avail":94345324,"statfs":{"total":96636764160,"available":96609611776,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25493504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35799999999999998,"15min":0.379},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.5,"5min":0.54900000000000004,"15min":0.58799999999999997},"last":0.247}]},{"osd":1,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.48299999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.41599999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41399999999999998,"15min":0.433},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.53300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":0.877,"15min":0.91600000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.61399999999999999},"min":{"1min":0.40300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96399999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0489999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":1.0409999999999999},"last":0.622}]},{"osd":6,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0780000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.877,"15min":1.0049999999999999},"last":0.54000000000000004}]},{"osd":7,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61299999999999999,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.02},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18836,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":17216,"kb_avail":94353004,"statfs":{"total":96636764160,"available":96617476096,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17629184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36199999999999999,"15min":0.35999999999999999},"max":{"1min":2.5600000000000001,"5min":2.5600000000000001,"15min":2.5600000000000001},"last":0.502},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.75800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575054,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25940,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":24320,"kb_avail":94345900,"statfs":{"total":96636764160,"available":96610201600,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24903680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.32500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.641,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":2.847,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.6059999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.432}]},{"osd":2,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73799999999999999,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.503}]},{"osd":3,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.127,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.627}]},{"osd":4,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.503,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1150000000000002,"5min":3.1150000000000002,"15min":3.1150000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":3.5659999999999998,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.52600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:19:52.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:19:52.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.196+0000 7f865f47a700 1 Processor -- start 2023-05-16T07:19:52.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.197+0000 7f865f47a700 1 -- start start 2023-05-16T07:19:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.197+0000 7f865f47a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.197+0000 7f865f47a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8658063e30 0x7f86580bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.197+0000 7f865f47a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658062b10 con 0x7f8658154180 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865f47a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8658068620 con 0x7f8658063e30 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865f47a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86580622b0 con 0x7f8658153da0 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8658063e30 0x7f86580bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8658063e30 0x7f86580bda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50152/0 (socket says 172.21.15.92:50152) 2023-05-16T07:19:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 -- 172.21.15.92:0/1229627012 learned_addr learned my addr 172.21.15.92:0/1229627012 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 -- 172.21.15.92:0/1229627012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 msgr2=0x7f8658069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 --2- 172.21.15.92:0/1229627012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 -- 172.21.15.92:0/1229627012 >> v1:172.21.15.158:6789/0 conn(0x7f8658154180 legacy=0x7f8658063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 -- 172.21.15.92:0/1229627012 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8658068340 con 0x7f8658063e30 2023-05-16T07:19:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f865da17700 1 --2- 172.21.15.92:0/1229627012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f86580bda00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f8650006fd0 tx=0x7f86500090f0).ready entity=mon.0 client_cookie=f144ebc18293f40d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f8657fff700 1 -- 172.21.15.92:0/1229627012 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000b600 con 0x7f8658063e30 2023-05-16T07:19:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f8657fff700 1 -- 172.21.15.92:0/1229627012 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865000b760 con 0x7f8658063e30 2023-05-16T07:19:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.198+0000 7f8657fff700 1 -- 172.21.15.92:0/1229627012 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000bae0 con 0x7f8658063e30 2023-05-16T07:19:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 -- 172.21.15.92:0/1229627012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 msgr2=0x7f86580bda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 --2- 172.21.15.92:0/1229627012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f86580bda00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f8650006fd0 tx=0x7f86500090f0).stop 2023-05-16T07:19:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 -- 172.21.15.92:0/1229627012 shutdown_connections 2023-05-16T07:19:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 --2- 172.21.15.92:0/1229627012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f86580bda00 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 --2- 172.21.15.92:0/1229627012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 -- 172.21.15.92:0/1229627012 >> 172.21.15.92:0/1229627012 conn(0x7f86580c8aa0 msgr2=0x7f86580c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:52.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 -- 172.21.15.92:0/1229627012 shutdown_connections 2023-05-16T07:19:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.199+0000 7f865f47a700 1 -- 172.21.15.92:0/1229627012 wait complete. 2023-05-16T07:19:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.200+0000 7f865f47a700 1 Processor -- start 2023-05-16T07:19:52.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.200+0000 7f865f47a700 1 -- start start 2023-05-16T07:19:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f8658163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8658154180 0x7f865814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86581651c0 con 0x7f8658063e30 2023-05-16T07:19:52.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86580c0b10 con 0x7f8658154180 2023-05-16T07:19:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8658164110 con 0x7f8658153da0 2023-05-16T07:19:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f8658163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f8658163b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50156/0 (socket says 172.21.15.92:50156) 2023-05-16T07:19:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 -- 172.21.15.92:0/354519007 learned_addr learned my addr 172.21.15.92:0/354519007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865da17700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8658154180 0x7f865814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865ca15700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 -- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 msgr2=0x7f8658164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658164cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 -- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8658154180 msgr2=0x7f865814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8658154180 0x7f865814ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 -- 172.21.15.92:0/354519007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650009370 con 0x7f8658063e30 2023-05-16T07:19:52.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865da17700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8658154180 0x7f865814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:52.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865d216700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f8658163b00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f864800bc30 tx=0x7f8648009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800c7a0 con 0x7f8658063e30 2023-05-16T07:19:52.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648010070 con 0x7f8658063e30 2023-05-16T07:19:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86581643f0 con 0x7f8658063e30 2023-05-16T07:19:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800c960 con 0x7f8658063e30 2023-05-16T07:19:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.201+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86581649b0 con 0x7f8658063e30 2023-05-16T07:19:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.202+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865806c350 con 0x7f8658063e30 2023-05-16T07:19:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.203+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f864801c070 con 0x7f8658063e30 2023-05-16T07:19:52.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.203+0000 7f86557fa700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f864405ed90 0x7f8644061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.203+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f8648079510 con 0x7f8658063e30 2023-05-16T07:19:52.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.203+0000 7f865ca15700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f864405ed90 0x7f8644061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.204+0000 7f865ca15700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f864405ed90 0x7f8644061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8658147b40 tx=0x7f864c008040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.207+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f864804a850 con 0x7f8658063e30 2023-05-16T07:19:52.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.365+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86580624c0 con 0x7f864405ed90 2023-05-16T07:19:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.376+0000 7f86557fa700 1 -- 172.21.15.92:0/354519007 <== mgr.7159 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361468 (secure 0 0 0) 0x7f86580624c0 con 0x7f864405ed90 2023-05-16T07:19:52.389 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f864405ed90 msgr2=0x7f8644061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f864405ed90 0x7f8644061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8658147b40 tx=0x7f864c008040).stop 2023-05-16T07:19:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 msgr2=0x7f8658163b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f8658163b00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f864800bc30 tx=0x7f8648009840).stop 2023-05-16T07:19:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 shutdown_connections 2023-05-16T07:19:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f864405ed90 0x7f8644061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8658154180 0x7f865814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8658063e30 0x7f8658163b00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 --2- 172.21.15.92:0/354519007 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8658153da0 0x7f8658164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 >> 172.21.15.92:0/354519007 conn(0x7f86580c8aa0 msgr2=0x7f86580ab060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 shutdown_connections 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.379+0000 7f865f47a700 1 -- 172.21.15.92:0/354519007 wait complete. 2023-05-16T07:19:52.393 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:19:52.394 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:19:50.416764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":173384,"kb_used_data":12936,"kb_used_omap":0,"kb_used_meta":160384,"kb_avail":754801336,"statfs":{"total":773094113280,"available":772916568064,"internally_reserved":0,"allocated":13246464,"data_stored":6949522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":164233216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11159,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.585454"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093174+0000","last_change":"2023-05-16T07:19:49.093174+0000","last_active":"2023-05-16T07:19:49.093174+0000","last_peered":"2023-05-16T07:19:49.093174+0000","last_clean":"2023-05-16T07:19:49.093174+0000","last_became_active":"2023-05-16T07:19:49.092883+0000","last_became_peered":"2023-05-16T07:19:49.092883+0000","last_unstale":"2023-05-16T07:19:49.093174+0000","last_undegraded":"2023-05-16T07:19:49.093174+0000","last_fullsized":"2023-05-16T07:19:49.093174+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"134'3","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089809+0000","last_change":"2023-05-16T07:19:48.711876+0000","last_active":"2023-05-16T07:19:49.089809+0000","last_peered":"2023-05-16T07:19:49.089809+0000","last_clean":"2023-05-16T07:19:49.089809+0000","last_became_active":"2023-05-16T07:19:48.711598+0000","last_became_peered":"2023-05-16T07:19:48.711598+0000","last_unstale":"2023-05-16T07:19:49.089809+0000","last_undegraded":"2023-05-16T07:19:49.089809+0000","last_fullsized":"2023-05-16T07:19:49.089809+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.082353+0000","last_change":"2023-05-16T07:19:49.082353+0000","last_active":"2023-05-16T07:19:49.082353+0000","last_peered":"2023-05-16T07:19:49.082353+0000","last_clean":"2023-05-16T07:19:49.082353+0000","last_became_active":"2023-05-16T07:19:49.082075+0000","last_became_peered":"2023-05-16T07:19:49.082075+0000","last_unstale":"2023-05-16T07:19:49.082353+0000","last_undegraded":"2023-05-16T07:19:49.082353+0000","last_fullsized":"2023-05-16T07:19:49.082353+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719395+0000","last_change":"2023-05-16T07:19:48.714290+0000","last_active":"2023-05-16T07:19:48.719395+0000","last_peered":"2023-05-16T07:19:48.719395+0000","last_clean":"2023-05-16T07:19:48.719395+0000","last_became_active":"2023-05-16T07:19:48.714026+0000","last_became_peered":"2023-05-16T07:19:48.714026+0000","last_unstale":"2023-05-16T07:19:48.719395+0000","last_undegraded":"2023-05-16T07:19:48.719395+0000","last_fullsized":"2023-05-16T07:19:48.719395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.080691+0000","last_change":"2023-05-16T07:19:49.080691+0000","last_active":"2023-05-16T07:19:49.080691+0000","last_peered":"2023-05-16T07:19:49.080691+0000","last_clean":"2023-05-16T07:19:49.080691+0000","last_became_active":"2023-05-16T07:19:49.080486+0000","last_became_peered":"2023-05-16T07:19:49.080486+0000","last_unstale":"2023-05-16T07:19:49.080691+0000","last_undegraded":"2023-05-16T07:19:49.080691+0000","last_fullsized":"2023-05-16T07:19:49.080691+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720955+0000","last_change":"2023-05-16T07:19:48.719963+0000","last_active":"2023-05-16T07:19:48.720955+0000","last_peered":"2023-05-16T07:19:48.720955+0000","last_clean":"2023-05-16T07:19:48.720955+0000","last_became_active":"2023-05-16T07:19:48.719680+0000","last_became_peered":"2023-05-16T07:19:48.719680+0000","last_unstale":"2023-05-16T07:19:48.720955+0000","last_undegraded":"2023-05-16T07:19:48.720955+0000","last_fullsized":"2023-05-16T07:19:48.720955+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086571+0000","last_change":"2023-05-16T07:19:49.086571+0000","last_active":"2023-05-16T07:19:49.086571+0000","last_peered":"2023-05-16T07:19:49.086571+0000","last_clean":"2023-05-16T07:19:49.086571+0000","last_became_active":"2023-05-16T07:19:49.086297+0000","last_became_peered":"2023-05-16T07:19:49.086297+0000","last_unstale":"2023-05-16T07:19:49.086571+0000","last_undegraded":"2023-05-16T07:19:49.086571+0000","last_fullsized":"2023-05-16T07:19:49.086571+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099097+0000","last_change":"2023-05-16T07:19:48.720083+0000","last_active":"2023-05-16T07:19:49.099097+0000","last_peered":"2023-05-16T07:19:49.099097+0000","last_clean":"2023-05-16T07:19:49.099097+0000","last_became_active":"2023-05-16T07:19:48.719666+0000","last_became_peered":"2023-05-16T07:19:48.719666+0000","last_unstale":"2023-05-16T07:19:49.099097+0000","last_undegraded":"2023-05-16T07:19:49.099097+0000","last_fullsized":"2023-05-16T07:19:49.099097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.075151+0000","last_change":"2023-05-16T07:19:49.075151+0000","last_active":"2023-05-16T07:19:49.075151+0000","last_peered":"2023-05-16T07:19:49.075151+0000","last_clean":"2023-05-16T07:19:49.075151+0000","last_became_active":"2023-05-16T07:19:49.074891+0000","last_became_peered":"2023-05-16T07:19:49.074891+0000","last_unstale":"2023-05-16T07:19:49.075151+0000","last_undegraded":"2023-05-16T07:19:49.075151+0000","last_fullsized":"2023-05-16T07:19:49.075151+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.075042+0000","last_change":"2023-05-16T07:19:48.715541+0000","last_active":"2023-05-16T07:19:49.075042+0000","last_peered":"2023-05-16T07:19:49.075042+0000","last_clean":"2023-05-16T07:19:49.075042+0000","last_became_active":"2023-05-16T07:19:48.715256+0000","last_became_peered":"2023-05-16T07:19:48.715256+0000","last_unstale":"2023-05-16T07:19:49.075042+0000","last_undegraded":"2023-05-16T07:19:49.075042+0000","last_fullsized":"2023-05-16T07:19:49.075042+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093115+0000","last_change":"2023-05-16T07:19:49.093115+0000","last_active":"2023-05-16T07:19:49.093115+0000","last_peered":"2023-05-16T07:19:49.093115+0000","last_clean":"2023-05-16T07:19:49.093115+0000","last_became_active":"2023-05-16T07:19:49.092862+0000","last_became_peered":"2023-05-16T07:19:49.092862+0000","last_unstale":"2023-05-16T07:19:49.093115+0000","last_undegraded":"2023-05-16T07:19:49.093115+0000","last_fullsized":"2023-05-16T07:19:49.093115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.078112+0000","last_change":"2023-05-16T07:19:48.707523+0000","last_active":"2023-05-16T07:19:49.078112+0000","last_peered":"2023-05-16T07:19:49.078112+0000","last_clean":"2023-05-16T07:19:49.078112+0000","last_became_active":"2023-05-16T07:19:48.706874+0000","last_became_peered":"2023-05-16T07:19:48.706874+0000","last_unstale":"2023-05-16T07:19:49.078112+0000","last_undegraded":"2023-05-16T07:19:49.078112+0000","last_fullsized":"2023-05-16T07:19:49.078112+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089012+0000","last_change":"2023-05-16T07:19:49.089012+0000","last_active":"2023-05-16T07:19:49.089012+0000","last_peered":"2023-05-16T07:19:49.089012+0000","last_clean":"2023-05-16T07:19:49.089012+0000","last_became_active":"2023-05-16T07:19:49.088777+0000","last_became_peered":"2023-05-16T07:19:49.088777+0000","last_unstale":"2023-05-16T07:19:49.089012+0000","last_undegraded":"2023-05-16T07:19:49.089012+0000","last_fullsized":"2023-05-16T07:19:49.089012+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714356+0000","last_change":"2023-05-16T07:19:48.711407+0000","last_active":"2023-05-16T07:19:48.714356+0000","last_peered":"2023-05-16T07:19:48.714356+0000","last_clean":"2023-05-16T07:19:48.714356+0000","last_became_active":"2023-05-16T07:19:48.710678+0000","last_became_peered":"2023-05-16T07:19:48.710678+0000","last_unstale":"2023-05-16T07:19:48.714356+0000","last_undegraded":"2023-05-16T07:19:48.714356+0000","last_fullsized":"2023-05-16T07:19:48.714356+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.080756+0000","last_change":"2023-05-16T07:19:49.080756+0000","last_active":"2023-05-16T07:19:49.080756+0000","last_peered":"2023-05-16T07:19:49.080756+0000","last_clean":"2023-05-16T07:19:49.080756+0000","last_became_active":"2023-05-16T07:19:49.080485+0000","last_became_peered":"2023-05-16T07:19:49.080485+0000","last_unstale":"2023-05-16T07:19:49.080756+0000","last_undegraded":"2023-05-16T07:19:49.080756+0000","last_fullsized":"2023-05-16T07:19:49.080756+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719618+0000","last_change":"2023-05-16T07:19:48.714875+0000","last_active":"2023-05-16T07:19:48.719618+0000","last_peered":"2023-05-16T07:19:48.719618+0000","last_clean":"2023-05-16T07:19:48.719618+0000","last_became_active":"2023-05-16T07:19:48.714526+0000","last_became_peered":"2023-05-16T07:19:48.714526+0000","last_unstale":"2023-05-16T07:19:48.719618+0000","last_undegraded":"2023-05-16T07:19:48.719618+0000","last_fullsized":"2023-05-16T07:19:48.719618+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090049+0000","last_change":"2023-05-16T07:19:49.090049+0000","last_active":"2023-05-16T07:19:49.090049+0000","last_peered":"2023-05-16T07:19:49.090049+0000","last_clean":"2023-05-16T07:19:49.090049+0000","last_became_active":"2023-05-16T07:19:49.089209+0000","last_became_peered":"2023-05-16T07:19:49.089209+0000","last_unstale":"2023-05-16T07:19:49.090049+0000","last_undegraded":"2023-05-16T07:19:49.090049+0000","last_fullsized":"2023-05-16T07:19:49.090049+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723402+0000","last_change":"2023-05-16T07:19:48.720485+0000","last_active":"2023-05-16T07:19:48.723402+0000","last_peered":"2023-05-16T07:19:48.723402+0000","last_clean":"2023-05-16T07:19:48.723402+0000","last_became_active":"2023-05-16T07:19:48.720202+0000","last_became_peered":"2023-05-16T07:19:48.720202+0000","last_unstale":"2023-05-16T07:19:48.723402+0000","last_undegraded":"2023-05-16T07:19:48.723402+0000","last_fullsized":"2023-05-16T07:19:48.723402+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086671+0000","last_change":"2023-05-16T07:19:49.086671+0000","last_active":"2023-05-16T07:19:49.086671+0000","last_peered":"2023-05-16T07:19:49.086671+0000","last_clean":"2023-05-16T07:19:49.086671+0000","last_became_active":"2023-05-16T07:19:49.084485+0000","last_became_peered":"2023-05-16T07:19:49.084485+0000","last_unstale":"2023-05-16T07:19:49.086671+0000","last_undegraded":"2023-05-16T07:19:49.086671+0000","last_fullsized":"2023-05-16T07:19:49.086671+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721339+0000","last_change":"2023-05-16T07:19:48.712449+0000","last_active":"2023-05-16T07:19:48.721339+0000","last_peered":"2023-05-16T07:19:48.721339+0000","last_clean":"2023-05-16T07:19:48.721339+0000","last_became_active":"2023-05-16T07:19:48.712111+0000","last_became_peered":"2023-05-16T07:19:48.712111+0000","last_unstale":"2023-05-16T07:19:48.721339+0000","last_undegraded":"2023-05-16T07:19:48.721339+0000","last_fullsized":"2023-05-16T07:19:48.721339+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.100072+0000","last_change":"2023-05-16T07:19:49.100072+0000","last_active":"2023-05-16T07:19:49.100072+0000","last_peered":"2023-05-16T07:19:49.100072+0000","last_clean":"2023-05-16T07:19:49.100072+0000","last_became_active":"2023-05-16T07:19:49.099855+0000","last_became_peered":"2023-05-16T07:19:49.099855+0000","last_unstale":"2023-05-16T07:19:49.100072+0000","last_undegraded":"2023-05-16T07:19:49.100072+0000","last_fullsized":"2023-05-16T07:19:49.100072+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719533+0000","last_change":"2023-05-16T07:19:48.714615+0000","last_active":"2023-05-16T07:19:48.719533+0000","last_peered":"2023-05-16T07:19:48.719533+0000","last_clean":"2023-05-16T07:19:48.719533+0000","last_became_active":"2023-05-16T07:19:48.714114+0000","last_became_peered":"2023-05-16T07:19:48.714114+0000","last_unstale":"2023-05-16T07:19:48.719533+0000","last_undegraded":"2023-05-16T07:19:48.719533+0000","last_fullsized":"2023-05-16T07:19:48.719533+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.092508+0000","last_change":"2023-05-16T07:19:49.092508+0000","last_active":"2023-05-16T07:19:49.092508+0000","last_peered":"2023-05-16T07:19:49.092508+0000","last_clean":"2023-05-16T07:19:49.092508+0000","last_became_active":"2023-05-16T07:19:49.092022+0000","last_became_peered":"2023-05-16T07:19:49.092022+0000","last_unstale":"2023-05-16T07:19:49.092508+0000","last_undegraded":"2023-05-16T07:19:49.092508+0000","last_fullsized":"2023-05-16T07:19:49.092508+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099628+0000","last_change":"2023-05-16T07:19:48.715882+0000","last_active":"2023-05-16T07:19:49.099628+0000","last_peered":"2023-05-16T07:19:49.099628+0000","last_clean":"2023-05-16T07:19:49.099628+0000","last_became_active":"2023-05-16T07:19:48.715595+0000","last_became_peered":"2023-05-16T07:19:48.715595+0000","last_unstale":"2023-05-16T07:19:49.099628+0000","last_undegraded":"2023-05-16T07:19:49.099628+0000","last_fullsized":"2023-05-16T07:19:49.099628+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.096249+0000","last_change":"2023-05-16T07:19:49.096249+0000","last_active":"2023-05-16T07:19:49.096249+0000","last_peered":"2023-05-16T07:19:49.096249+0000","last_clean":"2023-05-16T07:19:49.096249+0000","last_became_active":"2023-05-16T07:19:49.095791+0000","last_became_peered":"2023-05-16T07:19:49.095791+0000","last_unstale":"2023-05-16T07:19:49.096249+0000","last_undegraded":"2023-05-16T07:19:49.096249+0000","last_fullsized":"2023-05-16T07:19:49.096249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.072369+0000","last_change":"2023-05-16T07:19:48.719099+0000","last_active":"2023-05-16T07:19:49.072369+0000","last_peered":"2023-05-16T07:19:49.072369+0000","last_clean":"2023-05-16T07:19:49.072369+0000","last_became_active":"2023-05-16T07:19:48.718788+0000","last_became_peered":"2023-05-16T07:19:48.718788+0000","last_unstale":"2023-05-16T07:19:49.072369+0000","last_undegraded":"2023-05-16T07:19:49.072369+0000","last_fullsized":"2023-05-16T07:19:49.072369+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085323+0000","last_change":"2023-05-16T07:19:49.085323+0000","last_active":"2023-05-16T07:19:49.085323+0000","last_peered":"2023-05-16T07:19:49.085323+0000","last_clean":"2023-05-16T07:19:49.085323+0000","last_became_active":"2023-05-16T07:19:49.084896+0000","last_became_peered":"2023-05-16T07:19:49.084896+0000","last_unstale":"2023-05-16T07:19:49.085323+0000","last_undegraded":"2023-05-16T07:19:49.085323+0000","last_fullsized":"2023-05-16T07:19:49.085323+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715367+0000","last_change":"2023-05-16T07:19:48.713283+0000","last_active":"2023-05-16T07:19:48.715367+0000","last_peered":"2023-05-16T07:19:48.715367+0000","last_clean":"2023-05-16T07:19:48.715367+0000","last_became_active":"2023-05-16T07:19:48.713015+0000","last_became_peered":"2023-05-16T07:19:48.713015+0000","last_unstale":"2023-05-16T07:19:48.715367+0000","last_undegraded":"2023-05-16T07:19:48.715367+0000","last_fullsized":"2023-05-16T07:19:48.715367+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.075003+0000","last_change":"2023-05-16T07:19:49.075003+0000","last_active":"2023-05-16T07:19:49.075003+0000","last_peered":"2023-05-16T07:19:49.075003+0000","last_clean":"2023-05-16T07:19:49.075003+0000","last_became_active":"2023-05-16T07:19:49.074782+0000","last_became_peered":"2023-05-16T07:19:49.074782+0000","last_unstale":"2023-05-16T07:19:49.075003+0000","last_undegraded":"2023-05-16T07:19:49.075003+0000","last_fullsized":"2023-05-16T07:19:49.075003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723077+0000","last_change":"2023-05-16T07:19:48.718570+0000","last_active":"2023-05-16T07:19:48.723077+0000","last_peered":"2023-05-16T07:19:48.723077+0000","last_clean":"2023-05-16T07:19:48.723077+0000","last_became_active":"2023-05-16T07:19:48.718292+0000","last_became_peered":"2023-05-16T07:19:48.718292+0000","last_unstale":"2023-05-16T07:19:48.723077+0000","last_undegraded":"2023-05-16T07:19:48.723077+0000","last_fullsized":"2023-05-16T07:19:48.723077+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.091333+0000","last_change":"2023-05-16T07:19:49.091333+0000","last_active":"2023-05-16T07:19:49.091333+0000","last_peered":"2023-05-16T07:19:49.091333+0000","last_clean":"2023-05-16T07:19:49.091333+0000","last_became_active":"2023-05-16T07:19:49.091101+0000","last_became_peered":"2023-05-16T07:19:49.091101+0000","last_unstale":"2023-05-16T07:19:49.091333+0000","last_undegraded":"2023-05-16T07:19:49.091333+0000","last_fullsized":"2023-05-16T07:19:49.091333+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099091+0000","last_change":"2023-05-16T07:19:48.716129+0000","last_active":"2023-05-16T07:19:49.099091+0000","last_peered":"2023-05-16T07:19:49.099091+0000","last_clean":"2023-05-16T07:19:49.099091+0000","last_became_active":"2023-05-16T07:19:48.715860+0000","last_became_peered":"2023-05-16T07:19:48.715860+0000","last_unstale":"2023-05-16T07:19:49.099091+0000","last_undegraded":"2023-05-16T07:19:49.099091+0000","last_fullsized":"2023-05-16T07:19:49.099091+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099291+0000","last_change":"2023-05-16T07:19:49.099291+0000","last_active":"2023-05-16T07:19:49.099291+0000","last_peered":"2023-05-16T07:19:49.099291+0000","last_clean":"2023-05-16T07:19:49.099291+0000","last_became_active":"2023-05-16T07:19:49.098985+0000","last_became_peered":"2023-05-16T07:19:49.098985+0000","last_unstale":"2023-05-16T07:19:49.099291+0000","last_undegraded":"2023-05-16T07:19:49.099291+0000","last_fullsized":"2023-05-16T07:19:49.099291+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719487+0000","last_change":"2023-05-16T07:19:48.717134+0000","last_active":"2023-05-16T07:19:48.719487+0000","last_peered":"2023-05-16T07:19:48.719487+0000","last_clean":"2023-05-16T07:19:48.719487+0000","last_became_active":"2023-05-16T07:19:48.716805+0000","last_became_peered":"2023-05-16T07:19:48.716805+0000","last_unstale":"2023-05-16T07:19:48.719487+0000","last_undegraded":"2023-05-16T07:19:48.719487+0000","last_fullsized":"2023-05-16T07:19:48.719487+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088827+0000","last_change":"2023-05-16T07:19:49.088827+0000","last_active":"2023-05-16T07:19:49.088827+0000","last_peered":"2023-05-16T07:19:49.088827+0000","last_clean":"2023-05-16T07:19:49.088827+0000","last_became_active":"2023-05-16T07:19:49.088537+0000","last_became_peered":"2023-05-16T07:19:49.088537+0000","last_unstale":"2023-05-16T07:19:49.088827+0000","last_undegraded":"2023-05-16T07:19:49.088827+0000","last_fullsized":"2023-05-16T07:19:49.088827+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.718345+0000","last_change":"2023-05-16T07:19:48.714398+0000","last_active":"2023-05-16T07:19:48.718345+0000","last_peered":"2023-05-16T07:19:48.718345+0000","last_clean":"2023-05-16T07:19:48.718345+0000","last_became_active":"2023-05-16T07:19:48.714098+0000","last_became_peered":"2023-05-16T07:19:48.714098+0000","last_unstale":"2023-05-16T07:19:48.718345+0000","last_undegraded":"2023-05-16T07:19:48.718345+0000","last_fullsized":"2023-05-16T07:19:48.718345+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093197+0000","last_change":"2023-05-16T07:19:49.093197+0000","last_active":"2023-05-16T07:19:49.093197+0000","last_peered":"2023-05-16T07:19:49.093197+0000","last_clean":"2023-05-16T07:19:49.093197+0000","last_became_active":"2023-05-16T07:19:49.092927+0000","last_became_peered":"2023-05-16T07:19:49.092927+0000","last_unstale":"2023-05-16T07:19:49.093197+0000","last_undegraded":"2023-05-16T07:19:49.093197+0000","last_fullsized":"2023-05-16T07:19:49.093197+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.071309+0000","last_change":"2023-05-16T07:19:48.716306+0000","last_active":"2023-05-16T07:19:49.071309+0000","last_peered":"2023-05-16T07:19:49.071309+0000","last_clean":"2023-05-16T07:19:49.071309+0000","last_became_active":"2023-05-16T07:19:48.715984+0000","last_became_peered":"2023-05-16T07:19:48.715984+0000","last_unstale":"2023-05-16T07:19:49.071309+0000","last_undegraded":"2023-05-16T07:19:49.071309+0000","last_fullsized":"2023-05-16T07:19:49.071309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088701+0000","last_change":"2023-05-16T07:19:49.088701+0000","last_active":"2023-05-16T07:19:49.088701+0000","last_peered":"2023-05-16T07:19:49.088701+0000","last_clean":"2023-05-16T07:19:49.088701+0000","last_became_active":"2023-05-16T07:19:49.088435+0000","last_became_peered":"2023-05-16T07:19:49.088435+0000","last_unstale":"2023-05-16T07:19:49.088701+0000","last_undegraded":"2023-05-16T07:19:49.088701+0000","last_fullsized":"2023-05-16T07:19:49.088701+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.718907+0000","last_change":"2023-05-16T07:19:48.711741+0000","last_active":"2023-05-16T07:19:48.718907+0000","last_peered":"2023-05-16T07:19:48.718907+0000","last_clean":"2023-05-16T07:19:48.718907+0000","last_became_active":"2023-05-16T07:19:48.711508+0000","last_became_peered":"2023-05-16T07:19:48.711508+0000","last_unstale":"2023-05-16T07:19:48.718907+0000","last_undegraded":"2023-05-16T07:19:48.718907+0000","last_fullsized":"2023-05-16T07:19:48.718907+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085844+0000","last_change":"2023-05-16T07:19:49.085844+0000","last_active":"2023-05-16T07:19:49.085844+0000","last_peered":"2023-05-16T07:19:49.085844+0000","last_clean":"2023-05-16T07:19:49.085844+0000","last_became_active":"2023-05-16T07:19:49.085554+0000","last_became_peered":"2023-05-16T07:19:49.085554+0000","last_unstale":"2023-05-16T07:19:49.085844+0000","last_undegraded":"2023-05-16T07:19:49.085844+0000","last_fullsized":"2023-05-16T07:19:49.085844+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.076491+0000","last_change":"2023-05-16T07:19:48.718393+0000","last_active":"2023-05-16T07:19:49.076491+0000","last_peered":"2023-05-16T07:19:49.076491+0000","last_clean":"2023-05-16T07:19:49.076491+0000","last_became_active":"2023-05-16T07:19:48.718151+0000","last_became_peered":"2023-05-16T07:19:48.718151+0000","last_unstale":"2023-05-16T07:19:49.076491+0000","last_undegraded":"2023-05-16T07:19:49.076491+0000","last_fullsized":"2023-05-16T07:19:49.076491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087091+0000","last_change":"2023-05-16T07:19:49.087091+0000","last_active":"2023-05-16T07:19:49.087091+0000","last_peered":"2023-05-16T07:19:49.087091+0000","last_clean":"2023-05-16T07:19:49.087091+0000","last_became_active":"2023-05-16T07:19:49.086778+0000","last_became_peered":"2023-05-16T07:19:49.086778+0000","last_unstale":"2023-05-16T07:19:49.087091+0000","last_undegraded":"2023-05-16T07:19:49.087091+0000","last_fullsized":"2023-05-16T07:19:49.087091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721818+0000","last_change":"2023-05-16T07:19:48.717012+0000","last_active":"2023-05-16T07:19:48.721818+0000","last_peered":"2023-05-16T07:19:48.721818+0000","last_clean":"2023-05-16T07:19:48.721818+0000","last_became_active":"2023-05-16T07:19:48.716750+0000","last_became_peered":"2023-05-16T07:19:48.716750+0000","last_unstale":"2023-05-16T07:19:48.721818+0000","last_undegraded":"2023-05-16T07:19:48.721818+0000","last_fullsized":"2023-05-16T07:19:48.721818+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087451+0000","last_change":"2023-05-16T07:19:49.087451+0000","last_active":"2023-05-16T07:19:49.087451+0000","last_peered":"2023-05-16T07:19:49.087451+0000","last_clean":"2023-05-16T07:19:49.087451+0000","last_became_active":"2023-05-16T07:19:49.087066+0000","last_became_peered":"2023-05-16T07:19:49.087066+0000","last_unstale":"2023-05-16T07:19:49.087451+0000","last_undegraded":"2023-05-16T07:19:49.087451+0000","last_fullsized":"2023-05-16T07:19:49.087451+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719428+0000","last_change":"2023-05-16T07:19:48.715679+0000","last_active":"2023-05-16T07:19:48.719428+0000","last_peered":"2023-05-16T07:19:48.719428+0000","last_clean":"2023-05-16T07:19:48.719428+0000","last_became_active":"2023-05-16T07:19:48.715393+0000","last_became_peered":"2023-05-16T07:19:48.715393+0000","last_unstale":"2023-05-16T07:19:48.719428+0000","last_undegraded":"2023-05-16T07:19:48.719428+0000","last_fullsized":"2023-05-16T07:19:48.719428+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088080+0000","last_change":"2023-05-16T07:19:49.088080+0000","last_active":"2023-05-16T07:19:49.088080+0000","last_peered":"2023-05-16T07:19:49.088080+0000","last_clean":"2023-05-16T07:19:49.088080+0000","last_became_active":"2023-05-16T07:19:49.087793+0000","last_became_peered":"2023-05-16T07:19:49.087793+0000","last_unstale":"2023-05-16T07:19:49.088080+0000","last_undegraded":"2023-05-16T07:19:49.088080+0000","last_fullsized":"2023-05-16T07:19:49.088080+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.717698+0000","last_change":"2023-05-16T07:19:48.710180+0000","last_active":"2023-05-16T07:19:48.717698+0000","last_peered":"2023-05-16T07:19:48.717698+0000","last_clean":"2023-05-16T07:19:48.717698+0000","last_became_active":"2023-05-16T07:19:48.709728+0000","last_became_peered":"2023-05-16T07:19:48.709728+0000","last_unstale":"2023-05-16T07:19:48.717698+0000","last_undegraded":"2023-05-16T07:19:48.717698+0000","last_fullsized":"2023-05-16T07:19:48.717698+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.092263+0000","last_change":"2023-05-16T07:19:49.092263+0000","last_active":"2023-05-16T07:19:49.092263+0000","last_peered":"2023-05-16T07:19:49.092263+0000","last_clean":"2023-05-16T07:19:49.092263+0000","last_became_active":"2023-05-16T07:19:49.089550+0000","last_became_peered":"2023-05-16T07:19:49.089550+0000","last_unstale":"2023-05-16T07:19:49.092263+0000","last_undegraded":"2023-05-16T07:19:49.092263+0000","last_fullsized":"2023-05-16T07:19:49.092263+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719506+0000","last_change":"2023-05-16T07:19:48.717542+0000","last_active":"2023-05-16T07:19:48.719506+0000","last_peered":"2023-05-16T07:19:48.719506+0000","last_clean":"2023-05-16T07:19:48.719506+0000","last_became_active":"2023-05-16T07:19:48.717270+0000","last_became_peered":"2023-05-16T07:19:48.717270+0000","last_unstale":"2023-05-16T07:19:48.719506+0000","last_undegraded":"2023-05-16T07:19:48.719506+0000","last_fullsized":"2023-05-16T07:19:48.719506+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088736+0000","last_change":"2023-05-16T07:19:49.088736+0000","last_active":"2023-05-16T07:19:49.088736+0000","last_peered":"2023-05-16T07:19:49.088736+0000","last_clean":"2023-05-16T07:19:49.088736+0000","last_became_active":"2023-05-16T07:19:49.088429+0000","last_became_peered":"2023-05-16T07:19:49.088429+0000","last_unstale":"2023-05-16T07:19:49.088736+0000","last_undegraded":"2023-05-16T07:19:49.088736+0000","last_fullsized":"2023-05-16T07:19:49.088736+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721318+0000","last_change":"2023-05-16T07:19:48.712342+0000","last_active":"2023-05-16T07:19:48.721318+0000","last_peered":"2023-05-16T07:19:48.721318+0000","last_clean":"2023-05-16T07:19:48.721318+0000","last_became_active":"2023-05-16T07:19:48.711861+0000","last_became_peered":"2023-05-16T07:19:48.711861+0000","last_unstale":"2023-05-16T07:19:48.721318+0000","last_undegraded":"2023-05-16T07:19:48.721318+0000","last_fullsized":"2023-05-16T07:19:48.721318+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086365+0000","last_change":"2023-05-16T07:19:49.086365+0000","last_active":"2023-05-16T07:19:49.086365+0000","last_peered":"2023-05-16T07:19:49.086365+0000","last_clean":"2023-05-16T07:19:49.086365+0000","last_became_active":"2023-05-16T07:19:49.086098+0000","last_became_peered":"2023-05-16T07:19:49.086098+0000","last_unstale":"2023-05-16T07:19:49.086365+0000","last_undegraded":"2023-05-16T07:19:49.086365+0000","last_fullsized":"2023-05-16T07:19:49.086365+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.717700+0000","last_change":"2023-05-16T07:19:48.711741+0000","last_active":"2023-05-16T07:19:48.717700+0000","last_peered":"2023-05-16T07:19:48.717700+0000","last_clean":"2023-05-16T07:19:48.717700+0000","last_became_active":"2023-05-16T07:19:48.711454+0000","last_became_peered":"2023-05-16T07:19:48.711454+0000","last_unstale":"2023-05-16T07:19:48.717700+0000","last_undegraded":"2023-05-16T07:19:48.717700+0000","last_fullsized":"2023-05-16T07:19:48.717700+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083717+0000","last_change":"2023-05-16T07:19:49.083717+0000","last_active":"2023-05-16T07:19:49.083717+0000","last_peered":"2023-05-16T07:19:49.083717+0000","last_clean":"2023-05-16T07:19:49.083717+0000","last_became_active":"2023-05-16T07:19:49.083449+0000","last_became_peered":"2023-05-16T07:19:49.083449+0000","last_unstale":"2023-05-16T07:19:49.083717+0000","last_undegraded":"2023-05-16T07:19:49.083717+0000","last_fullsized":"2023-05-16T07:19:49.083717+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721677+0000","last_change":"2023-05-16T07:19:48.716559+0000","last_active":"2023-05-16T07:19:48.721677+0000","last_peered":"2023-05-16T07:19:48.721677+0000","last_clean":"2023-05-16T07:19:48.721677+0000","last_became_active":"2023-05-16T07:19:48.716264+0000","last_became_peered":"2023-05-16T07:19:48.716264+0000","last_unstale":"2023-05-16T07:19:48.721677+0000","last_undegraded":"2023-05-16T07:19:48.721677+0000","last_fullsized":"2023-05-16T07:19:48.721677+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083881+0000","last_change":"2023-05-16T07:19:49.083881+0000","last_active":"2023-05-16T07:19:49.083881+0000","last_peered":"2023-05-16T07:19:49.083881+0000","last_clean":"2023-05-16T07:19:49.083881+0000","last_became_active":"2023-05-16T07:19:49.083551+0000","last_became_peered":"2023-05-16T07:19:49.083551+0000","last_unstale":"2023-05-16T07:19:49.083881+0000","last_undegraded":"2023-05-16T07:19:49.083881+0000","last_fullsized":"2023-05-16T07:19:49.083881+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719626+0000","last_change":"2023-05-16T07:19:48.713299+0000","last_active":"2023-05-16T07:19:48.719626+0000","last_peered":"2023-05-16T07:19:48.719626+0000","last_clean":"2023-05-16T07:19:48.719626+0000","last_became_active":"2023-05-16T07:19:48.713036+0000","last_became_peered":"2023-05-16T07:19:48.713036+0000","last_unstale":"2023-05-16T07:19:48.719626+0000","last_undegraded":"2023-05-16T07:19:48.719626+0000","last_fullsized":"2023-05-16T07:19:48.719626+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086498+0000","last_change":"2023-05-16T07:19:49.086498+0000","last_active":"2023-05-16T07:19:49.086498+0000","last_peered":"2023-05-16T07:19:49.086498+0000","last_clean":"2023-05-16T07:19:49.086498+0000","last_became_active":"2023-05-16T07:19:49.085990+0000","last_became_peered":"2023-05-16T07:19:49.085990+0000","last_unstale":"2023-05-16T07:19:49.086498+0000","last_undegraded":"2023-05-16T07:19:49.086498+0000","last_fullsized":"2023-05-16T07:19:49.086498+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.722723+0000","last_change":"2023-05-16T07:19:48.722141+0000","last_active":"2023-05-16T07:19:48.722723+0000","last_peered":"2023-05-16T07:19:48.722723+0000","last_clean":"2023-05-16T07:19:48.722723+0000","last_became_active":"2023-05-16T07:19:48.721868+0000","last_became_peered":"2023-05-16T07:19:48.721868+0000","last_unstale":"2023-05-16T07:19:48.722723+0000","last_undegraded":"2023-05-16T07:19:48.722723+0000","last_fullsized":"2023-05-16T07:19:48.722723+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.084605+0000","last_change":"2023-05-16T07:19:49.084605+0000","last_active":"2023-05-16T07:19:49.084605+0000","last_peered":"2023-05-16T07:19:49.084605+0000","last_clean":"2023-05-16T07:19:49.084605+0000","last_became_active":"2023-05-16T07:19:49.084088+0000","last_became_peered":"2023-05-16T07:19:49.084088+0000","last_unstale":"2023-05-16T07:19:49.084605+0000","last_undegraded":"2023-05-16T07:19:49.084605+0000","last_fullsized":"2023-05-16T07:19:49.084605+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721153+0000","last_change":"2023-05-16T07:19:48.711305+0000","last_active":"2023-05-16T07:19:48.721153+0000","last_peered":"2023-05-16T07:19:48.721153+0000","last_clean":"2023-05-16T07:19:48.721153+0000","last_became_active":"2023-05-16T07:19:48.710749+0000","last_became_peered":"2023-05-16T07:19:48.710749+0000","last_unstale":"2023-05-16T07:19:48.721153+0000","last_undegraded":"2023-05-16T07:19:48.721153+0000","last_fullsized":"2023-05-16T07:19:48.721153+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086417+0000","last_change":"2023-05-16T07:19:49.086417+0000","last_active":"2023-05-16T07:19:49.086417+0000","last_peered":"2023-05-16T07:19:49.086417+0000","last_clean":"2023-05-16T07:19:49.086417+0000","last_became_active":"2023-05-16T07:19:49.086138+0000","last_became_peered":"2023-05-16T07:19:49.086138+0000","last_unstale":"2023-05-16T07:19:49.086417+0000","last_undegraded":"2023-05-16T07:19:49.086417+0000","last_fullsized":"2023-05-16T07:19:49.086417+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714338+0000","last_change":"2023-05-16T07:19:48.708850+0000","last_active":"2023-05-16T07:19:48.714338+0000","last_peered":"2023-05-16T07:19:48.714338+0000","last_clean":"2023-05-16T07:19:48.714338+0000","last_became_active":"2023-05-16T07:19:48.708211+0000","last_became_peered":"2023-05-16T07:19:48.708211+0000","last_unstale":"2023-05-16T07:19:48.714338+0000","last_undegraded":"2023-05-16T07:19:48.714338+0000","last_fullsized":"2023-05-16T07:19:48.714338+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099670+0000","last_change":"2023-05-16T07:19:49.099670+0000","last_active":"2023-05-16T07:19:49.099670+0000","last_peered":"2023-05-16T07:19:49.099670+0000","last_clean":"2023-05-16T07:19:49.099670+0000","last_became_active":"2023-05-16T07:19:49.099413+0000","last_became_peered":"2023-05-16T07:19:49.099413+0000","last_unstale":"2023-05-16T07:19:49.099670+0000","last_undegraded":"2023-05-16T07:19:49.099670+0000","last_fullsized":"2023-05-16T07:19:49.099670+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.074935+0000","last_change":"2023-05-16T07:19:48.710175+0000","last_active":"2023-05-16T07:19:49.074935+0000","last_peered":"2023-05-16T07:19:49.074935+0000","last_clean":"2023-05-16T07:19:49.074935+0000","last_became_active":"2023-05-16T07:19:48.709900+0000","last_became_peered":"2023-05-16T07:19:48.709900+0000","last_unstale":"2023-05-16T07:19:49.074935+0000","last_undegraded":"2023-05-16T07:19:49.074935+0000","last_fullsized":"2023-05-16T07:19:49.074935+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089056+0000","last_change":"2023-05-16T07:19:49.089056+0000","last_active":"2023-05-16T07:19:49.089056+0000","last_peered":"2023-05-16T07:19:49.089056+0000","last_clean":"2023-05-16T07:19:49.089056+0000","last_became_active":"2023-05-16T07:19:49.088721+0000","last_became_peered":"2023-05-16T07:19:49.088721+0000","last_unstale":"2023-05-16T07:19:49.089056+0000","last_undegraded":"2023-05-16T07:19:49.089056+0000","last_fullsized":"2023-05-16T07:19:49.089056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.082209+0000","last_change":"2023-05-16T07:19:48.712805+0000","last_active":"2023-05-16T07:19:49.082209+0000","last_peered":"2023-05-16T07:19:49.082209+0000","last_clean":"2023-05-16T07:19:49.082209+0000","last_became_active":"2023-05-16T07:19:48.712530+0000","last_became_peered":"2023-05-16T07:19:48.712530+0000","last_unstale":"2023-05-16T07:19:49.082209+0000","last_undegraded":"2023-05-16T07:19:49.082209+0000","last_fullsized":"2023-05-16T07:19:49.082209+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083826+0000","last_change":"2023-05-16T07:19:49.083826+0000","last_active":"2023-05-16T07:19:49.083826+0000","last_peered":"2023-05-16T07:19:49.083826+0000","last_clean":"2023-05-16T07:19:49.083826+0000","last_became_active":"2023-05-16T07:19:49.083469+0000","last_became_peered":"2023-05-16T07:19:49.083469+0000","last_unstale":"2023-05-16T07:19:49.083826+0000","last_undegraded":"2023-05-16T07:19:49.083826+0000","last_fullsized":"2023-05-16T07:19:49.083826+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090850+0000","last_change":"2023-05-16T07:19:48.716287+0000","last_active":"2023-05-16T07:19:49.090850+0000","last_peered":"2023-05-16T07:19:49.090850+0000","last_clean":"2023-05-16T07:19:49.090850+0000","last_became_active":"2023-05-16T07:19:48.715656+0000","last_became_peered":"2023-05-16T07:19:48.715656+0000","last_unstale":"2023-05-16T07:19:49.090850+0000","last_undegraded":"2023-05-16T07:19:49.090850+0000","last_fullsized":"2023-05-16T07:19:49.090850+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086397+0000","last_change":"2023-05-16T07:19:49.086397+0000","last_active":"2023-05-16T07:19:49.086397+0000","last_peered":"2023-05-16T07:19:49.086397+0000","last_clean":"2023-05-16T07:19:49.086397+0000","last_became_active":"2023-05-16T07:19:49.086143+0000","last_became_peered":"2023-05-16T07:19:49.086143+0000","last_unstale":"2023-05-16T07:19:49.086397+0000","last_undegraded":"2023-05-16T07:19:49.086397+0000","last_fullsized":"2023-05-16T07:19:49.086397+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.713088+0000","last_change":"2023-05-16T07:19:48.707314+0000","last_active":"2023-05-16T07:19:48.713088+0000","last_peered":"2023-05-16T07:19:48.713088+0000","last_clean":"2023-05-16T07:19:48.713088+0000","last_became_active":"2023-05-16T07:19:48.707009+0000","last_became_peered":"2023-05-16T07:19:48.707009+0000","last_unstale":"2023-05-16T07:19:48.713088+0000","last_undegraded":"2023-05-16T07:19:48.713088+0000","last_fullsized":"2023-05-16T07:19:48.713088+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090568+0000","last_change":"2023-05-16T07:19:49.090568+0000","last_active":"2023-05-16T07:19:49.090568+0000","last_peered":"2023-05-16T07:19:49.090568+0000","last_clean":"2023-05-16T07:19:49.090568+0000","last_became_active":"2023-05-16T07:19:49.090351+0000","last_became_peered":"2023-05-16T07:19:49.090351+0000","last_unstale":"2023-05-16T07:19:49.090568+0000","last_undegraded":"2023-05-16T07:19:49.090568+0000","last_fullsized":"2023-05-16T07:19:49.090568+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719643+0000","last_change":"2023-05-16T07:19:48.714373+0000","last_active":"2023-05-16T07:19:48.719643+0000","last_peered":"2023-05-16T07:19:48.719643+0000","last_clean":"2023-05-16T07:19:48.719643+0000","last_became_active":"2023-05-16T07:19:48.713710+0000","last_became_peered":"2023-05-16T07:19:48.713710+0000","last_unstale":"2023-05-16T07:19:48.719643+0000","last_undegraded":"2023-05-16T07:19:48.719643+0000","last_fullsized":"2023-05-16T07:19:48.719643+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.094157+0000","last_change":"2023-05-16T07:19:49.094157+0000","last_active":"2023-05-16T07:19:49.094157+0000","last_peered":"2023-05-16T07:19:49.094157+0000","last_clean":"2023-05-16T07:19:49.094157+0000","last_became_active":"2023-05-16T07:19:49.093843+0000","last_became_peered":"2023-05-16T07:19:49.093843+0000","last_unstale":"2023-05-16T07:19:49.094157+0000","last_undegraded":"2023-05-16T07:19:49.094157+0000","last_fullsized":"2023-05-16T07:19:49.094157+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723228+0000","last_change":"2023-05-16T07:19:48.718660+0000","last_active":"2023-05-16T07:19:48.723228+0000","last_peered":"2023-05-16T07:19:48.723228+0000","last_clean":"2023-05-16T07:19:48.723228+0000","last_became_active":"2023-05-16T07:19:48.718386+0000","last_became_peered":"2023-05-16T07:19:48.718386+0000","last_unstale":"2023-05-16T07:19:48.723228+0000","last_undegraded":"2023-05-16T07:19:48.723228+0000","last_fullsized":"2023-05-16T07:19:48.723228+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.080166+0000","last_change":"2023-05-16T07:19:49.080166+0000","last_active":"2023-05-16T07:19:49.080166+0000","last_peered":"2023-05-16T07:19:49.080166+0000","last_clean":"2023-05-16T07:19:49.080166+0000","last_became_active":"2023-05-16T07:19:49.079941+0000","last_became_peered":"2023-05-16T07:19:49.079941+0000","last_unstale":"2023-05-16T07:19:49.080166+0000","last_undegraded":"2023-05-16T07:19:49.080166+0000","last_fullsized":"2023-05-16T07:19:49.080166+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714335+0000","last_change":"2023-05-16T07:19:48.710777+0000","last_active":"2023-05-16T07:19:48.714335+0000","last_peered":"2023-05-16T07:19:48.714335+0000","last_clean":"2023-05-16T07:19:48.714335+0000","last_became_active":"2023-05-16T07:19:48.710476+0000","last_became_peered":"2023-05-16T07:19:48.710476+0000","last_unstale":"2023-05-16T07:19:48.714335+0000","last_undegraded":"2023-05-16T07:19:48.714335+0000","last_fullsized":"2023-05-16T07:19:48.714335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093403+0000","last_change":"2023-05-16T07:19:49.093403+0000","last_active":"2023-05-16T07:19:49.093403+0000","last_peered":"2023-05-16T07:19:49.093403+0000","last_clean":"2023-05-16T07:19:49.093403+0000","last_became_active":"2023-05-16T07:19:49.093143+0000","last_became_peered":"2023-05-16T07:19:49.093143+0000","last_unstale":"2023-05-16T07:19:49.093403+0000","last_undegraded":"2023-05-16T07:19:49.093403+0000","last_fullsized":"2023-05-16T07:19:49.093403+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723272+0000","last_change":"2023-05-16T07:19:48.718851+0000","last_active":"2023-05-16T07:19:48.723272+0000","last_peered":"2023-05-16T07:19:48.723272+0000","last_clean":"2023-05-16T07:19:48.723272+0000","last_became_active":"2023-05-16T07:19:48.718573+0000","last_became_peered":"2023-05-16T07:19:48.718573+0000","last_unstale":"2023-05-16T07:19:48.723272+0000","last_undegraded":"2023-05-16T07:19:48.723272+0000","last_fullsized":"2023-05-16T07:19:48.723272+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099763+0000","last_change":"2023-05-16T07:19:49.099763+0000","last_active":"2023-05-16T07:19:49.099763+0000","last_peered":"2023-05-16T07:19:49.099763+0000","last_clean":"2023-05-16T07:19:49.099763+0000","last_became_active":"2023-05-16T07:19:49.099478+0000","last_became_peered":"2023-05-16T07:19:49.099478+0000","last_unstale":"2023-05-16T07:19:49.099763+0000","last_undegraded":"2023-05-16T07:19:49.099763+0000","last_fullsized":"2023-05-16T07:19:49.099763+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083502+0000","last_change":"2023-05-16T07:19:48.710485+0000","last_active":"2023-05-16T07:19:49.083502+0000","last_peered":"2023-05-16T07:19:49.083502+0000","last_clean":"2023-05-16T07:19:49.083502+0000","last_became_active":"2023-05-16T07:19:48.708387+0000","last_became_peered":"2023-05-16T07:19:48.708387+0000","last_unstale":"2023-05-16T07:19:49.083502+0000","last_undegraded":"2023-05-16T07:19:49.083502+0000","last_fullsized":"2023-05-16T07:19:49.083502+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085869+0000","last_change":"2023-05-16T07:19:49.085869+0000","last_active":"2023-05-16T07:19:49.085869+0000","last_peered":"2023-05-16T07:19:49.085869+0000","last_clean":"2023-05-16T07:19:49.085869+0000","last_became_active":"2023-05-16T07:19:49.085611+0000","last_became_peered":"2023-05-16T07:19:49.085611+0000","last_unstale":"2023-05-16T07:19:49.085869+0000","last_undegraded":"2023-05-16T07:19:49.085869+0000","last_fullsized":"2023-05-16T07:19:49.085869+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.098907+0000","last_change":"2023-05-16T07:19:48.715081+0000","last_active":"2023-05-16T07:19:49.098907+0000","last_peered":"2023-05-16T07:19:49.098907+0000","last_clean":"2023-05-16T07:19:49.098907+0000","last_became_active":"2023-05-16T07:19:48.714412+0000","last_became_peered":"2023-05-16T07:19:48.714412+0000","last_unstale":"2023-05-16T07:19:49.098907+0000","last_undegraded":"2023-05-16T07:19:49.098907+0000","last_fullsized":"2023-05-16T07:19:49.098907+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.091808+0000","last_change":"2023-05-16T07:19:49.091808+0000","last_active":"2023-05-16T07:19:49.091808+0000","last_peered":"2023-05-16T07:19:49.091808+0000","last_clean":"2023-05-16T07:19:49.091808+0000","last_became_active":"2023-05-16T07:19:49.089497+0000","last_became_peered":"2023-05-16T07:19:49.089497+0000","last_unstale":"2023-05-16T07:19:49.091808+0000","last_undegraded":"2023-05-16T07:19:49.091808+0000","last_fullsized":"2023-05-16T07:19:49.091808+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720555+0000","last_change":"2023-05-16T07:19:48.718161+0000","last_active":"2023-05-16T07:19:48.720555+0000","last_peered":"2023-05-16T07:19:48.720555+0000","last_clean":"2023-05-16T07:19:48.720555+0000","last_became_active":"2023-05-16T07:19:48.716022+0000","last_became_peered":"2023-05-16T07:19:48.716022+0000","last_unstale":"2023-05-16T07:19:48.720555+0000","last_undegraded":"2023-05-16T07:19:48.720555+0000","last_fullsized":"2023-05-16T07:19:48.720555+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093169+0000","last_change":"2023-05-16T07:19:49.093169+0000","last_active":"2023-05-16T07:19:49.093169+0000","last_peered":"2023-05-16T07:19:49.093169+0000","last_clean":"2023-05-16T07:19:49.093169+0000","last_became_active":"2023-05-16T07:19:49.092907+0000","last_became_peered":"2023-05-16T07:19:49.092907+0000","last_unstale":"2023-05-16T07:19:49.093169+0000","last_undegraded":"2023-05-16T07:19:49.093169+0000","last_fullsized":"2023-05-16T07:19:49.093169+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.722504+0000","last_change":"2023-05-16T07:19:48.721859+0000","last_active":"2023-05-16T07:19:48.722504+0000","last_peered":"2023-05-16T07:19:48.722504+0000","last_clean":"2023-05-16T07:19:48.722504+0000","last_became_active":"2023-05-16T07:19:48.721641+0000","last_became_peered":"2023-05-16T07:19:48.721641+0000","last_unstale":"2023-05-16T07:19:48.722504+0000","last_undegraded":"2023-05-16T07:19:48.722504+0000","last_fullsized":"2023-05-16T07:19:48.722504+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099114+0000","last_change":"2023-05-16T07:19:49.099114+0000","last_active":"2023-05-16T07:19:49.099114+0000","last_peered":"2023-05-16T07:19:49.099114+0000","last_clean":"2023-05-16T07:19:49.099114+0000","last_became_active":"2023-05-16T07:19:49.098842+0000","last_became_peered":"2023-05-16T07:19:49.098842+0000","last_unstale":"2023-05-16T07:19:49.099114+0000","last_undegraded":"2023-05-16T07:19:49.099114+0000","last_fullsized":"2023-05-16T07:19:49.099114+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.072739+0000","last_change":"2023-05-16T07:19:48.719930+0000","last_active":"2023-05-16T07:19:49.072739+0000","last_peered":"2023-05-16T07:19:49.072739+0000","last_clean":"2023-05-16T07:19:49.072739+0000","last_became_active":"2023-05-16T07:19:48.719621+0000","last_became_peered":"2023-05-16T07:19:48.719621+0000","last_unstale":"2023-05-16T07:19:49.072739+0000","last_undegraded":"2023-05-16T07:19:49.072739+0000","last_fullsized":"2023-05-16T07:19:49.072739+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085507+0000","last_change":"2023-05-16T07:19:49.085507+0000","last_active":"2023-05-16T07:19:49.085507+0000","last_peered":"2023-05-16T07:19:49.085507+0000","last_clean":"2023-05-16T07:19:49.085507+0000","last_became_active":"2023-05-16T07:19:49.085049+0000","last_became_peered":"2023-05-16T07:19:49.085049+0000","last_unstale":"2023-05-16T07:19:49.085507+0000","last_undegraded":"2023-05-16T07:19:49.085507+0000","last_fullsized":"2023-05-16T07:19:49.085507+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720970+0000","last_change":"2023-05-16T07:19:48.720029+0000","last_active":"2023-05-16T07:19:48.720970+0000","last_peered":"2023-05-16T07:19:48.720970+0000","last_clean":"2023-05-16T07:19:48.720970+0000","last_became_active":"2023-05-16T07:19:48.719761+0000","last_became_peered":"2023-05-16T07:19:48.719761+0000","last_unstale":"2023-05-16T07:19:48.720970+0000","last_undegraded":"2023-05-16T07:19:48.720970+0000","last_fullsized":"2023-05-16T07:19:48.720970+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.084371+0000","last_change":"2023-05-16T07:19:49.084371+0000","last_active":"2023-05-16T07:19:49.084371+0000","last_peered":"2023-05-16T07:19:49.084371+0000","last_clean":"2023-05-16T07:19:49.084371+0000","last_became_active":"2023-05-16T07:19:49.083649+0000","last_became_peered":"2023-05-16T07:19:49.083649+0000","last_unstale":"2023-05-16T07:19:49.084371+0000","last_undegraded":"2023-05-16T07:19:49.084371+0000","last_fullsized":"2023-05-16T07:19:49.084371+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721734+0000","last_change":"2023-05-16T07:19:48.716582+0000","last_active":"2023-05-16T07:19:48.721734+0000","last_peered":"2023-05-16T07:19:48.721734+0000","last_clean":"2023-05-16T07:19:48.721734+0000","last_became_active":"2023-05-16T07:19:48.716297+0000","last_became_peered":"2023-05-16T07:19:48.716297+0000","last_unstale":"2023-05-16T07:19:48.721734+0000","last_undegraded":"2023-05-16T07:19:48.721734+0000","last_fullsized":"2023-05-16T07:19:48.721734+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.091787+0000","last_change":"2023-05-16T07:19:49.091787+0000","last_active":"2023-05-16T07:19:49.091787+0000","last_peered":"2023-05-16T07:19:49.091787+0000","last_clean":"2023-05-16T07:19:49.091787+0000","last_became_active":"2023-05-16T07:19:49.091491+0000","last_became_peered":"2023-05-16T07:19:49.091491+0000","last_unstale":"2023-05-16T07:19:49.091787+0000","last_undegraded":"2023-05-16T07:19:49.091787+0000","last_fullsized":"2023-05-16T07:19:49.091787+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.720917+0000","last_change":"2023-05-16T07:19:48.719269+0000","last_active":"2023-05-16T07:19:48.720917+0000","last_peered":"2023-05-16T07:19:48.720917+0000","last_clean":"2023-05-16T07:19:48.720917+0000","last_became_active":"2023-05-16T07:19:48.718794+0000","last_became_peered":"2023-05-16T07:19:48.718794+0000","last_unstale":"2023-05-16T07:19:48.720917+0000","last_undegraded":"2023-05-16T07:19:48.720917+0000","last_fullsized":"2023-05-16T07:19:48.720917+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.096108+0000","last_change":"2023-05-16T07:19:49.096108+0000","last_active":"2023-05-16T07:19:49.096108+0000","last_peered":"2023-05-16T07:19:49.096108+0000","last_clean":"2023-05-16T07:19:49.096108+0000","last_became_active":"2023-05-16T07:19:49.095835+0000","last_became_peered":"2023-05-16T07:19:49.095835+0000","last_unstale":"2023-05-16T07:19:49.096108+0000","last_undegraded":"2023-05-16T07:19:49.096108+0000","last_fullsized":"2023-05-16T07:19:49.096108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088488+0000","last_change":"2023-05-16T07:19:48.718846+0000","last_active":"2023-05-16T07:19:49.088488+0000","last_peered":"2023-05-16T07:19:49.088488+0000","last_clean":"2023-05-16T07:19:49.088488+0000","last_became_active":"2023-05-16T07:19:48.718043+0000","last_became_peered":"2023-05-16T07:19:48.718043+0000","last_unstale":"2023-05-16T07:19:49.088488+0000","last_undegraded":"2023-05-16T07:19:49.088488+0000","last_fullsized":"2023-05-16T07:19:49.088488+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.084868+0000","last_change":"2023-05-16T07:19:49.084868+0000","last_active":"2023-05-16T07:19:49.084868+0000","last_peered":"2023-05-16T07:19:49.084868+0000","last_clean":"2023-05-16T07:19:49.084868+0000","last_became_active":"2023-05-16T07:19:49.084601+0000","last_became_peered":"2023-05-16T07:19:49.084601+0000","last_unstale":"2023-05-16T07:19:49.084868+0000","last_undegraded":"2023-05-16T07:19:49.084868+0000","last_fullsized":"2023-05-16T07:19:49.084868+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715406+0000","last_change":"2023-05-16T07:19:48.711247+0000","last_active":"2023-05-16T07:19:48.715406+0000","last_peered":"2023-05-16T07:19:48.715406+0000","last_clean":"2023-05-16T07:19:48.715406+0000","last_became_active":"2023-05-16T07:19:48.710944+0000","last_became_peered":"2023-05-16T07:19:48.710944+0000","last_unstale":"2023-05-16T07:19:48.715406+0000","last_undegraded":"2023-05-16T07:19:48.715406+0000","last_fullsized":"2023-05-16T07:19:48.715406+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.093421+0000","last_change":"2023-05-16T07:19:49.093421+0000","last_active":"2023-05-16T07:19:49.093421+0000","last_peered":"2023-05-16T07:19:49.093421+0000","last_clean":"2023-05-16T07:19:49.093421+0000","last_became_active":"2023-05-16T07:19:49.093130+0000","last_became_peered":"2023-05-16T07:19:49.093130+0000","last_unstale":"2023-05-16T07:19:49.093421+0000","last_undegraded":"2023-05-16T07:19:49.093421+0000","last_fullsized":"2023-05-16T07:19:49.093421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715402+0000","last_change":"2023-05-16T07:19:48.711117+0000","last_active":"2023-05-16T07:19:48.715402+0000","last_peered":"2023-05-16T07:19:48.715402+0000","last_clean":"2023-05-16T07:19:48.715402+0000","last_became_active":"2023-05-16T07:19:48.710839+0000","last_became_peered":"2023-05-16T07:19:48.710839+0000","last_unstale":"2023-05-16T07:19:48.715402+0000","last_undegraded":"2023-05-16T07:19:48.715402+0000","last_fullsized":"2023-05-16T07:19:48.715402+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.085287+0000","last_change":"2023-05-16T07:19:49.085287+0000","last_active":"2023-05-16T07:19:49.085287+0000","last_peered":"2023-05-16T07:19:49.085287+0000","last_clean":"2023-05-16T07:19:49.085287+0000","last_became_active":"2023-05-16T07:19:49.085011+0000","last_became_peered":"2023-05-16T07:19:49.085011+0000","last_unstale":"2023-05-16T07:19:49.085287+0000","last_undegraded":"2023-05-16T07:19:49.085287+0000","last_fullsized":"2023-05-16T07:19:49.085287+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723328+0000","last_change":"2023-05-16T07:19:48.719669+0000","last_active":"2023-05-16T07:19:48.723328+0000","last_peered":"2023-05-16T07:19:48.723328+0000","last_clean":"2023-05-16T07:19:48.723328+0000","last_became_active":"2023-05-16T07:19:48.719390+0000","last_became_peered":"2023-05-16T07:19:48.719390+0000","last_unstale":"2023-05-16T07:19:48.723328+0000","last_undegraded":"2023-05-16T07:19:48.723328+0000","last_fullsized":"2023-05-16T07:19:48.723328+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083465+0000","last_change":"2023-05-16T07:19:49.083465+0000","last_active":"2023-05-16T07:19:49.083465+0000","last_peered":"2023-05-16T07:19:49.083465+0000","last_clean":"2023-05-16T07:19:49.083465+0000","last_became_active":"2023-05-16T07:19:49.083224+0000","last_became_peered":"2023-05-16T07:19:49.083224+0000","last_unstale":"2023-05-16T07:19:49.083465+0000","last_undegraded":"2023-05-16T07:19:49.083465+0000","last_fullsized":"2023-05-16T07:19:49.083465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.713001+0000","last_change":"2023-05-16T07:19:48.706422+0000","last_active":"2023-05-16T07:19:48.713001+0000","last_peered":"2023-05-16T07:19:48.713001+0000","last_clean":"2023-05-16T07:19:48.713001+0000","last_became_active":"2023-05-16T07:19:48.706135+0000","last_became_peered":"2023-05-16T07:19:48.706135+0000","last_unstale":"2023-05-16T07:19:48.713001+0000","last_undegraded":"2023-05-16T07:19:48.713001+0000","last_fullsized":"2023-05-16T07:19:48.713001+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.074133+0000","last_change":"2023-05-16T07:19:49.074133+0000","last_active":"2023-05-16T07:19:49.074133+0000","last_peered":"2023-05-16T07:19:49.074133+0000","last_clean":"2023-05-16T07:19:49.074133+0000","last_became_active":"2023-05-16T07:19:49.073881+0000","last_became_peered":"2023-05-16T07:19:49.073881+0000","last_unstale":"2023-05-16T07:19:49.074133+0000","last_undegraded":"2023-05-16T07:19:49.074133+0000","last_fullsized":"2023-05-16T07:19:49.074133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.719041+0000","last_change":"2023-05-16T07:19:48.712776+0000","last_active":"2023-05-16T07:19:48.719041+0000","last_peered":"2023-05-16T07:19:48.719041+0000","last_clean":"2023-05-16T07:19:48.719041+0000","last_became_active":"2023-05-16T07:19:48.709470+0000","last_became_peered":"2023-05-16T07:19:48.709470+0000","last_unstale":"2023-05-16T07:19:48.719041+0000","last_undegraded":"2023-05-16T07:19:48.719041+0000","last_fullsized":"2023-05-16T07:19:48.719041+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.099077+0000","last_change":"2023-05-16T07:19:49.099077+0000","last_active":"2023-05-16T07:19:49.099077+0000","last_peered":"2023-05-16T07:19:49.099077+0000","last_clean":"2023-05-16T07:19:49.099077+0000","last_became_active":"2023-05-16T07:19:49.098877+0000","last_became_peered":"2023-05-16T07:19:49.098877+0000","last_unstale":"2023-05-16T07:19:49.099077+0000","last_undegraded":"2023-05-16T07:19:49.099077+0000","last_fullsized":"2023-05-16T07:19:49.099077+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.723312+0000","last_change":"2023-05-16T07:19:48.719129+0000","last_active":"2023-05-16T07:19:48.723312+0000","last_peered":"2023-05-16T07:19:48.723312+0000","last_clean":"2023-05-16T07:19:48.723312+0000","last_became_active":"2023-05-16T07:19:48.718857+0000","last_became_peered":"2023-05-16T07:19:48.718857+0000","last_unstale":"2023-05-16T07:19:48.723312+0000","last_undegraded":"2023-05-16T07:19:48.723312+0000","last_fullsized":"2023-05-16T07:19:48.723312+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.090881+0000","last_change":"2023-05-16T07:19:49.090881+0000","last_active":"2023-05-16T07:19:49.090881+0000","last_peered":"2023-05-16T07:19:49.090881+0000","last_clean":"2023-05-16T07:19:49.090881+0000","last_became_active":"2023-05-16T07:19:49.090622+0000","last_became_peered":"2023-05-16T07:19:49.090622+0000","last_unstale":"2023-05-16T07:19:49.090881+0000","last_undegraded":"2023-05-16T07:19:49.090881+0000","last_fullsized":"2023-05-16T07:19:49.090881+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.715425+0000","last_change":"2023-05-16T07:19:48.711822+0000","last_active":"2023-05-16T07:19:48.715425+0000","last_peered":"2023-05-16T07:19:48.715425+0000","last_clean":"2023-05-16T07:19:48.715425+0000","last_became_active":"2023-05-16T07:19:48.711537+0000","last_became_peered":"2023-05-16T07:19:48.711537+0000","last_unstale":"2023-05-16T07:19:48.715425+0000","last_undegraded":"2023-05-16T07:19:48.715425+0000","last_fullsized":"2023-05-16T07:19:48.715425+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.088371+0000","last_change":"2023-05-16T07:19:49.088371+0000","last_active":"2023-05-16T07:19:49.088371+0000","last_peered":"2023-05-16T07:19:49.088371+0000","last_clean":"2023-05-16T07:19:49.088371+0000","last_became_active":"2023-05-16T07:19:49.088075+0000","last_became_peered":"2023-05-16T07:19:49.088075+0000","last_unstale":"2023-05-16T07:19:49.088371+0000","last_undegraded":"2023-05-16T07:19:49.088371+0000","last_fullsized":"2023-05-16T07:19:49.088371+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.716004+0000","last_change":"2023-05-16T07:19:48.710746+0000","last_active":"2023-05-16T07:19:48.716004+0000","last_peered":"2023-05-16T07:19:48.716004+0000","last_clean":"2023-05-16T07:19:48.716004+0000","last_became_active":"2023-05-16T07:19:48.710437+0000","last_became_peered":"2023-05-16T07:19:48.710437+0000","last_unstale":"2023-05-16T07:19:48.716004+0000","last_undegraded":"2023-05-16T07:19:48.716004+0000","last_fullsized":"2023-05-16T07:19:48.716004+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.696071+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:19:48.696071+0000","last_peered":"2023-05-16T07:19:48.696071+0000","last_clean":"2023-05-16T07:19:48.696071+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:19:48.696071+0000","last_undegraded":"2023-05-16T07:19:48.696071+0000","last_fullsized":"2023-05-16T07:19:48.696071+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083154+0000","last_change":"2023-05-16T07:19:49.083154+0000","last_active":"2023-05-16T07:19:49.083154+0000","last_peered":"2023-05-16T07:19:49.083154+0000","last_clean":"2023-05-16T07:19:49.083154+0000","last_became_active":"2023-05-16T07:19:49.082866+0000","last_became_peered":"2023-05-16T07:19:49.082866+0000","last_unstale":"2023-05-16T07:19:49.083154+0000","last_undegraded":"2023-05-16T07:19:49.083154+0000","last_fullsized":"2023-05-16T07:19:49.083154+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721778+0000","last_change":"2023-05-16T07:19:48.716933+0000","last_active":"2023-05-16T07:19:48.721778+0000","last_peered":"2023-05-16T07:19:48.721778+0000","last_clean":"2023-05-16T07:19:48.721778+0000","last_became_active":"2023-05-16T07:19:48.716393+0000","last_became_peered":"2023-05-16T07:19:48.716393+0000","last_unstale":"2023-05-16T07:19:48.721778+0000","last_undegraded":"2023-05-16T07:19:48.721778+0000","last_fullsized":"2023-05-16T07:19:48.721778+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":101,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.705673+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:19:48.705673+0000","last_peered":"2023-05-16T07:19:48.705673+0000","last_clean":"2023-05-16T07:19:48.705673+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:19:48.705673+0000","last_undegraded":"2023-05-16T07:19:48.705673+0000","last_fullsized":"2023-05-16T07:19:48.705673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.089115+0000","last_change":"2023-05-16T07:19:49.089115+0000","last_active":"2023-05-16T07:19:49.089115+0000","last_peered":"2023-05-16T07:19:49.089115+0000","last_clean":"2023-05-16T07:19:49.089115+0000","last_became_active":"2023-05-16T07:19:49.088833+0000","last_became_peered":"2023-05-16T07:19:49.088833+0000","last_unstale":"2023-05-16T07:19:49.089115+0000","last_undegraded":"2023-05-16T07:19:49.089115+0000","last_fullsized":"2023-05-16T07:19:49.089115+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.721798+0000","last_change":"2023-05-16T07:19:48.716929+0000","last_active":"2023-05-16T07:19:48.721798+0000","last_peered":"2023-05-16T07:19:48.721798+0000","last_clean":"2023-05-16T07:19:48.721798+0000","last_became_active":"2023-05-16T07:19:48.716406+0000","last_became_peered":"2023-05-16T07:19:48.716406+0000","last_unstale":"2023-05-16T07:19:48.721798+0000","last_undegraded":"2023-05-16T07:19:48.721798+0000","last_fullsized":"2023-05-16T07:19:48.721798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.703576+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:19:48.703576+0000","last_peered":"2023-05-16T07:19:48.703576+0000","last_clean":"2023-05-16T07:19:48.703576+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:19:48.703576+0000","last_undegraded":"2023-05-16T07:19:48.703576+0000","last_fullsized":"2023-05-16T07:19:48.703576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087814+0000","last_change":"2023-05-16T07:19:49.087814+0000","last_active":"2023-05-16T07:19:49.087814+0000","last_peered":"2023-05-16T07:19:49.087814+0000","last_clean":"2023-05-16T07:19:49.087814+0000","last_became_active":"2023-05-16T07:19:49.087536+0000","last_became_peered":"2023-05-16T07:19:49.087536+0000","last_unstale":"2023-05-16T07:19:49.087814+0000","last_undegraded":"2023-05-16T07:19:49.087814+0000","last_fullsized":"2023-05-16T07:19:49.087814+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.094125+0000","last_change":"2023-05-16T07:19:48.710011+0000","last_active":"2023-05-16T07:19:49.094125+0000","last_peered":"2023-05-16T07:19:49.094125+0000","last_clean":"2023-05-16T07:19:49.094125+0000","last_became_active":"2023-05-16T07:19:48.709388+0000","last_became_peered":"2023-05-16T07:19:48.709388+0000","last_unstale":"2023-05-16T07:19:49.094125+0000","last_undegraded":"2023-05-16T07:19:49.094125+0000","last_fullsized":"2023-05-16T07:19:49.094125+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":101,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.705613+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:19:48.705613+0000","last_peered":"2023-05-16T07:19:48.705613+0000","last_clean":"2023-05-16T07:19:48.705613+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:19:48.705613+0000","last_undegraded":"2023-05-16T07:19:48.705613+0000","last_fullsized":"2023-05-16T07:19:48.705613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.706285+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:19:48.706285+0000","last_peered":"2023-05-16T07:19:48.706285+0000","last_clean":"2023-05-16T07:19:48.706285+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:19:48.706285+0000","last_undegraded":"2023-05-16T07:19:48.706285+0000","last_fullsized":"2023-05-16T07:19:48.706285+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.086658+0000","last_change":"2023-05-16T07:19:49.086658+0000","last_active":"2023-05-16T07:19:49.086658+0000","last_peered":"2023-05-16T07:19:49.086658+0000","last_clean":"2023-05-16T07:19:49.086658+0000","last_became_active":"2023-05-16T07:19:49.086348+0000","last_became_peered":"2023-05-16T07:19:49.086348+0000","last_unstale":"2023-05-16T07:19:49.086658+0000","last_undegraded":"2023-05-16T07:19:49.086658+0000","last_fullsized":"2023-05-16T07:19:49.086658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.069473+0000","last_change":"2023-05-16T07:19:48.709921+0000","last_active":"2023-05-16T07:19:49.069473+0000","last_peered":"2023-05-16T07:19:49.069473+0000","last_clean":"2023-05-16T07:19:49.069473+0000","last_became_active":"2023-05-16T07:19:48.709639+0000","last_became_peered":"2023-05-16T07:19:48.709639+0000","last_unstale":"2023-05-16T07:19:49.069473+0000","last_undegraded":"2023-05-16T07:19:49.069473+0000","last_fullsized":"2023-05-16T07:19:49.069473+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.706367+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:19:48.706367+0000","last_peered":"2023-05-16T07:19:48.706367+0000","last_clean":"2023-05-16T07:19:48.706367+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:19:48.706367+0000","last_undegraded":"2023-05-16T07:19:48.706367+0000","last_fullsized":"2023-05-16T07:19:48.706367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087160+0000","last_change":"2023-05-16T07:19:49.087160+0000","last_active":"2023-05-16T07:19:49.087160+0000","last_peered":"2023-05-16T07:19:49.087160+0000","last_clean":"2023-05-16T07:19:49.087160+0000","last_became_active":"2023-05-16T07:19:49.086865+0000","last_became_peered":"2023-05-16T07:19:49.086865+0000","last_unstale":"2023-05-16T07:19:49.087160+0000","last_undegraded":"2023-05-16T07:19:49.087160+0000","last_fullsized":"2023-05-16T07:19:49.087160+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.714803+0000","last_change":"2023-05-16T07:19:48.712443+0000","last_active":"2023-05-16T07:19:48.714803+0000","last_peered":"2023-05-16T07:19:48.714803+0000","last_clean":"2023-05-16T07:19:48.714803+0000","last_became_active":"2023-05-16T07:19:48.712066+0000","last_became_peered":"2023-05-16T07:19:48.712066+0000","last_unstale":"2023-05-16T07:19:48.714803+0000","last_undegraded":"2023-05-16T07:19:48.714803+0000","last_fullsized":"2023-05-16T07:19:48.714803+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":102,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.709337+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:19:48.709337+0000","last_peered":"2023-05-16T07:19:48.709337+0000","last_clean":"2023-05-16T07:19:48.709337+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:19:48.709337+0000","last_undegraded":"2023-05-16T07:19:48.709337+0000","last_fullsized":"2023-05-16T07:19:48.709337+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083108+0000","last_change":"2023-05-16T07:19:49.083108+0000","last_active":"2023-05-16T07:19:49.083108+0000","last_peered":"2023-05-16T07:19:49.083108+0000","last_clean":"2023-05-16T07:19:49.083108+0000","last_became_active":"2023-05-16T07:19:49.082702+0000","last_became_peered":"2023-05-16T07:19:49.082702+0000","last_unstale":"2023-05-16T07:19:49.083108+0000","last_undegraded":"2023-05-16T07:19:49.083108+0000","last_fullsized":"2023-05-16T07:19:49.083108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"134'3","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.095505+0000","last_change":"2023-05-16T07:19:48.716009+0000","last_active":"2023-05-16T07:19:49.095505+0000","last_peered":"2023-05-16T07:19:49.095505+0000","last_clean":"2023-05-16T07:19:49.095505+0000","last_became_active":"2023-05-16T07:19:48.715769+0000","last_became_peered":"2023-05-16T07:19:48.715769+0000","last_unstale":"2023-05-16T07:19:49.095505+0000","last_undegraded":"2023-05-16T07:19:49.095505+0000","last_fullsized":"2023-05-16T07:19:49.095505+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":107,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.696318+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:19:48.696318+0000","last_peered":"2023-05-16T07:19:48.696318+0000","last_clean":"2023-05-16T07:19:48.696318+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:19:48.696318+0000","last_undegraded":"2023-05-16T07:19:48.696318+0000","last_fullsized":"2023-05-16T07:19:48.696318+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.087217+0000","last_change":"2023-05-16T07:19:49.087217+0000","last_active":"2023-05-16T07:19:49.087217+0000","last_peered":"2023-05-16T07:19:49.087217+0000","last_clean":"2023-05-16T07:19:49.087217+0000","last_became_active":"2023-05-16T07:19:49.086919+0000","last_became_peered":"2023-05-16T07:19:49.086919+0000","last_unstale":"2023-05-16T07:19:49.087217+0000","last_undegraded":"2023-05-16T07:19:49.087217+0000","last_fullsized":"2023-05-16T07:19:49.087217+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:48.687294+0000","last_clean_scrub_stamp":"2023-05-16T07:19:48.687294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-05-16T07:19:49.083485+0000","last_change":"2023-05-16T07:19:48.707143+0000","last_active":"2023-05-16T07:19:49.083485+0000","last_peered":"2023-05-16T07:19:49.083485+0000","last_clean":"2023-05-16T07:19:49.083485+0000","last_became_active":"2023-05-16T07:19:48.706879+0000","last_became_peered":"2023-05-16T07:19:48.706879+0000","last_unstale":"2023-05-16T07:19:49.083485+0000","last_undegraded":"2023-05-16T07:19:49.083485+0000","last_fullsized":"2023-05-16T07:19:49.083485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:19:47.682119+0000","last_clean_scrub_stamp":"2023-05-16T07:19:47.682119+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-05-16T07:19:48.700854+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:19:48.700854+0000","last_peered":"2023-05-16T07:19:48.700854+0000","last_clean":"2023-05-16T07:19:48.700854+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:19:48.700854+0000","last_undegraded":"2023-05-16T07:19:48.700854+0000","last_fullsized":"2023-05-16T07:19:48.700854+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575054,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23212,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":21568,"kb_avail":94348628,"statfs":{"total":96636764160,"available":96612995072,"internally_reserved":0,"allocated":1675264,"data_stored":874634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22085632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.56200000000000006},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.77700000000000002,"15min":0.84199999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.54200000000000004,"15min":0.56499999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73299999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.623}]},{"osd":1,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.55800000000000005},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.54400000000000004,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.749,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56499999999999995},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.79300000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54500000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65000000000000002,"5min":0.752,"15min":0.81999999999999995},"last":0.47799999999999998}]},{"osd":5,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.499},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68200000000000005,"5min":0.748,"15min":0.751},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue May 16 07:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65700000000000003,"5min":0.879,"15min":0.879},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.53100000000000003,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.84299999999999997},"last":0.54900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575054,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22884,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":94348956,"statfs":{"total":96636764160,"available":96613330944,"internally_reserved":0,"allocated":1667072,"data_stored":874994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53600000000000003,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.85099999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71499999999999997,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78400000000000003,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38},"max":{"1min":0.73999999999999999,"5min":0.79300000000000004,"15min":0.83699999999999997},"last":0.42999999999999999}]},{"osd":2,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70999999999999996,"5min":0.90400000000000003,"15min":0.96199999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.377,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.371},"max":{"1min":0.73899999999999999,"5min":0.84199999999999997,"15min":0.93300000000000005},"last":0.495}]},{"osd":4,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57699999999999996,"5min":0.749,"15min":0.89400000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":4.048,"15min":4.048},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.92300000000000004},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue May 16 07:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70699999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575055,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18772,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":17152,"kb_avail":94353068,"statfs":{"total":96636764160,"available":96617541632,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17563648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.69899999999999995},"last":0.316},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.39800000000000002,"15min":0.42099999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.5,"5min":0.56399999999999995,"15min":0.69199999999999995},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35699999999999998,"15min":0.379},"min":{"1min":0.27200000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41099999999999998,"5min":0.56999999999999995,"15min":0.65100000000000002},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38400000000000001,"15min":0.40500000000000003},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52100000000000002,"5min":0.64100000000000001,"15min":0.65500000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.68500000000000005},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.377,"15min":0.40000000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46300000000000002,"5min":0.622,"15min":0.64000000000000001},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.38500000000000001},"max":{"1min":0.80000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.623},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":0.92200000000000004,"15min":0.95499999999999996},"last":0.45700000000000002}]},{"osd":5,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61099999999999999,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77400000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.502},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.90700000000000003,"15min":0.91600000000000004},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.63500000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85599999999999998,"5min":0.94699999999999995,"15min":1.087},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.65500000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.93700000000000006,"15min":1.204},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue May 16 07:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.89200000000000002,"15min":0.96899999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.47399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575055,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17748,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":16128,"kb_avail":94354092,"statfs":{"total":96636764160,"available":96618590208,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16515072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.36299999999999999,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.67100000000000004},"last":0.34499999999999997}]},{"osd":2,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34599999999999997,"15min":0.372},"min":{"1min":0.23300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.437,"5min":0.51900000000000002,"15min":0.57799999999999996},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.40500000000000003},"min":{"1min":0.251,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.66900000000000004},"last":0.311}]},{"osd":3,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.40799999999999997},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.49399999999999999,"5min":0.60099999999999998,"15min":0.63600000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.64400000000000002},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65500000000000003,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0669999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue May 16 07:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.69799999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575055,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19476,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":17856,"kb_avail":94352364,"statfs":{"total":96636764160,"available":96616820736,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18284544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33700000000000002,"15min":0.36099999999999999},"min":{"1min":0.23499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.59699999999999998,"15min":0.64900000000000002},"last":0.251},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.373},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.58199999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38600000000000001,"15min":0.40699999999999997},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.71199999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40600000000000003,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.23200000000000001},"max":{"1min":0.621,"5min":0.626,"15min":0.72899999999999998},"last":0.379}]},{"osd":3,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36299999999999999,"15min":0.39100000000000001},"min":{"1min":0.245,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.64700000000000002},"last":0.316},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39700000000000002,"15min":0.434},"min":{"1min":0.29799999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56399999999999995,"5min":0.64900000000000002,"15min":0.72899999999999998},"last":0.28999999999999998}]},{"osd":4,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.79200000000000004,"5min":1.0580000000000001,"15min":1.5009999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61099999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.113},"last":0.45400000000000001}]},{"osd":5,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.48299999999999998,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":1.5469999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.63},"min":{"1min":0.432,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.5609999999999999},"last":0.435},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63200000000000001,"15min":0.66300000000000003},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.47},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue May 16 07:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.443,"5min":0.39200000000000002,"15min":0.36399999999999999},"max":{"1min":0.88400000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.41199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575054,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26516,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":24896,"kb_avail":94345324,"statfs":{"total":96636764160,"available":96609611776,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25493504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34999999999999998,"15min":0.36299999999999999},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.60199999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35799999999999998,"15min":0.379},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.5,"5min":0.54900000000000004,"15min":0.58799999999999997},"last":0.247}]},{"osd":1,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.48299999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.41599999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63600000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41399999999999998,"15min":0.433},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.53300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":0.877,"15min":0.91600000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.61399999999999999},"min":{"1min":0.40300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96399999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0489999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":1.0409999999999999},"last":0.622}]},{"osd":6,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0780000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.877,"15min":1.0049999999999999},"last":0.54000000000000004}]},{"osd":7,"last update":"Tue May 16 07:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61299999999999999,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.02},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18836,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":17216,"kb_avail":94353004,"statfs":{"total":96636764160,"available":96617476096,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17629184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.26200000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.36199999999999999,"15min":0.35999999999999999},"max":{"1min":2.5600000000000001,"5min":2.5600000000000001,"15min":2.5600000000000001},"last":0.502},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.33600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.36399999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.47399999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue May 16 07:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.75800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575054,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25940,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":24320,"kb_avail":94345900,"statfs":{"total":96636764160,"available":96610201600,"internally_reserved":0,"allocated":1650688,"data_stored":866649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24903680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.32500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.641,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":2.847,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.6059999999999999,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.432}]},{"osd":2,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73799999999999999,"5min":6.2800000000000002,"15min":6.2800000000000002},"last":0.503}]},{"osd":3,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.127,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.627}]},{"osd":4,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.503,"15min":0.47899999999999998},"min":{"1min":0.316,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1150000000000002,"5min":3.1150000000000002,"15min":3.1150000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":3.5659999999999998,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.52600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue May 16 07:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:19:52.403 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:19:52.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:19:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.548+0000 7ff638c50700 1 Processor -- start 2023-05-16T07:19:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.548+0000 7ff638c50700 1 -- start start 2023-05-16T07:19:52.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.549+0000 7ff638c50700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff6341542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.549+0000 7ff638c50700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff6340be280 0x7ff634066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.549+0000 7ff638c50700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff634155870 con 0x7ff634062cc0 2023-05-16T07:19:52.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.549+0000 7ff638c50700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff634067340 con 0x7ff6340be280 2023-05-16T07:19:52.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.549+0000 7ff638c50700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff634155670 con 0x7ff634153ee0 2023-05-16T07:19:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff63259c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff6341542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff6340be280 0x7ff634066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff6340be280 0x7ff634066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:53804/0 (socket says 172.21.15.92:53804) 2023-05-16T07:19:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff631d9b700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7ff634062cc0 0x7ff6340bdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:36282/0 (socket says 172.21.15.92:36282) 2023-05-16T07:19:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 -- 172.21.15.92:0/2985501225 learned_addr learned my addr 172.21.15.92:0/2985501225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:52.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 -- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 msgr2=0x7ff6341542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:52.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 --2- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff6341542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 -- 172.21.15.92:0/2985501225 >> v1:172.21.15.158:6789/0 conn(0x7ff634062cc0 legacy=0x7ff6340bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 -- 172.21.15.92:0/2985501225 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634063120 con 0x7ff6340be280 2023-05-16T07:19:52.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff63159a700 1 -- 172.21.15.92:0/2985501225 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452045029 0 0) 0x7ff634155870 con 0x7ff634062cc0 2023-05-16T07:19:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff63259c700 1 --2- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff6341542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:19:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff632d9d700 1 --2- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff634066e50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff62c009830 tx=0x7ff62c0090f0).ready entity=mon.1 client_cookie=d0b33f4f0d392066 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff63159a700 1 -- 172.21.15.92:0/2985501225 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00b580 con 0x7ff6340be280 2023-05-16T07:19:52.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff63159a700 1 -- 172.21.15.92:0/2985501225 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c00b6e0 con 0x7ff6340be280 2023-05-16T07:19:52.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.550+0000 7ff63159a700 1 -- 172.21.15.92:0/2985501225 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00ba60 con 0x7ff6340be280 2023-05-16T07:19:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 -- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 msgr2=0x7ff634066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 --2- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff634066e50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff62c009830 tx=0x7ff62c0090f0).stop 2023-05-16T07:19:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 -- 172.21.15.92:0/2985501225 shutdown_connections 2023-05-16T07:19:52.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 --2- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff634066e50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 --2- 172.21.15.92:0/2985501225 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff6341542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 -- 172.21.15.92:0/2985501225 >> 172.21.15.92:0/2985501225 conn(0x7ff6340c8420 msgr2=0x7ff6340c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 -- 172.21.15.92:0/2985501225 shutdown_connections 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.551+0000 7ff638c50700 1 -- 172.21.15.92:0/2985501225 wait complete. 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.552+0000 7ff638c50700 1 Processor -- start 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.552+0000 7ff638c50700 1 -- start start 2023-05-16T07:19:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 0x7ff634156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff6340ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff63414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff634150840 con 0x7ff634062cc0 2023-05-16T07:19:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff634156670 con 0x7ff6340be280 2023-05-16T07:19:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff634166b60 con 0x7ff634153ee0 2023-05-16T07:19:52.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 0x7ff634156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 0x7ff634156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50170/0 (socket says 172.21.15.92:50170) 2023-05-16T07:19:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 -- 172.21.15.92:0/3689662195 learned_addr learned my addr 172.21.15.92:0/3689662195 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff631d9b700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff6340ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 -- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 msgr2=0x7ff63414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff63414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 -- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 msgr2=0x7ff6340ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff6340ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 -- 172.21.15.92:0/3689662195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624009a20 con 0x7ff634062cc0 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff631d9b700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff6340ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff63259c700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 0x7ff634156180 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7ff62400c0c0 tx=0x7ff62400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62400cd60 con 0x7ff634062cc0 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62400f040 con 0x7ff634062cc0 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624013610 con 0x7ff634062cc0 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c009370 con 0x7ff634062cc0 2023-05-16T07:19:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.553+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff634166ef0 con 0x7ff634062cc0 2023-05-16T07:19:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.554+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff634150ae0 con 0x7ff634062cc0 2023-05-16T07:19:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.555+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff6240139c0 con 0x7ff634062cc0 2023-05-16T07:19:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.555+0000 7ff62a7fc700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff61405ed90 0x7ff614061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.555+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7ff62407a910 con 0x7ff634062cc0 2023-05-16T07:19:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.555+0000 7ff631d9b700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff61405ed90 0x7ff614061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.556+0000 7ff631d9b700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff61405ed90 0x7ff614061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff6340aa460 tx=0x7ff61c009040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.559+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff62404bc50 con 0x7ff634062cc0 2023-05-16T07:19:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.782+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7ff63406c350 con 0x7ff634062cc0 2023-05-16T07:19:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.786+0000 7ff62a7fc700 1 -- 172.21.15.92:0/3689662195 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff62404f270 con 0x7ff634062cc0 2023-05-16T07:19:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff61405ed90 msgr2=0x7ff614061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff61405ed90 0x7ff614061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff6340aa460 tx=0x7ff61c009040).stop 2023-05-16T07:19:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 msgr2=0x7ff634156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 0x7ff634156180 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7ff62400c0c0 tx=0x7ff62400b040).stop 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 shutdown_connections 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7ff61405ed90 0x7ff614061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff634153ee0 0x7ff63414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634062cc0 0x7ff634156180 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 --2- 172.21.15.92:0/3689662195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6340be280 0x7ff6340ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 >> 172.21.15.92:0/3689662195 conn(0x7ff6340c8420 msgr2=0x7ff6340bcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 shutdown_connections 2023-05-16T07:19:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.788+0000 7ff638c50700 1 -- 172.21.15.92:0/3689662195 wait complete. 2023-05-16T07:19:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:19:52.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:19:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.911+0000 7fbcaf796700 1 Processor -- start 2023-05-16T07:19:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.912+0000 7fbcaf796700 1 -- start start 2023-05-16T07:19:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.912+0000 7fbcaf796700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca80c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.912+0000 7fbcaf796700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbca8069db0 0x7fbca80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.912+0000 7fbcaf796700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80c0660 con 0x7fbca8154550 2023-05-16T07:19:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.912+0000 7fbcaf796700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca80c1b00 con 0x7fbca8069db0 2023-05-16T07:19:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcaf796700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80bfd30 con 0x7fbca8154170 2023-05-16T07:19:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca80c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcadd33700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbca8069db0 0x7fbca80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca80c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:42750/0 (socket says 172.21.15.92:42750) 2023-05-16T07:19:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 -- 172.21.15.92:0/2685131372 learned_addr learned my addr 172.21.15.92:0/2685131372 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:52.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 -- 172.21.15.92:0/2685131372 >> v2:172.21.15.158:3300/0 conn(0x7fbca8069db0 msgr2=0x7fbca80c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 --2- 172.21.15.92:0/2685131372 >> v2:172.21.15.158:3300/0 conn(0x7fbca8069db0 0x7fbca80c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 -- 172.21.15.92:0/2685131372 >> v1:172.21.15.158:6789/0 conn(0x7fbca8154550 legacy=0x7fbca80696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 -- 172.21.15.92:0/2685131372 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80c0010 con 0x7fbca8154170 2023-05-16T07:19:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.913+0000 7fbcad532700 1 --2- 172.21.15.92:0/2685131372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca80c5c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fbc98009f20 tx=0x7fbc98009840).ready entity=mon.2 client_cookie=817700e151dbb461 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbca7fff700 1 -- 172.21.15.92:0/2685131372 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800e070 con 0x7fbca8154170 2023-05-16T07:19:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbca7fff700 1 -- 172.21.15.92:0/2685131372 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9800b9d0 con 0x7fbca8154170 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbca7fff700 1 -- 172.21.15.92:0/2685131372 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800bd50 con 0x7fbca8154170 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 -- 172.21.15.92:0/2685131372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 msgr2=0x7fbca80c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 --2- 172.21.15.92:0/2685131372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca80c5c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fbc98009f20 tx=0x7fbc98009840).stop 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 -- 172.21.15.92:0/2685131372 shutdown_connections 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 --2- 172.21.15.92:0/2685131372 >> v2:172.21.15.158:3300/0 conn(0x7fbca8069db0 0x7fbca80c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 --2- 172.21.15.92:0/2685131372 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca80c5c70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 -- 172.21.15.92:0/2685131372 >> 172.21.15.92:0/2685131372 conn(0x7fbca80cb700 msgr2=0x7fbca80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 -- 172.21.15.92:0/2685131372 shutdown_connections 2023-05-16T07:19:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.914+0000 7fbcaf796700 1 -- 172.21.15.92:0/2685131372 wait complete. 2023-05-16T07:19:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.915+0000 7fbcaf796700 1 Processor -- start 2023-05-16T07:19:52.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.915+0000 7fbcaf796700 1 -- start start 2023-05-16T07:19:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcaf796700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbca8069db0 0x7fbca8165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcaf796700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca814dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcaf796700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 0x7fbca80c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcaf796700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8151d10 con 0x7fbca8154550 2023-05-16T07:19:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcaf796700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8168cd0 con 0x7fbca8069db0 2023-05-16T07:19:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcaf796700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8164f90 con 0x7fbca8154170 2023-05-16T07:19:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcadd33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 0x7fbca80c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcadd33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 0x7fbca80c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50188/0 (socket says 172.21.15.92:50188) 2023-05-16T07:19:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcadd33700 1 -- 172.21.15.92:0/353031458 learned_addr learned my addr 172.21.15.92:0/353031458 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcacd31700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca814dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.916+0000 7fbcad532700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbca8069db0 0x7fbca8165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcadd33700 1 -- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 msgr2=0x7fbca814dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcadd33700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca814dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcadd33700 1 -- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbca8069db0 msgr2=0x7fbca8165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcadd33700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbca8069db0 0x7fbca8165420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:52.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcadd33700 1 -- 172.21.15.92:0/353031458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc98009ae0 con 0x7fbca8154550 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcadd33700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 0x7fbca80c91e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fbca000e900 tx=0x7fbca0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0012600 con 0x7fbca8154550 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca0012760 con 0x7fbca8154550 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0012ae0 con 0x7fbca8154550 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca814dfd0 con 0x7fbca8154550 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.917+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca8164940 con 0x7fbca8154550 2023-05-16T07:19:52.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.918+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80666c0 con 0x7fbca8154550 2023-05-16T07:19:52.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.918+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbca0022070 con 0x7fbca8154550 2023-05-16T07:19:52.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.919+0000 7fbca57fa700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc9405ed60 0x7fbc94061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:52.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.919+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fbca007a090 con 0x7fbca8154550 2023-05-16T07:19:52.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.919+0000 7fbcad532700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc9405ed60 0x7fbc94061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:52.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.919+0000 7fbcad532700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc9405ed60 0x7fbc94061220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbc98009930 tx=0x7fbc98009840).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:52.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:52.923+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbca004b3d0 con 0x7fbca8154550 2023-05-16T07:19:53.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.128+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbca8164770 con 0x7fbca8154550 2023-05-16T07:19:53.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.129+0000 7fbca57fa700 1 -- 172.21.15.92:0/353031458 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fbca0016090 con 0x7fbca8154550 2023-05-16T07:19:53.133 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:19:53.134 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:49.065391+0000","modified":"2023-05-16T07:19:51.075477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2186588149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2186588149},{"type":"v1","addr":"172.21.15.92:6835","nonce":2186588149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-05-16T07:19:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc9405ed60 msgr2=0x7fbc94061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc9405ed60 0x7fbc94061220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbc98009930 tx=0x7fbc98009840).stop 2023-05-16T07:19:53.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 msgr2=0x7fbca80c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 0x7fbca80c91e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fbca000e900 tx=0x7fbca0009060).stop 2023-05-16T07:19:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 shutdown_connections 2023-05-16T07:19:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fbc9405ed60 0x7fbc94061220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8154550 0x7fbca80c91e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbca8069db0 0x7fbca8165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 --2- 172.21.15.92:0/353031458 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbca8154170 0x7fbca814dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 >> 172.21.15.92:0/353031458 conn(0x7fbca80cb700 msgr2=0x7fbca814f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.130+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 shutdown_connections 2023-05-16T07:19:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.131+0000 7fbcaf796700 1 -- 172.21.15.92:0/353031458 wait complete. 2023-05-16T07:19:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 49 2023-05-16T07:19:53.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:19:49.065391+0000', 'modified': '2023-05-16T07:19:51.075477+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7875}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7875': {'gid': 7875, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6835/2186588149', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6833', 'nonce': 2186588149}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2186588149}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:19:53.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:19:53.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-16T07:19:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.661+0000 7f078098a700 1 Processor -- start 2023-05-16T07:19:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.661+0000 7f078098a700 1 -- start start 2023-05-16T07:19:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.662+0000 7f078098a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f077c0698b0 0x7f077c069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.662+0000 7f078098a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f077c066e50 0x7f077c0659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.662+0000 7f078098a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c13a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.662+0000 7f078098a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c0672e0 con 0x7f077c0698b0 2023-05-16T07:19:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.662+0000 7f078098a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c1558f0 con 0x7f077c066e50 2023-05-16T07:19:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.662+0000 7f078098a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c065eb0 con 0x7f077c13a170 2023-05-16T07:19:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f077c0698b0 0x7f077c069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f077c0698b0 0x7f077c069c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50192/0 (socket says 172.21.15.92:50192) 2023-05-16T07:19:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 -- 172.21.15.92:0/1347956527 learned_addr learned my addr 172.21.15.92:0/1347956527 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f0779d9b700 1 --2- 172.21.15.92:0/1347956527 >> v2:172.21.15.158:3300/0 conn(0x7f077c066e50 0x7f077c0659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077ad9d700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c13a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 -- 172.21.15.92:0/1347956527 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 msgr2=0x7f077c13a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c13a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 -- 172.21.15.92:0/1347956527 >> v2:172.21.15.158:3300/0 conn(0x7f077c066e50 msgr2=0x7f077c0659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 --2- 172.21.15.92:0/1347956527 >> v2:172.21.15.158:3300/0 conn(0x7f077c066e50 0x7f077c0659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 -- 172.21.15.92:0/1347956527 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0674c0 con 0x7f077c0698b0 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f0779d9b700 1 --2- 172.21.15.92:0/1347956527 >> v2:172.21.15.158:3300/0 conn(0x7f077c066e50 0x7f077c0659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077ad9d700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c13a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077a59c700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c0698b0 0x7f077c069c90 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f077c0690a0 tx=0x7f076c0090f0).ready entity=mon.0 client_cookie=24f2eca1fd4ff607 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077959a700 1 -- 172.21.15.92:0/1347956527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c00b600 con 0x7f077c0698b0 2023-05-16T07:19:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077959a700 1 -- 172.21.15.92:0/1347956527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c00b760 con 0x7f077c0698b0 2023-05-16T07:19:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.663+0000 7f077959a700 1 -- 172.21.15.92:0/1347956527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c00bae0 con 0x7f077c0698b0 2023-05-16T07:19:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 -- 172.21.15.92:0/1347956527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c0698b0 msgr2=0x7f077c069c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c0698b0 0x7f077c069c90 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f077c0690a0 tx=0x7f076c0090f0).stop 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 -- 172.21.15.92:0/1347956527 shutdown_connections 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c13a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 --2- 172.21.15.92:0/1347956527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c0698b0 0x7f077c069c90 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 --2- 172.21.15.92:0/1347956527 >> v2:172.21.15.158:3300/0 conn(0x7f077c066e50 0x7f077c0659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 -- 172.21.15.92:0/1347956527 >> 172.21.15.92:0/1347956527 conn(0x7f077c063630 msgr2=0x7f077c0c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 -- 172.21.15.92:0/1347956527 shutdown_connections 2023-05-16T07:19:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.664+0000 7f078098a700 1 -- 172.21.15.92:0/1347956527 wait complete. 2023-05-16T07:19:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.665+0000 7f078098a700 1 Processor -- start 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.665+0000 7f078098a700 1 -- start start 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 0x7f077c158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f077c0698b0 0x7f077c139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0a8310 con 0x7f077c066e50 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c158600 con 0x7f077c0698b0 2023-05-16T07:19:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c139c60 con 0x7f077c13a170 2023-05-16T07:19:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 0x7f077c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 0x7f077c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50196/0 (socket says 172.21.15.92:50196) 2023-05-16T07:19:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 -- 172.21.15.92:0/2095853025 learned_addr learned my addr 172.21.15.92:0/2095853025 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f077c0698b0 0x7f077c139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077ad9d700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 -- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 msgr2=0x7f077c155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 -- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f077c0698b0 msgr2=0x7f077c139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f077c0698b0 0x7f077c139770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 -- 172.21.15.92:0/2095853025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c009370 con 0x7f077c066e50 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f077c0698b0 0x7f077c139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f077a59c700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 0x7f077c158110 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f077c06aa00 tx=0x7f076c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f076a7fc700 1 -- 172.21.15.92:0/2095853025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c00b8d0 con 0x7f077c066e50 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f076a7fc700 1 -- 172.21.15.92:0/2095853025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c00ba70 con 0x7f077c066e50 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f076a7fc700 1 -- 172.21.15.92:0/2095853025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c01b660 con 0x7f077c066e50 2023-05-16T07:19:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c158850 con 0x7f077c066e50 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.666+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c158e10 con 0x7f077c066e50 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.668+0000 7f076a7fc700 1 -- 172.21.15.92:0/2095853025 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f076c02b070 con 0x7f077c066e50 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.668+0000 7f076a7fc700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f075c05ed90 0x7f075c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.668+0000 7f076a7fc700 1 -- 172.21.15.92:0/2095853025 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f076c07b580 con 0x7f077c066e50 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.668+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f075c05ed90 0x7f075c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.669+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f075c05ed90 0x7f075c061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f077c159bd0 tx=0x7f0770008040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.670+0000 7f079231cb80 1 Processor -- start 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.670+0000 7f079231cb80 1 -- start start 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce67f560 0x5591ce662c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce67f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f079231cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 0x5591ce635710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f079231cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce6005e0 con 0x5591ce63b300 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce5778a0 con 0x5591ce67f560 2023-05-16T07:19:53.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591ce58c900 con 0x5591ce684480 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 0x5591ce635710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 0x5591ce635710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50208/0 (socket says 172.21.15.92:50208) 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 -- 172.21.15.92:0/1048859744 learned_addr learned my addr 172.21.15.92:0/1048859744 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077a59c700 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce67f560 0x5591ce662c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f0779d9b700 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce67f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 -- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 msgr2=0x5591ce67f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce67f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 -- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce67f560 msgr2=0x5591ce662c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce67f560 0x5591ce662c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 -- 172.21.15.92:0/1048859744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ce600450 con 0x5591ce63b300 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.671+0000 7f077ad9d700 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 0x5591ce635710 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f0774007460 tx=0x7f0774009b60).ready entity=mon.0 client_cookie=3dcfacb523698fab server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f075b7fe700 1 -- 172.21.15.92:0/1048859744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774007e30 con 0x5591ce63b300 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f075b7fe700 1 -- 172.21.15.92:0/1048859744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400f030 con 0x5591ce63b300 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f075b7fe700 1 -- 172.21.15.92:0/1048859744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740026a0 con 0x5591ce63b300 2023-05-16T07:19:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 -- 172.21.15.92:0/1048859744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 msgr2=0x5591ce635710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 0x5591ce635710 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f0774007460 tx=0x7f0774009b60).stop 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 -- 172.21.15.92:0/1048859744 shutdown_connections 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce63b300 0x5591ce635710 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce67f560 0x5591ce662c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 --2- 172.21.15.92:0/1048859744 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce67f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 -- 172.21.15.92:0/1048859744 >> 172.21.15.92:0/1048859744 conn(0x5591ce3f7e40 msgr2=0x5591ce6839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 -- 172.21.15.92:0/1048859744 shutdown_connections 2023-05-16T07:19:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.672+0000 7f079231cb80 1 -- 172.21.15.92:0/1048859744 wait complete. 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.673+0000 7f079231cb80 1 -- start start 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.673+0000 7f079231cb80 1 Processor -- start 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce65dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce690040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce553ed0 con 0x5591ce67f560 2023-05-16T07:19:53.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce67c630 con 0x5591ce63b300 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591ce635170 con 0x5591ce684480 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f079231cb80 10 client.0 ll_register_callbacks cb 0x5591ce5fa1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50212/0 (socket says 172.21.15.92:50212) 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 -- 172.21.15.92:0/4449954 learned_addr learned my addr 172.21.15.92:0/4449954 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f077ad9d700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce690040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f077a59c700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce65dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 -- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 msgr2=0x5591ce690040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce690040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 -- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 msgr2=0x5591ce65dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce65dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 -- 172.21.15.92:0/4449954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0774009e20 con 0x5591ce67f560 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f077ad9d700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce690040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0779d9b700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6366a0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f0770013550 tx=0x7f0770016040).ready entity=mon.0 client_cookie=ed7705348c152464 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.674+0000 7f0758ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:19:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f0758ff9700 1 -- 172.21.15.92:0/4449954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770013e70 con 0x5591ce67f560 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f0758ff9700 1 -- 172.21.15.92:0/4449954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f0770012570 con 0x5591ce67f560 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591ce62e460 con 0x5591ce67f560 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f0758ff9700 1 -- 172.21.15.92:0/4449954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0770012940 con 0x5591ce67f560 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 10 client.8025 fetch_fsmap learned FSMap version 49 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5591ce42f310 con 0x5591ce67f560 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f0758ff9700 1 -- 172.21.15.92:0/4449954 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591ce62e460 con 0x5591ce67f560 2023-05-16T07:19:53.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f0758ff9700 1 -- 172.21.15.92:0/4449954 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 49) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0770012aa0 con 0x5591ce67f560 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 10 client.8025 fetch_fsmap finished waiting for FSMap version 49 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 10 client.8025 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 20 client.8025 populate_metadata read hostname 'smithi092' 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce664080 0x5591ce664460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 4 client.8025 mds_command: new command op to 7875 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 --> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591ce3f3ca0 con 0x5591ce664080 2023-05-16T07:19:53.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f077a59c700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce664080 0x5591ce664460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f0758ff9700 10 client.8025 ms_handle_connect on v2:172.21.15.92:6833/2186588149 2023-05-16T07:19:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.675+0000 7f077a59c700 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce664080 0x5591ce664460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c3eacd2548dc9525 server_cookie=786387dcbb4d209e in_seq=0 out_seq=0 2023-05-16T07:19:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f0758ff9700 1 -- 172.21.15.92:0/4449954 <== mds.0 v2:172.21.15.92:6833/2186588149 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5591ce3f3ca0 con 0x5591ce664080 2023-05-16T07:19:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f0758ff9700 10 client.8025 handle_command_reply: tid=0 2023-05-16T07:19:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 client.8025 shutdown 2023-05-16T07:19:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 msgr2=0x5591ce6366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6366a0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f0770013550 tx=0x7f0770016040).stop 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 shutdown_connections 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce664080 0x5591ce664460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce690040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f0758ff9700 0 client.8025 ms_handle_reset on v2:172.21.15.92:6833/2186588149 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce65dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 --2- 172.21.15.92:0/4449954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6366a0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 >> 172.21.15.92:0/4449954 conn(0x5591ce3f7e40 msgr2=0x5591ce63a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 shutdown_connections 2023-05-16T07:19:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 1 -- 172.21.15.92:0/4449954 wait complete. 2023-05-16T07:19:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.676+0000 7f079231cb80 20 client.8025 trim_cache size 0 max 16384 2023-05-16T07:19:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.692+0000 7f079231cb80 1 Processor -- start 2023-05-16T07:19:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 -- start start 2023-05-16T07:19:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce6347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce67f560 0x5591ce65fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 0x5591ce6601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce635970 con 0x5591ce684480 2023-05-16T07:19:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce67be90 con 0x5591ce63b300 2023-05-16T07:19:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591ce3f3ca0 con 0x5591ce67f560 2023-05-16T07:19:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f077ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 0x5591ce6601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f077ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 0x5591ce6601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50222/0 (socket says 172.21.15.92:50222) 2023-05-16T07:19:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.693+0000 7f077ad9d700 1 -- 172.21.15.92:0/317605252 learned_addr learned my addr 172.21.15.92:0/317605252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077a59c700 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce6347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f0779d9b700 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce67f560 0x5591ce65fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077ad9d700 1 -- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce67f560 msgr2=0x5591ce65fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077ad9d700 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce67f560 0x5591ce65fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077ad9d700 1 -- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 msgr2=0x5591ce6347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077ad9d700 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce6347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077ad9d700 1 -- 172.21.15.92:0/317605252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ce585880 con 0x5591ce684480 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077a59c700 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce6347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f077ad9d700 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 0x5591ce6601d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f0774007bb0 tx=0x7f0774019000).ready entity=mon.0 client_cookie=332214bcb8776d62 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f075bfff700 1 -- 172.21.15.92:0/317605252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774007e30 con 0x5591ce684480 2023-05-16T07:19:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f075bfff700 1 -- 172.21.15.92:0/317605252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774002490 con 0x5591ce684480 2023-05-16T07:19:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f075bfff700 1 -- 172.21.15.92:0/317605252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774002a90 con 0x5591ce684480 2023-05-16T07:19:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 -- 172.21.15.92:0/317605252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 msgr2=0x5591ce6601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 0x5591ce6601d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f0774007bb0 tx=0x7f0774019000).stop 2023-05-16T07:19:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 -- 172.21.15.92:0/317605252 shutdown_connections 2023-05-16T07:19:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce684480 0x5591ce6601d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce6347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 --2- 172.21.15.92:0/317605252 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce67f560 0x5591ce65fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 -- 172.21.15.92:0/317605252 >> 172.21.15.92:0/317605252 conn(0x5591ce3f7e40 msgr2=0x5591ce679f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 -- 172.21.15.92:0/317605252 shutdown_connections 2023-05-16T07:19:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.694+0000 7f079231cb80 1 -- 172.21.15.92:0/317605252 wait complete. 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.695+0000 7f079231cb80 1 -- start start 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.695+0000 7f079231cb80 1 Processor -- start 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce665360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce6e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce639ec0 con 0x5591ce67f560 2023-05-16T07:19:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ce683a70 con 0x5591ce63b300 2023-05-16T07:19:53.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591ce665850 con 0x5591ce684480 2023-05-16T07:19:53.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f079231cb80 10 client.0 ll_register_callbacks cb 0x5591ce5fa1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:19:53.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6e5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50226/0 (socket says 172.21.15.92:50226) 2023-05-16T07:19:53.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 -- 172.21.15.92:0/2603268803 learned_addr learned my addr 172.21.15.92:0/2603268803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f077a59c700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce665360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f077ad9d700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce6e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 -- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 msgr2=0x5591ce6e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce6e5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 -- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 msgr2=0x5591ce665360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce665360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 -- 172.21.15.92:0/2603268803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591ce600450 con 0x5591ce67f560 2023-05-16T07:19:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f077ad9d700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce6e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0779d9b700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6e5160 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f0770029040 tx=0x7f0770020000).ready entity=mon.0 client_cookie=4aca39e89a180a05 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.696+0000 7f0758ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:19:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f0758ff9700 1 -- 172.21.15.92:0/2603268803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770033030 con 0x5591ce67f560 2023-05-16T07:19:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f0758ff9700 1 -- 172.21.15.92:0/2603268803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f0770034070 con 0x5591ce67f560 2023-05-16T07:19:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f077400f030 con 0x5591ce67f560 2023-05-16T07:19:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f0758ff9700 1 -- 172.21.15.92:0/2603268803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0770012570 con 0x5591ce67f560 2023-05-16T07:19:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 10 client.8031 fetch_fsmap learned FSMap version 49 2023-05-16T07:19:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5591ce581070 con 0x5591ce67f560 2023-05-16T07:19:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f0758ff9700 1 -- 172.21.15.92:0/2603268803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400f030 con 0x5591ce67f560 2023-05-16T07:19:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f0758ff9700 1 -- 172.21.15.92:0/2603268803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 49) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0770005880 con 0x5591ce67f560 2023-05-16T07:19:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 10 client.8031 fetch_fsmap finished waiting for FSMap version 49 2023-05-16T07:19:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 10 client.8031 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-16T07:19:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 20 client.8031 populate_metadata read hostname 'smithi092' 2023-05-16T07:19:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce68e8b0 0x5591ce68ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 4 client.8031 mds_command: new command op to 7875 tid=0[{"prefix": "status"}] 2023-05-16T07:19:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 --> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5591ce42aca0 con 0x5591ce68e8b0 2023-05-16T07:19:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f077a59c700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce68e8b0 0x5591ce68ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f0758ff9700 10 client.8031 ms_handle_connect on v2:172.21.15.92:6833/2186588149 2023-05-16T07:19:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.697+0000 7f077a59c700 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce68e8b0 0x5591ce68ec90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d407e78a55c83784 server_cookie=6896e6dd73c3c80e in_seq=0 out_seq=0 2023-05-16T07:19:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f0758ff9700 1 -- 172.21.15.92:0/2603268803 <== mds.0 v2:172.21.15.92:6833/2186588149 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5591ce42aca0 con 0x5591ce68e8b0 2023-05-16T07:19:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f0758ff9700 10 client.8031 handle_command_reply: tid=0 2023-05-16T07:19:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 client.8031 shutdown 2023-05-16T07:19:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 msgr2=0x5591ce6e5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6e5160 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f0770029040 tx=0x7f0770020000).stop 2023-05-16T07:19:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 shutdown_connections 2023-05-16T07:19:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:6833/2186588149,v1:172.21.15.92:6835/2186588149] conn(0x5591ce68e8b0 0x5591ce68ec90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5591ce684480 0x5591ce6e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f0758ff9700 0 client.8031 ms_handle_reset on v2:172.21.15.92:6833/2186588149 2023-05-16T07:19:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5591ce63b300 0x5591ce665360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 --2- 172.21.15.92:0/2603268803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591ce67f560 0x5591ce6e5160 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 >> 172.21.15.92:0/2603268803 conn(0x5591ce3f7e40 msgr2=0x5591ce6347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 shutdown_connections 2023-05-16T07:19:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 1 -- 172.21.15.92:0/2603268803 wait complete. 2023-05-16T07:19:53.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f079231cb80 20 client.8031 trim_cache size 0 max 16384 2023-05-16T07:19:53.720 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:19:53.721 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:19:53.721 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:19:53.721 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7875, 2023-05-16T07:19:53.722 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:19:53.722 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:19:53.722 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:19:53.722 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.6317519999999996, 2023-05-16T07:19:53.722 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 49, 2023-05-16T07:19:53.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 135, 2023-05-16T07:19:53.723 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 135, 2023-05-16T07:19:53.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.1763606860000007 2023-05-16T07:19:53.723 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:19:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f075c05ed90 msgr2=0x7f075c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f075c05ed90 0x7f075c061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f077c159bd0 tx=0x7f0770008040).stop 2023-05-16T07:19:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 msgr2=0x7f077c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 0x7f077c158110 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f077c06aa00 tx=0x7f076c017000).stop 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 shutdown_connections 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f075c05ed90 0x7f075c061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f077c13a170 0x7f077c155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f077c066e50 0x7f077c158110 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 --2- 172.21.15.92:0/2095853025 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f077c0698b0 0x7f077c139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 >> 172.21.15.92:0/2095853025 conn(0x7f077c063630 msgr2=0x7f077c0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 shutdown_connections 2023-05-16T07:19:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.698+0000 7f078098a700 1 -- 172.21.15.92:0/2095853025 wait complete. 2023-05-16T07:19:53.726 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:19:53.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.822+0000 7fb0ff06b700 1 Processor -- start 2023-05-16T07:19:53.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.822+0000 7fb0ff06b700 1 -- start start 2023-05-16T07:19:53.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0ff06b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0f8153dc0 0x7fb0f80c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0ff06b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0ff06b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb0f80c5c50 0x7fb0f8063ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0ff06b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f8063fd0 con 0x7fb0f8153dc0 2023-05-16T07:19:53.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0ff06b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f80aaf40 con 0x7fb0f80c5c50 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0ff06b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f80c1450 con 0x7fb0f81541a0 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0f8153dc0 0x7fb0f80c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0f8153dc0 0x7fb0f80c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50232/0 (socket says 172.21.15.92:50232) 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 -- 172.21.15.92:0/198941413 learned_addr learned my addr 172.21.15.92:0/198941413 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fd608700 1 --2- 172.21.15.92:0/198941413 >> v2:172.21.15.158:3300/0 conn(0x7fb0f80c5c50 0x7fb0f8063ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0f7fff700 1 --2- 172.21.15.92:0/198941413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 -- 172.21.15.92:0/198941413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 msgr2=0x7fb0f80635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 --2- 172.21.15.92:0/198941413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 -- 172.21.15.92:0/198941413 >> v2:172.21.15.158:3300/0 conn(0x7fb0f80c5c50 msgr2=0x7fb0f8063ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 --2- 172.21.15.92:0/198941413 >> v2:172.21.15.158:3300/0 conn(0x7fb0f80c5c50 0x7fb0f8063ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 -- 172.21.15.92:0/198941413 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f80c1730 con 0x7fb0f8153dc0 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fd608700 1 --2- 172.21.15.92:0/198941413 >> v2:172.21.15.158:3300/0 conn(0x7fb0f80c5c50 0x7fb0f8063ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0fce07700 1 --2- 172.21.15.92:0/198941413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f8153dc0 0x7fb0f80c0bc0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7fb0f80aaeb0 tx=0x7fb0e80090f0).ready entity=mon.0 client_cookie=aab9d5d2f225844e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0f77fe700 1 -- 172.21.15.92:0/198941413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800b600 con 0x7fb0f8153dc0 2023-05-16T07:19:53.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0f77fe700 1 -- 172.21.15.92:0/198941413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800b760 con 0x7fb0f8153dc0 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.823+0000 7fb0f77fe700 1 -- 172.21.15.92:0/198941413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800bae0 con 0x7fb0f8153dc0 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 -- 172.21.15.92:0/198941413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f8153dc0 msgr2=0x7fb0f80c0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/198941413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f8153dc0 0x7fb0f80c0bc0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7fb0f80aaeb0 tx=0x7fb0e80090f0).stop 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 -- 172.21.15.92:0/198941413 shutdown_connections 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/198941413 >> v2:172.21.15.158:3300/0 conn(0x7fb0f80c5c50 0x7fb0f8063ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/198941413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f8153dc0 0x7fb0f80c0bc0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/198941413 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 -- 172.21.15.92:0/198941413 >> 172.21.15.92:0/198941413 conn(0x7fb0f80caca0 msgr2=0x7fb0f80cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:53.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 -- 172.21.15.92:0/198941413 shutdown_connections 2023-05-16T07:19:53.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.824+0000 7fb0ff06b700 1 -- 172.21.15.92:0/198941413 wait complete. 2023-05-16T07:19:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.825+0000 7fb0ff06b700 1 Processor -- start 2023-05-16T07:19:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.825+0000 7fb0ff06b700 1 -- start start 2023-05-16T07:19:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0ff06b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 0x7fb0f8167f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0ff06b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0f8153dc0 0x7fb0f8163930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0ff06b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0ff06b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f813bfc0 con 0x7fb0f80c5c50 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0ff06b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f80aecd0 con 0x7fb0f8153dc0 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0ff06b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f8168420 con 0x7fb0f81541a0 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 0x7fb0f8167f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 0x7fb0f8167f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50238/0 (socket says 172.21.15.92:50238) 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 -- 172.21.15.92:0/4057295965 learned_addr learned my addr 172.21.15.92:0/4057295965 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fd608700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0f7fff700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0f8153dc0 0x7fb0f8163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 -- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 msgr2=0x7fb0f80c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 -- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0f8153dc0 msgr2=0x7fb0f8163930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0f8153dc0 0x7fb0f8163930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 -- 172.21.15.92:0/4057295965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8009370 con 0x7fb0f80c5c50 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0fce07700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 0x7fb0f8167f30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7fb0e8006fd0 tx=0x7fb0e8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.826+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800b8d0 con 0x7fb0f80c5c50 2023-05-16T07:19:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.827+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800ba30 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.827+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f8163e20 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.827+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e801b660 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.827+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f80ae3a0 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.827+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f805efa0 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.828+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb0e800f070 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.828+0000 7fb0f4ff9700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb0dc05ed90 0x7fb0dc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.828+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fb0e807b0b0 con 0x7fb0f80c5c50 2023-05-16T07:19:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.828+0000 7fb0f7fff700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb0dc05ed90 0x7fb0dc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.829+0000 7fb0f7fff700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb0dc05ed90 0x7fb0dc061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb0e0006fd0 tx=0x7fb0e0008040).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:53.832+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb0e804c470 con 0x7fb0f80c5c50 2023-05-16T07:19:54.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.038+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0f80c1a10 con 0x7fb0f80c5c50 2023-05-16T07:19:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.038+0000 7fb0f4ff9700 1 -- 172.21.15.92:0/4057295965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0e801f070 con 0x7fb0f80c5c50 2023-05-16T07:19:54.042 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb0dc05ed90 msgr2=0x7fb0dc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb0dc05ed90 0x7fb0dc061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb0e0006fd0 tx=0x7fb0e0008040).stop 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 msgr2=0x7fb0f8167f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 0x7fb0f8167f30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7fb0e8006fd0 tx=0x7fb0e8017000).stop 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 shutdown_connections 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb0dc05ed90 0x7fb0dc061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb0f81541a0 0x7fb0f80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0f80c5c50 0x7fb0f8167f30 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 --2- 172.21.15.92:0/4057295965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0f8153dc0 0x7fb0f8163930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 >> 172.21.15.92:0/4057295965 conn(0x7fb0f80caca0 msgr2=0x7fb0f8151590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 shutdown_connections 2023-05-16T07:19:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.040+0000 7fb0ff06b700 1 -- 172.21.15.92:0/4057295965 wait complete. 2023-05-16T07:19:54.054 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:19:54.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:19:54.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:19:54.055 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:19:54.056 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:54.056 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:19:54.056 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:19:54.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:19:54.057 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:19:54.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.058 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:19:54.072 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:19:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:54.074 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:19:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:54.074 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:19:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:54.074 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:19:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:19:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 40961sec preferred_lft 40961sec 2023-05-16T07:19:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:19:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.075 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:19:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:19:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:19:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:19:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:19:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:19:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:19:54.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.079 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:19:54.135 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:19:54.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.136 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:19:54.191 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:19:54.192 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:19:54.192 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:19:54.192 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:19:54.192 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:19:54.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.193 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:19:54.193 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:19:54.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:19:54.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:19:54.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:19:54.195 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:19:54.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.290 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:19:54.304 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:19:54.305 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:19:54.305 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:19:54.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:19:54.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.369 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:54.422 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:19:54.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:19:54.500 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:19:54.501 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:19:54.543 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:19:54.544 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:19:54.545 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:19:54.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:19:54.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:19:54.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:19:54.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.755+0000 7f644f448700 1 Processor -- start 2023-05-16T07:19:54.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.755+0000 7f644f448700 1 -- start start 2023-05-16T07:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.756+0000 7f644f448700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f64480696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.756+0000 7f644f448700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64480c0660 con 0x7f6448154170 2023-05-16T07:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.756+0000 7f644f448700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644806a020 con 0x7f6448069b90 2023-05-16T07:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.756+0000 7f644f448700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64480bfd30 con 0x7f6448154550 2023-05-16T07:19:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.756+0000 7f644d1e4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6448154170 0x7f64480c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32768/0 (socket says 172.21.15.92:32768) 2023-05-16T07:19:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.756+0000 7f644d1e4700 1 -- 172.21.15.92:0/2551408470 learned_addr learned my addr 172.21.15.92:0/2551408470 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:54.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229008234 0 0) 0x7f64480c0660 con 0x7f6448154170 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f64480696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64300033f0 con 0x7f6448154170 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710694057 0 0) 0x7f64300033f0 con 0x7f6448154170 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 msgr2=0x7f64480696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f64480696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 >> v1:172.21.15.158:6789/0 conn(0x7f6448069b90 legacy=0x7f64480c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64480c0010 con 0x7f6448154170 2023-05-16T07:19:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f64380017f0 con 0x7f6448154170 2023-05-16T07:19:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 >> v1:172.21.15.92:6789/0 conn(0x7f6448154170 legacy=0x7f64480c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6430003910 0x7f6430003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f6430004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f6430005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64480bfd30 con 0x7f6430004690 2023-05-16T07:19:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643000b8e0 con 0x7f6430003910 2023-05-16T07:19:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643000ba50 con 0x7f6448154550 2023-05-16T07:19:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f6430004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644d1e4700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6430003910 0x7f6430003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644d9e5700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f6430005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 -- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 msgr2=0x7f6430005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f6430005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 -- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6430003910 msgr2=0x7f6430003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6430003910 0x7f6430003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644c9e3700 1 -- 172.21.15.92:0/2551408470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300033f0 con 0x7f6430004690 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.757+0000 7f644d9e5700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448154550 0x7f6430005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:54.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644c9e3700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f6430004b10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f6448154930 tx=0x7f643c009060).ready entity=mon.0 client_cookie=a6fc7588f521cd14 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c009430 con 0x7f6430004690 2023-05-16T07:19:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c009e70 con 0x7f6430004690 2023-05-16T07:19:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f6447fff700 1 -- 172.21.15.92:0/2551408470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00e7f0 con 0x7f6430004690 2023-05-16T07:19:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644f448700 1 -- 172.21.15.92:0/2551408470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 msgr2=0x7f6430004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644f448700 1 --2- 172.21.15.92:0/2551408470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f6430004b10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f6448154930 tx=0x7f643c009060).stop 2023-05-16T07:19:54.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644d9e5700 1 -- 172.21.15.92:0/2551408470 reap_dead start 2023-05-16T07:19:54.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644f448700 1 -- 172.21.15.92:0/2551408470 shutdown_connections 2023-05-16T07:19:54.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644f448700 1 -- 172.21.15.92:0/2551408470 >> 172.21.15.92:0/2551408470 conn(0x7f64480cb750 msgr2=0x7f64480c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:54.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644f448700 1 -- 172.21.15.92:0/2551408470 shutdown_connections 2023-05-16T07:19:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.758+0000 7f644f448700 1 -- 172.21.15.92:0/2551408470 wait complete. 2023-05-16T07:19:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.759+0000 7f644f448700 1 Processor -- start 2023-05-16T07:19:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.759+0000 7f644f448700 1 -- start start 2023-05-16T07:19:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644f448700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f644814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644f448700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448150eb0 0x7f64480adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644f448700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64480ae140 0x7f644814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644f448700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64480c2150 con 0x7f6430004690 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644f448700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64480aec40 con 0x7f64480ae140 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644f448700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64480ae680 con 0x7f6448150eb0 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644d1e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f644814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644d1e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f644814c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50264/0 (socket says 172.21.15.92:50264) 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.760+0000 7f644d1e4700 1 -- 172.21.15.92:0/3745720559 learned_addr learned my addr 172.21.15.92:0/3745720559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644c9e3700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448150eb0 0x7f64480adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d9e5700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64480ae140 0x7f644814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d1e4700 1 -- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448150eb0 msgr2=0x7f64480adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d1e4700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448150eb0 0x7f64480adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d1e4700 1 -- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64480ae140 msgr2=0x7f644814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d1e4700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64480ae140 0x7f644814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d1e4700 1 -- 172.21.15.92:0/3745720559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64480ae960 con 0x7f6430004690 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644d1e4700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f644814c320 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f6438007710 tx=0x7f6438001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643800fa00 con 0x7f6430004690 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643800fb60 con 0x7f6430004690 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64480c1de0 con 0x7f6430004690 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643800fee0 con 0x7f6430004690 2023-05-16T07:19:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.761+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644814cce0 con 0x7f6430004690 2023-05-16T07:19:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.762+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f643801b030 con 0x7f6430004690 2023-05-16T07:19:54.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.762+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64480666c0 con 0x7f6430004690 2023-05-16T07:19:54.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.762+0000 7f64457fa700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f643405ed60 0x7f6434061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:54.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.763+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f6438079990 con 0x7f6430004690 2023-05-16T07:19:54.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.763+0000 7f644c9e3700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f643405ed60 0x7f6434061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:54.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.763+0000 7f644c9e3700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f643405ed60 0x7f6434061220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f64480b0770 tx=0x7f643c01a000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:54.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.767+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f643804ad20 con 0x7f6430004690 2023-05-16T07:19:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.953+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f64480c2290 con 0x7f643405ed60 2023-05-16T07:19:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.955+0000 7f64457fa700 1 -- 172.21.15.92:0/3745720559 <== mgr.7159 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f643c00e740 con 0x7f643405ed60 2023-05-16T07:19:54.959 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:19:54.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:19:54.959 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:19:54.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:19:54.959 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:19:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f643405ed60 msgr2=0x7f6434061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f643405ed60 0x7f6434061220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f64480b0770 tx=0x7f643c01a000).stop 2023-05-16T07:19:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 msgr2=0x7f644814c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f644814c320 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f6438007710 tx=0x7f6438001d90).stop 2023-05-16T07:19:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 shutdown_connections 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f643405ed60 0x7f6434061220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64480ae140 0x7f644814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6430004690 0x7f644814c320 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 --2- 172.21.15.92:0/3745720559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6448150eb0 0x7f64480adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 >> 172.21.15.92:0/3745720559 conn(0x7f64480cb750 msgr2=0x7f644814f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 shutdown_connections 2023-05-16T07:19:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:54.956+0000 7f644f448700 1 -- 172.21.15.92:0/3745720559 wait complete. 2023-05-16T07:19:54.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:19:54.990 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:19:54.985+0000 7f2059a99700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:19:55.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.081+0000 7f1815c5b700 1 Processor -- start 2023-05-16T07:19:55.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.081+0000 7f1815c5b700 1 -- start start 2023-05-16T07:19:55.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1815c5b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18100c1160 0x7f18100c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:55.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1815c5b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f18100c61f0 0x7f1810154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:55.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1815c5b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181006a140 con 0x7f1810153f90 2023-05-16T07:19:55.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1815c5b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1810069f60 con 0x7f18100c1160 2023-05-16T07:19:55.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1815c5b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181006a9e0 con 0x7f18100c61f0 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1814c59700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1810153f90 0x7f18100a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32788/0 (socket says 172.21.15.92:32788) 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f1814c59700 1 -- 172.21.15.92:0/2634473875 learned_addr learned my addr 172.21.15.92:0/2634473875 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f180ffff700 1 --2- 172.21.15.92:0/2634473875 >> v2:172.21.15.92:3300/0 conn(0x7f18100c1160 0x7f18100c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f181545a700 1 --2- 172.21.15.92:0/2634473875 >> v2:172.21.15.158:3300/0 conn(0x7f18100c61f0 0x7f1810154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f180f7fe700 1 -- 172.21.15.92:0/2634473875 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937579844 0 0) 0x7f181006a140 con 0x7f1810153f90 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f180f7fe700 1 -- 172.21.15.92:0/2634473875 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f80033f0 con 0x7f1810153f90 2023-05-16T07:19:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f180ffff700 1 -- 172.21.15.92:0/2634473875 >> v2:172.21.15.158:3300/0 conn(0x7f18100c61f0 msgr2=0x7f1810154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f180ffff700 1 --2- 172.21.15.92:0/2634473875 >> v2:172.21.15.158:3300/0 conn(0x7f18100c61f0 0x7f1810154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.082+0000 7f180ffff700 1 -- 172.21.15.92:0/2634473875 >> v1:172.21.15.92:6789/0 conn(0x7f1810153f90 legacy=0x7f18100a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f180ffff700 1 -- 172.21.15.92:0/2634473875 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181006a420 con 0x7f18100c1160 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f181545a700 1 --2- 172.21.15.92:0/2634473875 >> v2:172.21.15.158:3300/0 conn(0x7f18100c61f0 0x7f1810154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f180ffff700 1 --2- 172.21.15.92:0/2634473875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f18100c5d00 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f18100c1b50 tx=0x7f1804009060).ready entity=mon.0 client_cookie=cf52d453d09bc16f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f180f7fe700 1 -- 172.21.15.92:0/2634473875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180400f510 con 0x7f18100c1160 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f180f7fe700 1 -- 172.21.15.92:0/2634473875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180400f670 con 0x7f18100c1160 2023-05-16T07:19:55.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f180f7fe700 1 -- 172.21.15.92:0/2634473875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180400f9f0 con 0x7f18100c1160 2023-05-16T07:19:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 -- 172.21.15.92:0/2634473875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 msgr2=0x7f18100c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 --2- 172.21.15.92:0/2634473875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f18100c5d00 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f18100c1b50 tx=0x7f1804009060).stop 2023-05-16T07:19:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 -- 172.21.15.92:0/2634473875 shutdown_connections 2023-05-16T07:19:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 --2- 172.21.15.92:0/2634473875 >> v2:172.21.15.158:3300/0 conn(0x7f18100c61f0 0x7f1810154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 --2- 172.21.15.92:0/2634473875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f18100c5d00 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 -- 172.21.15.92:0/2634473875 >> 172.21.15.92:0/2634473875 conn(0x7f18100cb600 msgr2=0x7f18100c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:55.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 -- 172.21.15.92:0/2634473875 shutdown_connections 2023-05-16T07:19:55.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.083+0000 7f1815c5b700 1 -- 172.21.15.92:0/2634473875 wait complete. 2023-05-16T07:19:55.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.084+0000 7f1815c5b700 1 Processor -- start 2023-05-16T07:19:55.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.084+0000 7f1815c5b700 1 -- start start 2023-05-16T07:19:55.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1815c5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f1810150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:55.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1815c5b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18100c61f0 0x7f18100a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:55.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1815c5b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1810153f90 0x7f18100a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:55.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1815c5b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18100a3160 con 0x7f18100c1160 2023-05-16T07:19:55.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1815c5b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18100aea20 con 0x7f1810153f90 2023-05-16T07:19:55.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1815c5b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18100adc60 con 0x7f18100c61f0 2023-05-16T07:19:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1814c59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f1810150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1814c59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f1810150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50278/0 (socket says 172.21.15.92:50278) 2023-05-16T07:19:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.085+0000 7f1814c59700 1 -- 172.21.15.92:0/1240643116 learned_addr learned my addr 172.21.15.92:0/1240643116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:55.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f181545a700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1810153f90 0x7f18100a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:55.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f180ffff700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18100c61f0 0x7f18100a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:55.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1814c59700 1 -- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18100c61f0 msgr2=0x7f18100a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:55.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1814c59700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18100c61f0 0x7f18100a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:55.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1814c59700 1 -- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1810153f90 msgr2=0x7f18100a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:55.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1814c59700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1810153f90 0x7f18100a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:55.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1814c59700 1 -- 172.21.15.92:0/1240643116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040092a0 con 0x7f18100c1160 2023-05-16T07:19:55.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f180ffff700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18100c61f0 0x7f18100a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:19:55.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1814c59700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f1810150640 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f180000c820 tx=0x7f1800001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:55.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800005d30 con 0x7f18100c1160 2023-05-16T07:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1800005e90 con 0x7f18100c1160 2023-05-16T07:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18100adf40 con 0x7f18100c1160 2023-05-16T07:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800007360 con 0x7f18100c1160 2023-05-16T07:19:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.086+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18100ae500 con 0x7f18100c1160 2023-05-16T07:19:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.087+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1810083120 con 0x7f18100c1160 2023-05-16T07:19:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.090+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 55) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1800010030 con 0x7f18100c1160 2023-05-16T07:19:55.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.090+0000 7f180cff9700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17fc05ed90 0x7f17fc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:55.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.090+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f180007a390 con 0x7f18100c1160 2023-05-16T07:19:55.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.090+0000 7f180ffff700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17fc05ed90 0x7f17fc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:55.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.090+0000 7f180ffff700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17fc05ed90 0x7f17fc061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f18100c15f0 tx=0x7f1804017000).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:55.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.093+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f180004b6f0 con 0x7f18100c1160 2023-05-16T07:19:55.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:55.301+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1810047150 con 0x7f18100c1160 2023-05-16T07:19:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.103+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 56) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f180007a080 con 0x7f18100c1160 2023-05-16T07:19:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.104+0000 7f180cff9700 1 -- 172.21.15.92:0/1240643116 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v56) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1800077240 con 0x7f18100c1160 2023-05-16T07:19:56.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17fc05ed90 msgr2=0x7f17fc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17fc05ed90 0x7f17fc061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f18100c15f0 tx=0x7f1804017000).stop 2023-05-16T07:19:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 msgr2=0x7f1810150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f1810150640 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f180000c820 tx=0x7f1800001650).stop 2023-05-16T07:19:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 shutdown_connections 2023-05-16T07:19:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f17fc05ed90 0x7f17fc061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1810153f90 0x7f18100a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18100c1160 0x7f1810150640 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 --2- 172.21.15.92:0/1240643116 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18100c61f0 0x7f18100a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 >> 172.21.15.92:0/1240643116 conn(0x7f18100cb600 msgr2=0x7f18100a9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 shutdown_connections 2023-05-16T07:19:56.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.107+0000 7f1815c5b700 1 -- 172.21.15.92:0/1240643116 wait complete. 2023-05-16T07:19:56.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:19:56.243 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:56.239+0000 7f8b8810d680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:56.254 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:56.249+0000 7fedb258d680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:19:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.281+0000 7f264995d700 1 Processor -- start 2023-05-16T07:19:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f264995d700 1 -- start start 2023-05-16T07:19:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f264995d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2644063fd0 0x7f26441540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f264995d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f2644062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f264995d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26440bf950 con 0x7f2644153d00 2023-05-16T07:19:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f264995d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2644067f60 con 0x7f2644063fd0 2023-05-16T07:19:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f264995d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2644062c30 con 0x7f26440622b0 2023-05-16T07:19:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f26427fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2644063fd0 0x7f26441540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.282+0000 7f2642ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2644153d00 0x7f26440bf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32830/0 (socket says 172.21.15.92:32830) 2023-05-16T07:19:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f2642ffd700 1 -- 172.21.15.92:0/2931206661 learned_addr learned my addr 172.21.15.92:0/2931206661 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f26437fe700 1 --2- 172.21.15.92:0/2931206661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f2644062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f26427fc700 1 -- 172.21.15.92:0/2931206661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 msgr2=0x7f2644062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:56.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f26427fc700 1 --2- 172.21.15.92:0/2931206661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f2644062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f26427fc700 1 -- 172.21.15.92:0/2931206661 >> v1:172.21.15.92:6789/0 conn(0x7f2644153d00 legacy=0x7f26440bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f2641ffb700 1 -- 172.21.15.92:0/2931206661 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058757313 0 0) 0x7f26440bf950 con 0x7f2644153d00 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f26427fc700 1 -- 172.21.15.92:0/2931206661 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440a7c90 con 0x7f2644063fd0 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f26427fc700 1 --2- 172.21.15.92:0/2931206661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644063fd0 0x7f26441540e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f2634009760 tx=0x7f2634009060).ready entity=mon.0 client_cookie=a6308b5861590718 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f2641ffb700 1 -- 172.21.15.92:0/2931206661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400b580 con 0x7f2644063fd0 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f2641ffb700 1 -- 172.21.15.92:0/2931206661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263400b6e0 con 0x7f2644063fd0 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.283+0000 7f2641ffb700 1 -- 172.21.15.92:0/2931206661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400ba60 con 0x7f2644063fd0 2023-05-16T07:19:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 -- 172.21.15.92:0/2931206661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644063fd0 msgr2=0x7f26441540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 --2- 172.21.15.92:0/2931206661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644063fd0 0x7f26441540e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f2634009760 tx=0x7f2634009060).stop 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 -- 172.21.15.92:0/2931206661 shutdown_connections 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 --2- 172.21.15.92:0/2931206661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f2644062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 --2- 172.21.15.92:0/2931206661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644063fd0 0x7f26441540e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 -- 172.21.15.92:0/2931206661 >> 172.21.15.92:0/2931206661 conn(0x7f264405e820 msgr2=0x7f26440c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 -- 172.21.15.92:0/2931206661 shutdown_connections 2023-05-16T07:19:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.284+0000 7f264995d700 1 -- 172.21.15.92:0/2931206661 wait complete. 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.285+0000 7f264995d700 1 Processor -- start 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.285+0000 7f264995d700 1 -- start start 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f264414f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2644063fd0 0x7f2644152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 0x7f26440c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2644165140 con 0x7f2644153d00 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440c0950 con 0x7f2644063fd0 2023-05-16T07:19:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2644164af0 con 0x7f26440622b0 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 0x7f26440c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 0x7f26440c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50324/0 (socket says 172.21.15.92:50324) 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 -- 172.21.15.92:0/4137466615 learned_addr learned my addr 172.21.15.92:0/4137466615 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f2642ffd700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f264414f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26427fc700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2644063fd0 0x7f2644152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 -- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 msgr2=0x7f264414f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f264414f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 -- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2644063fd0 msgr2=0x7f2644152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2644063fd0 0x7f2644152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 -- 172.21.15.92:0/4137466615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340092a0 con 0x7f2644153d00 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f26437fe700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 0x7f26440c4d50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f263c009fa0 tx=0x7f263c009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c00ebc0 con 0x7f2644153d00 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c00ed20 con 0x7f2644153d00 2023-05-16T07:19:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2644164dd0 con 0x7f2644153d00 2023-05-16T07:19:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c014610 con 0x7f2644153d00 2023-05-16T07:19:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.286+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2644153460 con 0x7f2644153d00 2023-05-16T07:19:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.287+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2644153650 con 0x7f2644153d00 2023-05-16T07:19:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.288+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 56) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f263c01b030 con 0x7f2644153d00 2023-05-16T07:19:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.288+0000 7f262affd700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f263005ede0 0x7f26300612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.288+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f263c07a090 con 0x7f2644153d00 2023-05-16T07:19:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.288+0000 7f2642ffd700 1 -- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f263005ede0 msgr2=0x7f26300612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.288+0000 7f2642ffd700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f263005ede0 0x7f26300612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.292+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f263c04b4a0 con 0x7f2644153d00 2023-05-16T07:19:56.304 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:56.301+0000 7f8b8810d680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:56.320 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:56.315+0000 7fedb258d680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:19:56.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.462+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f26440bd9f0 con 0x7f2644153d00 2023-05-16T07:19:56.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.467+0000 7f262affd700 1 -- 172.21.15.92:0/4137466615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7f263c023030 con 0x7f2644153d00 2023-05-16T07:19:56.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.469+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f263005ede0 msgr2=0x7f26300612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:19:56.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.469+0000 7f264995d700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f263005ede0 0x7f26300612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 msgr2=0x7f26440c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 0x7f26440c4d50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f263c009fa0 tx=0x7f263c009900).stop 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 shutdown_connections 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f263005ede0 0x7f26300612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f26440622b0 0x7f264414f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2644153d00 0x7f26440c4d50 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 --2- 172.21.15.92:0/4137466615 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2644063fd0 0x7f2644152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 >> 172.21.15.92:0/4137466615 conn(0x7f264405e820 msgr2=0x7f26440a70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:56.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 shutdown_connections 2023-05-16T07:19:56.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.470+0000 7f264995d700 1 -- 172.21.15.92:0/4137466615 wait complete. 2023-05-16T07:19:56.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:56.488 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:19:56.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:19:56.507 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-05-16T07:19:56.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-05-16T07:19:56.574 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:56.571+0000 7f8b8810d680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:56.602 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:56.597+0000 7fedb258d680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:19:56.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.720+0000 7f9b60abe700 1 Processor -- start 2023-05-16T07:19:56.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.720+0000 7f9b60abe700 1 -- start start 2023-05-16T07:19:56.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.720+0000 7f9b60abe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b5c153bb0 0x7f9b5c0698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b60abe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9b5c0bdf20 0x7f9b5c067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b60abe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c05e760 con 0x7f9b5c153f90 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b60abe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c069ef0 con 0x7f9b5c153bb0 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b60abe700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c0bf540 con 0x7f9b5c0bdf20 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b5c153bb0 0x7f9b5c0698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9b5c153f90 0x7f9b5c0bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32890/0 (socket says 172.21.15.92:32890) 2023-05-16T07:19:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5affd700 1 -- 172.21.15.92:0/3487235929 learned_addr learned my addr 172.21.15.92:0/3487235929 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/3487235929 >> v2:172.21.15.158:3300/0 conn(0x7f9b5c0bdf20 0x7f9b5c067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/3487235929 >> v2:172.21.15.158:3300/0 conn(0x7f9b5c0bdf20 msgr2=0x7f9b5c067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/3487235929 >> v2:172.21.15.158:3300/0 conn(0x7f9b5c0bdf20 0x7f9b5c067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/3487235929 >> v1:172.21.15.92:6789/0 conn(0x7f9b5c153f90 legacy=0x7f9b5c0bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/3487235929 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c068450 con 0x7f9b5c153bb0 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5a7fc700 1 -- 172.21.15.92:0/3487235929 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718390474 0 0) 0x7f9b5c05e760 con 0x7f9b5c153f90 2023-05-16T07:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/3487235929 >> v2:172.21.15.158:3300/0 conn(0x7f9b5c0bdf20 0x7f9b5c067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/3487235929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153bb0 0x7f9b5c0698c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f9b5c158a20 tx=0x7f9b4c009060).ready entity=mon.0 client_cookie=54769afa1b981780 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5a7fc700 1 -- 172.21.15.92:0/3487235929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00b580 con 0x7f9b5c153bb0 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5a7fc700 1 -- 172.21.15.92:0/3487235929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c00b6e0 con 0x7f9b5c153bb0 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.721+0000 7f9b5a7fc700 1 -- 172.21.15.92:0/3487235929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00ba60 con 0x7f9b5c153bb0 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 -- 172.21.15.92:0/3487235929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153bb0 msgr2=0x7f9b5c0698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 --2- 172.21.15.92:0/3487235929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153bb0 0x7f9b5c0698c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f9b5c158a20 tx=0x7f9b4c009060).stop 2023-05-16T07:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 -- 172.21.15.92:0/3487235929 shutdown_connections 2023-05-16T07:19:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 --2- 172.21.15.92:0/3487235929 >> v2:172.21.15.158:3300/0 conn(0x7f9b5c0bdf20 0x7f9b5c067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 --2- 172.21.15.92:0/3487235929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153bb0 0x7f9b5c0698c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 -- 172.21.15.92:0/3487235929 >> 172.21.15.92:0/3487235929 conn(0x7f9b5c062d80 msgr2=0x7f9b5c05dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:19:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 -- 172.21.15.92:0/3487235929 shutdown_connections 2023-05-16T07:19:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.722+0000 7f9b60abe700 1 -- 172.21.15.92:0/3487235929 wait complete. 2023-05-16T07:19:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.723+0000 7f9b60abe700 1 Processor -- start 2023-05-16T07:19:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.723+0000 7f9b60abe700 1 -- start start 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b60abe700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b5c0bdf20 0x7f9b5c0adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b60abe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b5c153bb0 0x7f9b5c167420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b60abe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 0x7f9b5c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b60abe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c167910 con 0x7f9b5c153f90 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b60abe700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0bfb80 con 0x7f9b5c0bdf20 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b60abe700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1683a0 con 0x7f9b5c153bb0 2023-05-16T07:19:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 0x7f9b5c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 0x7f9b5c152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50400/0 (socket says 172.21.15.92:50400) 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1513079705 learned_addr learned my addr 172.21.15.92:0/1513079705 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b5c0bdf20 0x7f9b5c0adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.724+0000 7f9b5affd700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b5c153bb0 0x7f9b5c167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b5c153bb0 msgr2=0x7f9b5c167420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b5c153bb0 0x7f9b5c167420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b5c0bdf20 msgr2=0x7f9b5c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:19:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b5c0bdf20 0x7f9b5c0adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b5bfff700 1 -- 172.21.15.92:0/1513079705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0092a0 con 0x7f9b5c153f90 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b5bfff700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 0x7f9b5c152420 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f9b5400d180 tx=0x7f9b54009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400dcc0 con 0x7f9b5c153f90 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5400f040 con 0x7f9b5c153f90 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c167b00 con 0x7f9b5c153f90 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540133f0 con 0x7f9b5c153f90 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.725+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c1680c0 con 0x7f9b5c153f90 2023-05-16T07:19:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.726+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 56) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f9b540137a0 con 0x7f9b5c153f90 2023-05-16T07:19:56.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.726+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c0bf950 con 0x7f9b5c153f90 2023-05-16T07:19:56.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.726+0000 7f9b437fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:19:56.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.727+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:56.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.727+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f9b5407a520 con 0x7f9b5c153f90 2023-05-16T07:19:56.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.727+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:19:56.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.731+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9b5c0bf950 con 0x7f9b5c153f90 2023-05-16T07:19:56.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.923+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b5c0bfcf0 con 0x7f9b4805ede0 2023-05-16T07:19:56.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.928+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:56.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:56.928+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:19:57.188 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:57.184+0000 7f8b8810d680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:57.222 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:57.218+0000 7fedb258d680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:19:57.259 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:57.256+0000 7f8b8810d680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:57.294 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:57.290+0000 7fedb258d680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:19:57.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:57.329+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:57.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:57.329+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:19:57.444 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:57.440+0000 7f8b8810d680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:57.477 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:57.472+0000 7fedb258d680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:19:57.859 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:57.856+0000 7f8b8810d680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:57.888 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:57.884+0000 7fedb258d680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:19:57.960 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:57.957+0000 7f8b8810d680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:57.990 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:57.985+0000 7fedb258d680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:58.113 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:58.109+0000 7f8b8810d680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:58.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:58.130+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:58.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:58.130+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:19:58.140 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:58.135+0000 7fedb258d680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:58.186 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:58.183+0000 7f8b8810d680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:58.214 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:58.209+0000 7fedb258d680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:19:58.252 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:58.248+0000 7f8b8810d680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:58.279 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:58.275+0000 7fedb258d680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:19:58.505 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:58.501+0000 7f8b8810d680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:58.532 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:58.527+0000 7fedb258d680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:19:58.687 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:58.684+0000 7f8b8810d680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:58.714 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:58.710+0000 7fedb258d680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:19:59.327 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.324+0000 7f8b8810d680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:59.351 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.347+0000 7fedb258d680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:19:59.394 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.391+0000 7f8b8810d680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:59.418 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.414+0000 7fedb258d680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:19:59.467 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.464+0000 7f8b8810d680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:59.491 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.487+0000 7fedb258d680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:19:59.610 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.607+0000 7f8b8810d680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:59.635 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.630+0000 7fedb258d680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:19:59.679 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.676+0000 7f8b8810d680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:59.703 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.699+0000 7fedb258d680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:19:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:59.731+0000 7f9b5b7fe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/86301 2023-05-16T07:19:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:19:59.731+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:19:59.792 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.789+0000 7f8b8810d680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:59.816 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.811+0000 7fedb258d680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:19:59.897 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:19:59.894+0000 7f8b8810d680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:19:59.921 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:19:59.918+0000 7fedb258d680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:20:00.186 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:00.183+0000 7f8b8810d680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:20:00.206 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:20:00.203+0000 7fedb258d680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:20:00.255 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:00.252+0000 7f8b8810d680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:20:00.276 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:20:00.272+0000 7fedb258d680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:20:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:01.060+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 57) v1 ==== 75109+0+0 (secure 0 0 0) 0x7f9b540781c0 con 0x7f9b5c153f90 2023-05-16T07:20:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:01.061+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b480612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:20:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:01.061+0000 7f9b437fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:20:01.121 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b038e9700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.122 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b038e9700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.122 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b038e9700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.122 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b038e9700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.122 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b038e9700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.122 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b068ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.122 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b068ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.123 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b068ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.123 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b068ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:01.123 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:20:01.118+0000 7f8b068ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:20:02.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.068+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 58) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f9b54078540 con 0x7f9b5c153f90 2023-05-16T07:20:02.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.069+0000 7f9b437fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b4802fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.069+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b5c0bfcf0 con 0x7f9b4805ede0 2023-05-16T07:20:02.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.070+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b4802fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.071+0000 7f9b5b7fe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b4802fd40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f9b4c009fd0 tx=0x7f9b4c017000).ready entity=mgr.7453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:20:02.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.084+0000 7f9b437fe700 1 -- 172.21.15.92:0/1513079705 <== mgr.7453 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9b5c0bfcf0 con 0x7f9b4805ede0 2023-05-16T07:20:02.087 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-05-16T07:20:02.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 msgr2=0x7f9b4802fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b4802fd40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f9b4c009fd0 tx=0x7f9b4c017000).stop 2023-05-16T07:20:02.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 msgr2=0x7f9b5c152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 0x7f9b5c152420 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f9b5400d180 tx=0x7f9b54009a50).stop 2023-05-16T07:20:02.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 shutdown_connections 2023-05-16T07:20:02.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f9b4805ede0 0x7f9b4802fd40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c153f90 0x7f9b5c152420 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b5c0bdf20 0x7f9b5c0adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 --2- 172.21.15.92:0/1513079705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b5c153bb0 0x7f9b5c167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 >> 172.21.15.92:0/1513079705 conn(0x7f9b5c062d80 msgr2=0x7f9b5c0bd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:20:02.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 shutdown_connections 2023-05-16T07:20:02.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.086+0000 7f9b60abe700 1 -- 172.21.15.92:0/1513079705 wait complete. 2023-05-16T07:20:02.104 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-05-16T07:20:02.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-05-16T07:20:02.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.222+0000 7fb7103da700 1 Processor -- start 2023-05-16T07:20:02.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.222+0000 7fb7103da700 1 -- start start 2023-05-16T07:20:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb7103da700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb708154090 0x7fb7080aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb7103da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb708154470 0x7fb7080a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb7103da700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080c94d0 con 0x7fb7080aa150 2023-05-16T07:20:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb7103da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080aa5a0 con 0x7fb708154470 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb7103da700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080cb7e0 con 0x7fb708154090 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb70d975700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb708154470 0x7fb7080a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb70d975700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb708154470 0x7fb7080a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50510/0 (socket says 172.21.15.92:50510) 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.223+0000 7fb70d975700 1 -- 172.21.15.92:0/1930127328 learned_addr learned my addr 172.21.15.92:0/1930127328 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70e176700 1 --2- 172.21.15.92:0/1930127328 >> v2:172.21.15.158:3300/0 conn(0x7fb708154090 0x7fb7080aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d975700 1 -- 172.21.15.92:0/1930127328 >> v2:172.21.15.158:3300/0 conn(0x7fb708154090 msgr2=0x7fb7080aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:20:02.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d975700 1 --2- 172.21.15.92:0/1930127328 >> v2:172.21.15.158:3300/0 conn(0x7fb708154090 0x7fb7080aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d975700 1 -- 172.21.15.92:0/1930127328 >> v1:172.21.15.92:6789/0 conn(0x7fb7080aa150 legacy=0x7fb708068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d174700 1 -- 172.21.15.92:0/1930127328 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418890761 0 0) 0x7fb7080c94d0 con 0x7fb7080aa150 2023-05-16T07:20:02.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d975700 1 -- 172.21.15.92:0/1930127328 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7080c88c0 con 0x7fb708154470 2023-05-16T07:20:02.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70e176700 1 --2- 172.21.15.92:0/1930127328 >> v2:172.21.15.158:3300/0 conn(0x7fb708154090 0x7fb7080aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:20:02.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d975700 1 --2- 172.21.15.92:0/1930127328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7080a9c60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fb70806bd10 tx=0x7fb6fc0090f0).ready entity=mon.0 client_cookie=9cf91bf75954289a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:20:02.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d174700 1 -- 172.21.15.92:0/1930127328 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00b600 con 0x7fb708154470 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d174700 1 -- 172.21.15.92:0/1930127328 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00b760 con 0x7fb708154470 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb70d174700 1 -- 172.21.15.92:0/1930127328 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00bae0 con 0x7fb708154470 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 -- 172.21.15.92:0/1930127328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 msgr2=0x7fb7080a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 --2- 172.21.15.92:0/1930127328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7080a9c60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fb70806bd10 tx=0x7fb6fc0090f0).stop 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 -- 172.21.15.92:0/1930127328 shutdown_connections 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 --2- 172.21.15.92:0/1930127328 >> v2:172.21.15.158:3300/0 conn(0x7fb708154090 0x7fb7080aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 --2- 172.21.15.92:0/1930127328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7080a9c60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 -- 172.21.15.92:0/1930127328 >> 172.21.15.92:0/1930127328 conn(0x7fb708067920 msgr2=0x7fb7080cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:20:02.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 -- 172.21.15.92:0/1930127328 shutdown_connections 2023-05-16T07:20:02.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.224+0000 7fb7103da700 1 -- 172.21.15.92:0/1930127328 wait complete. 2023-05-16T07:20:02.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.225+0000 7fb7103da700 1 Processor -- start 2023-05-16T07:20:02.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 -- start start 2023-05-16T07:20:02.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7080aa150 0x7fb7080b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb708154090 0x7fb708167c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7081416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080adc80 con 0x7fb708154470 2023-05-16T07:20:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080adf60 con 0x7fb7080aa150 2023-05-16T07:20:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb7103da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080ae240 con 0x7fb708154090 2023-05-16T07:20:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb70e977700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7081416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb70e977700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7081416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50518/0 (socket says 172.21.15.92:50518) 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.226+0000 7fb70e977700 1 -- 172.21.15.92:0/2613055417 learned_addr learned my addr 172.21.15.92:0/2613055417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e176700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7080aa150 0x7fb7080b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70d975700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb708154090 0x7fb708167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e977700 1 -- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb708154090 msgr2=0x7fb708167c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e977700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb708154090 0x7fb708167c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e977700 1 -- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7080aa150 msgr2=0x7fb7080b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e977700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7080aa150 0x7fb7080b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e977700 1 -- 172.21.15.92:0/2613055417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc009370 con 0x7fb708154470 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e176700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7080aa150 0x7fb7080b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb70e977700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7081416e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7fb70000c490 tx=0x7fb700007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f470 con 0x7fb708154470 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70000f5d0 con 0x7fb708154470 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7080ae590 con 0x7fb708154470 2023-05-16T07:20:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f970 con 0x7fb708154470 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.227+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7080b0a50 con 0x7fb708154470 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.228+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb708066730 con 0x7fb708154470 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.228+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 58) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fb700019030 con 0x7fb708154470 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.228+0000 7fb7067fc700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6f405ed70 0x7fb6f4061230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.229+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fb700079250 con 0x7fb708154470 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.229+0000 7fb70e176700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6f405ed70 0x7fb6f4061230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:20:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.229+0000 7fb70e176700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6f405ed70 0x7fb6f4061230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb6f80076e0 tx=0x7fb6f80098f0).ready entity=mgr.7453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:20:02.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.233+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb70004a580 con 0x7fb708154470 2023-05-16T07:20:02.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.423+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb70806c350 con 0x7fb6f405ed70 2023-05-16T07:20:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.428+0000 7fb7067fc700 1 -- 172.21.15.92:0/2613055417 <== mgr.7453 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7fb70806c350 con 0x7fb6f405ed70 2023-05-16T07:20:02.433 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-05-16T07:20:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6f405ed70 msgr2=0x7fb6f4061230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6f405ed70 0x7fb6f4061230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb6f80076e0 tx=0x7fb6f80098f0).stop 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 msgr2=0x7fb7081416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7081416e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7fb70000c490 tx=0x7fb700007bf0).stop 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 shutdown_connections 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fb6f405ed70 0x7fb6f4061230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb708154470 0x7fb7081416e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb7080aa150 0x7fb7080b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 --2- 172.21.15.92:0/2613055417 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb708154090 0x7fb708167c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:20:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 >> 172.21.15.92:0/2613055417 conn(0x7fb708067920 msgr2=0x7fb7080c1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:20:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 shutdown_connections 2023-05-16T07:20:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:20:02.430+0000 7fb7103da700 1 -- 172.21.15.92:0/2613055417 wait complete. 2023-05-16T07:20:02.445 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-05-16T07:20:07.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:20:07.479 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:20:37.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:20:37.543 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:21:00.436 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:21:00.432+0000 7f8af60ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:21:00.436 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:21:00.432+0000 7f8af60ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:21:00.436 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:21:00.432+0000 7f8af60ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:21:00.437 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:21:00.432+0000 7f8af60ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:21:00.437 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:21:00.432+0000 7f8af60ce700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:21:07.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:21:07.624 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:21:37.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:21:37.685 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:22:07.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:22:07.775 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:22:37.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:22:37.850 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:23:07.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:23:07.923 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:23:38.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:23:38.019 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:24:08.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:24:08.101 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:24:38.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:24:38.170 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:25:08.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:25:08.243 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:25:38.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:25:38.305 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:26:08.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:26:08.376 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:26:38.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:26:38.441 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:27:08.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:27:08.514 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:27:38.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:27:38.593 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:28:08.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:28:08.659 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:28:38.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:28:38.747 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:29:08.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:29:08.824 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:29:38.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:29:38.879 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:30:08.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:30:08.981 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:30:39.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:30:39.047 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:31:09.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:31:09.124 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:31:15.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:31:15.447 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-05-16T07:31:15.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_21_00_UTC 2023-05-16T07:31:15.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_22_00_UTC 2023-05-16T07:31:15.466 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_23_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_24_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_25_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_26_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_27_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_28_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_29_00_UTC 2023-05-16T07:31:15.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_30_00_UTC 2023-05-16T07:31:15.468 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_31_00_UTC 2023-05-16T07:31:15.468 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_21_00_UTC', 'scheduled-2023-05-16-07_22_00_UTC', 'scheduled-2023-05-16-07_23_00_UTC', 'scheduled-2023-05-16-07_24_00_UTC', 'scheduled-2023-05-16-07_25_00_UTC', 'scheduled-2023-05-16-07_26_00_UTC', 'scheduled-2023-05-16-07_27_00_UTC', 'scheduled-2023-05-16-07_28_00_UTC', 'scheduled-2023-05-16-07_29_00_UTC', 'scheduled-2023-05-16-07_30_00_UTC', 'scheduled-2023-05-16-07_31_00_UTC'] 2023-05-16T07:31:15.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-05-16T07:31:15.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.629+0000 7f8143fff700 1 Processor -- start 2023-05-16T07:31:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.629+0000 7f8143fff700 1 -- start start 2023-05-16T07:31:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f8143fff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8144063e50 0x7f8144064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f8143fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f8143fff700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8144068f90 con 0x7f8144069420 2023-05-16T07:31:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f8143fff700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8144065f50 con 0x7f8144063e50 2023-05-16T07:31:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f8143fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8144152e90 con 0x7f8144152990 2023-05-16T07:31:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f8142ffd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8144063e50 0x7f8144064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f81437fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f81427fc700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f8144069420 0x7f8144152280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:36642/0 (socket says 172.21.15.92:36642) 2023-05-16T07:31:15.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.630+0000 7f81427fc700 1 -- 172.21.15.92:0/343464017 learned_addr learned my addr 172.21.15.92:0/343464017 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:31:15.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8142ffd700 1 -- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 msgr2=0x7f81440c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:31:15.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8142ffd700 1 --2- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8142ffd700 1 -- 172.21.15.92:0/343464017 >> v1:172.21.15.158:6789/0 conn(0x7f8144069420 legacy=0x7f8144152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8141ffb700 1 -- 172.21.15.92:0/343464017 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694696406 0 0) 0x7f8144068f90 con 0x7f8144069420 2023-05-16T07:31:15.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8142ffd700 1 -- 172.21.15.92:0/343464017 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144153170 con 0x7f8144063e50 2023-05-16T07:31:15.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f81437fe700 1 --2- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:31:15.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8142ffd700 1 --2- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144064230 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8134009e60 tx=0x7f8134009780).ready entity=mon.1 client_cookie=fd471e499b127554 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:31:15.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8141ffb700 1 -- 172.21.15.92:0/343464017 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400e070 con 0x7f8144063e50 2023-05-16T07:31:15.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8141ffb700 1 -- 172.21.15.92:0/343464017 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813400bb60 con 0x7f8144063e50 2023-05-16T07:31:15.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.631+0000 7f8141ffb700 1 -- 172.21.15.92:0/343464017 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134012410 con 0x7f8144063e50 2023-05-16T07:31:15.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 -- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 msgr2=0x7f8144064230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 --2- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144064230 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8134009e60 tx=0x7f8134009780).stop 2023-05-16T07:31:15.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 -- 172.21.15.92:0/343464017 shutdown_connections 2023-05-16T07:31:15.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 --2- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144064230 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 --2- 172.21.15.92:0/343464017 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 -- 172.21.15.92:0/343464017 >> 172.21.15.92:0/343464017 conn(0x7f8144067920 msgr2=0x7f81440c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:31:15.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 -- 172.21.15.92:0/343464017 shutdown_connections 2023-05-16T07:31:15.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.632+0000 7f8143fff700 1 -- 172.21.15.92:0/343464017 wait complete. 2023-05-16T07:31:15.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.633+0000 7f8143fff700 1 Processor -- start 2023-05-16T07:31:15.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.633+0000 7f8143fff700 1 -- start start 2023-05-16T07:31:15.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8143fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8143fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 0x7f81440ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8143fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8143fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81440ac230 con 0x7f8144069420 2023-05-16T07:31:15.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8143fff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8144144020 con 0x7f8144063e50 2023-05-16T07:31:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8143fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81440a6990 con 0x7f8144152990 2023-05-16T07:31:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 0x7f81440ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 0x7f81440ab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50528/0 (socket says 172.21.15.92:50528) 2023-05-16T07:31:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 -- 172.21.15.92:0/3541653365 learned_addr learned my addr 172.21.15.92:0/3541653365 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:31:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81437fe700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8142ffd700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 -- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 msgr2=0x7f81440abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 -- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 msgr2=0x7f8144143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f81427fc700 1 -- 172.21.15.92:0/3541653365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0092a0 con 0x7f8144069420 2023-05-16T07:31:15.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.634+0000 7f8142ffd700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:31:15.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.635+0000 7f81427fc700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 0x7f81440ab850 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f812c00c4e0 tx=0x7f812c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:31:15.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.635+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c0072e0 con 0x7f8144069420 2023-05-16T07:31:15.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.635+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c007440 con 0x7f8144069420 2023-05-16T07:31:15.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.635+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8134009a20 con 0x7f8144069420 2023-05-16T07:31:15.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.635+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c0057a0 con 0x7f8144069420 2023-05-16T07:31:15.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.635+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81440a6340 con 0x7f8144069420 2023-05-16T07:31:15.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.636+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81440a6b20 con 0x7f8144069420 2023-05-16T07:31:15.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.636+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 61) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f812c01a060 con 0x7f8144069420 2023-05-16T07:31:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.637+0000 7f813affd700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f812405ee70 0x7f8124061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.637+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6963+0+0 (secure 0 0 0) 0x7f812c079820 con 0x7f8144069420 2023-05-16T07:31:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.637+0000 7f8142ffd700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f812405ee70 0x7f8124061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.637+0000 7f8142ffd700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f812405ee70 0x7f8124061330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f8134009e60 tx=0x7f8134009780).ready entity=mgr.7453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:31:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.641+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f812c04aa50 con 0x7f8144069420 2023-05-16T07:31:15.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.828+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 --> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8144047150 con 0x7f812405ee70 2023-05-16T07:31:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.831+0000 7f813affd700 1 -- 172.21.15.92:0/3541653365 <== mgr.7453 v2:172.21.15.158:6832/86301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f8144047150 con 0x7f812405ee70 2023-05-16T07:31:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:31:15.834 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:20:02", "first": "2023-05-16T07:21:00", "last": "2023-05-16T07:31:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-05-16T07:31:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f812405ee70 msgr2=0x7f8124061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f812405ee70 0x7f8124061330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f8134009e60 tx=0x7f8134009780).stop 2023-05-16T07:31:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 msgr2=0x7f81440ab850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 0x7f81440ab850 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f812c00c4e0 tx=0x7f812c007c70).stop 2023-05-16T07:31:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 shutdown_connections 2023-05-16T07:31:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7f812405ee70 0x7f8124061330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8144152990 0x7f81440abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8144063e50 0x7f8144143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 --2- 172.21.15.92:0/3541653365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8144069420 0x7f81440ab850 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 >> 172.21.15.92:0/3541653365 conn(0x7f8144067920 msgr2=0x7f814406a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:31:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 shutdown_connections 2023-05-16T07:31:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.832+0000 7f8143fff700 1 -- 172.21.15.92:0/3541653365 wait complete. 2023-05-16T07:31:15.847 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:20:02", "first": "2023-05-16T07:21:00", "last": "2023-05-16T07:31:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-05-16T07:31:15.847 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:20:02', 'first': '2023-05-16T07:21:00', 'last': '2023-05-16T07:31:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-05-16T07:31:15.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-05-16T07:31:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.954+0000 7fc6e219a700 1 Processor -- start 2023-05-16T07:31:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.954+0000 7fc6e219a700 1 -- start start 2023-05-16T07:31:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6e219a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 0x7fc6dc0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6e219a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc0c0660 con 0x7fc6dc069b90 2023-05-16T07:31:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6e219a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc06a020 con 0x7fc6dc154170 2023-05-16T07:31:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6e219a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0bfd30 con 0x7fc6dc154550 2023-05-16T07:31:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6dbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc6dc069b90 0x7fc6dc0c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33054/0 (socket says 172.21.15.92:33054) 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6dbfff700 1 -- 172.21.15.92:0/1496914785 learned_addr learned my addr 172.21.15.92:0/1496914785 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6daffd700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 0x7fc6dc0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425354966 0 0) 0x7fc6dc0c0660 con 0x7fc6dc069b90 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c40033f0 con 0x7fc6dc069b90 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624742487 0 0) 0x7fc6c40033f0 con 0x7fc6dc069b90 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 msgr2=0x7fc6dc0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:31:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.955+0000 7fc6da7fc700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 0x7fc6dc0696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 >> v1:172.21.15.158:6789/0 conn(0x7fc6dc154170 legacy=0x7fc6dc0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0c0010 con 0x7fc6dc069b90 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fc6d4001470 con 0x7fc6dc069b90 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 >> v1:172.21.15.92:6789/0 conn(0x7fc6dc069b90 legacy=0x7fc6dc0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 0x7fc6c4003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6c4003e00 0x7fc6c4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c4004b80 0x7fc6c4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0bfd30 con 0x7fc6c4004b80 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400b990 con 0x7fc6c4003e00 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400bb50 con 0x7fc6dc154550 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c4004b80 0x7fc6c4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6db7fe700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 0x7fc6c4003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6daffd700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6c4003e00 0x7fc6c4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 -- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 msgr2=0x7fc6c4003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:31:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc154550 0x7fc6c4003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 -- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6c4003e00 msgr2=0x7fc6c4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6c4003e00 0x7fc6c4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 -- 172.21.15.92:0/1496914785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c40033f0 con 0x7fc6c4004b80 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6dbfff700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c4004b80 0x7fc6c4005000 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7fc6d400c190 tx=0x7fc6d4001e70).ready entity=mon.0 client_cookie=45b2f624b9b12224 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4007a40 con 0x7fc6c4004b80 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d400cac0 con 0x7fc6c4004b80 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.956+0000 7fc6da7fc700 1 -- 172.21.15.92:0/1496914785 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400ce40 con 0x7fc6c4004b80 2023-05-16T07:31:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6e219a700 1 -- 172.21.15.92:0/1496914785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c4004b80 msgr2=0x7fc6c4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1496914785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c4004b80 0x7fc6c4005000 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7fc6d400c190 tx=0x7fc6d4001e70).stop 2023-05-16T07:31:15.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6dbfff700 1 -- 172.21.15.92:0/1496914785 reap_dead start 2023-05-16T07:31:15.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6e219a700 1 -- 172.21.15.92:0/1496914785 shutdown_connections 2023-05-16T07:31:15.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6e219a700 1 -- 172.21.15.92:0/1496914785 >> 172.21.15.92:0/1496914785 conn(0x7fc6dc0cb700 msgr2=0x7fc6dc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:31:15.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6e219a700 1 -- 172.21.15.92:0/1496914785 shutdown_connections 2023-05-16T07:31:15.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.957+0000 7fc6e219a700 1 -- 172.21.15.92:0/1496914785 wait complete. 2023-05-16T07:31:15.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.958+0000 7fc6e219a700 1 Processor -- start 2023-05-16T07:31:15.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.958+0000 7fc6e219a700 1 -- start start 2023-05-16T07:31:15.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6e219a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6dc0a9f20 0x7fc6dc0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6e219a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 0x7fc6dc0c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6e219a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc14bfd0 0x7fc6dc1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6e219a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc1492a0 con 0x7fc6dc0aa300 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6e219a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0c27a0 con 0x7fc6dc0a9f20 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6e219a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0c2360 con 0x7fc6dc14bfd0 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 0x7fc6dc0c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 0x7fc6dc0c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50548/0 (socket says 172.21.15.92:50548) 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 -- 172.21.15.92:0/1900097738 learned_addr learned my addr 172.21.15.92:0/1900097738 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6db7fe700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6dc0a9f20 0x7fc6dc0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6dbfff700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc14bfd0 0x7fc6dc1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 -- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc14bfd0 msgr2=0x7fc6dc1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc14bfd0 0x7fc6dc1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 -- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6dc0a9f20 msgr2=0x7fc6dc0c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:15.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6dc0a9f20 0x7fc6dc0c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 -- 172.21.15.92:0/1900097738 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0c1de0 con 0x7fc6dc0aa300 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6db7fe700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6dc0a9f20 0x7fc6dc0c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6dbfff700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc14bfd0 0x7fc6dc1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6daffd700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 0x7fc6dc0c4800 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fc6d0011a20 tx=0x7fc6d0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000f940 con 0x7fc6dc0aa300 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d000faa0 con 0x7fc6dc0aa300 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.959+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000fe20 con 0x7fc6dc0aa300 2023-05-16T07:31:15.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.960+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc0c2080 con 0x7fc6dc0aa300 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.960+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc0c4cf0 con 0x7fc6dc0aa300 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.960+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc0666c0 con 0x7fc6dc0aa300 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.961+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 61) v1 ==== 75339+0+0 (secure 0 0 0) 0x7fc6d0024070 con 0x7fc6dc0aa300 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.961+0000 7fc6c37fe700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6c805ee40 0x7fc6c8061300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.961+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6963+0+0 (secure 0 0 0) 0x7fc6d007aa40 con 0x7fc6dc0aa300 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.961+0000 7fc6db7fe700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6c805ee40 0x7fc6c8061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:31:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.962+0000 7fc6db7fe700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6c805ee40 0x7fc6c8061300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fc6cc000f80 tx=0x7fc6cc007f10).ready entity=mgr.7453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:31:15.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:15.965+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc6d004bc70 con 0x7fc6dc0aa300 2023-05-16T07:31:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.171+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc6dc0c2910 con 0x7fc6dc0aa300 2023-05-16T07:31:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.174+0000 7fc6c37fe700 1 -- 172.21.15.92:0/1900097738 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v61) v1 ==== 74+0+148685 (secure 0 0 0) 0x7fc6d004f290 con 0x7fc6dc0aa300 2023-05-16T07:31:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:31:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6c805ee40 msgr2=0x7fc6c8061300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6c805ee40 0x7fc6c8061300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fc6cc000f80 tx=0x7fc6cc007f10).stop 2023-05-16T07:31:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 msgr2=0x7fc6dc0c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 0x7fc6dc0c4800 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fc6d0011a20 tx=0x7fc6d0009060).stop 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 shutdown_connections 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:6832/86301,v1:172.21.15.158:6833/86301] conn(0x7fc6c805ee40 0x7fc6c8061300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc6dc14bfd0 0x7fc6dc1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6dc0a9f20 0x7fc6dc0c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 --2- 172.21.15.92:0/1900097738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6dc0aa300 0x7fc6dc0c4800 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:31:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 >> 172.21.15.92:0/1900097738 conn(0x7fc6dc0cb700 msgr2=0x7fc6dc0ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:31:16.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 shutdown_connections 2023-05-16T07:31:16.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:31:16.176+0000 7fc6e219a700 1 -- 172.21.15.92:0/1900097738 wait complete. 2023-05-16T07:31:16.191 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"active_gid":7453,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":86301},{"type":"v1","addr":"172.21.15.158:6833","nonce":86301}]},"active_addr":"172.21.15.158:6833/86301","active_change":"2023-05-16T07:20:00.266662+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":8064,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":136,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":117216949}]},{"addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":4145570814}]},{"addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":3660652651}]},{"addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":4009528875}]}]} 2023-05-16T07:31:16.195 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-05-16T07:31:16.195 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-05-16T07:31:16.196 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-05-16T07:31:16.196 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-05-16T07:31:16.196 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:31:16.225 INFO:tasks.ceph.mgr.x:Stopped 2023-05-16T07:31:16.225 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-05-16T07:31:16.267 INFO:tasks.ceph.mgr.x:Started 2023-05-16T07:31:16.448 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:16.446+0000 7f6d0e7fa680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:31:16.510 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:16.507+0000 7f6d0e7fa680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:31:16.779 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:16.776+0000 7f6d0e7fa680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:31:17.381 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:17.378+0000 7f6d0e7fa680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:31:17.542 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:17.449+0000 7f6d0e7fa680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:31:17.637 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:17.634+0000 7f6d0e7fa680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:31:18.049 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.046+0000 7f6d0e7fa680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:31:18.150 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.147+0000 7f6d0e7fa680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:31:18.301 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.298+0000 7f6d0e7fa680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:31:18.374 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.371+0000 7f6d0e7fa680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:31:18.440 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.437+0000 7f6d0e7fa680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:31:18.695 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.692+0000 7f6d0e7fa680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:31:18.888 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:18.885+0000 7f6d0e7fa680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:31:19.531 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:19.528+0000 7f6d0e7fa680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:31:19.598 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:19.595+0000 7f6d0e7fa680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:31:19.670 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:19.668+0000 7f6d0e7fa680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:31:19.813 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:19.810+0000 7f6d0e7fa680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:31:19.882 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:19.879+0000 7f6d0e7fa680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:31:19.995 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:19.992+0000 7f6d0e7fa680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:31:20.101 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.098+0000 7f6d0e7fa680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:31:20.389 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.386+0000 7f6d0e7fa680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:31:20.458 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.455+0000 7f6d0e7fa680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:31:20.592 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c897d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.593 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c897d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.593 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c8c7db700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.593 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c8c7db700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.593 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c8c7db700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.593 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c8c7db700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.594 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c897d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.594 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c8c7db700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.594 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c897d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:20.594 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:31:20.589+0000 7f6c897d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:31:39.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:31:39.195 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:32:00.586 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:32:00.583+0000 7f6c92fe8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:32:00.586 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:32:00.583+0000 7f6c92fe8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:32:00.586 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:32:00.583+0000 7f6c92fe8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:32:00.586 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:32:00.583+0000 7f6c92fe8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:32:00.586 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:32:00.583+0000 7f6c92fe8700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:32:09.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:32:09.253 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:32:39.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:32:39.333 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:33:09.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:33:09.412 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:33:39.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:33:39.477 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:34:09.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:34:09.556 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:34:39.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:34:39.618 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:35:09.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:35:09.714 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:35:39.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:35:39.809 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:36:09.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:36:09.873 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:36:16.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-05-16T07:36:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.395+0000 7f46297dd700 1 Processor -- start 2023-05-16T07:36:16.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.395+0000 7f46297dd700 1 -- start start 2023-05-16T07:36:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46297dd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4624154470 0x7f46240aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46297dd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624068160 0x7f46240685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46297dd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240c94d0 con 0x7f4624154090 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46297dd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240cb7e0 con 0x7f4624154470 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46297dd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4624068aa0 con 0x7f4624068160 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4624154470 0x7f46240aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4624154470 0x7f46240aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50758/0 (socket says 172.21.15.92:50758) 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 -- 172.21.15.92:0/4236073292 learned_addr learned my addr 172.21.15.92:0/4236073292 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46237fe700 1 --2- 172.21.15.92:0/4236073292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624068160 0x7f46240685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 -- 172.21.15.92:0/4236073292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624068160 msgr2=0x7f46240685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 --2- 172.21.15.92:0/4236073292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624068160 0x7f46240685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 -- 172.21.15.92:0/4236073292 >> v1:172.21.15.158:6789/0 conn(0x7f4624154090 legacy=0x7f46240a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 -- 172.21.15.92:0/4236073292 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240c88c0 con 0x7f4624154470 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f46227fc700 1 --2- 172.21.15.92:0/4236073292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154470 0x7f46240aaca0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f460c006fd0 tx=0x7f460c0090f0).ready entity=mon.0 client_cookie=b0d43cefb88aee4d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f4621ffb700 1 -- 172.21.15.92:0/4236073292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00b580 con 0x7f4624154470 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f4621ffb700 1 -- 172.21.15.92:0/4236073292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00b6e0 con 0x7f4624154470 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.396+0000 7f4621ffb700 1 -- 172.21.15.92:0/4236073292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00ba60 con 0x7f4624154470 2023-05-16T07:36:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 -- 172.21.15.92:0/4236073292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154470 msgr2=0x7f46240aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 --2- 172.21.15.92:0/4236073292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154470 0x7f46240aaca0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f460c006fd0 tx=0x7f460c0090f0).stop 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 -- 172.21.15.92:0/4236073292 shutdown_connections 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 --2- 172.21.15.92:0/4236073292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624068160 0x7f46240685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 --2- 172.21.15.92:0/4236073292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154470 0x7f46240aaca0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 -- 172.21.15.92:0/4236073292 >> 172.21.15.92:0/4236073292 conn(0x7f4624067920 msgr2=0x7f46240cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 -- 172.21.15.92:0/4236073292 shutdown_connections 2023-05-16T07:36:16.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.397+0000 7f46297dd700 1 -- 172.21.15.92:0/4236073292 wait complete. 2023-05-16T07:36:16.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.398+0000 7f46297dd700 1 Processor -- start 2023-05-16T07:36:16.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.398+0000 7f46297dd700 1 -- start start 2023-05-16T07:36:16.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46297dd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4624068160 0x7f462414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46297dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 0x7f46240c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46297dd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624154470 0x7f4624156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46297dd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624152760 con 0x7f4624154090 2023-05-16T07:36:16.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46297dd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462414f230 con 0x7f4624068160 2023-05-16T07:36:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46297dd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4624157480 con 0x7f4624154470 2023-05-16T07:36:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 0x7f46240c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 0x7f46240c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50768/0 (socket says 172.21.15.92:50768) 2023-05-16T07:36:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 -- 172.21.15.92:0/3432951244 learned_addr learned my addr 172.21.15.92:0/3432951244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46237fe700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624154470 0x7f4624156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f4622ffd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4624068160 0x7f462414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 -- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624154470 msgr2=0x7f4624156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624154470 0x7f4624156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 -- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4624068160 msgr2=0x7f462414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4624068160 0x7f462414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f46227fc700 1 -- 172.21.15.92:0/3432951244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c009370 con 0x7f4624154090 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.399+0000 7f4622ffd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4624068160 0x7f462414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.400+0000 7f46227fc700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 0x7f46240c1cd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f460c009130 tx=0x7f460c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.400+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00b850 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.400+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00b9b0 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.400+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4624152110 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.400+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c01b660 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.400+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46240c21c0 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.401+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4624066730 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.401+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 66) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f460c02b070 con 0x7f4624154090 2023-05-16T07:36:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.401+0000 7f461affd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f460405ee70 0x7f4604061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.401+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7135+0+0 (secure 0 0 0) 0x7f460c07b8d0 con 0x7f4624154090 2023-05-16T07:36:16.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.402+0000 7f4622ffd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f460405ee70 0x7f4604061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.402+0000 7f4622ffd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f460405ee70 0x7f4604061330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f4624151bb0 tx=0x7f4614009040).ready entity=mgr.7522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:16.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.406+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f460c04ca60 con 0x7f4624154090 2023-05-16T07:36:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.594+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f462406c350 con 0x7f460405ee70 2023-05-16T07:36:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.599+0000 7f461affd700 1 -- 172.21.15.92:0/3432951244 <== mgr.7522 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f462406c350 con 0x7f460405ee70 2023-05-16T07:36:16.603 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-05-16T07:36:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f460405ee70 msgr2=0x7f4604061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f460405ee70 0x7f4604061330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f4624151bb0 tx=0x7f4614009040).stop 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 msgr2=0x7f46240c1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 0x7f46240c1cd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f460c009130 tx=0x7f460c017000).stop 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 shutdown_connections 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f460405ee70 0x7f4604061330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4624154470 0x7f4624156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4624068160 0x7f462414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 --2- 172.21.15.92:0/3432951244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4624154090 0x7f46240c1cd0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 >> 172.21.15.92:0/3432951244 conn(0x7f4624067920 msgr2=0x7f46240abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 shutdown_connections 2023-05-16T07:36:16.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.601+0000 7f46297dd700 1 -- 172.21.15.92:0/3432951244 wait complete. 2023-05-16T07:36:16.616 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-05-16T07:36:16.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:16.617 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-05-16T07:36:16.632 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_21_00_UTC 2023-05-16T07:36:16.632 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_22_00_UTC 2023-05-16T07:36:16.632 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_23_00_UTC 2023-05-16T07:36:16.632 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_24_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_25_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_26_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_27_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_28_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_29_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_30_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_31_00_UTC 2023-05-16T07:36:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_32_00_UTC 2023-05-16T07:36:16.634 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_33_00_UTC 2023-05-16T07:36:16.634 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_34_00_UTC 2023-05-16T07:36:16.634 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_35_00_UTC 2023-05-16T07:36:16.634 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_36_00_UTC 2023-05-16T07:36:16.634 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-05-16-07_21_00_UTC', 'scheduled-2023-05-16-07_22_00_UTC', 'scheduled-2023-05-16-07_23_00_UTC', 'scheduled-2023-05-16-07_24_00_UTC', 'scheduled-2023-05-16-07_25_00_UTC', 'scheduled-2023-05-16-07_26_00_UTC', 'scheduled-2023-05-16-07_27_00_UTC', 'scheduled-2023-05-16-07_28_00_UTC', 'scheduled-2023-05-16-07_29_00_UTC', 'scheduled-2023-05-16-07_30_00_UTC', 'scheduled-2023-05-16-07_31_00_UTC', 'scheduled-2023-05-16-07_32_00_UTC', 'scheduled-2023-05-16-07_33_00_UTC', 'scheduled-2023-05-16-07_34_00_UTC', 'scheduled-2023-05-16-07_35_00_UTC', 'scheduled-2023-05-16-07_36_00_UTC'] 2023-05-16T07:36:16.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-05-16T07:36:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.780+0000 7f8d3a9ca700 1 Processor -- start 2023-05-16T07:36:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.780+0000 7f8d3a9ca700 1 -- start start 2023-05-16T07:36:16.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d3a9ca700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8d34154090 0x7f8d340aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d3a9ca700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 0x7f8d340a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d3a9ca700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d340c94d0 con 0x7f8d340aa150 2023-05-16T07:36:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d3a9ca700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d340aa5c0 con 0x7f8d34154090 2023-05-16T07:36:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d3a9ca700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340cb7e0 con 0x7f8d34154470 2023-05-16T07:36:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d38f67700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d340aa150 0x7f8d34068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33288/0 (socket says 172.21.15.92:33288) 2023-05-16T07:36:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.781+0000 7f8d38f67700 1 -- 172.21.15.92:0/3677993411 learned_addr learned my addr 172.21.15.92:0/3677993411 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d33fff700 1 --2- 172.21.15.92:0/3677993411 >> v2:172.21.15.158:3300/0 conn(0x7f8d34154090 0x7f8d340aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 0x7f8d340a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155821129 0 0) 0x7f8d340c94d0 con 0x7f8d340aa150 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d200033f0 con 0x7f8d340aa150 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900408190 0 0) 0x7f8d200033f0 con 0x7f8d340aa150 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 msgr2=0x7f8d340a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 0x7f8d340a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 >> v2:172.21.15.158:3300/0 conn(0x7f8d34154090 msgr2=0x7f8d340aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 --2- 172.21.15.92:0/3677993411 >> v2:172.21.15.158:3300/0 conn(0x7f8d34154090 0x7f8d340aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340c88c0 con 0x7f8d340aa150 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f8d2c0017f0 con 0x7f8d340aa150 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 >> v1:172.21.15.92:6789/0 conn(0x7f8d340aa150 legacy=0x7f8d34068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d200038e0 0x7f8d20003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d20004690 0x7f8d20004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 0x7f8d20005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340cb7e0 con 0x7f8d20004690 2023-05-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200044b0 con 0x7f8d200038e0 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2000b910 con 0x7f8d34154470 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d20004690 0x7f8d20004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d33fff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d200038e0 0x7f8d20003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d38f67700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 0x7f8d20005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 -- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 msgr2=0x7f8d20005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d34154470 0x7f8d20005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 -- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d200038e0 msgr2=0x7f8d20003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d200038e0 0x7f8d20003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 -- 172.21.15.92:0/3677993411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200033f0 con 0x7f8d20004690 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d33fff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d200038e0 0x7f8d20003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.782+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d20004690 0x7f8d20004b10 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f8d1c009a60 tx=0x7f8d1c009060).ready entity=mon.0 client_cookie=bc6464f154af90be server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c009ea0 con 0x7f8d20004690 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c009430 con 0x7f8d20004690 2023-05-16T07:36:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d337fe700 1 -- 172.21.15.92:0/3677993411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00e870 con 0x7f8d20004690 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/3677993411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d20004690 msgr2=0x7f8d20004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/3677993411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d20004690 0x7f8d20004b10 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f8d1c009a60 tx=0x7f8d1c009060).stop 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d38f67700 1 -- 172.21.15.92:0/3677993411 reap_dead start 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/3677993411 shutdown_connections 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/3677993411 >> 172.21.15.92:0/3677993411 conn(0x7f8d34067920 msgr2=0x7f8d340cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/3677993411 shutdown_connections 2023-05-16T07:36:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.783+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/3677993411 wait complete. 2023-05-16T07:36:16.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.784+0000 7f8d3a9ca700 1 Processor -- start 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.784+0000 7f8d3a9ca700 1 -- start start 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d3a9ca700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d20004690 0x7f8d340b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d3a9ca700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d34168060 0x7f8d34167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d3a9ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 0x7f8d3414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d3a9ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340ae0d0 con 0x7f8d34167930 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d3a9ca700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34167d90 con 0x7f8d34168060 2023-05-16T07:36:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d3a9ca700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340ae320 con 0x7f8d20004690 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 0x7f8d3414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 0x7f8d3414d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50788/0 (socket says 172.21.15.92:50788) 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 -- 172.21.15.92:0/130195529 learned_addr learned my addr 172.21.15.92:0/130195529 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d33fff700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d20004690 0x7f8d340b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d34168060 0x7f8d34167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 -- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d20004690 msgr2=0x7f8d340b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d20004690 0x7f8d340b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 -- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d34168060 msgr2=0x7f8d34167440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d34168060 0x7f8d34167440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 -- 172.21.15.92:0/130195529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340ae600 con 0x7f8d34167930 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d2bfff700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d34168060 0x7f8d34167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.785+0000 7f8d38f67700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 0x7f8d3414d0b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f8d2c001350 tx=0x7f8d2c0014e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.786+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c005e70 con 0x7f8d34167930 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.786+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c007e40 con 0x7f8d34167930 2023-05-16T07:36:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.786+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d340ae8e0 con 0x7f8d34167930 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.786+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00ea80 con 0x7f8d34167930 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.786+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d340aeea0 con 0x7f8d34167930 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.786+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d34066730 con 0x7f8d34167930 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.787+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 66) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f8d2c014430 con 0x7f8d34167930 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.787+0000 7f8d30ff9700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8d1405eea0 0x7f8d14061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.787+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7135+0+0 (secure 0 0 0) 0x7f8d2c079da0 con 0x7f8d34167930 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.787+0000 7f8d33fff700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8d1405eea0 0x7f8d14061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:16.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.788+0000 7f8d33fff700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8d1405eea0 0x7f8d14061360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8d24009d60 tx=0x7f8d24008040).ready entity=mgr.7522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:16.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.791+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8d2c0498f0 con 0x7f8d34167930 2023-05-16T07:36:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.976+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d3406c3f0 con 0x7f8d1405eea0 2023-05-16T07:36:16.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.978+0000 7f8d30ff9700 1 -- 172.21.15.92:0/130195529 <== mgr.7522 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f8d3406c3f0 con 0x7f8d1405eea0 2023-05-16T07:36:16.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:16.982 INFO:teuthology.orchestra.run.smithi092.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:20:02", "first": "2023-05-16T07:21:00", "last": "2023-05-16T07:36:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-05-16T07:36:16.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8d1405eea0 msgr2=0x7f8d14061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8d1405eea0 0x7f8d14061360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8d24009d60 tx=0x7f8d24008040).stop 2023-05-16T07:36:16.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 msgr2=0x7f8d3414d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:16.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 0x7f8d3414d0b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f8d2c001350 tx=0x7f8d2c0014e0).stop 2023-05-16T07:36:16.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 shutdown_connections 2023-05-16T07:36:16.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8d1405eea0 0x7f8d14061360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8d20004690 0x7f8d340b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d34167930 0x7f8d3414d0b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 --2- 172.21.15.92:0/130195529 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d34168060 0x7f8d34167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 >> 172.21.15.92:0/130195529 conn(0x7f8d34067920 msgr2=0x7f8d340c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 shutdown_connections 2023-05-16T07:36:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:16.980+0000 7f8d3a9ca700 1 -- 172.21.15.92:0/130195529 wait complete. 2023-05-16T07:36:16.995 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-05-16T00:00:00", "created": "2023-05-16T07:20:02", "first": "2023-05-16T07:21:00", "last": "2023-05-16T07:36:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-05-16T07:36:16.996 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-05-16T00:00:00', 'created': '2023-05-16T07:20:02', 'first': '2023-05-16T07:21:00', 'last': '2023-05-16T07:36:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-05-16T07:36:16.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-05-16T07:36:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.106+0000 7f4dcffff700 1 Processor -- start 2023-05-16T07:36:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.106+0000 7f4dcffff700 1 -- start start 2023-05-16T07:36:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dcffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4dd00c1160 0x7f4dd00c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd00c61f0 0x7f4dd0154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dcffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00c6640 con 0x7f4dd0153f90 2023-05-16T07:36:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dcffff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd0069f60 con 0x7f4dd00c1160 2023-05-16T07:36:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dcffff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd006a9e0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dce7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4dd00c1160 0x7f4dd00c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dceffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4dd0153f90 0x7f4dd00a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33308/0 (socket says 172.21.15.92:33308) 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dceffd700 1 -- 172.21.15.92:0/3989818474 learned_addr learned my addr 172.21.15.92:0/3989818474 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.107+0000 7f4dcf7fe700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd00c61f0 0x7f4dd0154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dce7fc700 1 -- 172.21.15.92:0/3989818474 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd00c61f0 msgr2=0x7f4dd0154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dce7fc700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd00c61f0 0x7f4dd0154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dce7fc700 1 -- 172.21.15.92:0/3989818474 >> v1:172.21.15.92:6789/0 conn(0x7f4dd0153f90 legacy=0x7f4dd00a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:17.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dce7fc700 1 -- 172.21.15.92:0/3989818474 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd006a140 con 0x7f4dd00c1160 2023-05-16T07:36:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcf7fe700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd00c61f0 0x7f4dd0154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dce7fc700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c1160 0x7f4dd00c5d00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f4db8009760 tx=0x7f4db8009060).ready entity=mon.0 client_cookie=c8285327fe38df5d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcdffb700 1 -- 172.21.15.92:0/3989818474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800b580 con 0x7f4dd00c1160 2023-05-16T07:36:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcdffb700 1 -- 172.21.15.92:0/3989818474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db800b6e0 con 0x7f4dd00c1160 2023-05-16T07:36:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcdffb700 1 -- 172.21.15.92:0/3989818474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800ba60 con 0x7f4dd00c1160 2023-05-16T07:36:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 -- 172.21.15.92:0/3989818474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c1160 msgr2=0x7f4dd00c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c1160 0x7f4dd00c5d00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f4db8009760 tx=0x7f4db8009060).stop 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 -- 172.21.15.92:0/3989818474 shutdown_connections 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd00c61f0 0x7f4dd0154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 --2- 172.21.15.92:0/3989818474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c1160 0x7f4dd00c5d00 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 -- 172.21.15.92:0/3989818474 >> 172.21.15.92:0/3989818474 conn(0x7f4dd00cb600 msgr2=0x7f4dd00c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 -- 172.21.15.92:0/3989818474 shutdown_connections 2023-05-16T07:36:17.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.108+0000 7f4dcffff700 1 -- 172.21.15.92:0/3989818474 wait complete. 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 Processor -- start 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 -- start start 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4dd00c1160 0x7f4dd0141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 0x7f4dd00aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd0153f90 0x7f4dd00af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00af840 con 0x7f4dd00c61f0 2023-05-16T07:36:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00b0650 con 0x7f4dd00c1160 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dcffff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0154670 con 0x7f4dd0153f90 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dce7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 0x7f4dd00aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dce7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 0x7f4dd00aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50800/0 (socket says 172.21.15.92:50800) 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.110+0000 7f4dce7fc700 1 -- 172.21.15.92:0/980928978 learned_addr learned my addr 172.21.15.92:0/980928978 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dcf7fe700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd0153f90 0x7f4dd00af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dceffd700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4dd00c1160 0x7f4dd0141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:17.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dce7fc700 1 -- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd0153f90 msgr2=0x7f4dd00af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dce7fc700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd0153f90 0x7f4dd00af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dce7fc700 1 -- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4dd00c1160 msgr2=0x7f4dd0141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dce7fc700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4dd00c1160 0x7f4dd0141570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dce7fc700 1 -- 172.21.15.92:0/980928978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80092a0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dcf7fe700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd0153f90 0x7f4dd00af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dceffd700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4dd00c1160 0x7f4dd0141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dce7fc700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 0x7f4dd00aee60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f4dd00c1b80 tx=0x7f4db8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800b850 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db800b9b0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00b0ae0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db801b660 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.111+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd00afaf0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.112+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd00c7eb0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.112+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 66) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f4db802b070 con 0x7f4dd00c61f0 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.113+0000 7f4dc6ffd700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4db005ee70 0x7f4db0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:17.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.113+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7135+0+0 (secure 0 0 0) 0x7f4db807bba0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.113+0000 7f4dceffd700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4db005ee70 0x7f4db0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.113+0000 7f4dceffd700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4db005ee70 0x7f4db0061330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f4dc0001da0 tx=0x7f4dc0009040).ready entity=mgr.7522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.117+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4db804cdb0 con 0x7f4dd00c61f0 2023-05-16T07:36:17.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.304+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dd0083120 con 0x7f4db005ee70 2023-05-16T07:36:17.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.307+0000 7f4dc6ffd700 1 -- 172.21.15.92:0/980928978 <== mgr.7522 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f4dd0083120 con 0x7f4db005ee70 2023-05-16T07:36:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-05-16T07:36:17.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4db005ee70 msgr2=0x7f4db0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:17.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4db005ee70 0x7f4db0061330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f4dc0001da0 tx=0x7f4dc0009040).stop 2023-05-16T07:36:17.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 msgr2=0x7f4dd00aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:17.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 0x7f4dd00aee60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f4dd00c1b80 tx=0x7f4db8017000).stop 2023-05-16T07:36:17.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 shutdown_connections 2023-05-16T07:36:17.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4db005ee70 0x7f4db0061330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4dd0153f90 0x7f4dd00af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4dd00c1160 0x7f4dd0141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 --2- 172.21.15.92:0/980928978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd00c61f0 0x7f4dd00aee60 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:17.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 >> 172.21.15.92:0/980928978 conn(0x7f4dd00cb600 msgr2=0x7f4dd0068240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:17.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 shutdown_connections 2023-05-16T07:36:17.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:17.309+0000 7f4dcffff700 1 -- 172.21.15.92:0/980928978 wait complete. 2023-05-16T07:36:17.325 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-05-16T07:36:17.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-05-16T07:36:17.340 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_21_00_UTC 2023-05-16T07:36:17.340 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_22_00_UTC 2023-05-16T07:36:17.340 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_23_00_UTC 2023-05-16T07:36:17.340 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_24_00_UTC 2023-05-16T07:36:17.340 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_25_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_26_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_27_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_28_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_29_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_30_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_31_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_32_00_UTC 2023-05-16T07:36:17.341 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_33_00_UTC 2023-05-16T07:36:17.342 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_34_00_UTC 2023-05-16T07:36:17.342 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_35_00_UTC 2023-05-16T07:36:17.342 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_36_00_UTC 2023-05-16T07:36:17.343 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_21_00_UTC 2023-05-16T07:36:17.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_21_00_UTC) 2023-05-16T07:36:17.402 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_22_00_UTC 2023-05-16T07:36:17.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_22_00_UTC) 2023-05-16T07:36:17.460 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_23_00_UTC 2023-05-16T07:36:17.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.461 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_23_00_UTC) 2023-05-16T07:36:17.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_24_00_UTC 2023-05-16T07:36:17.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.521 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_24_00_UTC) 2023-05-16T07:36:17.580 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_25_00_UTC 2023-05-16T07:36:17.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.581 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_25_00_UTC) 2023-05-16T07:36:17.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_26_00_UTC 2023-05-16T07:36:17.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.642 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_26_00_UTC) 2023-05-16T07:36:17.700 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_27_00_UTC 2023-05-16T07:36:17.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_27_00_UTC) 2023-05-16T07:36:17.759 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_28_00_UTC 2023-05-16T07:36:17.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.760 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_28_00_UTC) 2023-05-16T07:36:17.819 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_29_00_UTC 2023-05-16T07:36:17.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.820 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_29_00_UTC) 2023-05-16T07:36:17.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_30_00_UTC 2023-05-16T07:36:17.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.881 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_30_00_UTC) 2023-05-16T07:36:17.940 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_31_00_UTC 2023-05-16T07:36:17.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:17.941 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_31_00_UTC) 2023-05-16T07:36:17.999 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_32_00_UTC 2023-05-16T07:36:17.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:18.000 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_32_00_UTC) 2023-05-16T07:36:18.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_33_00_UTC 2023-05-16T07:36:18.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:18.063 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_33_00_UTC) 2023-05-16T07:36:18.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_34_00_UTC 2023-05-16T07:36:18.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:18.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_34_00_UTC) 2023-05-16T07:36:18.184 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_35_00_UTC 2023-05-16T07:36:18.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:18.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_35_00_UTC) 2023-05-16T07:36:18.243 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_36_00_UTC 2023-05-16T07:36:18.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:18.244 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-05-16-07_36_00_UTC) 2023-05-16T07:36:18.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:18.303 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-05-16T07:36:18.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:36:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 Processor -- start 2023-05-16T07:36:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 -- start start 2023-05-16T07:36:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bc014f3b0 0x7f8bc014f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:18.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc005e460 0x7f8bc0064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc00c3170 con 0x7f8bc0152bc0 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc005ec20 con 0x7f8bc014f3b0 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.508+0000 7f8bbf59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc005e940 con 0x7f8bc005e460 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bc014f3b0 0x7f8bc014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bc014f3b0 0x7f8bc014f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50806/0 (socket says 172.21.15.92:50806) 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 -- 172.21.15.92:0/2746214897 learned_addr learned my addr 172.21.15.92:0/2746214897 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbdd9b700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc005e460 0x7f8bc0064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 -- 172.21.15.92:0/2746214897 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc005e460 msgr2=0x7f8bc0064150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc005e460 0x7f8bc0064150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 -- 172.21.15.92:0/2746214897 >> v1:172.21.15.158:6789/0 conn(0x7f8bc0152bc0 legacy=0x7f8bc00a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 -- 172.21.15.92:0/2746214897 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc0063670 con 0x7f8bc014f3b0 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbdd9b700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc005e460 0x7f8bc0064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbe59c700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc014f3b0 0x7f8bc014f790 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f8bb0009f20 tx=0x7f8bb0009840).ready entity=mon.0 client_cookie=39e90a18fd8874ed server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:18.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbd59a700 1 -- 172.21.15.92:0/2746214897 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000e070 con 0x7f8bc014f3b0 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbd59a700 1 -- 172.21.15.92:0/2746214897 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb000ba80 con 0x7f8bc014f3b0 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.509+0000 7f8bbd59a700 1 -- 172.21.15.92:0/2746214897 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000be00 con 0x7f8bc014f3b0 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 -- 172.21.15.92:0/2746214897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc014f3b0 msgr2=0x7f8bc014f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc014f3b0 0x7f8bc014f790 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f8bb0009f20 tx=0x7f8bb0009840).stop 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 -- 172.21.15.92:0/2746214897 shutdown_connections 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc014f3b0 0x7f8bc014f790 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/2746214897 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc005e460 0x7f8bc0064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:18.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 -- 172.21.15.92:0/2746214897 >> 172.21.15.92:0/2746214897 conn(0x7f8bc0062d80 msgr2=0x7f8bc005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:18.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 -- 172.21.15.92:0/2746214897 shutdown_connections 2023-05-16T07:36:18.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.510+0000 7f8bbf59e700 1 -- 172.21.15.92:0/2746214897 wait complete. 2023-05-16T07:36:18.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.511+0000 7f8bbf59e700 1 Processor -- start 2023-05-16T07:36:18.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.511+0000 7f8bbf59e700 1 -- start start 2023-05-16T07:36:18.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bc005e460 0x7f8bc0140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:18.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc014f3b0 0x7f8bc014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:18.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 0x7f8bc01555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:18.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0154be0 con 0x7f8bc0152bc0 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0154340 con 0x7f8bc005e460 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0154620 con 0x7f8bc014f3b0 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 0x7f8bc01555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 0x7f8bc01555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50816/0 (socket says 172.21.15.92:50816) 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 -- 172.21.15.92:0/1568533140 learned_addr learned my addr 172.21.15.92:0/1568533140 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbe59c700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bc005e460 0x7f8bc0140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:18.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbdd9b700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc014f3b0 0x7f8bc014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 -- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc014f3b0 msgr2=0x7f8bc014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc014f3b0 0x7f8bc014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 -- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bc005e460 msgr2=0x7f8bc0140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bc005e460 0x7f8bc0140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 -- 172.21.15.92:0/1568533140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0009ae0 con 0x7f8bc0152bc0 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbed9d700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 0x7f8bc01555e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f8bb800c440 tx=0x7f8bb8007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80123f0 con 0x7f8bc0152bc0 2023-05-16T07:36:18.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb8012550 con 0x7f8bc0152bc0 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc0154900 con 0x7f8bc0152bc0 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80128d0 con 0x7f8bc0152bc0 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.512+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc0140740 con 0x7f8bc0152bc0 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.513+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc00abb40 con 0x7f8bc0152bc0 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.514+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 66) v1 ==== 75339+0+0 (secure 0 0 0) 0x7f8bb8010070 con 0x7f8bc0152bc0 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.514+0000 7f8bb67fc700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8ba005ee70 0x7f8ba0061330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:18.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.514+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7135+0+0 (secure 0 0 0) 0x7f8bb80792b0 con 0x7f8bc0152bc0 2023-05-16T07:36:18.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.514+0000 7f8bbe59c700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8ba005ee70 0x7f8ba0061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:18.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.515+0000 7f8bbe59c700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8ba005ee70 0x7f8ba0061330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8bb000ba50 tx=0x7f8bb0009840).ready entity=mgr.7522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:18.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.518+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8bb80478f0 con 0x7f8bc0152bc0 2023-05-16T07:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:18.723+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8bc0047150 con 0x7f8bc0152bc0 2023-05-16T07:36:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.318+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f8bb8076da0 con 0x7f8bc0152bc0 2023-05-16T07:36:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.319+0000 7f8bb67fc700 1 -- 172.21.15.92:0/1568533140 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v67) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8bb80743d0 con 0x7f8bc0152bc0 2023-05-16T07:36:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8ba005ee70 msgr2=0x7f8ba0061330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8ba005ee70 0x7f8ba0061330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8bb000ba50 tx=0x7f8bb0009840).stop 2023-05-16T07:36:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 msgr2=0x7f8bc01555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 0x7f8bc01555e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f8bb800c440 tx=0x7f8bb8007dd0).stop 2023-05-16T07:36:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 shutdown_connections 2023-05-16T07:36:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc0152bc0 0x7f8bc01555e0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8bc005e460 0x7f8bc0140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8ba005ee70 0x7f8ba0061330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 --2- 172.21.15.92:0/1568533140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8bc014f3b0 0x7f8bc014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 >> 172.21.15.92:0/1568533140 conn(0x7f8bc0062d80 msgr2=0x7f8bc00bf210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 shutdown_connections 2023-05-16T07:36:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.321+0000 7f8bbf59e700 1 -- 172.21.15.92:0/1568533140 wait complete. 2023-05-16T07:36:19.338 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:36:19.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:36:19.367 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:36:19.397 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:36:19.398 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:36:19.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:19.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:36:19.464 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:19.459+0000 7f34b9cce680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:36:19.470 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:19.467+0000 7f6544971680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:36:19.525 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:19.521+0000 7f34b9cce680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:36:19.531 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:19.528+0000 7f6544971680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:36:19.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:36:19.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:19.557 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:36:19.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:19.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.722+0000 7f0305e4e700 1 Processor -- start 2023-05-16T07:36:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.722+0000 7f0305e4e700 1 -- start start 2023-05-16T07:36:19.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f0305e4e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c7ac0 0x7f0300153820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:19.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f0305e4e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0300153d10 0x7f03000c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f0305e4e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0300158900 con 0x7f03000c76e0 2023-05-16T07:36:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f0305e4e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0300157cf0 con 0x7f0300153d10 2023-05-16T07:36:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f0305e4e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0300157fd0 con 0x7f03000c7ac0 2023-05-16T07:36:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0300153d10 0x7f03000c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0300153d10 0x7f03000c0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50910/0 (socket says 172.21.15.92:50910) 2023-05-16T07:36:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 -- 172.21.15.92:0/2219456276 learned_addr learned my addr 172.21.15.92:0/2219456276 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f02fffff700 1 --2- 172.21.15.92:0/2219456276 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c7ac0 0x7f0300153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 -- 172.21.15.92:0/2219456276 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c7ac0 msgr2=0x7f0300153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 --2- 172.21.15.92:0/2219456276 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c7ac0 0x7f0300153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 -- 172.21.15.92:0/2219456276 >> v1:172.21.15.158:6789/0 conn(0x7f03000c76e0 legacy=0x7f03000a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 -- 172.21.15.92:0/2219456276 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03001582b0 con 0x7f0300153d10 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f030564d700 1 --2- 172.21.15.92:0/2219456276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f03000c0090 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f02f8009f20 tx=0x7f02f8009840).ready entity=mon.0 client_cookie=46eec978321ff40b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.723+0000 7f02ff7fe700 1 -- 172.21.15.92:0/2219456276 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f800e070 con 0x7f0300153d10 2023-05-16T07:36:19.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f02ff7fe700 1 -- 172.21.15.92:0/2219456276 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f800b9d0 con 0x7f0300153d10 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f02ff7fe700 1 -- 172.21.15.92:0/2219456276 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f800bd50 con 0x7f0300153d10 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 -- 172.21.15.92:0/2219456276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 msgr2=0x7f03000c0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2219456276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f03000c0090 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f02f8009f20 tx=0x7f02f8009840).stop 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 -- 172.21.15.92:0/2219456276 shutdown_connections 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2219456276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f03000c0090 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2219456276 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c7ac0 0x7f0300153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 -- 172.21.15.92:0/2219456276 >> 172.21.15.92:0/2219456276 conn(0x7f0300068330 msgr2=0x7f0300068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:19.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 -- 172.21.15.92:0/2219456276 shutdown_connections 2023-05-16T07:36:19.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.724+0000 7f0305e4e700 1 -- 172.21.15.92:0/2219456276 wait complete. 2023-05-16T07:36:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.725+0000 7f0305e4e700 1 Processor -- start 2023-05-16T07:36:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.725+0000 7f0305e4e700 1 -- start start 2023-05-16T07:36:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0305e4e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c76e0 0x7f0300152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0305e4e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03000c7ac0 0x7f030014d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0305e4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f0300167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0305e4e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c46b0 con 0x7f0300153d10 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0305e4e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300168ab0 con 0x7f03000c7ac0 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0305e4e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0300150c10 con 0x7f03000c76e0 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f030564d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f0300167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f030564d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f0300167bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50916/0 (socket says 172.21.15.92:50916) 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f030564d700 1 -- 172.21.15.92:0/2993481829 learned_addr learned my addr 172.21.15.92:0/2993481829 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f0304e4c700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c76e0 0x7f0300152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f02fffff700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03000c7ac0 0x7f030014d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f030564d700 1 -- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c76e0 msgr2=0x7f0300152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.726+0000 7f030564d700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c76e0 0x7f0300152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f030564d700 1 -- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03000c7ac0 msgr2=0x7f030014d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f030564d700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03000c7ac0 0x7f030014d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f030564d700 1 -- 172.21.15.92:0/2993481829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f8009ae0 con 0x7f0300153d10 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f030564d700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f0300167bc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f0300083bd0 tx=0x7f0300158870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80092e0 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f801e470 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03001532f0 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f801e830 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.727+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03001502e0 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.728+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0300047150 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.728+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f02f8025030 con 0x7f0300153d10 2023-05-16T07:36:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.728+0000 7f02fcff9700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 0x7f02e40612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.729+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7135+0+0 (secure 0 0 0) 0x7f02f807b5a0 con 0x7f0300153d10 2023-05-16T07:36:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.729+0000 7f0304e4c700 1 -- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 msgr2=0x7f02e40612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.729+0000 7f0304e4c700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 0x7f02e40612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:19.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.736+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f02f804c7e0 con 0x7f0300153d10 2023-05-16T07:36:19.798 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:19.795+0000 7f6544971680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:36:19.813 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:19.809+0000 7f34b9cce680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:36:19.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.930+0000 7f0304e4c700 1 -- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 msgr2=0x7f02e40612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:19.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.930+0000 7f0304e4c700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 0x7f02e40612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:36:19.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.946+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03000bfb90 con 0x7f0300153d10 2023-05-16T07:36:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.946+0000 7f02fcff9700 1 -- 172.21.15.92:0/2993481829 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f02f8023070 con 0x7f0300153d10 2023-05-16T07:36:19.951 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:19.951 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:49.065391+0000","modified":"2023-05-16T07:19:51.075477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2186588149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2186588149},{"type":"v1","addr":"172.21.15.92:6835","nonce":2186588149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-05-16T07:36:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 msgr2=0x7f02e40612b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 0x7f02e40612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 msgr2=0x7f0300167bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f0300167bc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f0300083bd0 tx=0x7f0300158870).stop 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 shutdown_connections 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f02e405edf0 0x7f02e40612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03000c76e0 0x7f0300152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0300153d10 0x7f0300167bc0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 --2- 172.21.15.92:0/2993481829 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03000c7ac0 0x7f030014d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 >> 172.21.15.92:0/2993481829 conn(0x7f0300068330 msgr2=0x7f03000c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 shutdown_connections 2023-05-16T07:36:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:19.948+0000 7f0305e4e700 1 -- 172.21.15.92:0/2993481829 wait complete. 2023-05-16T07:36:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 49 2023-05-16T07:36:19.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:20.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.074+0000 7febc1d41700 1 Processor -- start 2023-05-16T07:36:20.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.075+0000 7febc1d41700 1 -- start start 2023-05-16T07:36:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.075+0000 7febc1d41700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7febbc154040 0x7febbc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.075+0000 7febc1d41700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febbc063e30 0x7febbc062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.075+0000 7febc1d41700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc062960 con 0x7febbc0bf4f0 2023-05-16T07:36:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.075+0000 7febc1d41700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc154420 con 0x7febbc063e30 2023-05-16T07:36:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.075+0000 7febc1d41700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc069fb0 con 0x7febbc154040 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febbc063e30 0x7febbc062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febb2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7febbc0bf4f0 0x7febbc063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33472/0 (socket says 172.21.15.92:33472) 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febb2ffd700 1 -- 172.21.15.92:0/3317012381 learned_addr learned my addr 172.21.15.92:0/3317012381 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbb7fe700 1 --2- 172.21.15.92:0/3317012381 >> v2:172.21.15.158:3300/0 conn(0x7febbc154040 0x7febbc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbbfff700 1 -- 172.21.15.92:0/3317012381 >> v2:172.21.15.158:3300/0 conn(0x7febbc154040 msgr2=0x7febbc069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbbfff700 1 --2- 172.21.15.92:0/3317012381 >> v2:172.21.15.158:3300/0 conn(0x7febbc154040 0x7febbc069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbbfff700 1 -- 172.21.15.92:0/3317012381 >> v1:172.21.15.92:6789/0 conn(0x7febbc0bf4f0 legacy=0x7febbc063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbbfff700 1 -- 172.21.15.92:0/3317012381 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc062350 con 0x7febbc063e30 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbaffd700 1 -- 172.21.15.92:0/3317012381 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609384460 0 0) 0x7febbc062960 con 0x7febbc0bf4f0 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbbfff700 1 --2- 172.21.15.92:0/3317012381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc063e30 0x7febbc062df0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7febbc0c89b0 tx=0x7febb4009780).ready entity=mon.0 client_cookie=f814f49fef4066db server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbaffd700 1 -- 172.21.15.92:0/3317012381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb400e070 con 0x7febbc063e30 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbaffd700 1 -- 172.21.15.92:0/3317012381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb400bb60 con 0x7febbc063e30 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.076+0000 7febbaffd700 1 -- 172.21.15.92:0/3317012381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4012410 con 0x7febbc063e30 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 -- 172.21.15.92:0/3317012381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc063e30 msgr2=0x7febbc062df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 --2- 172.21.15.92:0/3317012381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc063e30 0x7febbc062df0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7febbc0c89b0 tx=0x7febb4009780).stop 2023-05-16T07:36:20.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 -- 172.21.15.92:0/3317012381 shutdown_connections 2023-05-16T07:36:20.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 --2- 172.21.15.92:0/3317012381 >> v2:172.21.15.158:3300/0 conn(0x7febbc154040 0x7febbc069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 --2- 172.21.15.92:0/3317012381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc063e30 0x7febbc062df0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 -- 172.21.15.92:0/3317012381 >> 172.21.15.92:0/3317012381 conn(0x7febbc0c8aa0 msgr2=0x7febbc0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:20.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 -- 172.21.15.92:0/3317012381 shutdown_connections 2023-05-16T07:36:20.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.077+0000 7febc1d41700 1 -- 172.21.15.92:0/3317012381 wait complete. 2023-05-16T07:36:20.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.078+0000 7febc1d41700 1 Processor -- start 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.078+0000 7febc1d41700 1 -- start start 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febbc063e30 0x7febbc152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 0x7febbc151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febbc154040 0x7febbc14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc166fb0 con 0x7febbc0bf4f0 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0a85a0 con 0x7febbc063e30 2023-05-16T07:36:20.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc14f630 con 0x7febbc154040 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 0x7febbc151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 0x7febbc151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50962/0 (socket says 172.21.15.92:50962) 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 -- 172.21.15.92:0/4204689324 learned_addr learned my addr 172.21.15.92:0/4204689324 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febbb7fe700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febbc063e30 0x7febbc152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febbbfff700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febbc154040 0x7febbc14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 -- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febbc154040 msgr2=0x7febbc14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febbc154040 0x7febbc14f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 -- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febbc063e30 msgr2=0x7febbc152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febbc063e30 0x7febbc152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 -- 172.21.15.92:0/4204689324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb4009a20 con 0x7febbc0bf4f0 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb2ffd700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 0x7febbc151600 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7feba400c4e0 tx=0x7feba4007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400ce90 con 0x7febbc0bf4f0 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40072e0 con 0x7febbc0bf4f0 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.079+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc152bc0 con 0x7febbc0bf4f0 2023-05-16T07:36:20.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.080+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4007660 con 0x7febbc0bf4f0 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.080+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc0a8b20 con 0x7febbc0bf4f0 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.081+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc06c350 con 0x7febbc0bf4f0 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.081+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7feba4010070 con 0x7febbc0bf4f0 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.081+0000 7febb3fff700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7feb9c05ee20 0x7feb9c0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.081+0000 7febbb7fe700 1 -- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7feb9c05ee20 msgr2=0x7feb9c0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.081+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7135+0+0 (secure 0 0 0) 0x7feba4079530 con 0x7febbc0bf4f0 2023-05-16T07:36:20.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.081+0000 7febbb7fe700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7feb9c05ee20 0x7feb9c0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:20.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.085+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7feba404a6c0 con 0x7febbc0bf4f0 2023-05-16T07:36:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.257+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febbc062ce0 con 0x7febbc0bf4f0 2023-05-16T07:36:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.257+0000 7febb3fff700 1 -- 172.21.15.92:0/4204689324 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+18514 (secure 0 0 0) 0x7feba404dce0 con 0x7febbc0bf4f0 2023-05-16T07:36:20.262 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:20.262 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":139,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:31:20.470670+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:47.601645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:48.023978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3660652651":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/4009528875":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/4145570814":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/3403657457":"2023-05-17T07:20:00.266255+0000","172.21.15.158:0/1546480240":"2023-05-17T07:20:00.266255+0000","172.21.15.158:0/117216949":"2023-05-17T07:31:20.470620+0000","172.21.15.158:6833/86301":"2023-05-17T07:20:00.266255+0000","172.21.15.158:6832/86301":"2023-05-17T07:20:00.266255+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7feb9c05ee20 msgr2=0x7feb9c0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7feb9c05ee20 0x7feb9c0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 msgr2=0x7febbc151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 0x7febbc151600 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7feba400c4e0 tx=0x7feba4007c70).stop 2023-05-16T07:36:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 shutdown_connections 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7feb9c05ee20 0x7feb9c0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febbc154040 0x7febbc14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febbc063e30 0x7febbc152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 --2- 172.21.15.92:0/4204689324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febbc0bf4f0 0x7febbc151600 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.259+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 >> 172.21.15.92:0/4204689324 conn(0x7febbc0c8aa0 msgr2=0x7febbc154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.260+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 shutdown_connections 2023-05-16T07:36:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.260+0000 7febc1d41700 1 -- 172.21.15.92:0/4204689324 wait complete. 2023-05-16T07:36:20.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:19:47.601645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:19:48.023978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:36:20.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:36:20.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.412+0000 7f65d4c59700 1 Processor -- start 2023-05-16T07:36:20.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.412+0000 7f65d4c59700 1 -- start start 2023-05-16T07:36:20.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.413+0000 7f65d4c59700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65d0150bb0 0x7f65d0067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65d4c59700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f65d00a6e90 0x7f65d0068450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65d4c59700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d0064e90 con 0x7f65d0150f90 2023-05-16T07:36:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65d4c59700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d0068940 con 0x7f65d0150bb0 2023-05-16T07:36:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65d4c59700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d0065a10 con 0x7f65d00a6e90 2023-05-16T07:36:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65d0150bb0 0x7f65d0067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65d0150bb0 0x7f65d0067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50968/0 (socket says 172.21.15.92:50968) 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 -- 172.21.15.92:0/2876241562 learned_addr learned my addr 172.21.15.92:0/2876241562 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 -- 172.21.15.92:0/2876241562 >> v2:172.21.15.158:3300/0 conn(0x7f65d00a6e90 msgr2=0x7f65d0068450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 --2- 172.21.15.92:0/2876241562 >> v2:172.21.15.158:3300/0 conn(0x7f65d00a6e90 0x7f65d0068450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 -- 172.21.15.92:0/2876241562 >> v1:172.21.15.158:6789/0 conn(0x7f65d0150f90 legacy=0x7f65d00a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 -- 172.21.15.92:0/2876241562 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00635b0 con 0x7f65d0150bb0 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.414+0000 7f65ce59c700 1 --2- 172.21.15.92:0/2876241562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150bb0 0x7f65d0067f60 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f65d00c7900 tx=0x7f65c00090f0).ready entity=mon.0 client_cookie=46ca570181043a25 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.415+0000 7f65cd59a700 1 -- 172.21.15.92:0/2876241562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000b600 con 0x7f65d0150bb0 2023-05-16T07:36:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.415+0000 7f65cd59a700 1 -- 172.21.15.92:0/2876241562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000b760 con 0x7f65d0150bb0 2023-05-16T07:36:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.415+0000 7f65cd59a700 1 -- 172.21.15.92:0/2876241562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000bae0 con 0x7f65d0150bb0 2023-05-16T07:36:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.415+0000 7f65d4c59700 1 -- 172.21.15.92:0/2876241562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150bb0 msgr2=0x7f65d0067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.415+0000 7f65d4c59700 1 --2- 172.21.15.92:0/2876241562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150bb0 0x7f65d0067f60 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f65d00c7900 tx=0x7f65c00090f0).stop 2023-05-16T07:36:20.426 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:20.415+0000 7f6544971680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:36:20.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.424+0000 7f65d4c59700 1 -- 172.21.15.92:0/2876241562 shutdown_connections 2023-05-16T07:36:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.424+0000 7f65d4c59700 1 --2- 172.21.15.92:0/2876241562 >> v2:172.21.15.158:3300/0 conn(0x7f65d00a6e90 0x7f65d0068450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.424+0000 7f65d4c59700 1 --2- 172.21.15.92:0/2876241562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150bb0 0x7f65d0067f60 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.424+0000 7f65d4c59700 1 -- 172.21.15.92:0/2876241562 >> 172.21.15.92:0/2876241562 conn(0x7f65d00c8420 msgr2=0x7f65d00c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:20.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.436+0000 7f65d4c59700 1 -- 172.21.15.92:0/2876241562 shutdown_connections 2023-05-16T07:36:20.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.445+0000 7f65d4c59700 1 -- 172.21.15.92:0/2876241562 wait complete. 2023-05-16T07:36:20.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.446+0000 7f65d4c59700 1 Processor -- start 2023-05-16T07:36:20.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.454+0000 7f65d4c59700 1 -- start start 2023-05-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65d4c59700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 0x7f65d0162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65d4c59700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d0150bb0 0x7f65d00c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65d4c59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150f90 0x7f65d00c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65d4c59700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0162840 con 0x7f65d0150f90 2023-05-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65d4c59700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00a8410 con 0x7f65d0150bb0 2023-05-16T07:36:20.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65d4c59700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00a8cb0 con 0x7f65d00a6e90 2023-05-16T07:36:20.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65ce59c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 0x7f65d0162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65ce59c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 0x7f65d0162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:43546/0 (socket says 172.21.15.92:43546) 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65ce59c700 1 -- 172.21.15.92:0/1261176627 learned_addr learned my addr 172.21.15.92:0/1261176627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65cdd9b700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d0150bb0 0x7f65d00c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65cdd9b700 1 -- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 msgr2=0x7f65d0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65cdd9b700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 0x7f65d0162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65cdd9b700 1 -- 172.21.15.92:0/1261176627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150f90 msgr2=0x7f65d00c14c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65cdd9b700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150f90 0x7f65d00c14c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65cdd9b700 1 -- 172.21.15.92:0/1261176627 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009370 con 0x7f65d0150bb0 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.455+0000 7f65ce59c700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 0x7f65d0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.456+0000 7f65cdd9b700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d0150bb0 0x7f65d00c0fd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f65c400c440 tx=0x7f65c4007d40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.456+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4007020 con 0x7f65d0150bb0 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.456+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4007180 con 0x7f65d0150bb0 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.456+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00bfe40 con 0x7f65d0150bb0 2023-05-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.456+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00c1b40 con 0x7f65d0150bb0 2023-05-16T07:36:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.457+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4007520 con 0x7f65d0150bb0 2023-05-16T07:36:20.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.457+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d00a8e20 con 0x7f65d0150bb0 2023-05-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.458+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f65c4010070 con 0x7f65d0150bb0 2023-05-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.458+0000 7f65be7fc700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65b005ee20 0x7f65b00612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.458+0000 7f65ce59c700 1 -- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65b005ee20 msgr2=0x7f65b00612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.458+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 7223+0+0 (secure 0 0 0) 0x7f65c4079180 con 0x7f65d0150bb0 2023-05-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.458+0000 7f65ce59c700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65b005ee20 0x7f65b00612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:20.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.463+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f65c40478f0 con 0x7f65d0150bb0 2023-05-16T07:36:20.467 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:20.463+0000 7f34b9cce680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:36:20.497 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:20.494+0000 7f6544971680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:36:20.539 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:20.535+0000 7f34b9cce680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:36:20.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.639+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f65d006c350 con 0x7f65d0150bb0 2023-05-16T07:36:20.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.640+0000 7f65be7fc700 1 -- 172.21.15.92:0/1261176627 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v140) v1 ==== 110+0+10 (secure 0 0 0) 0x7f65c404a2b0 con 0x7f65d0150bb0 2023-05-16T07:36:20.644 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:36:20.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65b005ee20 msgr2=0x7f65b00612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65b005ee20 0x7f65b00612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d0150bb0 msgr2=0x7f65d00c0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d0150bb0 0x7f65d00c0fd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f65c400c440 tx=0x7f65c4007d40).stop 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 shutdown_connections 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65b005ee20 0x7f65b00612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65d00a6e90 0x7f65d0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d0150f90 0x7f65d00c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 --2- 172.21.15.92:0/1261176627 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65d0150bb0 0x7f65d00c0fd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 >> 172.21.15.92:0/1261176627 conn(0x7f65d00c8420 msgr2=0x7f65d00c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 shutdown_connections 2023-05-16T07:36:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.642+0000 7f65d4c59700 1 -- 172.21.15.92:0/1261176627 wait complete. 2023-05-16T07:36:20.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:36:20.680 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:20.677+0000 7f6544971680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:36:20.743 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:20.738+0000 7f34b9cce680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:36:20.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.818+0000 7f9b64a8c700 1 Processor -- start 2023-05-16T07:36:20.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.818+0000 7f9b64a8c700 1 -- start start 2023-05-16T07:36:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b64a8c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9b6014e210 0x7f9b6014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b64a8c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b60158060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b64a8c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b60068ba0 0x7f9b60068fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b64a8c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6006a470 con 0x7f9b60068ba0 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b64a8c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b60137670 con 0x7f9b6014e210 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b64a8c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b60158a50 con 0x7f9b60157be0 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b5ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b60068ba0 0x7f9b60068fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b5ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b60068ba0 0x7f9b60068fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51040/0 (socket says 172.21.15.92:51040) 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b5ed9d700 1 -- 172.21.15.92:0/1728999861 learned_addr learned my addr 172.21.15.92:0/1728999861 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b5e59c700 1 --2- 172.21.15.92:0/1728999861 >> v2:172.21.15.158:3300/0 conn(0x7f9b6014e210 0x7f9b6014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.819+0000 7f9b5dd9b700 1 --2- 172.21.15.92:0/1728999861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b60158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5ed9d700 1 -- 172.21.15.92:0/1728999861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 msgr2=0x7f9b60158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5ed9d700 1 --2- 172.21.15.92:0/1728999861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b60158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5ed9d700 1 -- 172.21.15.92:0/1728999861 >> v2:172.21.15.158:3300/0 conn(0x7f9b6014e210 msgr2=0x7f9b6014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5ed9d700 1 --2- 172.21.15.92:0/1728999861 >> v2:172.21.15.158:3300/0 conn(0x7f9b6014e210 0x7f9b6014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5ed9d700 1 -- 172.21.15.92:0/1728999861 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60158550 con 0x7f9b60068ba0 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5ed9d700 1 --2- 172.21.15.92:0/1728999861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60068fd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f9b600c4df0 tx=0x7f9b58009840).ready entity=mon.0 client_cookie=12eb20fb49d89042 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5d59a700 1 -- 172.21.15.92:0/1728999861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800e070 con 0x7f9b60068ba0 2023-05-16T07:36:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5d59a700 1 -- 172.21.15.92:0/1728999861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5800b970 con 0x7f9b60068ba0 2023-05-16T07:36:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b5d59a700 1 -- 172.21.15.92:0/1728999861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800bcf0 con 0x7f9b60068ba0 2023-05-16T07:36:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b64a8c700 1 -- 172.21.15.92:0/1728999861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 msgr2=0x7f9b60068fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.820+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/1728999861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60068fd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f9b600c4df0 tx=0x7f9b58009840).stop 2023-05-16T07:36:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 -- 172.21.15.92:0/1728999861 shutdown_connections 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/1728999861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60068fd0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/1728999861 >> v2:172.21.15.158:3300/0 conn(0x7f9b6014e210 0x7f9b6014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/1728999861 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b60158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 -- 172.21.15.92:0/1728999861 >> 172.21.15.92:0/1728999861 conn(0x7f9b600c4600 msgr2=0x7f9b60066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 -- 172.21.15.92:0/1728999861 shutdown_connections 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.821+0000 7f9b64a8c700 1 -- 172.21.15.92:0/1728999861 wait complete. 2023-05-16T07:36:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 Processor -- start 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 -- start start 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60135b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b6014e210 0x7f9b600c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b600c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c6870 con 0x7f9b60068ba0 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b60156d40 con 0x7f9b6014e210 2023-05-16T07:36:20.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.822+0000 7f9b64a8c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b60157020 con 0x7f9b60157be0 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60135b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60135b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51042/0 (socket says 172.21.15.92:51042) 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 -- 172.21.15.92:0/3724625972 learned_addr learned my addr 172.21.15.92:0/3724625972 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5dd9b700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b6014e210 0x7f9b600c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5ed9d700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b600c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 -- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 msgr2=0x7f9b600c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b600c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 -- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b6014e210 msgr2=0x7f9b600c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b6014e210 0x7f9b600c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 -- 172.21.15.92:0/3724625972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58009ae0 con 0x7f9b60068ba0 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b5e59c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60135b80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f9b5000e930 tx=0x7f9b50009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50012600 con 0x7f9b60068ba0 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50012760 con 0x7f9b60068ba0 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b60157300 con 0x7f9b60068ba0 2023-05-16T07:36:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50012ae0 con 0x7f9b60068ba0 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.823+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b60136070 con 0x7f9b60068ba0 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.824+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6006c350 con 0x7f9b60068ba0 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.825+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f9b50022070 con 0x7f9b60068ba0 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.825+0000 7f9b4e7fc700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9b4005ee20 0x7f9b400612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.825+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 7223+0+0 (secure 0 0 0) 0x7f9b5007a2b0 con 0x7f9b60068ba0 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.825+0000 7f9b5dd9b700 1 -- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9b4005ee20 msgr2=0x7f9b400612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.825+0000 7f9b5dd9b700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9b4005ee20 0x7f9b400612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:20.830+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9b5004b390 con 0x7f9b60068ba0 2023-05-16T07:36:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.001+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9b600697f0 con 0x7f9b60068ba0 2023-05-16T07:36:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.002+0000 7f9b4e7fc700 1 -- 172.21.15.92:0/3724625972 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v140) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9b5004e9b0 con 0x7f9b60068ba0 2023-05-16T07:36:21.006 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:36:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9b4005ee20 msgr2=0x7f9b400612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9b4005ee20 0x7f9b400612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 msgr2=0x7f9b60135b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60135b80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f9b5000e930 tx=0x7f9b50009060).stop 2023-05-16T07:36:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 shutdown_connections 2023-05-16T07:36:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9b4005ee20 0x7f9b400612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9b60157be0 0x7f9b600c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b60068ba0 0x7f9b60135b80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 --2- 172.21.15.92:0/3724625972 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b6014e210 0x7f9b600c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 >> 172.21.15.92:0/3724625972 conn(0x7f9b600c4600 msgr2=0x7f9b60064b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 shutdown_connections 2023-05-16T07:36:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.004+0000 7f9b64a8c700 1 -- 172.21.15.92:0/3724625972 wait complete. 2023-05-16T07:36:21.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:36:21.131 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.128+0000 7f6544971680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:36:21.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.135+0000 7f3f88f16700 1 Processor -- start 2023-05-16T07:36:21.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.135+0000 7f3f88f16700 1 -- start start 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f88f16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f84157710 0x7f3f840a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f88f16700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f840bfcc0 con 0x7f3f840aa1f0 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f88f16700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8406a140 con 0x7f3f840c62c0 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f88f16700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8406a9a0 con 0x7f3f84157710 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f84157710 0x7f3f840a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f7affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f840aa1f0 0x7f3f840c5bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33566/0 (socket says 172.21.15.92:33566) 2023-05-16T07:36:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f7affd700 1 -- 172.21.15.92:0/1797619126 learned_addr learned my addr 172.21.15.92:0/1797619126 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f837fe700 1 -- 172.21.15.92:0/1797619126 >> v1:172.21.15.158:6789/0 conn(0x7f3f840c62c0 legacy=0x7f3f8415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f837fe700 1 -- 172.21.15.92:0/1797619126 >> v1:172.21.15.92:6789/0 conn(0x7f3f840aa1f0 legacy=0x7f3f840c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f837fe700 1 -- 172.21.15.92:0/1797619126 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8406a3e0 con 0x7f3f84157710 2023-05-16T07:36:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f82ffd700 1 -- 172.21.15.92:0/1797619126 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606675310 0 0) 0x7f3f840bfcc0 con 0x7f3f840aa1f0 2023-05-16T07:36:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.137+0000 7f3f837fe700 1 --2- 172.21.15.92:0/1797619126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f84157710 0x7f3f840a9d00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f3f74009760 tx=0x7f3f74009060).ready entity=mon.0 client_cookie=f80daaa53249d4eb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f82ffd700 1 -- 172.21.15.92:0/1797619126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7400b580 con 0x7f3f84157710 2023-05-16T07:36:21.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f82ffd700 1 -- 172.21.15.92:0/1797619126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7400b6e0 con 0x7f3f84157710 2023-05-16T07:36:21.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f82ffd700 1 -- 172.21.15.92:0/1797619126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7400ba60 con 0x7f3f84157710 2023-05-16T07:36:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f88f16700 1 -- 172.21.15.92:0/1797619126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f84157710 msgr2=0x7f3f840a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f88f16700 1 --2- 172.21.15.92:0/1797619126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f84157710 0x7f3f840a9d00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f3f74009760 tx=0x7f3f74009060).stop 2023-05-16T07:36:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f88f16700 1 -- 172.21.15.92:0/1797619126 shutdown_connections 2023-05-16T07:36:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f88f16700 1 --2- 172.21.15.92:0/1797619126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f84157710 0x7f3f840a9d00 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.138+0000 7f3f88f16700 1 -- 172.21.15.92:0/1797619126 >> 172.21.15.92:0/1797619126 conn(0x7f3f840cb600 msgr2=0x7f3f840c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.139+0000 7f3f88f16700 1 -- 172.21.15.92:0/1797619126 shutdown_connections 2023-05-16T07:36:21.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.139+0000 7f3f88f16700 1 -- 172.21.15.92:0/1797619126 wait complete. 2023-05-16T07:36:21.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.140+0000 7f3f88f16700 1 Processor -- start 2023-05-16T07:36:21.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.140+0000 7f3f88f16700 1 -- start start 2023-05-16T07:36:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.140+0000 7f3f88f16700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f840aa1f0 0x7f3f840aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.140+0000 7f3f88f16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 0x7f3f84148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.140+0000 7f3f88f16700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f84157710 0x7f3f840a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.140+0000 7f3f88f16700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840a2e80 con 0x7f3f840c62c0 2023-05-16T07:36:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f88f16700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840a2270 con 0x7f3f84157710 2023-05-16T07:36:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f88f16700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840a2550 con 0x7f3f840aa1f0 2023-05-16T07:36:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 0x7f3f84148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 0x7f3f84148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51058/0 (socket says 172.21.15.92:51058) 2023-05-16T07:36:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 -- 172.21.15.92:0/2950585383 learned_addr learned my addr 172.21.15.92:0/2950585383 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f837fe700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f840aa1f0 0x7f3f840aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f83fff700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f84157710 0x7f3f840a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 -- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f840aa1f0 msgr2=0x7f3f840aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f840aa1f0 0x7f3f840aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 -- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f84157710 msgr2=0x7f3f840a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f84157710 0x7f3f840a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 -- 172.21.15.92:0/2950585383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740092a0 con 0x7f3f840c62c0 2023-05-16T07:36:21.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7affd700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 0x7f3f84148920 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f3f6c00c4c0 tx=0x7f3f6c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:21.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00ce30 con 0x7f3f840c62c0 2023-05-16T07:36:21.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0072e0 con 0x7f3f840c62c0 2023-05-16T07:36:21.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f840a2830 con 0x7f3f840c62c0 2023-05-16T07:36:21.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0057a0 con 0x7f3f840c62c0 2023-05-16T07:36:21.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.141+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f840a6640 con 0x7f3f840c62c0 2023-05-16T07:36:21.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.142+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f84083120 con 0x7f3f840c62c0 2023-05-16T07:36:21.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.143+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f3f6c01a060 con 0x7f3f840c62c0 2023-05-16T07:36:21.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.143+0000 7f3f7bfff700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f6405ee20 0x7f3f640612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.143+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 7223+0+0 (secure 0 0 0) 0x7f3f6c079630 con 0x7f3f840c62c0 2023-05-16T07:36:21.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.143+0000 7f3f837fe700 1 -- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f6405ee20 msgr2=0x7f3f640612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:21.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.143+0000 7f3f837fe700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f6405ee20 0x7f3f640612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:21.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.147+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3f6c04a760 con 0x7f3f840c62c0 2023-05-16T07:36:21.219 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:21.216+0000 7f34b9cce680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:36:21.235 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.233+0000 7f6544971680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.315+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f84047150 con 0x7f3f840c62c0 2023-05-16T07:36:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.315+0000 7f3f7bfff700 1 -- 172.21.15.92:0/2950585383 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f6c04dd80 con 0x7f3f840c62c0 2023-05-16T07:36:21.319 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:36:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f6405ee20 msgr2=0x7f3f640612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f6405ee20 0x7f3f640612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 msgr2=0x7f3f84148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 0x7f3f84148920 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f3f6c00c4c0 tx=0x7f3f6c007c10).stop 2023-05-16T07:36:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 shutdown_connections 2023-05-16T07:36:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f6405ee20 0x7f3f640612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f840aa1f0 0x7f3f840aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f84157710 0x7f3f840a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 --2- 172.21.15.92:0/2950585383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f840c62c0 0x7f3f84148920 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 >> 172.21.15.92:0/2950585383 conn(0x7f3f840cb600 msgr2=0x7f3f840c3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 shutdown_connections 2023-05-16T07:36:21.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.317+0000 7f3f88f16700 1 -- 172.21.15.92:0/2950585383 wait complete. 2023-05-16T07:36:21.325 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:21.322+0000 7f34b9cce680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:21.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:36:21.391 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.388+0000 7f6544971680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:21.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.452+0000 7f3441bda700 1 Processor -- start 2023-05-16T07:36:21.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.453+0000 7f3441bda700 1 -- start start 2023-05-16T07:36:21.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.453+0000 7f3441bda700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f343c153ee0 0x7f343c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.453+0000 7f3441bda700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f343c0be280 0x7f343c066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.453+0000 7f3441bda700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c155870 con 0x7f343c062cc0 2023-05-16T07:36:21.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.453+0000 7f3441bda700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c067340 con 0x7f343c153ee0 2023-05-16T07:36:21.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.453+0000 7f3441bda700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c155670 con 0x7f343c0be280 2023-05-16T07:36:21.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f343c153ee0 0x7f343c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f343c062cc0 0x7f343c0bdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33582/0 (socket says 172.21.15.92:33582) 2023-05-16T07:36:21.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343affd700 1 -- 172.21.15.92:0/3825266777 learned_addr learned my addr 172.21.15.92:0/3825266777 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343bfff700 1 --2- 172.21.15.92:0/3825266777 >> v2:172.21.15.158:3300/0 conn(0x7f343c0be280 0x7f343c066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343b7fe700 1 -- 172.21.15.92:0/3825266777 >> v2:172.21.15.158:3300/0 conn(0x7f343c0be280 msgr2=0x7f343c066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343b7fe700 1 --2- 172.21.15.92:0/3825266777 >> v2:172.21.15.158:3300/0 conn(0x7f343c0be280 0x7f343c066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343a7fc700 1 -- 172.21.15.92:0/3825266777 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388520549 0 0) 0x7f343c155870 con 0x7f343c062cc0 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343b7fe700 1 -- 172.21.15.92:0/3825266777 >> v1:172.21.15.92:6789/0 conn(0x7f343c062cc0 legacy=0x7f343c0bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343b7fe700 1 -- 172.21.15.92:0/3825266777 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c063120 con 0x7f343c153ee0 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343b7fe700 1 --2- 172.21.15.92:0/3825266777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c153ee0 0x7f343c1542c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f343c0c5430 tx=0x7f342c009060).ready entity=mon.0 client_cookie=aaf95fb408118299 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:21.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343a7fc700 1 -- 172.21.15.92:0/3825266777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c00f510 con 0x7f343c153ee0 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343a7fc700 1 -- 172.21.15.92:0/3825266777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c00f670 con 0x7f343c153ee0 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.454+0000 7f343a7fc700 1 -- 172.21.15.92:0/3825266777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c00f9f0 con 0x7f343c153ee0 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 -- 172.21.15.92:0/3825266777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c153ee0 msgr2=0x7f343c1542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 --2- 172.21.15.92:0/3825266777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c153ee0 0x7f343c1542c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f343c0c5430 tx=0x7f342c009060).stop 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 -- 172.21.15.92:0/3825266777 shutdown_connections 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 --2- 172.21.15.92:0/3825266777 >> v2:172.21.15.158:3300/0 conn(0x7f343c0be280 0x7f343c066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 --2- 172.21.15.92:0/3825266777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c153ee0 0x7f343c1542c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 -- 172.21.15.92:0/3825266777 >> 172.21.15.92:0/3825266777 conn(0x7f343c0c8420 msgr2=0x7f343c0c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 -- 172.21.15.92:0/3825266777 shutdown_connections 2023-05-16T07:36:21.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.455+0000 7f3441bda700 1 -- 172.21.15.92:0/3825266777 wait complete. 2023-05-16T07:36:21.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.456+0000 7f3441bda700 1 Processor -- start 2023-05-16T07:36:21.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.456+0000 7f3441bda700 1 -- start start 2023-05-16T07:36:21.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f3441bda700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f343c062cc0 0x7f343c0a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f3441bda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 0x7f343c164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f3441bda700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f343c153ee0 0x7f343c148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f3441bda700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c167a40 con 0x7f343c0be280 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f3441bda700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c167ce0 con 0x7f343c153ee0 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f3441bda700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c164960 con 0x7f343c062cc0 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 0x7f343c164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 0x7f343c164470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51072/0 (socket says 172.21.15.92:51072) 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 -- 172.21.15.92:0/3065224031 learned_addr learned my addr 172.21.15.92:0/3065224031 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343bfff700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f343c153ee0 0x7f343c148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343b7fe700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f343c062cc0 0x7f343c0a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 -- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f343c062cc0 msgr2=0x7f343c0a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f343c062cc0 0x7f343c0a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 -- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f343c153ee0 msgr2=0x7f343c148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f343c153ee0 0x7f343c148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 -- 172.21.15.92:0/3065224031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0092a0 con 0x7f343c0be280 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.457+0000 7f343affd700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 0x7f343c164470 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f343000c4e0 tx=0x7f3430007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.458+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000ce90 con 0x7f343c0be280 2023-05-16T07:36:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.458+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300072e0 con 0x7f343c0be280 2023-05-16T07:36:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.458+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c164c30 con 0x7f343c0be280 2023-05-16T07:36:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.458+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430007660 con 0x7f343c0be280 2023-05-16T07:36:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.458+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c1651f0 con 0x7f343c0be280 2023-05-16T07:36:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.458+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0a89f0 con 0x7f343c0be280 2023-05-16T07:36:21.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.459+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f343001a060 con 0x7f343c0be280 2023-05-16T07:36:21.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.459+0000 7f34237fe700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f342805ee20 0x7f34280612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.459+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f34300796a0 con 0x7f343c0be280 2023-05-16T07:36:21.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.459+0000 7f343b7fe700 1 -- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f342805ee20 msgr2=0x7f34280612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:21.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.459+0000 7f343b7fe700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f342805ee20 0x7f34280612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:21.470 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.462+0000 7f6544971680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:36:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.463+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f343004a7e0 con 0x7f343c0be280 2023-05-16T07:36:21.487 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:21.484+0000 7f34b9cce680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:21.533 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.530+0000 7f6544971680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:36:21.564 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:21.560+0000 7f34b9cce680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:36:21.631 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:21.628+0000 7f34b9cce680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:36:21.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.633+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f343c165410 con 0x7f343c0be280 2023-05-16T07:36:21.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.634+0000 7f34237fe700 1 -- 172.21.15.92:0/3065224031 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+11 (secure 0 0 0) 0x7f343004de00 con 0x7f343c0be280 2023-05-16T07:36:21.637 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:36:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f342805ee20 msgr2=0x7f34280612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f342805ee20 0x7f34280612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 msgr2=0x7f343c164470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 0x7f343c164470 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f343000c4e0 tx=0x7f3430007c70).stop 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 shutdown_connections 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f342805ee20 0x7f34280612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f343c062cc0 0x7f343c0a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f343c153ee0 0x7f343c148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 --2- 172.21.15.92:0/3065224031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343c0be280 0x7f343c164470 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 >> 172.21.15.92:0/3065224031 conn(0x7f343c0c8420 msgr2=0x7f343c0c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 shutdown_connections 2023-05-16T07:36:21.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.636+0000 7f3441bda700 1 -- 172.21.15.92:0/3065224031 wait complete. 2023-05-16T07:36:21.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.762+0000 7f03c20ef700 1 Processor -- start 2023-05-16T07:36:21.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.762+0000 7f03c20ef700 1 -- start start 2023-05-16T07:36:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03c20ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc0bf4f0 0x7f03bc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03c20ef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc062960 con 0x7f03bc063e30 2023-05-16T07:36:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03c20ef700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc069fb0 con 0x7f03bc154040 2023-05-16T07:36:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03c20ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc154420 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03baffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f03bc063e30 0x7f03bc0bda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33596/0 (socket says 172.21.15.92:33596) 2023-05-16T07:36:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03baffd700 1 -- 172.21.15.92:0/1791339850 learned_addr learned my addr 172.21.15.92:0/1791339850 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03bbfff700 1 --2- 172.21.15.92:0/1791339850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc0bf4f0 0x7f03bc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987757928 0 0) 0x7f03bc062960 con 0x7f03bc063e30 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.763+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a40033f0 con 0x7f03bc063e30 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931104420 0 0) 0x7f03bc069fb0 con 0x7f03bc154040 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03bc062960 con 0x7f03bc154040 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03bbfff700 1 -- 172.21.15.92:0/1791339850 >> v1:172.21.15.158:6789/0 conn(0x7f03bc154040 legacy=0x7f03bc063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03bbfff700 1 -- 172.21.15.92:0/1791339850 >> v1:172.21.15.92:6789/0 conn(0x7f03bc063e30 legacy=0x7f03bc0bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3595518026 0 0) 0x7f03a40033f0 con 0x7f03bc063e30 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03bbfff700 1 -- 172.21.15.92:0/1791339850 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc062350 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03bbfff700 1 --2- 172.21.15.92:0/1791339850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc0bf4f0 0x7f03bc069ac0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f03b4009e90 tx=0x7f03b4009780).ready entity=mon.2 client_cookie=f0bf2c72eb057b53 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400e070 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400b7a0 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03ba7fc700 1 -- 172.21.15.92:0/1791339850 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0001470 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03c20ef700 1 -- 172.21.15.92:0/1791339850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc0bf4f0 msgr2=0x7f03bc069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03c20ef700 1 --2- 172.21.15.92:0/1791339850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc0bf4f0 0x7f03bc069ac0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f03b4009e90 tx=0x7f03b4009780).stop 2023-05-16T07:36:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03c20ef700 1 -- 172.21.15.92:0/1791339850 shutdown_connections 2023-05-16T07:36:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03c20ef700 1 --2- 172.21.15.92:0/1791339850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc0bf4f0 0x7f03bc069ac0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.764+0000 7f03c20ef700 1 -- 172.21.15.92:0/1791339850 >> 172.21.15.92:0/1791339850 conn(0x7f03bc0c8aa0 msgr2=0x7f03bc0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.765+0000 7f03c20ef700 1 -- 172.21.15.92:0/1791339850 shutdown_connections 2023-05-16T07:36:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.765+0000 7f03c20ef700 1 -- 172.21.15.92:0/1791339850 wait complete. 2023-05-16T07:36:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 Processor -- start 2023-05-16T07:36:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 -- start start 2023-05-16T07:36:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03bc063e30 0x7f03bc0a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 0x7f03bc1504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc154040 0x7f03bc14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0aa120 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc150990 con 0x7f03bc063e30 2023-05-16T07:36:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.766+0000 7f03c20ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0a9880 con 0x7f03bc154040 2023-05-16T07:36:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 0x7f03bc1504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 0x7f03bc1504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51086/0 (socket says 172.21.15.92:51086) 2023-05-16T07:36:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 -- 172.21.15.92:0/2330653536 learned_addr learned my addr 172.21.15.92:0/2330653536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03bb7fe700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03bc063e30 0x7f03bc0a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03bbfff700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc154040 0x7f03bc14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 -- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc154040 msgr2=0x7f03bc14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc154040 0x7f03bc14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 -- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03bc063e30 msgr2=0x7f03bc0a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03bc063e30 0x7f03bc0a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 -- 172.21.15.92:0/2330653536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4009a20 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03baffd700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 0x7f03bc1504a0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f03b000c790 tx=0x7f03b0001a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00072e0 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0007440 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc0a9b60 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00057a0 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.767+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc0ac240 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.768+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc0ac0a0 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.769+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f03b0010070 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.769+0000 7f03a37fe700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 0x7f03a80612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.769+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f03b0079b00 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.769+0000 7f03bb7fe700 1 -- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 msgr2=0x7f03a80612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:21.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.769+0000 7f03bb7fe700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 0x7f03a80612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:21.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.773+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f03b004ac60 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.788 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.785+0000 7f6544971680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:36:21.902 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:21.899+0000 7f34b9cce680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:36:21.971 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:21.968+0000 7f6544971680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:21.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.970+0000 7f03bb7fe700 1 -- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 msgr2=0x7f03a80612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:21.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.970+0000 7f03bb7fe700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 0x7f03a80612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:36:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.988+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03bc0ac0a0 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.989+0000 7f03a37fe700 1 -- 172.21.15.92:0/2330653536 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f03bc0ac0a0 con 0x7f03bc0bf4f0 2023-05-16T07:36:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:21.993 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:49.065391+0000","modified":"2023-05-16T07:19:51.075477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2186588149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2186588149},{"type":"v1","addr":"172.21.15.92:6835","nonce":2186588149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-05-16T07:36:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 msgr2=0x7f03a80612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 0x7f03a80612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 msgr2=0x7f03bc1504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 0x7f03bc1504a0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f03b000c790 tx=0x7f03b0001a50).stop 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 shutdown_connections 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f03a805ee20 0x7f03a80612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f03bc154040 0x7f03bc14e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f03bc063e30 0x7f03bc0a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 --2- 172.21.15.92:0/2330653536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0bf4f0 0x7f03bc1504a0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 >> 172.21.15.92:0/2330653536 conn(0x7f03bc0c8aa0 msgr2=0x7f03bc14f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 shutdown_connections 2023-05-16T07:36:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:21.991+0000 7f03c20ef700 1 -- 172.21.15.92:0/2330653536 wait complete. 2023-05-16T07:36:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 49 2023-05-16T07:36:22.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:22.097 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:22.094+0000 7f34b9cce680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:22.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.121+0000 7f64a22a3700 1 Processor -- start 2023-05-16T07:36:22.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.122+0000 7f64a22a3700 1 -- start start 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.122+0000 7f64a22a3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.122+0000 7f64a22a3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 0x7f649c062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.122+0000 7f64a22a3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c062960 con 0x7f649c0bf4f0 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.122+0000 7f64a22a3700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c154420 con 0x7f649c063e30 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.122+0000 7f64a22a3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c069fb0 con 0x7f649c154040 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f64a0840700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 0x7f649c062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:43666/0 (socket says 172.21.15.92:43666) 2023-05-16T07:36:22.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f64a0840700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 0x7f649c062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:54734/0 (socket says 172.21.15.92:54734) 2023-05-16T07:36:22.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 -- 172.21.15.92:0/4026713550 learned_addr learned my addr 172.21.15.92:0/4026713550 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 -- 172.21.15.92:0/4026713550 >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 msgr2=0x7f649c062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 --2- 172.21.15.92:0/4026713550 >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 0x7f649c062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 -- 172.21.15.92:0/4026713550 >> v1:172.21.15.158:6789/0 conn(0x7f649c0bf4f0 legacy=0x7f649c063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 -- 172.21.15.92:0/4026713550 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c062350 con 0x7f649c154040 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f64a0840700 1 --2- 172.21.15.92:0/4026713550 >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 0x7f649c062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649bfff700 1 --2- 172.21.15.92:0/4026713550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c069ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f648c006fd0 tx=0x7f648c0090f0).ready entity=mon.2 client_cookie=806b0e7be75db177 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649affd700 1 -- 172.21.15.92:0/4026713550 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00b580 con 0x7f649c154040 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649affd700 1 -- 172.21.15.92:0/4026713550 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c00b6e0 con 0x7f649c154040 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.123+0000 7f649affd700 1 -- 172.21.15.92:0/4026713550 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00ba60 con 0x7f649c154040 2023-05-16T07:36:22.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 -- 172.21.15.92:0/4026713550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 msgr2=0x7f649c069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 --2- 172.21.15.92:0/4026713550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c069ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f648c006fd0 tx=0x7f648c0090f0).stop 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 -- 172.21.15.92:0/4026713550 shutdown_connections 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 --2- 172.21.15.92:0/4026713550 >> v2:172.21.15.158:3300/0 conn(0x7f649c063e30 0x7f649c062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 --2- 172.21.15.92:0/4026713550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c069ac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 -- 172.21.15.92:0/4026713550 >> 172.21.15.92:0/4026713550 conn(0x7f649c0c8aa0 msgr2=0x7f649c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 -- 172.21.15.92:0/4026713550 shutdown_connections 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.124+0000 7f64a22a3700 1 -- 172.21.15.92:0/4026713550 wait complete. 2023-05-16T07:36:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.125+0000 7f64a22a3700 1 Processor -- start 2023-05-16T07:36:22.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.125+0000 7f64a22a3700 1 -- start start 2023-05-16T07:36:22.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a22a3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f649c063e30 0x7f649c145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a22a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 0x7f649c0aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a22a3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c0a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a22a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c145b60 con 0x7f649c0bf4f0 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a22a3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0aa200 con 0x7f649c063e30 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a22a3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0a9ed0 con 0x7f649c154040 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 0x7f649c0aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 0x7f649c0aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51102/0 (socket says 172.21.15.92:51102) 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 -- 172.21.15.92:0/670948530 learned_addr learned my addr 172.21.15.92:0/670948530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:22.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f64a0840700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c0a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649bfff700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f649c063e30 0x7f649c145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 -- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 msgr2=0x7f649c0a8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c0a8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 -- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f649c063e30 msgr2=0x7f649c145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f649c063e30 0x7f649c145670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 -- 172.21.15.92:0/670948530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494009a20 con 0x7f649c0bf4f0 2023-05-16T07:36:22.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f649b7fe700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 0x7f649c0aa690 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f649000c410 tx=0x7f6490007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.126+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490012400 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.127+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490012560 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.127+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648c009370 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.127+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490012900 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.127+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c0a4d00 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.128+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f6490019030 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.128+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c06c350 con 0x7f649c0bf4f0 2023-05-16T07:36:22.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.128+0000 7f6483fff700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f648805ee20 0x7f64880612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.128+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f6490079220 con 0x7f649c0bf4f0 2023-05-16T07:36:22.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.128+0000 7f649bfff700 1 -- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f648805ee20 msgr2=0x7f64880612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:22.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.128+0000 7f649bfff700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f648805ee20 0x7f64880612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:22.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.132+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f64900488f0 con 0x7f649c0bf4f0 2023-05-16T07:36:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.315+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f649c0c2c10 con 0x7f649c0bf4f0 2023-05-16T07:36:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.316+0000 7f6483fff700 1 -- 172.21.15.92:0/670948530 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+18653 (secure 0 0 0) 0x7f6490017070 con 0x7f649c0bf4f0 2023-05-16T07:36:22.319 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:22.320 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":141,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:36:21.317961+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:47.601645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:48.023978+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3660652651":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/4009528875":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/4145570814":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/3403657457":"2023-05-17T07:20:00.266255+0000","172.21.15.158:0/1546480240":"2023-05-17T07:20:00.266255+0000","172.21.15.158:0/117216949":"2023-05-17T07:31:20.470620+0000","172.21.15.158:6833/86301":"2023-05-17T07:20:00.266255+0000","172.21.15.158:6832/86301":"2023-05-17T07:20:00.266255+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:22.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f648805ee20 msgr2=0x7f64880612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:22.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f648805ee20 0x7f64880612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:22.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 msgr2=0x7f649c0aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 0x7f649c0aa690 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f649000c410 tx=0x7f6490007d90).stop 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 shutdown_connections 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f648805ee20 0x7f64880612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f649c154040 0x7f649c0a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f649c063e30 0x7f649c145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 --2- 172.21.15.92:0/670948530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649c0bf4f0 0x7f649c0aa690 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 >> 172.21.15.92:0/670948530 conn(0x7f649c0c8aa0 msgr2=0x7f649c0c4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 shutdown_connections 2023-05-16T07:36:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.318+0000 7f64a22a3700 1 -- 172.21.15.92:0/670948530 wait complete. 2023-05-16T07:36:22.334 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:36:22.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:22.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.445+0000 7fd69a8a1700 1 Processor -- start 2023-05-16T07:36:22.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.445+0000 7fd69a8a1700 1 -- start start 2023-05-16T07:36:22.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd69a8a1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd694154170 0x7fd6940aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd69a8a1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6940c5cb0 0x7fd6940c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd69a8a1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6940c6650 con 0x7fd694154550 2023-05-16T07:36:22.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd69a8a1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940ab200 con 0x7fd6940c5cb0 2023-05-16T07:36:22.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd69a8a1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940c08b0 con 0x7fd694154170 2023-05-16T07:36:22.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd698e3e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6940c5cb0 0x7fd6940c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd698e3e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6940c5cb0 0x7fd6940c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51144/0 (socket says 172.21.15.92:51144) 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd698e3e700 1 -- 172.21.15.92:0/3309507971 learned_addr learned my addr 172.21.15.92:0/3309507971 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.446+0000 7fd693fff700 1 --2- 172.21.15.92:0/3309507971 >> v2:172.21.15.158:3300/0 conn(0x7fd694154170 0x7fd6940aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd698e3e700 1 -- 172.21.15.92:0/3309507971 >> v2:172.21.15.158:3300/0 conn(0x7fd694154170 msgr2=0x7fd6940aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd698e3e700 1 --2- 172.21.15.92:0/3309507971 >> v2:172.21.15.158:3300/0 conn(0x7fd694154170 0x7fd6940aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd698e3e700 1 -- 172.21.15.92:0/3309507971 >> v1:172.21.15.158:6789/0 conn(0x7fd694154550 legacy=0x7fd6940a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd698e3e700 1 -- 172.21.15.92:0/3309507971 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940bfca0 con 0x7fd6940c5cb0 2023-05-16T07:36:22.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd698e3e700 1 --2- 172.21.15.92:0/3309507971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940c5cb0 0x7fd6940c6160 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fd694083030 tx=0x7fd68c009840).ready entity=mon.0 client_cookie=9199b058bbace59f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd6937fe700 1 -- 172.21.15.92:0/3309507971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00e070 con 0x7fd6940c5cb0 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd6937fe700 1 -- 172.21.15.92:0/3309507971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00ba10 con 0x7fd6940c5cb0 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd6937fe700 1 -- 172.21.15.92:0/3309507971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00bd90 con 0x7fd6940c5cb0 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 -- 172.21.15.92:0/3309507971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940c5cb0 msgr2=0x7fd6940c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/3309507971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940c5cb0 0x7fd6940c6160 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fd694083030 tx=0x7fd68c009840).stop 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 -- 172.21.15.92:0/3309507971 shutdown_connections 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/3309507971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940c5cb0 0x7fd6940c6160 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/3309507971 >> v2:172.21.15.158:3300/0 conn(0x7fd694154170 0x7fd6940aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 -- 172.21.15.92:0/3309507971 >> 172.21.15.92:0/3309507971 conn(0x7fd6940cb6d0 msgr2=0x7fd6940c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:22.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 -- 172.21.15.92:0/3309507971 shutdown_connections 2023-05-16T07:36:22.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.447+0000 7fd69a8a1700 1 -- 172.21.15.92:0/3309507971 wait complete. 2023-05-16T07:36:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 Processor -- start 2023-05-16T07:36:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 -- start start 2023-05-16T07:36:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd6940c5cb0 0x7fd6940adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd694154170 0x7fd6941489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 0x7fd6941593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940ae140 con 0x7fd694154550 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940ae3c0 con 0x7fd6940c5cb0 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd69a8a1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940ae6a0 con 0x7fd694154170 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd698e3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 0x7fd6941593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd698e3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 0x7fd6941593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51150/0 (socket says 172.21.15.92:51150) 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd698e3e700 1 -- 172.21.15.92:0/2520829677 learned_addr learned my addr 172.21.15.92:0/2520829677 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd693fff700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd6940c5cb0 0x7fd6940adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.449+0000 7fd68bfff700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd694154170 0x7fd6941489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd698e3e700 1 -- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd694154170 msgr2=0x7fd6941489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd698e3e700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd694154170 0x7fd6941489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd698e3e700 1 -- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd6940c5cb0 msgr2=0x7fd6940adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd698e3e700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd6940c5cb0 0x7fd6940adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd698e3e700 1 -- 172.21.15.92:0/2520829677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009ae0 con 0x7fd694154550 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd698e3e700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 0x7fd6941593a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7fd68c009540 tx=0x7fd68c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c013e60 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c0092e0 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c01d640 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6940ae980 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.450+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6940aef40 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.451+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940666c0 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.451+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7fd68c01b040 con 0x7fd694154550 2023-05-16T07:36:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.452+0000 7fd690ff9700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 0x7fd6740612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.452+0000 7fd693fff700 1 -- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 msgr2=0x7fd6740612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:22.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.452+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7fd68c07b850 con 0x7fd694154550 2023-05-16T07:36:22.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.452+0000 7fd693fff700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 0x7fd6740612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:22.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.456+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd68c04c990 con 0x7fd694154550 2023-05-16T07:36:22.616 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:22.613+0000 7f6544971680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:36:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.653+0000 7fd693fff700 1 -- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 msgr2=0x7fd6740612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.653+0000 7fd693fff700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 0x7fd6740612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:36:22.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.664+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6940c6650 con 0x7fd694154550 2023-05-16T07:36:22.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.665+0000 7fd690ff9700 1 -- 172.21.15.92:0/2520829677 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fd6940c6650 con 0x7fd694154550 2023-05-16T07:36:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:22.669 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:49.065391+0000","modified":"2023-05-16T07:19:51.075477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2186588149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2186588149},{"type":"v1","addr":"172.21.15.92:6835","nonce":2186588149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-05-16T07:36:22.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 msgr2=0x7fd6740612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 0x7fd6740612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 msgr2=0x7fd6941593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 0x7fd6941593a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7fd68c009540 tx=0x7fd68c009840).stop 2023-05-16T07:36:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 shutdown_connections 2023-05-16T07:36:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd694154550 0x7fd6941593a0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd6940c5cb0 0x7fd6940adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd67405ee20 0x7fd6740612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:22.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 --2- 172.21.15.92:0/2520829677 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd694154170 0x7fd6941489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 >> 172.21.15.92:0/2520829677 conn(0x7fd6940cb6d0 msgr2=0x7fd6940a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:22.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 shutdown_connections 2023-05-16T07:36:22.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.667+0000 7fd69a8a1700 1 -- 172.21.15.92:0/2520829677 wait complete. 2023-05-16T07:36:22.673 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 49 2023-05-16T07:36:22.682 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:22.679+0000 7f6544971680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:36:22.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:22.754 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:22.751+0000 7f6544971680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:36:22.776 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:22.773+0000 7f34b9cce680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:36:22.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.794+0000 7f2610547700 1 Processor -- start 2023-05-16T07:36:22.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.794+0000 7f2610547700 1 -- start start 2023-05-16T07:36:22.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f2610547700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2608068060 0x7f26080bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f2610547700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080bdbd0 con 0x7f26080bd780 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f2610547700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2608154b80 con 0x7f260813a5a0 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f2610547700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2608154e60 con 0x7f2608068060 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f260dae2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2608068060 0x7f26080bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f260eae4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f26080bd780 0x7f26080c2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33676/0 (socket says 172.21.15.92:33676) 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.795+0000 7f260eae4700 1 -- 172.21.15.92:0/2314540338 learned_addr learned my addr 172.21.15.92:0/2314540338 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260dae2700 1 -- 172.21.15.92:0/2314540338 >> v1:172.21.15.158:6789/0 conn(0x7f260813a5a0 legacy=0x7f2608068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260dae2700 1 -- 172.21.15.92:0/2314540338 >> v1:172.21.15.92:6789/0 conn(0x7f26080bd780 legacy=0x7f26080c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260dae2700 1 -- 172.21.15.92:0/2314540338 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608155140 con 0x7f2608068060 2023-05-16T07:36:22.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260d2e1700 1 -- 172.21.15.92:0/2314540338 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288501479 0 0) 0x7f26080bdbd0 con 0x7f26080bd780 2023-05-16T07:36:22.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260dae2700 1 --2- 172.21.15.92:0/2314540338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2608068060 0x7f26080bd290 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f2604009760 tx=0x7f2604009060).ready entity=mon.0 client_cookie=90186595230afb6b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:22.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260d2e1700 1 -- 172.21.15.92:0/2314540338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400b580 con 0x7f2608068060 2023-05-16T07:36:22.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260d2e1700 1 -- 172.21.15.92:0/2314540338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400b6e0 con 0x7f2608068060 2023-05-16T07:36:22.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.796+0000 7f260d2e1700 1 -- 172.21.15.92:0/2314540338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400ba60 con 0x7f2608068060 2023-05-16T07:36:22.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 -- 172.21.15.92:0/2314540338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2608068060 msgr2=0x7f26080bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 --2- 172.21.15.92:0/2314540338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2608068060 0x7f26080bd290 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f2604009760 tx=0x7f2604009060).stop 2023-05-16T07:36:22.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 -- 172.21.15.92:0/2314540338 shutdown_connections 2023-05-16T07:36:22.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 --2- 172.21.15.92:0/2314540338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2608068060 0x7f26080bd290 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 -- 172.21.15.92:0/2314540338 >> 172.21.15.92:0/2314540338 conn(0x7f26080c4430 msgr2=0x7f26080c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:22.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 -- 172.21.15.92:0/2314540338 shutdown_connections 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.797+0000 7f2610547700 1 -- 172.21.15.92:0/2314540338 wait complete. 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.798+0000 7f2610547700 1 Processor -- start 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.798+0000 7f2610547700 1 -- start start 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f2610547700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2608068060 0x7f26081508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f2610547700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 0x7f26080a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f2610547700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f260813a5a0 0x7f26081650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f2610547700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26080aa1b0 con 0x7f26080bd780 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f2610547700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608168680 con 0x7f260813a5a0 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f2610547700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2608167d90 con 0x7f2608068060 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 0x7f26080a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 0x7f26080a9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51164/0 (socket says 172.21.15.92:51164) 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 -- 172.21.15.92:0/2779400427 learned_addr learned my addr 172.21.15.92:0/2779400427 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260e2e3700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2608068060 0x7f26081508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260eae4700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f260813a5a0 0x7f26081650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 -- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2608068060 msgr2=0x7f26081508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2608068060 0x7f26081508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 -- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f260813a5a0 msgr2=0x7f26081650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f260813a5a0 0x7f26081650b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 -- 172.21.15.92:0/2779400427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040092a0 con 0x7f26080bd780 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260eae4700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f260813a5a0 0x7f26081650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f260dae2700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 0x7f26080a9cc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f2604011000 tx=0x7f2604017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.799+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400b850 con 0x7f26080bd780 2023-05-16T07:36:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.800+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400b9d0 con 0x7f26080bd780 2023-05-16T07:36:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.800+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2608150f40 con 0x7f26080bd780 2023-05-16T07:36:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.800+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260401b660 con 0x7f26080bd780 2023-05-16T07:36:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.800+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2608168310 con 0x7f26080bd780 2023-05-16T07:36:22.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.801+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260806c350 con 0x7f26080bd780 2023-05-16T07:36:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.801+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f260400f070 con 0x7f26080bd780 2023-05-16T07:36:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.801+0000 7f25fa7fc700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 0x7f25ec0612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:22.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.801+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f260407ba30 con 0x7f26080bd780 2023-05-16T07:36:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.801+0000 7f260e2e3700 1 -- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 msgr2=0x7f25ec0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.801+0000 7f260e2e3700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 0x7f25ec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:22.805+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f260404cb70 con 0x7f26080bd780 2023-05-16T07:36:22.849 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:22.846+0000 7f34b9cce680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:36:22.898 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:22.895+0000 7f6544971680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:36:22.925 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:22.922+0000 7f34b9cce680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:36:22.966 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:22.963+0000 7f6544971680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:36:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.002+0000 7f260e2e3700 1 -- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 msgr2=0x7f25ec0612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.002+0000 7f260e2e3700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 0x7f25ec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:36:23.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.022+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2608068e70 con 0x7f26080bd780 2023-05-16T07:36:23.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.023+0000 7f25fa7fc700 1 -- 172.21.15.92:0/2779400427 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f2604006b30 con 0x7f26080bd780 2023-05-16T07:36:23.026 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:23.026 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":49,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:19:49.065391+0000","modified":"2023-05-16T07:19:51.075477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6835/2186588149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":2186588149},{"type":"v1","addr":"172.21.15.92:6835","nonce":2186588149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-05-16T07:36:23.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.024+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 msgr2=0x7f25ec0612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:23.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.024+0000 7f2610547700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 0x7f25ec0612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:23.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.024+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 msgr2=0x7f26080a9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.024+0000 7f2610547700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 0x7f26080a9cc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f2604011000 tx=0x7f2604017000).stop 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 shutdown_connections 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f25ec05ee20 0x7f25ec0612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2608068060 0x7f26081508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f260813a5a0 0x7f26081650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 --2- 172.21.15.92:0/2779400427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26080bd780 0x7f26080a9cc0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 >> 172.21.15.92:0/2779400427 conn(0x7f26080c4430 msgr2=0x7f26080c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 shutdown_connections 2023-05-16T07:36:23.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.025+0000 7f2610547700 1 -- 172.21.15.92:0/2779400427 wait complete. 2023-05-16T07:36:23.030 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 49 2023-05-16T07:36:23.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:23.076 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:23.073+0000 7f34b9cce680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:36:23.078 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:23.075+0000 7f6544971680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:36:23.149 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:23.145+0000 7f34b9cce680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:36:23.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 Processor -- start 2023-05-16T07:36:23.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 -- start start 2023-05-16T07:36:23.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2874064280 0x7f28741506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2874151190 0x7f28740c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2874065450 con 0x7f2874150db0 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28740bd380 con 0x7f2874151190 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.152+0000 7f28788ca700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28741515b0 con 0x7f2874064280 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f287259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2874150db0 0x7f287414ffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33688/0 (socket says 172.21.15.92:33688) 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f287259c700 1 -- 172.21.15.92:0/1907651396 learned_addr learned my addr 172.21.15.92:0/1907651396 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2872d9d700 1 --2- 172.21.15.92:0/1907651396 >> v2:172.21.15.92:3300/0 conn(0x7f2874151190 0x7f28740c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f286bfff700 1 --2- 172.21.15.92:0/1907651396 >> v2:172.21.15.158:3300/0 conn(0x7f2874064280 0x7f28741506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2871d9b700 1 -- 172.21.15.92:0/1907651396 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802691952 0 0) 0x7f2874065450 con 0x7f2874150db0 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2871d9b700 1 -- 172.21.15.92:0/1907651396 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28600033f0 con 0x7f2874150db0 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2872d9d700 1 -- 172.21.15.92:0/1907651396 >> v2:172.21.15.158:3300/0 conn(0x7f2874064280 msgr2=0x7f28741506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2872d9d700 1 --2- 172.21.15.92:0/1907651396 >> v2:172.21.15.158:3300/0 conn(0x7f2874064280 0x7f28741506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2872d9d700 1 -- 172.21.15.92:0/1907651396 >> v1:172.21.15.92:6789/0 conn(0x7f2874150db0 legacy=0x7f287414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2872d9d700 1 -- 172.21.15.92:0/1907651396 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740c1370 con 0x7f2874151190 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2872d9d700 1 --2- 172.21.15.92:0/1907651396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874151190 0x7f28740c0e80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f2874155260 tx=0x7f286c009780).ready entity=mon.0 client_cookie=77661282befc2596 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:23.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2871d9b700 1 -- 172.21.15.92:0/1907651396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c00e070 con 0x7f2874151190 2023-05-16T07:36:23.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2871d9b700 1 -- 172.21.15.92:0/1907651396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c00ba70 con 0x7f2874151190 2023-05-16T07:36:23.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.153+0000 7f2871d9b700 1 -- 172.21.15.92:0/1907651396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c00bdf0 con 0x7f2874151190 2023-05-16T07:36:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 -- 172.21.15.92:0/1907651396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874151190 msgr2=0x7f28740c0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 --2- 172.21.15.92:0/1907651396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874151190 0x7f28740c0e80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f2874155260 tx=0x7f286c009780).stop 2023-05-16T07:36:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 -- 172.21.15.92:0/1907651396 shutdown_connections 2023-05-16T07:36:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 --2- 172.21.15.92:0/1907651396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874151190 0x7f28740c0e80 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 --2- 172.21.15.92:0/1907651396 >> v2:172.21.15.158:3300/0 conn(0x7f2874064280 0x7f28741506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 -- 172.21.15.92:0/1907651396 >> 172.21.15.92:0/1907651396 conn(0x7f28740c7350 msgr2=0x7f28740c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 -- 172.21.15.92:0/1907651396 shutdown_connections 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.154+0000 7f28788ca700 1 -- 172.21.15.92:0/1907651396 wait complete. 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.155+0000 7f28788ca700 1 Processor -- start 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.155+0000 7f28788ca700 1 -- start start 2023-05-16T07:36:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2874064280 0x7f28740a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 0x7f28741648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2874151190 0x7f287413cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740ab340 con 0x7f2874150db0 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740aa4d0 con 0x7f2874151190 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28740aa730 con 0x7f2874064280 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 0x7f28741648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 0x7f28741648c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51180/0 (socket says 172.21.15.92:51180) 2023-05-16T07:36:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 -- 172.21.15.92:0/2575207151 learned_addr learned my addr 172.21.15.92:0/2575207151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f2872d9d700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2874151190 0x7f287413cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f287259c700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2874064280 0x7f28740a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 -- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2874064280 msgr2=0x7f28740a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2874064280 0x7f28740a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 -- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2874151190 msgr2=0x7f287413cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2874151190 0x7f287413cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 -- 172.21.15.92:0/2575207151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c009a20 con 0x7f2874150db0 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f287259c700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2874064280 0x7f28740a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f2872d9d700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2874151190 0x7f287413cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286bfff700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 0x7f28741648c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f285c00e900 tx=0x7f285c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00c640 con 0x7f2874150db0 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00c7a0 con 0x7f2874150db0 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00cb20 con 0x7f2874150db0 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28740aaa10 con 0x7f2874150db0 2023-05-16T07:36:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.156+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28740aafd0 con 0x7f2874150db0 2023-05-16T07:36:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.157+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28740ab4b0 con 0x7f2874150db0 2023-05-16T07:36:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.158+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7f285c022070 con 0x7f2874150db0 2023-05-16T07:36:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.158+0000 7f286a7fc700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f285405ee20 0x7f28540612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.158+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f285c07a320 con 0x7f2874150db0 2023-05-16T07:36:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.158+0000 7f287259c700 1 -- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f285405ee20 msgr2=0x7f28540612e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.158+0000 7f287259c700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f285405ee20 0x7f28540612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.162+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f285c04b460 con 0x7f2874150db0 2023-05-16T07:36:23.183 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:23.181+0000 7f6544971680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:23.267 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:23.264+0000 7f34b9cce680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:36:23.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.334+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28740c4bd0 con 0x7f2874150db0 2023-05-16T07:36:23.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.334+0000 7f286a7fc700 1 -- 172.21.15.92:0/2575207151 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+18653 (secure 0 0 0) 0x7f285c016090 con 0x7f2874150db0 2023-05-16T07:36:23.338 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:23.338 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":141,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:36:21.317961+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:19:47.601645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-05-16T07:19:48.023978+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3660652651":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/4009528875":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/4145570814":"2023-05-17T07:31:20.470620+0000","172.21.15.158:0/3403657457":"2023-05-17T07:20:00.266255+0000","172.21.15.158:0/1546480240":"2023-05-17T07:20:00.266255+0000","172.21.15.158:0/117216949":"2023-05-17T07:31:20.470620+0000","172.21.15.158:6833/86301":"2023-05-17T07:20:00.266255+0000","172.21.15.158:6832/86301":"2023-05-17T07:20:00.266255+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f285405ee20 msgr2=0x7f28540612e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f285405ee20 0x7f28540612e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 msgr2=0x7f28741648c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 0x7f28741648c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f285c00e900 tx=0x7f285c009060).stop 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 shutdown_connections 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f285405ee20 0x7f28540612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2874064280 0x7f28740a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2874151190 0x7f287413cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 --2- 172.21.15.92:0/2575207151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2874150db0 0x7f28741648c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 >> 172.21.15.92:0/2575207151 conn(0x7f28740c7350 msgr2=0x7f28740c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:23.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 shutdown_connections 2023-05-16T07:36:23.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.337+0000 7f28788ca700 1 -- 172.21.15.92:0/2575207151 wait complete. 2023-05-16T07:36:23.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:36:23.377 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:23.374+0000 7f34b9cce680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:23.472 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:23.469+0000 7f6544971680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:36:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.491+0000 7ff39931c700 1 Processor -- start 2023-05-16T07:36:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 -- start start 2023-05-16T07:36:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff39413a5c0 0x7ff3940c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff39413a9a0 0x7ff394067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff394068450 0x7ff3941549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3940c3340 con 0x7ff39413a5c0 2023-05-16T07:36:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3940c2ab0 con 0x7ff39413a9a0 2023-05-16T07:36:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff39931c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940688a0 con 0x7ff394068450 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.492+0000 7ff392ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff39413a5c0 0x7ff3940c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff39413a5c0 0x7ff3940c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51186/0 (socket says 172.21.15.92:51186) 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 -- 172.21.15.92:0/247256797 learned_addr learned my addr 172.21.15.92:0/247256797 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff3937fe700 1 --2- 172.21.15.92:0/247256797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff394068450 0x7ff3941549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff3927fc700 1 --2- 172.21.15.92:0/247256797 >> v2:172.21.15.158:3300/0 conn(0x7ff39413a9a0 0x7ff394067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 -- 172.21.15.92:0/247256797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff394068450 msgr2=0x7ff3941549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 --2- 172.21.15.92:0/247256797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff394068450 0x7ff3941549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 -- 172.21.15.92:0/247256797 >> v2:172.21.15.158:3300/0 conn(0x7ff39413a9a0 msgr2=0x7ff394067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 --2- 172.21.15.92:0/247256797 >> v2:172.21.15.158:3300/0 conn(0x7ff39413a9a0 0x7ff394067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 -- 172.21.15.92:0/247256797 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940c2d10 con 0x7ff39413a5c0 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff392ffd700 1 --2- 172.21.15.92:0/247256797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39413a5c0 0x7ff3940c37d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7ff3940c2f00 tx=0x7ff3880090f0).ready entity=mon.0 client_cookie=314906d81d9d384a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff391ffb700 1 -- 172.21.15.92:0/247256797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800f510 con 0x7ff39413a5c0 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff391ffb700 1 -- 172.21.15.92:0/247256797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38800f670 con 0x7ff39413a5c0 2023-05-16T07:36:23.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.493+0000 7ff391ffb700 1 -- 172.21.15.92:0/247256797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800f9f0 con 0x7ff39413a5c0 2023-05-16T07:36:23.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 -- 172.21.15.92:0/247256797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39413a5c0 msgr2=0x7ff3940c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 --2- 172.21.15.92:0/247256797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39413a5c0 0x7ff3940c37d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7ff3940c2f00 tx=0x7ff3880090f0).stop 2023-05-16T07:36:23.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 -- 172.21.15.92:0/247256797 shutdown_connections 2023-05-16T07:36:23.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 --2- 172.21.15.92:0/247256797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff394068450 0x7ff3941549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 --2- 172.21.15.92:0/247256797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff39413a5c0 0x7ff3940c37d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 --2- 172.21.15.92:0/247256797 >> v2:172.21.15.158:3300/0 conn(0x7ff39413a9a0 0x7ff394067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 -- 172.21.15.92:0/247256797 >> 172.21.15.92:0/247256797 conn(0x7ff3940c4430 msgr2=0x7ff3940c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 -- 172.21.15.92:0/247256797 shutdown_connections 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.494+0000 7ff39931c700 1 -- 172.21.15.92:0/247256797 wait complete. 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.495+0000 7ff39931c700 1 Processor -- start 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.495+0000 7ff39931c700 1 -- start start 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff39931c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff394068450 0x7ff3940bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff39931c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff39413a5c0 0x7ff3940c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff39931c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 0x7ff394160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff39931c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940bea20 con 0x7ff3940c7140 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff39931c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940bf2c0 con 0x7ff394068450 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff39931c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940c75c0 con 0x7ff39413a5c0 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 0x7ff394160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 0x7ff394160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51196/0 (socket says 172.21.15.92:51196) 2023-05-16T07:36:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 -- 172.21.15.92:0/13940621 learned_addr learned my addr 172.21.15.92:0/13940621 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:23.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff392ffd700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff394068450 0x7ff3940bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3927fc700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff39413a5c0 0x7ff3940c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:23.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff39413a5c0 msgr2=0x7ff3940c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff39413a5c0 0x7ff3940c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff394068450 msgr2=0x7ff3940bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:23.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff394068450 0x7ff3940bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 -- 172.21.15.92:0/13940621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff388009370 con 0x7ff3940c7140 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.496+0000 7ff3937fe700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 0x7ff394160330 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7ff38c009e50 tx=0x7ff38c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.497+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00caa0 con 0x7ff3940c7140 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.497+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c00f040 con 0x7ff3940c7140 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.497+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3941610a0 con 0x7ff3940c7140 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.497+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00ce10 con 0x7ff3940c7140 2023-05-16T07:36:23.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.497+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3940bef50 con 0x7ff3940c7140 2023-05-16T07:36:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.498+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3940bf450 con 0x7ff3940c7140 2023-05-16T07:36:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.498+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 67) v1 ==== 75322+0+0 (secure 0 0 0) 0x7ff38c013610 con 0x7ff3940c7140 2023-05-16T07:36:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.498+0000 7ff37affd700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 0x7ff3800612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.498+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7207+0+0 (secure 0 0 0) 0x7ff38c07a7f0 con 0x7ff3940c7140 2023-05-16T07:36:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.498+0000 7ff392ffd700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 msgr2=0x7ff3800612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.498+0000 7ff392ffd700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 0x7ff3800612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:23.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.503+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff38c04b930 con 0x7ff3940c7140 2023-05-16T07:36:23.542 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:23.539+0000 7f6544971680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:36:23.673 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:23.670+0000 7f34b9cce680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:36:23.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.699+0000 7ff392ffd700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 msgr2=0x7ff3800612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:23.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.699+0000 7ff392ffd700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 0x7ff3800612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:36:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:23.715+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff39406c350 con 0x7ff3940c7140 2023-05-16T07:36:23.745 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:23.742+0000 7f34b9cce680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:36:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.100+0000 7ff392ffd700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 msgr2=0x7ff3800612b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.100+0000 7ff392ffd700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 0x7ff3800612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:36:24.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.336+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 68) v1 ==== 75092+0+0 (secure 0 0 0) 0x7ff38c078290 con 0x7ff3940c7140 2023-05-16T07:36:24.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.337+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 msgr2=0x7ff3800612b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:24.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.337+0000 7ff37affd700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 0x7ff3800612b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:24.389 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c4956700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.390 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c795c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.390 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c4956700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.390 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c795c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.390 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c4956700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.390 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c4956700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.391 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c4956700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.391 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c795c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.391 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c795c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.391 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:24.386+0000 7f64c795c700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:24.392 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:36:24.389+0000 7f2056292700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:36:24.392 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:36:24.389+0000 7f2056292700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:36:24.392 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-05-16T07:36:24.389+0000 7f2056292700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:36:24.406 INFO:tasks.ceph.mds.a.smithi092.stderr: -4> 2023-05-16T07:36:24.389+0000 7f2056292700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:36:24.407 INFO:tasks.ceph.mds.a.smithi092.stderr: -3> 2023-05-16T07:36:24.389+0000 7f2056292700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:36:24.407 INFO:tasks.ceph.mds.a.smithi092.stderr: -1> 2023-05-16T07:36:24.389+0000 7f2056292700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:36:24.425 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:36:24.556 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:36:24.553+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:36:24.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.558+0000 7ff37affd700 1 -- 172.21.15.92:0/13940621 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff38c06e0d0 con 0x7ff3940c7140 2023-05-16T07:36:24.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 msgr2=0x7ff394160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 0x7ff394160330 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7ff38c009e50 tx=0x7ff38c00b040).stop 2023-05-16T07:36:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 shutdown_connections 2023-05-16T07:36:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3940c7140 0x7ff394160330 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff394068450 0x7ff3940bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff38005edf0 0x7ff3800612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 --2- 172.21.15.92:0/13940621 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff39413a5c0 0x7ff3940c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 >> 172.21.15.92:0/13940621 conn(0x7ff3940c4430 msgr2=0x7ff394066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 shutdown_connections 2023-05-16T07:36:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.560+0000 7ff39931c700 1 -- 172.21.15.92:0/13940621 wait complete. 2023-05-16T07:36:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:36:24.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:36:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.685+0000 7fcf9f0f8700 1 Processor -- start 2023-05-16T07:36:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.685+0000 7fcf9f0f8700 1 -- start start 2023-05-16T07:36:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf9f0f8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf980aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf9f0f8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf980c94d0 con 0x7fcf98154470 2023-05-16T07:36:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf9f0f8700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf980cb7e0 con 0x7fcf98154090 2023-05-16T07:36:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf9f0f8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980c88c0 con 0x7fcf98068870 2023-05-16T07:36:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf97fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcf98154470 0x7fcf98068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33744/0 (socket says 172.21.15.92:33744) 2023-05-16T07:36:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf97fff700 1 -- 172.21.15.92:0/3483361473 learned_addr learned my addr 172.21.15.92:0/3483361473 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf9d695700 1 --2- 172.21.15.92:0/3483361473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf980aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.686+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208402613 0 0) 0x7fcf980c94d0 con 0x7fcf98154470 2023-05-16T07:36:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf800033f0 con 0x7fcf98154470 2023-05-16T07:36:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189371411 0 0) 0x7fcf980cb7e0 con 0x7fcf98154090 2023-05-16T07:36:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf980c94d0 con 0x7fcf98154090 2023-05-16T07:36:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf9d695700 1 -- 172.21.15.92:0/3483361473 >> v1:172.21.15.158:6789/0 conn(0x7fcf98154090 legacy=0x7fcf980a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf9d695700 1 -- 172.21.15.92:0/3483361473 >> v1:172.21.15.92:6789/0 conn(0x7fcf98154470 legacy=0x7fcf98068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf9d695700 1 -- 172.21.15.92:0/3483361473 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf980c8ba0 con 0x7fcf98068870 2023-05-16T07:36:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf9d695700 1 --2- 172.21.15.92:0/3483361473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf980aaca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fcf90009e90 tx=0x7fcf90009780).ready entity=mon.2 client_cookie=183cf539b31beb48 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000e070 con 0x7fcf98068870 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9000bab0 con 0x7fcf98068870 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf977fe700 1 -- 172.21.15.92:0/3483361473 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000be30 con 0x7fcf98068870 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3483361473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 msgr2=0x7fcf980aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.687+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3483361473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf980aaca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fcf90009e90 tx=0x7fcf90009780).stop 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.688+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3483361473 shutdown_connections 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.688+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3483361473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf980aaca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.688+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3483361473 >> 172.21.15.92:0/3483361473 conn(0x7fcf98067920 msgr2=0x7fcf980cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:24.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.688+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3483361473 shutdown_connections 2023-05-16T07:36:24.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.688+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3483361473 wait complete. 2023-05-16T07:36:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.689+0000 7fcf9f0f8700 1 Processor -- start 2023-05-16T07:36:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.689+0000 7fcf9f0f8700 1 -- start start 2023-05-16T07:36:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.689+0000 7fcf9f0f8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf98156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf98154090 0x7fcf98154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 0x7fcf9814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980b0c70 con 0x7fcf98154470 2023-05-16T07:36:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf98155780 con 0x7fcf98154090 2023-05-16T07:36:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf98167aa0 con 0x7fcf98068870 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 0x7fcf9814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 0x7fcf9814e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51236/0 (socket says 172.21.15.92:51236) 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 -- 172.21.15.92:0/3471879543 learned_addr learned my addr 172.21.15.92:0/3471879543 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf97fff700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf98154090 0x7fcf98154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9ce94700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf98156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 -- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 msgr2=0x7fcf98156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf98156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 -- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf98154090 msgr2=0x7fcf98154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf98154090 0x7fcf98154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 -- 172.21.15.92:0/3471879543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90009a20 con 0x7fcf98154470 2023-05-16T07:36:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9d695700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 0x7fcf9814e950 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fcf9806bb30 tx=0x7fcf90009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9001c660 con 0x7fcf98154470 2023-05-16T07:36:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9001c7e0 con 0x7fcf98154470 2023-05-16T07:36:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf98154ee0 con 0x7fcf98154470 2023-05-16T07:36:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9001cba0 con 0x7fcf98154470 2023-05-16T07:36:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.690+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf981554a0 con 0x7fcf98154470 2023-05-16T07:36:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.691+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf98066730 con 0x7fcf98154470 2023-05-16T07:36:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.692+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 68) v1 ==== 75092+0+0 (secure 0 0 0) 0x7fcf90027460 con 0x7fcf98154470 2023-05-16T07:36:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.692+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 7507+0+0 (secure 0 0 0) 0x7fcf9007c2d0 con 0x7fcf98154470 2023-05-16T07:36:24.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.696+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcf9004d340 con 0x7fcf98154470 2023-05-16T07:36:24.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:24.901+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcf9806c350 con 0x7fcf98154470 2023-05-16T07:36:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.340+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 69) v1 ==== 75162+0+0 (secure 0 0 0) 0x7fcf90079d00 con 0x7fcf98154470 2023-05-16T07:36:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.340+0000 7fcf94ff9700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcf840301b0 0x7fcf84068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.340+0000 7fcf9ce94700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcf840301b0 0x7fcf84068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.342+0000 7fcf9ce94700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcf840301b0 0x7fcf84068070 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fcf88000c00 tx=0x7fcf88009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:25.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.559+0000 7fcf94ff9700 1 -- 172.21.15.92:0/3471879543 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf90055340 con 0x7fcf98154470 2023-05-16T07:36:25.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcf840301b0 msgr2=0x7fcf84068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcf840301b0 0x7fcf84068070 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fcf88000c00 tx=0x7fcf88009040).stop 2023-05-16T07:36:25.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 msgr2=0x7fcf9814e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 0x7fcf9814e950 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fcf9806bb30 tx=0x7fcf90009780).stop 2023-05-16T07:36:25.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 shutdown_connections 2023-05-16T07:36:25.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcf840301b0 0x7fcf84068070 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcf98068870 0x7fcf98156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf98154470 0x7fcf9814e950 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 --2- 172.21.15.92:0/3471879543 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf98154090 0x7fcf98154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 >> 172.21.15.92:0/3471879543 conn(0x7fcf98067920 msgr2=0x7fcf980bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 shutdown_connections 2023-05-16T07:36:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.561+0000 7fcf9f0f8700 1 -- 172.21.15.92:0/3471879543 wait complete. 2023-05-16T07:36:25.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:36:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.691+0000 7f1b6d84e700 1 Processor -- start 2023-05-16T07:36:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.691+0000 7f1b6d84e700 1 -- start start 2023-05-16T07:36:25.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b6d84e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68154090 0x7f1b680aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b6d84e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680c94d0 con 0x7f1b68068160 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b6d84e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680cb7e0 con 0x7f1b68154470 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b6d84e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680c88c0 con 0x7f1b68154090 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b677fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b68068160 0x7f1b68068580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33776/0 (socket says 172.21.15.92:33776) 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b677fe700 1 -- 172.21.15.92:0/1751725747 learned_addr learned my addr 172.21.15.92:0/1751725747 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b66ffd700 1 --2- 172.21.15.92:0/1751725747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68154090 0x7f1b680aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786450131 0 0) 0x7f1b680c94d0 con 0x7f1b68068160 2023-05-16T07:36:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b540033f0 con 0x7f1b68068160 2023-05-16T07:36:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769076487 0 0) 0x7f1b680cb7e0 con 0x7f1b68154470 2023-05-16T07:36:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b680c94d0 con 0x7f1b68154470 2023-05-16T07:36:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b66ffd700 1 -- 172.21.15.92:0/1751725747 >> v1:172.21.15.158:6789/0 conn(0x7f1b68154470 legacy=0x7f1b680a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b66ffd700 1 -- 172.21.15.92:0/1751725747 >> v1:172.21.15.92:6789/0 conn(0x7f1b68068160 legacy=0x7f1b68068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b66ffd700 1 -- 172.21.15.92:0/1751725747 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680c8ba0 con 0x7f1b68154090 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.692+0000 7f1b66ffd700 1 --2- 172.21.15.92:0/1751725747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68154090 0x7f1b680aaca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f1b58009e90 tx=0x7f1b58009780).ready entity=mon.2 client_cookie=3a8f84f1de850352 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800e070 con 0x7f1b68154090 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5800bab0 con 0x7f1b68154090 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b65ffb700 1 -- 172.21.15.92:0/1751725747 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800be30 con 0x7f1b68154090 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 -- 172.21.15.92:0/1751725747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68154090 msgr2=0x7f1b680aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/1751725747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68154090 0x7f1b680aaca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f1b58009e90 tx=0x7f1b58009780).stop 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 -- 172.21.15.92:0/1751725747 shutdown_connections 2023-05-16T07:36:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/1751725747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68154090 0x7f1b680aaca0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 -- 172.21.15.92:0/1751725747 >> 172.21.15.92:0/1751725747 conn(0x7f1b68067920 msgr2=0x7f1b680cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 -- 172.21.15.92:0/1751725747 shutdown_connections 2023-05-16T07:36:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.693+0000 7f1b6d84e700 1 -- 172.21.15.92:0/1751725747 wait complete. 2023-05-16T07:36:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.694+0000 7f1b6d84e700 1 Processor -- start 2023-05-16T07:36:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.694+0000 7f1b6d84e700 1 -- start start 2023-05-16T07:36:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b6d84e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68068160 0x7f1b6814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b6d84e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 0x7f1b68154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b6d84e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b68154470 0x7f1b68152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b6d84e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68156240 con 0x7f1b68154090 2023-05-16T07:36:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b6d84e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681556c0 con 0x7f1b68154470 2023-05-16T07:36:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b6d84e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681559a0 con 0x7f1b68068160 2023-05-16T07:36:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 0x7f1b68154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 0x7f1b68154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51264/0 (socket says 172.21.15.92:51264) 2023-05-16T07:36:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b667fc700 1 -- 172.21.15.92:0/316993747 learned_addr learned my addr 172.21.15.92:0/316993747 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b66ffd700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68068160 0x7f1b6814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.695+0000 7f1b677fe700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b68154470 0x7f1b68152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b667fc700 1 -- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68068160 msgr2=0x7f1b6814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b667fc700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68068160 0x7f1b6814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b667fc700 1 -- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b68154470 msgr2=0x7f1b68152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b667fc700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b68154470 0x7f1b68152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b667fc700 1 -- 172.21.15.92:0/316993747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58009a20 con 0x7f1b68154090 2023-05-16T07:36:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b667fc700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 0x7f1b68154ce0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f1b5000c420 tx=0x7f1b50007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50012400 con 0x7f1b68154090 2023-05-16T07:36:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50012560 con 0x7f1b68154090 2023-05-16T07:36:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b68155c80 con 0x7f1b68154090 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50012920 con 0x7f1b68154090 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.696+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6814de50 con 0x7f1b68154090 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.697+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b68066730 con 0x7f1b68154090 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.697+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 69) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f1b50019030 con 0x7f1b68154090 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.697+0000 7f1b5effd700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1b4805ed20 0x7f1b480611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.697+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 7507+0+0 (secure 0 0 0) 0x7f1b500795b0 con 0x7f1b68154090 2023-05-16T07:36:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.698+0000 7f1b66ffd700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1b4805ed20 0x7f1b480611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:25.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.698+0000 7f1b66ffd700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1b4805ed20 0x7f1b480611e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1b58001190 tx=0x7f1b58009780).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:25.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.701+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1b5004a6a0 con 0x7f1b68154090 2023-05-16T07:36:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:25.866+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b6806c350 con 0x7f1b68154090 2023-05-16T07:36:26.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.561+0000 7f1b5effd700 1 -- 172.21.15.92:0/316993747 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1b5004dcc0 con 0x7f1b68154090 2023-05-16T07:36:26.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1b4805ed20 msgr2=0x7f1b480611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1b4805ed20 0x7f1b480611e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1b58001190 tx=0x7f1b58009780).stop 2023-05-16T07:36:26.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 msgr2=0x7f1b68154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 0x7f1b68154ce0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f1b5000c420 tx=0x7f1b50007bd0).stop 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 shutdown_connections 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1b4805ed20 0x7f1b480611e0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0x7f1b58001190 tx=0x7f1b58009780).stop 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b68068160 0x7f1b6814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b68154470 0x7f1b68152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 --2- 172.21.15.92:0/316993747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b68154090 0x7f1b68154ce0 secure :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0x7f1b5000c420 tx=0x7f1b50007bd0).stop 2023-05-16T07:36:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 >> 172.21.15.92:0/316993747 conn(0x7f1b68067920 msgr2=0x7f1b680c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.564+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 shutdown_connections 2023-05-16T07:36:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.565+0000 7f1b6d84e700 1 -- 172.21.15.92:0/316993747 wait complete. 2023-05-16T07:36:26.569 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:36:26.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:36:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.696+0000 7f42767e2700 1 Processor -- start 2023-05-16T07:36:26.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.696+0000 7f42767e2700 1 -- start start 2023-05-16T07:36:26.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f42767e2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f427005e460 0x7f427014f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:26.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f42767e2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42700c3170 con 0x7f427014f3b0 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f42767e2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427005e940 con 0x7f42700a7e00 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f42767e2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427005ec20 con 0x7f427005e460 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f4275fe1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f427005e460 0x7f427014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f42757e0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f427014f3b0 0x7f42700a76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33786/0 (socket says 172.21.15.92:33786) 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.697+0000 7f42757e0700 1 -- 172.21.15.92:0/3932296257 learned_addr learned my addr 172.21.15.92:0/3932296257 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f4275fe1700 1 -- 172.21.15.92:0/3932296257 >> v1:172.21.15.158:6789/0 conn(0x7f42700a7e00 legacy=0x7f42700bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f4275fe1700 1 -- 172.21.15.92:0/3932296257 >> v1:172.21.15.92:6789/0 conn(0x7f427014f3b0 legacy=0x7f42700a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f426ffff700 1 -- 172.21.15.92:0/3932296257 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988859070 0 0) 0x7f42700c3170 con 0x7f427014f3b0 2023-05-16T07:36:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f4275fe1700 1 -- 172.21.15.92:0/3932296257 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270063670 con 0x7f427005e460 2023-05-16T07:36:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f4275fe1700 1 --2- 172.21.15.92:0/3932296257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f427005e460 0x7f427014f790 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f4268009760 tx=0x7f4268009060).ready entity=mon.0 client_cookie=b26a1e37e0262863 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f426ffff700 1 -- 172.21.15.92:0/3932296257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800b580 con 0x7f427005e460 2023-05-16T07:36:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f426ffff700 1 -- 172.21.15.92:0/3932296257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426800b6e0 con 0x7f427005e460 2023-05-16T07:36:26.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f426ffff700 1 -- 172.21.15.92:0/3932296257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800ba60 con 0x7f427005e460 2023-05-16T07:36:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f42767e2700 1 -- 172.21.15.92:0/3932296257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f427005e460 msgr2=0x7f427014f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.698+0000 7f42767e2700 1 --2- 172.21.15.92:0/3932296257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f427005e460 0x7f427014f790 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f4268009760 tx=0x7f4268009060).stop 2023-05-16T07:36:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.699+0000 7f42767e2700 1 -- 172.21.15.92:0/3932296257 shutdown_connections 2023-05-16T07:36:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.699+0000 7f42767e2700 1 --2- 172.21.15.92:0/3932296257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f427005e460 0x7f427014f790 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.699+0000 7f42767e2700 1 -- 172.21.15.92:0/3932296257 >> 172.21.15.92:0/3932296257 conn(0x7f4270062d80 msgr2=0x7f427005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.699+0000 7f42767e2700 1 -- 172.21.15.92:0/3932296257 shutdown_connections 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.699+0000 7f42767e2700 1 -- 172.21.15.92:0/3932296257 wait complete. 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.700+0000 7f42767e2700 1 Processor -- start 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.700+0000 7f42767e2700 1 -- start start 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f427005e460 0x7f4270161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 0x7f42701636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f427014f3b0 0x7f4270068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4270147ec0 con 0x7f42700a7e00 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4270163250 con 0x7f427005e460 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42701623c0 con 0x7f427014f3b0 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 0x7f42701636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 0x7f42701636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51278/0 (socket says 172.21.15.92:51278) 2023-05-16T07:36:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 -- 172.21.15.92:0/2607876012 learned_addr learned my addr 172.21.15.92:0/2607876012 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42757e0700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f427005e460 0x7f4270161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4275fe1700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f427014f3b0 0x7f4270068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 -- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f427014f3b0 msgr2=0x7f4270068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f427014f3b0 0x7f4270068a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 -- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f427005e460 msgr2=0x7f4270161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f427005e460 0x7f4270161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 -- 172.21.15.92:0/2607876012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680092a0 con 0x7f42700a7e00 2023-05-16T07:36:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42757e0700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f427005e460 0x7f4270161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f4274fdf700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 0x7f42701636e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f426400c410 tx=0x7f4264007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264007020 con 0x7f42700a7e00 2023-05-16T07:36:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264007180 con 0x7f42700a7e00 2023-05-16T07:36:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4270162640 con 0x7f42700a7e00 2023-05-16T07:36:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264007520 con 0x7f42700a7e00 2023-05-16T07:36:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.701+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4270162c00 con 0x7f42700a7e00 2023-05-16T07:36:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.702+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42701633e0 con 0x7f42700a7e00 2023-05-16T07:36:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.703+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4264010070 con 0x7f42700a7e00 2023-05-16T07:36:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.703+0000 7f426d7fa700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f425c05ed60 0x7f425c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.703+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 7011+0+0 (secure 0 0 0) 0x7f4264079160 con 0x7f42700a7e00 2023-05-16T07:36:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.703+0000 7f42757e0700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f425c05ed60 0x7f425c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.703+0000 7f42757e0700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f425c05ed60 0x7f425c061220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4260007fd0 tx=0x7f4260009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.707+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f42640478f0 con 0x7f42700a7e00 2023-05-16T07:36:26.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:26.878+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4270047150 con 0x7f42700a7e00 2023-05-16T07:36:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.569+0000 7f426d7fa700 1 -- 172.21.15.92:0/2607876012 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) v1 ==== 169+0+0 (secure 0 0 0) 0x7f426404ddb0 con 0x7f42700a7e00 2023-05-16T07:36:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f425c05ed60 msgr2=0x7f425c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f425c05ed60 0x7f425c061220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4260007fd0 tx=0x7f4260009040).stop 2023-05-16T07:36:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 msgr2=0x7f42701636e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 0x7f42701636e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f426400c410 tx=0x7f4264007da0).stop 2023-05-16T07:36:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 shutdown_connections 2023-05-16T07:36:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f425c05ed60 0x7f425c061220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f427014f3b0 0x7f4270068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f427005e460 0x7f4270161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 --2- 172.21.15.92:0/2607876012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42700a7e00 0x7f42701636e0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 >> 172.21.15.92:0/2607876012 conn(0x7f4270062d80 msgr2=0x7f42700bf210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 shutdown_connections 2023-05-16T07:36:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.572+0000 7f42767e2700 1 -- 172.21.15.92:0/2607876012 wait complete. 2023-05-16T07:36:27.578 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:36:27.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:36:27.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.701+0000 7f0b0f59e700 1 Processor -- start 2023-05-16T07:36:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.701+0000 7f0b0f59e700 1 -- start start 2023-05-16T07:36:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0b100a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1006a140 con 0x7f0b10155380 2023-05-16T07:36:27.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0f59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1006a420 con 0x7f0b100c5d00 2023-05-16T07:36:27.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10069f60 con 0x7f0b10153f90 2023-05-16T07:36:27.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b10155380 0x7f0b100c0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33800/0 (socket says 172.21.15.92:33800) 2023-05-16T07:36:27.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0ed9d700 1 -- 172.21.15.92:0/485393813 learned_addr learned my addr 172.21.15.92:0/485393813 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:27.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132708137 0 0) 0x7f0b1006a140 con 0x7f0b10155380 2023-05-16T07:36:27.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0b100a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.702+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af80033f0 con 0x7f0b10155380 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 898380793 0 0) 0x7f0af80033f0 con 0x7f0b10155380 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 msgr2=0x7f0b100a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0b100a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 >> v1:172.21.15.158:6789/0 conn(0x7f0b100c5d00 legacy=0x7f0b100c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1006a700 con 0x7f0b10155380 2023-05-16T07:36:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f0b080017f0 con 0x7f0b10155380 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 >> v1:172.21.15.92:6789/0 conn(0x7f0b10155380 legacy=0x7f0b100c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0af8003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0af8004690 0x7f0af8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0af8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10069f60 con 0x7f0af8003910 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af800b910 con 0x7f0af8004690 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af800bad0 con 0x7f0b10153f90 2023-05-16T07:36:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0af8003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0dd9b700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0af8004690 0x7f0af8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0ed9d700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0af8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 -- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 msgr2=0x7f0af8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0af8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 -- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0af8004690 msgr2=0x7f0af8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0af8004690 0x7f0af8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 -- 172.21.15.92:0/485393813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80033f0 con 0x7f0af8003910 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0ed9d700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b10153f90 0x7f0af8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0af8003d30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f0b00009a80 tx=0x7f0b00009060).ready entity=mon.0 client_cookie=d87eee6808b3909f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009430 con 0x7f0af8003910 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000e640 con 0x7f0af8003910 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.703+0000 7f0b0d59a700 1 -- 172.21.15.92:0/485393813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000e9c0 con 0x7f0af8003910 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0f59e700 1 -- 172.21.15.92:0/485393813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 msgr2=0x7f0af8003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/485393813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0af8003d30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f0b00009a80 tx=0x7f0b00009060).stop 2023-05-16T07:36:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0ed9d700 1 -- 172.21.15.92:0/485393813 reap_dead start 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0f59e700 1 -- 172.21.15.92:0/485393813 shutdown_connections 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0f59e700 1 -- 172.21.15.92:0/485393813 >> 172.21.15.92:0/485393813 conn(0x7f0b100cb600 msgr2=0x7f0b100c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0f59e700 1 -- 172.21.15.92:0/485393813 shutdown_connections 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.704+0000 7f0b0f59e700 1 -- 172.21.15.92:0/485393813 wait complete. 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.705+0000 7f0b0f59e700 1 Processor -- start 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.705+0000 7f0b0f59e700 1 -- start start 2023-05-16T07:36:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0b10142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b100c1c80 0x7f0b100c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b100c2100 0x7f0b100c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b100c0dd0 con 0x7f0af8003910 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b100af670 con 0x7f0b100c1c80 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b100c2ee0 con 0x7f0b100c2100 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0b10142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0b10142940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51292/0 (socket says 172.21.15.92:51292) 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 -- 172.21.15.92:0/277585617 learned_addr learned my addr 172.21.15.92:0/277585617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 -- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b100c2100 msgr2=0x7f0b100c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0ed9d700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b100c2100 0x7f0b100c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b100c2100 0x7f0b100c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0dd9b700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b100c1c80 0x7f0b100c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 -- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b100c1c80 msgr2=0x7f0b100c03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b100c1c80 0x7f0b100c03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 -- 172.21.15.92:0/277585617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10156a30 con 0x7f0af8003910 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0dd9b700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b100c1c80 0x7f0b100c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0e59c700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0b10142940 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f0b0000e4b0 tx=0x7f0b0001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000ed30 con 0x7f0af8003910 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000ee90 con 0x7f0af8003910 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b100c3a20 con 0x7f0af8003910 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00026440 con 0x7f0af8003910 2023-05-16T07:36:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.706+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b100c3460 con 0x7f0af8003910 2023-05-16T07:36:27.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.707+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b10083120 con 0x7f0af8003910 2023-05-16T07:36:27.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.708+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0b0002c020 con 0x7f0af8003910 2023-05-16T07:36:27.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.708+0000 7f0afe7fc700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0af005ed90 0x7f0af0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:27.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.708+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0b0007b940 con 0x7f0af8003910 2023-05-16T07:36:27.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.708+0000 7f0b0dd9b700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0af005ed90 0x7f0af0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:27.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.709+0000 7f0b0dd9b700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0af005ed90 0x7f0af0061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0b04007a40 tx=0x7f0b04009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.712+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0b0004cd00 con 0x7f0af8003910 2023-05-16T07:36:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.878+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0b10047150 con 0x7f0af8003910 2023-05-16T07:36:27.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.882+0000 7f0afe7fc700 1 -- 172.21.15.92:0/277585617 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b000160a0 con 0x7f0af8003910 2023-05-16T07:36:27.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0af005ed90 msgr2=0x7f0af0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0af005ed90 0x7f0af0061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0b04007a40 tx=0x7f0b04009040).stop 2023-05-16T07:36:27.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 msgr2=0x7f0b10142940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:27.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0b10142940 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f0b0000e4b0 tx=0x7f0b0001a000).stop 2023-05-16T07:36:27.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 shutdown_connections 2023-05-16T07:36:27.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0af005ed90 0x7f0af0061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0b100c2100 0x7f0b100c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0af8003910 0x7f0b10142940 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 --2- 172.21.15.92:0/277585617 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0b100c1c80 0x7f0b100c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:27.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 >> 172.21.15.92:0/277585617 conn(0x7f0b100cb600 msgr2=0x7f0b100a9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:27.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 shutdown_connections 2023-05-16T07:36:27.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:27.884+0000 7f0b0f59e700 1 -- 172.21.15.92:0/277585617 wait complete. 2023-05-16T07:36:27.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:36:28.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.007+0000 7f967759e700 1 Processor -- start 2023-05-16T07:36:28.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.007+0000 7f967759e700 1 -- start start 2023-05-16T07:36:28.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.008+0000 7f967759e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f967814f3b0 0x7f967814f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.008+0000 7f967759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96780c3170 con 0x7f96780a7e00 2023-05-16T07:36:28.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.008+0000 7f967759e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967805e940 con 0x7f967805e460 2023-05-16T07:36:28.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.008+0000 7f967759e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967805ec20 con 0x7f967814f3b0 2023-05-16T07:36:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f9676d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96780a7e00 0x7f96780bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33816/0 (socket says 172.21.15.92:33816) 2023-05-16T07:36:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f9676d9d700 1 -- 172.21.15.92:0/4060718695 learned_addr learned my addr 172.21.15.92:0/4060718695 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967659c700 1 --2- 172.21.15.92:0/4060718695 >> v2:172.21.15.158:3300/0 conn(0x7f967814f3b0 0x7f967814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673289231 0 0) 0x7f96780c3170 con 0x7f96780a7e00 2023-05-16T07:36:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96640033f0 con 0x7f96780a7e00 2023-05-16T07:36:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199997103 0 0) 0x7f96640033f0 con 0x7f96780a7e00 2023-05-16T07:36:28.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 >> v2:172.21.15.158:3300/0 conn(0x7f967814f3b0 msgr2=0x7f967814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 --2- 172.21.15.92:0/4060718695 >> v2:172.21.15.158:3300/0 conn(0x7f967814f3b0 0x7f967814f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 >> v1:172.21.15.158:6789/0 conn(0x7f967805e460 legacy=0x7f96780a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678063670 con 0x7f96780a7e00 2023-05-16T07:36:28.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f9670001470 con 0x7f96780a7e00 2023-05-16T07:36:28.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967659c700 1 --2- 172.21.15.92:0/4060718695 >> v2:172.21.15.158:3300/0 conn(0x7f967814f3b0 0x7f967814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 >> v1:172.21.15.92:6789/0 conn(0x7f96780a7e00 legacy=0x7f96780bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96640038e0 0x7f9664003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9664004690 0x7f9664004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9664005000 0x7f966400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967805ec20 con 0x7f9664004690 2023-05-16T07:36:28.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966400beb0 con 0x7f96640038e0 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966400c070 con 0x7f9664005000 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f9675d9b700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9664004690 0x7f9664004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f967659c700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96640038e0 0x7f9664003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.009+0000 7f9676d9d700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9664005000 0x7f966400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9675d9b700 1 -- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9664005000 msgr2=0x7f966400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9675d9b700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9664005000 0x7f966400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9675d9b700 1 -- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96640038e0 msgr2=0x7f9664003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9675d9b700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f96640038e0 0x7f9664003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9675d9b700 1 -- 172.21.15.92:0/4060718695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640033f0 con 0x7f9664004690 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9676d9d700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9664005000 0x7f966400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9676d9d700 1 -- 172.21.15.92:0/4060718695 reap_dead start 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f9675d9b700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9664004690 0x7f9664004b10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f9660001700 tx=0x7f9660007bb0).ready entity=mon.0 client_cookie=69042e0dfec82e03 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660007020 con 0x7f9664004690 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660007180 con 0x7f9664004690 2023-05-16T07:36:28.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967559a700 1 -- 172.21.15.92:0/4060718695 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660007520 con 0x7f9664004690 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 -- 172.21.15.92:0/4060718695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9664004690 msgr2=0x7f9664004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9664004690 0x7f9664004b10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f9660001700 tx=0x7f9660007bb0).stop 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 -- 172.21.15.92:0/4060718695 shutdown_connections 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 --2- 172.21.15.92:0/4060718695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9664004690 0x7f9664004b10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 -- 172.21.15.92:0/4060718695 >> 172.21.15.92:0/4060718695 conn(0x7f9678062d80 msgr2=0x7f967805dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 -- 172.21.15.92:0/4060718695 shutdown_connections 2023-05-16T07:36:28.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.010+0000 7f967759e700 1 -- 172.21.15.92:0/4060718695 wait complete. 2023-05-16T07:36:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.011+0000 7f967759e700 1 Processor -- start 2023-05-16T07:36:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 -- start start 2023-05-16T07:36:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9664004690 0x7f967814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 0x7f96780c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f967814cc70 0x7f967814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96780ab480 con 0x7f96780c0130 2023-05-16T07:36:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967814d5e0 con 0x7f9664004690 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967759e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96780a8230 con 0x7f967814cc70 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f9675d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 0x7f96780c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f9675d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 0x7f96780c05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51312/0 (socket says 172.21.15.92:51312) 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f9675d9b700 1 -- 172.21.15.92:0/3461859388 learned_addr learned my addr 172.21.15.92:0/3461859388 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f967659c700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9664004690 0x7f967814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.012+0000 7f9676d9d700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f967814cc70 0x7f967814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9675d9b700 1 -- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f967814cc70 msgr2=0x7f967814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9675d9b700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f967814cc70 0x7f967814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9675d9b700 1 -- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9664004690 msgr2=0x7f967814c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9675d9b700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9664004690 0x7f967814c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9675d9b700 1 -- 172.21.15.92:0/3461859388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96780a84f0 con 0x7f96780c0130 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9676d9d700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f967814cc70 0x7f967814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f9675d9b700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 0x7f96780c05b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f96600069b0 tx=0x7f9660016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660006e90 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660006ff0 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96780a87d0 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966001e670 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.013+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96780a8d90 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.014+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9678047150 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.014+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9660025030 con 0x7f96780c0130 2023-05-16T07:36:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.014+0000 7f966e7fc700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f965805ed90 0x7f9658061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.014+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f966007a8c0 con 0x7f96780c0130 2023-05-16T07:36:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.015+0000 7f967659c700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f965805ed90 0x7f9658061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.015+0000 7f967659c700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f965805ed90 0x7f9658061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f96680094d0 tx=0x7f966800b040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.018+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f966004bb70 con 0x7f96780c0130 2023-05-16T07:36:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.180+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f9678063950 con 0x7f96780c0130 2023-05-16T07:36:28.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.183+0000 7f966e7fc700 1 -- 172.21.15.92:0/3461859388 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7f966004f190 con 0x7f96780c0130 2023-05-16T07:36:28.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f965805ed90 msgr2=0x7f9658061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f965805ed90 0x7f9658061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f96680094d0 tx=0x7f966800b040).stop 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 msgr2=0x7f96780c05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 0x7f96780c05b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f96600069b0 tx=0x7f9660016000).stop 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 shutdown_connections 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f965805ed90 0x7f9658061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f967814cc70 0x7f967814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9664004690 0x7f967814c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 --2- 172.21.15.92:0/3461859388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96780c0130 0x7f96780c05b0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 >> 172.21.15.92:0/3461859388 conn(0x7f9678062d80 msgr2=0x7f967814fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 shutdown_connections 2023-05-16T07:36:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.185+0000 7f967759e700 1 -- 172.21.15.92:0/3461859388 wait complete. 2023-05-16T07:36:28.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-05-16T07:36:28.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.307+0000 7fccfd290700 1 Processor -- start 2023-05-16T07:36:28.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.307+0000 7fccfd290700 1 -- start start 2023-05-16T07:36:28.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccfd290700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fccf8063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccfd290700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf8062b10 con 0x7fccf8062fa0 2023-05-16T07:36:28.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccfd290700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf8068340 con 0x7fccf8153da0 2023-05-16T07:36:28.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccfd290700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8068620 con 0x7fccf8154180 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fccf8062fa0 0x7fccf80bda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33840/0 (socket says 172.21.15.92:33840) 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf77fe700 1 -- 172.21.15.92:0/3633864850 learned_addr learned my addr 172.21.15.92:0/3633864850 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fcceffff700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fccf8063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008444218 0 0) 0x7fccf8062b10 con 0x7fccf8062fa0 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce40033f0 con 0x7fccf8062fa0 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4045879586 0 0) 0x7fcce40033f0 con 0x7fccf8062fa0 2023-05-16T07:36:28.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 msgr2=0x7fccf8063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fccf8063720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 >> v1:172.21.15.158:6789/0 conn(0x7fccf8153da0 legacy=0x7fccf8069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf80624c0 con 0x7fccf8062fa0 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fccf00017f0 con 0x7fccf8062fa0 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fcceffff700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fccf8063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 >> v1:172.21.15.92:6789/0 conn(0x7fccf8062fa0 legacy=0x7fccf80bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4003910 0x7fcce4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce4004690 0x7fcce4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fcce4005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8068620 con 0x7fcce4004690 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce400b990 con 0x7fcce4003910 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.308+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce400bb50 con 0x7fccf8154180 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce4004690 0x7fcce4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fccf6ffd700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4003910 0x7fcce4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fccf77fe700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fcce4005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 -- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 msgr2=0x7fcce4005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fcce4005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 -- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4003910 msgr2=0x7fcce4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4003910 0x7fcce4003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 -- 172.21.15.92:0/3633864850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40033f0 con 0x7fcce4004690 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fccf77fe700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8154180 0x7fcce4005050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:28.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fcceffff700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce4004690 0x7fcce4004b10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7fcce000baa0 tx=0x7fcce0008040).ready entity=mon.0 client_cookie=78350c6fe4b4abec server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000b410 con 0x7fcce4004690 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce000d5d0 con 0x7fcce4004690 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.309+0000 7fccf67fc700 1 -- 172.21.15.92:0/3633864850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000d990 con 0x7fcce4004690 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccfd290700 1 -- 172.21.15.92:0/3633864850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce4004690 msgr2=0x7fcce4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccfd290700 1 --2- 172.21.15.92:0/3633864850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce4004690 0x7fcce4004b10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7fcce000baa0 tx=0x7fcce0008040).stop 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccf77fe700 1 -- 172.21.15.92:0/3633864850 reap_dead start 2023-05-16T07:36:28.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccfd290700 1 -- 172.21.15.92:0/3633864850 shutdown_connections 2023-05-16T07:36:28.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccfd290700 1 -- 172.21.15.92:0/3633864850 >> 172.21.15.92:0/3633864850 conn(0x7fccf80c8aa0 msgr2=0x7fccf80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:28.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccfd290700 1 -- 172.21.15.92:0/3633864850 shutdown_connections 2023-05-16T07:36:28.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.310+0000 7fccfd290700 1 -- 172.21.15.92:0/3633864850 wait complete. 2023-05-16T07:36:28.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 Processor -- start 2023-05-16T07:36:28.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 -- start start 2023-05-16T07:36:28.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4004690 0x7fccf80c1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 0x7fccf81519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8151ed0 0x7fccf8156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf814ee70 con 0x7fccf814e490 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf814e910 con 0x7fcce4004690 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fccfd290700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80c2360 con 0x7fccf8151ed0 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.311+0000 7fcceffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 0x7fccf81519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 0x7fccf81519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51334/0 (socket says 172.21.15.92:51334) 2023-05-16T07:36:28.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 -- 172.21.15.92:0/3326010 learned_addr learned my addr 172.21.15.92:0/3326010 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fccf6ffd700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4004690 0x7fccf80c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fccf77fe700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8151ed0 0x7fccf8156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 -- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8151ed0 msgr2=0x7fccf8156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8151ed0 0x7fccf8156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 -- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4004690 msgr2=0x7fccf80c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4004690 0x7fccf80c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 -- 172.21.15.92:0/3326010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8152350 con 0x7fccf814e490 2023-05-16T07:36:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fccf6ffd700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4004690 0x7fccf80c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceffff700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 0x7fccf81519e0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fcce000b540 tx=0x7fcce0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0012630 con 0x7fccf814e490 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce0012790 con 0x7fccf814e490 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0012b50 con 0x7fccf814e490 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf80c1750 con 0x7fccf814e490 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.312+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf80c1d10 con 0x7fccf814e490 2023-05-16T07:36:28.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.313+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf806c350 con 0x7fccf814e490 2023-05-16T07:36:28.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.313+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fcce0025470 con 0x7fccf814e490 2023-05-16T07:36:28.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.314+0000 7fcceeffd700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fccd805ed90 0x7fccd8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.314+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcce007b6e0 con 0x7fccf814e490 2023-05-16T07:36:28.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.314+0000 7fccf6ffd700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fccd805ed90 0x7fccd8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.314+0000 7fccf6ffd700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fccd805ed90 0x7fccd8061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcce8000f60 tx=0x7fcce8009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.318+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcce004ca40 con 0x7fccf814e490 2023-05-16T07:36:28.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.542+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fccf80bf5a0 con 0x7fccf814e490 2023-05-16T07:36:28.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.570+0000 7fcceeffd700 1 -- 172.21.15.92:0/3326010 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v1166) v1 ==== 165+0+0 (secure 0 0 0) 0x7fcce002a070 con 0x7fccf814e490 2023-05-16T07:36:28.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fccd805ed90 msgr2=0x7fccd8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fccd805ed90 0x7fccd8061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcce8000f60 tx=0x7fcce8009040).stop 2023-05-16T07:36:28.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 msgr2=0x7fccf81519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 0x7fccf81519e0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fcce000b540 tx=0x7fcce0008040).stop 2023-05-16T07:36:28.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 shutdown_connections 2023-05-16T07:36:28.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fccd805ed90 0x7fccd8061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fccf8151ed0 0x7fccf8156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcce4004690 0x7fccf80c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 --2- 172.21.15.92:0/3326010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf814e490 0x7fccf81519e0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 >> 172.21.15.92:0/3326010 conn(0x7fccf80c8aa0 msgr2=0x7fccf80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:28.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 shutdown_connections 2023-05-16T07:36:28.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.572+0000 7fccfd290700 1 -- 172.21.15.92:0/3326010 wait complete. 2023-05-16T07:36:28.587 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-05-16T07:36:28.588 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-05-16T07:36:28.588 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-05-16T07:36:28.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-05-16T07:36:28.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.695+0000 7f9596541700 1 Processor -- start 2023-05-16T07:36:28.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.695+0000 7f9596541700 1 -- start start 2023-05-16T07:36:28.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9596541700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9596541700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f95900c7760 0x7f95900acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9596541700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9590158970 con 0x7f95900ad040 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9596541700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9590155340 con 0x7f95900c7760 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9596541700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9590157d60 con 0x7f95900c7380 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f959553f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9594d3e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f95900c7760 0x7f95900acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9594d3e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f95900c7760 0x7f95900acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:54978/0 (socket says 172.21.15.92:54978) 2023-05-16T07:36:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9595d40700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f95900ad040 0x7f959014dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:37458/0 (socket says 172.21.15.92:37458) 2023-05-16T07:36:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f959553f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:43912/0 (socket says 172.21.15.92:43912) 2023-05-16T07:36:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.696+0000 7f9594d3e700 1 -- 172.21.15.92:0/3652718673 learned_addr learned my addr 172.21.15.92:0/3652718673 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9594d3e700 1 -- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 msgr2=0x7f9590154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9594d3e700 1 --2- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9594d3e700 1 -- 172.21.15.92:0/3652718673 >> v1:172.21.15.158:6789/0 conn(0x7f95900ad040 legacy=0x7f959014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9594d3e700 1 -- 172.21.15.92:0/3652718673 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9590158040 con 0x7f95900c7760 2023-05-16T07:36:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f958ffff700 1 -- 172.21.15.92:0/3652718673 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854897057 0 0) 0x7f9590158970 con 0x7f95900ad040 2023-05-16T07:36:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9594d3e700 1 --2- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900acb50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f9584006fd0 tx=0x7f95840090f0).ready entity=mon.1 client_cookie=eb95e556cdb3524a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f958ffff700 1 -- 172.21.15.92:0/3652718673 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400b580 con 0x7f95900c7760 2023-05-16T07:36:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f958ffff700 1 -- 172.21.15.92:0/3652718673 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958400b6e0 con 0x7f95900c7760 2023-05-16T07:36:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f958ffff700 1 -- 172.21.15.92:0/3652718673 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400ba60 con 0x7f95900c7760 2023-05-16T07:36:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9596541700 1 -- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 msgr2=0x7f95900acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.697+0000 7f9596541700 1 --2- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900acb50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f9584006fd0 tx=0x7f95840090f0).stop 2023-05-16T07:36:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.698+0000 7f9596541700 1 -- 172.21.15.92:0/3652718673 shutdown_connections 2023-05-16T07:36:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.698+0000 7f9596541700 1 --2- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.698+0000 7f9596541700 1 --2- 172.21.15.92:0/3652718673 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900acb50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.698+0000 7f9596541700 1 -- 172.21.15.92:0/3652718673 >> 172.21.15.92:0/3652718673 conn(0x7f9590068330 msgr2=0x7f9590068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.698+0000 7f9596541700 1 -- 172.21.15.92:0/3652718673 shutdown_connections 2023-05-16T07:36:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.698+0000 7f9596541700 1 -- 172.21.15.92:0/3652718673 wait complete. 2023-05-16T07:36:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.699+0000 7f9596541700 1 Processor -- start 2023-05-16T07:36:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.699+0000 7f9596541700 1 -- start start 2023-05-16T07:36:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 0x7f95900c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900c9240 con 0x7f95900ad040 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900c8980 con 0x7f95900c7760 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95901650a0 con 0x7f95900c7380 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 0x7f95900c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 0x7f95900c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51346/0 (socket says 172.21.15.92:51346) 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 -- 172.21.15.92:0/3769127238 learned_addr learned my addr 172.21.15.92:0/3769127238 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9595d40700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9594d3e700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 -- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 msgr2=0x7f9590164bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590164bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 -- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 msgr2=0x7f95900adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 -- 172.21.15.92:0/3769127238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584009370 con 0x7f95900ad040 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f959553f700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 0x7f95900c3c30 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f9580009f20 tx=0x7f958000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000cb70 con 0x7f95900ad040 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000f040 con 0x7f95900ad040 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95900c8bf0 con 0x7f95900ad040 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000cee0 con 0x7f95900ad040 2023-05-16T07:36:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.700+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95900adf90 con 0x7f95900ad040 2023-05-16T07:36:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.701+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95900c9520 con 0x7f95900ad040 2023-05-16T07:36:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.702+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9580013610 con 0x7f95900ad040 2023-05-16T07:36:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.702+0000 7f958d7fa700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f957c05ed90 0x7f957c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.702+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f958007a700 con 0x7f95900ad040 2023-05-16T07:36:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.702+0000 7f9594d3e700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f957c05ed90 0x7f957c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.703+0000 7f9594d3e700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f957c05ed90 0x7f957c061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9584006fd0 tx=0x7f9584017000).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:28.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.706+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f958004bac0 con 0x7f95900ad040 2023-05-16T07:36:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:28.933+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f9590047150 con 0x7f95900ad040 2023-05-16T07:36:29.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.573+0000 7f958d7fa700 1 -- 172.21.15.92:0/3769127238 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v1167) v1 ==== 166+0+0 (secure 0 0 0) 0x7f958004f0e0 con 0x7f95900ad040 2023-05-16T07:36:29.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.575+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f957c05ed90 msgr2=0x7f957c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.575+0000 7f9596541700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f957c05ed90 0x7f957c061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9584006fd0 tx=0x7f9584017000).stop 2023-05-16T07:36:29.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.575+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 msgr2=0x7f95900c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.575+0000 7f9596541700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 0x7f95900c3c30 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f9580009f20 tx=0x7f958000b040).stop 2023-05-16T07:36:29.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 shutdown_connections 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95900c7760 0x7f95900adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95900ad040 0x7f95900c3c30 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f957c05ed90 0x7f957c061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 --2- 172.21.15.92:0/3769127238 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95900c7380 0x7f9590164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 >> 172.21.15.92:0/3769127238 conn(0x7f9590068330 msgr2=0x7f959014f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 shutdown_connections 2023-05-16T07:36:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.576+0000 7f9596541700 1 -- 172.21.15.92:0/3769127238 wait complete. 2023-05-16T07:36:29.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:36:29.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.704+0000 7f73f58d7700 1 Processor -- start 2023-05-16T07:36:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.704+0000 7f73f58d7700 1 -- start start 2023-05-16T07:36:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f58d7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 0x7f73f0062980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f58d7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f73f00a97c0 0x7f73f00a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f58d7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f01530a0 con 0x7f73f0065070 2023-05-16T07:36:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f58d7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f0152240 con 0x7f73f00a97c0 2023-05-16T07:36:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f58d7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00aa110 con 0x7f73f00acff0 2023-05-16T07:36:29.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f50d6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f73f0065070 0x7f73f0065520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33872/0 (socket says 172.21.15.92:33872) 2023-05-16T07:36:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.705+0000 7f73f50d6700 1 -- 172.21.15.92:0/2231202851 learned_addr learned my addr 172.21.15.92:0/2231202851 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73f48d5700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 0x7f73f0062980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 --2- 172.21.15.92:0/2231202851 >> v2:172.21.15.158:3300/0 conn(0x7f73f00a97c0 0x7f73f00a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917708628 0 0) 0x7f73f01530a0 con 0x7f73f0065070 2023-05-16T07:36:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc0033f0 con 0x7f73f0065070 2023-05-16T07:36:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669778585 0 0) 0x7f73dc0033f0 con 0x7f73f0065070 2023-05-16T07:36:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 msgr2=0x7f73f0062980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 0x7f73f0062980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 >> v2:172.21.15.158:3300/0 conn(0x7f73f00a97c0 msgr2=0x7f73f00a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:29.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 --2- 172.21.15.92:0/2231202851 >> v2:172.21.15.158:3300/0 conn(0x7f73f00a97c0 0x7f73f00a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0152490 con 0x7f73f0065070 2023-05-16T07:36:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f73e8001470 con 0x7f73f0065070 2023-05-16T07:36:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 >> v1:172.21.15.92:6789/0 conn(0x7f73f0065070 legacy=0x7f73f0065520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73dc0038e0 0x7f73dc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73dc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 0x7f73dc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0152240 con 0x7f73dc004690 2023-05-16T07:36:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00aa110 con 0x7f73dc0038e0 2023-05-16T07:36:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc00b8e0 con 0x7f73f00acff0 2023-05-16T07:36:29.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73dc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73f48d5700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73dc0038e0 0x7f73dc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73f50d6700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 0x7f73dc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 -- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 msgr2=0x7f73dc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:29.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f00acff0 0x7f73dc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 -- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73dc0038e0 msgr2=0x7f73dc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73dc0038e0 0x7f73dc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 -- 172.21.15.92:0/2231202851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc0033f0 con 0x7f73dc004690 2023-05-16T07:36:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73f48d5700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73dc0038e0 0x7f73dc003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73effff700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73dc004b10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f73f0064d20 tx=0x7f73d800b040).ready entity=mon.0 client_cookie=7106af129786cf07 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8009ac0 con 0x7f73dc004690 2023-05-16T07:36:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d800f040 con 0x7f73dc004690 2023-05-16T07:36:29.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.706+0000 7f73ef7fe700 1 -- 172.21.15.92:0/2231202851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d800eca0 con 0x7f73dc004690 2023-05-16T07:36:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f58d7700 1 -- 172.21.15.92:0/2231202851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 msgr2=0x7f73dc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f58d7700 1 --2- 172.21.15.92:0/2231202851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73dc004b10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f73f0064d20 tx=0x7f73d800b040).stop 2023-05-16T07:36:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f50d6700 1 -- 172.21.15.92:0/2231202851 reap_dead start 2023-05-16T07:36:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f58d7700 1 -- 172.21.15.92:0/2231202851 shutdown_connections 2023-05-16T07:36:29.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f58d7700 1 -- 172.21.15.92:0/2231202851 >> 172.21.15.92:0/2231202851 conn(0x7f73f0068a30 msgr2=0x7f73f00c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f58d7700 1 -- 172.21.15.92:0/2231202851 shutdown_connections 2023-05-16T07:36:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.707+0000 7f73f58d7700 1 -- 172.21.15.92:0/2231202851 wait complete. 2023-05-16T07:36:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.708+0000 7f73f58d7700 1 Processor -- start 2023-05-16T07:36:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.708+0000 7f73f58d7700 1 -- start start 2023-05-16T07:36:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f58d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73f0145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f58d7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f01552b0 0x7f73f00c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f58d7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73f0154da0 0x7f73f00c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f58d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00c7300 con 0x7f73dc004690 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f58d7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0164f60 con 0x7f73f0154da0 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f58d7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0164350 con 0x7f73f01552b0 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73f0145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73f0145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51366/0 (socket says 172.21.15.92:51366) 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 -- 172.21.15.92:0/1767581902 learned_addr learned my addr 172.21.15.92:0/1767581902 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:29.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f50d6700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73f0154da0 0x7f73f00c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73effff700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f01552b0 0x7f73f00c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 -- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f01552b0 msgr2=0x7f73f00c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f01552b0 0x7f73f00c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 -- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73f0154da0 msgr2=0x7f73f00c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73f0154da0 0x7f73f00c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 -- 172.21.15.92:0/1767581902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0164630 con 0x7f73dc004690 2023-05-16T07:36:29.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.709+0000 7f73f48d5700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73f0145570 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f73e0011960 tx=0x7f73e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.710+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000f6d0 con 0x7f73dc004690 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.710+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e000f830 con 0x7f73dc004690 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.710+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f0164910 con 0x7f73dc004690 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.710+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000fbb0 con 0x7f73dc004690 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.710+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f0145a60 con 0x7f73dc004690 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.711+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f0047150 con 0x7f73dc004690 2023-05-16T07:36:29.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.711+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f73e0025070 con 0x7f73dc004690 2023-05-16T07:36:29.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.711+0000 7f73ecff9700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f73d005ed90 0x7f73d0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:29.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.711+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f73e007a480 con 0x7f73dc004690 2023-05-16T07:36:29.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.711+0000 7f73effff700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f73d005ed90 0x7f73d0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:29.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.712+0000 7f73effff700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f73d005ed90 0x7f73d0061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f73d800afd0 tx=0x7f73d801a000).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:29.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.715+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f73e004b840 con 0x7f73dc004690 2023-05-16T07:36:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.878+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f73f0152770 con 0x7f73dc004690 2023-05-16T07:36:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.883+0000 7f73ecff9700 1 -- 172.21.15.92:0/1767581902 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f73e004ee60 con 0x7f73dc004690 2023-05-16T07:36:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f73d005ed90 msgr2=0x7f73d0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f73d005ed90 0x7f73d0061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f73d800afd0 tx=0x7f73d801a000).stop 2023-05-16T07:36:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 msgr2=0x7f73f0145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73f0145570 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f73e0011960 tx=0x7f73e0008040).stop 2023-05-16T07:36:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 shutdown_connections 2023-05-16T07:36:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73f0154da0 0x7f73f00c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73dc004690 0x7f73f0145570 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f73d005ed90 0x7f73d0061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 --2- 172.21.15.92:0/1767581902 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f73f01552b0 0x7f73f00c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 >> 172.21.15.92:0/1767581902 conn(0x7f73f0068a30 msgr2=0x7f73f00c5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 shutdown_connections 2023-05-16T07:36:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:29.885+0000 7f73f58d7700 1 -- 172.21.15.92:0/1767581902 wait complete. 2023-05-16T07:36:29.900 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:36:29.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:36:29.927 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:36:29.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:36:29.960 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:36:29.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.075+0000 7fc2b99c5700 1 Processor -- start 2023-05-16T07:36:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.075+0000 7fc2b99c5700 1 -- start start 2023-05-16T07:36:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b99c5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b4154e50 0x7fc2b40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b99c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2b40ad040 0x7fc2b414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b99c5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b4158970 con 0x7fc2b40c7380 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b99c5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b4157d60 con 0x7fc2b40ad040 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b99c5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4158040 con 0x7fc2b4154e50 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b91c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2b40ad040 0x7fc2b414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b89c3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc2b40c7380 0x7fc2b40c7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33890/0 (socket says 172.21.15.92:33890) 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b89c3700 1 -- 172.21.15.92:0/1812851712 learned_addr learned my addr 172.21.15.92:0/1812851712 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b3fff700 1 --2- 172.21.15.92:0/1812851712 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b4154e50 0x7fc2b40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b91c4700 1 -- 172.21.15.92:0/1812851712 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b4154e50 msgr2=0x7fc2b40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:30.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b91c4700 1 --2- 172.21.15.92:0/1812851712 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b4154e50 0x7fc2b40acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b91c4700 1 -- 172.21.15.92:0/1812851712 >> v1:172.21.15.92:6789/0 conn(0x7fc2b40c7380 legacy=0x7fc2b40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b37fe700 1 -- 172.21.15.92:0/1812851712 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095421656 0 0) 0x7fc2b4158970 con 0x7fc2b40c7380 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b91c4700 1 -- 172.21.15.92:0/1812851712 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b4158320 con 0x7fc2b40ad040 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b91c4700 1 --2- 172.21.15.92:0/1812851712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40ad040 0x7fc2b414dd70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fc29c009760 tx=0x7fc29c009060).ready entity=mon.0 client_cookie=db696fffe0cdd18d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b37fe700 1 -- 172.21.15.92:0/1812851712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00b580 con 0x7fc2b40ad040 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b37fe700 1 -- 172.21.15.92:0/1812851712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c00b6e0 con 0x7fc2b40ad040 2023-05-16T07:36:30.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.076+0000 7fc2b37fe700 1 -- 172.21.15.92:0/1812851712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00ba60 con 0x7fc2b40ad040 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 -- 172.21.15.92:0/1812851712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40ad040 msgr2=0x7fc2b414dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/1812851712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40ad040 0x7fc2b414dd70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fc29c009760 tx=0x7fc29c009060).stop 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 -- 172.21.15.92:0/1812851712 shutdown_connections 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/1812851712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40ad040 0x7fc2b414dd70 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/1812851712 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b4154e50 0x7fc2b40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 -- 172.21.15.92:0/1812851712 >> 172.21.15.92:0/1812851712 conn(0x7fc2b4068330 msgr2=0x7fc2b4068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 -- 172.21.15.92:0/1812851712 shutdown_connections 2023-05-16T07:36:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.077+0000 7fc2b99c5700 1 -- 172.21.15.92:0/1812851712 wait complete. 2023-05-16T07:36:30.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.078+0000 7fc2b99c5700 1 Processor -- start 2023-05-16T07:36:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.078+0000 7fc2b99c5700 1 -- start start 2023-05-16T07:36:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b99c5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b40ad040 0x7fc2b4146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b99c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 0x7fc2b4152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b99c5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2b4154e50 0x7fc2b4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b99c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40c4ad0 con 0x7fc2b40c7380 2023-05-16T07:36:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b99c5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40c1f50 con 0x7fc2b4154e50 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b99c5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4152f30 con 0x7fc2b40ad040 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 0x7fc2b4152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 0x7fc2b4152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51382/0 (socket says 172.21.15.92:51382) 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 -- 172.21.15.92:0/3539783911 learned_addr learned my addr 172.21.15.92:0/3539783911 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b91c4700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2b4154e50 0x7fc2b4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b89c3700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b40ad040 0x7fc2b4146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 -- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b40ad040 msgr2=0x7fc2b4146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b40ad040 0x7fc2b4146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 -- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2b4154e50 msgr2=0x7fc2b4152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2b4154e50 0x7fc2b4152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 -- 172.21.15.92:0/3539783911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c0092a0 con 0x7fc2b40c7380 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b89c3700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b40ad040 0x7fc2b4146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b91c4700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2b4154e50 0x7fc2b4152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.079+0000 7fc2b3fff700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 0x7fc2b4152550 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7fc2a400cfc0 tx=0x7fc2a40078b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.080+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4016400 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.080+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a4016560 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.080+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b40c4230 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.080+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40168e0 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.080+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b40c47f0 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.080+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b4047150 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.081+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fc2a400e400 con 0x7fc2b40c7380 2023-05-16T07:36:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.081+0000 7fc2b0ff9700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc29405ed90 0x7fc294061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.081+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc2a407ac20 con 0x7fc2b40c7380 2023-05-16T07:36:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.081+0000 7fc2b89c3700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc29405ed90 0x7fc294061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.082+0000 7fc2b89c3700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc29405ed90 0x7fc294061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc2ac001750 tx=0x7fc2ac009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.085+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc2a404bf80 con 0x7fc2b40c7380 2023-05-16T07:36:30.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.288+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2b40bfb90 con 0x7fc2b40c7380 2023-05-16T07:36:30.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.288+0000 7fc2b0ff9700 1 -- 172.21.15.92:0/3539783911 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc2a404f5a0 con 0x7fc2b40c7380 2023-05-16T07:36:30.292 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:30.292 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/1573248347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1573248347},{"type":"v1","addr":"172.21.15.158:6836","nonce":1573248347}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/698704695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":698704695},{"type":"v1","addr":"172.21.15.158:6837","nonce":698704695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7872,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/4193933800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4193933800},{"type":"v1","addr":"172.21.15.92:6834","nonce":4193933800}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":8316,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/595399741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":595399741},{"type":"v1","addr":"172.21.15.92:6835","nonce":595399741}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[]} 2023-05-16T07:36:30.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc29405ed90 msgr2=0x7fc294061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc29405ed90 0x7fc294061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc2ac001750 tx=0x7fc2ac009040).stop 2023-05-16T07:36:30.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 msgr2=0x7fc2b4152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 0x7fc2b4152550 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7fc2a400cfc0 tx=0x7fc2a40078b0).stop 2023-05-16T07:36:30.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 shutdown_connections 2023-05-16T07:36:30.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc29405ed90 0x7fc294061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc2b40ad040 0x7fc2b4146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2b4154e50 0x7fc2b4152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 --2- 172.21.15.92:0/3539783911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b40c7380 0x7fc2b4152550 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 >> 172.21.15.92:0/3539783911 conn(0x7fc2b4068330 msgr2=0x7fc2b41509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:30.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 shutdown_connections 2023-05-16T07:36:30.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.290+0000 7fc2b99c5700 1 -- 172.21.15.92:0/3539783911 wait complete. 2023-05-16T07:36:30.296 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 51 2023-05-16T07:36:30.305 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:36:30.306 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:36:30.306 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:36:30.306 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:36:30.306 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:36:30.306 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:36:30.307 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:36:30.307 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:36:30.307 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:36:30.307 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:36:30.308 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:36:30.308 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:36:30.308 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:36:30.308 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:36:30.308 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:36:30.309 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:36:30.314 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:36:30.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:36:30.315 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:36:30.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:36:30.319 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:36:30.319 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:36:30.320 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:36:30.320 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:36:30.357 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:36:30.357 DEBUG:teuthology.parallel:result is None 2023-05-16T07:36:30.358 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:36:30.358 DEBUG:teuthology.parallel:result is None 2023-05-16T07:36:30.361 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:36:30.361 DEBUG:teuthology.parallel:result is None 2023-05-16T07:36:30.362 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:36:30.362 DEBUG:teuthology.parallel:result is None 2023-05-16T07:36:30.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:36:30.426 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:36:30.427 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:36:30.440 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:36:30.441 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:36:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.482+0000 7f95d7adc700 1 Processor -- start 2023-05-16T07:36:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.482+0000 7f95d7adc700 1 -- start start 2023-05-16T07:36:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d7adc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95d01555c0 0x7f95d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d7adc700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f95d0068650 0x7f95d00bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d7adc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d0155a40 con 0x7f95d0151ff0 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d7adc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00c55d0 con 0x7f95d01555c0 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d7adc700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d0155130 con 0x7f95d0068650 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5077700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95d01555c0 0x7f95d0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5878700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f95d0151ff0 0x7f95d00a9ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33928/0 (socket says 172.21.15.92:33928) 2023-05-16T07:36:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5878700 1 -- 172.21.15.92:0/136908607 learned_addr learned my addr 172.21.15.92:0/136908607 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d6079700 1 --2- 172.21.15.92:0/136908607 >> v2:172.21.15.158:3300/0 conn(0x7f95d0068650 0x7f95d00bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5077700 1 -- 172.21.15.92:0/136908607 >> v2:172.21.15.158:3300/0 conn(0x7f95d0068650 msgr2=0x7f95d00bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5077700 1 --2- 172.21.15.92:0/136908607 >> v2:172.21.15.158:3300/0 conn(0x7f95d0068650 0x7f95d00bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5077700 1 -- 172.21.15.92:0/136908607 >> v1:172.21.15.92:6789/0 conn(0x7f95d0151ff0 legacy=0x7f95d00a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d4876700 1 -- 172.21.15.92:0/136908607 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106076586 0 0) 0x7f95cc001110 con 0x7f95d0151ff0 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5077700 1 -- 172.21.15.92:0/136908607 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00aa5f0 con 0x7f95d01555c0 2023-05-16T07:36:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d6079700 1 --2- 172.21.15.92:0/136908607 >> v2:172.21.15.158:3300/0 conn(0x7f95d0068650 0x7f95d00bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d5077700 1 --2- 172.21.15.92:0/136908607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0068160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f95d0071480 tx=0x7f95c0009060).ready entity=mon.0 client_cookie=4839f6d90a13b669 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d4876700 1 -- 172.21.15.92:0/136908607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000b580 con 0x7f95d01555c0 2023-05-16T07:36:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d4876700 1 -- 172.21.15.92:0/136908607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c000b6e0 con 0x7f95d01555c0 2023-05-16T07:36:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.483+0000 7f95d4876700 1 -- 172.21.15.92:0/136908607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000ba60 con 0x7f95d01555c0 2023-05-16T07:36:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 -- 172.21.15.92:0/136908607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 msgr2=0x7f95d0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 --2- 172.21.15.92:0/136908607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0068160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f95d0071480 tx=0x7f95c0009060).stop 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 -- 172.21.15.92:0/136908607 shutdown_connections 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 --2- 172.21.15.92:0/136908607 >> v2:172.21.15.158:3300/0 conn(0x7f95d0068650 0x7f95d00bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 --2- 172.21.15.92:0/136908607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0068160 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 -- 172.21.15.92:0/136908607 >> 172.21.15.92:0/136908607 conn(0x7f95d0067920 msgr2=0x7f95d00cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 -- 172.21.15.92:0/136908607 shutdown_connections 2023-05-16T07:36:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.484+0000 7f95d7adc700 1 -- 172.21.15.92:0/136908607 wait complete. 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.485+0000 7f95d7adc700 1 Processor -- start 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.485+0000 7f95d7adc700 1 -- start start 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95d0068650 0x7f95d0168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95d0151ff0 0x7f95d014c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0167f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c4f10 con 0x7f95d01555c0 2023-05-16T07:36:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00ac3a0 con 0x7f95d0151ff0 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00aba70 con 0x7f95d0068650 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0167f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0167f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51422/0 (socket says 172.21.15.92:51422) 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 -- 172.21.15.92:0/1796782860 learned_addr learned my addr 172.21.15.92:0/1796782860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d5878700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95d0068650 0x7f95d0168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d5077700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95d0151ff0 0x7f95d014c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 -- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95d0068650 msgr2=0x7f95d0168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95d0068650 0x7f95d0168ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 -- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95d0151ff0 msgr2=0x7f95d014c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95d0151ff0 0x7f95d014c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 -- 172.21.15.92:0/1796782860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00092a0 con 0x7f95d01555c0 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d6079700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0167f10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f95c800d100 tx=0x7f95c80099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800dc40 con 0x7f95d01555c0 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800f040 con 0x7f95d01555c0 2023-05-16T07:36:30.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d00abd50 con 0x7f95d01555c0 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.486+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80133f0 con 0x7f95d01555c0 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.487+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00ac830 con 0x7f95d01555c0 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.488+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d00666c0 con 0x7f95d01555c0 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.488+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f95c80137e0 con 0x7f95d01555c0 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.488+0000 7f95c5ffb700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f95b405ed60 0x7f95b4061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.488+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f95c807a3f0 con 0x7f95d01555c0 2023-05-16T07:36:30.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.488+0000 7f95d5878700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f95b405ed60 0x7f95b4061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.489+0000 7f95d5878700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f95b405ed60 0x7f95b4061220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f95cc0075c0 tx=0x7f95cc007a10).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.492+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f95c804b750 con 0x7f95d01555c0 2023-05-16T07:36:30.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:30.668+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f95d006c350 con 0x7f95d01555c0 2023-05-16T07:36:31.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.581+0000 7f95c5ffb700 1 -- 172.21.15.92:0/1796782860 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) v1 ==== 97+0+0 (secure 0 0 0) 0x7f95c804ed70 con 0x7f95d01555c0 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f95b405ed60 msgr2=0x7f95b4061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f95b405ed60 0x7f95b4061220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f95cc0075c0 tx=0x7f95cc007a10).stop 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 msgr2=0x7f95d0167f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0167f10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f95c800d100 tx=0x7f95c80099b0).stop 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 shutdown_connections 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f95b405ed60 0x7f95b4061220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f95d0068650 0x7f95d0168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d01555c0 0x7f95d0167f10 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 --2- 172.21.15.92:0/1796782860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f95d0151ff0 0x7f95d014c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 >> 172.21.15.92:0/1796782860 conn(0x7f95d0067920 msgr2=0x7f95d0156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:31.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 shutdown_connections 2023-05-16T07:36:31.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.583+0000 7f95d7adc700 1 -- 172.21.15.92:0/1796782860 wait complete. 2023-05-16T07:36:31.589 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:36:31.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:36:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.710+0000 7ffba0ff2700 1 Processor -- start 2023-05-16T07:36:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.710+0000 7ffba0ff2700 1 -- start start 2023-05-16T07:36:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffba0ff2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 0x7ffb9c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffba0ff2700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ffb9c154550 0x7ffb9c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffba0ff2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c0c0660 con 0x7ffb9c0696a0 2023-05-16T07:36:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffba0ff2700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c0c1b00 con 0x7ffb9c154550 2023-05-16T07:36:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffba0ff2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0bfd30 con 0x7ffb9c154170 2023-05-16T07:36:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffb9c0696a0 0x7ffb9c157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33950/0 (socket says 172.21.15.92:33950) 2023-05-16T07:36:31.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9ad9d700 1 -- 172.21.15.92:0/3747967113 learned_addr learned my addr 172.21.15.92:0/3747967113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 0x7ffb9c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb99d9b700 1 --2- 172.21.15.92:0/3747967113 >> v2:172.21.15.158:3300/0 conn(0x7ffb9c154550 0x7ffb9c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582185569 0 0) 0x7ffb9c0c0660 con 0x7ffb9c0696a0 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb840033f0 con 0x7ffb9c0696a0 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1049397069 0 0) 0x7ffb840033f0 con 0x7ffb9c0696a0 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 msgr2=0x7ffb9c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.711+0000 7ffb9959a700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 0x7ffb9c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 >> v2:172.21.15.158:3300/0 conn(0x7ffb9c154550 msgr2=0x7ffb9c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 --2- 172.21.15.92:0/3747967113 >> v2:172.21.15.158:3300/0 conn(0x7ffb9c154550 0x7ffb9c0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c0c0010 con 0x7ffb9c0696a0 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7ffb8c0017f0 con 0x7ffb9c0696a0 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 >> v1:172.21.15.92:6789/0 conn(0x7ffb9c0696a0 legacy=0x7ffb9c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb840038e0 0x7ffb84003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb84004690 0x7ffb84004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 0x7ffb84005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0c1b00 con 0x7ffb840038e0 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0bfd30 con 0x7ffb84004690 2023-05-16T07:36:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8400b8e0 con 0x7ffb9c154170 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb840038e0 0x7ffb84003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9ad9d700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 0x7ffb84005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb99d9b700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb84004690 0x7ffb84004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 -- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 msgr2=0x7ffb84005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c154170 0x7ffb84005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 -- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb84004690 msgr2=0x7ffb84004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb84004690 0x7ffb84004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 -- 172.21.15.92:0/3747967113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb840033f0 con 0x7ffb840038e0 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb840038e0 0x7ffb84003d00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7ffb94009a60 tx=0x7ffb94009060).ready entity=mon.0 client_cookie=6696b13cc442e276 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94009ea0 con 0x7ffb840038e0 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94009430 con 0x7ffb840038e0 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.712+0000 7ffb9959a700 1 -- 172.21.15.92:0/3747967113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400e870 con 0x7ffb840038e0 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3747967113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb840038e0 msgr2=0x7ffb84003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3747967113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb840038e0 0x7ffb84003d00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7ffb94009a60 tx=0x7ffb94009060).stop 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffb9ad9d700 1 -- 172.21.15.92:0/3747967113 reap_dead start 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3747967113 shutdown_connections 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3747967113 >> 172.21.15.92:0/3747967113 conn(0x7ffb9c0cb700 msgr2=0x7ffb9c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3747967113 shutdown_connections 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.713+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3747967113 wait complete. 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.714+0000 7ffba0ff2700 1 Processor -- start 2023-05-16T07:36:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.714+0000 7ffba0ff2700 1 -- start start 2023-05-16T07:36:31.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.714+0000 7ffba0ff2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb840038e0 0x7ffb9c13b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c1505f0 0x7ffb9c150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 0x7ffb9c152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c14ed50 con 0x7ffb9c152620 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c150f60 con 0x7ffb840038e0 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c153290 con 0x7ffb9c1505f0 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 0x7ffb9c152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 0x7ffb9c152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51446/0 (socket says 172.21.15.92:51446) 2023-05-16T07:36:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 -- 172.21.15.92:0/3710717614 learned_addr learned my addr 172.21.15.92:0/3710717614 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb840038e0 0x7ffb9c13b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb99d9b700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c1505f0 0x7ffb9c150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 -- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c1505f0 msgr2=0x7ffb9c150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c1505f0 0x7ffb9c150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 -- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb840038e0 msgr2=0x7ffb9c13b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb840038e0 0x7ffb9c13b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 -- 172.21.15.92:0/3710717614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c153570 con 0x7ffb9c152620 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb99d9b700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c1505f0 0x7ffb9c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb840038e0 0x7ffb9c13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb9ad9d700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 0x7ffb9c152aa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7ffb9c083060 tx=0x7ffb8c001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c001030 con 0x7ffb9c152620 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c00e5a0 con 0x7ffb9c152620 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c153850 con 0x7ffb9c152620 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00e940 con 0x7ffb9c152620 2023-05-16T07:36:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.715+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c152f90 con 0x7ffb9c152620 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.716+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c0666c0 con 0x7ffb9c152620 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.717+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ffb8c00ed30 con 0x7ffb9c152620 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.717+0000 7ffb8a7fc700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ffb7c05ed60 0x7ffb7c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.717+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5909+0+0 (secure 0 0 0) 0x7ffb8c079760 con 0x7ffb9c152620 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.717+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ffb7c05ed60 0x7ffb7c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.717+0000 7ffb9a59c700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ffb7c05ed60 0x7ffb7c061220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ffb94006d50 tx=0x7ffb9401a000).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.721+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ffb8c0498f0 con 0x7ffb9c152620 2023-05-16T07:36:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.949+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffb9c153a40 con 0x7ffb9c152620 2023-05-16T07:36:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.950+0000 7ffb8a7fc700 1 -- 172.21.15.92:0/3710717614 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+5298 (secure 0 0 0) 0x7ffb8c04e800 con 0x7ffb9c152620 2023-05-16T07:36:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:36:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:36:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:36:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:36:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:36:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:36:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:36:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:36:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:36:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:36:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:36:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:36:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:36:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:36:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:36:31.977 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:36:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ffb7c05ed60 msgr2=0x7ffb7c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ffb7c05ed60 0x7ffb7c061220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ffb94006d50 tx=0x7ffb9401a000).stop 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 msgr2=0x7ffb9c152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 0x7ffb9c152aa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7ffb9c083060 tx=0x7ffb8c001500).stop 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 shutdown_connections 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb9c152620 0x7ffb9c152aa0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.951+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffb840038e0 0x7ffb9c13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.952+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ffb7c05ed60 0x7ffb7c061220 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.952+0000 7ffba0ff2700 1 --2- 172.21.15.92:0/3710717614 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ffb9c1505f0 0x7ffb9c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.952+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 >> 172.21.15.92:0/3710717614 conn(0x7ffb9c0cb700 msgr2=0x7ffb9c0acc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.952+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 shutdown_connections 2023-05-16T07:36:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:31.952+0000 7ffba0ff2700 1 -- 172.21.15.92:0/3710717614 wait complete. 2023-05-16T07:36:31.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.088+0000 7f16c2e3e700 1 Processor -- start 2023-05-16T07:36:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.088+0000 7f16c2e3e700 1 -- start start 2023-05-16T07:36:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c2e3e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16bc154170 0x7f16bc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c2e3e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f16bc154550 0x7f16bc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c2e3e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc0c0660 con 0x7f16bc0696a0 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c2e3e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0c1b00 con 0x7f16bc154170 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c2e3e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0bfd30 con 0x7f16bc154550 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c0bda700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16bc154170 0x7f16bc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c13db700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f16bc0696a0 0x7f16bc157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33970/0 (socket says 172.21.15.92:33970) 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.089+0000 7f16c13db700 1 -- 172.21.15.92:0/2417154303 learned_addr learned my addr 172.21.15.92:0/2417154303 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16b3fff700 1 --2- 172.21.15.92:0/2417154303 >> v2:172.21.15.158:3300/0 conn(0x7f16bc154550 0x7f16bc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c0bda700 1 -- 172.21.15.92:0/2417154303 >> v2:172.21.15.158:3300/0 conn(0x7f16bc154550 msgr2=0x7f16bc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c0bda700 1 --2- 172.21.15.92:0/2417154303 >> v2:172.21.15.158:3300/0 conn(0x7f16bc154550 0x7f16bc0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c0bda700 1 -- 172.21.15.92:0/2417154303 >> v1:172.21.15.92:6789/0 conn(0x7f16bc0696a0 legacy=0x7f16bc157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c0bda700 1 -- 172.21.15.92:0/2417154303 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0c0010 con 0x7f16bc154170 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16b37fe700 1 -- 172.21.15.92:0/2417154303 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745540838 0 0) 0x7f16bc0c0660 con 0x7f16bc0696a0 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16b3fff700 1 --2- 172.21.15.92:0/2417154303 >> v2:172.21.15.158:3300/0 conn(0x7f16bc154550 0x7f16bc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c0bda700 1 --2- 172.21.15.92:0/2417154303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc154170 0x7f16bc0c5c70 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f16bc0844e0 tx=0x7f16b8009780).ready entity=mon.0 client_cookie=2d8534c9528510bd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16b37fe700 1 -- 172.21.15.92:0/2417154303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800e070 con 0x7f16bc154170 2023-05-16T07:36:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16b37fe700 1 -- 172.21.15.92:0/2417154303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800bc30 con 0x7f16bc154170 2023-05-16T07:36:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16b37fe700 1 -- 172.21.15.92:0/2417154303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8012410 con 0x7f16bc154170 2023-05-16T07:36:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c2e3e700 1 -- 172.21.15.92:0/2417154303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc154170 msgr2=0x7f16bc0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.090+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/2417154303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc154170 0x7f16bc0c5c70 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f16bc0844e0 tx=0x7f16b8009780).stop 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.091+0000 7f16c2e3e700 1 -- 172.21.15.92:0/2417154303 shutdown_connections 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.091+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/2417154303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc154170 0x7f16bc0c5c70 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.091+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/2417154303 >> v2:172.21.15.158:3300/0 conn(0x7f16bc154550 0x7f16bc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.091+0000 7f16c2e3e700 1 -- 172.21.15.92:0/2417154303 >> 172.21.15.92:0/2417154303 conn(0x7f16bc0cb700 msgr2=0x7f16bc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.091+0000 7f16c2e3e700 1 -- 172.21.15.92:0/2417154303 shutdown_connections 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.091+0000 7f16c2e3e700 1 -- 172.21.15.92:0/2417154303 wait complete. 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.092+0000 7f16c2e3e700 1 Processor -- start 2023-05-16T07:36:32.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.092+0000 7f16c2e3e700 1 -- start start 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 0x7f16bc0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc154170 0x7f16bc0aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f16bc154550 0x7f16bc14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0ae730 con 0x7f16bc0696a0 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0ae3e0 con 0x7f16bc154170 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0c4690 con 0x7f16bc154550 2023-05-16T07:36:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 0x7f16bc0c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 0x7f16bc0c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51456/0 (socket says 172.21.15.92:51456) 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 -- 172.21.15.92:0/1340435064 learned_addr learned my addr 172.21.15.92:0/1340435064 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16b3fff700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc154170 0x7f16bc0aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c13db700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f16bc154550 0x7f16bc14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 -- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f16bc154550 msgr2=0x7f16bc14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f16bc154550 0x7f16bc14f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 -- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc154170 msgr2=0x7f16bc0aa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc154170 0x7f16bc0aa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 -- 172.21.15.92:0/1340435064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009a20 con 0x7f16bc0696a0 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c13db700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f16bc154550 0x7f16bc14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c0bda700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 0x7f16bc0c2960 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f16bc083060 tx=0x7f16b8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b801d630 con 0x7f16bc0696a0 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b801d790 con 0x7f16bc0696a0 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc0c4970 con 0x7f16bc0696a0 2023-05-16T07:36:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b801db50 con 0x7f16bc0696a0 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.093+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc0aa500 con 0x7f16bc0696a0 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.094+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc0aa720 con 0x7f16bc0696a0 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.095+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f16b8027460 con 0x7f16bc0696a0 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.095+0000 7f16b0ff9700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f16a805ed90 0x7f16a8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.095+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5909+0+0 (secure 0 0 0) 0x7f16b807bd10 con 0x7f16bc0696a0 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.095+0000 7f16b3fff700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f16a805ed90 0x7f16a8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.095+0000 7f16b3fff700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f16a805ed90 0x7f16a8061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f16bc153940 tx=0x7f16ac008040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.099+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f16bc0aa720 con 0x7f16bc0696a0 2023-05-16T07:36:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.262+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16bc0ae240 con 0x7f16bc0696a0 2023-05-16T07:36:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.263+0000 7f16b0ff9700 1 -- 172.21.15.92:0/1340435064 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+14744 (secure 0 0 0) 0x7f16bc0ae240 con 0x7f16bc0696a0 2023-05-16T07:36:32.266 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:32.266 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":150,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:36:31.574762+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.264+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f16a805ed90 msgr2=0x7f16a8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.264+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f16a805ed90 0x7f16a8061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f16bc153940 tx=0x7f16ac008040).stop 2023-05-16T07:36:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.264+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 msgr2=0x7f16bc0c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.264+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 0x7f16bc0c2960 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f16bc083060 tx=0x7f16b8009780).stop 2023-05-16T07:36:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 shutdown_connections 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f16a805ed90 0x7f16a8061250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f16bc154550 0x7f16bc14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16bc0696a0 0x7f16bc0c2960 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 --2- 172.21.15.92:0/1340435064 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f16bc154170 0x7f16bc0aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 >> 172.21.15.92:0/1340435064 conn(0x7f16bc0cb700 msgr2=0x7f16bc0abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 shutdown_connections 2023-05-16T07:36:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.265+0000 7f16c2e3e700 1 -- 172.21.15.92:0/1340435064 wait complete. 2023-05-16T07:36:32.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:36:32.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:36:32.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.391+0000 7fb444534700 1 Processor -- start 2023-05-16T07:36:32.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.391+0000 7fb444534700 1 -- start start 2023-05-16T07:36:32.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb444534700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb43c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb444534700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb43c068160 0x7fb43c0685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb444534700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0c94d0 con 0x7fb43c154090 2023-05-16T07:36:32.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb444534700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43c0cb7e0 con 0x7fb43c068160 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb444534700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c068ac0 con 0x7fb43c154470 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4422d0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb43c154090 0x7fb43c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33980/0 (socket says 172.21.15.92:33980) 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4422d0700 1 -- 172.21.15.92:0/488586517 learned_addr learned my addr 172.21.15.92:0/488586517 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282509358 0 0) 0x7fb43c0c94d0 con 0x7fb43c154090 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb442ad1700 1 --2- 172.21.15.92:0/488586517 >> v2:172.21.15.158:3300/0 conn(0x7fb43c068160 0x7fb43c0685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4280033f0 con 0x7fb43c154090 2023-05-16T07:36:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb441acf700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb43c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695962687 0 0) 0x7fb4280033f0 con 0x7fb43c154090 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 msgr2=0x7fb43c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb43c0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 >> v2:172.21.15.158:3300/0 conn(0x7fb43c068160 msgr2=0x7fb43c0685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 --2- 172.21.15.92:0/488586517 >> v2:172.21.15.158:3300/0 conn(0x7fb43c068160 0x7fb43c0685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0c88c0 con 0x7fb43c154090 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fb434001470 con 0x7fb43c154090 2023-05-16T07:36:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 >> v1:172.21.15.92:6789/0 conn(0x7fb43c154090 legacy=0x7fb43c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4280038e0 0x7fb428003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428004690 0x7fb428004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb428005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0cb7e0 con 0x7fb428004690 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c068ac0 con 0x7fb4280038e0 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.392+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42800b8e0 con 0x7fb43c154470 2023-05-16T07:36:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428004690 0x7fb428004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb4422d0700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4280038e0 0x7fb428003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb442ad1700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb428005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 -- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 msgr2=0x7fb428005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb428005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 -- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4280038e0 msgr2=0x7fb428003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb4280038e0 0x7fb428003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 -- 172.21.15.92:0/488586517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280033f0 con 0x7fb428004690 2023-05-16T07:36:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb442ad1700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb43c154470 0x7fb428005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb441acf700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428004690 0x7fb428004b10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fb42c009a80 tx=0x7fb42c009060).ready entity=mon.0 client_cookie=27f2e59857b3d236 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c009430 con 0x7fb428004690 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c00e640 con 0x7fb428004690 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb4412ce700 1 -- 172.21.15.92:0/488586517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00e9c0 con 0x7fb428004690 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb444534700 1 -- 172.21.15.92:0/488586517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428004690 msgr2=0x7fb428004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb444534700 1 --2- 172.21.15.92:0/488586517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428004690 0x7fb428004b10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fb42c009a80 tx=0x7fb42c009060).stop 2023-05-16T07:36:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.393+0000 7fb442ad1700 1 -- 172.21.15.92:0/488586517 reap_dead start 2023-05-16T07:36:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.394+0000 7fb444534700 1 -- 172.21.15.92:0/488586517 shutdown_connections 2023-05-16T07:36:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.394+0000 7fb444534700 1 -- 172.21.15.92:0/488586517 >> 172.21.15.92:0/488586517 conn(0x7fb43c067920 msgr2=0x7fb43c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.394+0000 7fb444534700 1 -- 172.21.15.92:0/488586517 shutdown_connections 2023-05-16T07:36:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.394+0000 7fb444534700 1 -- 172.21.15.92:0/488586517 wait complete. 2023-05-16T07:36:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 Processor -- start 2023-05-16T07:36:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 -- start start 2023-05-16T07:36:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb428004690 0x7fb43c14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 0x7fb43c1417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb43c152e10 0x7fb43c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0aec40 con 0x7fb43c1517c0 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c153780 con 0x7fb43c152e10 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.395+0000 7fb444534700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c151d10 con 0x7fb428004690 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 0x7fb43c1417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 0x7fb43c1417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51478/0 (socket says 172.21.15.92:51478) 2023-05-16T07:36:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 -- 172.21.15.92:0/1664439187 learned_addr learned my addr 172.21.15.92:0/1664439187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb442ad1700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb43c152e10 0x7fb43c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb4422d0700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb428004690 0x7fb43c14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 -- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb428004690 msgr2=0x7fb43c14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb428004690 0x7fb43c14fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 -- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb43c152e10 msgr2=0x7fb43c153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb43c152e10 0x7fb43c153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 -- 172.21.15.92:0/1664439187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c151ff0 con 0x7fb43c1517c0 2023-05-16T07:36:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb441acf700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 0x7fb43c1417b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fb428004480 tx=0x7fb42c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00ed30 con 0x7fb43c1517c0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c00ee90 con 0x7fb43c1517c0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c1522d0 con 0x7fb43c1517c0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c026440 con 0x7fb43c1517c0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.396+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c1501c0 con 0x7fb43c1517c0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.397+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c066730 con 0x7fb43c1517c0 2023-05-16T07:36:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.397+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb42c02c020 con 0x7fb43c1517c0 2023-05-16T07:36:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.398+0000 7fb4327fc700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb42005ed60 0x7fb420061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.398+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5909+0+0 (secure 0 0 0) 0x7fb42c07b590 con 0x7fb43c1517c0 2023-05-16T07:36:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.398+0000 7fb4422d0700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb42005ed60 0x7fb420061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.398+0000 7fb4422d0700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb42005ed60 0x7fb420061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb434001200 tx=0x7fb434009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.402+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb42c04cc00 con 0x7fb43c1517c0 2023-05-16T07:36:32.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.564+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb43c06c350 con 0x7fb43c1517c0 2023-05-16T07:36:32.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.564+0000 7fb4327fc700 1 -- 172.21.15.92:0/1664439187 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v150) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb42c016020 con 0x7fb43c1517c0 2023-05-16T07:36:32.568 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb42005ed60 msgr2=0x7fb420061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb42005ed60 0x7fb420061220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb434001200 tx=0x7fb434009040).stop 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 msgr2=0x7fb43c1417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 0x7fb43c1417b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fb428004480 tx=0x7fb42c01a000).stop 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 shutdown_connections 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb42005ed60 0x7fb420061220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb428004690 0x7fb43c14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb43c152e10 0x7fb43c153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 --2- 172.21.15.92:0/1664439187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb43c1517c0 0x7fb43c1417b0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 >> 172.21.15.92:0/1664439187 conn(0x7fb43c067920 msgr2=0x7fb43c0a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:32.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 shutdown_connections 2023-05-16T07:36:32.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.566+0000 7fb444534700 1 -- 172.21.15.92:0/1664439187 wait complete. 2023-05-16T07:36:32.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:36:32.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.691+0000 7f5fde3d2700 1 Processor -- start 2023-05-16T07:36:32.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.691+0000 7f5fde3d2700 1 -- start start 2023-05-16T07:36:32.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fde3d2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fd8154470 0x7f5fd80aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fde3d2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80c94d0 con 0x7f5fd8068160 2023-05-16T07:36:32.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fde3d2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80cb7e0 con 0x7f5fd8154090 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fde3d2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd80c88c0 con 0x7f5fd8154470 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fd77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fd8154470 0x7f5fd80aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fdc96f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5fd8068160 0x7f5fd80c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34002/0 (socket says 172.21.15.92:34002) 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fdc96f700 1 -- 172.21.15.92:0/950147594 learned_addr learned my addr 172.21.15.92:0/950147594 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fd77fe700 1 -- 172.21.15.92:0/950147594 >> v1:172.21.15.158:6789/0 conn(0x7f5fd8154090 legacy=0x7f5fd80a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fd77fe700 1 -- 172.21.15.92:0/950147594 >> v1:172.21.15.92:6789/0 conn(0x7f5fd8068160 legacy=0x7f5fd80c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fd77fe700 1 -- 172.21.15.92:0/950147594 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80c8ba0 con 0x7f5fd8154470 2023-05-16T07:36:32.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fd6ffd700 1 -- 172.21.15.92:0/950147594 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578664029 0 0) 0x7f5fd80c94d0 con 0x7f5fd8068160 2023-05-16T07:36:32.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.692+0000 7f5fd77fe700 1 --2- 172.21.15.92:0/950147594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154470 0x7f5fd80aaca0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f5fcc009760 tx=0x7f5fcc009060).ready entity=mon.0 client_cookie=d5bb2b0c8f247df server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fd6ffd700 1 -- 172.21.15.92:0/950147594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00b580 con 0x7f5fd8154470 2023-05-16T07:36:32.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fd6ffd700 1 -- 172.21.15.92:0/950147594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc00b6e0 con 0x7f5fd8154470 2023-05-16T07:36:32.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fd6ffd700 1 -- 172.21.15.92:0/950147594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00ba60 con 0x7f5fd8154470 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 -- 172.21.15.92:0/950147594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154470 msgr2=0x7f5fd80aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/950147594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154470 0x7f5fd80aaca0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f5fcc009760 tx=0x7f5fcc009060).stop 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 -- 172.21.15.92:0/950147594 shutdown_connections 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/950147594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154470 0x7f5fd80aaca0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 -- 172.21.15.92:0/950147594 >> 172.21.15.92:0/950147594 conn(0x7f5fd8067920 msgr2=0x7f5fd80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 -- 172.21.15.92:0/950147594 shutdown_connections 2023-05-16T07:36:32.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.693+0000 7f5fde3d2700 1 -- 172.21.15.92:0/950147594 wait complete. 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.694+0000 7f5fde3d2700 1 Processor -- start 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.694+0000 7f5fde3d2700 1 -- start start 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fde3d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5fd8068160 0x7f5fd8154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fde3d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 0x7f5fd80aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fde3d2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fd8154470 0x7f5fd813f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fde3d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8155cb0 con 0x7f5fd8154090 2023-05-16T07:36:32.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fde3d2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8155f90 con 0x7f5fd8154470 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fde3d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81550a0 con 0x7f5fd8068160 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 0x7f5fd80aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 0x7f5fd80aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51492/0 (socket says 172.21.15.92:51492) 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 -- 172.21.15.92:0/3453085777 learned_addr learned my addr 172.21.15.92:0/3453085777 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fdc96f700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fd8154470 0x7f5fd813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 -- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5fd8068160 msgr2=0x7f5fd8154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5fd8068160 0x7f5fd8154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 -- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fd8154470 msgr2=0x7f5fd813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.695+0000 7f5fd77fe700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fd8154470 0x7f5fd813f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fd77fe700 1 -- 172.21.15.92:0/3453085777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0092a0 con 0x7f5fd8154090 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fdc96f700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fd8154470 0x7f5fd813f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fd77fe700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 0x7f5fd80aef20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f5fcc009260 tx=0x7f5fcc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00ba60 con 0x7f5fd8154090 2023-05-16T07:36:32.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc000c90 con 0x7f5fd8154090 2023-05-16T07:36:32.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd8155380 con 0x7f5fd8154090 2023-05-16T07:36:32.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01b670 con 0x7f5fd8154090 2023-05-16T07:36:32.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.696+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd8154d40 con 0x7f5fd8154090 2023-05-16T07:36:32.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.697+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd8066730 con 0x7f5fd8154090 2023-05-16T07:36:32.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.697+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5fcc02a070 con 0x7f5fd8154090 2023-05-16T07:36:32.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.697+0000 7f5fbffff700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fc405ed90 0x7f5fc4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.697+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5909+0+0 (secure 0 0 0) 0x7f5fcc07b240 con 0x7f5fd8154090 2023-05-16T07:36:32.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.698+0000 7f5fd7fff700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fc405ed90 0x7f5fc4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.698+0000 7f5fd7fff700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fc405ed90 0x7f5fc4061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5fc8001510 tx=0x7f5fc80079c0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:32.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.702+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5fcc04c7a0 con 0x7f5fd8154090 2023-05-16T07:36:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.865+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5fd80c8e80 con 0x7f5fd8154090 2023-05-16T07:36:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.865+0000 7f5fbffff700 1 -- 172.21.15.92:0/3453085777 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v150) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5fcc0280a0 con 0x7f5fd8154090 2023-05-16T07:36:32.869 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:36:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fc405ed90 msgr2=0x7f5fc4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fc405ed90 0x7f5fc4061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5fc8001510 tx=0x7f5fc80079c0).stop 2023-05-16T07:36:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 msgr2=0x7f5fd80aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 0x7f5fd80aef20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f5fcc009260 tx=0x7f5fcc017000).stop 2023-05-16T07:36:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 shutdown_connections 2023-05-16T07:36:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fc405ed90 0x7f5fc4061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5fd8068160 0x7f5fd8154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fd8154470 0x7f5fd813f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 --2- 172.21.15.92:0/3453085777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd8154090 0x7f5fd80aef20 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 >> 172.21.15.92:0/3453085777 conn(0x7f5fd8067920 msgr2=0x7f5fd80c11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 shutdown_connections 2023-05-16T07:36:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.867+0000 7f5fde3d2700 1 -- 172.21.15.92:0/3453085777 wait complete. 2023-05-16T07:36:32.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:36:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.989+0000 7f2ef2363700 1 Processor -- start 2023-05-16T07:36:32.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.989+0000 7f2ef2363700 1 -- start start 2023-05-16T07:36:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2ef2363700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec13b930 0x7f2eec0c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2ef2363700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eec0bdc30 con 0x7f2eec13b550 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2ef2363700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eec0c4e40 con 0x7f2eec063e30 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2ef2363700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0c5400 con 0x7f2eec13b930 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2eebfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2eec13b550 0x7f2eec063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34014/0 (socket says 172.21.15.92:34014) 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2eebfff700 1 -- 172.21.15.92:0/2986184726 learned_addr learned my addr 172.21.15.92:0/2986184726 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212028376 0 0) 0x7f2eec0bdc30 con 0x7f2eec13b550 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed40033f0 con 0x7f2eec13b550 2023-05-16T07:36:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.990+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec13b930 0x7f2eec0c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233995684 0 0) 0x7f2ed40033f0 con 0x7f2eec13b550 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec13b930 msgr2=0x7f2eec0c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec13b930 0x7f2eec0c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 >> v1:172.21.15.158:6789/0 conn(0x7f2eec063e30 legacy=0x7f2eec0be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0c5120 con 0x7f2eec13b550 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f2edc001470 con 0x7f2eec13b550 2023-05-16T07:36:32.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 >> v1:172.21.15.92:6789/0 conn(0x7f2eec13b550 legacy=0x7f2eec063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4003910 0x7f2ed4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2ed4004690 0x7f2ed4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4005000 0x7f2ed400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0c5400 con 0x7f2ed4005000 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed400beb0 con 0x7f2ed4003910 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed400c070 con 0x7f2ed4004690 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec13b930 0x7f2eec0c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:32.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4005000 0x7f2ed400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eebfff700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4003910 0x7f2ed4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeb7fe700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2ed4004690 0x7f2ed4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 -- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2ed4004690 msgr2=0x7f2ed4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2ed4004690 0x7f2ed4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 -- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4003910 msgr2=0x7f2ed4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4003910 0x7f2ed4003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 -- 172.21.15.92:0/2986184726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40033f0 con 0x7f2ed4005000 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4005000 0x7f2ed400b8e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f2eec13bd10 tx=0x7f2ee40097b0).ready entity=mon.0 client_cookie=41990ac646c1b91e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.991+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4009aa0 con 0x7f2ed4005000 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee400f040 con 0x7f2ed4005000 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2eeaffd700 1 -- 172.21.15.92:0/2986184726 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4007da0 con 0x7f2ed4005000 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef2363700 1 -- 172.21.15.92:0/2986184726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4005000 msgr2=0x7f2ed400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2986184726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4005000 0x7f2ed400b8e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f2eec13bd10 tx=0x7f2ee40097b0).stop 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef0900700 1 -- 172.21.15.92:0/2986184726 reap_dead start 2023-05-16T07:36:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef2363700 1 -- 172.21.15.92:0/2986184726 shutdown_connections 2023-05-16T07:36:33.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef2363700 1 -- 172.21.15.92:0/2986184726 >> 172.21.15.92:0/2986184726 conn(0x7f2eec0c8aa0 msgr2=0x7f2eec0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:33.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef2363700 1 -- 172.21.15.92:0/2986184726 shutdown_connections 2023-05-16T07:36:33.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.992+0000 7f2ef2363700 1 -- 172.21.15.92:0/2986184726 wait complete. 2023-05-16T07:36:33.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.993+0000 7f2ef2363700 1 Processor -- start 2023-05-16T07:36:33.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.993+0000 7f2ef2363700 1 -- start start 2023-05-16T07:36:33.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef2363700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4005000 0x7f2eec0c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef2363700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec12c440 0x7f2eec0622b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef2363700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 0x7f2eec0c1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef2363700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0c11c0 con 0x7f2eec0627a0 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef2363700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0c0840 con 0x7f2ed4005000 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef2363700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec062c20 con 0x7f2eec12c440 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 0x7f2eec0c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 0x7f2eec0c1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51512/0 (socket says 172.21.15.92:51512) 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 -- 172.21.15.92:0/2148977713 learned_addr learned my addr 172.21.15.92:0/2148977713 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2eebfff700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4005000 0x7f2eec0c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2eeb7fe700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec12c440 0x7f2eec0622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 -- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec12c440 msgr2=0x7f2eec0622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec12c440 0x7f2eec0622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 -- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4005000 msgr2=0x7f2eec0c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4005000 0x7f2eec0c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 -- 172.21.15.92:0/2148977713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0bfc70 con 0x7f2eec0627a0 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2eebfff700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4005000 0x7f2eec0c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:33.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.994+0000 7f2ef0900700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 0x7f2eec0c1970 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f2eec13bd40 tx=0x7f2ee4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.995+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4009aa0 con 0x7f2eec0627a0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.995+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee4014e30 con 0x7f2eec0627a0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.995+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec12c8c0 con 0x7f2eec0627a0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.995+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee401f6a0 con 0x7f2eec0627a0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.995+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec0c01f0 con 0x7f2eec0627a0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.996+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec06c350 con 0x7f2eec0627a0 2023-05-16T07:36:33.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.996+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2ee4026030 con 0x7f2eec0627a0 2023-05-16T07:36:33.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.996+0000 7f2ed3fff700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2ed805ed90 0x7f2ed8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.996+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5909+0+0 (secure 0 0 0) 0x7f2ee407b120 con 0x7f2eec0627a0 2023-05-16T07:36:33.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.997+0000 7f2eebfff700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2ed805ed90 0x7f2ed8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:32.997+0000 7f2eebfff700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2ed805ed90 0x7f2ed8061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2edc0015a0 tx=0x7f2edc009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.001+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2ee404c840 con 0x7f2eec0627a0 2023-05-16T07:36:33.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.165+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2eec0c5ca0 con 0x7f2eec0627a0 2023-05-16T07:36:33.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.165+0000 7f2ed3fff700 1 -- 172.21.15.92:0/2148977713 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2ee4024020 con 0x7f2eec0627a0 2023-05-16T07:36:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:33.169 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:36:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:36:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:36:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:36:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:36:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:36:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:33.170 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:36:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2ed805ed90 msgr2=0x7f2ed8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2ed805ed90 0x7f2ed8061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2edc0015a0 tx=0x7f2edc009040).stop 2023-05-16T07:36:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 msgr2=0x7f2eec0c1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 0x7f2eec0c1970 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f2eec13bd40 tx=0x7f2ee4017000).stop 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 shutdown_connections 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2eec0627a0 0x7f2eec0c1970 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2ed4005000 0x7f2eec0c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2ed805ed90 0x7f2ed8061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 --2- 172.21.15.92:0/2148977713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2eec12c440 0x7f2eec0622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 >> 172.21.15.92:0/2148977713 conn(0x7f2eec0c8aa0 msgr2=0x7f2eec0c79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 shutdown_connections 2023-05-16T07:36:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.167+0000 7f2ef2363700 1 -- 172.21.15.92:0/2148977713 wait complete. 2023-05-16T07:36:33.182 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:36:33.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:36:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.294+0000 7fd3b2d94700 1 Processor -- start 2023-05-16T07:36:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.294+0000 7fd3b2d94700 1 -- start start 2023-05-16T07:36:33.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b2d94700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd3ac0699c0 0x7fd3ac155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b2d94700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd3ac0bd3b0 0x7fd3ac0bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b2d94700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ac151c70 con 0x7fd3ac154f70 2023-05-16T07:36:33.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b2d94700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac152100 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b2d94700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac151120 con 0x7fd3ac0699c0 2023-05-16T07:36:33.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b1331700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd3ac0bd3b0 0x7fd3ac0bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b0b30700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd3ac154f70 0x7fd3ac0692b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34032/0 (socket says 172.21.15.92:34032) 2023-05-16T07:36:33.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b0b30700 1 -- 172.21.15.92:0/3077448070 learned_addr learned my addr 172.21.15.92:0/3077448070 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b1331700 1 -- 172.21.15.92:0/3077448070 >> v2:172.21.15.158:3300/0 conn(0x7fd3ac0699c0 msgr2=0x7fd3ac155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b1331700 1 --2- 172.21.15.92:0/3077448070 >> v2:172.21.15.158:3300/0 conn(0x7fd3ac0699c0 0x7fd3ac155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b1331700 1 -- 172.21.15.92:0/3077448070 >> v1:172.21.15.92:6789/0 conn(0x7fd3ac154f70 legacy=0x7fd3ac0692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b1331700 1 -- 172.21.15.92:0/3077448070 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac151340 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3a37fe700 1 -- 172.21.15.92:0/3077448070 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279211523 0 0) 0x7fd3ac151c70 con 0x7fd3ac154f70 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3b1331700 1 --2- 172.21.15.92:0/3077448070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac0bd7c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fd3ac064c40 tx=0x7fd3a4009780).ready entity=mon.0 client_cookie=eb24b446c0dff336 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3a37fe700 1 -- 172.21.15.92:0/3077448070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400e070 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3a37fe700 1 -- 172.21.15.92:0/3077448070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a400bc30 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.295+0000 7fd3a37fe700 1 -- 172.21.15.92:0/3077448070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4012410 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 -- 172.21.15.92:0/3077448070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 msgr2=0x7fd3ac0bd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/3077448070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac0bd7c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fd3ac064c40 tx=0x7fd3a4009780).stop 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 -- 172.21.15.92:0/3077448070 shutdown_connections 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/3077448070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac0bd7c0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/3077448070 >> v2:172.21.15.158:3300/0 conn(0x7fd3ac0699c0 0x7fd3ac155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:33.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 -- 172.21.15.92:0/3077448070 >> 172.21.15.92:0/3077448070 conn(0x7fd3ac0c8470 msgr2=0x7fd3ac0c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:33.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 -- 172.21.15.92:0/3077448070 shutdown_connections 2023-05-16T07:36:33.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.296+0000 7fd3b2d94700 1 -- 172.21.15.92:0/3077448070 wait complete. 2023-05-16T07:36:33.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.297+0000 7fd3b2d94700 1 Processor -- start 2023-05-16T07:36:33.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.297+0000 7fd3b2d94700 1 -- start start 2023-05-16T07:36:33.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3ac0699c0 0x7fd3ac14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac064f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3ac154f70 0x7fd3ac065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c0400 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0bf880 con 0x7fd3ac154f70 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0bfad0 con 0x7fd3ac0699c0 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac064f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac064f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51524/0 (socket says 172.21.15.92:51524) 2023-05-16T07:36:33.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 -- 172.21.15.92:0/341824909 learned_addr learned my addr 172.21.15.92:0/341824909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 -- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3ac0699c0 msgr2=0x7fd3ac14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3ac0699c0 0x7fd3ac14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b1331700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3ac154f70 0x7fd3ac065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 -- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3ac154f70 msgr2=0x7fd3ac065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3ac154f70 0x7fd3ac065480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 -- 172.21.15.92:0/341824909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4009a20 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a3fff700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac064f90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7fd39c00b460 tx=0x7fd39c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c002320 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c0024a0 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac0bfdb0 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.298+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c002860 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.299+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac0beb80 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.300+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac06c350 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.300+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd39c00f030 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.300+0000 7fd3a0ff9700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd39805ed90 0x7fd398061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.300+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5909+0+0 (secure 0 0 0) 0x7fd39c079760 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.300+0000 7fd3b0b30700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd39805ed90 0x7fd398061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.301+0000 7fd3b0b30700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd39805ed90 0x7fd398061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd3a80011e0 tx=0x7fd3a8007850).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.304+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd39c04add0 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.466+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fd3ac0c8880 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.591+0000 7fd3a0ff9700 1 -- 172.21.15.92:0/341824909 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v151) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd39c04e3f0 con 0x7fd3ac0bd3b0 2023-05-16T07:36:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.595+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd39805ed90 msgr2=0x7fd398061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.595+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd39805ed90 0x7fd398061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd3a80011e0 tx=0x7fd3a8007850).stop 2023-05-16T07:36:33.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.595+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 msgr2=0x7fd3ac064f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.595+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac064f90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7fd39c00b460 tx=0x7fd39c009060).stop 2023-05-16T07:36:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 shutdown_connections 2023-05-16T07:36:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd39805ed90 0x7fd398061250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd3ac0699c0 0x7fd3ac14acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd3ac154f70 0x7fd3ac065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 --2- 172.21.15.92:0/341824909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac0bd3b0 0x7fd3ac064f90 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 >> 172.21.15.92:0/341824909 conn(0x7fd3ac0c8470 msgr2=0x7fd3ac14d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.598+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 shutdown_connections 2023-05-16T07:36:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.599+0000 7fd3b2d94700 1 -- 172.21.15.92:0/341824909 wait complete. 2023-05-16T07:36:33.604 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:36:33.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:36:33.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.728+0000 7f366cdf9700 1 Processor -- start 2023-05-16T07:36:33.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.728+0000 7f366cdf9700 1 -- start start 2023-05-16T07:36:33.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f366cdf9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3668068870 0x7f36680aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f366cdf9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680c94d0 con 0x7f3668154090 2023-05-16T07:36:33.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f366cdf9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680cb7e0 con 0x7f3668154470 2023-05-16T07:36:33.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f366cdf9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36680c88c0 con 0x7f3668068870 2023-05-16T07:36:33.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f366659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3668154090 0x7f36680a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34046/0 (socket says 172.21.15.92:34046) 2023-05-16T07:36:33.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f366659c700 1 -- 172.21.15.92:0/1828926212 learned_addr learned my addr 172.21.15.92:0/1828926212 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:33.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.729+0000 7f3666d9d700 1 --2- 172.21.15.92:0/1828926212 >> v2:172.21.15.158:3300/0 conn(0x7f3668068870 0x7f36680aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921693803 0 0) 0x7f36680c94d0 con 0x7f3668154090 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36540033f0 con 0x7f3668154090 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 402774937 0 0) 0x7f36540033f0 con 0x7f3668154090 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 >> v2:172.21.15.158:3300/0 conn(0x7f3668068870 msgr2=0x7f36680aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 --2- 172.21.15.92:0/1828926212 >> v2:172.21.15.158:3300/0 conn(0x7f3668068870 0x7f36680aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 >> v1:172.21.15.158:6789/0 conn(0x7f3668154470 legacy=0x7f3668068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680c8ba0 con 0x7f3668154090 2023-05-16T07:36:33.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f36580017f0 con 0x7f3668154090 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 >> v1:172.21.15.92:6789/0 conn(0x7f3668154090 legacy=0x7f36680a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36540038e0 0x7f3654003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3654004690 0x7f3654004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f365400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680c88c0 con 0x7f3654005000 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365400beb0 con 0x7f3654004690 2023-05-16T07:36:33.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365400c070 con 0x7f36540038e0 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f365400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3665d9b700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3654004690 0x7f3654004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f366659c700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36540038e0 0x7f3654003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 -- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36540038e0 msgr2=0x7f3654003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36540038e0 0x7f3654003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 -- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3654004690 msgr2=0x7f3654004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3654004690 0x7f3654004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 -- 172.21.15.92:0/1828926212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540033f0 con 0x7f3654005000 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 -- 172.21.15.92:0/1828926212 reap_dead start 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.730+0000 7f3666d9d700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f365400b8e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f3660009a80 tx=0x7f3660009060).ready entity=mon.0 client_cookie=b347a4157baf33cd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660009430 con 0x7f3654005000 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366000e640 con 0x7f3654005000 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366559a700 1 -- 172.21.15.92:0/1828926212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000ea00 con 0x7f3654005000 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 -- 172.21.15.92:0/1828926212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 msgr2=0x7f365400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f365400b8e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f3660009a80 tx=0x7f3660009060).stop 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 -- 172.21.15.92:0/1828926212 shutdown_connections 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 --2- 172.21.15.92:0/1828926212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f365400b8e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 -- 172.21.15.92:0/1828926212 >> 172.21.15.92:0/1828926212 conn(0x7f3668067920 msgr2=0x7f36680cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 -- 172.21.15.92:0/1828926212 shutdown_connections 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.731+0000 7f366cdf9700 1 -- 172.21.15.92:0/1828926212 wait complete. 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.732+0000 7f366cdf9700 1 Processor -- start 2023-05-16T07:36:33.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.732+0000 7f366cdf9700 1 -- start start 2023-05-16T07:36:33.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366cdf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f366814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366cdf9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f366814fe10 0x7f366814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366cdf9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f366814e880 0x7f36680c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366cdf9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680c4c40 con 0x7f3654005000 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366cdf9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680ae6b0 con 0x7f366814fe10 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366cdf9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680add80 con 0x7f366814e880 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f366814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f366814f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51542/0 (socket says 172.21.15.92:51542) 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 -- 172.21.15.92:0/2529716426 learned_addr learned my addr 172.21.15.92:0/2529716426 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f3666d9d700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f366814e880 0x7f36680c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f3665d9b700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f366814fe10 0x7f366814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 -- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f366814e880 msgr2=0x7f36680c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f366814e880 0x7f36680c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 -- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f366814fe10 msgr2=0x7f366814e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:33.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f366814fe10 0x7f366814e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 -- 172.21.15.92:0/2529716426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680ae060 con 0x7f3654005000 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f3665d9b700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f366814fe10 0x7f366814e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.733+0000 7f366659c700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f366814f920 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f3658007710 tx=0x7f3658001d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.734+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800f8d0 con 0x7f3654005000 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.734+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800fa30 con 0x7f3654005000 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.734+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36680ae340 con 0x7f3654005000 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.734+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800fdf0 con 0x7f3654005000 2023-05-16T07:36:33.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.734+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366814ed00 con 0x7f3654005000 2023-05-16T07:36:33.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.735+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680ae990 con 0x7f3654005000 2023-05-16T07:36:33.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.735+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f365801b030 con 0x7f3654005000 2023-05-16T07:36:33.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.735+0000 7f365e7fc700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f364805ed90 0x7f3648061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:33.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.736+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6279+0+0 (secure 0 0 0) 0x7f3658079850 con 0x7f3654005000 2023-05-16T07:36:33.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.736+0000 7f3665d9b700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f364805ed90 0x7f3648061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:33.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.736+0000 7f3665d9b700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f364805ed90 0x7f3648061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3650007cc0 tx=0x7f3650009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:33.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.740+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f365804ac40 con 0x7f3654005000 2023-05-16T07:36:33.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:33.906+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3668066730 con 0x7f3654005000 2023-05-16T07:36:34.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.597+0000 7f365e7fc700 1 -- 172.21.15.92:0/2529716426 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v152) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3658019020 con 0x7f3654005000 2023-05-16T07:36:34.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.604+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f364805ed90 msgr2=0x7f3648061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.604+0000 7f366cdf9700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f364805ed90 0x7f3648061250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3650007cc0 tx=0x7f3650009040).stop 2023-05-16T07:36:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.604+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 msgr2=0x7f366814f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.604+0000 7f366cdf9700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f366814f920 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f3658007710 tx=0x7f3658001d10).stop 2023-05-16T07:36:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.606+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 shutdown_connections 2023-05-16T07:36:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.606+0000 7f366cdf9700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f364805ed90 0x7f3648061250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.606+0000 7f366cdf9700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f366814e880 0x7f36680c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.606+0000 7f366cdf9700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3654005000 0x7f366814f920 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.606+0000 7f366cdf9700 1 --2- 172.21.15.92:0/2529716426 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f366814fe10 0x7f366814e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.606+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 >> 172.21.15.92:0/2529716426 conn(0x7f3668067920 msgr2=0x7f36680c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:34.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.607+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 shutdown_connections 2023-05-16T07:36:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.608+0000 7f366cdf9700 1 -- 172.21.15.92:0/2529716426 wait complete. 2023-05-16T07:36:34.623 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:36:34.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:36:34.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.748+0000 7f269c5a9700 1 Processor -- start 2023-05-16T07:36:34.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.748+0000 7f269c5a9700 1 -- start start 2023-05-16T07:36:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269c5a9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2694154090 0x7f26940aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269c5a9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694068160 0x7f26940685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269c5a9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940c94d0 con 0x7f2694154470 2023-05-16T07:36:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269c5a9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26940cb7e0 con 0x7f2694154090 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269c5a9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694068aa0 con 0x7f2694068160 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269a345700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2694154090 0x7f26940aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269a345700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2694154090 0x7f26940aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51550/0 (socket says 172.21.15.92:51550) 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.749+0000 7f269a345700 1 -- 172.21.15.92:0/3608593032 learned_addr learned my addr 172.21.15.92:0/3608593032 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269a345700 1 -- 172.21.15.92:0/3608593032 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694068160 msgr2=0x7f26940685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269a345700 1 --2- 172.21.15.92:0/3608593032 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694068160 0x7f26940685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269a345700 1 -- 172.21.15.92:0/3608593032 >> v1:172.21.15.158:6789/0 conn(0x7f2694154470 legacy=0x7f26940a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269a345700 1 -- 172.21.15.92:0/3608593032 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940c88c0 con 0x7f2694154090 2023-05-16T07:36:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269a345700 1 --2- 172.21.15.92:0/3608593032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154090 0x7f26940aaca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f2684009f20 tx=0x7f2684009840).ready entity=mon.0 client_cookie=6ed5f6f6873b40b0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f2699343700 1 -- 172.21.15.92:0/3608593032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400e070 con 0x7f2694154090 2023-05-16T07:36:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f2699343700 1 -- 172.21.15.92:0/3608593032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268400ba40 con 0x7f2694154090 2023-05-16T07:36:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f2699343700 1 -- 172.21.15.92:0/3608593032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400bdc0 con 0x7f2694154090 2023-05-16T07:36:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 -- 172.21.15.92:0/3608593032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154090 msgr2=0x7f26940aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 --2- 172.21.15.92:0/3608593032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154090 0x7f26940aaca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f2684009f20 tx=0x7f2684009840).stop 2023-05-16T07:36:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 -- 172.21.15.92:0/3608593032 shutdown_connections 2023-05-16T07:36:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 --2- 172.21.15.92:0/3608593032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154090 0x7f26940aaca0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 --2- 172.21.15.92:0/3608593032 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694068160 0x7f26940685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 -- 172.21.15.92:0/3608593032 >> 172.21.15.92:0/3608593032 conn(0x7f2694067920 msgr2=0x7f26940cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 -- 172.21.15.92:0/3608593032 shutdown_connections 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.750+0000 7f269c5a9700 1 -- 172.21.15.92:0/3608593032 wait complete. 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.751+0000 7f269c5a9700 1 Processor -- start 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 -- start start 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2694068160 0x7f26940a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:34.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694154090 0x7f2694164430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 0x7f2694152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694156770 con 0x7f2694154470 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694155ba0 con 0x7f2694068160 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269c5a9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694155e40 con 0x7f2694154090 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269ab46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 0x7f2694152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269ab46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 0x7f2694152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51558/0 (socket says 172.21.15.92:51558) 2023-05-16T07:36:34.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.752+0000 7f269ab46700 1 -- 172.21.15.92:0/2880431983 learned_addr learned my addr 172.21.15.92:0/2880431983 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269a345700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2694068160 0x7f26940a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f2699b44700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694154090 0x7f2694164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269ab46700 1 -- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694154090 msgr2=0x7f2694164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269ab46700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694154090 0x7f2694164430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269ab46700 1 -- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2694068160 msgr2=0x7f26940a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269ab46700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2694068160 0x7f26940a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269ab46700 1 -- 172.21.15.92:0/2880431983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684009ae0 con 0x7f2694154470 2023-05-16T07:36:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269ab46700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 0x7f2694152d80 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f268c00b5e0 tx=0x7f268c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c002320 con 0x7f2694154470 2023-05-16T07:36:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c0024a0 con 0x7f2694154470 2023-05-16T07:36:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2694156120 con 0x7f2694154470 2023-05-16T07:36:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c002860 con 0x7f2694154470 2023-05-16T07:36:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.753+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26940aa4e0 con 0x7f2694154470 2023-05-16T07:36:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.754+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694156920 con 0x7f2694154470 2023-05-16T07:36:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.754+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f268c00f030 con 0x7f2694154470 2023-05-16T07:36:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.755+0000 7f26927fc700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f268005ed60 0x7f2680061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.755+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f268c079a00 con 0x7f2694154470 2023-05-16T07:36:34.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.755+0000 7f269a345700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f268005ed60 0x7f2680061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.755+0000 7f269a345700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f268005ed60 0x7f2680061220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f268400c010 tx=0x7f2684009840).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:34.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.759+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2694156920 con 0x7f2694154470 2023-05-16T07:36:34.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.961+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f26941568b0 con 0x7f2694154470 2023-05-16T07:36:34.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.966+0000 7f26927fc700 1 -- 172.21.15.92:0/2880431983 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f268c04e7b0 con 0x7f2694154470 2023-05-16T07:36:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f268005ed60 msgr2=0x7f2680061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f268005ed60 0x7f2680061220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f268400c010 tx=0x7f2684009840).stop 2023-05-16T07:36:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 msgr2=0x7f2694152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 0x7f2694152d80 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f268c00b5e0 tx=0x7f268c009060).stop 2023-05-16T07:36:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 shutdown_connections 2023-05-16T07:36:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2694154470 0x7f2694152d80 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2694068160 0x7f26940a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f268005ed60 0x7f2680061220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 --2- 172.21.15.92:0/2880431983 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2694154090 0x7f2694164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:34.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.969+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 >> 172.21.15.92:0/2880431983 conn(0x7f2694067920 msgr2=0x7f26940b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:34.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.972+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 shutdown_connections 2023-05-16T07:36:34.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:34.976+0000 7f269c5a9700 1 -- 172.21.15.92:0/2880431983 wait complete. 2023-05-16T07:36:34.983 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 19 and data pool 20 2023-05-16T07:36:34.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.106+0000 7f4f3c066700 1 Processor -- start 2023-05-16T07:36:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.106+0000 7f4f3c066700 1 -- start start 2023-05-16T07:36:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f3c066700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4f3414e200 0x7f4f3414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f3c066700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f340683d0 con 0x7f4f34158160 2023-05-16T07:36:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f3c066700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f340be060 con 0x7f4f34066e50 2023-05-16T07:36:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f3c066700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f34137470 con 0x7f4f3414e200 2023-05-16T07:36:35.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f3a603700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4f34158160 0x7f4f3415ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34084/0 (socket says 172.21.15.92:34084) 2023-05-16T07:36:35.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f3a603700 1 -- 172.21.15.92:0/2683892954 learned_addr learned my addr 172.21.15.92:0/2683892954 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:35.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f39601700 1 --1- 172.21.15.92:0/2683892954 >> v1:172.21.15.158:6789/0 conn(0x7f4f34066e50 0x7f4f34157a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:37680/0 (socket says 172.21.15.92:37680) 2023-05-16T07:36:35.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f39e02700 1 --2- 172.21.15.92:0/2683892954 >> v2:172.21.15.158:3300/0 conn(0x7f4f3414e200 0x7f4f3414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720221328 0 0) 0x7f4f340683d0 con 0x7f4f34158160 2023-05-16T07:36:35.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f200033f0 con 0x7f4f34158160 2023-05-16T07:36:35.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.107+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657695082 0 0) 0x7f4f340be060 con 0x7f4f34066e50 2023-05-16T07:36:35.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f340683d0 con 0x7f4f34066e50 2023-05-16T07:36:35.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f39e02700 1 -- 172.21.15.92:0/2683892954 >> v1:172.21.15.158:6789/0 conn(0x7f4f34066e50 legacy=0x7f4f34157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f39e02700 1 -- 172.21.15.92:0/2683892954 >> v1:172.21.15.92:6789/0 conn(0x7f4f34158160 legacy=0x7f4f3415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f39e02700 1 -- 172.21.15.92:0/2683892954 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f340677c0 con 0x7f4f3414e200 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f39e02700 1 --2- 172.21.15.92:0/2683892954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f3414e5e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f4f30009760 tx=0x7f4f30009060).ready entity=mon.1 client_cookie=efa48303749c7176 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000d4a0 con 0x7f4f3414e200 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3000d600 con 0x7f4f3414e200 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.108+0000 7f4f38e00700 1 -- 172.21.15.92:0/2683892954 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000d980 con 0x7f4f3414e200 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 -- 172.21.15.92:0/2683892954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 msgr2=0x7f4f3414e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 --2- 172.21.15.92:0/2683892954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f3414e5e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f4f30009760 tx=0x7f4f30009060).stop 2023-05-16T07:36:35.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 -- 172.21.15.92:0/2683892954 shutdown_connections 2023-05-16T07:36:35.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 --2- 172.21.15.92:0/2683892954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f3414e5e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 -- 172.21.15.92:0/2683892954 >> 172.21.15.92:0/2683892954 conn(0x7f4f340c7bc0 msgr2=0x7f4f340c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:35.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 -- 172.21.15.92:0/2683892954 shutdown_connections 2023-05-16T07:36:35.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.109+0000 7f4f3c066700 1 -- 172.21.15.92:0/2683892954 wait complete. 2023-05-16T07:36:35.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.110+0000 7f4f3c066700 1 Processor -- start 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.110+0000 7f4f3c066700 1 -- start start 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f34066e50 0x7f4f34136370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f340be210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 0x7f4f34065770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340bebf0 con 0x7f4f34158160 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340beed0 con 0x7f4f3414e200 2023-05-16T07:36:35.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340bf1b0 con 0x7f4f34066e50 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 0x7f4f34065770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 0x7f4f34065770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51574/0 (socket says 172.21.15.92:51574) 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 -- 172.21.15.92:0/1429526151 learned_addr learned my addr 172.21.15.92:0/1429526151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f39601700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f340be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f39e02700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f34066e50 0x7f4f34136370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 -- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f34066e50 msgr2=0x7f4f34136370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f34066e50 0x7f4f34136370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 -- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 msgr2=0x7f4f340be210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f340be210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 -- 172.21.15.92:0/1429526151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300092a0 con 0x7f4f34158160 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f39601700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f340be210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3a603700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 0x7f4f34065770 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f4f2c001560 tx=0x7f4f2c0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c005d30 con 0x7f4f34158160 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c005e90 con 0x7f4f34158160 2023-05-16T07:36:35.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f340bf490 con 0x7f4f34158160 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.111+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c007320 con 0x7f4f34158160 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.112+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f340acdb0 con 0x7f4f34158160 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.112+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f340bf9b0 con 0x7f4f34158160 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.113+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4f2c010030 con 0x7f4f34158160 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.113+0000 7f4f29ffb700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4f1805ed60 0x7f4f18061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.113+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f4f2c079d70 con 0x7f4f34158160 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.113+0000 7f4f39e02700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4f1805ed60 0x7f4f18061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.114+0000 7f4f39e02700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4f1805ed60 0x7f4f18061220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4f30006d60 tx=0x7f4f30016000).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.117+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4f340bf9b0 con 0x7f4f34158160 2023-05-16T07:36:35.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.282+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f340bf9b0 con 0x7f4f34158160 2023-05-16T07:36:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.283+0000 7f4f29ffb700 1 -- 172.21.15.92:0/1429526151 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+17977 (secure 0 0 0) 0x7f4f340bf9b0 con 0x7f4f34158160 2023-05-16T07:36:35.286 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:35.287 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":153,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:36:34.962663+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:36:33.491970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-05-16T07:36:33.931369+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4f1805ed60 msgr2=0x7f4f18061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4f1805ed60 0x7f4f18061220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4f30006d60 tx=0x7f4f30016000).stop 2023-05-16T07:36:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 msgr2=0x7f4f34065770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 0x7f4f34065770 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f4f2c001560 tx=0x7f4f2c0015b0).stop 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 shutdown_connections 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4f1805ed60 0x7f4f18061220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4f34066e50 0x7f4f34136370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f34158160 0x7f4f34065770 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 --2- 172.21.15.92:0/1429526151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4f3414e200 0x7f4f340be210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 >> 172.21.15.92:0/1429526151 conn(0x7f4f340c7bc0 msgr2=0x7f4f340c4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 shutdown_connections 2023-05-16T07:36:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.285+0000 7f4f3c066700 1 -- 172.21.15.92:0/1429526151 wait complete. 2023-05-16T07:36:35.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:35.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 Processor -- start 2023-05-16T07:36:35.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 -- start start 2023-05-16T07:36:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f54b0154550 0x7f54b00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b00c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00c0660 con 0x7f54b0154170 2023-05-16T07:36:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b006a000 con 0x7f54b0154550 2023-05-16T07:36:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.408+0000 7f54b60c1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00bfd30 con 0x7f54b0069b90 2023-05-16T07:36:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54aeffd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f54b0154550 0x7f54b00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b00c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54af7fe700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f54b0154170 0x7f54b00c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:37694/0 (socket says 172.21.15.92:37694) 2023-05-16T07:36:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54af7fe700 1 -- 172.21.15.92:0/3208878615 learned_addr learned my addr 172.21.15.92:0/3208878615 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:35.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b00c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:44152/0 (socket says 172.21.15.92:44152) 2023-05-16T07:36:35.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 -- 172.21.15.92:0/3208878615 >> v2:172.21.15.158:3300/0 conn(0x7f54b0154550 msgr2=0x7f54b00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:35.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 --2- 172.21.15.92:0/3208878615 >> v2:172.21.15.158:3300/0 conn(0x7f54b0154550 0x7f54b00696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 -- 172.21.15.92:0/3208878615 >> v1:172.21.15.158:6789/0 conn(0x7f54b0154170 legacy=0x7f54b00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 -- 172.21.15.92:0/3208878615 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00c0010 con 0x7f54b0069b90 2023-05-16T07:36:35.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54affff700 1 --2- 172.21.15.92:0/3208878615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b00c0af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f54a8009e60 tx=0x7f54a8009780).ready entity=mon.2 client_cookie=1bcd3bad8edd66d2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54ae7fc700 1 -- 172.21.15.92:0/3208878615 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800e070 con 0x7f54b0069b90 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54ae7fc700 1 -- 172.21.15.92:0/3208878615 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a800bb60 con 0x7f54b0069b90 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.409+0000 7f54ae7fc700 1 -- 172.21.15.92:0/3208878615 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8012410 con 0x7f54b0069b90 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 -- 172.21.15.92:0/3208878615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 msgr2=0x7f54b00c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3208878615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b00c0af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f54a8009e60 tx=0x7f54a8009780).stop 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 -- 172.21.15.92:0/3208878615 shutdown_connections 2023-05-16T07:36:35.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3208878615 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b00c0af0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3208878615 >> v2:172.21.15.158:3300/0 conn(0x7f54b0154550 0x7f54b00696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 -- 172.21.15.92:0/3208878615 >> 172.21.15.92:0/3208878615 conn(0x7f54b00cb700 msgr2=0x7f54b00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 -- 172.21.15.92:0/3208878615 shutdown_connections 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.410+0000 7f54b60c1700 1 -- 172.21.15.92:0/3208878615 wait complete. 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.411+0000 7f54b60c1700 1 Processor -- start 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.411+0000 7f54b60c1700 1 -- start start 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54b60c1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54b60c1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 0x7f54b0164650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54b60c1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54b0154550 0x7f54b00c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54b60c1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00afdf0 con 0x7f54b0154170 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54b60c1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0165600 con 0x7f54b0154550 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54b60c1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00c2150 con 0x7f54b0069b90 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 0x7f54b0164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 0x7f54b0164650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51586/0 (socket says 172.21.15.92:51586) 2023-05-16T07:36:35.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 -- 172.21.15.92:0/3135687854 learned_addr learned my addr 172.21.15.92:0/3135687854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54affff700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54b0154550 0x7f54b00c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54af7fe700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 -- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 msgr2=0x7f54b014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 -- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54b0154550 msgr2=0x7f54b00c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54b0154550 0x7f54b00c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.412+0000 7f54aeffd700 1 -- 172.21.15.92:0/3135687854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8009a20 con 0x7f54b0154170 2023-05-16T07:36:35.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.413+0000 7f54aeffd700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 0x7f54b0164650 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f54a400e900 tx=0x7f54a4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.413+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4012600 con 0x7f54b0154170 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.413+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4012760 con 0x7f54b0154170 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.413+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b0165040 con 0x7f54b0154170 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.413+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4012ae0 con 0x7f54b0154170 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.413+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b00c1de0 con 0x7f54b0154170 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.414+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b00666c0 con 0x7f54b0154170 2023-05-16T07:36:35.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.414+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f54a4022070 con 0x7f54b0154170 2023-05-16T07:36:35.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.414+0000 7f54977fe700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f549c05ed90 0x7f549c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.414+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f54a4079f10 con 0x7f54b0154170 2023-05-16T07:36:35.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.414+0000 7f54af7fe700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f549c05ed90 0x7f549c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.415+0000 7f54af7fe700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f549c05ed90 0x7f549c061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f54b00c3670 tx=0x7f54a0007d30).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.418+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f54a404b230 con 0x7f54b0154170 2023-05-16T07:36:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.580+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b006c350 con 0x7f54b0154170 2023-05-16T07:36:35.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.581+0000 7f54977fe700 1 -- 172.21.15.92:0/3135687854 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+17977 (secure 0 0 0) 0x7f54a4016070 con 0x7f54b0154170 2023-05-16T07:36:35.585 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:35.585 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":153,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:36:34.962663+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:36:33.491970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-05-16T07:36:33.931369+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f549c05ed90 msgr2=0x7f549c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f549c05ed90 0x7f549c061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f54b00c3670 tx=0x7f54a0007d30).stop 2023-05-16T07:36:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 msgr2=0x7f54b0164650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 0x7f54b0164650 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f54a400e900 tx=0x7f54a4009060).stop 2023-05-16T07:36:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 shutdown_connections 2023-05-16T07:36:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f549c05ed90 0x7f549c061250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f54b0069b90 0x7f54b014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54b0154550 0x7f54b00c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 --2- 172.21.15.92:0/3135687854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54b0154170 0x7f54b0164650 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 >> 172.21.15.92:0/3135687854 conn(0x7f54b00cb700 msgr2=0x7f54b00ab530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 shutdown_connections 2023-05-16T07:36:35.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.583+0000 7f54b60c1700 1 -- 172.21.15.92:0/3135687854 wait complete. 2023-05-16T07:36:35.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:36:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.709+0000 7f6d0cdad700 1 Processor -- start 2023-05-16T07:36:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.709+0000 7f6d0cdad700 1 -- start start 2023-05-16T07:36:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.709+0000 7f6d0cdad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d08154090 0x7f6d080aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0cdad700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6d08154470 0x7f6d080a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0cdad700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080c94d0 con 0x7f6d080aa150 2023-05-16T07:36:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0cdad700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080aa5a0 con 0x7f6d08154090 2023-05-16T07:36:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0cdad700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080cb7e0 con 0x7f6d08154470 2023-05-16T07:36:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d08154090 0x7f6d080aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d06d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6d080aa150 0x7f6d08068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34112/0 (socket says 172.21.15.92:34112) 2023-05-16T07:36:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d06d9d700 1 -- 172.21.15.92:0/2196472691 learned_addr learned my addr 172.21.15.92:0/2196472691 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6cfdd9b700 1 --2- 172.21.15.92:0/2196472691 >> v2:172.21.15.158:3300/0 conn(0x7f6d08154470 0x7f6d080a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0659c700 1 -- 172.21.15.92:0/2196472691 >> v2:172.21.15.158:3300/0 conn(0x7f6d08154470 msgr2=0x7f6d080a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0659c700 1 --2- 172.21.15.92:0/2196472691 >> v2:172.21.15.158:3300/0 conn(0x7f6d08154470 0x7f6d080a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0659c700 1 -- 172.21.15.92:0/2196472691 >> v1:172.21.15.92:6789/0 conn(0x7f6d080aa150 legacy=0x7f6d08068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0659c700 1 -- 172.21.15.92:0/2196472691 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080c88c0 con 0x7f6d08154090 2023-05-16T07:36:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d05d9b700 1 -- 172.21.15.92:0/2196472691 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079999223 0 0) 0x7f6d080c94d0 con 0x7f6d080aa150 2023-05-16T07:36:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6cfdd9b700 1 --2- 172.21.15.92:0/2196472691 >> v2:172.21.15.158:3300/0 conn(0x7f6d08154470 0x7f6d080a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d0659c700 1 --2- 172.21.15.92:0/2196472691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080aaca0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f6d08084320 tx=0x7f6cf8009060).ready entity=mon.0 client_cookie=992e8727969eb096 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d05d9b700 1 -- 172.21.15.92:0/2196472691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800b580 con 0x7f6d08154090 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d05d9b700 1 -- 172.21.15.92:0/2196472691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf800b6e0 con 0x7f6d08154090 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.710+0000 7f6d05d9b700 1 -- 172.21.15.92:0/2196472691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800ba60 con 0x7f6d08154090 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 -- 172.21.15.92:0/2196472691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 msgr2=0x7f6d080aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/2196472691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080aaca0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f6d08084320 tx=0x7f6cf8009060).stop 2023-05-16T07:36:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 -- 172.21.15.92:0/2196472691 shutdown_connections 2023-05-16T07:36:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/2196472691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080aaca0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/2196472691 >> v2:172.21.15.158:3300/0 conn(0x7f6d08154470 0x7f6d080a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 -- 172.21.15.92:0/2196472691 >> 172.21.15.92:0/2196472691 conn(0x7f6d08067920 msgr2=0x7f6d080cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 -- 172.21.15.92:0/2196472691 shutdown_connections 2023-05-16T07:36:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.711+0000 7f6d0cdad700 1 -- 172.21.15.92:0/2196472691 wait complete. 2023-05-16T07:36:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.712+0000 7f6d0cdad700 1 Processor -- start 2023-05-16T07:36:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.712+0000 7f6d0cdad700 1 -- start start 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d080aa150 0x7f6d08157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6d08154470 0x7f6d080c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08154f20 con 0x7f6d08154090 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080c4ed0 con 0x7f6d080aa150 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080c51b0 con 0x7f6d08154470 2023-05-16T07:36:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51600/0 (socket says 172.21.15.92:51600) 2023-05-16T07:36:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 -- 172.21.15.92:0/3859397804 learned_addr learned my addr 172.21.15.92:0/3859397804 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0659c700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d080aa150 0x7f6d08157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 -- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6d08154470 msgr2=0x7f6d080c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6d08154470 0x7f6d080c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 -- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d080aa150 msgr2=0x7f6d08157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d080aa150 0x7f6d08157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 -- 172.21.15.92:0/3859397804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80092a0 con 0x7f6d08154090 2023-05-16T07:36:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0659c700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d080aa150 0x7f6d08157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfdd9b700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080c4440 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f6cf000d890 tx=0x7f6cf0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00125f0 con 0x7f6d08154090 2023-05-16T07:36:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0012750 con 0x7f6d08154090 2023-05-16T07:36:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d080c5490 con 0x7f6d08154090 2023-05-16T07:36:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0012ad0 con 0x7f6d08154090 2023-05-16T07:36:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.713+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d08157720 con 0x7f6d08154090 2023-05-16T07:36:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.714+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08066730 con 0x7f6d08154090 2023-05-16T07:36:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.715+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6cf0013480 con 0x7f6d08154090 2023-05-16T07:36:35.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.715+0000 7f6cfeffd700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6ce805ed90 0x7f6ce8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.715+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f6cf007a090 con 0x7f6d08154090 2023-05-16T07:36:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.715+0000 7f6d0659c700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6ce805ed90 0x7f6ce8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:35.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.716+0000 7f6d0659c700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6ce805ed90 0x7f6ce8061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6cf800b340 tx=0x7f6cf8017000).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.719+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6cf004b350 con 0x7f6d08154090 2023-05-16T07:36:35.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.924+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6d0806c350 con 0x7f6d08154090 2023-05-16T07:36:35.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.968+0000 7f6cfeffd700 1 -- 172.21.15.92:0/3859397804 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6cf004e970 con 0x7f6d08154090 2023-05-16T07:36:35.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6ce805ed90 msgr2=0x7f6ce8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6ce805ed90 0x7f6ce8061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6cf800b340 tx=0x7f6cf8017000).stop 2023-05-16T07:36:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 msgr2=0x7f6d080c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080c4440 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f6cf000d890 tx=0x7f6cf0008040).stop 2023-05-16T07:36:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 shutdown_connections 2023-05-16T07:36:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6ce805ed90 0x7f6ce8061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6d08154470 0x7f6d080c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d080aa150 0x7f6d08157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 --2- 172.21.15.92:0/3859397804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08154090 0x7f6d080c4440 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 >> 172.21.15.92:0/3859397804 conn(0x7f6d08067920 msgr2=0x7f6d0814e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 shutdown_connections 2023-05-16T07:36:35.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:35.970+0000 7f6d0cdad700 1 -- 172.21.15.92:0/3859397804 wait complete. 2023-05-16T07:36:35.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:36:36.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 Processor -- start 2023-05-16T07:36:36.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 -- start start 2023-05-16T07:36:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2430066e50 0x7f24300672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 0x7f24300a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24300622b0 con 0x7f243013b420 2023-05-16T07:36:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2430063110 con 0x7f2430066e50 2023-05-16T07:36:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.099+0000 7f2436c98700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300677a0 con 0x7f24300a7820 2023-05-16T07:36:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f2435235700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 0x7f24300a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2430066e50 0x7f24300672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f2435235700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 0x7f24300a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:44182/0 (socket says 172.21.15.92:44182) 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f2434a34700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f243013b420 0x7f24300be0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:37722/0 (socket says 172.21.15.92:37722) 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f2435235700 1 -- 172.21.15.92:0/977291151 learned_addr learned my addr 172.21.15.92:0/977291151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2430066e50 0x7f24300672b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:55244/0 (socket says 172.21.15.92:55244) 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 -- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 msgr2=0x7f24300a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 --2- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 0x7f24300a7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 -- 172.21.15.92:0/977291151 >> v1:172.21.15.158:6789/0 conn(0x7f243013b420 legacy=0x7f24300be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242f7fe700 1 -- 172.21.15.92:0/977291151 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388716160 0 0) 0x7f24300622b0 con 0x7f243013b420 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 -- 172.21.15.92:0/977291151 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430063600 con 0x7f2430066e50 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f2435235700 1 --2- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 0x7f24300a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.100+0000 7f242ffff700 1 --2- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f24300672b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f2424009760 tx=0x7f2424009060).ready entity=mon.1 client_cookie=acf9022b6c19a1b1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f242f7fe700 1 -- 172.21.15.92:0/977291151 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242400f510 con 0x7f2430066e50 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f242f7fe700 1 -- 172.21.15.92:0/977291151 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242400f670 con 0x7f2430066e50 2023-05-16T07:36:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f242f7fe700 1 -- 172.21.15.92:0/977291151 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242400f9f0 con 0x7f2430066e50 2023-05-16T07:36:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 -- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 msgr2=0x7f24300672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 --2- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f24300672b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f2424009760 tx=0x7f2424009060).stop 2023-05-16T07:36:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 -- 172.21.15.92:0/977291151 shutdown_connections 2023-05-16T07:36:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 --2- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f24300a7820 0x7f24300a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 --2- 172.21.15.92:0/977291151 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f24300672b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 -- 172.21.15.92:0/977291151 >> 172.21.15.92:0/977291151 conn(0x7f24300c8470 msgr2=0x7f24300c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 -- 172.21.15.92:0/977291151 shutdown_connections 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.101+0000 7f2436c98700 1 -- 172.21.15.92:0/977291151 wait complete. 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.102+0000 7f2436c98700 1 Processor -- start 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.102+0000 7f2436c98700 1 -- start start 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2436c98700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f2430139350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2436c98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 0x7f24300a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2436c98700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f243013b420 0x7f24300a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2436c98700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430162f50 con 0x7f24300a7820 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2436c98700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430162620 con 0x7f2430066e50 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2436c98700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2430162900 con 0x7f243013b420 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f242ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 0x7f24300a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f242ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 0x7f24300a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51614/0 (socket says 172.21.15.92:51614) 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f242ffff700 1 -- 172.21.15.92:0/3164653559 learned_addr learned my addr 172.21.15.92:0/3164653559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2434a34700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f2430139350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:36.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.103+0000 7f2435235700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f243013b420 0x7f24300a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242ffff700 1 -- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f243013b420 msgr2=0x7f24300a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242ffff700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f243013b420 0x7f24300a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242ffff700 1 -- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 msgr2=0x7f2430139350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242ffff700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f2430139350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242ffff700 1 -- 172.21.15.92:0/3164653559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009370 con 0x7f24300a7820 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242ffff700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 0x7f24300a6d20 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f2424009e00 tx=0x7f2424017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242400fe80 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242400f7e0 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24240092a0 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242401b640 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.104+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2430139840 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.105+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24301630c0 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.105+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f242400f070 con 0x7f24300a7820 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.106+0000 7f242cff9700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f241c05ed90 0x7f241c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.106+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f242407b610 con 0x7f24300a7820 2023-05-16T07:36:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.106+0000 7f2434a34700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f241c05ed90 0x7f241c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.106+0000 7f2434a34700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f241c05ed90 0x7f241c061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f243015fef0 tx=0x7f2420001500).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:36.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.110+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f242404c930 con 0x7f24300a7820 2023-05-16T07:36:36.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.312+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f243006c350 con 0x7f24300a7820 2023-05-16T07:36:36.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.974+0000 7f242cff9700 1 -- 172.21.15.92:0/3164653559 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f242404ff50 con 0x7f24300a7820 2023-05-16T07:36:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f241c05ed90 msgr2=0x7f241c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f241c05ed90 0x7f241c061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f243015fef0 tx=0x7f2420001500).stop 2023-05-16T07:36:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 msgr2=0x7f24300a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 0x7f24300a6d20 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f2424009e00 tx=0x7f2424017000).stop 2023-05-16T07:36:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 shutdown_connections 2023-05-16T07:36:36.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f241c05ed90 0x7f241c061250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f243013b420 0x7f24300a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2430066e50 0x7f2430139350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 --2- 172.21.15.92:0/3164653559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24300a7820 0x7f24300a6d20 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:36.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 >> 172.21.15.92:0/3164653559 conn(0x7f24300c8470 msgr2=0x7f24300c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:36.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 shutdown_connections 2023-05-16T07:36:36.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:36.977+0000 7f2436c98700 1 -- 172.21.15.92:0/3164653559 wait complete. 2023-05-16T07:36:36.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:37.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.108+0000 7f09e483c700 1 Processor -- start 2023-05-16T07:36:37.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.108+0000 7f09e483c700 1 -- start start 2023-05-16T07:36:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.109+0000 7f09e483c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e00c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.109+0000 7f09e483c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09e0154550 0x7f09e00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.109+0000 7f09e483c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f09e00696a0 0x7f09e0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.109+0000 7f09e483c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e00c0660 con 0x7f09e0154550 2023-05-16T07:36:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.109+0000 7f09e483c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e006a020 con 0x7f09e00696a0 2023-05-16T07:36:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.109+0000 7f09e483c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00bfd30 con 0x7f09e0154170 2023-05-16T07:36:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09e0154550 0x7f09e00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09e0154550 0x7f09e00c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51624/0 (socket says 172.21.15.92:51624) 2023-05-16T07:36:37.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 -- 172.21.15.92:0/1470579103 learned_addr learned my addr 172.21.15.92:0/1470579103 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ded9d700 1 --2- 172.21.15.92:0/1470579103 >> v2:172.21.15.158:3300/0 conn(0x7f09e00696a0 0x7f09e0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09de59c700 1 --2- 172.21.15.92:0/1470579103 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e00c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 -- 172.21.15.92:0/1470579103 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 msgr2=0x7f09e00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 --2- 172.21.15.92:0/1470579103 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e00c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 -- 172.21.15.92:0/1470579103 >> v2:172.21.15.158:3300/0 conn(0x7f09e00696a0 msgr2=0x7f09e0069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 --2- 172.21.15.92:0/1470579103 >> v2:172.21.15.158:3300/0 conn(0x7f09e00696a0 0x7f09e0069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 -- 172.21.15.92:0/1470579103 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e00c0010 con 0x7f09e0154550 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09ddd9b700 1 --2- 172.21.15.92:0/1470579103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0154550 0x7f09e00c6160 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f09e0083be0 tx=0x7f09d4009840).ready entity=mon.0 client_cookie=708dbda31104f5d0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09dd59a700 1 -- 172.21.15.92:0/1470579103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400e070 con 0x7f09e0154550 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09dd59a700 1 -- 172.21.15.92:0/1470579103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d400ba50 con 0x7f09e0154550 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.110+0000 7f09dd59a700 1 -- 172.21.15.92:0/1470579103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400bdd0 con 0x7f09e0154550 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 -- 172.21.15.92:0/1470579103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0154550 msgr2=0x7f09e00c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 --2- 172.21.15.92:0/1470579103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0154550 0x7f09e00c6160 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f09e0083be0 tx=0x7f09d4009840).stop 2023-05-16T07:36:37.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 -- 172.21.15.92:0/1470579103 shutdown_connections 2023-05-16T07:36:37.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 --2- 172.21.15.92:0/1470579103 >> v2:172.21.15.158:3300/0 conn(0x7f09e00696a0 0x7f09e0069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 --2- 172.21.15.92:0/1470579103 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e00c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 --2- 172.21.15.92:0/1470579103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0154550 0x7f09e00c6160 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 -- 172.21.15.92:0/1470579103 >> 172.21.15.92:0/1470579103 conn(0x7f09e00cb700 msgr2=0x7f09e00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:37.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 -- 172.21.15.92:0/1470579103 shutdown_connections 2023-05-16T07:36:37.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.111+0000 7f09e483c700 1 -- 172.21.15.92:0/1470579103 wait complete. 2023-05-16T07:36:37.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.112+0000 7f09e483c700 1 Processor -- start 2023-05-16T07:36:37.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.112+0000 7f09e483c700 1 -- start start 2023-05-16T07:36:37.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 0x7f09e013f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0154550 0x7f09e00aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00c8b10 con 0x7f09e00696a0 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e013fe50 con 0x7f09e0154550 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00afaa0 con 0x7f09e0154170 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 0x7f09e013f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 0x7f09e013f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51626/0 (socket says 172.21.15.92:51626) 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 -- 172.21.15.92:0/3439233990 learned_addr learned my addr 172.21.15.92:0/3439233990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:37.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09ddd9b700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09ded9d700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0154550 0x7f09e00aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 -- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 msgr2=0x7f09e014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e014fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 -- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0154550 msgr2=0x7f09e00aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0154550 0x7f09e00aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 -- 172.21.15.92:0/3439233990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4009ae0 con 0x7f09e00696a0 2023-05-16T07:36:37.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09de59c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 0x7f09e013f960 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f09d800d890 tx=0x7f09d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8011590 con 0x7f09e00696a0 2023-05-16T07:36:37.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d80116f0 con 0x7f09e00696a0 2023-05-16T07:36:37.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e00afd80 con 0x7f09e00696a0 2023-05-16T07:36:37.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8011a70 con 0x7f09e00696a0 2023-05-16T07:36:37.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.113+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e00b0060 con 0x7f09e00696a0 2023-05-16T07:36:37.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.114+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e00666c0 con 0x7f09e00696a0 2023-05-16T07:36:37.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.114+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f09d8012420 con 0x7f09e00696a0 2023-05-16T07:36:37.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.115+0000 7f09ce7fc700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f09c005ed90 0x7f09c0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.115+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f09d80798c0 con 0x7f09e00696a0 2023-05-16T07:36:37.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.115+0000 7f09ddd9b700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f09c005ed90 0x7f09c0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.115+0000 7f09ddd9b700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f09c005ed90 0x7f09c0061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f09e0083be0 tx=0x7f09d4009840).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.119+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f09d804ab80 con 0x7f09e00696a0 2023-05-16T07:36:37.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.323+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09e006c350 con 0x7f09e00696a0 2023-05-16T07:36:37.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.324+0000 7f09ce7fc700 1 -- 172.21.15.92:0/3439233990 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f09d804e1a0 con 0x7f09e00696a0 2023-05-16T07:36:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:37.328 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:36:34.963007+0000","modified":"2023-05-16T07:36:36.970661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/1435424518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1435424518},{"type":"v1","addr":"172.21.15.158:6837","nonce":1435424518}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-05-16T07:36:37.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f09c005ed90 msgr2=0x7f09c0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f09c005ed90 0x7f09c0061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f09e0083be0 tx=0x7f09d4009840).stop 2023-05-16T07:36:37.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 msgr2=0x7f09e013f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 0x7f09e013f960 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f09d800d890 tx=0x7f09d8008040).stop 2023-05-16T07:36:37.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 shutdown_connections 2023-05-16T07:36:37.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09e0154550 0x7f09e00aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e00696a0 0x7f09e013f960 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f09c005ed90 0x7f09c0061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 --2- 172.21.15.92:0/3439233990 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f09e0154170 0x7f09e014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 >> 172.21.15.92:0/3439233990 conn(0x7f09e00cb700 msgr2=0x7f09e00b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:37.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 shutdown_connections 2023-05-16T07:36:37.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.326+0000 7f09e483c700 1 -- 172.21.15.92:0/3439233990 wait complete. 2023-05-16T07:36:37.334 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-05-16T07:36:37.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:36:37.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.450+0000 7f36e5ff6700 1 Processor -- start 2023-05-16T07:36:37.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.450+0000 7f36e5ff6700 1 -- start start 2023-05-16T07:36:37.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36e5ff6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f36e0154550 0x7f36e00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36e5ff6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e0069b90 0x7f36e00c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36e5ff6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e00c0660 con 0x7f36e0154170 2023-05-16T07:36:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36e5ff6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e0069fc0 con 0x7f36e0069b90 2023-05-16T07:36:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36e5ff6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e00bfd30 con 0x7f36e0154550 2023-05-16T07:36:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36dffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e0069b90 0x7f36e00c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36dffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e0069b90 0x7f36e00c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51636/0 (socket says 172.21.15.92:51636) 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36dffff700 1 -- 172.21.15.92:0/3337228958 learned_addr learned my addr 172.21.15.92:0/3337228958 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.451+0000 7f36deffd700 1 --2- 172.21.15.92:0/3337228958 >> v2:172.21.15.158:3300/0 conn(0x7f36e0154550 0x7f36e00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36dffff700 1 -- 172.21.15.92:0/3337228958 >> v2:172.21.15.158:3300/0 conn(0x7f36e0154550 msgr2=0x7f36e00696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36dffff700 1 --2- 172.21.15.92:0/3337228958 >> v2:172.21.15.158:3300/0 conn(0x7f36e0154550 0x7f36e00696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36dffff700 1 -- 172.21.15.92:0/3337228958 >> v1:172.21.15.158:6789/0 conn(0x7f36e0154170 legacy=0x7f36e00c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36dffff700 1 -- 172.21.15.92:0/3337228958 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e00c0010 con 0x7f36e0069b90 2023-05-16T07:36:37.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36dffff700 1 --2- 172.21.15.92:0/3337228958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0069b90 0x7f36e00c0af0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f36e00844e0 tx=0x7f36d80090f0).ready entity=mon.0 client_cookie=86e8e5652cb7541f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36de7fc700 1 -- 172.21.15.92:0/3337228958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800b520 con 0x7f36e0069b90 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36de7fc700 1 -- 172.21.15.92:0/3337228958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d800b680 con 0x7f36e0069b90 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36de7fc700 1 -- 172.21.15.92:0/3337228958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800ba20 con 0x7f36e0069b90 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3337228958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0069b90 msgr2=0x7f36e00c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3337228958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0069b90 0x7f36e00c0af0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f36e00844e0 tx=0x7f36d80090f0).stop 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3337228958 shutdown_connections 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3337228958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0069b90 0x7f36e00c0af0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3337228958 >> v2:172.21.15.158:3300/0 conn(0x7f36e0154550 0x7f36e00696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3337228958 >> 172.21.15.92:0/3337228958 conn(0x7f36e00cb700 msgr2=0x7f36e00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3337228958 shutdown_connections 2023-05-16T07:36:37.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.452+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3337228958 wait complete. 2023-05-16T07:36:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 Processor -- start 2023-05-16T07:36:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 -- start start 2023-05-16T07:36:37.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36e0069b90 0x7f36e014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f36e0154170 0x7f36e01505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 0x7f36e00aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e00c1d40 con 0x7f36e0154550 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0152dc0 con 0x7f36e0154170 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36e5ff6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0153050 con 0x7f36e0069b90 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 0x7f36e00aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 0x7f36e00aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51642/0 (socket says 172.21.15.92:51642) 2023-05-16T07:36:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.454+0000 7f36dffff700 1 -- 172.21.15.92:0/3825460904 learned_addr learned my addr 172.21.15.92:0/3825460904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36deffd700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f36e0154170 0x7f36e01505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36df7fe700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36e0069b90 0x7f36e014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36dffff700 1 -- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36e0069b90 msgr2=0x7f36e014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36dffff700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36e0069b90 0x7f36e014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36dffff700 1 -- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f36e0154170 msgr2=0x7f36e01505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36dffff700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f36e0154170 0x7f36e01505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36dffff700 1 -- 172.21.15.92:0/3825460904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d8009370 con 0x7f36e0154550 2023-05-16T07:36:37.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36dffff700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 0x7f36e00aec00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f36e0083060 tx=0x7f36d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800b520 con 0x7f36e0154550 2023-05-16T07:36:37.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8006970 con 0x7f36e0154550 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e0153330 con 0x7f36e0154550 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8006d90 con 0x7f36e0154550 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.455+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e014eed0 con 0x7f36e0154550 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.456+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e00666c0 con 0x7f36e0154550 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.456+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f36d802c070 con 0x7f36e0154550 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.457+0000 7f36c77fe700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f36cc05ed90 0x7f36cc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.457+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f36d807b800 con 0x7f36e0154550 2023-05-16T07:36:37.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.457+0000 7f36df7fe700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f36cc05ed90 0x7f36cc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.457+0000 7f36df7fe700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f36cc05ed90 0x7f36cc061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f36d0002d40 tx=0x7f36d0000f40).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.461+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f36d804cb20 con 0x7f36e0154550 2023-05-16T07:36:37.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.624+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36e006c350 con 0x7f36e0154550 2023-05-16T07:36:37.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.625+0000 7f36c77fe700 1 -- 172.21.15.92:0/3825460904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+17964 (secure 0 0 0) 0x7f36d802a020 con 0x7f36e0154550 2023-05-16T07:36:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:37.629 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":154,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:36:35.964643+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:36:33.491970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-05-16T07:36:33.931369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:36:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f36cc05ed90 msgr2=0x7f36cc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f36cc05ed90 0x7f36cc061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f36d0002d40 tx=0x7f36d0000f40).stop 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 msgr2=0x7f36e00aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 0x7f36e00aec00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f36e0083060 tx=0x7f36d8017000).stop 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 shutdown_connections 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f36cc05ed90 0x7f36cc061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f36e0069b90 0x7f36e014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e0154550 0x7f36e00aec00 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 --2- 172.21.15.92:0/3825460904 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f36e0154170 0x7f36e01505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 >> 172.21.15.92:0/3825460904 conn(0x7f36e00cb700 msgr2=0x7f36e014fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 shutdown_connections 2023-05-16T07:36:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.627+0000 7f36e5ff6700 1 -- 172.21.15.92:0/3825460904 wait complete. 2023-05-16T07:36:37.642 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:36:37.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:36:37.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.750+0000 7f4b7b854700 1 Processor -- start 2023-05-16T07:36:37.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.750+0000 7f4b7b854700 1 -- start start 2023-05-16T07:36:37.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b7b854700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b74153da0 0x7f4b74069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b7b854700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b74062b10 con 0x7f4b74154180 2023-05-16T07:36:37.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b7b854700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b74068340 con 0x7f4b74063e30 2023-05-16T07:36:37.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b7b854700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b74068620 con 0x7f4b74153da0 2023-05-16T07:36:37.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b795f0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b74153da0 0x7f4b74069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b78def700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4b74154180 0x7f4b74063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34166/0 (socket says 172.21.15.92:34166) 2023-05-16T07:36:37.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.751+0000 7f4b78def700 1 -- 172.21.15.92:0/3156523316 learned_addr learned my addr 172.21.15.92:0/3156523316 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b795f0700 1 -- 172.21.15.92:0/3156523316 >> v1:172.21.15.158:6789/0 conn(0x7f4b74063e30 legacy=0x7f4b740bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b795f0700 1 -- 172.21.15.92:0/3156523316 >> v1:172.21.15.92:6789/0 conn(0x7f4b74154180 legacy=0x7f4b74063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b795f0700 1 -- 172.21.15.92:0/3156523316 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740624c0 con 0x7f4b74153da0 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b73fff700 1 -- 172.21.15.92:0/3156523316 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804571106 0 0) 0x7f4b74062b10 con 0x7f4b74154180 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b795f0700 1 --2- 172.21.15.92:0/3156523316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b74069a10 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7f4b64009e60 tx=0x7f4b64009780).ready entity=mon.0 client_cookie=de1aa934d84e7ca2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b73fff700 1 -- 172.21.15.92:0/3156523316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400e070 con 0x7f4b74153da0 2023-05-16T07:36:37.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b73fff700 1 -- 172.21.15.92:0/3156523316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6400bbf0 con 0x7f4b74153da0 2023-05-16T07:36:37.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b73fff700 1 -- 172.21.15.92:0/3156523316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64012410 con 0x7f4b74153da0 2023-05-16T07:36:37.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b7b854700 1 -- 172.21.15.92:0/3156523316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 msgr2=0x7f4b74069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b7b854700 1 --2- 172.21.15.92:0/3156523316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b74069a10 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7f4b64009e60 tx=0x7f4b64009780).stop 2023-05-16T07:36:37.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b7b854700 1 -- 172.21.15.92:0/3156523316 shutdown_connections 2023-05-16T07:36:37.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b7b854700 1 --2- 172.21.15.92:0/3156523316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b74069a10 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.752+0000 7f4b7b854700 1 -- 172.21.15.92:0/3156523316 >> 172.21.15.92:0/3156523316 conn(0x7f4b740c8aa0 msgr2=0x7f4b740c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:37.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.753+0000 7f4b7b854700 1 -- 172.21.15.92:0/3156523316 shutdown_connections 2023-05-16T07:36:37.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.753+0000 7f4b7b854700 1 -- 172.21.15.92:0/3156523316 wait complete. 2023-05-16T07:36:37.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 Processor -- start 2023-05-16T07:36:37.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 -- start start 2023-05-16T07:36:37.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b74063e30 0x7f4b74146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b740c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4b74154180 0x7f4b74164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741579e0 con 0x7f4b74153da0 2023-05-16T07:36:37.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74158840 con 0x7f4b74063e30 2023-05-16T07:36:37.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.754+0000 7f4b7b854700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74157c30 con 0x7f4b74154180 2023-05-16T07:36:37.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b740c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b740c1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51656/0 (socket says 172.21.15.92:51656) 2023-05-16T07:36:37.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 -- 172.21.15.92:0/1387835921 learned_addr learned my addr 172.21.15.92:0/1387835921 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b795f0700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b74063e30 0x7f4b74146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b79df1700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4b74154180 0x7f4b74164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 -- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4b74154180 msgr2=0x7f4b74164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4b74154180 0x7f4b74164440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 -- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b74063e30 msgr2=0x7f4b74146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b74063e30 0x7f4b74146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 -- 172.21.15.92:0/1387835921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b64009a20 con 0x7f4b74153da0 2023-05-16T07:36:37.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b79df1700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4b74154180 0x7f4b74164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:37.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b78def700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b740c1830 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f4b68001eb0 tx=0x7f4b68007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680072e0 con 0x7f4b74153da0 2023-05-16T07:36:37.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68007440 con 0x7f4b74153da0 2023-05-16T07:36:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b74157f10 con 0x7f4b74153da0 2023-05-16T07:36:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680057a0 con 0x7f4b74153da0 2023-05-16T07:36:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.755+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b741584d0 con 0x7f4b74153da0 2023-05-16T07:36:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.756+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7406c350 con 0x7f4b74153da0 2023-05-16T07:36:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.757+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4b6801a060 con 0x7f4b74153da0 2023-05-16T07:36:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.757+0000 7f4b717fa700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4b6005ed90 0x7f4b60061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.757+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f4b68079af0 con 0x7f4b74153da0 2023-05-16T07:36:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.758+0000 7f4b795f0700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4b6005ed90 0x7f4b60061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.758+0000 7f4b795f0700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4b6005ed90 0x7f4b60061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4b6400b9b0 tx=0x7f4b64009780).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:37.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.761+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4b6804ae90 con 0x7f4b74153da0 2023-05-16T07:36:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.921+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b74158980 con 0x7f4b6005ed90 2023-05-16T07:36:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.933+0000 7f4b717fa700 1 -- 172.21.15.92:0/1387835921 <== mgr.7591 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361599 (secure 0 0 0) 0x7f4b74158980 con 0x7f4b6005ed90 2023-05-16T07:36:37.943 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:37.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4b6005ed90 msgr2=0x7f4b60061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4b6005ed90 0x7f4b60061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4b6400b9b0 tx=0x7f4b64009780).stop 2023-05-16T07:36:37.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 msgr2=0x7f4b740c1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:37.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b740c1830 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f4b68001eb0 tx=0x7f4b68007e90).stop 2023-05-16T07:36:37.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 shutdown_connections 2023-05-16T07:36:37.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4b6005ed90 0x7f4b60061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4b74154180 0x7f4b74164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4b74063e30 0x7f4b74146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 --2- 172.21.15.92:0/1387835921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b74153da0 0x7f4b740c1830 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 >> 172.21.15.92:0/1387835921 conn(0x7f4b740c8aa0 msgr2=0x7f4b740acfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 shutdown_connections 2023-05-16T07:36:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:37.936+0000 7f4b7b854700 1 -- 172.21.15.92:0/1387835921 wait complete. 2023-05-16T07:36:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:36:37.952 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:36:36.346664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":188652,"kb_used_data":14892,"kb_used_omap":0,"kb_used_meta":173696,"kb_avail":754786068,"statfs":{"total":773094113280,"available":772900933632,"internally_reserved":0,"allocated":15249408,"data_stored":8089739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":177864704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-593589,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-208,"num_write_kb":-720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-979,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.587198"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983780+0000","last_change":"2023-05-16T07:36:34.983780+0000","last_active":"2023-05-16T07:36:34.983780+0000","last_peered":"2023-05-16T07:36:34.983780+0000","last_clean":"2023-05-16T07:36:34.983780+0000","last_became_active":"2023-05-16T07:36:34.983451+0000","last_became_peered":"2023-05-16T07:36:34.983451+0000","last_unstale":"2023-05-16T07:36:34.983780+0000","last_undegraded":"2023-05-16T07:36:34.983780+0000","last_fullsized":"2023-05-16T07:36:34.983780+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"153'3","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982822+0000","last_change":"2023-05-16T07:36:34.617363+0000","last_active":"2023-05-16T07:36:34.982822+0000","last_peered":"2023-05-16T07:36:34.982822+0000","last_clean":"2023-05-16T07:36:34.982822+0000","last_became_active":"2023-05-16T07:36:34.617058+0000","last_became_peered":"2023-05-16T07:36:34.617058+0000","last_unstale":"2023-05-16T07:36:34.982822+0000","last_undegraded":"2023-05-16T07:36:34.982822+0000","last_fullsized":"2023-05-16T07:36:34.982822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992649+0000","last_change":"2023-05-16T07:36:34.992649+0000","last_active":"2023-05-16T07:36:34.992649+0000","last_peered":"2023-05-16T07:36:34.992649+0000","last_clean":"2023-05-16T07:36:34.992649+0000","last_became_active":"2023-05-16T07:36:34.992380+0000","last_became_peered":"2023-05-16T07:36:34.992380+0000","last_unstale":"2023-05-16T07:36:34.992649+0000","last_undegraded":"2023-05-16T07:36:34.992649+0000","last_fullsized":"2023-05-16T07:36:34.992649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618704+0000","last_change":"2023-05-16T07:36:34.616207+0000","last_active":"2023-05-16T07:36:34.618704+0000","last_peered":"2023-05-16T07:36:34.618704+0000","last_clean":"2023-05-16T07:36:34.618704+0000","last_became_active":"2023-05-16T07:36:34.615750+0000","last_became_peered":"2023-05-16T07:36:34.615750+0000","last_unstale":"2023-05-16T07:36:34.618704+0000","last_undegraded":"2023-05-16T07:36:34.618704+0000","last_fullsized":"2023-05-16T07:36:34.618704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983354+0000","last_change":"2023-05-16T07:36:34.983354+0000","last_active":"2023-05-16T07:36:34.983354+0000","last_peered":"2023-05-16T07:36:34.983354+0000","last_clean":"2023-05-16T07:36:34.983354+0000","last_became_active":"2023-05-16T07:36:34.983091+0000","last_became_peered":"2023-05-16T07:36:34.983091+0000","last_unstale":"2023-05-16T07:36:34.983354+0000","last_undegraded":"2023-05-16T07:36:34.983354+0000","last_fullsized":"2023-05-16T07:36:34.983354+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619788+0000","last_change":"2023-05-16T07:36:34.611818+0000","last_active":"2023-05-16T07:36:34.619788+0000","last_peered":"2023-05-16T07:36:34.619788+0000","last_clean":"2023-05-16T07:36:34.619788+0000","last_became_active":"2023-05-16T07:36:34.611542+0000","last_became_peered":"2023-05-16T07:36:34.611542+0000","last_unstale":"2023-05-16T07:36:34.619788+0000","last_undegraded":"2023-05-16T07:36:34.619788+0000","last_fullsized":"2023-05-16T07:36:34.619788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985708+0000","last_change":"2023-05-16T07:36:34.985708+0000","last_active":"2023-05-16T07:36:34.985708+0000","last_peered":"2023-05-16T07:36:34.985708+0000","last_clean":"2023-05-16T07:36:34.985708+0000","last_became_active":"2023-05-16T07:36:34.985435+0000","last_became_peered":"2023-05-16T07:36:34.985435+0000","last_unstale":"2023-05-16T07:36:34.985708+0000","last_undegraded":"2023-05-16T07:36:34.985708+0000","last_fullsized":"2023-05-16T07:36:34.985708+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987479+0000","last_change":"2023-05-16T07:36:34.615743+0000","last_active":"2023-05-16T07:36:34.987479+0000","last_peered":"2023-05-16T07:36:34.987479+0000","last_clean":"2023-05-16T07:36:34.987479+0000","last_became_active":"2023-05-16T07:36:34.615465+0000","last_became_peered":"2023-05-16T07:36:34.615465+0000","last_unstale":"2023-05-16T07:36:34.987479+0000","last_undegraded":"2023-05-16T07:36:34.987479+0000","last_fullsized":"2023-05-16T07:36:34.987479+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989497+0000","last_change":"2023-05-16T07:36:34.989497+0000","last_active":"2023-05-16T07:36:34.989497+0000","last_peered":"2023-05-16T07:36:34.989497+0000","last_clean":"2023-05-16T07:36:34.989497+0000","last_became_active":"2023-05-16T07:36:34.989243+0000","last_became_peered":"2023-05-16T07:36:34.989243+0000","last_unstale":"2023-05-16T07:36:34.989497+0000","last_undegraded":"2023-05-16T07:36:34.989497+0000","last_fullsized":"2023-05-16T07:36:34.989497+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.972889+0000","last_change":"2023-05-16T07:36:34.614729+0000","last_active":"2023-05-16T07:36:34.972889+0000","last_peered":"2023-05-16T07:36:34.972889+0000","last_clean":"2023-05-16T07:36:34.972889+0000","last_became_active":"2023-05-16T07:36:34.614125+0000","last_became_peered":"2023-05-16T07:36:34.614125+0000","last_unstale":"2023-05-16T07:36:34.972889+0000","last_undegraded":"2023-05-16T07:36:34.972889+0000","last_fullsized":"2023-05-16T07:36:34.972889+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991809+0000","last_change":"2023-05-16T07:36:34.991809+0000","last_active":"2023-05-16T07:36:34.991809+0000","last_peered":"2023-05-16T07:36:34.991809+0000","last_clean":"2023-05-16T07:36:34.991809+0000","last_became_active":"2023-05-16T07:36:34.991541+0000","last_became_peered":"2023-05-16T07:36:34.991541+0000","last_unstale":"2023-05-16T07:36:34.991809+0000","last_undegraded":"2023-05-16T07:36:34.991809+0000","last_fullsized":"2023-05-16T07:36:34.991809+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3a","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.978278+0000","last_change":"2023-05-16T07:36:34.620171+0000","last_active":"2023-05-16T07:36:34.978278+0000","last_peered":"2023-05-16T07:36:34.978278+0000","last_clean":"2023-05-16T07:36:34.978278+0000","last_became_active":"2023-05-16T07:36:34.619924+0000","last_became_peered":"2023-05-16T07:36:34.619924+0000","last_unstale":"2023-05-16T07:36:34.978278+0000","last_undegraded":"2023-05-16T07:36:34.978278+0000","last_fullsized":"2023-05-16T07:36:34.978278+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989195+0000","last_change":"2023-05-16T07:36:34.989195+0000","last_active":"2023-05-16T07:36:34.989195+0000","last_peered":"2023-05-16T07:36:34.989195+0000","last_clean":"2023-05-16T07:36:34.989195+0000","last_became_active":"2023-05-16T07:36:34.988803+0000","last_became_peered":"2023-05-16T07:36:34.988803+0000","last_unstale":"2023-05-16T07:36:34.989195+0000","last_undegraded":"2023-05-16T07:36:34.989195+0000","last_fullsized":"2023-05-16T07:36:34.989195+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.613075+0000","last_change":"2023-05-16T07:36:34.611442+0000","last_active":"2023-05-16T07:36:34.613075+0000","last_peered":"2023-05-16T07:36:34.613075+0000","last_clean":"2023-05-16T07:36:34.613075+0000","last_became_active":"2023-05-16T07:36:34.611197+0000","last_became_peered":"2023-05-16T07:36:34.611197+0000","last_unstale":"2023-05-16T07:36:34.613075+0000","last_undegraded":"2023-05-16T07:36:34.613075+0000","last_fullsized":"2023-05-16T07:36:34.613075+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980696+0000","last_change":"2023-05-16T07:36:34.980696+0000","last_active":"2023-05-16T07:36:34.980696+0000","last_peered":"2023-05-16T07:36:34.980696+0000","last_clean":"2023-05-16T07:36:34.980696+0000","last_became_active":"2023-05-16T07:36:34.980429+0000","last_became_peered":"2023-05-16T07:36:34.980429+0000","last_unstale":"2023-05-16T07:36:34.980696+0000","last_undegraded":"2023-05-16T07:36:34.980696+0000","last_fullsized":"2023-05-16T07:36:34.980696+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620815+0000","last_change":"2023-05-16T07:36:34.619509+0000","last_active":"2023-05-16T07:36:34.620815+0000","last_peered":"2023-05-16T07:36:34.620815+0000","last_clean":"2023-05-16T07:36:34.620815+0000","last_became_active":"2023-05-16T07:36:34.619189+0000","last_became_peered":"2023-05-16T07:36:34.619189+0000","last_unstale":"2023-05-16T07:36:34.620815+0000","last_undegraded":"2023-05-16T07:36:34.620815+0000","last_fullsized":"2023-05-16T07:36:34.620815+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987085+0000","last_change":"2023-05-16T07:36:34.987085+0000","last_active":"2023-05-16T07:36:34.987085+0000","last_peered":"2023-05-16T07:36:34.987085+0000","last_clean":"2023-05-16T07:36:34.987085+0000","last_became_active":"2023-05-16T07:36:34.986868+0000","last_became_peered":"2023-05-16T07:36:34.986868+0000","last_unstale":"2023-05-16T07:36:34.987085+0000","last_undegraded":"2023-05-16T07:36:34.987085+0000","last_fullsized":"2023-05-16T07:36:34.987085+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620162+0000","last_change":"2023-05-16T07:36:34.618400+0000","last_active":"2023-05-16T07:36:34.620162+0000","last_peered":"2023-05-16T07:36:34.620162+0000","last_clean":"2023-05-16T07:36:34.620162+0000","last_became_active":"2023-05-16T07:36:34.618104+0000","last_became_peered":"2023-05-16T07:36:34.618104+0000","last_unstale":"2023-05-16T07:36:34.620162+0000","last_undegraded":"2023-05-16T07:36:34.620162+0000","last_fullsized":"2023-05-16T07:36:34.620162+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.975409+0000","last_change":"2023-05-16T07:36:34.975409+0000","last_active":"2023-05-16T07:36:34.975409+0000","last_peered":"2023-05-16T07:36:34.975409+0000","last_clean":"2023-05-16T07:36:34.975409+0000","last_became_active":"2023-05-16T07:36:34.975120+0000","last_became_peered":"2023-05-16T07:36:34.975120+0000","last_unstale":"2023-05-16T07:36:34.975409+0000","last_undegraded":"2023-05-16T07:36:34.975409+0000","last_fullsized":"2023-05-16T07:36:34.975409+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618268+0000","last_change":"2023-05-16T07:36:34.616038+0000","last_active":"2023-05-16T07:36:34.618268+0000","last_peered":"2023-05-16T07:36:34.618268+0000","last_clean":"2023-05-16T07:36:34.618268+0000","last_became_active":"2023-05-16T07:36:34.615407+0000","last_became_peered":"2023-05-16T07:36:34.615407+0000","last_unstale":"2023-05-16T07:36:34.618268+0000","last_undegraded":"2023-05-16T07:36:34.618268+0000","last_fullsized":"2023-05-16T07:36:34.618268+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.977938+0000","last_change":"2023-05-16T07:36:34.977938+0000","last_active":"2023-05-16T07:36:34.977938+0000","last_peered":"2023-05-16T07:36:34.977938+0000","last_clean":"2023-05-16T07:36:34.977938+0000","last_became_active":"2023-05-16T07:36:34.977665+0000","last_became_peered":"2023-05-16T07:36:34.977665+0000","last_unstale":"2023-05-16T07:36:34.977938+0000","last_undegraded":"2023-05-16T07:36:34.977938+0000","last_fullsized":"2023-05-16T07:36:34.977938+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621487+0000","last_change":"2023-05-16T07:36:34.610721+0000","last_active":"2023-05-16T07:36:34.621487+0000","last_peered":"2023-05-16T07:36:34.621487+0000","last_clean":"2023-05-16T07:36:34.621487+0000","last_became_active":"2023-05-16T07:36:34.610302+0000","last_became_peered":"2023-05-16T07:36:34.610302+0000","last_unstale":"2023-05-16T07:36:34.621487+0000","last_undegraded":"2023-05-16T07:36:34.621487+0000","last_fullsized":"2023-05-16T07:36:34.621487+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992706+0000","last_change":"2023-05-16T07:36:34.992706+0000","last_active":"2023-05-16T07:36:34.992706+0000","last_peered":"2023-05-16T07:36:34.992706+0000","last_clean":"2023-05-16T07:36:34.992706+0000","last_became_active":"2023-05-16T07:36:34.992415+0000","last_became_peered":"2023-05-16T07:36:34.992415+0000","last_unstale":"2023-05-16T07:36:34.992706+0000","last_undegraded":"2023-05-16T07:36:34.992706+0000","last_fullsized":"2023-05-16T07:36:34.992706+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987795+0000","last_change":"2023-05-16T07:36:34.618547+0000","last_active":"2023-05-16T07:36:34.987795+0000","last_peered":"2023-05-16T07:36:34.987795+0000","last_clean":"2023-05-16T07:36:34.987795+0000","last_became_active":"2023-05-16T07:36:34.618245+0000","last_became_peered":"2023-05-16T07:36:34.618245+0000","last_unstale":"2023-05-16T07:36:34.987795+0000","last_undegraded":"2023-05-16T07:36:34.987795+0000","last_fullsized":"2023-05-16T07:36:34.987795+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983398+0000","last_change":"2023-05-16T07:36:34.983398+0000","last_active":"2023-05-16T07:36:34.983398+0000","last_peered":"2023-05-16T07:36:34.983398+0000","last_clean":"2023-05-16T07:36:34.983398+0000","last_became_active":"2023-05-16T07:36:34.983131+0000","last_became_peered":"2023-05-16T07:36:34.983131+0000","last_unstale":"2023-05-16T07:36:34.983398+0000","last_undegraded":"2023-05-16T07:36:34.983398+0000","last_fullsized":"2023-05-16T07:36:34.983398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.33","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.971856+0000","last_change":"2023-05-16T07:36:34.616384+0000","last_active":"2023-05-16T07:36:34.971856+0000","last_peered":"2023-05-16T07:36:34.971856+0000","last_clean":"2023-05-16T07:36:34.971856+0000","last_became_active":"2023-05-16T07:36:34.615622+0000","last_became_peered":"2023-05-16T07:36:34.615622+0000","last_unstale":"2023-05-16T07:36:34.971856+0000","last_undegraded":"2023-05-16T07:36:34.971856+0000","last_fullsized":"2023-05-16T07:36:34.971856+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.995085+0000","last_change":"2023-05-16T07:36:34.995085+0000","last_active":"2023-05-16T07:36:34.995085+0000","last_peered":"2023-05-16T07:36:34.995085+0000","last_clean":"2023-05-16T07:36:34.995085+0000","last_became_active":"2023-05-16T07:36:34.994814+0000","last_became_peered":"2023-05-16T07:36:34.994814+0000","last_unstale":"2023-05-16T07:36:34.995085+0000","last_undegraded":"2023-05-16T07:36:34.995085+0000","last_fullsized":"2023-05-16T07:36:34.995085+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621982+0000","last_change":"2023-05-16T07:36:34.620363+0000","last_active":"2023-05-16T07:36:34.621982+0000","last_peered":"2023-05-16T07:36:34.621982+0000","last_clean":"2023-05-16T07:36:34.621982+0000","last_became_active":"2023-05-16T07:36:34.620066+0000","last_became_peered":"2023-05-16T07:36:34.620066+0000","last_unstale":"2023-05-16T07:36:34.621982+0000","last_undegraded":"2023-05-16T07:36:34.621982+0000","last_fullsized":"2023-05-16T07:36:34.621982+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.988596+0000","last_change":"2023-05-16T07:36:34.988596+0000","last_active":"2023-05-16T07:36:34.988596+0000","last_peered":"2023-05-16T07:36:34.988596+0000","last_clean":"2023-05-16T07:36:34.988596+0000","last_became_active":"2023-05-16T07:36:34.988294+0000","last_became_peered":"2023-05-16T07:36:34.988294+0000","last_unstale":"2023-05-16T07:36:34.988596+0000","last_undegraded":"2023-05-16T07:36:34.988596+0000","last_fullsized":"2023-05-16T07:36:34.988596+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.623044+0000","last_change":"2023-05-16T07:36:34.617933+0000","last_active":"2023-05-16T07:36:34.623044+0000","last_peered":"2023-05-16T07:36:34.623044+0000","last_clean":"2023-05-16T07:36:34.623044+0000","last_became_active":"2023-05-16T07:36:34.617651+0000","last_became_peered":"2023-05-16T07:36:34.617651+0000","last_unstale":"2023-05-16T07:36:34.623044+0000","last_undegraded":"2023-05-16T07:36:34.623044+0000","last_fullsized":"2023-05-16T07:36:34.623044+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990611+0000","last_change":"2023-05-16T07:36:34.990611+0000","last_active":"2023-05-16T07:36:34.990611+0000","last_peered":"2023-05-16T07:36:34.990611+0000","last_clean":"2023-05-16T07:36:34.990611+0000","last_became_active":"2023-05-16T07:36:34.990360+0000","last_became_peered":"2023-05-16T07:36:34.990360+0000","last_unstale":"2023-05-16T07:36:34.990611+0000","last_undegraded":"2023-05-16T07:36:34.990611+0000","last_fullsized":"2023-05-16T07:36:34.990611+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985474+0000","last_change":"2023-05-16T07:36:34.614358+0000","last_active":"2023-05-16T07:36:34.985474+0000","last_peered":"2023-05-16T07:36:34.985474+0000","last_clean":"2023-05-16T07:36:34.985474+0000","last_became_active":"2023-05-16T07:36:34.614082+0000","last_became_peered":"2023-05-16T07:36:34.614082+0000","last_unstale":"2023-05-16T07:36:34.985474+0000","last_undegraded":"2023-05-16T07:36:34.985474+0000","last_fullsized":"2023-05-16T07:36:34.985474+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989366+0000","last_change":"2023-05-16T07:36:34.989366+0000","last_active":"2023-05-16T07:36:34.989366+0000","last_peered":"2023-05-16T07:36:34.989366+0000","last_clean":"2023-05-16T07:36:34.989366+0000","last_became_active":"2023-05-16T07:36:34.989162+0000","last_became_peered":"2023-05-16T07:36:34.989162+0000","last_unstale":"2023-05-16T07:36:34.989366+0000","last_undegraded":"2023-05-16T07:36:34.989366+0000","last_fullsized":"2023-05-16T07:36:34.989366+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.612991+0000","last_change":"2023-05-16T07:36:34.610890+0000","last_active":"2023-05-16T07:36:34.612991+0000","last_peered":"2023-05-16T07:36:34.612991+0000","last_clean":"2023-05-16T07:36:34.612991+0000","last_became_active":"2023-05-16T07:36:34.610638+0000","last_became_peered":"2023-05-16T07:36:34.610638+0000","last_unstale":"2023-05-16T07:36:34.612991+0000","last_undegraded":"2023-05-16T07:36:34.612991+0000","last_fullsized":"2023-05-16T07:36:34.612991+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990447+0000","last_change":"2023-05-16T07:36:34.990447+0000","last_active":"2023-05-16T07:36:34.990447+0000","last_peered":"2023-05-16T07:36:34.990447+0000","last_clean":"2023-05-16T07:36:34.990447+0000","last_became_active":"2023-05-16T07:36:34.989836+0000","last_became_peered":"2023-05-16T07:36:34.989836+0000","last_unstale":"2023-05-16T07:36:34.990447+0000","last_undegraded":"2023-05-16T07:36:34.990447+0000","last_fullsized":"2023-05-16T07:36:34.990447+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.617965+0000","last_change":"2023-05-16T07:36:34.612974+0000","last_active":"2023-05-16T07:36:34.617965+0000","last_peered":"2023-05-16T07:36:34.617965+0000","last_clean":"2023-05-16T07:36:34.617965+0000","last_became_active":"2023-05-16T07:36:34.612753+0000","last_became_peered":"2023-05-16T07:36:34.612753+0000","last_unstale":"2023-05-16T07:36:34.617965+0000","last_undegraded":"2023-05-16T07:36:34.617965+0000","last_fullsized":"2023-05-16T07:36:34.617965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990515+0000","last_change":"2023-05-16T07:36:34.990515+0000","last_active":"2023-05-16T07:36:34.990515+0000","last_peered":"2023-05-16T07:36:34.990515+0000","last_clean":"2023-05-16T07:36:34.990515+0000","last_became_active":"2023-05-16T07:36:34.990072+0000","last_became_peered":"2023-05-16T07:36:34.990072+0000","last_unstale":"2023-05-16T07:36:34.990515+0000","last_undegraded":"2023-05-16T07:36:34.990515+0000","last_fullsized":"2023-05-16T07:36:34.990515+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2d","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.973386+0000","last_change":"2023-05-16T07:36:34.612576+0000","last_active":"2023-05-16T07:36:34.973386+0000","last_peered":"2023-05-16T07:36:34.973386+0000","last_clean":"2023-05-16T07:36:34.973386+0000","last_became_active":"2023-05-16T07:36:34.612300+0000","last_became_peered":"2023-05-16T07:36:34.612300+0000","last_unstale":"2023-05-16T07:36:34.973386+0000","last_undegraded":"2023-05-16T07:36:34.973386+0000","last_fullsized":"2023-05-16T07:36:34.973386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989016+0000","last_change":"2023-05-16T07:36:34.989016+0000","last_active":"2023-05-16T07:36:34.989016+0000","last_peered":"2023-05-16T07:36:34.989016+0000","last_clean":"2023-05-16T07:36:34.989016+0000","last_became_active":"2023-05-16T07:36:34.988714+0000","last_became_peered":"2023-05-16T07:36:34.988714+0000","last_unstale":"2023-05-16T07:36:34.989016+0000","last_undegraded":"2023-05-16T07:36:34.989016+0000","last_fullsized":"2023-05-16T07:36:34.989016+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619259+0000","last_change":"2023-05-16T07:36:34.616368+0000","last_active":"2023-05-16T07:36:34.619259+0000","last_peered":"2023-05-16T07:36:34.619259+0000","last_clean":"2023-05-16T07:36:34.619259+0000","last_became_active":"2023-05-16T07:36:34.616019+0000","last_became_peered":"2023-05-16T07:36:34.616019+0000","last_unstale":"2023-05-16T07:36:34.619259+0000","last_undegraded":"2023-05-16T07:36:34.619259+0000","last_fullsized":"2023-05-16T07:36:34.619259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.973269+0000","last_change":"2023-05-16T07:36:34.973269+0000","last_active":"2023-05-16T07:36:34.973269+0000","last_peered":"2023-05-16T07:36:34.973269+0000","last_clean":"2023-05-16T07:36:34.973269+0000","last_became_active":"2023-05-16T07:36:34.973021+0000","last_became_peered":"2023-05-16T07:36:34.973021+0000","last_unstale":"2023-05-16T07:36:34.973269+0000","last_undegraded":"2023-05-16T07:36:34.973269+0000","last_fullsized":"2023-05-16T07:36:34.973269+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.973002+0000","last_change":"2023-05-16T07:36:34.614844+0000","last_active":"2023-05-16T07:36:34.973002+0000","last_peered":"2023-05-16T07:36:34.973002+0000","last_clean":"2023-05-16T07:36:34.973002+0000","last_became_active":"2023-05-16T07:36:34.614475+0000","last_became_peered":"2023-05-16T07:36:34.614475+0000","last_unstale":"2023-05-16T07:36:34.973002+0000","last_undegraded":"2023-05-16T07:36:34.973002+0000","last_fullsized":"2023-05-16T07:36:34.973002+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986742+0000","last_change":"2023-05-16T07:36:34.986742+0000","last_active":"2023-05-16T07:36:34.986742+0000","last_peered":"2023-05-16T07:36:34.986742+0000","last_clean":"2023-05-16T07:36:34.986742+0000","last_became_active":"2023-05-16T07:36:34.986461+0000","last_became_peered":"2023-05-16T07:36:34.986461+0000","last_unstale":"2023-05-16T07:36:34.986742+0000","last_undegraded":"2023-05-16T07:36:34.986742+0000","last_fullsized":"2023-05-16T07:36:34.986742+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621496+0000","last_change":"2023-05-16T07:36:34.610983+0000","last_active":"2023-05-16T07:36:34.621496+0000","last_peered":"2023-05-16T07:36:34.621496+0000","last_clean":"2023-05-16T07:36:34.621496+0000","last_became_active":"2023-05-16T07:36:34.610687+0000","last_became_peered":"2023-05-16T07:36:34.610687+0000","last_unstale":"2023-05-16T07:36:34.621496+0000","last_undegraded":"2023-05-16T07:36:34.621496+0000","last_fullsized":"2023-05-16T07:36:34.621496+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987031+0000","last_change":"2023-05-16T07:36:34.987031+0000","last_active":"2023-05-16T07:36:34.987031+0000","last_peered":"2023-05-16T07:36:34.987031+0000","last_clean":"2023-05-16T07:36:34.987031+0000","last_became_active":"2023-05-16T07:36:34.986738+0000","last_became_peered":"2023-05-16T07:36:34.986738+0000","last_unstale":"2023-05-16T07:36:34.987031+0000","last_undegraded":"2023-05-16T07:36:34.987031+0000","last_fullsized":"2023-05-16T07:36:34.987031+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620472+0000","last_change":"2023-05-16T07:36:34.617921+0000","last_active":"2023-05-16T07:36:34.620472+0000","last_peered":"2023-05-16T07:36:34.620472+0000","last_clean":"2023-05-16T07:36:34.620472+0000","last_became_active":"2023-05-16T07:36:34.617416+0000","last_became_peered":"2023-05-16T07:36:34.617416+0000","last_unstale":"2023-05-16T07:36:34.620472+0000","last_undegraded":"2023-05-16T07:36:34.620472+0000","last_fullsized":"2023-05-16T07:36:34.620472+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983361+0000","last_change":"2023-05-16T07:36:34.983361+0000","last_active":"2023-05-16T07:36:34.983361+0000","last_peered":"2023-05-16T07:36:34.983361+0000","last_clean":"2023-05-16T07:36:34.983361+0000","last_became_active":"2023-05-16T07:36:34.983083+0000","last_became_peered":"2023-05-16T07:36:34.983083+0000","last_unstale":"2023-05-16T07:36:34.983361+0000","last_undegraded":"2023-05-16T07:36:34.983361+0000","last_fullsized":"2023-05-16T07:36:34.983361+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621763+0000","last_change":"2023-05-16T07:36:34.613032+0000","last_active":"2023-05-16T07:36:34.621763+0000","last_peered":"2023-05-16T07:36:34.621763+0000","last_clean":"2023-05-16T07:36:34.621763+0000","last_became_active":"2023-05-16T07:36:34.612699+0000","last_became_peered":"2023-05-16T07:36:34.612699+0000","last_unstale":"2023-05-16T07:36:34.621763+0000","last_undegraded":"2023-05-16T07:36:34.621763+0000","last_fullsized":"2023-05-16T07:36:34.621763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982784+0000","last_change":"2023-05-16T07:36:34.982784+0000","last_active":"2023-05-16T07:36:34.982784+0000","last_peered":"2023-05-16T07:36:34.982784+0000","last_clean":"2023-05-16T07:36:34.982784+0000","last_became_active":"2023-05-16T07:36:34.982539+0000","last_became_peered":"2023-05-16T07:36:34.982539+0000","last_unstale":"2023-05-16T07:36:34.982784+0000","last_undegraded":"2023-05-16T07:36:34.982784+0000","last_fullsized":"2023-05-16T07:36:34.982784+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619779+0000","last_change":"2023-05-16T07:36:34.616330+0000","last_active":"2023-05-16T07:36:34.619779+0000","last_peered":"2023-05-16T07:36:34.619779+0000","last_clean":"2023-05-16T07:36:34.619779+0000","last_became_active":"2023-05-16T07:36:34.615577+0000","last_became_peered":"2023-05-16T07:36:34.615577+0000","last_unstale":"2023-05-16T07:36:34.619779+0000","last_undegraded":"2023-05-16T07:36:34.619779+0000","last_fullsized":"2023-05-16T07:36:34.619779+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984407+0000","last_change":"2023-05-16T07:36:34.984407+0000","last_active":"2023-05-16T07:36:34.984407+0000","last_peered":"2023-05-16T07:36:34.984407+0000","last_clean":"2023-05-16T07:36:34.984407+0000","last_became_active":"2023-05-16T07:36:34.984097+0000","last_became_peered":"2023-05-16T07:36:34.984097+0000","last_unstale":"2023-05-16T07:36:34.984407+0000","last_undegraded":"2023-05-16T07:36:34.984407+0000","last_fullsized":"2023-05-16T07:36:34.984407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619192+0000","last_change":"2023-05-16T07:36:34.614889+0000","last_active":"2023-05-16T07:36:34.619192+0000","last_peered":"2023-05-16T07:36:34.619192+0000","last_clean":"2023-05-16T07:36:34.619192+0000","last_became_active":"2023-05-16T07:36:34.614257+0000","last_became_peered":"2023-05-16T07:36:34.614257+0000","last_unstale":"2023-05-16T07:36:34.619192+0000","last_undegraded":"2023-05-16T07:36:34.619192+0000","last_fullsized":"2023-05-16T07:36:34.619192+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.988839+0000","last_change":"2023-05-16T07:36:34.988839+0000","last_active":"2023-05-16T07:36:34.988839+0000","last_peered":"2023-05-16T07:36:34.988839+0000","last_clean":"2023-05-16T07:36:34.988839+0000","last_became_active":"2023-05-16T07:36:34.985450+0000","last_became_peered":"2023-05-16T07:36:34.985450+0000","last_unstale":"2023-05-16T07:36:34.988839+0000","last_undegraded":"2023-05-16T07:36:34.988839+0000","last_fullsized":"2023-05-16T07:36:34.988839+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622876+0000","last_change":"2023-05-16T07:36:34.615159+0000","last_active":"2023-05-16T07:36:34.622876+0000","last_peered":"2023-05-16T07:36:34.622876+0000","last_clean":"2023-05-16T07:36:34.622876+0000","last_became_active":"2023-05-16T07:36:34.614875+0000","last_became_peered":"2023-05-16T07:36:34.614875+0000","last_unstale":"2023-05-16T07:36:34.622876+0000","last_undegraded":"2023-05-16T07:36:34.622876+0000","last_fullsized":"2023-05-16T07:36:34.622876+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982480+0000","last_change":"2023-05-16T07:36:34.982480+0000","last_active":"2023-05-16T07:36:34.982480+0000","last_peered":"2023-05-16T07:36:34.982480+0000","last_clean":"2023-05-16T07:36:34.982480+0000","last_became_active":"2023-05-16T07:36:34.982180+0000","last_became_peered":"2023-05-16T07:36:34.982180+0000","last_unstale":"2023-05-16T07:36:34.982480+0000","last_undegraded":"2023-05-16T07:36:34.982480+0000","last_fullsized":"2023-05-16T07:36:34.982480+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619097+0000","last_change":"2023-05-16T07:36:34.614696+0000","last_active":"2023-05-16T07:36:34.619097+0000","last_peered":"2023-05-16T07:36:34.619097+0000","last_clean":"2023-05-16T07:36:34.619097+0000","last_became_active":"2023-05-16T07:36:34.614181+0000","last_became_peered":"2023-05-16T07:36:34.614181+0000","last_unstale":"2023-05-16T07:36:34.619097+0000","last_undegraded":"2023-05-16T07:36:34.619097+0000","last_fullsized":"2023-05-16T07:36:34.619097+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986237+0000","last_change":"2023-05-16T07:36:34.986237+0000","last_active":"2023-05-16T07:36:34.986237+0000","last_peered":"2023-05-16T07:36:34.986237+0000","last_clean":"2023-05-16T07:36:34.986237+0000","last_became_active":"2023-05-16T07:36:34.986004+0000","last_became_peered":"2023-05-16T07:36:34.986004+0000","last_unstale":"2023-05-16T07:36:34.986237+0000","last_undegraded":"2023-05-16T07:36:34.986237+0000","last_fullsized":"2023-05-16T07:36:34.986237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619421+0000","last_change":"2023-05-16T07:36:34.609318+0000","last_active":"2023-05-16T07:36:34.619421+0000","last_peered":"2023-05-16T07:36:34.619421+0000","last_clean":"2023-05-16T07:36:34.619421+0000","last_became_active":"2023-05-16T07:36:34.609025+0000","last_became_peered":"2023-05-16T07:36:34.609025+0000","last_unstale":"2023-05-16T07:36:34.619421+0000","last_undegraded":"2023-05-16T07:36:34.619421+0000","last_fullsized":"2023-05-16T07:36:34.619421+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984987+0000","last_change":"2023-05-16T07:36:34.984987+0000","last_active":"2023-05-16T07:36:34.984987+0000","last_peered":"2023-05-16T07:36:34.984987+0000","last_clean":"2023-05-16T07:36:34.984987+0000","last_became_active":"2023-05-16T07:36:34.984698+0000","last_became_peered":"2023-05-16T07:36:34.984698+0000","last_unstale":"2023-05-16T07:36:34.984987+0000","last_undegraded":"2023-05-16T07:36:34.984987+0000","last_fullsized":"2023-05-16T07:36:34.984987+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621248+0000","last_change":"2023-05-16T07:36:34.618135+0000","last_active":"2023-05-16T07:36:34.621248+0000","last_peered":"2023-05-16T07:36:34.621248+0000","last_clean":"2023-05-16T07:36:34.621248+0000","last_became_active":"2023-05-16T07:36:34.617856+0000","last_became_peered":"2023-05-16T07:36:34.617856+0000","last_unstale":"2023-05-16T07:36:34.621248+0000","last_undegraded":"2023-05-16T07:36:34.621248+0000","last_fullsized":"2023-05-16T07:36:34.621248+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982142+0000","last_change":"2023-05-16T07:36:34.982142+0000","last_active":"2023-05-16T07:36:34.982142+0000","last_peered":"2023-05-16T07:36:34.982142+0000","last_clean":"2023-05-16T07:36:34.982142+0000","last_became_active":"2023-05-16T07:36:34.981490+0000","last_became_peered":"2023-05-16T07:36:34.981490+0000","last_unstale":"2023-05-16T07:36:34.982142+0000","last_undegraded":"2023-05-16T07:36:34.982142+0000","last_fullsized":"2023-05-16T07:36:34.982142+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622410+0000","last_change":"2023-05-16T07:36:34.621445+0000","last_active":"2023-05-16T07:36:34.622410+0000","last_peered":"2023-05-16T07:36:34.622410+0000","last_clean":"2023-05-16T07:36:34.622410+0000","last_became_active":"2023-05-16T07:36:34.620796+0000","last_became_peered":"2023-05-16T07:36:34.620796+0000","last_unstale":"2023-05-16T07:36:34.622410+0000","last_undegraded":"2023-05-16T07:36:34.622410+0000","last_fullsized":"2023-05-16T07:36:34.622410+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989141+0000","last_change":"2023-05-16T07:36:34.989141+0000","last_active":"2023-05-16T07:36:34.989141+0000","last_peered":"2023-05-16T07:36:34.989141+0000","last_clean":"2023-05-16T07:36:34.989141+0000","last_became_active":"2023-05-16T07:36:34.985473+0000","last_became_peered":"2023-05-16T07:36:34.985473+0000","last_unstale":"2023-05-16T07:36:34.989141+0000","last_undegraded":"2023-05-16T07:36:34.989141+0000","last_fullsized":"2023-05-16T07:36:34.989141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618631+0000","last_change":"2023-05-16T07:36:34.615595+0000","last_active":"2023-05-16T07:36:34.618631+0000","last_peered":"2023-05-16T07:36:34.618631+0000","last_clean":"2023-05-16T07:36:34.618631+0000","last_became_active":"2023-05-16T07:36:34.614397+0000","last_became_peered":"2023-05-16T07:36:34.614397+0000","last_unstale":"2023-05-16T07:36:34.618631+0000","last_undegraded":"2023-05-16T07:36:34.618631+0000","last_fullsized":"2023-05-16T07:36:34.618631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.988261+0000","last_change":"2023-05-16T07:36:34.988261+0000","last_active":"2023-05-16T07:36:34.988261+0000","last_peered":"2023-05-16T07:36:34.988261+0000","last_clean":"2023-05-16T07:36:34.988261+0000","last_became_active":"2023-05-16T07:36:34.987993+0000","last_became_peered":"2023-05-16T07:36:34.987993+0000","last_unstale":"2023-05-16T07:36:34.988261+0000","last_undegraded":"2023-05-16T07:36:34.988261+0000","last_fullsized":"2023-05-16T07:36:34.988261+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.970267+0000","last_change":"2023-05-16T07:36:34.619459+0000","last_active":"2023-05-16T07:36:34.970267+0000","last_peered":"2023-05-16T07:36:34.970267+0000","last_clean":"2023-05-16T07:36:34.970267+0000","last_became_active":"2023-05-16T07:36:34.619163+0000","last_became_peered":"2023-05-16T07:36:34.619163+0000","last_unstale":"2023-05-16T07:36:34.970267+0000","last_undegraded":"2023-05-16T07:36:34.970267+0000","last_fullsized":"2023-05-16T07:36:34.970267+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990373+0000","last_change":"2023-05-16T07:36:34.990373+0000","last_active":"2023-05-16T07:36:34.990373+0000","last_peered":"2023-05-16T07:36:34.990373+0000","last_clean":"2023-05-16T07:36:34.990373+0000","last_became_active":"2023-05-16T07:36:34.990098+0000","last_became_peered":"2023-05-16T07:36:34.990098+0000","last_unstale":"2023-05-16T07:36:34.990373+0000","last_undegraded":"2023-05-16T07:36:34.990373+0000","last_fullsized":"2023-05-16T07:36:34.990373+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.967298+0000","last_change":"2023-05-16T07:36:34.615087+0000","last_active":"2023-05-16T07:36:34.967298+0000","last_peered":"2023-05-16T07:36:34.967298+0000","last_clean":"2023-05-16T07:36:34.967298+0000","last_became_active":"2023-05-16T07:36:34.614627+0000","last_became_peered":"2023-05-16T07:36:34.614627+0000","last_unstale":"2023-05-16T07:36:34.967298+0000","last_undegraded":"2023-05-16T07:36:34.967298+0000","last_fullsized":"2023-05-16T07:36:34.967298+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989528+0000","last_change":"2023-05-16T07:36:34.989528+0000","last_active":"2023-05-16T07:36:34.989528+0000","last_peered":"2023-05-16T07:36:34.989528+0000","last_clean":"2023-05-16T07:36:34.989528+0000","last_became_active":"2023-05-16T07:36:34.989013+0000","last_became_peered":"2023-05-16T07:36:34.989013+0000","last_unstale":"2023-05-16T07:36:34.989528+0000","last_undegraded":"2023-05-16T07:36:34.989528+0000","last_fullsized":"2023-05-16T07:36:34.989528+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986844+0000","last_change":"2023-05-16T07:36:34.617560+0000","last_active":"2023-05-16T07:36:34.986844+0000","last_peered":"2023-05-16T07:36:34.986844+0000","last_clean":"2023-05-16T07:36:34.986844+0000","last_became_active":"2023-05-16T07:36:34.617153+0000","last_became_peered":"2023-05-16T07:36:34.617153+0000","last_unstale":"2023-05-16T07:36:34.986844+0000","last_undegraded":"2023-05-16T07:36:34.986844+0000","last_fullsized":"2023-05-16T07:36:34.986844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.977731+0000","last_change":"2023-05-16T07:36:34.977731+0000","last_active":"2023-05-16T07:36:34.977731+0000","last_peered":"2023-05-16T07:36:34.977731+0000","last_clean":"2023-05-16T07:36:34.977731+0000","last_became_active":"2023-05-16T07:36:34.977462+0000","last_became_peered":"2023-05-16T07:36:34.977462+0000","last_unstale":"2023-05-16T07:36:34.977731+0000","last_undegraded":"2023-05-16T07:36:34.977731+0000","last_fullsized":"2023-05-16T07:36:34.977731+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.616562+0000","last_change":"2023-05-16T07:36:34.613312+0000","last_active":"2023-05-16T07:36:34.616562+0000","last_peered":"2023-05-16T07:36:34.616562+0000","last_clean":"2023-05-16T07:36:34.616562+0000","last_became_active":"2023-05-16T07:36:34.613008+0000","last_became_peered":"2023-05-16T07:36:34.613008+0000","last_unstale":"2023-05-16T07:36:34.616562+0000","last_undegraded":"2023-05-16T07:36:34.616562+0000","last_fullsized":"2023-05-16T07:36:34.616562+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984956+0000","last_change":"2023-05-16T07:36:34.984956+0000","last_active":"2023-05-16T07:36:34.984956+0000","last_peered":"2023-05-16T07:36:34.984956+0000","last_clean":"2023-05-16T07:36:34.984956+0000","last_became_active":"2023-05-16T07:36:34.984620+0000","last_became_peered":"2023-05-16T07:36:34.984620+0000","last_unstale":"2023-05-16T07:36:34.984956+0000","last_undegraded":"2023-05-16T07:36:34.984956+0000","last_fullsized":"2023-05-16T07:36:34.984956+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.613864+0000","last_change":"2023-05-16T07:36:34.610195+0000","last_active":"2023-05-16T07:36:34.613864+0000","last_peered":"2023-05-16T07:36:34.613864+0000","last_clean":"2023-05-16T07:36:34.613864+0000","last_became_active":"2023-05-16T07:36:34.609930+0000","last_became_peered":"2023-05-16T07:36:34.609930+0000","last_unstale":"2023-05-16T07:36:34.613864+0000","last_undegraded":"2023-05-16T07:36:34.613864+0000","last_fullsized":"2023-05-16T07:36:34.613864+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989470+0000","last_change":"2023-05-16T07:36:34.989470+0000","last_active":"2023-05-16T07:36:34.989470+0000","last_peered":"2023-05-16T07:36:34.989470+0000","last_clean":"2023-05-16T07:36:34.989470+0000","last_became_active":"2023-05-16T07:36:34.989249+0000","last_became_peered":"2023-05-16T07:36:34.989249+0000","last_unstale":"2023-05-16T07:36:34.989470+0000","last_undegraded":"2023-05-16T07:36:34.989470+0000","last_fullsized":"2023-05-16T07:36:34.989470+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.617845+0000","last_change":"2023-05-16T07:36:34.612461+0000","last_active":"2023-05-16T07:36:34.617845+0000","last_peered":"2023-05-16T07:36:34.617845+0000","last_clean":"2023-05-16T07:36:34.617845+0000","last_became_active":"2023-05-16T07:36:34.612206+0000","last_became_peered":"2023-05-16T07:36:34.612206+0000","last_unstale":"2023-05-16T07:36:34.617845+0000","last_undegraded":"2023-05-16T07:36:34.617845+0000","last_fullsized":"2023-05-16T07:36:34.617845+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.975366+0000","last_change":"2023-05-16T07:36:34.975366+0000","last_active":"2023-05-16T07:36:34.975366+0000","last_peered":"2023-05-16T07:36:34.975366+0000","last_clean":"2023-05-16T07:36:34.975366+0000","last_became_active":"2023-05-16T07:36:34.975111+0000","last_became_peered":"2023-05-16T07:36:34.975111+0000","last_unstale":"2023-05-16T07:36:34.975366+0000","last_undegraded":"2023-05-16T07:36:34.975366+0000","last_fullsized":"2023-05-16T07:36:34.975366+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620639+0000","last_change":"2023-05-16T07:36:34.618794+0000","last_active":"2023-05-16T07:36:34.620639+0000","last_peered":"2023-05-16T07:36:34.620639+0000","last_clean":"2023-05-16T07:36:34.620639+0000","last_became_active":"2023-05-16T07:36:34.618524+0000","last_became_peered":"2023-05-16T07:36:34.618524+0000","last_unstale":"2023-05-16T07:36:34.620639+0000","last_undegraded":"2023-05-16T07:36:34.620639+0000","last_fullsized":"2023-05-16T07:36:34.620639+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985348+0000","last_change":"2023-05-16T07:36:34.985348+0000","last_active":"2023-05-16T07:36:34.985348+0000","last_peered":"2023-05-16T07:36:34.985348+0000","last_clean":"2023-05-16T07:36:34.985348+0000","last_became_active":"2023-05-16T07:36:34.985035+0000","last_became_peered":"2023-05-16T07:36:34.985035+0000","last_unstale":"2023-05-16T07:36:34.985348+0000","last_undegraded":"2023-05-16T07:36:34.985348+0000","last_fullsized":"2023-05-16T07:36:34.985348+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619981+0000","last_change":"2023-05-16T07:36:34.614893+0000","last_active":"2023-05-16T07:36:34.619981+0000","last_peered":"2023-05-16T07:36:34.619981+0000","last_clean":"2023-05-16T07:36:34.619981+0000","last_became_active":"2023-05-16T07:36:34.614627+0000","last_became_peered":"2023-05-16T07:36:34.614627+0000","last_unstale":"2023-05-16T07:36:34.619981+0000","last_undegraded":"2023-05-16T07:36:34.619981+0000","last_fullsized":"2023-05-16T07:36:34.619981+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984697+0000","last_change":"2023-05-16T07:36:34.984697+0000","last_active":"2023-05-16T07:36:34.984697+0000","last_peered":"2023-05-16T07:36:34.984697+0000","last_clean":"2023-05-16T07:36:34.984697+0000","last_became_active":"2023-05-16T07:36:34.984426+0000","last_became_peered":"2023-05-16T07:36:34.984426+0000","last_unstale":"2023-05-16T07:36:34.984697+0000","last_undegraded":"2023-05-16T07:36:34.984697+0000","last_fullsized":"2023-05-16T07:36:34.984697+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.979653+0000","last_change":"2023-05-16T07:36:34.614052+0000","last_active":"2023-05-16T07:36:34.979653+0000","last_peered":"2023-05-16T07:36:34.979653+0000","last_clean":"2023-05-16T07:36:34.979653+0000","last_became_active":"2023-05-16T07:36:34.613764+0000","last_became_peered":"2023-05-16T07:36:34.613764+0000","last_unstale":"2023-05-16T07:36:34.979653+0000","last_undegraded":"2023-05-16T07:36:34.979653+0000","last_fullsized":"2023-05-16T07:36:34.979653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987880+0000","last_change":"2023-05-16T07:36:34.987880+0000","last_active":"2023-05-16T07:36:34.987880+0000","last_peered":"2023-05-16T07:36:34.987880+0000","last_clean":"2023-05-16T07:36:34.987880+0000","last_became_active":"2023-05-16T07:36:34.987601+0000","last_became_peered":"2023-05-16T07:36:34.987601+0000","last_unstale":"2023-05-16T07:36:34.987880+0000","last_undegraded":"2023-05-16T07:36:34.987880+0000","last_fullsized":"2023-05-16T07:36:34.987880+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982386+0000","last_change":"2023-05-16T07:36:34.619298+0000","last_active":"2023-05-16T07:36:34.982386+0000","last_peered":"2023-05-16T07:36:34.982386+0000","last_clean":"2023-05-16T07:36:34.982386+0000","last_became_active":"2023-05-16T07:36:34.619023+0000","last_became_peered":"2023-05-16T07:36:34.619023+0000","last_unstale":"2023-05-16T07:36:34.982386+0000","last_undegraded":"2023-05-16T07:36:34.982386+0000","last_fullsized":"2023-05-16T07:36:34.982386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985195+0000","last_change":"2023-05-16T07:36:34.985195+0000","last_active":"2023-05-16T07:36:34.985195+0000","last_peered":"2023-05-16T07:36:34.985195+0000","last_clean":"2023-05-16T07:36:34.985195+0000","last_became_active":"2023-05-16T07:36:34.984345+0000","last_became_peered":"2023-05-16T07:36:34.984345+0000","last_unstale":"2023-05-16T07:36:34.985195+0000","last_undegraded":"2023-05-16T07:36:34.985195+0000","last_fullsized":"2023-05-16T07:36:34.985195+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620666+0000","last_change":"2023-05-16T07:36:34.619313+0000","last_active":"2023-05-16T07:36:34.620666+0000","last_peered":"2023-05-16T07:36:34.620666+0000","last_clean":"2023-05-16T07:36:34.620666+0000","last_became_active":"2023-05-16T07:36:34.619061+0000","last_became_peered":"2023-05-16T07:36:34.619061+0000","last_unstale":"2023-05-16T07:36:34.620666+0000","last_undegraded":"2023-05-16T07:36:34.620666+0000","last_fullsized":"2023-05-16T07:36:34.620666+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992971+0000","last_change":"2023-05-16T07:36:34.992971+0000","last_active":"2023-05-16T07:36:34.992971+0000","last_peered":"2023-05-16T07:36:34.992971+0000","last_clean":"2023-05-16T07:36:34.992971+0000","last_became_active":"2023-05-16T07:36:34.992501+0000","last_became_peered":"2023-05-16T07:36:34.992501+0000","last_unstale":"2023-05-16T07:36:34.992971+0000","last_undegraded":"2023-05-16T07:36:34.992971+0000","last_fullsized":"2023-05-16T07:36:34.992971+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619810+0000","last_change":"2023-05-16T07:36:34.617109+0000","last_active":"2023-05-16T07:36:34.619810+0000","last_peered":"2023-05-16T07:36:34.619810+0000","last_clean":"2023-05-16T07:36:34.619810+0000","last_became_active":"2023-05-16T07:36:34.616829+0000","last_became_peered":"2023-05-16T07:36:34.616829+0000","last_unstale":"2023-05-16T07:36:34.619810+0000","last_undegraded":"2023-05-16T07:36:34.619810+0000","last_fullsized":"2023-05-16T07:36:34.619810+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.978670+0000","last_change":"2023-05-16T07:36:34.978670+0000","last_active":"2023-05-16T07:36:34.978670+0000","last_peered":"2023-05-16T07:36:34.978670+0000","last_clean":"2023-05-16T07:36:34.978670+0000","last_became_active":"2023-05-16T07:36:34.978400+0000","last_became_peered":"2023-05-16T07:36:34.978400+0000","last_unstale":"2023-05-16T07:36:34.978670+0000","last_undegraded":"2023-05-16T07:36:34.978670+0000","last_fullsized":"2023-05-16T07:36:34.978670+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.978856+0000","last_change":"2023-05-16T07:36:34.612723+0000","last_active":"2023-05-16T07:36:34.978856+0000","last_peered":"2023-05-16T07:36:34.978856+0000","last_clean":"2023-05-16T07:36:34.978856+0000","last_became_active":"2023-05-16T07:36:34.612325+0000","last_became_peered":"2023-05-16T07:36:34.612325+0000","last_unstale":"2023-05-16T07:36:34.978856+0000","last_undegraded":"2023-05-16T07:36:34.978856+0000","last_fullsized":"2023-05-16T07:36:34.978856+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985599+0000","last_change":"2023-05-16T07:36:34.985599+0000","last_active":"2023-05-16T07:36:34.985599+0000","last_peered":"2023-05-16T07:36:34.985599+0000","last_clean":"2023-05-16T07:36:34.985599+0000","last_became_active":"2023-05-16T07:36:34.985333+0000","last_became_peered":"2023-05-16T07:36:34.985333+0000","last_unstale":"2023-05-16T07:36:34.985599+0000","last_undegraded":"2023-05-16T07:36:34.985599+0000","last_fullsized":"2023-05-16T07:36:34.985599+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620322+0000","last_change":"2023-05-16T07:36:34.614480+0000","last_active":"2023-05-16T07:36:34.620322+0000","last_peered":"2023-05-16T07:36:34.620322+0000","last_clean":"2023-05-16T07:36:34.620322+0000","last_became_active":"2023-05-16T07:36:34.614189+0000","last_became_peered":"2023-05-16T07:36:34.614189+0000","last_unstale":"2023-05-16T07:36:34.620322+0000","last_undegraded":"2023-05-16T07:36:34.620322+0000","last_fullsized":"2023-05-16T07:36:34.620322+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984760+0000","last_change":"2023-05-16T07:36:34.984760+0000","last_active":"2023-05-16T07:36:34.984760+0000","last_peered":"2023-05-16T07:36:34.984760+0000","last_clean":"2023-05-16T07:36:34.984760+0000","last_became_active":"2023-05-16T07:36:34.984499+0000","last_became_peered":"2023-05-16T07:36:34.984499+0000","last_unstale":"2023-05-16T07:36:34.984760+0000","last_undegraded":"2023-05-16T07:36:34.984760+0000","last_fullsized":"2023-05-16T07:36:34.984760+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622699+0000","last_change":"2023-05-16T07:36:34.621657+0000","last_active":"2023-05-16T07:36:34.622699+0000","last_peered":"2023-05-16T07:36:34.622699+0000","last_clean":"2023-05-16T07:36:34.622699+0000","last_became_active":"2023-05-16T07:36:34.621156+0000","last_became_peered":"2023-05-16T07:36:34.621156+0000","last_unstale":"2023-05-16T07:36:34.622699+0000","last_undegraded":"2023-05-16T07:36:34.622699+0000","last_fullsized":"2023-05-16T07:36:34.622699+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.995605+0000","last_change":"2023-05-16T07:36:34.995605+0000","last_active":"2023-05-16T07:36:34.995605+0000","last_peered":"2023-05-16T07:36:34.995605+0000","last_clean":"2023-05-16T07:36:34.995605+0000","last_became_active":"2023-05-16T07:36:34.995329+0000","last_became_peered":"2023-05-16T07:36:34.995329+0000","last_unstale":"2023-05-16T07:36:34.995605+0000","last_undegraded":"2023-05-16T07:36:34.995605+0000","last_fullsized":"2023-05-16T07:36:34.995605+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615942+0000","last_change":"2023-05-16T07:36:34.610720+0000","last_active":"2023-05-16T07:36:34.615942+0000","last_peered":"2023-05-16T07:36:34.615942+0000","last_clean":"2023-05-16T07:36:34.615942+0000","last_became_active":"2023-05-16T07:36:34.610449+0000","last_became_peered":"2023-05-16T07:36:34.610449+0000","last_unstale":"2023-05-16T07:36:34.615942+0000","last_undegraded":"2023-05-16T07:36:34.615942+0000","last_fullsized":"2023-05-16T07:36:34.615942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.974933+0000","last_change":"2023-05-16T07:36:34.974933+0000","last_active":"2023-05-16T07:36:34.974933+0000","last_peered":"2023-05-16T07:36:34.974933+0000","last_clean":"2023-05-16T07:36:34.974933+0000","last_became_active":"2023-05-16T07:36:34.974668+0000","last_became_peered":"2023-05-16T07:36:34.974668+0000","last_unstale":"2023-05-16T07:36:34.974933+0000","last_undegraded":"2023-05-16T07:36:34.974933+0000","last_fullsized":"2023-05-16T07:36:34.974933+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992918+0000","last_change":"2023-05-16T07:36:34.616834+0000","last_active":"2023-05-16T07:36:34.992918+0000","last_peered":"2023-05-16T07:36:34.992918+0000","last_clean":"2023-05-16T07:36:34.992918+0000","last_became_active":"2023-05-16T07:36:34.616550+0000","last_became_peered":"2023-05-16T07:36:34.616550+0000","last_unstale":"2023-05-16T07:36:34.992918+0000","last_undegraded":"2023-05-16T07:36:34.992918+0000","last_fullsized":"2023-05-16T07:36:34.992918+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983542+0000","last_change":"2023-05-16T07:36:34.983542+0000","last_active":"2023-05-16T07:36:34.983542+0000","last_peered":"2023-05-16T07:36:34.983542+0000","last_clean":"2023-05-16T07:36:34.983542+0000","last_became_active":"2023-05-16T07:36:34.983275+0000","last_became_peered":"2023-05-16T07:36:34.983275+0000","last_unstale":"2023-05-16T07:36:34.983542+0000","last_undegraded":"2023-05-16T07:36:34.983542+0000","last_fullsized":"2023-05-16T07:36:34.983542+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618653+0000","last_change":"2023-05-16T07:36:34.615853+0000","last_active":"2023-05-16T07:36:34.618653+0000","last_peered":"2023-05-16T07:36:34.618653+0000","last_clean":"2023-05-16T07:36:34.618653+0000","last_became_active":"2023-05-16T07:36:34.614255+0000","last_became_peered":"2023-05-16T07:36:34.614255+0000","last_unstale":"2023-05-16T07:36:34.618653+0000","last_undegraded":"2023-05-16T07:36:34.618653+0000","last_fullsized":"2023-05-16T07:36:34.618653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992017+0000","last_change":"2023-05-16T07:36:34.992017+0000","last_active":"2023-05-16T07:36:34.992017+0000","last_peered":"2023-05-16T07:36:34.992017+0000","last_clean":"2023-05-16T07:36:34.992017+0000","last_became_active":"2023-05-16T07:36:34.991757+0000","last_became_peered":"2023-05-16T07:36:34.991757+0000","last_unstale":"2023-05-16T07:36:34.992017+0000","last_undegraded":"2023-05-16T07:36:34.992017+0000","last_fullsized":"2023-05-16T07:36:34.992017+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620522+0000","last_change":"2023-05-16T07:36:34.618066+0000","last_active":"2023-05-16T07:36:34.620522+0000","last_peered":"2023-05-16T07:36:34.620522+0000","last_clean":"2023-05-16T07:36:34.620522+0000","last_became_active":"2023-05-16T07:36:34.617837+0000","last_became_peered":"2023-05-16T07:36:34.617837+0000","last_unstale":"2023-05-16T07:36:34.620522+0000","last_undegraded":"2023-05-16T07:36:34.620522+0000","last_fullsized":"2023-05-16T07:36:34.620522+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986256+0000","last_change":"2023-05-16T07:36:34.986256+0000","last_active":"2023-05-16T07:36:34.986256+0000","last_peered":"2023-05-16T07:36:34.986256+0000","last_clean":"2023-05-16T07:36:34.986256+0000","last_became_active":"2023-05-16T07:36:34.985988+0000","last_became_peered":"2023-05-16T07:36:34.985988+0000","last_unstale":"2023-05-16T07:36:34.986256+0000","last_undegraded":"2023-05-16T07:36:34.986256+0000","last_fullsized":"2023-05-16T07:36:34.986256+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615919+0000","last_change":"2023-05-16T07:36:34.612892+0000","last_active":"2023-05-16T07:36:34.615919+0000","last_peered":"2023-05-16T07:36:34.615919+0000","last_clean":"2023-05-16T07:36:34.615919+0000","last_became_active":"2023-05-16T07:36:34.612628+0000","last_became_peered":"2023-05-16T07:36:34.612628+0000","last_unstale":"2023-05-16T07:36:34.615919+0000","last_undegraded":"2023-05-16T07:36:34.615919+0000","last_fullsized":"2023-05-16T07:36:34.615919+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980103+0000","last_change":"2023-05-16T07:36:34.980103+0000","last_active":"2023-05-16T07:36:34.980103+0000","last_peered":"2023-05-16T07:36:34.980103+0000","last_clean":"2023-05-16T07:36:34.980103+0000","last_became_active":"2023-05-16T07:36:34.979829+0000","last_became_peered":"2023-05-16T07:36:34.979829+0000","last_unstale":"2023-05-16T07:36:34.980103+0000","last_undegraded":"2023-05-16T07:36:34.980103+0000","last_fullsized":"2023-05-16T07:36:34.980103+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622378+0000","last_change":"2023-05-16T07:36:34.618023+0000","last_active":"2023-05-16T07:36:34.622378+0000","last_peered":"2023-05-16T07:36:34.622378+0000","last_clean":"2023-05-16T07:36:34.622378+0000","last_became_active":"2023-05-16T07:36:34.617623+0000","last_became_peered":"2023-05-16T07:36:34.617623+0000","last_unstale":"2023-05-16T07:36:34.622378+0000","last_undegraded":"2023-05-16T07:36:34.622378+0000","last_fullsized":"2023-05-16T07:36:34.622378+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980083+0000","last_change":"2023-05-16T07:36:34.980083+0000","last_active":"2023-05-16T07:36:34.980083+0000","last_peered":"2023-05-16T07:36:34.980083+0000","last_clean":"2023-05-16T07:36:34.980083+0000","last_became_active":"2023-05-16T07:36:34.979812+0000","last_became_peered":"2023-05-16T07:36:34.979812+0000","last_unstale":"2023-05-16T07:36:34.980083+0000","last_undegraded":"2023-05-16T07:36:34.980083+0000","last_fullsized":"2023-05-16T07:36:34.980083+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615920+0000","last_change":"2023-05-16T07:36:34.612837+0000","last_active":"2023-05-16T07:36:34.615920+0000","last_peered":"2023-05-16T07:36:34.615920+0000","last_clean":"2023-05-16T07:36:34.615920+0000","last_became_active":"2023-05-16T07:36:34.612568+0000","last_became_peered":"2023-05-16T07:36:34.612568+0000","last_unstale":"2023-05-16T07:36:34.615920+0000","last_undegraded":"2023-05-16T07:36:34.615920+0000","last_fullsized":"2023-05-16T07:36:34.615920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986594+0000","last_change":"2023-05-16T07:36:34.986594+0000","last_active":"2023-05-16T07:36:34.986594+0000","last_peered":"2023-05-16T07:36:34.986594+0000","last_clean":"2023-05-16T07:36:34.986594+0000","last_became_active":"2023-05-16T07:36:34.986343+0000","last_became_peered":"2023-05-16T07:36:34.986343+0000","last_unstale":"2023-05-16T07:36:34.986594+0000","last_undegraded":"2023-05-16T07:36:34.986594+0000","last_fullsized":"2023-05-16T07:36:34.986594+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615744+0000","last_change":"2023-05-16T07:36:34.612134+0000","last_active":"2023-05-16T07:36:34.615744+0000","last_peered":"2023-05-16T07:36:34.615744+0000","last_clean":"2023-05-16T07:36:34.615744+0000","last_became_active":"2023-05-16T07:36:34.611857+0000","last_became_peered":"2023-05-16T07:36:34.611857+0000","last_unstale":"2023-05-16T07:36:34.615744+0000","last_undegraded":"2023-05-16T07:36:34.615744+0000","last_fullsized":"2023-05-16T07:36:34.615744+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.995270+0000","last_change":"2023-05-16T07:36:34.995270+0000","last_active":"2023-05-16T07:36:34.995270+0000","last_peered":"2023-05-16T07:36:34.995270+0000","last_clean":"2023-05-16T07:36:34.995270+0000","last_became_active":"2023-05-16T07:36:34.994987+0000","last_became_peered":"2023-05-16T07:36:34.994987+0000","last_unstale":"2023-05-16T07:36:34.995270+0000","last_undegraded":"2023-05-16T07:36:34.995270+0000","last_fullsized":"2023-05-16T07:36:34.995270+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.624357+0000","last_change":"2023-05-16T07:36:34.622141+0000","last_active":"2023-05-16T07:36:34.624357+0000","last_peered":"2023-05-16T07:36:34.624357+0000","last_clean":"2023-05-16T07:36:34.624357+0000","last_became_active":"2023-05-16T07:36:34.621906+0000","last_became_peered":"2023-05-16T07:36:34.621906+0000","last_unstale":"2023-05-16T07:36:34.624357+0000","last_undegraded":"2023-05-16T07:36:34.624357+0000","last_fullsized":"2023-05-16T07:36:34.624357+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990517+0000","last_change":"2023-05-16T07:36:34.990517+0000","last_active":"2023-05-16T07:36:34.990517+0000","last_peered":"2023-05-16T07:36:34.990517+0000","last_clean":"2023-05-16T07:36:34.990517+0000","last_became_active":"2023-05-16T07:36:34.990237+0000","last_became_peered":"2023-05-16T07:36:34.990237+0000","last_unstale":"2023-05-16T07:36:34.990517+0000","last_undegraded":"2023-05-16T07:36:34.990517+0000","last_fullsized":"2023-05-16T07:36:34.990517+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"153'3","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.994834+0000","last_change":"2023-05-16T07:36:34.621386+0000","last_active":"2023-05-16T07:36:34.994834+0000","last_peered":"2023-05-16T07:36:34.994834+0000","last_clean":"2023-05-16T07:36:34.994834+0000","last_became_active":"2023-05-16T07:36:34.621118+0000","last_became_peered":"2023-05-16T07:36:34.621118+0000","last_unstale":"2023-05-16T07:36:34.994834+0000","last_undegraded":"2023-05-16T07:36:34.994834+0000","last_fullsized":"2023-05-16T07:36:34.994834+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.605492+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:36:34.605492+0000","last_peered":"2023-05-16T07:36:34.605492+0000","last_clean":"2023-05-16T07:36:34.605492+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:36:34.605492+0000","last_undegraded":"2023-05-16T07:36:34.605492+0000","last_fullsized":"2023-05-16T07:36:34.605492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.979602+0000","last_change":"2023-05-16T07:36:34.979602+0000","last_active":"2023-05-16T07:36:34.979602+0000","last_peered":"2023-05-16T07:36:34.979602+0000","last_clean":"2023-05-16T07:36:34.979602+0000","last_became_active":"2023-05-16T07:36:34.979389+0000","last_became_peered":"2023-05-16T07:36:34.979389+0000","last_unstale":"2023-05-16T07:36:34.979602+0000","last_undegraded":"2023-05-16T07:36:34.979602+0000","last_fullsized":"2023-05-16T07:36:34.979602+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.971966+0000","last_change":"2023-05-16T07:36:34.616220+0000","last_active":"2023-05-16T07:36:34.971966+0000","last_peered":"2023-05-16T07:36:34.971966+0000","last_clean":"2023-05-16T07:36:34.971966+0000","last_became_active":"2023-05-16T07:36:34.614466+0000","last_became_peered":"2023-05-16T07:36:34.614466+0000","last_unstale":"2023-05-16T07:36:34.971966+0000","last_undegraded":"2023-05-16T07:36:34.971966+0000","last_fullsized":"2023-05-16T07:36:34.971966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":117,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.610427+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:36:34.610427+0000","last_peered":"2023-05-16T07:36:34.610427+0000","last_clean":"2023-05-16T07:36:34.610427+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:36:34.610427+0000","last_undegraded":"2023-05-16T07:36:34.610427+0000","last_fullsized":"2023-05-16T07:36:34.610427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980212+0000","last_change":"2023-05-16T07:36:34.980212+0000","last_active":"2023-05-16T07:36:34.980212+0000","last_peered":"2023-05-16T07:36:34.980212+0000","last_clean":"2023-05-16T07:36:34.980212+0000","last_became_active":"2023-05-16T07:36:34.979958+0000","last_became_peered":"2023-05-16T07:36:34.979958+0000","last_unstale":"2023-05-16T07:36:34.980212+0000","last_undegraded":"2023-05-16T07:36:34.980212+0000","last_fullsized":"2023-05-16T07:36:34.980212+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991627+0000","last_change":"2023-05-16T07:36:34.613619+0000","last_active":"2023-05-16T07:36:34.991627+0000","last_peered":"2023-05-16T07:36:34.991627+0000","last_clean":"2023-05-16T07:36:34.991627+0000","last_became_active":"2023-05-16T07:36:34.613332+0000","last_became_peered":"2023-05-16T07:36:34.613332+0000","last_unstale":"2023-05-16T07:36:34.991627+0000","last_undegraded":"2023-05-16T07:36:34.991627+0000","last_fullsized":"2023-05-16T07:36:34.991627+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.609923+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:36:34.609923+0000","last_peered":"2023-05-16T07:36:34.609923+0000","last_clean":"2023-05-16T07:36:34.609923+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:36:34.609923+0000","last_undegraded":"2023-05-16T07:36:34.609923+0000","last_fullsized":"2023-05-16T07:36:34.609923+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991823+0000","last_change":"2023-05-16T07:36:34.991823+0000","last_active":"2023-05-16T07:36:34.991823+0000","last_peered":"2023-05-16T07:36:34.991823+0000","last_clean":"2023-05-16T07:36:34.991823+0000","last_became_active":"2023-05-16T07:36:34.991526+0000","last_became_peered":"2023-05-16T07:36:34.991526+0000","last_unstale":"2023-05-16T07:36:34.991823+0000","last_undegraded":"2023-05-16T07:36:34.991823+0000","last_fullsized":"2023-05-16T07:36:34.991823+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622344+0000","last_change":"2023-05-16T07:36:34.621108+0000","last_active":"2023-05-16T07:36:34.622344+0000","last_peered":"2023-05-16T07:36:34.622344+0000","last_clean":"2023-05-16T07:36:34.622344+0000","last_became_active":"2023-05-16T07:36:34.620846+0000","last_became_peered":"2023-05-16T07:36:34.620846+0000","last_unstale":"2023-05-16T07:36:34.622344+0000","last_undegraded":"2023-05-16T07:36:34.622344+0000","last_fullsized":"2023-05-16T07:36:34.622344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":117,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.611155+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:36:34.611155+0000","last_peered":"2023-05-16T07:36:34.611155+0000","last_clean":"2023-05-16T07:36:34.611155+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:36:34.611155+0000","last_undegraded":"2023-05-16T07:36:34.611155+0000","last_fullsized":"2023-05-16T07:36:34.611155+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.604838+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:36:34.604838+0000","last_peered":"2023-05-16T07:36:34.604838+0000","last_clean":"2023-05-16T07:36:34.604838+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:36:34.604838+0000","last_undegraded":"2023-05-16T07:36:34.604838+0000","last_fullsized":"2023-05-16T07:36:34.604838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990228+0000","last_change":"2023-05-16T07:36:34.990228+0000","last_active":"2023-05-16T07:36:34.990228+0000","last_peered":"2023-05-16T07:36:34.990228+0000","last_clean":"2023-05-16T07:36:34.990228+0000","last_became_active":"2023-05-16T07:36:34.989391+0000","last_became_peered":"2023-05-16T07:36:34.989391+0000","last_unstale":"2023-05-16T07:36:34.990228+0000","last_undegraded":"2023-05-16T07:36:34.990228+0000","last_fullsized":"2023-05-16T07:36:34.990228+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620451+0000","last_change":"2023-05-16T07:36:34.617741+0000","last_active":"2023-05-16T07:36:34.620451+0000","last_peered":"2023-05-16T07:36:34.620451+0000","last_clean":"2023-05-16T07:36:34.620451+0000","last_became_active":"2023-05-16T07:36:34.617064+0000","last_became_peered":"2023-05-16T07:36:34.617064+0000","last_unstale":"2023-05-16T07:36:34.620451+0000","last_undegraded":"2023-05-16T07:36:34.620451+0000","last_fullsized":"2023-05-16T07:36:34.620451+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.604941+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:36:34.604941+0000","last_peered":"2023-05-16T07:36:34.604941+0000","last_clean":"2023-05-16T07:36:34.604941+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:36:34.604941+0000","last_undegraded":"2023-05-16T07:36:34.604941+0000","last_fullsized":"2023-05-16T07:36:34.604941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982352+0000","last_change":"2023-05-16T07:36:34.982352+0000","last_active":"2023-05-16T07:36:34.982352+0000","last_peered":"2023-05-16T07:36:34.982352+0000","last_clean":"2023-05-16T07:36:34.982352+0000","last_became_active":"2023-05-16T07:36:34.982074+0000","last_became_peered":"2023-05-16T07:36:34.982074+0000","last_unstale":"2023-05-16T07:36:34.982352+0000","last_undegraded":"2023-05-16T07:36:34.982352+0000","last_fullsized":"2023-05-16T07:36:34.982352+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987087+0000","last_change":"2023-05-16T07:36:34.618250+0000","last_active":"2023-05-16T07:36:34.987087+0000","last_peered":"2023-05-16T07:36:34.987087+0000","last_clean":"2023-05-16T07:36:34.987087+0000","last_became_active":"2023-05-16T07:36:34.617991+0000","last_became_peered":"2023-05-16T07:36:34.617991+0000","last_unstale":"2023-05-16T07:36:34.987087+0000","last_undegraded":"2023-05-16T07:36:34.987087+0000","last_fullsized":"2023-05-16T07:36:34.987087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.609006+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:36:34.609006+0000","last_peered":"2023-05-16T07:36:34.609006+0000","last_clean":"2023-05-16T07:36:34.609006+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:36:34.609006+0000","last_undegraded":"2023-05-16T07:36:34.609006+0000","last_fullsized":"2023-05-16T07:36:34.609006+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991949+0000","last_change":"2023-05-16T07:36:34.991949+0000","last_active":"2023-05-16T07:36:34.991949+0000","last_peered":"2023-05-16T07:36:34.991949+0000","last_clean":"2023-05-16T07:36:34.991949+0000","last_became_active":"2023-05-16T07:36:34.991707+0000","last_became_peered":"2023-05-16T07:36:34.991707+0000","last_unstale":"2023-05-16T07:36:34.991949+0000","last_undegraded":"2023-05-16T07:36:34.991949+0000","last_fullsized":"2023-05-16T07:36:34.991949+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622436+0000","last_change":"2023-05-16T07:36:34.618778+0000","last_active":"2023-05-16T07:36:34.622436+0000","last_peered":"2023-05-16T07:36:34.622436+0000","last_clean":"2023-05-16T07:36:34.622436+0000","last_became_active":"2023-05-16T07:36:34.618463+0000","last_became_peered":"2023-05-16T07:36:34.618463+0000","last_unstale":"2023-05-16T07:36:34.622436+0000","last_undegraded":"2023-05-16T07:36:34.622436+0000","last_fullsized":"2023-05-16T07:36:34.622436+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.604595+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:36:34.604595+0000","last_peered":"2023-05-16T07:36:34.604595+0000","last_clean":"2023-05-16T07:36:34.604595+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:36:34.604595+0000","last_undegraded":"2023-05-16T07:36:34.604595+0000","last_fullsized":"2023-05-16T07:36:34.604595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.981458+0000","last_change":"2023-05-16T07:36:34.981458+0000","last_active":"2023-05-16T07:36:34.981458+0000","last_peered":"2023-05-16T07:36:34.981458+0000","last_clean":"2023-05-16T07:36:34.981458+0000","last_became_active":"2023-05-16T07:36:34.980886+0000","last_became_peered":"2023-05-16T07:36:34.980886+0000","last_unstale":"2023-05-16T07:36:34.981458+0000","last_undegraded":"2023-05-16T07:36:34.981458+0000","last_fullsized":"2023-05-16T07:36:34.981458+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621556+0000","last_change":"2023-05-16T07:36:34.614164+0000","last_active":"2023-05-16T07:36:34.621556+0000","last_peered":"2023-05-16T07:36:34.621556+0000","last_clean":"2023-05-16T07:36:34.621556+0000","last_became_active":"2023-05-16T07:36:34.613896+0000","last_became_peered":"2023-05-16T07:36:34.613896+0000","last_unstale":"2023-05-16T07:36:34.621556+0000","last_undegraded":"2023-05-16T07:36:34.621556+0000","last_fullsized":"2023-05-16T07:36:34.621556+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.605849+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:36:34.605849+0000","last_peered":"2023-05-16T07:36:34.605849+0000","last_clean":"2023-05-16T07:36:34.605849+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:36:34.605849+0000","last_undegraded":"2023-05-16T07:36:34.605849+0000","last_fullsized":"2023-05-16T07:36:34.605849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575253,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24408,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":22528,"kb_avail":94347432,"statfs":{"total":96636764160,"available":96611770368,"internally_reserved":0,"allocated":1916928,"data_stored":1014544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23068672},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.76400000000000001,"15min":0.80500000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48299999999999998}]},{"osd":1,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36499999999999999},"max":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75,"5min":0.77600000000000002,"15min":0.79800000000000004},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.80200000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56799999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81999999999999995},"last":0.46800000000000003}]},{"osd":3,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79200000000000004,"15min":0.81899999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73499999999999999,"5min":0.80000000000000004,"15min":0.81100000000000005},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46700000000000003,"15min":0.46200000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.72699999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.76000000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.28399999999999997},"max":{"1min":0.63,"5min":0.75600000000000001,"15min":0.79100000000000004},"last":0.433},{"interface":"front","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":0.80800000000000005},"last":0.438}]},{"osd":6,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":0.70599999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.38100000000000001,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.495}]}]},{"osd":6,"up_from":13,"seq":55834575253,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24064,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":94347776,"statfs":{"total":96636764160,"available":96612122624,"internally_reserved":0,"allocated":1892352,"data_stored":1005991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.26700000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.27900000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.86299999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.48299999999999998,"5min":0.438,"15min":0.33400000000000002},"max":{"1min":0.79500000000000004,"5min":1.5,"15min":1.5},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69699999999999995}]},{"osd":2,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61099999999999999,"15min":0.58199999999999996},"min":{"1min":0.50600000000000001,"5min":0.41999999999999998,"15min":0.373},"max":{"1min":0.73699999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":1.486,"15min":1.486},"last":0.65400000000000003}]},{"osd":3,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.62},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.64600000000000002},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82999999999999996,"5min":1.532,"15min":1.532},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.377,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.76500000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.499},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.30199999999999999},"max":{"1min":0.77700000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.497},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.31900000000000001},"max":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.79900000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.44900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.81299999999999994},"last":0.81200000000000006}]},{"osd":7,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.36799999999999999,"15min":0.34699999999999998},"max":{"1min":0.80300000000000005,"5min":0.876,"15min":0.876},"last":0.754},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.99399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575254,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19776,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":17920,"kb_avail":94352064,"statfs":{"total":96636764160,"available":96616513536,"internally_reserved":0,"allocated":1892352,"data_stored":1005991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18350080},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50800000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55500000000000005,"5min":0.63,"15min":0.64100000000000001},"last":0.29599999999999999}]},{"osd":2,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38400000000000001,"15min":0.38500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.47199999999999998,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.252},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.54600000000000004,"5min":0.65700000000000003,"15min":0.70199999999999996},"last":0.23200000000000001}]},{"osd":3,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.435,"15min":0.436},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.56599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26200000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.50900000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.46300000000000002,"15min":0.38500000000000001},"max":{"1min":0.81499999999999995,"5min":0.873,"15min":0.88900000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.45000000000000001,"15min":0.40100000000000002},"max":{"1min":0.872,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.38500000000000001},"max":{"1min":0.80700000000000005,"5min":0.86399999999999999,"15min":0.91000000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85099999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.44400000000000001}]},{"osd":6,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":0.872,"15min":0.998},"last":0.51800000000000002}]},{"osd":7,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.873,"15min":0.94199999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.53100000000000003,"5min":0.48999999999999999,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.89900000000000002,"15min":0.93700000000000006},"last":0.48999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19672,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":17792,"kb_avail":94352168,"statfs":{"total":96636764160,"available":96616620032,"internally_reserved":0,"allocated":1916928,"data_stored":1013936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18219008},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.36799999999999999,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.222},"max":{"1min":0.5,"5min":0.57599999999999996,"15min":0.58899999999999997},"last":0.223},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.27500000000000002,"15min":0.24199999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.627},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39100000000000001,"15min":0.38600000000000001},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.63100000000000001},"last":0.315},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.65700000000000003},"last":0.311}]},{"osd":3,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26300000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.64600000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.438,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.70199999999999996},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80500000000000005,"5min":0.92700000000000005,"15min":0.98699999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.41099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.91000000000000003,"15min":1.0549999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84799999999999998,"5min":0.94899999999999995,"15min":1.0680000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.439},"max":{"1min":0.877,"5min":0.93500000000000005,"15min":1.095},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.94099999999999995,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.505},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.67100000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40899999999999997},"max":{"1min":0.83399999999999996,"5min":0.92900000000000005,"15min":1},"last":0.46999999999999997}]},{"osd":7,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.873,"5min":0.91500000000000004,"15min":1.03},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.44400000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.056},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575254,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21332,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":19456,"kb_avail":94350508,"statfs":{"total":96636764160,"available":96614920192,"internally_reserved":0,"allocated":1912832,"data_stored":1014398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19922944},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.47499999999999998,"5min":0.747,"15min":0.747},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.50900000000000001,"5min":0.623,"15min":0.65600000000000003},"last":0.28199999999999997}]},{"osd":1,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.41299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.48899999999999999,"5min":0.59299999999999997,"15min":0.67200000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.33200000000000002,"5min":0.32900000000000001,"15min":0.30199999999999999},"max":{"1min":0.54100000000000004,"5min":0.62,"15min":0.67200000000000004},"last":0.313}]},{"osd":3,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39000000000000001,"15min":0.39500000000000002},"min":{"1min":0.27300000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.69399999999999995},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.63600000000000001},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.62,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.377},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":1.0389999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.76700000000000002,"5min":0.90100000000000002,"15min":1.048},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.88100000000000001,"15min":1.0069999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.997},"last":0.48599999999999999}]},{"osd":6,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":1.01},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.495,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81399999999999995,"5min":0.93799999999999994,"15min":1.0229999999999999},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82699999999999996,"5min":0.92800000000000005,"15min":0.998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65100000000000002,"15min":0.66000000000000003},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.90800000000000003,"15min":0.96499999999999997},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575253,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":26240,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1912832,"data_stored":1014454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26869760},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.36399999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.51200000000000001,"15min":0.61899999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.28699999999999998}]},{"osd":1,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":0.60099999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.37,"5min":0.41399999999999998,"15min":0.41499999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.72999999999999998},"last":0.27300000000000002}]},{"osd":2,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.42999999999999999,"15min":0.433},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55200000000000005,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.315},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.44900000000000001,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.70099999999999996},"last":0.308}]},{"osd":4,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":1.014,"15min":1.014},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.371},"max":{"1min":0.78300000000000003,"5min":1.024,"15min":1.024},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":1.05,"15min":1.05},"last":0.441},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41699999999999998},"max":{"1min":0.83599999999999997,"5min":0.96599999999999997,"15min":0.97299999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.65800000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.442},"max":{"1min":0.80700000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80300000000000005,"5min":0.98899999999999999,"15min":1.012},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575253,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24212,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":22336,"kb_avail":94347628,"statfs":{"total":96636764160,"available":96611971072,"internally_reserved":0,"allocated":1912832,"data_stored":1014434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22872064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.69799999999999995,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.77800000000000002,"15min":0.83799999999999997},"last":0.439}]},{"osd":1,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.29399999999999998},"max":{"1min":0.75600000000000001,"5min":0.78600000000000003,"15min":0.81899999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.48399999999999999}]},{"osd":2,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.72999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36299999999999999},"max":{"1min":0.81000000000000005,"5min":0.82099999999999995,"15min":0.82199999999999995},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.373},"max":{"1min":0.76300000000000001,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79000000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59899999999999998}]},{"osd":5,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.29099999999999998},"max":{"1min":0.70999999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"max":{"1min":0.65600000000000003,"5min":0.755,"15min":0.755},"last":0.5},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.318},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.33500000000000002},"max":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.82099999999999995,"15min":0.85899999999999999},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575253,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":1892352,"data_stored":1005991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.70399999999999996,"5min":0.79500000000000004,"15min":0.80400000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36599999999999999},"max":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.80800000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72699999999999998,"5min":0.747,"15min":0.84699999999999998},"last":0.64100000000000001}]},{"osd":3,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.47899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79200000000000004,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.629},"min":{"1min":0.49199999999999999,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.85599999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.65700000000000003,"5min":0.74199999999999999,"15min":0.84899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.34999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.82699999999999996,"15min":0.83299999999999996},"last":0.627}]},{"osd":6,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.505,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32400000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80400000000000005,"5min":0.85299999999999998,"15min":0.86299999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.34999999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:36:37.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:36:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.099+0000 7f1a85f81700 1 Processor -- start 2023-05-16T07:36:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.100+0000 7f1a85f81700 1 -- start start 2023-05-16T07:36:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.100+0000 7f1a85f81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a800c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.100+0000 7f1a85f81700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1a80154550 0x7f1a800c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.100+0000 7f1a85f81700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a800c0660 con 0x7f1a800696a0 2023-05-16T07:36:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.100+0000 7f1a85f81700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a800c1b00 con 0x7f1a80154550 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.100+0000 7f1a85f81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a800bfd30 con 0x7f1a80154170 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a800c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7effd700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1a80154550 0x7f1a800c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a800c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:44234/0 (socket says 172.21.15.92:44234) 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7ffff700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f1a800696a0 0x7f1a80157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:37778/0 (socket says 172.21.15.92:37778) 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7f7fe700 1 -- 172.21.15.92:0/3458684981 learned_addr learned my addr 172.21.15.92:0/3458684981 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7effd700 1 -- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 msgr2=0x7f1a800c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7effd700 1 --2- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a800c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7effd700 1 -- 172.21.15.92:0/3458684981 >> v1:172.21.15.158:6789/0 conn(0x7f1a800696a0 legacy=0x7f1a80157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7effd700 1 -- 172.21.15.92:0/3458684981 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a800c0010 con 0x7f1a80154550 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7f7fe700 1 --2- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a800c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7effd700 1 --2- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a80154550 0x7f1a800c6160 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f1a74009760 tx=0x7f1a74009060).ready entity=mon.1 client_cookie=f63a5e290b12018e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7e7fc700 1 -- 172.21.15.92:0/3458684981 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740090d0 con 0x7f1a80154550 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7e7fc700 1 -- 172.21.15.92:0/3458684981 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7400f510 con 0x7f1a80154550 2023-05-16T07:36:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.101+0000 7f1a7e7fc700 1 -- 172.21.15.92:0/3458684981 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400f890 con 0x7f1a80154550 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 -- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a80154550 msgr2=0x7f1a800c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 --2- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a80154550 0x7f1a800c6160 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f1a74009760 tx=0x7f1a74009060).stop 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 -- 172.21.15.92:0/3458684981 shutdown_connections 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 --2- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a800c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 --2- 172.21.15.92:0/3458684981 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a80154550 0x7f1a800c6160 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 -- 172.21.15.92:0/3458684981 >> 172.21.15.92:0/3458684981 conn(0x7f1a800cb700 msgr2=0x7f1a800c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 -- 172.21.15.92:0/3458684981 shutdown_connections 2023-05-16T07:36:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.102+0000 7f1a85f81700 1 -- 172.21.15.92:0/3458684981 wait complete. 2023-05-16T07:36:38.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.103+0000 7f1a85f81700 1 Processor -- start 2023-05-16T07:36:38.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.103+0000 7f1a85f81700 1 -- start start 2023-05-16T07:36:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a85f81700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a800696a0 0x7f1a800af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a85f81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a80159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a85f81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 0x7f1a800c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a85f81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a80159960 con 0x7f1a80154550 2023-05-16T07:36:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a85f81700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a800c4e90 con 0x7f1a800696a0 2023-05-16T07:36:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a85f81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a800ae850 con 0x7f1a80154170 2023-05-16T07:36:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 0x7f1a800c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 0x7f1a800c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51670/0 (socket says 172.21.15.92:51670) 2023-05-16T07:36:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 -- 172.21.15.92:0/2422294634 learned_addr learned my addr 172.21.15.92:0/2422294634 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7effd700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a80159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7f7fe700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a800696a0 0x7f1a800af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 -- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 msgr2=0x7f1a80159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a80159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 -- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a800696a0 msgr2=0x7f1a800af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a800696a0 0x7f1a800af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 -- 172.21.15.92:0/2422294634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70009ae0 con 0x7f1a80154550 2023-05-16T07:36:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.104+0000 7f1a7ffff700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 0x7f1a800c1d40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f1a7800c410 tx=0x7f1a78007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.105+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a780070d0 con 0x7f1a80154550 2023-05-16T07:36:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.105+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a78007230 con 0x7f1a80154550 2023-05-16T07:36:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.105+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a740092a0 con 0x7f1a80154550 2023-05-16T07:36:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.105+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a780056e0 con 0x7f1a80154550 2023-05-16T07:36:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.105+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a800aee10 con 0x7f1a80154550 2023-05-16T07:36:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.105+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a800666c0 con 0x7f1a80154550 2023-05-16T07:36:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.106+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1a7801a060 con 0x7f1a80154550 2023-05-16T07:36:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.107+0000 7f1a677fe700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1a6c05ed90 0x7f1a6c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.107+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f1a78079990 con 0x7f1a80154550 2023-05-16T07:36:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.107+0000 7f1a7f7fe700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1a6c05ed90 0x7f1a6c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.107+0000 7f1a7f7fe700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1a6c05ed90 0x7f1a6c061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1a700072d0 tx=0x7f1a7000b040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.110+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1a7804ad30 con 0x7f1a80154550 2023-05-16T07:36:38.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.268+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a8006c350 con 0x7f1a6c05ed90 2023-05-16T07:36:38.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.278+0000 7f1a677fe700 1 -- 172.21.15.92:0/2422294634 <== mgr.7591 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361599 (secure 0 0 0) 0x7f1a8006c350 con 0x7f1a6c05ed90 2023-05-16T07:36:38.285 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:38.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1a6c05ed90 msgr2=0x7f1a6c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1a6c05ed90 0x7f1a6c061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1a700072d0 tx=0x7f1a7000b040).stop 2023-05-16T07:36:38.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 msgr2=0x7f1a800c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 0x7f1a800c1d40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f1a7800c410 tx=0x7f1a78007d70).stop 2023-05-16T07:36:38.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 shutdown_connections 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80154550 0x7f1a800c1d40 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1a800696a0 0x7f1a800af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1a6c05ed90 0x7f1a6c061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 --2- 172.21.15.92:0/2422294634 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1a80154170 0x7f1a80159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 >> 172.21.15.92:0/2422294634 conn(0x7f1a800cb700 msgr2=0x7f1a8014fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 shutdown_connections 2023-05-16T07:36:38.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.282+0000 7f1a85f81700 1 -- 172.21.15.92:0/2422294634 wait complete. 2023-05-16T07:36:38.289 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:36:38.297 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:36:36.346664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":188652,"kb_used_data":14892,"kb_used_omap":0,"kb_used_meta":173696,"kb_avail":754786068,"statfs":{"total":773094113280,"available":772900933632,"internally_reserved":0,"allocated":15249408,"data_stored":8089739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":177864704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-593589,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-208,"num_write_kb":-720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-979,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.587198"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983780+0000","last_change":"2023-05-16T07:36:34.983780+0000","last_active":"2023-05-16T07:36:34.983780+0000","last_peered":"2023-05-16T07:36:34.983780+0000","last_clean":"2023-05-16T07:36:34.983780+0000","last_became_active":"2023-05-16T07:36:34.983451+0000","last_became_peered":"2023-05-16T07:36:34.983451+0000","last_unstale":"2023-05-16T07:36:34.983780+0000","last_undegraded":"2023-05-16T07:36:34.983780+0000","last_fullsized":"2023-05-16T07:36:34.983780+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"153'3","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982822+0000","last_change":"2023-05-16T07:36:34.617363+0000","last_active":"2023-05-16T07:36:34.982822+0000","last_peered":"2023-05-16T07:36:34.982822+0000","last_clean":"2023-05-16T07:36:34.982822+0000","last_became_active":"2023-05-16T07:36:34.617058+0000","last_became_peered":"2023-05-16T07:36:34.617058+0000","last_unstale":"2023-05-16T07:36:34.982822+0000","last_undegraded":"2023-05-16T07:36:34.982822+0000","last_fullsized":"2023-05-16T07:36:34.982822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992649+0000","last_change":"2023-05-16T07:36:34.992649+0000","last_active":"2023-05-16T07:36:34.992649+0000","last_peered":"2023-05-16T07:36:34.992649+0000","last_clean":"2023-05-16T07:36:34.992649+0000","last_became_active":"2023-05-16T07:36:34.992380+0000","last_became_peered":"2023-05-16T07:36:34.992380+0000","last_unstale":"2023-05-16T07:36:34.992649+0000","last_undegraded":"2023-05-16T07:36:34.992649+0000","last_fullsized":"2023-05-16T07:36:34.992649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618704+0000","last_change":"2023-05-16T07:36:34.616207+0000","last_active":"2023-05-16T07:36:34.618704+0000","last_peered":"2023-05-16T07:36:34.618704+0000","last_clean":"2023-05-16T07:36:34.618704+0000","last_became_active":"2023-05-16T07:36:34.615750+0000","last_became_peered":"2023-05-16T07:36:34.615750+0000","last_unstale":"2023-05-16T07:36:34.618704+0000","last_undegraded":"2023-05-16T07:36:34.618704+0000","last_fullsized":"2023-05-16T07:36:34.618704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983354+0000","last_change":"2023-05-16T07:36:34.983354+0000","last_active":"2023-05-16T07:36:34.983354+0000","last_peered":"2023-05-16T07:36:34.983354+0000","last_clean":"2023-05-16T07:36:34.983354+0000","last_became_active":"2023-05-16T07:36:34.983091+0000","last_became_peered":"2023-05-16T07:36:34.983091+0000","last_unstale":"2023-05-16T07:36:34.983354+0000","last_undegraded":"2023-05-16T07:36:34.983354+0000","last_fullsized":"2023-05-16T07:36:34.983354+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619788+0000","last_change":"2023-05-16T07:36:34.611818+0000","last_active":"2023-05-16T07:36:34.619788+0000","last_peered":"2023-05-16T07:36:34.619788+0000","last_clean":"2023-05-16T07:36:34.619788+0000","last_became_active":"2023-05-16T07:36:34.611542+0000","last_became_peered":"2023-05-16T07:36:34.611542+0000","last_unstale":"2023-05-16T07:36:34.619788+0000","last_undegraded":"2023-05-16T07:36:34.619788+0000","last_fullsized":"2023-05-16T07:36:34.619788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985708+0000","last_change":"2023-05-16T07:36:34.985708+0000","last_active":"2023-05-16T07:36:34.985708+0000","last_peered":"2023-05-16T07:36:34.985708+0000","last_clean":"2023-05-16T07:36:34.985708+0000","last_became_active":"2023-05-16T07:36:34.985435+0000","last_became_peered":"2023-05-16T07:36:34.985435+0000","last_unstale":"2023-05-16T07:36:34.985708+0000","last_undegraded":"2023-05-16T07:36:34.985708+0000","last_fullsized":"2023-05-16T07:36:34.985708+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987479+0000","last_change":"2023-05-16T07:36:34.615743+0000","last_active":"2023-05-16T07:36:34.987479+0000","last_peered":"2023-05-16T07:36:34.987479+0000","last_clean":"2023-05-16T07:36:34.987479+0000","last_became_active":"2023-05-16T07:36:34.615465+0000","last_became_peered":"2023-05-16T07:36:34.615465+0000","last_unstale":"2023-05-16T07:36:34.987479+0000","last_undegraded":"2023-05-16T07:36:34.987479+0000","last_fullsized":"2023-05-16T07:36:34.987479+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989497+0000","last_change":"2023-05-16T07:36:34.989497+0000","last_active":"2023-05-16T07:36:34.989497+0000","last_peered":"2023-05-16T07:36:34.989497+0000","last_clean":"2023-05-16T07:36:34.989497+0000","last_became_active":"2023-05-16T07:36:34.989243+0000","last_became_peered":"2023-05-16T07:36:34.989243+0000","last_unstale":"2023-05-16T07:36:34.989497+0000","last_undegraded":"2023-05-16T07:36:34.989497+0000","last_fullsized":"2023-05-16T07:36:34.989497+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.972889+0000","last_change":"2023-05-16T07:36:34.614729+0000","last_active":"2023-05-16T07:36:34.972889+0000","last_peered":"2023-05-16T07:36:34.972889+0000","last_clean":"2023-05-16T07:36:34.972889+0000","last_became_active":"2023-05-16T07:36:34.614125+0000","last_became_peered":"2023-05-16T07:36:34.614125+0000","last_unstale":"2023-05-16T07:36:34.972889+0000","last_undegraded":"2023-05-16T07:36:34.972889+0000","last_fullsized":"2023-05-16T07:36:34.972889+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991809+0000","last_change":"2023-05-16T07:36:34.991809+0000","last_active":"2023-05-16T07:36:34.991809+0000","last_peered":"2023-05-16T07:36:34.991809+0000","last_clean":"2023-05-16T07:36:34.991809+0000","last_became_active":"2023-05-16T07:36:34.991541+0000","last_became_peered":"2023-05-16T07:36:34.991541+0000","last_unstale":"2023-05-16T07:36:34.991809+0000","last_undegraded":"2023-05-16T07:36:34.991809+0000","last_fullsized":"2023-05-16T07:36:34.991809+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3a","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.978278+0000","last_change":"2023-05-16T07:36:34.620171+0000","last_active":"2023-05-16T07:36:34.978278+0000","last_peered":"2023-05-16T07:36:34.978278+0000","last_clean":"2023-05-16T07:36:34.978278+0000","last_became_active":"2023-05-16T07:36:34.619924+0000","last_became_peered":"2023-05-16T07:36:34.619924+0000","last_unstale":"2023-05-16T07:36:34.978278+0000","last_undegraded":"2023-05-16T07:36:34.978278+0000","last_fullsized":"2023-05-16T07:36:34.978278+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989195+0000","last_change":"2023-05-16T07:36:34.989195+0000","last_active":"2023-05-16T07:36:34.989195+0000","last_peered":"2023-05-16T07:36:34.989195+0000","last_clean":"2023-05-16T07:36:34.989195+0000","last_became_active":"2023-05-16T07:36:34.988803+0000","last_became_peered":"2023-05-16T07:36:34.988803+0000","last_unstale":"2023-05-16T07:36:34.989195+0000","last_undegraded":"2023-05-16T07:36:34.989195+0000","last_fullsized":"2023-05-16T07:36:34.989195+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.613075+0000","last_change":"2023-05-16T07:36:34.611442+0000","last_active":"2023-05-16T07:36:34.613075+0000","last_peered":"2023-05-16T07:36:34.613075+0000","last_clean":"2023-05-16T07:36:34.613075+0000","last_became_active":"2023-05-16T07:36:34.611197+0000","last_became_peered":"2023-05-16T07:36:34.611197+0000","last_unstale":"2023-05-16T07:36:34.613075+0000","last_undegraded":"2023-05-16T07:36:34.613075+0000","last_fullsized":"2023-05-16T07:36:34.613075+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980696+0000","last_change":"2023-05-16T07:36:34.980696+0000","last_active":"2023-05-16T07:36:34.980696+0000","last_peered":"2023-05-16T07:36:34.980696+0000","last_clean":"2023-05-16T07:36:34.980696+0000","last_became_active":"2023-05-16T07:36:34.980429+0000","last_became_peered":"2023-05-16T07:36:34.980429+0000","last_unstale":"2023-05-16T07:36:34.980696+0000","last_undegraded":"2023-05-16T07:36:34.980696+0000","last_fullsized":"2023-05-16T07:36:34.980696+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620815+0000","last_change":"2023-05-16T07:36:34.619509+0000","last_active":"2023-05-16T07:36:34.620815+0000","last_peered":"2023-05-16T07:36:34.620815+0000","last_clean":"2023-05-16T07:36:34.620815+0000","last_became_active":"2023-05-16T07:36:34.619189+0000","last_became_peered":"2023-05-16T07:36:34.619189+0000","last_unstale":"2023-05-16T07:36:34.620815+0000","last_undegraded":"2023-05-16T07:36:34.620815+0000","last_fullsized":"2023-05-16T07:36:34.620815+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987085+0000","last_change":"2023-05-16T07:36:34.987085+0000","last_active":"2023-05-16T07:36:34.987085+0000","last_peered":"2023-05-16T07:36:34.987085+0000","last_clean":"2023-05-16T07:36:34.987085+0000","last_became_active":"2023-05-16T07:36:34.986868+0000","last_became_peered":"2023-05-16T07:36:34.986868+0000","last_unstale":"2023-05-16T07:36:34.987085+0000","last_undegraded":"2023-05-16T07:36:34.987085+0000","last_fullsized":"2023-05-16T07:36:34.987085+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620162+0000","last_change":"2023-05-16T07:36:34.618400+0000","last_active":"2023-05-16T07:36:34.620162+0000","last_peered":"2023-05-16T07:36:34.620162+0000","last_clean":"2023-05-16T07:36:34.620162+0000","last_became_active":"2023-05-16T07:36:34.618104+0000","last_became_peered":"2023-05-16T07:36:34.618104+0000","last_unstale":"2023-05-16T07:36:34.620162+0000","last_undegraded":"2023-05-16T07:36:34.620162+0000","last_fullsized":"2023-05-16T07:36:34.620162+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.975409+0000","last_change":"2023-05-16T07:36:34.975409+0000","last_active":"2023-05-16T07:36:34.975409+0000","last_peered":"2023-05-16T07:36:34.975409+0000","last_clean":"2023-05-16T07:36:34.975409+0000","last_became_active":"2023-05-16T07:36:34.975120+0000","last_became_peered":"2023-05-16T07:36:34.975120+0000","last_unstale":"2023-05-16T07:36:34.975409+0000","last_undegraded":"2023-05-16T07:36:34.975409+0000","last_fullsized":"2023-05-16T07:36:34.975409+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618268+0000","last_change":"2023-05-16T07:36:34.616038+0000","last_active":"2023-05-16T07:36:34.618268+0000","last_peered":"2023-05-16T07:36:34.618268+0000","last_clean":"2023-05-16T07:36:34.618268+0000","last_became_active":"2023-05-16T07:36:34.615407+0000","last_became_peered":"2023-05-16T07:36:34.615407+0000","last_unstale":"2023-05-16T07:36:34.618268+0000","last_undegraded":"2023-05-16T07:36:34.618268+0000","last_fullsized":"2023-05-16T07:36:34.618268+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.977938+0000","last_change":"2023-05-16T07:36:34.977938+0000","last_active":"2023-05-16T07:36:34.977938+0000","last_peered":"2023-05-16T07:36:34.977938+0000","last_clean":"2023-05-16T07:36:34.977938+0000","last_became_active":"2023-05-16T07:36:34.977665+0000","last_became_peered":"2023-05-16T07:36:34.977665+0000","last_unstale":"2023-05-16T07:36:34.977938+0000","last_undegraded":"2023-05-16T07:36:34.977938+0000","last_fullsized":"2023-05-16T07:36:34.977938+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621487+0000","last_change":"2023-05-16T07:36:34.610721+0000","last_active":"2023-05-16T07:36:34.621487+0000","last_peered":"2023-05-16T07:36:34.621487+0000","last_clean":"2023-05-16T07:36:34.621487+0000","last_became_active":"2023-05-16T07:36:34.610302+0000","last_became_peered":"2023-05-16T07:36:34.610302+0000","last_unstale":"2023-05-16T07:36:34.621487+0000","last_undegraded":"2023-05-16T07:36:34.621487+0000","last_fullsized":"2023-05-16T07:36:34.621487+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992706+0000","last_change":"2023-05-16T07:36:34.992706+0000","last_active":"2023-05-16T07:36:34.992706+0000","last_peered":"2023-05-16T07:36:34.992706+0000","last_clean":"2023-05-16T07:36:34.992706+0000","last_became_active":"2023-05-16T07:36:34.992415+0000","last_became_peered":"2023-05-16T07:36:34.992415+0000","last_unstale":"2023-05-16T07:36:34.992706+0000","last_undegraded":"2023-05-16T07:36:34.992706+0000","last_fullsized":"2023-05-16T07:36:34.992706+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987795+0000","last_change":"2023-05-16T07:36:34.618547+0000","last_active":"2023-05-16T07:36:34.987795+0000","last_peered":"2023-05-16T07:36:34.987795+0000","last_clean":"2023-05-16T07:36:34.987795+0000","last_became_active":"2023-05-16T07:36:34.618245+0000","last_became_peered":"2023-05-16T07:36:34.618245+0000","last_unstale":"2023-05-16T07:36:34.987795+0000","last_undegraded":"2023-05-16T07:36:34.987795+0000","last_fullsized":"2023-05-16T07:36:34.987795+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983398+0000","last_change":"2023-05-16T07:36:34.983398+0000","last_active":"2023-05-16T07:36:34.983398+0000","last_peered":"2023-05-16T07:36:34.983398+0000","last_clean":"2023-05-16T07:36:34.983398+0000","last_became_active":"2023-05-16T07:36:34.983131+0000","last_became_peered":"2023-05-16T07:36:34.983131+0000","last_unstale":"2023-05-16T07:36:34.983398+0000","last_undegraded":"2023-05-16T07:36:34.983398+0000","last_fullsized":"2023-05-16T07:36:34.983398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.33","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.971856+0000","last_change":"2023-05-16T07:36:34.616384+0000","last_active":"2023-05-16T07:36:34.971856+0000","last_peered":"2023-05-16T07:36:34.971856+0000","last_clean":"2023-05-16T07:36:34.971856+0000","last_became_active":"2023-05-16T07:36:34.615622+0000","last_became_peered":"2023-05-16T07:36:34.615622+0000","last_unstale":"2023-05-16T07:36:34.971856+0000","last_undegraded":"2023-05-16T07:36:34.971856+0000","last_fullsized":"2023-05-16T07:36:34.971856+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.995085+0000","last_change":"2023-05-16T07:36:34.995085+0000","last_active":"2023-05-16T07:36:34.995085+0000","last_peered":"2023-05-16T07:36:34.995085+0000","last_clean":"2023-05-16T07:36:34.995085+0000","last_became_active":"2023-05-16T07:36:34.994814+0000","last_became_peered":"2023-05-16T07:36:34.994814+0000","last_unstale":"2023-05-16T07:36:34.995085+0000","last_undegraded":"2023-05-16T07:36:34.995085+0000","last_fullsized":"2023-05-16T07:36:34.995085+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621982+0000","last_change":"2023-05-16T07:36:34.620363+0000","last_active":"2023-05-16T07:36:34.621982+0000","last_peered":"2023-05-16T07:36:34.621982+0000","last_clean":"2023-05-16T07:36:34.621982+0000","last_became_active":"2023-05-16T07:36:34.620066+0000","last_became_peered":"2023-05-16T07:36:34.620066+0000","last_unstale":"2023-05-16T07:36:34.621982+0000","last_undegraded":"2023-05-16T07:36:34.621982+0000","last_fullsized":"2023-05-16T07:36:34.621982+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.988596+0000","last_change":"2023-05-16T07:36:34.988596+0000","last_active":"2023-05-16T07:36:34.988596+0000","last_peered":"2023-05-16T07:36:34.988596+0000","last_clean":"2023-05-16T07:36:34.988596+0000","last_became_active":"2023-05-16T07:36:34.988294+0000","last_became_peered":"2023-05-16T07:36:34.988294+0000","last_unstale":"2023-05-16T07:36:34.988596+0000","last_undegraded":"2023-05-16T07:36:34.988596+0000","last_fullsized":"2023-05-16T07:36:34.988596+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.623044+0000","last_change":"2023-05-16T07:36:34.617933+0000","last_active":"2023-05-16T07:36:34.623044+0000","last_peered":"2023-05-16T07:36:34.623044+0000","last_clean":"2023-05-16T07:36:34.623044+0000","last_became_active":"2023-05-16T07:36:34.617651+0000","last_became_peered":"2023-05-16T07:36:34.617651+0000","last_unstale":"2023-05-16T07:36:34.623044+0000","last_undegraded":"2023-05-16T07:36:34.623044+0000","last_fullsized":"2023-05-16T07:36:34.623044+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990611+0000","last_change":"2023-05-16T07:36:34.990611+0000","last_active":"2023-05-16T07:36:34.990611+0000","last_peered":"2023-05-16T07:36:34.990611+0000","last_clean":"2023-05-16T07:36:34.990611+0000","last_became_active":"2023-05-16T07:36:34.990360+0000","last_became_peered":"2023-05-16T07:36:34.990360+0000","last_unstale":"2023-05-16T07:36:34.990611+0000","last_undegraded":"2023-05-16T07:36:34.990611+0000","last_fullsized":"2023-05-16T07:36:34.990611+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985474+0000","last_change":"2023-05-16T07:36:34.614358+0000","last_active":"2023-05-16T07:36:34.985474+0000","last_peered":"2023-05-16T07:36:34.985474+0000","last_clean":"2023-05-16T07:36:34.985474+0000","last_became_active":"2023-05-16T07:36:34.614082+0000","last_became_peered":"2023-05-16T07:36:34.614082+0000","last_unstale":"2023-05-16T07:36:34.985474+0000","last_undegraded":"2023-05-16T07:36:34.985474+0000","last_fullsized":"2023-05-16T07:36:34.985474+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989366+0000","last_change":"2023-05-16T07:36:34.989366+0000","last_active":"2023-05-16T07:36:34.989366+0000","last_peered":"2023-05-16T07:36:34.989366+0000","last_clean":"2023-05-16T07:36:34.989366+0000","last_became_active":"2023-05-16T07:36:34.989162+0000","last_became_peered":"2023-05-16T07:36:34.989162+0000","last_unstale":"2023-05-16T07:36:34.989366+0000","last_undegraded":"2023-05-16T07:36:34.989366+0000","last_fullsized":"2023-05-16T07:36:34.989366+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.612991+0000","last_change":"2023-05-16T07:36:34.610890+0000","last_active":"2023-05-16T07:36:34.612991+0000","last_peered":"2023-05-16T07:36:34.612991+0000","last_clean":"2023-05-16T07:36:34.612991+0000","last_became_active":"2023-05-16T07:36:34.610638+0000","last_became_peered":"2023-05-16T07:36:34.610638+0000","last_unstale":"2023-05-16T07:36:34.612991+0000","last_undegraded":"2023-05-16T07:36:34.612991+0000","last_fullsized":"2023-05-16T07:36:34.612991+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990447+0000","last_change":"2023-05-16T07:36:34.990447+0000","last_active":"2023-05-16T07:36:34.990447+0000","last_peered":"2023-05-16T07:36:34.990447+0000","last_clean":"2023-05-16T07:36:34.990447+0000","last_became_active":"2023-05-16T07:36:34.989836+0000","last_became_peered":"2023-05-16T07:36:34.989836+0000","last_unstale":"2023-05-16T07:36:34.990447+0000","last_undegraded":"2023-05-16T07:36:34.990447+0000","last_fullsized":"2023-05-16T07:36:34.990447+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.617965+0000","last_change":"2023-05-16T07:36:34.612974+0000","last_active":"2023-05-16T07:36:34.617965+0000","last_peered":"2023-05-16T07:36:34.617965+0000","last_clean":"2023-05-16T07:36:34.617965+0000","last_became_active":"2023-05-16T07:36:34.612753+0000","last_became_peered":"2023-05-16T07:36:34.612753+0000","last_unstale":"2023-05-16T07:36:34.617965+0000","last_undegraded":"2023-05-16T07:36:34.617965+0000","last_fullsized":"2023-05-16T07:36:34.617965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990515+0000","last_change":"2023-05-16T07:36:34.990515+0000","last_active":"2023-05-16T07:36:34.990515+0000","last_peered":"2023-05-16T07:36:34.990515+0000","last_clean":"2023-05-16T07:36:34.990515+0000","last_became_active":"2023-05-16T07:36:34.990072+0000","last_became_peered":"2023-05-16T07:36:34.990072+0000","last_unstale":"2023-05-16T07:36:34.990515+0000","last_undegraded":"2023-05-16T07:36:34.990515+0000","last_fullsized":"2023-05-16T07:36:34.990515+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2d","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.973386+0000","last_change":"2023-05-16T07:36:34.612576+0000","last_active":"2023-05-16T07:36:34.973386+0000","last_peered":"2023-05-16T07:36:34.973386+0000","last_clean":"2023-05-16T07:36:34.973386+0000","last_became_active":"2023-05-16T07:36:34.612300+0000","last_became_peered":"2023-05-16T07:36:34.612300+0000","last_unstale":"2023-05-16T07:36:34.973386+0000","last_undegraded":"2023-05-16T07:36:34.973386+0000","last_fullsized":"2023-05-16T07:36:34.973386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989016+0000","last_change":"2023-05-16T07:36:34.989016+0000","last_active":"2023-05-16T07:36:34.989016+0000","last_peered":"2023-05-16T07:36:34.989016+0000","last_clean":"2023-05-16T07:36:34.989016+0000","last_became_active":"2023-05-16T07:36:34.988714+0000","last_became_peered":"2023-05-16T07:36:34.988714+0000","last_unstale":"2023-05-16T07:36:34.989016+0000","last_undegraded":"2023-05-16T07:36:34.989016+0000","last_fullsized":"2023-05-16T07:36:34.989016+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619259+0000","last_change":"2023-05-16T07:36:34.616368+0000","last_active":"2023-05-16T07:36:34.619259+0000","last_peered":"2023-05-16T07:36:34.619259+0000","last_clean":"2023-05-16T07:36:34.619259+0000","last_became_active":"2023-05-16T07:36:34.616019+0000","last_became_peered":"2023-05-16T07:36:34.616019+0000","last_unstale":"2023-05-16T07:36:34.619259+0000","last_undegraded":"2023-05-16T07:36:34.619259+0000","last_fullsized":"2023-05-16T07:36:34.619259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.973269+0000","last_change":"2023-05-16T07:36:34.973269+0000","last_active":"2023-05-16T07:36:34.973269+0000","last_peered":"2023-05-16T07:36:34.973269+0000","last_clean":"2023-05-16T07:36:34.973269+0000","last_became_active":"2023-05-16T07:36:34.973021+0000","last_became_peered":"2023-05-16T07:36:34.973021+0000","last_unstale":"2023-05-16T07:36:34.973269+0000","last_undegraded":"2023-05-16T07:36:34.973269+0000","last_fullsized":"2023-05-16T07:36:34.973269+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.973002+0000","last_change":"2023-05-16T07:36:34.614844+0000","last_active":"2023-05-16T07:36:34.973002+0000","last_peered":"2023-05-16T07:36:34.973002+0000","last_clean":"2023-05-16T07:36:34.973002+0000","last_became_active":"2023-05-16T07:36:34.614475+0000","last_became_peered":"2023-05-16T07:36:34.614475+0000","last_unstale":"2023-05-16T07:36:34.973002+0000","last_undegraded":"2023-05-16T07:36:34.973002+0000","last_fullsized":"2023-05-16T07:36:34.973002+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986742+0000","last_change":"2023-05-16T07:36:34.986742+0000","last_active":"2023-05-16T07:36:34.986742+0000","last_peered":"2023-05-16T07:36:34.986742+0000","last_clean":"2023-05-16T07:36:34.986742+0000","last_became_active":"2023-05-16T07:36:34.986461+0000","last_became_peered":"2023-05-16T07:36:34.986461+0000","last_unstale":"2023-05-16T07:36:34.986742+0000","last_undegraded":"2023-05-16T07:36:34.986742+0000","last_fullsized":"2023-05-16T07:36:34.986742+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621496+0000","last_change":"2023-05-16T07:36:34.610983+0000","last_active":"2023-05-16T07:36:34.621496+0000","last_peered":"2023-05-16T07:36:34.621496+0000","last_clean":"2023-05-16T07:36:34.621496+0000","last_became_active":"2023-05-16T07:36:34.610687+0000","last_became_peered":"2023-05-16T07:36:34.610687+0000","last_unstale":"2023-05-16T07:36:34.621496+0000","last_undegraded":"2023-05-16T07:36:34.621496+0000","last_fullsized":"2023-05-16T07:36:34.621496+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987031+0000","last_change":"2023-05-16T07:36:34.987031+0000","last_active":"2023-05-16T07:36:34.987031+0000","last_peered":"2023-05-16T07:36:34.987031+0000","last_clean":"2023-05-16T07:36:34.987031+0000","last_became_active":"2023-05-16T07:36:34.986738+0000","last_became_peered":"2023-05-16T07:36:34.986738+0000","last_unstale":"2023-05-16T07:36:34.987031+0000","last_undegraded":"2023-05-16T07:36:34.987031+0000","last_fullsized":"2023-05-16T07:36:34.987031+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620472+0000","last_change":"2023-05-16T07:36:34.617921+0000","last_active":"2023-05-16T07:36:34.620472+0000","last_peered":"2023-05-16T07:36:34.620472+0000","last_clean":"2023-05-16T07:36:34.620472+0000","last_became_active":"2023-05-16T07:36:34.617416+0000","last_became_peered":"2023-05-16T07:36:34.617416+0000","last_unstale":"2023-05-16T07:36:34.620472+0000","last_undegraded":"2023-05-16T07:36:34.620472+0000","last_fullsized":"2023-05-16T07:36:34.620472+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983361+0000","last_change":"2023-05-16T07:36:34.983361+0000","last_active":"2023-05-16T07:36:34.983361+0000","last_peered":"2023-05-16T07:36:34.983361+0000","last_clean":"2023-05-16T07:36:34.983361+0000","last_became_active":"2023-05-16T07:36:34.983083+0000","last_became_peered":"2023-05-16T07:36:34.983083+0000","last_unstale":"2023-05-16T07:36:34.983361+0000","last_undegraded":"2023-05-16T07:36:34.983361+0000","last_fullsized":"2023-05-16T07:36:34.983361+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621763+0000","last_change":"2023-05-16T07:36:34.613032+0000","last_active":"2023-05-16T07:36:34.621763+0000","last_peered":"2023-05-16T07:36:34.621763+0000","last_clean":"2023-05-16T07:36:34.621763+0000","last_became_active":"2023-05-16T07:36:34.612699+0000","last_became_peered":"2023-05-16T07:36:34.612699+0000","last_unstale":"2023-05-16T07:36:34.621763+0000","last_undegraded":"2023-05-16T07:36:34.621763+0000","last_fullsized":"2023-05-16T07:36:34.621763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982784+0000","last_change":"2023-05-16T07:36:34.982784+0000","last_active":"2023-05-16T07:36:34.982784+0000","last_peered":"2023-05-16T07:36:34.982784+0000","last_clean":"2023-05-16T07:36:34.982784+0000","last_became_active":"2023-05-16T07:36:34.982539+0000","last_became_peered":"2023-05-16T07:36:34.982539+0000","last_unstale":"2023-05-16T07:36:34.982784+0000","last_undegraded":"2023-05-16T07:36:34.982784+0000","last_fullsized":"2023-05-16T07:36:34.982784+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619779+0000","last_change":"2023-05-16T07:36:34.616330+0000","last_active":"2023-05-16T07:36:34.619779+0000","last_peered":"2023-05-16T07:36:34.619779+0000","last_clean":"2023-05-16T07:36:34.619779+0000","last_became_active":"2023-05-16T07:36:34.615577+0000","last_became_peered":"2023-05-16T07:36:34.615577+0000","last_unstale":"2023-05-16T07:36:34.619779+0000","last_undegraded":"2023-05-16T07:36:34.619779+0000","last_fullsized":"2023-05-16T07:36:34.619779+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984407+0000","last_change":"2023-05-16T07:36:34.984407+0000","last_active":"2023-05-16T07:36:34.984407+0000","last_peered":"2023-05-16T07:36:34.984407+0000","last_clean":"2023-05-16T07:36:34.984407+0000","last_became_active":"2023-05-16T07:36:34.984097+0000","last_became_peered":"2023-05-16T07:36:34.984097+0000","last_unstale":"2023-05-16T07:36:34.984407+0000","last_undegraded":"2023-05-16T07:36:34.984407+0000","last_fullsized":"2023-05-16T07:36:34.984407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619192+0000","last_change":"2023-05-16T07:36:34.614889+0000","last_active":"2023-05-16T07:36:34.619192+0000","last_peered":"2023-05-16T07:36:34.619192+0000","last_clean":"2023-05-16T07:36:34.619192+0000","last_became_active":"2023-05-16T07:36:34.614257+0000","last_became_peered":"2023-05-16T07:36:34.614257+0000","last_unstale":"2023-05-16T07:36:34.619192+0000","last_undegraded":"2023-05-16T07:36:34.619192+0000","last_fullsized":"2023-05-16T07:36:34.619192+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.988839+0000","last_change":"2023-05-16T07:36:34.988839+0000","last_active":"2023-05-16T07:36:34.988839+0000","last_peered":"2023-05-16T07:36:34.988839+0000","last_clean":"2023-05-16T07:36:34.988839+0000","last_became_active":"2023-05-16T07:36:34.985450+0000","last_became_peered":"2023-05-16T07:36:34.985450+0000","last_unstale":"2023-05-16T07:36:34.988839+0000","last_undegraded":"2023-05-16T07:36:34.988839+0000","last_fullsized":"2023-05-16T07:36:34.988839+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622876+0000","last_change":"2023-05-16T07:36:34.615159+0000","last_active":"2023-05-16T07:36:34.622876+0000","last_peered":"2023-05-16T07:36:34.622876+0000","last_clean":"2023-05-16T07:36:34.622876+0000","last_became_active":"2023-05-16T07:36:34.614875+0000","last_became_peered":"2023-05-16T07:36:34.614875+0000","last_unstale":"2023-05-16T07:36:34.622876+0000","last_undegraded":"2023-05-16T07:36:34.622876+0000","last_fullsized":"2023-05-16T07:36:34.622876+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982480+0000","last_change":"2023-05-16T07:36:34.982480+0000","last_active":"2023-05-16T07:36:34.982480+0000","last_peered":"2023-05-16T07:36:34.982480+0000","last_clean":"2023-05-16T07:36:34.982480+0000","last_became_active":"2023-05-16T07:36:34.982180+0000","last_became_peered":"2023-05-16T07:36:34.982180+0000","last_unstale":"2023-05-16T07:36:34.982480+0000","last_undegraded":"2023-05-16T07:36:34.982480+0000","last_fullsized":"2023-05-16T07:36:34.982480+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619097+0000","last_change":"2023-05-16T07:36:34.614696+0000","last_active":"2023-05-16T07:36:34.619097+0000","last_peered":"2023-05-16T07:36:34.619097+0000","last_clean":"2023-05-16T07:36:34.619097+0000","last_became_active":"2023-05-16T07:36:34.614181+0000","last_became_peered":"2023-05-16T07:36:34.614181+0000","last_unstale":"2023-05-16T07:36:34.619097+0000","last_undegraded":"2023-05-16T07:36:34.619097+0000","last_fullsized":"2023-05-16T07:36:34.619097+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986237+0000","last_change":"2023-05-16T07:36:34.986237+0000","last_active":"2023-05-16T07:36:34.986237+0000","last_peered":"2023-05-16T07:36:34.986237+0000","last_clean":"2023-05-16T07:36:34.986237+0000","last_became_active":"2023-05-16T07:36:34.986004+0000","last_became_peered":"2023-05-16T07:36:34.986004+0000","last_unstale":"2023-05-16T07:36:34.986237+0000","last_undegraded":"2023-05-16T07:36:34.986237+0000","last_fullsized":"2023-05-16T07:36:34.986237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619421+0000","last_change":"2023-05-16T07:36:34.609318+0000","last_active":"2023-05-16T07:36:34.619421+0000","last_peered":"2023-05-16T07:36:34.619421+0000","last_clean":"2023-05-16T07:36:34.619421+0000","last_became_active":"2023-05-16T07:36:34.609025+0000","last_became_peered":"2023-05-16T07:36:34.609025+0000","last_unstale":"2023-05-16T07:36:34.619421+0000","last_undegraded":"2023-05-16T07:36:34.619421+0000","last_fullsized":"2023-05-16T07:36:34.619421+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984987+0000","last_change":"2023-05-16T07:36:34.984987+0000","last_active":"2023-05-16T07:36:34.984987+0000","last_peered":"2023-05-16T07:36:34.984987+0000","last_clean":"2023-05-16T07:36:34.984987+0000","last_became_active":"2023-05-16T07:36:34.984698+0000","last_became_peered":"2023-05-16T07:36:34.984698+0000","last_unstale":"2023-05-16T07:36:34.984987+0000","last_undegraded":"2023-05-16T07:36:34.984987+0000","last_fullsized":"2023-05-16T07:36:34.984987+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621248+0000","last_change":"2023-05-16T07:36:34.618135+0000","last_active":"2023-05-16T07:36:34.621248+0000","last_peered":"2023-05-16T07:36:34.621248+0000","last_clean":"2023-05-16T07:36:34.621248+0000","last_became_active":"2023-05-16T07:36:34.617856+0000","last_became_peered":"2023-05-16T07:36:34.617856+0000","last_unstale":"2023-05-16T07:36:34.621248+0000","last_undegraded":"2023-05-16T07:36:34.621248+0000","last_fullsized":"2023-05-16T07:36:34.621248+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982142+0000","last_change":"2023-05-16T07:36:34.982142+0000","last_active":"2023-05-16T07:36:34.982142+0000","last_peered":"2023-05-16T07:36:34.982142+0000","last_clean":"2023-05-16T07:36:34.982142+0000","last_became_active":"2023-05-16T07:36:34.981490+0000","last_became_peered":"2023-05-16T07:36:34.981490+0000","last_unstale":"2023-05-16T07:36:34.982142+0000","last_undegraded":"2023-05-16T07:36:34.982142+0000","last_fullsized":"2023-05-16T07:36:34.982142+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622410+0000","last_change":"2023-05-16T07:36:34.621445+0000","last_active":"2023-05-16T07:36:34.622410+0000","last_peered":"2023-05-16T07:36:34.622410+0000","last_clean":"2023-05-16T07:36:34.622410+0000","last_became_active":"2023-05-16T07:36:34.620796+0000","last_became_peered":"2023-05-16T07:36:34.620796+0000","last_unstale":"2023-05-16T07:36:34.622410+0000","last_undegraded":"2023-05-16T07:36:34.622410+0000","last_fullsized":"2023-05-16T07:36:34.622410+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989141+0000","last_change":"2023-05-16T07:36:34.989141+0000","last_active":"2023-05-16T07:36:34.989141+0000","last_peered":"2023-05-16T07:36:34.989141+0000","last_clean":"2023-05-16T07:36:34.989141+0000","last_became_active":"2023-05-16T07:36:34.985473+0000","last_became_peered":"2023-05-16T07:36:34.985473+0000","last_unstale":"2023-05-16T07:36:34.989141+0000","last_undegraded":"2023-05-16T07:36:34.989141+0000","last_fullsized":"2023-05-16T07:36:34.989141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618631+0000","last_change":"2023-05-16T07:36:34.615595+0000","last_active":"2023-05-16T07:36:34.618631+0000","last_peered":"2023-05-16T07:36:34.618631+0000","last_clean":"2023-05-16T07:36:34.618631+0000","last_became_active":"2023-05-16T07:36:34.614397+0000","last_became_peered":"2023-05-16T07:36:34.614397+0000","last_unstale":"2023-05-16T07:36:34.618631+0000","last_undegraded":"2023-05-16T07:36:34.618631+0000","last_fullsized":"2023-05-16T07:36:34.618631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.988261+0000","last_change":"2023-05-16T07:36:34.988261+0000","last_active":"2023-05-16T07:36:34.988261+0000","last_peered":"2023-05-16T07:36:34.988261+0000","last_clean":"2023-05-16T07:36:34.988261+0000","last_became_active":"2023-05-16T07:36:34.987993+0000","last_became_peered":"2023-05-16T07:36:34.987993+0000","last_unstale":"2023-05-16T07:36:34.988261+0000","last_undegraded":"2023-05-16T07:36:34.988261+0000","last_fullsized":"2023-05-16T07:36:34.988261+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.970267+0000","last_change":"2023-05-16T07:36:34.619459+0000","last_active":"2023-05-16T07:36:34.970267+0000","last_peered":"2023-05-16T07:36:34.970267+0000","last_clean":"2023-05-16T07:36:34.970267+0000","last_became_active":"2023-05-16T07:36:34.619163+0000","last_became_peered":"2023-05-16T07:36:34.619163+0000","last_unstale":"2023-05-16T07:36:34.970267+0000","last_undegraded":"2023-05-16T07:36:34.970267+0000","last_fullsized":"2023-05-16T07:36:34.970267+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990373+0000","last_change":"2023-05-16T07:36:34.990373+0000","last_active":"2023-05-16T07:36:34.990373+0000","last_peered":"2023-05-16T07:36:34.990373+0000","last_clean":"2023-05-16T07:36:34.990373+0000","last_became_active":"2023-05-16T07:36:34.990098+0000","last_became_peered":"2023-05-16T07:36:34.990098+0000","last_unstale":"2023-05-16T07:36:34.990373+0000","last_undegraded":"2023-05-16T07:36:34.990373+0000","last_fullsized":"2023-05-16T07:36:34.990373+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.967298+0000","last_change":"2023-05-16T07:36:34.615087+0000","last_active":"2023-05-16T07:36:34.967298+0000","last_peered":"2023-05-16T07:36:34.967298+0000","last_clean":"2023-05-16T07:36:34.967298+0000","last_became_active":"2023-05-16T07:36:34.614627+0000","last_became_peered":"2023-05-16T07:36:34.614627+0000","last_unstale":"2023-05-16T07:36:34.967298+0000","last_undegraded":"2023-05-16T07:36:34.967298+0000","last_fullsized":"2023-05-16T07:36:34.967298+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989528+0000","last_change":"2023-05-16T07:36:34.989528+0000","last_active":"2023-05-16T07:36:34.989528+0000","last_peered":"2023-05-16T07:36:34.989528+0000","last_clean":"2023-05-16T07:36:34.989528+0000","last_became_active":"2023-05-16T07:36:34.989013+0000","last_became_peered":"2023-05-16T07:36:34.989013+0000","last_unstale":"2023-05-16T07:36:34.989528+0000","last_undegraded":"2023-05-16T07:36:34.989528+0000","last_fullsized":"2023-05-16T07:36:34.989528+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986844+0000","last_change":"2023-05-16T07:36:34.617560+0000","last_active":"2023-05-16T07:36:34.986844+0000","last_peered":"2023-05-16T07:36:34.986844+0000","last_clean":"2023-05-16T07:36:34.986844+0000","last_became_active":"2023-05-16T07:36:34.617153+0000","last_became_peered":"2023-05-16T07:36:34.617153+0000","last_unstale":"2023-05-16T07:36:34.986844+0000","last_undegraded":"2023-05-16T07:36:34.986844+0000","last_fullsized":"2023-05-16T07:36:34.986844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.977731+0000","last_change":"2023-05-16T07:36:34.977731+0000","last_active":"2023-05-16T07:36:34.977731+0000","last_peered":"2023-05-16T07:36:34.977731+0000","last_clean":"2023-05-16T07:36:34.977731+0000","last_became_active":"2023-05-16T07:36:34.977462+0000","last_became_peered":"2023-05-16T07:36:34.977462+0000","last_unstale":"2023-05-16T07:36:34.977731+0000","last_undegraded":"2023-05-16T07:36:34.977731+0000","last_fullsized":"2023-05-16T07:36:34.977731+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.616562+0000","last_change":"2023-05-16T07:36:34.613312+0000","last_active":"2023-05-16T07:36:34.616562+0000","last_peered":"2023-05-16T07:36:34.616562+0000","last_clean":"2023-05-16T07:36:34.616562+0000","last_became_active":"2023-05-16T07:36:34.613008+0000","last_became_peered":"2023-05-16T07:36:34.613008+0000","last_unstale":"2023-05-16T07:36:34.616562+0000","last_undegraded":"2023-05-16T07:36:34.616562+0000","last_fullsized":"2023-05-16T07:36:34.616562+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984956+0000","last_change":"2023-05-16T07:36:34.984956+0000","last_active":"2023-05-16T07:36:34.984956+0000","last_peered":"2023-05-16T07:36:34.984956+0000","last_clean":"2023-05-16T07:36:34.984956+0000","last_became_active":"2023-05-16T07:36:34.984620+0000","last_became_peered":"2023-05-16T07:36:34.984620+0000","last_unstale":"2023-05-16T07:36:34.984956+0000","last_undegraded":"2023-05-16T07:36:34.984956+0000","last_fullsized":"2023-05-16T07:36:34.984956+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.613864+0000","last_change":"2023-05-16T07:36:34.610195+0000","last_active":"2023-05-16T07:36:34.613864+0000","last_peered":"2023-05-16T07:36:34.613864+0000","last_clean":"2023-05-16T07:36:34.613864+0000","last_became_active":"2023-05-16T07:36:34.609930+0000","last_became_peered":"2023-05-16T07:36:34.609930+0000","last_unstale":"2023-05-16T07:36:34.613864+0000","last_undegraded":"2023-05-16T07:36:34.613864+0000","last_fullsized":"2023-05-16T07:36:34.613864+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.989470+0000","last_change":"2023-05-16T07:36:34.989470+0000","last_active":"2023-05-16T07:36:34.989470+0000","last_peered":"2023-05-16T07:36:34.989470+0000","last_clean":"2023-05-16T07:36:34.989470+0000","last_became_active":"2023-05-16T07:36:34.989249+0000","last_became_peered":"2023-05-16T07:36:34.989249+0000","last_unstale":"2023-05-16T07:36:34.989470+0000","last_undegraded":"2023-05-16T07:36:34.989470+0000","last_fullsized":"2023-05-16T07:36:34.989470+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.617845+0000","last_change":"2023-05-16T07:36:34.612461+0000","last_active":"2023-05-16T07:36:34.617845+0000","last_peered":"2023-05-16T07:36:34.617845+0000","last_clean":"2023-05-16T07:36:34.617845+0000","last_became_active":"2023-05-16T07:36:34.612206+0000","last_became_peered":"2023-05-16T07:36:34.612206+0000","last_unstale":"2023-05-16T07:36:34.617845+0000","last_undegraded":"2023-05-16T07:36:34.617845+0000","last_fullsized":"2023-05-16T07:36:34.617845+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.975366+0000","last_change":"2023-05-16T07:36:34.975366+0000","last_active":"2023-05-16T07:36:34.975366+0000","last_peered":"2023-05-16T07:36:34.975366+0000","last_clean":"2023-05-16T07:36:34.975366+0000","last_became_active":"2023-05-16T07:36:34.975111+0000","last_became_peered":"2023-05-16T07:36:34.975111+0000","last_unstale":"2023-05-16T07:36:34.975366+0000","last_undegraded":"2023-05-16T07:36:34.975366+0000","last_fullsized":"2023-05-16T07:36:34.975366+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620639+0000","last_change":"2023-05-16T07:36:34.618794+0000","last_active":"2023-05-16T07:36:34.620639+0000","last_peered":"2023-05-16T07:36:34.620639+0000","last_clean":"2023-05-16T07:36:34.620639+0000","last_became_active":"2023-05-16T07:36:34.618524+0000","last_became_peered":"2023-05-16T07:36:34.618524+0000","last_unstale":"2023-05-16T07:36:34.620639+0000","last_undegraded":"2023-05-16T07:36:34.620639+0000","last_fullsized":"2023-05-16T07:36:34.620639+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985348+0000","last_change":"2023-05-16T07:36:34.985348+0000","last_active":"2023-05-16T07:36:34.985348+0000","last_peered":"2023-05-16T07:36:34.985348+0000","last_clean":"2023-05-16T07:36:34.985348+0000","last_became_active":"2023-05-16T07:36:34.985035+0000","last_became_peered":"2023-05-16T07:36:34.985035+0000","last_unstale":"2023-05-16T07:36:34.985348+0000","last_undegraded":"2023-05-16T07:36:34.985348+0000","last_fullsized":"2023-05-16T07:36:34.985348+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619981+0000","last_change":"2023-05-16T07:36:34.614893+0000","last_active":"2023-05-16T07:36:34.619981+0000","last_peered":"2023-05-16T07:36:34.619981+0000","last_clean":"2023-05-16T07:36:34.619981+0000","last_became_active":"2023-05-16T07:36:34.614627+0000","last_became_peered":"2023-05-16T07:36:34.614627+0000","last_unstale":"2023-05-16T07:36:34.619981+0000","last_undegraded":"2023-05-16T07:36:34.619981+0000","last_fullsized":"2023-05-16T07:36:34.619981+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984697+0000","last_change":"2023-05-16T07:36:34.984697+0000","last_active":"2023-05-16T07:36:34.984697+0000","last_peered":"2023-05-16T07:36:34.984697+0000","last_clean":"2023-05-16T07:36:34.984697+0000","last_became_active":"2023-05-16T07:36:34.984426+0000","last_became_peered":"2023-05-16T07:36:34.984426+0000","last_unstale":"2023-05-16T07:36:34.984697+0000","last_undegraded":"2023-05-16T07:36:34.984697+0000","last_fullsized":"2023-05-16T07:36:34.984697+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.979653+0000","last_change":"2023-05-16T07:36:34.614052+0000","last_active":"2023-05-16T07:36:34.979653+0000","last_peered":"2023-05-16T07:36:34.979653+0000","last_clean":"2023-05-16T07:36:34.979653+0000","last_became_active":"2023-05-16T07:36:34.613764+0000","last_became_peered":"2023-05-16T07:36:34.613764+0000","last_unstale":"2023-05-16T07:36:34.979653+0000","last_undegraded":"2023-05-16T07:36:34.979653+0000","last_fullsized":"2023-05-16T07:36:34.979653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987880+0000","last_change":"2023-05-16T07:36:34.987880+0000","last_active":"2023-05-16T07:36:34.987880+0000","last_peered":"2023-05-16T07:36:34.987880+0000","last_clean":"2023-05-16T07:36:34.987880+0000","last_became_active":"2023-05-16T07:36:34.987601+0000","last_became_peered":"2023-05-16T07:36:34.987601+0000","last_unstale":"2023-05-16T07:36:34.987880+0000","last_undegraded":"2023-05-16T07:36:34.987880+0000","last_fullsized":"2023-05-16T07:36:34.987880+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982386+0000","last_change":"2023-05-16T07:36:34.619298+0000","last_active":"2023-05-16T07:36:34.982386+0000","last_peered":"2023-05-16T07:36:34.982386+0000","last_clean":"2023-05-16T07:36:34.982386+0000","last_became_active":"2023-05-16T07:36:34.619023+0000","last_became_peered":"2023-05-16T07:36:34.619023+0000","last_unstale":"2023-05-16T07:36:34.982386+0000","last_undegraded":"2023-05-16T07:36:34.982386+0000","last_fullsized":"2023-05-16T07:36:34.982386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985195+0000","last_change":"2023-05-16T07:36:34.985195+0000","last_active":"2023-05-16T07:36:34.985195+0000","last_peered":"2023-05-16T07:36:34.985195+0000","last_clean":"2023-05-16T07:36:34.985195+0000","last_became_active":"2023-05-16T07:36:34.984345+0000","last_became_peered":"2023-05-16T07:36:34.984345+0000","last_unstale":"2023-05-16T07:36:34.985195+0000","last_undegraded":"2023-05-16T07:36:34.985195+0000","last_fullsized":"2023-05-16T07:36:34.985195+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620666+0000","last_change":"2023-05-16T07:36:34.619313+0000","last_active":"2023-05-16T07:36:34.620666+0000","last_peered":"2023-05-16T07:36:34.620666+0000","last_clean":"2023-05-16T07:36:34.620666+0000","last_became_active":"2023-05-16T07:36:34.619061+0000","last_became_peered":"2023-05-16T07:36:34.619061+0000","last_unstale":"2023-05-16T07:36:34.620666+0000","last_undegraded":"2023-05-16T07:36:34.620666+0000","last_fullsized":"2023-05-16T07:36:34.620666+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992971+0000","last_change":"2023-05-16T07:36:34.992971+0000","last_active":"2023-05-16T07:36:34.992971+0000","last_peered":"2023-05-16T07:36:34.992971+0000","last_clean":"2023-05-16T07:36:34.992971+0000","last_became_active":"2023-05-16T07:36:34.992501+0000","last_became_peered":"2023-05-16T07:36:34.992501+0000","last_unstale":"2023-05-16T07:36:34.992971+0000","last_undegraded":"2023-05-16T07:36:34.992971+0000","last_fullsized":"2023-05-16T07:36:34.992971+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.619810+0000","last_change":"2023-05-16T07:36:34.617109+0000","last_active":"2023-05-16T07:36:34.619810+0000","last_peered":"2023-05-16T07:36:34.619810+0000","last_clean":"2023-05-16T07:36:34.619810+0000","last_became_active":"2023-05-16T07:36:34.616829+0000","last_became_peered":"2023-05-16T07:36:34.616829+0000","last_unstale":"2023-05-16T07:36:34.619810+0000","last_undegraded":"2023-05-16T07:36:34.619810+0000","last_fullsized":"2023-05-16T07:36:34.619810+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.978670+0000","last_change":"2023-05-16T07:36:34.978670+0000","last_active":"2023-05-16T07:36:34.978670+0000","last_peered":"2023-05-16T07:36:34.978670+0000","last_clean":"2023-05-16T07:36:34.978670+0000","last_became_active":"2023-05-16T07:36:34.978400+0000","last_became_peered":"2023-05-16T07:36:34.978400+0000","last_unstale":"2023-05-16T07:36:34.978670+0000","last_undegraded":"2023-05-16T07:36:34.978670+0000","last_fullsized":"2023-05-16T07:36:34.978670+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.978856+0000","last_change":"2023-05-16T07:36:34.612723+0000","last_active":"2023-05-16T07:36:34.978856+0000","last_peered":"2023-05-16T07:36:34.978856+0000","last_clean":"2023-05-16T07:36:34.978856+0000","last_became_active":"2023-05-16T07:36:34.612325+0000","last_became_peered":"2023-05-16T07:36:34.612325+0000","last_unstale":"2023-05-16T07:36:34.978856+0000","last_undegraded":"2023-05-16T07:36:34.978856+0000","last_fullsized":"2023-05-16T07:36:34.978856+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.985599+0000","last_change":"2023-05-16T07:36:34.985599+0000","last_active":"2023-05-16T07:36:34.985599+0000","last_peered":"2023-05-16T07:36:34.985599+0000","last_clean":"2023-05-16T07:36:34.985599+0000","last_became_active":"2023-05-16T07:36:34.985333+0000","last_became_peered":"2023-05-16T07:36:34.985333+0000","last_unstale":"2023-05-16T07:36:34.985599+0000","last_undegraded":"2023-05-16T07:36:34.985599+0000","last_fullsized":"2023-05-16T07:36:34.985599+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620322+0000","last_change":"2023-05-16T07:36:34.614480+0000","last_active":"2023-05-16T07:36:34.620322+0000","last_peered":"2023-05-16T07:36:34.620322+0000","last_clean":"2023-05-16T07:36:34.620322+0000","last_became_active":"2023-05-16T07:36:34.614189+0000","last_became_peered":"2023-05-16T07:36:34.614189+0000","last_unstale":"2023-05-16T07:36:34.620322+0000","last_undegraded":"2023-05-16T07:36:34.620322+0000","last_fullsized":"2023-05-16T07:36:34.620322+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.984760+0000","last_change":"2023-05-16T07:36:34.984760+0000","last_active":"2023-05-16T07:36:34.984760+0000","last_peered":"2023-05-16T07:36:34.984760+0000","last_clean":"2023-05-16T07:36:34.984760+0000","last_became_active":"2023-05-16T07:36:34.984499+0000","last_became_peered":"2023-05-16T07:36:34.984499+0000","last_unstale":"2023-05-16T07:36:34.984760+0000","last_undegraded":"2023-05-16T07:36:34.984760+0000","last_fullsized":"2023-05-16T07:36:34.984760+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622699+0000","last_change":"2023-05-16T07:36:34.621657+0000","last_active":"2023-05-16T07:36:34.622699+0000","last_peered":"2023-05-16T07:36:34.622699+0000","last_clean":"2023-05-16T07:36:34.622699+0000","last_became_active":"2023-05-16T07:36:34.621156+0000","last_became_peered":"2023-05-16T07:36:34.621156+0000","last_unstale":"2023-05-16T07:36:34.622699+0000","last_undegraded":"2023-05-16T07:36:34.622699+0000","last_fullsized":"2023-05-16T07:36:34.622699+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.995605+0000","last_change":"2023-05-16T07:36:34.995605+0000","last_active":"2023-05-16T07:36:34.995605+0000","last_peered":"2023-05-16T07:36:34.995605+0000","last_clean":"2023-05-16T07:36:34.995605+0000","last_became_active":"2023-05-16T07:36:34.995329+0000","last_became_peered":"2023-05-16T07:36:34.995329+0000","last_unstale":"2023-05-16T07:36:34.995605+0000","last_undegraded":"2023-05-16T07:36:34.995605+0000","last_fullsized":"2023-05-16T07:36:34.995605+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615942+0000","last_change":"2023-05-16T07:36:34.610720+0000","last_active":"2023-05-16T07:36:34.615942+0000","last_peered":"2023-05-16T07:36:34.615942+0000","last_clean":"2023-05-16T07:36:34.615942+0000","last_became_active":"2023-05-16T07:36:34.610449+0000","last_became_peered":"2023-05-16T07:36:34.610449+0000","last_unstale":"2023-05-16T07:36:34.615942+0000","last_undegraded":"2023-05-16T07:36:34.615942+0000","last_fullsized":"2023-05-16T07:36:34.615942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.974933+0000","last_change":"2023-05-16T07:36:34.974933+0000","last_active":"2023-05-16T07:36:34.974933+0000","last_peered":"2023-05-16T07:36:34.974933+0000","last_clean":"2023-05-16T07:36:34.974933+0000","last_became_active":"2023-05-16T07:36:34.974668+0000","last_became_peered":"2023-05-16T07:36:34.974668+0000","last_unstale":"2023-05-16T07:36:34.974933+0000","last_undegraded":"2023-05-16T07:36:34.974933+0000","last_fullsized":"2023-05-16T07:36:34.974933+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992918+0000","last_change":"2023-05-16T07:36:34.616834+0000","last_active":"2023-05-16T07:36:34.992918+0000","last_peered":"2023-05-16T07:36:34.992918+0000","last_clean":"2023-05-16T07:36:34.992918+0000","last_became_active":"2023-05-16T07:36:34.616550+0000","last_became_peered":"2023-05-16T07:36:34.616550+0000","last_unstale":"2023-05-16T07:36:34.992918+0000","last_undegraded":"2023-05-16T07:36:34.992918+0000","last_fullsized":"2023-05-16T07:36:34.992918+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.983542+0000","last_change":"2023-05-16T07:36:34.983542+0000","last_active":"2023-05-16T07:36:34.983542+0000","last_peered":"2023-05-16T07:36:34.983542+0000","last_clean":"2023-05-16T07:36:34.983542+0000","last_became_active":"2023-05-16T07:36:34.983275+0000","last_became_peered":"2023-05-16T07:36:34.983275+0000","last_unstale":"2023-05-16T07:36:34.983542+0000","last_undegraded":"2023-05-16T07:36:34.983542+0000","last_fullsized":"2023-05-16T07:36:34.983542+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.618653+0000","last_change":"2023-05-16T07:36:34.615853+0000","last_active":"2023-05-16T07:36:34.618653+0000","last_peered":"2023-05-16T07:36:34.618653+0000","last_clean":"2023-05-16T07:36:34.618653+0000","last_became_active":"2023-05-16T07:36:34.614255+0000","last_became_peered":"2023-05-16T07:36:34.614255+0000","last_unstale":"2023-05-16T07:36:34.618653+0000","last_undegraded":"2023-05-16T07:36:34.618653+0000","last_fullsized":"2023-05-16T07:36:34.618653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.992017+0000","last_change":"2023-05-16T07:36:34.992017+0000","last_active":"2023-05-16T07:36:34.992017+0000","last_peered":"2023-05-16T07:36:34.992017+0000","last_clean":"2023-05-16T07:36:34.992017+0000","last_became_active":"2023-05-16T07:36:34.991757+0000","last_became_peered":"2023-05-16T07:36:34.991757+0000","last_unstale":"2023-05-16T07:36:34.992017+0000","last_undegraded":"2023-05-16T07:36:34.992017+0000","last_fullsized":"2023-05-16T07:36:34.992017+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620522+0000","last_change":"2023-05-16T07:36:34.618066+0000","last_active":"2023-05-16T07:36:34.620522+0000","last_peered":"2023-05-16T07:36:34.620522+0000","last_clean":"2023-05-16T07:36:34.620522+0000","last_became_active":"2023-05-16T07:36:34.617837+0000","last_became_peered":"2023-05-16T07:36:34.617837+0000","last_unstale":"2023-05-16T07:36:34.620522+0000","last_undegraded":"2023-05-16T07:36:34.620522+0000","last_fullsized":"2023-05-16T07:36:34.620522+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986256+0000","last_change":"2023-05-16T07:36:34.986256+0000","last_active":"2023-05-16T07:36:34.986256+0000","last_peered":"2023-05-16T07:36:34.986256+0000","last_clean":"2023-05-16T07:36:34.986256+0000","last_became_active":"2023-05-16T07:36:34.985988+0000","last_became_peered":"2023-05-16T07:36:34.985988+0000","last_unstale":"2023-05-16T07:36:34.986256+0000","last_undegraded":"2023-05-16T07:36:34.986256+0000","last_fullsized":"2023-05-16T07:36:34.986256+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615919+0000","last_change":"2023-05-16T07:36:34.612892+0000","last_active":"2023-05-16T07:36:34.615919+0000","last_peered":"2023-05-16T07:36:34.615919+0000","last_clean":"2023-05-16T07:36:34.615919+0000","last_became_active":"2023-05-16T07:36:34.612628+0000","last_became_peered":"2023-05-16T07:36:34.612628+0000","last_unstale":"2023-05-16T07:36:34.615919+0000","last_undegraded":"2023-05-16T07:36:34.615919+0000","last_fullsized":"2023-05-16T07:36:34.615919+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980103+0000","last_change":"2023-05-16T07:36:34.980103+0000","last_active":"2023-05-16T07:36:34.980103+0000","last_peered":"2023-05-16T07:36:34.980103+0000","last_clean":"2023-05-16T07:36:34.980103+0000","last_became_active":"2023-05-16T07:36:34.979829+0000","last_became_peered":"2023-05-16T07:36:34.979829+0000","last_unstale":"2023-05-16T07:36:34.980103+0000","last_undegraded":"2023-05-16T07:36:34.980103+0000","last_fullsized":"2023-05-16T07:36:34.980103+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622378+0000","last_change":"2023-05-16T07:36:34.618023+0000","last_active":"2023-05-16T07:36:34.622378+0000","last_peered":"2023-05-16T07:36:34.622378+0000","last_clean":"2023-05-16T07:36:34.622378+0000","last_became_active":"2023-05-16T07:36:34.617623+0000","last_became_peered":"2023-05-16T07:36:34.617623+0000","last_unstale":"2023-05-16T07:36:34.622378+0000","last_undegraded":"2023-05-16T07:36:34.622378+0000","last_fullsized":"2023-05-16T07:36:34.622378+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980083+0000","last_change":"2023-05-16T07:36:34.980083+0000","last_active":"2023-05-16T07:36:34.980083+0000","last_peered":"2023-05-16T07:36:34.980083+0000","last_clean":"2023-05-16T07:36:34.980083+0000","last_became_active":"2023-05-16T07:36:34.979812+0000","last_became_peered":"2023-05-16T07:36:34.979812+0000","last_unstale":"2023-05-16T07:36:34.980083+0000","last_undegraded":"2023-05-16T07:36:34.980083+0000","last_fullsized":"2023-05-16T07:36:34.980083+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615920+0000","last_change":"2023-05-16T07:36:34.612837+0000","last_active":"2023-05-16T07:36:34.615920+0000","last_peered":"2023-05-16T07:36:34.615920+0000","last_clean":"2023-05-16T07:36:34.615920+0000","last_became_active":"2023-05-16T07:36:34.612568+0000","last_became_peered":"2023-05-16T07:36:34.612568+0000","last_unstale":"2023-05-16T07:36:34.615920+0000","last_undegraded":"2023-05-16T07:36:34.615920+0000","last_fullsized":"2023-05-16T07:36:34.615920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.986594+0000","last_change":"2023-05-16T07:36:34.986594+0000","last_active":"2023-05-16T07:36:34.986594+0000","last_peered":"2023-05-16T07:36:34.986594+0000","last_clean":"2023-05-16T07:36:34.986594+0000","last_became_active":"2023-05-16T07:36:34.986343+0000","last_became_peered":"2023-05-16T07:36:34.986343+0000","last_unstale":"2023-05-16T07:36:34.986594+0000","last_undegraded":"2023-05-16T07:36:34.986594+0000","last_fullsized":"2023-05-16T07:36:34.986594+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.615744+0000","last_change":"2023-05-16T07:36:34.612134+0000","last_active":"2023-05-16T07:36:34.615744+0000","last_peered":"2023-05-16T07:36:34.615744+0000","last_clean":"2023-05-16T07:36:34.615744+0000","last_became_active":"2023-05-16T07:36:34.611857+0000","last_became_peered":"2023-05-16T07:36:34.611857+0000","last_unstale":"2023-05-16T07:36:34.615744+0000","last_undegraded":"2023-05-16T07:36:34.615744+0000","last_fullsized":"2023-05-16T07:36:34.615744+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.995270+0000","last_change":"2023-05-16T07:36:34.995270+0000","last_active":"2023-05-16T07:36:34.995270+0000","last_peered":"2023-05-16T07:36:34.995270+0000","last_clean":"2023-05-16T07:36:34.995270+0000","last_became_active":"2023-05-16T07:36:34.994987+0000","last_became_peered":"2023-05-16T07:36:34.994987+0000","last_unstale":"2023-05-16T07:36:34.995270+0000","last_undegraded":"2023-05-16T07:36:34.995270+0000","last_fullsized":"2023-05-16T07:36:34.995270+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.624357+0000","last_change":"2023-05-16T07:36:34.622141+0000","last_active":"2023-05-16T07:36:34.624357+0000","last_peered":"2023-05-16T07:36:34.624357+0000","last_clean":"2023-05-16T07:36:34.624357+0000","last_became_active":"2023-05-16T07:36:34.621906+0000","last_became_peered":"2023-05-16T07:36:34.621906+0000","last_unstale":"2023-05-16T07:36:34.624357+0000","last_undegraded":"2023-05-16T07:36:34.624357+0000","last_fullsized":"2023-05-16T07:36:34.624357+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990517+0000","last_change":"2023-05-16T07:36:34.990517+0000","last_active":"2023-05-16T07:36:34.990517+0000","last_peered":"2023-05-16T07:36:34.990517+0000","last_clean":"2023-05-16T07:36:34.990517+0000","last_became_active":"2023-05-16T07:36:34.990237+0000","last_became_peered":"2023-05-16T07:36:34.990237+0000","last_unstale":"2023-05-16T07:36:34.990517+0000","last_undegraded":"2023-05-16T07:36:34.990517+0000","last_fullsized":"2023-05-16T07:36:34.990517+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"153'3","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.994834+0000","last_change":"2023-05-16T07:36:34.621386+0000","last_active":"2023-05-16T07:36:34.994834+0000","last_peered":"2023-05-16T07:36:34.994834+0000","last_clean":"2023-05-16T07:36:34.994834+0000","last_became_active":"2023-05-16T07:36:34.621118+0000","last_became_peered":"2023-05-16T07:36:34.621118+0000","last_unstale":"2023-05-16T07:36:34.994834+0000","last_undegraded":"2023-05-16T07:36:34.994834+0000","last_fullsized":"2023-05-16T07:36:34.994834+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.605492+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:36:34.605492+0000","last_peered":"2023-05-16T07:36:34.605492+0000","last_clean":"2023-05-16T07:36:34.605492+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:36:34.605492+0000","last_undegraded":"2023-05-16T07:36:34.605492+0000","last_fullsized":"2023-05-16T07:36:34.605492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.979602+0000","last_change":"2023-05-16T07:36:34.979602+0000","last_active":"2023-05-16T07:36:34.979602+0000","last_peered":"2023-05-16T07:36:34.979602+0000","last_clean":"2023-05-16T07:36:34.979602+0000","last_became_active":"2023-05-16T07:36:34.979389+0000","last_became_peered":"2023-05-16T07:36:34.979389+0000","last_unstale":"2023-05-16T07:36:34.979602+0000","last_undegraded":"2023-05-16T07:36:34.979602+0000","last_fullsized":"2023-05-16T07:36:34.979602+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.971966+0000","last_change":"2023-05-16T07:36:34.616220+0000","last_active":"2023-05-16T07:36:34.971966+0000","last_peered":"2023-05-16T07:36:34.971966+0000","last_clean":"2023-05-16T07:36:34.971966+0000","last_became_active":"2023-05-16T07:36:34.614466+0000","last_became_peered":"2023-05-16T07:36:34.614466+0000","last_unstale":"2023-05-16T07:36:34.971966+0000","last_undegraded":"2023-05-16T07:36:34.971966+0000","last_fullsized":"2023-05-16T07:36:34.971966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":117,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.610427+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:36:34.610427+0000","last_peered":"2023-05-16T07:36:34.610427+0000","last_clean":"2023-05-16T07:36:34.610427+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:36:34.610427+0000","last_undegraded":"2023-05-16T07:36:34.610427+0000","last_fullsized":"2023-05-16T07:36:34.610427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.980212+0000","last_change":"2023-05-16T07:36:34.980212+0000","last_active":"2023-05-16T07:36:34.980212+0000","last_peered":"2023-05-16T07:36:34.980212+0000","last_clean":"2023-05-16T07:36:34.980212+0000","last_became_active":"2023-05-16T07:36:34.979958+0000","last_became_peered":"2023-05-16T07:36:34.979958+0000","last_unstale":"2023-05-16T07:36:34.980212+0000","last_undegraded":"2023-05-16T07:36:34.980212+0000","last_fullsized":"2023-05-16T07:36:34.980212+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991627+0000","last_change":"2023-05-16T07:36:34.613619+0000","last_active":"2023-05-16T07:36:34.991627+0000","last_peered":"2023-05-16T07:36:34.991627+0000","last_clean":"2023-05-16T07:36:34.991627+0000","last_became_active":"2023-05-16T07:36:34.613332+0000","last_became_peered":"2023-05-16T07:36:34.613332+0000","last_unstale":"2023-05-16T07:36:34.991627+0000","last_undegraded":"2023-05-16T07:36:34.991627+0000","last_fullsized":"2023-05-16T07:36:34.991627+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.609923+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:36:34.609923+0000","last_peered":"2023-05-16T07:36:34.609923+0000","last_clean":"2023-05-16T07:36:34.609923+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:36:34.609923+0000","last_undegraded":"2023-05-16T07:36:34.609923+0000","last_fullsized":"2023-05-16T07:36:34.609923+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991823+0000","last_change":"2023-05-16T07:36:34.991823+0000","last_active":"2023-05-16T07:36:34.991823+0000","last_peered":"2023-05-16T07:36:34.991823+0000","last_clean":"2023-05-16T07:36:34.991823+0000","last_became_active":"2023-05-16T07:36:34.991526+0000","last_became_peered":"2023-05-16T07:36:34.991526+0000","last_unstale":"2023-05-16T07:36:34.991823+0000","last_undegraded":"2023-05-16T07:36:34.991823+0000","last_fullsized":"2023-05-16T07:36:34.991823+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622344+0000","last_change":"2023-05-16T07:36:34.621108+0000","last_active":"2023-05-16T07:36:34.622344+0000","last_peered":"2023-05-16T07:36:34.622344+0000","last_clean":"2023-05-16T07:36:34.622344+0000","last_became_active":"2023-05-16T07:36:34.620846+0000","last_became_peered":"2023-05-16T07:36:34.620846+0000","last_unstale":"2023-05-16T07:36:34.622344+0000","last_undegraded":"2023-05-16T07:36:34.622344+0000","last_fullsized":"2023-05-16T07:36:34.622344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":117,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.611155+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:36:34.611155+0000","last_peered":"2023-05-16T07:36:34.611155+0000","last_clean":"2023-05-16T07:36:34.611155+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:36:34.611155+0000","last_undegraded":"2023-05-16T07:36:34.611155+0000","last_fullsized":"2023-05-16T07:36:34.611155+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.604838+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:36:34.604838+0000","last_peered":"2023-05-16T07:36:34.604838+0000","last_clean":"2023-05-16T07:36:34.604838+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:36:34.604838+0000","last_undegraded":"2023-05-16T07:36:34.604838+0000","last_fullsized":"2023-05-16T07:36:34.604838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.990228+0000","last_change":"2023-05-16T07:36:34.990228+0000","last_active":"2023-05-16T07:36:34.990228+0000","last_peered":"2023-05-16T07:36:34.990228+0000","last_clean":"2023-05-16T07:36:34.990228+0000","last_became_active":"2023-05-16T07:36:34.989391+0000","last_became_peered":"2023-05-16T07:36:34.989391+0000","last_unstale":"2023-05-16T07:36:34.990228+0000","last_undegraded":"2023-05-16T07:36:34.990228+0000","last_fullsized":"2023-05-16T07:36:34.990228+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.620451+0000","last_change":"2023-05-16T07:36:34.617741+0000","last_active":"2023-05-16T07:36:34.620451+0000","last_peered":"2023-05-16T07:36:34.620451+0000","last_clean":"2023-05-16T07:36:34.620451+0000","last_became_active":"2023-05-16T07:36:34.617064+0000","last_became_peered":"2023-05-16T07:36:34.617064+0000","last_unstale":"2023-05-16T07:36:34.620451+0000","last_undegraded":"2023-05-16T07:36:34.620451+0000","last_fullsized":"2023-05-16T07:36:34.620451+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.604941+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:36:34.604941+0000","last_peered":"2023-05-16T07:36:34.604941+0000","last_clean":"2023-05-16T07:36:34.604941+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:36:34.604941+0000","last_undegraded":"2023-05-16T07:36:34.604941+0000","last_fullsized":"2023-05-16T07:36:34.604941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.982352+0000","last_change":"2023-05-16T07:36:34.982352+0000","last_active":"2023-05-16T07:36:34.982352+0000","last_peered":"2023-05-16T07:36:34.982352+0000","last_clean":"2023-05-16T07:36:34.982352+0000","last_became_active":"2023-05-16T07:36:34.982074+0000","last_became_peered":"2023-05-16T07:36:34.982074+0000","last_unstale":"2023-05-16T07:36:34.982352+0000","last_undegraded":"2023-05-16T07:36:34.982352+0000","last_fullsized":"2023-05-16T07:36:34.982352+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.987087+0000","last_change":"2023-05-16T07:36:34.618250+0000","last_active":"2023-05-16T07:36:34.987087+0000","last_peered":"2023-05-16T07:36:34.987087+0000","last_clean":"2023-05-16T07:36:34.987087+0000","last_became_active":"2023-05-16T07:36:34.617991+0000","last_became_peered":"2023-05-16T07:36:34.617991+0000","last_unstale":"2023-05-16T07:36:34.987087+0000","last_undegraded":"2023-05-16T07:36:34.987087+0000","last_fullsized":"2023-05-16T07:36:34.987087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.609006+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:36:34.609006+0000","last_peered":"2023-05-16T07:36:34.609006+0000","last_clean":"2023-05-16T07:36:34.609006+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:36:34.609006+0000","last_undegraded":"2023-05-16T07:36:34.609006+0000","last_fullsized":"2023-05-16T07:36:34.609006+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.991949+0000","last_change":"2023-05-16T07:36:34.991949+0000","last_active":"2023-05-16T07:36:34.991949+0000","last_peered":"2023-05-16T07:36:34.991949+0000","last_clean":"2023-05-16T07:36:34.991949+0000","last_became_active":"2023-05-16T07:36:34.991707+0000","last_became_peered":"2023-05-16T07:36:34.991707+0000","last_unstale":"2023-05-16T07:36:34.991949+0000","last_undegraded":"2023-05-16T07:36:34.991949+0000","last_fullsized":"2023-05-16T07:36:34.991949+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.622436+0000","last_change":"2023-05-16T07:36:34.618778+0000","last_active":"2023-05-16T07:36:34.622436+0000","last_peered":"2023-05-16T07:36:34.622436+0000","last_clean":"2023-05-16T07:36:34.622436+0000","last_became_active":"2023-05-16T07:36:34.618463+0000","last_became_peered":"2023-05-16T07:36:34.618463+0000","last_unstale":"2023-05-16T07:36:34.622436+0000","last_undegraded":"2023-05-16T07:36:34.622436+0000","last_fullsized":"2023-05-16T07:36:34.622436+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.604595+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:36:34.604595+0000","last_peered":"2023-05-16T07:36:34.604595+0000","last_clean":"2023-05-16T07:36:34.604595+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:36:34.604595+0000","last_undegraded":"2023-05-16T07:36:34.604595+0000","last_fullsized":"2023-05-16T07:36:34.604595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.981458+0000","last_change":"2023-05-16T07:36:34.981458+0000","last_active":"2023-05-16T07:36:34.981458+0000","last_peered":"2023-05-16T07:36:34.981458+0000","last_clean":"2023-05-16T07:36:34.981458+0000","last_became_active":"2023-05-16T07:36:34.980886+0000","last_became_peered":"2023-05-16T07:36:34.980886+0000","last_unstale":"2023-05-16T07:36:34.981458+0000","last_undegraded":"2023-05-16T07:36:34.981458+0000","last_fullsized":"2023-05-16T07:36:34.981458+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:34.590800+0000","last_clean_scrub_stamp":"2023-05-16T07:36:34.590800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.621556+0000","last_change":"2023-05-16T07:36:34.614164+0000","last_active":"2023-05-16T07:36:34.621556+0000","last_peered":"2023-05-16T07:36:34.621556+0000","last_clean":"2023-05-16T07:36:34.621556+0000","last_became_active":"2023-05-16T07:36:34.613896+0000","last_became_peered":"2023-05-16T07:36:34.613896+0000","last_unstale":"2023-05-16T07:36:34.621556+0000","last_undegraded":"2023-05-16T07:36:34.621556+0000","last_fullsized":"2023-05-16T07:36:34.621556+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:36:33.585965+0000","last_clean_scrub_stamp":"2023-05-16T07:36:33.585965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-05-16T07:36:34.605849+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:36:34.605849+0000","last_peered":"2023-05-16T07:36:34.605849+0000","last_clean":"2023-05-16T07:36:34.605849+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:36:34.605849+0000","last_undegraded":"2023-05-16T07:36:34.605849+0000","last_fullsized":"2023-05-16T07:36:34.605849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575253,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24408,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":22528,"kb_avail":94347432,"statfs":{"total":96636764160,"available":96611770368,"internally_reserved":0,"allocated":1916928,"data_stored":1014544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23068672},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.76400000000000001,"15min":0.80500000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48299999999999998}]},{"osd":1,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36499999999999999},"max":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75,"5min":0.77600000000000002,"15min":0.79800000000000004},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.80200000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56799999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.81999999999999995},"last":0.46800000000000003}]},{"osd":3,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79200000000000004,"15min":0.81899999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73499999999999999,"5min":0.80000000000000004,"15min":0.81100000000000005},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46700000000000003,"15min":0.46200000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.72699999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.76000000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.28399999999999997},"max":{"1min":0.63,"5min":0.75600000000000001,"15min":0.79100000000000004},"last":0.433},{"interface":"front","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":0.80800000000000005},"last":0.438}]},{"osd":6,"last update":"Tue May 16 07:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":0.70599999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.38100000000000001,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.495}]}]},{"osd":6,"up_from":13,"seq":55834575253,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24064,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":94347776,"statfs":{"total":96636764160,"available":96612122624,"internally_reserved":0,"allocated":1892352,"data_stored":1005991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.26700000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.59199999999999997,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.27900000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.86299999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.48299999999999998,"5min":0.438,"15min":0.33400000000000002},"max":{"1min":0.79500000000000004,"5min":1.5,"15min":1.5},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69699999999999995}]},{"osd":2,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61099999999999999,"15min":0.58199999999999996},"min":{"1min":0.50600000000000001,"5min":0.41999999999999998,"15min":0.373},"max":{"1min":0.73699999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":1.486,"15min":1.486},"last":0.65400000000000003}]},{"osd":3,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.62},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.64600000000000002},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82999999999999996,"5min":1.532,"15min":1.532},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.377,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.76500000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.499},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.30199999999999999},"max":{"1min":0.77700000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.497},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.31900000000000001},"max":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.79900000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.44900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.81299999999999994},"last":0.81200000000000006}]},{"osd":7,"last update":"Tue May 16 07:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.36799999999999999,"15min":0.34699999999999998},"max":{"1min":0.80300000000000005,"5min":0.876,"15min":0.876},"last":0.754},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.99399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575254,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19776,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":17920,"kb_avail":94352064,"statfs":{"total":96636764160,"available":96616513536,"internally_reserved":0,"allocated":1892352,"data_stored":1005991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18350080},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50800000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55500000000000005,"5min":0.63,"15min":0.64100000000000001},"last":0.29599999999999999}]},{"osd":2,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38400000000000001,"15min":0.38500000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.47199999999999998,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.252},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.54600000000000004,"5min":0.65700000000000003,"15min":0.70199999999999996},"last":0.23200000000000001}]},{"osd":3,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.435,"15min":0.436},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.56599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26200000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.50900000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.46300000000000002,"15min":0.38500000000000001},"max":{"1min":0.81499999999999995,"5min":0.873,"15min":0.88900000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.45000000000000001,"15min":0.40100000000000002},"max":{"1min":0.872,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.63900000000000001},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.38500000000000001},"max":{"1min":0.80700000000000005,"5min":0.86399999999999999,"15min":0.91000000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85099999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.44400000000000001}]},{"osd":6,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":0.872,"15min":0.998},"last":0.51800000000000002}]},{"osd":7,"last update":"Tue May 16 07:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.873,"15min":0.94199999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.53100000000000003,"5min":0.48999999999999999,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.89900000000000002,"15min":0.93700000000000006},"last":0.48999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19672,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":17792,"kb_avail":94352168,"statfs":{"total":96636764160,"available":96616620032,"internally_reserved":0,"allocated":1916928,"data_stored":1013936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18219008},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.36799999999999999,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.222},"max":{"1min":0.5,"5min":0.57599999999999996,"15min":0.58899999999999997},"last":0.223},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.27500000000000002,"15min":0.24199999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.627},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39100000000000001,"15min":0.38600000000000001},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.63100000000000001},"last":0.315},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.65700000000000003},"last":0.311}]},{"osd":3,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26300000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.64600000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.438,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.70199999999999996},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80500000000000005,"5min":0.92700000000000005,"15min":0.98699999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.41099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.91000000000000003,"15min":1.0549999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84799999999999998,"5min":0.94899999999999995,"15min":1.0680000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.439},"max":{"1min":0.877,"5min":0.93500000000000005,"15min":1.095},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.94099999999999995,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.505},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.67100000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40899999999999997},"max":{"1min":0.83399999999999996,"5min":0.92900000000000005,"15min":1},"last":0.46999999999999997}]},{"osd":7,"last update":"Tue May 16 07:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.873,"5min":0.91500000000000004,"15min":1.03},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.44400000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.056},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575254,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21332,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":19456,"kb_avail":94350508,"statfs":{"total":96636764160,"available":96614920192,"internally_reserved":0,"allocated":1912832,"data_stored":1014398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19922944},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.47499999999999998,"5min":0.747,"15min":0.747},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.50900000000000001,"5min":0.623,"15min":0.65600000000000003},"last":0.28199999999999997}]},{"osd":1,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.41299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.48899999999999999,"5min":0.59299999999999997,"15min":0.67200000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.33200000000000002,"5min":0.32900000000000001,"15min":0.30199999999999999},"max":{"1min":0.54100000000000004,"5min":0.62,"15min":0.67200000000000004},"last":0.313}]},{"osd":3,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39000000000000001,"15min":0.39500000000000002},"min":{"1min":0.27300000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.69399999999999995},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.63600000000000001},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.62,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.377},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":1.0389999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.76700000000000002,"5min":0.90100000000000002,"15min":1.048},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.88100000000000001,"15min":1.0069999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.997},"last":0.48599999999999999}]},{"osd":6,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.80000000000000004,"5min":0.84999999999999998,"15min":1.01},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.495,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81399999999999995,"5min":0.93799999999999994,"15min":1.0229999999999999},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue May 16 07:36:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82699999999999996,"5min":0.92800000000000005,"15min":0.998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65100000000000002,"15min":0.66000000000000003},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.90800000000000003,"15min":0.96499999999999997},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575253,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":26240,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1912832,"data_stored":1014454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26869760},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.36399999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.51200000000000001,"15min":0.61899999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.28699999999999998}]},{"osd":1,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":0.60099999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.37,"5min":0.41399999999999998,"15min":0.41499999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.72999999999999998},"last":0.27300000000000002}]},{"osd":2,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.42999999999999999,"15min":0.433},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55200000000000005,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.315},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.44900000000000001,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.70099999999999996},"last":0.308}]},{"osd":4,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":1.014,"15min":1.014},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.371},"max":{"1min":0.78300000000000003,"5min":1.024,"15min":1.024},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":1.05,"15min":1.05},"last":0.441},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41699999999999998},"max":{"1min":0.83599999999999997,"5min":0.96599999999999997,"15min":0.97299999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.65800000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.442},"max":{"1min":0.80700000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:36:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80300000000000005,"5min":0.98899999999999999,"15min":1.012},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575253,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24212,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":22336,"kb_avail":94347628,"statfs":{"total":96636764160,"available":96611971072,"internally_reserved":0,"allocated":1912832,"data_stored":1014434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22872064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.69799999999999995,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.77800000000000002,"15min":0.83799999999999997},"last":0.439}]},{"osd":1,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.29399999999999998},"max":{"1min":0.75600000000000001,"5min":0.78600000000000003,"15min":0.81899999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.48399999999999999}]},{"osd":2,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.72999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36299999999999999},"max":{"1min":0.81000000000000005,"5min":0.82099999999999995,"15min":0.82199999999999995},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.373},"max":{"1min":0.76300000000000001,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79000000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59899999999999998}]},{"osd":5,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.29099999999999998},"max":{"1min":0.70999999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.502,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"max":{"1min":0.65600000000000003,"5min":0.755,"15min":0.755},"last":0.5},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.318},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue May 16 07:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.33500000000000002},"max":{"1min":0.72299999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.82099999999999995,"15min":0.85899999999999999},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575253,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":1892352,"data_stored":1005991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.70399999999999996,"5min":0.79500000000000004,"15min":0.80400000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36599999999999999},"max":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.80800000000000005},"last":0.503},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.82399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72699999999999998,"5min":0.747,"15min":0.84699999999999998},"last":0.64100000000000001}]},{"osd":3,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.47899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79200000000000004,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.629},"min":{"1min":0.49199999999999999,"5min":0.436,"15min":0.41599999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.85599999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.46600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.65700000000000003,"5min":0.74199999999999999,"15min":0.84899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.34999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.82699999999999996,"15min":0.83299999999999996},"last":0.627}]},{"osd":6,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.505,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32400000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue May 16 07:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80400000000000005,"5min":0.85299999999999998,"15min":0.86299999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.34999999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:36:38.340 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:36:38.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:36:38.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.455+0000 7eff0178a700 1 Processor -- start 2023-05-16T07:36:38.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.455+0000 7eff0178a700 1 -- start start 2023-05-16T07:36:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.455+0000 7eff0178a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efefc068870 0x7efefc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.455+0000 7eff0178a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc0c94d0 con 0x7efefc154470 2023-05-16T07:36:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.455+0000 7eff0178a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc0cb7e0 con 0x7efefc154090 2023-05-16T07:36:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.455+0000 7eff0178a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc0c88c0 con 0x7efefc068870 2023-05-16T07:36:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efefc068870 0x7efefc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efef27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efefc154470 0x7efefc068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34194/0 (socket says 172.21.15.92:34194) 2023-05-16T07:36:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efefc068870 0x7efefc0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51676/0 (socket says 172.21.15.92:51676) 2023-05-16T07:36:38.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efef27fc700 1 -- 172.21.15.92:0/4074336015 learned_addr learned my addr 172.21.15.92:0/4074336015 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:38.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefb7fe700 1 -- 172.21.15.92:0/4074336015 >> v1:172.21.15.158:6789/0 conn(0x7efefc154090 legacy=0x7efefc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:38.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefb7fe700 1 -- 172.21.15.92:0/4074336015 >> v1:172.21.15.92:6789/0 conn(0x7efefc154470 legacy=0x7efefc068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefb7fe700 1 -- 172.21.15.92:0/4074336015 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc0c8ba0 con 0x7efefc068870 2023-05-16T07:36:38.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefb7fe700 1 --2- 172.21.15.92:0/4074336015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc068870 0x7efefc0aaca0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7efef4009e60 tx=0x7efef4009780).ready entity=mon.0 client_cookie=a219b81104a29b61 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefa7fc700 1 -- 172.21.15.92:0/4074336015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400e070 con 0x7efefc068870 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefa7fc700 1 -- 172.21.15.92:0/4074336015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400bbf0 con 0x7efefc068870 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.456+0000 7efefa7fc700 1 -- 172.21.15.92:0/4074336015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4012410 con 0x7efefc068870 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 -- 172.21.15.92:0/4074336015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc068870 msgr2=0x7efefc0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 --2- 172.21.15.92:0/4074336015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc068870 0x7efefc0aaca0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7efef4009e60 tx=0x7efef4009780).stop 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 -- 172.21.15.92:0/4074336015 shutdown_connections 2023-05-16T07:36:38.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 --2- 172.21.15.92:0/4074336015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc068870 0x7efefc0aaca0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 -- 172.21.15.92:0/4074336015 >> 172.21.15.92:0/4074336015 conn(0x7efefc067920 msgr2=0x7efefc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:38.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 -- 172.21.15.92:0/4074336015 shutdown_connections 2023-05-16T07:36:38.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.457+0000 7eff0178a700 1 -- 172.21.15.92:0/4074336015 wait complete. 2023-05-16T07:36:38.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.458+0000 7eff0178a700 1 Processor -- start 2023-05-16T07:36:38.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.458+0000 7eff0178a700 1 -- start start 2023-05-16T07:36:38.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efefc068870 0x7efefc164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 0x7efefc14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efefc154470 0x7efefc0abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc155ad0 con 0x7efefc154090 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0aee80 con 0x7efefc068870 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc155d70 con 0x7efefc154470 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 0x7efefc14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 0x7efefc14dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51682/0 (socket says 172.21.15.92:51682) 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 -- 172.21.15.92:0/119540705 learned_addr learned my addr 172.21.15.92:0/119540705 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 -- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efefc154470 msgr2=0x7efefc0abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efefc154470 0x7efefc0abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 -- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efefc068870 msgr2=0x7efefc164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efefaffd700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efefc068870 0x7efefc164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efefc068870 0x7efefc164b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 -- 172.21.15.92:0/119540705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4009a20 con 0x7efefc154090 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef27fc700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 0x7efefc14dc00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7efee4001eb0 tx=0x7efee4007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee40072e0 con 0x7efefc154090 2023-05-16T07:36:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4007440 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc156050 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee40057a0 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.459+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc14e0f0 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.460+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc14e2e0 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.461+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7efee401a060 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.461+0000 7efef37fe700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efedc05ed90 0x7efedc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.461+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7efee4079460 con 0x7efefc154090 2023-05-16T07:36:38.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.461+0000 7efefaffd700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efedc05ed90 0x7efedc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.462+0000 7efefaffd700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efedc05ed90 0x7efedc061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7efeec007fd0 tx=0x7efeec009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.465+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7efee404a780 con 0x7efefc154090 2023-05-16T07:36:38.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.690+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7efefc066730 con 0x7efefc154090 2023-05-16T07:36:38.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.694+0000 7efef37fe700 1 -- 172.21.15.92:0/119540705 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7efee404dda0 con 0x7efefc154090 2023-05-16T07:36:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efedc05ed90 msgr2=0x7efedc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efedc05ed90 0x7efedc061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7efeec007fd0 tx=0x7efeec009040).stop 2023-05-16T07:36:38.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 msgr2=0x7efefc14dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 0x7efefc14dc00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7efee4001eb0 tx=0x7efee4007e50).stop 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 shutdown_connections 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efedc05ed90 0x7efedc061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efefc154470 0x7efefc0abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efefc068870 0x7efefc164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 --2- 172.21.15.92:0/119540705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc154090 0x7efefc14dc00 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 >> 172.21.15.92:0/119540705 conn(0x7efefc067920 msgr2=0x7efefc0c0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 shutdown_connections 2023-05-16T07:36:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.696+0000 7eff0178a700 1 -- 172.21.15.92:0/119540705 wait complete. 2023-05-16T07:36:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:36:38.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:36:38.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.818+0000 7f7f7c2d6700 1 Processor -- start 2023-05-16T07:36:38.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.819+0000 7f7f7c2d6700 1 -- start start 2023-05-16T07:36:38.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.819+0000 7f7f7c2d6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f74068060 0x7f7f740bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.819+0000 7f7f7c2d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f740bdbd0 con 0x7f7f740bd780 2023-05-16T07:36:38.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.819+0000 7f7f7c2d6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f74154b80 con 0x7f7f7413a5a0 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.819+0000 7f7f7c2d6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f74154e60 con 0x7f7f74068060 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79871700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f74068060 0x7f7f740bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f7a873700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f740bd780 0x7f7f740c2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34208/0 (socket says 172.21.15.92:34208) 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f7a873700 1 -- 172.21.15.92:0/181715024 learned_addr learned my addr 172.21.15.92:0/181715024 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79871700 1 -- 172.21.15.92:0/181715024 >> v1:172.21.15.158:6789/0 conn(0x7f7f7413a5a0 legacy=0x7f7f74068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79871700 1 -- 172.21.15.92:0/181715024 >> v1:172.21.15.92:6789/0 conn(0x7f7f740bd780 legacy=0x7f7f740c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79070700 1 -- 172.21.15.92:0/181715024 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969280422 0 0) 0x7f7f740bdbd0 con 0x7f7f740bd780 2023-05-16T07:36:38.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79871700 1 -- 172.21.15.92:0/181715024 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74155140 con 0x7f7f74068060 2023-05-16T07:36:38.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79871700 1 --2- 172.21.15.92:0/181715024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f74068060 0x7f7f740bd290 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f7f64009760 tx=0x7f7f64009060).ready entity=mon.0 client_cookie=3693c318e38ceb82 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79070700 1 -- 172.21.15.92:0/181715024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400b580 con 0x7f7f74068060 2023-05-16T07:36:38.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79070700 1 -- 172.21.15.92:0/181715024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400b6e0 con 0x7f7f74068060 2023-05-16T07:36:38.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.820+0000 7f7f79070700 1 -- 172.21.15.92:0/181715024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400ba60 con 0x7f7f74068060 2023-05-16T07:36:38.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/181715024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f74068060 msgr2=0x7f7f740bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/181715024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f74068060 0x7f7f740bd290 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f7f64009760 tx=0x7f7f64009060).stop 2023-05-16T07:36:38.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/181715024 shutdown_connections 2023-05-16T07:36:38.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/181715024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f74068060 0x7f7f740bd290 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/181715024 >> 172.21.15.92:0/181715024 conn(0x7f7f740c4430 msgr2=0x7f7f740c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:38.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/181715024 shutdown_connections 2023-05-16T07:36:38.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.821+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/181715024 wait complete. 2023-05-16T07:36:38.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.822+0000 7f7f7c2d6700 1 Processor -- start 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.822+0000 7f7f7c2d6700 1 -- start start 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f74068060 0x7f7f741641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 0x7f7f7414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f7413a5a0 0x7f7f7414d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7414c450 con 0x7f7f740bd780 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7414cf90 con 0x7f7f74068060 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7414c660 con 0x7f7f7413a5a0 2023-05-16T07:36:38.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 0x7f7f7414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 0x7f7f7414fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51696/0 (socket says 172.21.15.92:51696) 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 -- 172.21.15.92:0/2158422195 learned_addr learned my addr 172.21.15.92:0/2158422195 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7a072700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f74068060 0x7f7f741641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7a873700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f7413a5a0 0x7f7f7414d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 -- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f7413a5a0 msgr2=0x7f7f7414d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f7413a5a0 0x7f7f7414d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 -- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f74068060 msgr2=0x7f7f741641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f74068060 0x7f7f741641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 -- 172.21.15.92:0/2158422195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640092a0 con 0x7f7f740bd780 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7a873700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f7413a5a0 0x7f7f7414d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f79871700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 0x7f7f7414fe90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f7f74064cf0 tx=0x7f7f64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400ba60 con 0x7f7f740bd780 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f64000c90 con 0x7f7f740bd780 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7414c940 con 0x7f7f740bd780 2023-05-16T07:36:38.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6401b640 con 0x7f7f740bd780 2023-05-16T07:36:38.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.823+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f74150380 con 0x7f7f740bd780 2023-05-16T07:36:38.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.824+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f740bcb20 con 0x7f7f740bd780 2023-05-16T07:36:38.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.825+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7f6400f070 con 0x7f7f740bd780 2023-05-16T07:36:38.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.825+0000 7f7f6a7fc700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7f5805ed90 0x7f7f58061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:38.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.825+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f7f6407b910 con 0x7f7f740bd780 2023-05-16T07:36:38.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.825+0000 7f7f7a072700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7f5805ed90 0x7f7f58061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:38.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.825+0000 7f7f7a072700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7f5805ed90 0x7f7f58061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7f70003630 tx=0x7f7f70009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:38.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:38.829+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7f6404cc30 con 0x7f7f740bd780 2023-05-16T07:36:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.035+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f7406c350 con 0x7f7f740bd780 2023-05-16T07:36:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.036+0000 7f7f6a7fc700 1 -- 172.21.15.92:0/2158422195 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f7f64050250 con 0x7f7f740bd780 2023-05-16T07:36:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:39.039 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:36:34.963007+0000","modified":"2023-05-16T07:36:36.970661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/1435424518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1435424518},{"type":"v1","addr":"172.21.15.158:6837","nonce":1435424518}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-05-16T07:36:39.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7f5805ed90 msgr2=0x7f7f58061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7f5805ed90 0x7f7f58061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7f70003630 tx=0x7f7f70009040).stop 2023-05-16T07:36:39.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 msgr2=0x7f7f7414fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 0x7f7f7414fe90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f7f74064cf0 tx=0x7f7f64017000).stop 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 shutdown_connections 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7f5805ed90 0x7f7f58061250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7f7413a5a0 0x7f7f7414d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f74068060 0x7f7f741641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 --2- 172.21.15.92:0/2158422195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f740bd780 0x7f7f7414fe90 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 >> 172.21.15.92:0/2158422195 conn(0x7f7f740c4430 msgr2=0x7f7f74065000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 shutdown_connections 2023-05-16T07:36:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.038+0000 7f7f7c2d6700 1 -- 172.21.15.92:0/2158422195 wait complete. 2023-05-16T07:36:39.043 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-05-16T07:36:39.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:36:34.963007+0000', 'modified': '2023-05-16T07:36:36.970661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8394': {'gid': 8394, 'name': 'b', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.158:6837/1435424518', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6835', 'nonce': 1435424518}, {'type': 'v1', 'addr': '172.21.15.158:6837', 'nonce': 1435424518}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:36:39.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:36:39.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-05-16T07:36:39.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.566+0000 7f009616e700 1 Processor -- start 2023-05-16T07:36:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.566+0000 7f009616e700 1 -- start start 2023-05-16T07:36:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f009616e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0090154550 0x7f00900696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f009616e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00900c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f009616e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900c0660 con 0x7f0090154170 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f009616e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009006a000 con 0x7f0090154550 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f009616e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900bfd30 con 0x7f0090069b90 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f008f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0090154170 0x7f00900c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34218/0 (socket says 172.21.15.92:34218) 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f008f7fe700 1 -- 172.21.15.92:0/1893182433 learned_addr learned my addr 172.21.15.92:0/1893182433 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f008effd700 1 --2- 172.21.15.92:0/1893182433 >> v2:172.21.15.158:3300/0 conn(0x7f0090154550 0x7f00900696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f008ffff700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00900c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832909918 0 0) 0x7f00900c0660 con 0x7f0090154170 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.567+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00780033f0 con 0x7f0090154170 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833139555 0 0) 0x7f00780033f0 con 0x7f0090154170 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 msgr2=0x7f00900c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00900c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 >> v2:172.21.15.158:3300/0 conn(0x7f0090154550 msgr2=0x7f00900696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 --2- 172.21.15.92:0/1893182433 >> v2:172.21.15.158:3300/0 conn(0x7f0090154550 0x7f00900696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900c0010 con 0x7f0090154170 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f00800017f0 con 0x7f0090154170 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 >> v1:172.21.15.92:6789/0 conn(0x7f0090154170 legacy=0x7f00900c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00780038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0078003e10 0x7f0078004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0078004b80 0x7f0078005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009006a000 con 0x7f0078004b80 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900bfd30 con 0x7f0078003e10 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007800b8e0 con 0x7f0090069b90 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0078004b80 0x7f0078005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008f7fe700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00780038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008effd700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0078003e10 0x7f0078004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 -- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 msgr2=0x7f00780038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00780038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 -- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0078003e10 msgr2=0x7f0078004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0078003e10 0x7f0078004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 -- 172.21.15.92:0/1893182433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780033f0 con 0x7f0078004b80 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008f7fe700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0090069b90 0x7f00780038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008effd700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0078003e10 0x7f0078004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008ffff700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0078004b80 0x7f0078005000 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f0088009a90 tx=0x7f0088009060).ready entity=mon.0 client_cookie=c8ed14a5e61c4893 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088009430 con 0x7f0078004b80 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008800e640 con 0x7f0078004b80 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.568+0000 7f008e7fc700 1 -- 172.21.15.92:0/1893182433 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008800e9c0 con 0x7f0078004b80 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f009616e700 1 -- 172.21.15.92:0/1893182433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0078004b80 msgr2=0x7f0078005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f009616e700 1 --2- 172.21.15.92:0/1893182433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0078004b80 0x7f0078005000 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f0088009a90 tx=0x7f0088009060).stop 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f008ffff700 1 -- 172.21.15.92:0/1893182433 reap_dead start 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f009616e700 1 -- 172.21.15.92:0/1893182433 shutdown_connections 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f009616e700 1 -- 172.21.15.92:0/1893182433 >> 172.21.15.92:0/1893182433 conn(0x7f00900cb700 msgr2=0x7f00900c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f009616e700 1 -- 172.21.15.92:0/1893182433 shutdown_connections 2023-05-16T07:36:39.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.569+0000 7f009616e700 1 -- 172.21.15.92:0/1893182433 wait complete. 2023-05-16T07:36:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.570+0000 7f009616e700 1 Processor -- start 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.570+0000 7f009616e700 1 -- start start 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f009616e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0078004b80 0x7f0090142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f009616e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 0x7f0090156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f009616e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0090152f10 0x7f0090153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f009616e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090156e30 con 0x7f0090168040 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f009616e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090153860 con 0x7f0090152f10 2023-05-16T07:36:39.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f009616e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00901500c0 con 0x7f0078004b80 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 0x7f0090156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 0x7f0090156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51716/0 (socket says 172.21.15.92:51716) 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 -- 172.21.15.92:0/1990421150 learned_addr learned my addr 172.21.15.92:0/1990421150 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008ffff700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0090152f10 0x7f0090153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008f7fe700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0078004b80 0x7f0090142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 -- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0078004b80 msgr2=0x7f0090142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0078004b80 0x7f0090142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 -- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0090152f10 msgr2=0x7f0090153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0090152f10 0x7f0090153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 -- 172.21.15.92:0/1990421150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900b0010 con 0x7f0090168040 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f008effd700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 0x7f0090156490 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f00840115f0 tx=0x7f0084008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f00777fe700 1 -- 172.21.15.92:0/1990421150 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840145f0 con 0x7f0090168040 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.571+0000 7f00777fe700 1 -- 172.21.15.92:0/1990421150 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084014750 con 0x7f0090168040 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.572+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00900b02f0 con 0x7f0090168040 2023-05-16T07:36:39.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.572+0000 7f00777fe700 1 -- 172.21.15.92:0/1990421150 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084014ad0 con 0x7f0090168040 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.572+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0090151860 con 0x7f0090168040 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.573+0000 7f00777fe700 1 -- 172.21.15.92:0/1990421150 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0084012040 con 0x7f0090168040 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.573+0000 7f00777fe700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f007c05ed60 0x7f007c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.573+0000 7f00777fe700 1 -- 172.21.15.92:0/1990421150 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f008407a320 con 0x7f0090168040 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.573+0000 7f008f7fe700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f007c05ed60 0x7f007c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.574+0000 7f008f7fe700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f007c05ed60 0x7f007c061220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0080007fd0 tx=0x7f0080009040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.575+0000 7f00a7b00b80 1 Processor -- start 2023-05-16T07:36:39.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.575+0000 7f00a7b00b80 1 -- start start 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2c6560 0x557fce2a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 0x557fce2c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce27c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce2475e0 con 0x557fce2cb480 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce1be8a0 con 0x557fce2c6560 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fce1d3900 con 0x557fce282300 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 0x557fce2c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 0x557fce2c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51722/0 (socket says 172.21.15.92:51722) 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 -- 172.21.15.92:0/2751735378 learned_addr learned my addr 172.21.15.92:0/2751735378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 -- 172.21.15.92:0/2751735378 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 msgr2=0x557fce27c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce27c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 -- 172.21.15.92:0/2751735378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2c6560 msgr2=0x557fce2a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2c6560 0x557fce2a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 -- 172.21.15.92:0/2751735378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fce247450 con 0x557fce2cb480 2023-05-16T07:36:39.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.576+0000 7f008effd700 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 0x557fce2c6940 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f008404eef0 tx=0x7f0084009060).ready entity=mon.0 client_cookie=f540e7a49b4ff0c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f0074ff9700 1 -- 172.21.15.92:0/2751735378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008401cdc0 con 0x557fce2cb480 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f0074ff9700 1 -- 172.21.15.92:0/2751735378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084053070 con 0x557fce2cb480 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f0074ff9700 1 -- 172.21.15.92:0/2751735378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008404d6a0 con 0x557fce2cb480 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2751735378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 msgr2=0x557fce2c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 0x557fce2c6940 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f008404eef0 tx=0x7f0084009060).stop 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2751735378 shutdown_connections 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce27c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2c6560 0x557fce2a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2751735378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2cb480 0x557fce2c6940 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2751735378 >> 172.21.15.92:0/2751735378 conn(0x557fce03ee40 msgr2=0x557fce2ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2751735378 shutdown_connections 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.577+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2751735378 wait complete. 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.578+0000 7f00a7b00b80 1 -- start start 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.578+0000 7f00a7b00b80 1 Processor -- start 2023-05-16T07:36:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce19aed0 con 0x557fce282300 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce2ac5c0 con 0x557fce2cb480 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fce27c500 con 0x557fce2c6560 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 10 client.0 ll_register_callbacks cb 0x557fce2411f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51728/0 (socket says 172.21.15.92:51728) 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 -- 172.21.15.92:0/2959127041 learned_addr learned my addr 172.21.15.92:0/2959127041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008ffff700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008effd700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 -- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 msgr2=0x557fce2ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 -- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 msgr2=0x557fce2ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 -- 172.21.15.92:0/2959127041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0090151860 con 0x557fce282300 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008ffff700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f008f7fe700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27d7f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f0080018830 tx=0x7f0080014000).ready entity=mon.0 client_cookie=679b0dfdcafc764b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f006dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f006dffb700 1 -- 172.21.15.92:0/2959127041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008001d070 con 0x557fce282300 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f006dffb700 1 -- 172.21.15.92:0/2959127041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f00800155d0 con 0x557fce282300 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fce275460 con 0x557fce282300 2023-05-16T07:36:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f006dffb700 1 -- 172.21.15.92:0/2959127041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00800159a0 con 0x557fce282300 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 10 client.8538 fetch_fsmap learned FSMap version 55 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.579+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x557fce076310 con 0x557fce282300 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f006dffb700 1 -- 172.21.15.92:0/2959127041 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557fce275460 con 0x557fce282300 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f006dffb700 1 -- 172.21.15.92:0/2959127041 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0080015b00 con 0x557fce282300 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f00a7b00b80 10 client.8538 fetch_fsmap finished waiting for FSMap version 55 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f00a7b00b80 10 client.8538 resolve_mds: resolved name 'b' to daemon mds.b 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f00a7b00b80 20 client.8538 populate_metadata read hostname 'smithi092' 2023-05-16T07:36:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce2b02b0 0x557fce2b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f00a7b00b80 4 client.8538 mds_command: new command op to 8394 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 --> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fce03aca0 con 0x557fce2b02b0 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f008effd700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce2b02b0 0x557fce2b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f006dffb700 10 client.8538 ms_handle_connect on v2:172.21.15.158:6835/1435424518 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.580+0000 7f008effd700 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce2b02b0 0x557fce2b0690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=32c3e7da6c468c08 server_cookie=a8c654c6c6b8cf30 in_seq=0 out_seq=0 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f006dffb700 1 -- 172.21.15.92:0/2959127041 <== mds.0 v2:172.21.15.158:6835/1435424518 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x557fce03aca0 con 0x557fce2b02b0 2023-05-16T07:36:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f006dffb700 10 client.8538 handle_command_reply: tid=0 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 client.8538 shutdown 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 msgr2=0x557fce27d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27d7f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f0080018830 tx=0x7f0080014000).stop 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 shutdown_connections 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce2b02b0 0x557fce2b0690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27d7f0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/2959127041 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f006dffb700 0 client.8538 ms_handle_reset on v2:172.21.15.158:6835/1435424518 2023-05-16T07:36:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 >> 172.21.15.92:0/2959127041 conn(0x557fce03ee40 msgr2=0x557fce2ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 shutdown_connections 2023-05-16T07:36:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 1 -- 172.21.15.92:0/2959127041 wait complete. 2023-05-16T07:36:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.581+0000 7f00a7b00b80 20 client.8538 trim_cache size 0 max 16384 2023-05-16T07:36:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.597+0000 7f00a7b00b80 1 Processor -- start 2023-05-16T07:36:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.597+0000 7f00a7b00b80 1 -- start start 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce277b00 con 0x557fce282300 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce2c5bd0 con 0x557fce2cb480 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fce2afe80 con 0x557fce2c6560 2023-05-16T07:36:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51736/0 (socket says 172.21.15.92:51736) 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 -- 172.21.15.92:0/4133427550 learned_addr learned my addr 172.21.15.92:0/4133427550 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 -- 172.21.15.92:0/4133427550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 msgr2=0x557fce2a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 -- 172.21.15.92:0/4133427550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 msgr2=0x557fce2c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 -- 172.21.15.92:0/4133427550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fce1cc880 con 0x557fce282300 2023-05-16T07:36:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f008f7fe700 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27bae0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f0080001010 tx=0x7f0080018000).ready entity=mon.0 client_cookie=65be6767d7f58b06 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.598+0000 7f00757fa700 1 -- 172.21.15.92:0/4133427550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800155d0 con 0x557fce282300 2023-05-16T07:36:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00757fa700 1 -- 172.21.15.92:0/4133427550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080001ff0 con 0x557fce282300 2023-05-16T07:36:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00757fa700 1 -- 172.21.15.92:0/4133427550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080015bb0 con 0x557fce282300 2023-05-16T07:36:39.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 -- 172.21.15.92:0/4133427550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 msgr2=0x557fce27bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27bae0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f0080001010 tx=0x7f0080018000).stop 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 -- 172.21.15.92:0/4133427550 shutdown_connections 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce282300 0x557fce27bae0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/4133427550 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce2c6560 0x557fce2a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 -- 172.21.15.92:0/4133427550 >> 172.21.15.92:0/4133427550 conn(0x557fce03ee40 msgr2=0x557fce2ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 -- 172.21.15.92:0/4133427550 shutdown_connections 2023-05-16T07:36:39.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.599+0000 7f00a7b00b80 1 -- 172.21.15.92:0/4133427550 wait complete. 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 -- start start 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 Processor -- start 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce2d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 0x557fce2e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2d8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce2aecb0 con 0x557fce2c6560 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fce2e3910 con 0x557fce2cb480 2023-05-16T07:36:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fce2d7dc0 con 0x557fce282300 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f00a7b00b80 10 client.0 ll_register_callbacks cb 0x557fce2411f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f008effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 0x557fce2e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f008effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 0x557fce2e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51744/0 (socket says 172.21.15.92:51744) 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.600+0000 7f008effd700 1 -- 172.21.15.92:0/273761872 learned_addr learned my addr 172.21.15.92:0/273761872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008ffff700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008f7fe700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce2d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008effd700 1 -- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 msgr2=0x557fce2d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008effd700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce2d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008effd700 1 -- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 msgr2=0x557fce2d8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008effd700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2d8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008effd700 1 -- 172.21.15.92:0/273761872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fce076310 con 0x557fce2c6560 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008ffff700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f008effd700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 0x557fce2e2f70 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f008404d650 tx=0x7f0084053000).ready entity=mon.0 client_cookie=c856b97dca0034c3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f006dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:36:39.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f006dffb700 1 -- 172.21.15.92:0/273761872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084014c30 con 0x557fce2c6560 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f006dffb700 1 -- 172.21.15.92:0/273761872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f008404ba10 con 0x557fce2c6560 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0084053070 con 0x557fce2c6560 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f006dffb700 1 -- 172.21.15.92:0/273761872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f008404bbd0 con 0x557fce2c6560 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 10 client.8544 fetch_fsmap learned FSMap version 55 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x557fce1c8070 con 0x557fce2c6560 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f006dffb700 1 -- 172.21.15.92:0/273761872 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084053070 con 0x557fce2c6560 2023-05-16T07:36:39.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f006dffb700 1 -- 172.21.15.92:0/273761872 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f008404b0f0 con 0x557fce2c6560 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 10 client.8544 fetch_fsmap finished waiting for FSMap version 55 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 10 client.8544 resolve_mds: resolved name 'b' to daemon mds.b 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 20 client.8544 populate_metadata read hostname 'smithi092' 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce32f790 0x557fce32b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 4 client.8544 mds_command: new command op to 8394 tid=0[{"prefix": "status"}] 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.601+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 --> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557fce071ca0 con 0x557fce32f790 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f008f7fe700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce32f790 0x557fce32b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f006dffb700 10 client.8544 ms_handle_connect on v2:172.21.15.158:6835/1435424518 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f008f7fe700 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce32f790 0x557fce32b5f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2587ae054745863d server_cookie=dd5e5cae46b78980 in_seq=0 out_seq=0 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f006dffb700 1 -- 172.21.15.92:0/273761872 <== mds.0 v2:172.21.15.158:6835/1435424518 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557fce071ca0 con 0x557fce32f790 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f006dffb700 10 client.8544 handle_command_reply: tid=0 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 client.8544 shutdown 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 msgr2=0x557fce2e2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 0x557fce2e2f70 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f008404d650 tx=0x7f0084053000).stop 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 shutdown_connections 2023-05-16T07:36:39.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:6835/1435424518,v1:172.21.15.158:6837/1435424518] conn(0x557fce32f790 0x557fce32b5f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x557fce282300 0x557fce2d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f006dffb700 0 client.8544 ms_handle_reset on v2:172.21.15.158:6835/1435424518 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x557fce2cb480 0x557fce2d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 --2- 172.21.15.92:0/273761872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fce2c6560 0x557fce2e2f70 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 >> 172.21.15.92:0/273761872 conn(0x557fce03ee40 msgr2=0x557fce27d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 shutdown_connections 2023-05-16T07:36:39.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 1 -- 172.21.15.92:0/273761872 wait complete. 2023-05-16T07:36:39.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.602+0000 7f00a7b00b80 20 client.8544 trim_cache size 0 max 16384 2023-05-16T07:36:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:36:39.616 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:36:39.616 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:36:39.616 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8394, 2023-05-16T07:36:39.616 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.6383570409999999, 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 55, 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 154, 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 154, 2023-05-16T07:36:39.617 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.1628929760000002 2023-05-16T07:36:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:36:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f007c05ed60 msgr2=0x7f007c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f007c05ed60 0x7f007c061220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0080007fd0 tx=0x7f0080009040).stop 2023-05-16T07:36:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 msgr2=0x7f0090156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 0x7f0090156490 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f00840115f0 tx=0x7f0084008040).stop 2023-05-16T07:36:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 shutdown_connections 2023-05-16T07:36:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f007c05ed60 0x7f007c061220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0078004b80 0x7f0090142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0090152f10 0x7f0090153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 --2- 172.21.15.92:0/1990421150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0090168040 0x7f0090156490 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 >> 172.21.15.92:0/1990421150 conn(0x7f00900cb700 msgr2=0x7f00900ab9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 shutdown_connections 2023-05-16T07:36:39.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.603+0000 7f009616e700 1 -- 172.21.15.92:0/1990421150 wait complete. 2023-05-16T07:36:39.621 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:36:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.717+0000 7f7c31b2c700 1 Processor -- start 2023-05-16T07:36:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.717+0000 7f7c31b2c700 1 -- start start 2023-05-16T07:36:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c31b2c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c1545b0 0x7f7c2c064a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c31b2c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c065380 con 0x7f7c2c064f10 2023-05-16T07:36:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c31b2c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0aa320 con 0x7f7c2c0c1460 2023-05-16T07:36:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c31b2c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c154a60 con 0x7f7c2c1545b0 2023-05-16T07:36:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c3132b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7c2c064f10 0x7f7c2c0bfa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34266/0 (socket says 172.21.15.92:34266) 2023-05-16T07:36:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c3132b700 1 -- 172.21.15.92:0/3093679629 learned_addr learned my addr 172.21.15.92:0/3093679629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c2bfff700 1 --2- 172.21.15.92:0/3093679629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c1545b0 0x7f7c2c064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896051604 0 0) 0x7f7c2c065380 con 0x7f7c2c064f10 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.718+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c180033f0 con 0x7f7c2c064f10 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314308809 0 0) 0x7f7c2c0aa320 con 0x7f7c2c0c1460 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c2c065380 con 0x7f7c2c0c1460 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807393912 0 0) 0x7f7c180033f0 con 0x7f7c2c064f10 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2bfff700 1 -- 172.21.15.92:0/3093679629 >> v1:172.21.15.158:6789/0 conn(0x7f7c2c0c1460 legacy=0x7f7c2c0a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2bfff700 1 -- 172.21.15.92:0/3093679629 >> v1:172.21.15.92:6789/0 conn(0x7f7c2c064f10 legacy=0x7f7c2c0bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2bfff700 1 -- 172.21.15.92:0/3093679629 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c154d40 con 0x7f7c2c1545b0 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2bfff700 1 --2- 172.21.15.92:0/3093679629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c1545b0 0x7f7c2c064a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f7c1c009e90 tx=0x7f7c1c009780).ready entity=mon.2 client_cookie=67f92d80105159e5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00e070 con 0x7f7c2c1545b0 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00b7a0 con 0x7f7c2c1545b0 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/3093679629 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24001470 con 0x7f7c2c1545b0 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 -- 172.21.15.92:0/3093679629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c1545b0 msgr2=0x7f7c2c064a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/3093679629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c1545b0 0x7f7c2c064a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f7c1c009e90 tx=0x7f7c1c009780).stop 2023-05-16T07:36:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 -- 172.21.15.92:0/3093679629 shutdown_connections 2023-05-16T07:36:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/3093679629 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c1545b0 0x7f7c2c064a20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 -- 172.21.15.92:0/3093679629 >> 172.21.15.92:0/3093679629 conn(0x7f7c2c0cb600 msgr2=0x7f7c2c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 -- 172.21.15.92:0/3093679629 shutdown_connections 2023-05-16T07:36:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.719+0000 7f7c31b2c700 1 -- 172.21.15.92:0/3093679629 wait complete. 2023-05-16T07:36:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 Processor -- start 2023-05-16T07:36:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 -- start start 2023-05-16T07:36:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 0x7f7c2c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c0c1460 0x7f7c2c0c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c2c1545b0 0x7f7c2c151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0aec00 con 0x7f7c2c064f10 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0ae080 con 0x7f7c2c1545b0 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.721+0000 7f7c31b2c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0ae360 con 0x7f7c2c0c1460 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 0x7f7c2c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 0x7f7c2c0adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51758/0 (socket says 172.21.15.92:51758) 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 -- 172.21.15.92:0/1083783521 learned_addr learned my addr 172.21.15.92:0/1083783521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c2bfff700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c0c1460 0x7f7c2c0c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c3132b700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c2c1545b0 0x7f7c2c151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 -- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c0c1460 msgr2=0x7f7c2c0c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c0c1460 0x7f7c2c0c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 -- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c2c1545b0 msgr2=0x7f7c2c151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c2c1545b0 0x7f7c2c151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 -- 172.21.15.92:0/1083783521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c009a20 con 0x7f7c2c064f10 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c30b2a700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 0x7f7c2c0adae0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7f7c1400c4a0 tx=0x7f7c14007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400ce50 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1400f470 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c0ae640 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400f810 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.722+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c0c49a0 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.723+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c0c1c60 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.723+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f7c14019030 con 0x7f7c2c064f10 2023-05-16T07:36:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.724+0000 7f7c28ff9700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7c0c05ed90 0x7f7c0c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:39.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.724+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f7c140791f0 con 0x7f7c2c064f10 2023-05-16T07:36:39.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.724+0000 7f7c2bfff700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7c0c05ed90 0x7f7c0c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:39.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.724+0000 7f7c2bfff700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7c0c05ed90 0x7f7c0c061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7c2c154990 tx=0x7f7c1c009780).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:39.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.728+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7c2c0c1c60 con 0x7f7c2c064f10 2023-05-16T07:36:39.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:36:39.934 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:36:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.933+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7c2c05efa0 con 0x7f7c2c064f10 2023-05-16T07:36:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.933+0000 7f7c28ff9700 1 -- 172.21.15.92:0/1083783521 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7c1404a4b0 con 0x7f7c2c064f10 2023-05-16T07:36:39.937 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:36:39.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7c0c05ed90 msgr2=0x7f7c0c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7c0c05ed90 0x7f7c0c061250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7c2c154990 tx=0x7f7c1c009780).stop 2023-05-16T07:36:39.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 msgr2=0x7f7c2c0adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 0x7f7c2c0adae0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7f7c1400c4a0 tx=0x7f7c14007c50).stop 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 shutdown_connections 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7c2c1545b0 0x7f7c2c151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c064f10 0x7f7c2c0adae0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7c0c05ed90 0x7f7c0c061250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 --2- 172.21.15.92:0/1083783521 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7c2c0c1460 0x7f7c2c0c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:39.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 >> 172.21.15.92:0/1083783521 conn(0x7f7c2c0cb600 msgr2=0x7f7c2c0c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:39.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 shutdown_connections 2023-05-16T07:36:39.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:39.935+0000 7f7c31b2c700 1 -- 172.21.15.92:0/1083783521 wait complete. 2023-05-16T07:36:39.949 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:36:39.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:36:39.950 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:36:39.950 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:36:39.950 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:36:39.950 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:36:39.951 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:36:39.951 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:36:39.951 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:36:39.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:39.952 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:36:40.044 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:36:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:36:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:36:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:36:40.045 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 39955sec preferred_lft 39955sec 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:36:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:36:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.048 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:36:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:36:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:36:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:36:40.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.049 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:36:40.106 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:36:40.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.108 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:36:40.164 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:36:40.164 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:36:40.164 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:36:40.164 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:36:40.164 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:36:40.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.165 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:36:40.165 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:36:40.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:36:40.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:36:40.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:36:40.167 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:36:40.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.261 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:36:40.275 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:36:40.276 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:36:40.276 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:36:40.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:36:40.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.339 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:36:40.392 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:36:40.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:36:40.470 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:36:40.470 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:36:40.513 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:36:40.513 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:36:40.515 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:36:40.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:36:40.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:36:40.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:36:40.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.733+0000 7fa36ce5b700 1 Processor -- start 2023-05-16T07:36:40.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.733+0000 7fa36ce5b700 1 -- start start 2023-05-16T07:36:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36ce5b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa368150b50 0x7fa368150f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36ce5b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa368068f10 0x7fa3680a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36ce5b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3680bc8f0 0x7fa3680bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36ce5b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3680bd270 con 0x7fa3680bc8f0 2023-05-16T07:36:40.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36ce5b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa368069410 con 0x7fa368150b50 2023-05-16T07:36:40.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36ce5b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680696f0 con 0x7fa368068f10 2023-05-16T07:36:40.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa366d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3680bc8f0 0x7fa3680bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa366d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3680bc8f0 0x7fa3680bcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51770/0 (socket says 172.21.15.92:51770) 2023-05-16T07:36:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa366d9d700 1 -- 172.21.15.92:0/3570470240 learned_addr learned my addr 172.21.15.92:0/3570470240 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa365d9b700 1 --2- 172.21.15.92:0/3570470240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa368068f10 0x7fa3680a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.734+0000 7fa36659c700 1 --2- 172.21.15.92:0/3570470240 >> v2:172.21.15.158:3300/0 conn(0x7fa368150b50 0x7fa368150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa366d9d700 1 -- 172.21.15.92:0/3570470240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa368068f10 msgr2=0x7fa3680a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa366d9d700 1 --2- 172.21.15.92:0/3570470240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa368068f10 0x7fa3680a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa366d9d700 1 -- 172.21.15.92:0/3570470240 >> v2:172.21.15.158:3300/0 conn(0x7fa368150b50 msgr2=0x7fa368150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa366d9d700 1 --2- 172.21.15.92:0/3570470240 >> v2:172.21.15.158:3300/0 conn(0x7fa368150b50 0x7fa368150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa366d9d700 1 -- 172.21.15.92:0/3570470240 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680699d0 con 0x7fa3680bc8f0 2023-05-16T07:36:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa366d9d700 1 --2- 172.21.15.92:0/3570470240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3680bc8f0 0x7fa3680bcd80 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7fa3680c6790 tx=0x7fa36000ba50).ready entity=mon.0 client_cookie=2569993bd9f0675f server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa36559a700 1 -- 172.21.15.92:0/3570470240 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360010070 con 0x7fa3680bc8f0 2023-05-16T07:36:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa36559a700 1 -- 172.21.15.92:0/3570470240 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000de90 con 0x7fa3680bc8f0 2023-05-16T07:36:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa36559a700 1 -- 172.21.15.92:0/3570470240 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000b2e0 con 0x7fa3680bc8f0 2023-05-16T07:36:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa36ce5b700 1 -- 172.21.15.92:0/3570470240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3680bc8f0 msgr2=0x7fa3680bcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.735+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/3570470240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3680bc8f0 0x7fa3680bcd80 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7fa3680c6790 tx=0x7fa36000ba50).stop 2023-05-16T07:36:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 -- 172.21.15.92:0/3570470240 shutdown_connections 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/3570470240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3680bc8f0 0x7fa3680bcd80 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/3570470240 >> v2:172.21.15.158:3300/0 conn(0x7fa368150b50 0x7fa368150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/3570470240 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa368068f10 0x7fa3680a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 -- 172.21.15.92:0/3570470240 >> 172.21.15.92:0/3570470240 conn(0x7fa3680c7350 msgr2=0x7fa3680c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 -- 172.21.15.92:0/3570470240 shutdown_connections 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.736+0000 7fa36ce5b700 1 -- 172.21.15.92:0/3570470240 wait complete. 2023-05-16T07:36:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 Processor -- start 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 -- start start 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa368068f10 0x7fa3680ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa3680bc8f0 0x7fa3680bf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 0x7fa3680c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680aaff0 con 0x7fa368150b50 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680aa730 con 0x7fa368068f10 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.737+0000 7fa36ce5b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680ab970 con 0x7fa3680bc8f0 2023-05-16T07:36:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 0x7fa3680c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 0x7fa3680c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51776/0 (socket says 172.21.15.92:51776) 2023-05-16T07:36:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 -- 172.21.15.92:0/1656395094 learned_addr learned my addr 172.21.15.92:0/1656395094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa36659c700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa368068f10 0x7fa3680ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa365d9b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa3680bc8f0 0x7fa3680bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 -- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa3680bc8f0 msgr2=0x7fa3680bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa3680bc8f0 0x7fa3680bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 -- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa368068f10 msgr2=0x7fa3680ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa368068f10 0x7fa3680ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 -- 172.21.15.92:0/1656395094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36000bcf0 con 0x7fa368150b50 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa366d9d700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 0x7fa3680c4690 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7fa36000d8c0 tx=0x7fa36000ba50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000b220 con 0x7fa368150b50 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa360015420 con 0x7fa368150b50 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3680aa9a0 con 0x7fa368150b50 2023-05-16T07:36:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600157c0 con 0x7fa368150b50 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.738+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36814fcb0 con 0x7fa368150b50 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.739+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3680ab1a0 con 0x7fa368150b50 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.740+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa36001d040 con 0x7fa368150b50 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.740+0000 7fa3567fc700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa34805ed90 0x7fa348061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.740+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fa36007da70 con 0x7fa368150b50 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.740+0000 7fa36659c700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa34805ed90 0x7fa348061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.741+0000 7fa36659c700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa34805ed90 0x7fa348061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa35c00bb10 tx=0x7fa35c008040).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.744+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa3680ab1a0 con 0x7fa368150b50 2023-05-16T07:36:40.894 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:36:40.891+0000 7f45857c0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:36:40.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.942+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3680ab1a0 con 0x7fa34805ed90 2023-05-16T07:36:40.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.944+0000 7fa3567fc700 1 -- 172.21.15.92:0/1656395094 <== mgr.7591 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa3680ab1a0 con 0x7fa34805ed90 2023-05-16T07:36:40.947 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:36:40.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:36:40.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:36:40.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:36:40.948 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:36:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa34805ed90 msgr2=0x7fa348061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa34805ed90 0x7fa348061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa35c00bb10 tx=0x7fa35c008040).stop 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 msgr2=0x7fa3680c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 0x7fa3680c4690 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7fa36000d8c0 tx=0x7fa36000ba50).stop 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 shutdown_connections 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa368150b50 0x7fa3680c4690 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa368068f10 0x7fa3680ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa34805ed90 0x7fa348061250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 --2- 172.21.15.92:0/1656395094 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa3680bc8f0 0x7fa3680bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 >> 172.21.15.92:0/1656395094 conn(0x7fa3680c7350 msgr2=0x7fa3680c1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 shutdown_connections 2023-05-16T07:36:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:40.946+0000 7fa36ce5b700 1 -- 172.21.15.92:0/1656395094 wait complete. 2023-05-16T07:36:40.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:36:41.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.075+0000 7f1b0c487700 1 Processor -- start 2023-05-16T07:36:41.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.075+0000 7f1b0c487700 1 -- start start 2023-05-16T07:36:41.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0c487700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b04063fd0 0x7f1b041540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:41.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0c487700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b040622b0 0x7f1b04062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:41.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0c487700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b040bf950 con 0x7f1b04153d00 2023-05-16T07:36:41.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0c487700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b04067f60 con 0x7f1b04063fd0 2023-05-16T07:36:41.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0c487700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04062c30 con 0x7f1b040622b0 2023-05-16T07:36:41.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b09a22700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b04063fd0 0x7f1b041540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:41.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0a223700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b04153d00 0x7f1b040bf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34298/0 (socket says 172.21.15.92:34298) 2023-05-16T07:36:41.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0a223700 1 -- 172.21.15.92:0/1599839462 learned_addr learned my addr 172.21.15.92:0/1599839462 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:41.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.076+0000 7f1b0aa24700 1 --2- 172.21.15.92:0/1599839462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b040622b0 0x7f1b04062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:41.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09a22700 1 -- 172.21.15.92:0/1599839462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b040622b0 msgr2=0x7f1b04062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09a22700 1 --2- 172.21.15.92:0/1599839462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b040622b0 0x7f1b04062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09a22700 1 -- 172.21.15.92:0/1599839462 >> v1:172.21.15.92:6789/0 conn(0x7f1b04153d00 legacy=0x7f1b040bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09a22700 1 -- 172.21.15.92:0/1599839462 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040a7c90 con 0x7f1b04063fd0 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09221700 1 -- 172.21.15.92:0/1599839462 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252424892 0 0) 0x7f1b040bf950 con 0x7f1b04153d00 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09a22700 1 --2- 172.21.15.92:0/1599839462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b04063fd0 0x7f1b041540e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f1b00009760 tx=0x7f1b00009060).ready entity=mon.0 client_cookie=beed47d71c5cdbd4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09221700 1 -- 172.21.15.92:0/1599839462 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000b580 con 0x7f1b04063fd0 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09221700 1 -- 172.21.15.92:0/1599839462 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000b6e0 con 0x7f1b04063fd0 2023-05-16T07:36:41.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b09221700 1 -- 172.21.15.92:0/1599839462 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000ba60 con 0x7f1b04063fd0 2023-05-16T07:36:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b0c487700 1 -- 172.21.15.92:0/1599839462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b04063fd0 msgr2=0x7f1b041540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.077+0000 7f1b0c487700 1 --2- 172.21.15.92:0/1599839462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b04063fd0 0x7f1b041540e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f1b00009760 tx=0x7f1b00009060).stop 2023-05-16T07:36:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.078+0000 7f1b0c487700 1 -- 172.21.15.92:0/1599839462 shutdown_connections 2023-05-16T07:36:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.078+0000 7f1b0c487700 1 --2- 172.21.15.92:0/1599839462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b040622b0 0x7f1b04062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.078+0000 7f1b0c487700 1 --2- 172.21.15.92:0/1599839462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b04063fd0 0x7f1b041540e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.078+0000 7f1b0c487700 1 -- 172.21.15.92:0/1599839462 >> 172.21.15.92:0/1599839462 conn(0x7f1b0405e820 msgr2=0x7f1b040c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.078+0000 7f1b0c487700 1 -- 172.21.15.92:0/1599839462 shutdown_connections 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.078+0000 7f1b0c487700 1 -- 172.21.15.92:0/1599839462 wait complete. 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.079+0000 7f1b0c487700 1 Processor -- start 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.079+0000 7f1b0c487700 1 -- start start 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.079+0000 7f1b0c487700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 0x7f1b0414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b04063fd0 0x7f1b040a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b04153d00 0x7f1b040a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040add60 con 0x7f1b040622b0 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c27a0 con 0x7f1b04153d00 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040ae000 con 0x7f1b04063fd0 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 0x7f1b0414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 0x7f1b0414def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51786/0 (socket says 172.21.15.92:51786) 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 -- 172.21.15.92:0/3411890662 learned_addr learned my addr 172.21.15.92:0/3411890662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0aa24700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b04153d00 0x7f1b040a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 -- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b04063fd0 msgr2=0x7f1b040a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b09a22700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b04063fd0 0x7f1b040a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b04063fd0 0x7f1b040a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 -- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b04153d00 msgr2=0x7f1b040a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b04153d00 0x7f1b040a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 -- 172.21.15.92:0/3411890662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000092a0 con 0x7f1b040622b0 2023-05-16T07:36:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0aa24700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b04153d00 0x7f1b040a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0a223700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 0x7f1b0414def0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f1af4001eb0 tx=0x7f1af4007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40072e0 con 0x7f1b040622b0 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4007440 con 0x7f1b040622b0 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40057a0 con 0x7f1b040622b0 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b040ae2e0 con 0x7f1b040622b0 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.080+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0414e3e0 con 0x7f1b040622b0 2023-05-16T07:36:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.081+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0406c350 con 0x7f1b040622b0 2023-05-16T07:36:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.082+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 70) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1af4010070 con 0x7f1b040622b0 2023-05-16T07:36:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.082+0000 7f1afa7fc700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1ae805ed90 0x7f1ae8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.082+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f1af4079470 con 0x7f1b040622b0 2023-05-16T07:36:41.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.082+0000 7f1b09a22700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1ae805ed90 0x7f1ae8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:41.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.083+0000 7f1b09a22700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1ae805ed90 0x7f1ae8061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1b0000b340 tx=0x7f1b00017000).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:41.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.086+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1af404a790 con 0x7f1b040622b0 2023-05-16T07:36:41.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.299+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1b040a7f00 con 0x7f1b040622b0 2023-05-16T07:36:41.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.994+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 71) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f1af40770f0 con 0x7f1b040622b0 2023-05-16T07:36:41.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.995+0000 7f1afa7fc700 1 -- 172.21.15.92:0/3411890662 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v71) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1af40767b0 con 0x7f1b040622b0 2023-05-16T07:36:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1ae805ed90 msgr2=0x7f1ae8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1ae805ed90 0x7f1ae8061250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1b0000b340 tx=0x7f1b00017000).stop 2023-05-16T07:36:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 msgr2=0x7f1b0414def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 0x7f1b0414def0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f1af4001eb0 tx=0x7f1af4007e90).stop 2023-05-16T07:36:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 shutdown_connections 2023-05-16T07:36:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1b04153d00 0x7f1b040a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b040622b0 0x7f1b0414def0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1ae805ed90 0x7f1ae8061250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 --2- 172.21.15.92:0/3411890662 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1b04063fd0 0x7f1b040a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 >> 172.21.15.92:0/3411890662 conn(0x7f1b0405e820 msgr2=0x7f1b040c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 shutdown_connections 2023-05-16T07:36:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:41.998+0000 7f1b0c487700 1 -- 172.21.15.92:0/3411890662 wait complete. 2023-05-16T07:36:42.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:36:42.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.128+0000 7f600f215700 1 Processor -- start 2023-05-16T07:36:42.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.128+0000 7f600f215700 1 -- start start 2023-05-16T07:36:42.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.129+0000 7f600f215700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60080acd80 0x7f60080ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.129+0000 7f600f215700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6008069a40 0x7f6008062980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.129+0000 7f600f215700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6008153250 con 0x7f60080a97c0 2023-05-16T07:36:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.129+0000 7f600f215700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6008062e70 con 0x7f60080acd80 2023-05-16T07:36:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.129+0000 7f600f215700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008069ec0 con 0x7f6008069a40 2023-05-16T07:36:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60080acd80 0x7f60080ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60080acd80 0x7f60080ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51820/0 (socket says 172.21.15.92:51820) 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 -- 172.21.15.92:0/3212364585 learned_addr learned my addr 172.21.15.92:0/3212364585 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600ea14700 1 --1- 172.21.15.92:0/3212364585 >> v1:172.21.15.92:6789/0 conn(0x7f60080a97c0 0x7f6008155bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34340/0 (socket says 172.21.15.92:34340) 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600da12700 1 --2- 172.21.15.92:0/3212364585 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6008069a40 0x7f6008062980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 -- 172.21.15.92:0/3212364585 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6008069a40 msgr2=0x7f6008062980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 --2- 172.21.15.92:0/3212364585 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6008069a40 0x7f6008062980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 -- 172.21.15.92:0/3212364585 >> v1:172.21.15.92:6789/0 conn(0x7f60080a97c0 legacy=0x7f6008155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 -- 172.21.15.92:0/3212364585 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60081526d0 con 0x7f60080acd80 2023-05-16T07:36:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600d211700 1 -- 172.21.15.92:0/3212364585 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876311592 0 0) 0x7f6008153250 con 0x7f60080a97c0 2023-05-16T07:36:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600e213700 1 --2- 172.21.15.92:0/3212364585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080acd80 0x7f60080ad160 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f6004006fd0 tx=0x7f60040090f0).ready entity=mon.0 client_cookie=81dbd3fd87085086 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600d211700 1 -- 172.21.15.92:0/3212364585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400b580 con 0x7f60080acd80 2023-05-16T07:36:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600d211700 1 -- 172.21.15.92:0/3212364585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400b6e0 con 0x7f60080acd80 2023-05-16T07:36:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.130+0000 7f600d211700 1 -- 172.21.15.92:0/3212364585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400ba60 con 0x7f60080acd80 2023-05-16T07:36:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 -- 172.21.15.92:0/3212364585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080acd80 msgr2=0x7f60080ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 --2- 172.21.15.92:0/3212364585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080acd80 0x7f60080ad160 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f6004006fd0 tx=0x7f60040090f0).stop 2023-05-16T07:36:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 -- 172.21.15.92:0/3212364585 shutdown_connections 2023-05-16T07:36:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 --2- 172.21.15.92:0/3212364585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080acd80 0x7f60080ad160 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 --2- 172.21.15.92:0/3212364585 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6008069a40 0x7f6008062980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 -- 172.21.15.92:0/3212364585 >> 172.21.15.92:0/3212364585 conn(0x7f6008068a30 msgr2=0x7f60080c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 -- 172.21.15.92:0/3212364585 shutdown_connections 2023-05-16T07:36:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.131+0000 7f600f215700 1 -- 172.21.15.92:0/3212364585 wait complete. 2023-05-16T07:36:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.132+0000 7f600f215700 1 Processor -- start 2023-05-16T07:36:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.132+0000 7f600f215700 1 -- start start 2023-05-16T07:36:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600f215700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6008069a40 0x7f60080a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600f215700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 0x7f6008147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600f215700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f60080acd80 0x7f60080a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600f215700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080a1e50 con 0x7f60080a97c0 2023-05-16T07:36:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600f215700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080a1240 con 0x7f6008069a40 2023-05-16T07:36:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600f215700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60080a1520 con 0x7f60080acd80 2023-05-16T07:36:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 0x7f6008147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 0x7f6008147aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51828/0 (socket says 172.21.15.92:51828) 2023-05-16T07:36:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 -- 172.21.15.92:0/2650892286 learned_addr learned my addr 172.21.15.92:0/2650892286 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600e213700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6008069a40 0x7f60080a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600ea14700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f60080acd80 0x7f60080a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 -- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f60080acd80 msgr2=0x7f60080a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f60080acd80 0x7f60080a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 -- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6008069a40 msgr2=0x7f60080a22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6008069a40 0x7f60080a22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.133+0000 7f600da12700 1 -- 172.21.15.92:0/2650892286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004009370 con 0x7f60080a97c0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.134+0000 7f600da12700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 0x7f6008147aa0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f5ff800e950 tx=0x7f5ff8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.134+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8012600 con 0x7f60080a97c0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.134+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff8012760 con 0x7f60080a97c0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.134+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60080a1800 con 0x7f60080a97c0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.134+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8012ae0 con 0x7f60080a97c0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.134+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008147f90 con 0x7f60080a97c0 2023-05-16T07:36:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.135+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60080a2000 con 0x7f60080a97c0 2023-05-16T07:36:42.144 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:42.138+0000 7f8f9ace5680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:36:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.136+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 71) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f5ff8022070 con 0x7f60080a97c0 2023-05-16T07:36:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.136+0000 7f5ffe7fc700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fec05ee10 0x7f5fec0612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.136+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f5ff807a160 con 0x7f60080a97c0 2023-05-16T07:36:42.145 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:42.136+0000 7f5e660bc680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:36:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.136+0000 7f600e213700 1 -- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fec05ee10 msgr2=0x7f5fec0612d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.136+0000 7f600e213700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fec05ee10 0x7f5fec0612d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.140+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5ff804b480 con 0x7f60080a97c0 2023-05-16T07:36:42.204 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:42.201+0000 7f8f9ace5680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:36:42.204 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:42.201+0000 7f5e660bc680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:36:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.311+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6008047150 con 0x7f60080a97c0 2023-05-16T07:36:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.315+0000 7f5ffe7fc700 1 -- 172.21.15.92:0/2650892286 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5ff80160d0 con 0x7f60080a97c0 2023-05-16T07:36:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fec05ee10 msgr2=0x7f5fec0612d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fec05ee10 0x7f5fec0612d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 msgr2=0x7f6008147aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 0x7f6008147aa0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f5ff800e950 tx=0x7f5ff8009060).stop 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 shutdown_connections 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5fec05ee10 0x7f5fec0612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f60080acd80 0x7f60080a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6008069a40 0x7f60080a22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 --2- 172.21.15.92:0/2650892286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60080a97c0 0x7f6008147aa0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 >> 172.21.15.92:0/2650892286 conn(0x7f6008068a30 msgr2=0x7f6008155bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:42.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 shutdown_connections 2023-05-16T07:36:42.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.317+0000 7f600f215700 1 -- 172.21.15.92:0/2650892286 wait complete. 2023-05-16T07:36:42.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:42.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:36:42.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-05-16T07:36:42.476 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:42.473+0000 7f8f9ace5680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:36:42.487 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:42.484+0000 7f5e660bc680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:36:42.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.509+0000 7f703b59e700 1 Processor -- start 2023-05-16T07:36:42.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.509+0000 7f703b59e700 1 -- start start 2023-05-16T07:36:42.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.510+0000 7f703b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c0a75b0 0x7f703c150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.510+0000 7f703b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c151520 con 0x7f703c062980 2023-05-16T07:36:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.510+0000 7f703b59e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c1511f0 con 0x7f703c150920 2023-05-16T07:36:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.510+0000 7f703b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c1520c0 con 0x7f703c0a75b0 2023-05-16T07:36:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f703c062980 0x7f703c154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34380/0 (socket says 172.21.15.92:34380) 2023-05-16T07:36:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703ad9d700 1 -- 172.21.15.92:0/482204721 learned_addr learned my addr 172.21.15.92:0/482204721 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f7039d9b700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c0a75b0 0x7f703c150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917255389 0 0) 0x7f703c151520 con 0x7f703c062980 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70280033f0 con 0x7f703c062980 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559675567 0 0) 0x7f703c1511f0 con 0x7f703c150920 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f703c151520 con 0x7f703c150920 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3906869716 0 0) 0x7f70280033f0 con 0x7f703c062980 2023-05-16T07:36:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c0a75b0 msgr2=0x7f703c150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c0a75b0 0x7f703c150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 >> v1:172.21.15.158:6789/0 conn(0x7f703c150920 legacy=0x7f703c1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0a7a30 con 0x7f703c062980 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f702c001470 con 0x7f703c062980 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 >> v1:172.21.15.92:6789/0 conn(0x7f703c062980 legacy=0x7f703c154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7028003aa0 0x7f7028003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028004690 0x7f7028004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7028005000 0x7f702800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c1520c0 con 0x7f7028003aa0 2023-05-16T07:36:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702800be80 con 0x7f7028004690 2023-05-16T07:36:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702800c040 con 0x7f7028005000 2023-05-16T07:36:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f7039d9b700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c0a75b0 0x7f703c150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703a59c700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7028003aa0 0x7f7028003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f703ad9d700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7028005000 0x7f702800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.511+0000 7f7039d9b700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028004690 0x7f7028004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703a59c700 1 -- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7028005000 msgr2=0x7f702800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703a59c700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7028005000 0x7f702800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703a59c700 1 -- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028004690 msgr2=0x7f7028004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703a59c700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028004690 0x7f7028004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703a59c700 1 -- 172.21.15.92:0/482204721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70280033f0 con 0x7f7028003aa0 2023-05-16T07:36:42.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703a59c700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7028003aa0 0x7f7028003ea0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f70340018d0 tx=0x7f7034007b90).ready entity=mon.0 client_cookie=4c0740fab05a4c7e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034007ea0 con 0x7f7028003aa0 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703400c420 con 0x7f7028003aa0 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703959a700 1 -- 172.21.15.92:0/482204721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400c7a0 con 0x7f7028003aa0 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703b59e700 1 -- 172.21.15.92:0/482204721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7028003aa0 msgr2=0x7f7028003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703b59e700 1 --2- 172.21.15.92:0/482204721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7028003aa0 0x7f7028003ea0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f70340018d0 tx=0x7f7034007b90).stop 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703ad9d700 1 -- 172.21.15.92:0/482204721 reap_dead start 2023-05-16T07:36:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703b59e700 1 -- 172.21.15.92:0/482204721 shutdown_connections 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703b59e700 1 -- 172.21.15.92:0/482204721 >> 172.21.15.92:0/482204721 conn(0x7f703c068a30 msgr2=0x7f703c0c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.512+0000 7f703b59e700 1 -- 172.21.15.92:0/482204721 shutdown_connections 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.513+0000 7f703b59e700 1 -- 172.21.15.92:0/482204721 wait complete. 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 Processor -- start 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 -- start start 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028003aa0 0x7f703c14f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c13bbc0 0x7f703c0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 0x7f703c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0ab330 con 0x7f703c13c020 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c14ebf0 con 0x7f7028003aa0 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703b59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0ad8a0 con 0x7f703c13bbc0 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 0x7f703c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703a59c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028003aa0 0x7f703c14f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 0x7f703c0aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51876/0 (socket says 172.21.15.92:51876) 2023-05-16T07:36:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.514+0000 7f703ad9d700 1 -- 172.21.15.92:0/3763039717 learned_addr learned my addr 172.21.15.92:0/3763039717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f7039d9b700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c13bbc0 0x7f703c0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703ad9d700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c13bbc0 msgr2=0x7f703c0aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703ad9d700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c13bbc0 0x7f703c0aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703ad9d700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028003aa0 msgr2=0x7f703c14f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703ad9d700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028003aa0 0x7f703c14f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703ad9d700 1 -- 172.21.15.92:0/3763039717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c14e2c0 con 0x7f703c13c020 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028003aa0 0x7f703c14f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703ad9d700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 0x7f703c0aae40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f703c06bee0 tx=0x7f702c001df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011590 con 0x7f703c13c020 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c011710 con 0x7f703c13c020 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c14e5a0 con 0x7f703c13c020 2023-05-16T07:36:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011ad0 con 0x7f703c13c020 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.515+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c0acb50 con 0x7f703c13c020 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.516+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c047150 con 0x7f703c13c020 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.516+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 71) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f702c01d060 con 0x7f703c13c020 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.517+0000 7f70327fc700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.517+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f702c079790 con 0x7f703c13c020 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.517+0000 7f703a59c700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:42.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.517+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:36:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.521+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f702c04ab30 con 0x7f703c13c020 2023-05-16T07:36:42.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.715+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f703c0a8480 con 0x7f701c05ede0 2023-05-16T07:36:42.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.718+0000 7f703a59c700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:42.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:42.718+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:36:43.091 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:43.086+0000 7f8f9ace5680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:36:43.103 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:43.100+0000 7f5e660bc680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:36:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:43.119+0000 7f703a59c700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:43.119+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:36:43.161 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:43.158+0000 7f8f9ace5680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:36:43.174 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:43.171+0000 7f5e660bc680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:36:43.345 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:43.342+0000 7f8f9ace5680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:36:43.355 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:43.352+0000 7f5e660bc680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:36:43.761 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:43.759+0000 7f8f9ace5680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:36:43.764 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:43.761+0000 7f5e660bc680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:36:43.863 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:43.860+0000 7f8f9ace5680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:43.865 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:43.862+0000 7f5e660bc680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:43.920+0000 7f703a59c700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:43.920+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:36:44.016 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:44.013+0000 7f8f9ace5680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:44.017 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:44.013+0000 7f5e660bc680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:44.089 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:44.086+0000 7f5e660bc680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:36:44.090 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:44.087+0000 7f8f9ace5680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:36:44.155 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:44.151+0000 7f5e660bc680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:36:44.157 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:44.154+0000 7f8f9ace5680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:36:44.406 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:44.403+0000 7f5e660bc680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:36:44.411 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:44.408+0000 7f8f9ace5680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:36:44.587 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:44.584+0000 7f5e660bc680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:44.594 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:44.591+0000 7f8f9ace5680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:36:45.222 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.219+0000 7f5e660bc680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:36:45.241 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.238+0000 7f8f9ace5680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:36:45.289 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.286+0000 7f5e660bc680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:36:45.308 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.305+0000 7f8f9ace5680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:36:45.361 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.358+0000 7f5e660bc680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:36:45.382 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.379+0000 7f8f9ace5680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:36:45.506 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.503+0000 7f5e660bc680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:36:45.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:45.522+0000 7f703a59c700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:36:45.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:45.522+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:36:45.527 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.524+0000 7f8f9ace5680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:36:45.576 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.573+0000 7f5e660bc680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:36:45.596 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.593+0000 7f8f9ace5680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:36:45.689 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.686+0000 7f5e660bc680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:36:45.711 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.708+0000 7f8f9ace5680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:36:45.794 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:45.791+0000 7f5e660bc680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:45.817 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:45.813+0000 7f8f9ace5680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:36:46.079 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:46.076+0000 7f5e660bc680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:36:46.107 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.102+0000 7f8f9ace5680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:36:46.148 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:36:46.144+0000 7f5e660bc680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:36:46.176 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.172+0000 7f8f9ace5680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:36:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:46.213+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 72) v1 ==== 75109+0+0 (secure 0 0 0) 0x7f702c077490 con 0x7f703c13c020 2023-05-16T07:36:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:46.214+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:36:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:46.214+0000 7f70327fc700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:36:46.273 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f164c1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.273 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f164c1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.274 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f164c1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.274 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f164c1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.274 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f164c1700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.275 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f194c7700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.275 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f194c7700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.275 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f194c7700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.275 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f194c7700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:46.276 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:36:46.268+0000 7f8f194c7700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:36:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.228+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 73) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f702c077810 con 0x7f703c13c020 2023-05-16T07:36:47.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.228+0000 7f70327fc700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c02fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.228+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f703c0a8480 con 0x7f701c05ede0 2023-05-16T07:36:47.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.229+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c02fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.231+0000 7f703a59c700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c02fd40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f7034007ca0 tx=0x7f7034017000).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.243+0000 7f70327fc700 1 -- 172.21.15.92:0/3763039717 <== mgr.7127 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f703c0a8480 con 0x7f701c05ede0 2023-05-16T07:36:47.247 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path snap_test_dir1 2023-05-16T07:36:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 msgr2=0x7f701c02fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c02fd40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f7034007ca0 tx=0x7f7034017000).stop 2023-05-16T07:36:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 msgr2=0x7f703c0aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 0x7f703c0aae40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f703c06bee0 tx=0x7f702c001df0).stop 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 shutdown_connections 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c13c020 0x7f703c0aae40 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7028003aa0 0x7f703c14f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f701c05ede0 0x7f701c02fd40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 --2- 172.21.15.92:0/3763039717 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f703c13bbc0 0x7f703c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 >> 172.21.15.92:0/3763039717 conn(0x7f703c068a30 msgr2=0x7f703c065690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 shutdown_connections 2023-05-16T07:36:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.246+0000 7f703b59e700 1 -- 172.21.15.92:0/3763039717 wait complete. 2023-05-16T07:36:47.263 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-05-16T07:36:47.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-05-16T07:36:47.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.381+0000 7f01968a8700 1 Processor -- start 2023-05-16T07:36:47.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f01968a8700 1 -- start start 2023-05-16T07:36:47.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f01968a8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0190154470 0x7f01900aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f01968a8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 0x7f01900685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f01968a8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01900c94d0 con 0x7f0190154090 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f01968a8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01900cb7e0 con 0x7f0190154470 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f01968a8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0190068aa0 con 0x7f0190068160 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.382+0000 7f018ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0190154090 0x7f01900a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34508/0 (socket says 172.21.15.92:34508) 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018ffff700 1 -- 172.21.15.92:0/3096662199 learned_addr learned my addr 172.21.15.92:0/3096662199 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:36:47.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018f7fe700 1 --2- 172.21.15.92:0/3096662199 >> v2:172.21.15.158:3300/0 conn(0x7f0190154470 0x7f01900aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 0x7f01900685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001294550 0 0) 0x7f01900c94d0 con 0x7f0190154090 2023-05-16T07:36:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01780033f0 con 0x7f0190154090 2023-05-16T07:36:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4098927090 0 0) 0x7f01780033f0 con 0x7f0190154090 2023-05-16T07:36:47.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 msgr2=0x7f01900685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 0x7f01900685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 >> v2:172.21.15.158:3300/0 conn(0x7f0190154470 msgr2=0x7f01900aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 --2- 172.21.15.92:0/3096662199 >> v2:172.21.15.158:3300/0 conn(0x7f0190154470 0x7f01900aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900c88c0 con 0x7f0190154090 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f0180001470 con 0x7f0190154090 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 >> v1:172.21.15.92:6789/0 conn(0x7f0190154090 legacy=0x7f01900a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 0x7f01780038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0178003e10 0x7f0178004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178004b80 0x7f0178005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900cb7e0 con 0x7f0178004b80 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190068aa0 con 0x7f0178003e10 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017800b8e0 con 0x7f0190068160 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178004b80 0x7f0178005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018ffff700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 0x7f01780038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f018f7fe700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0178003e10 0x7f0178004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 -- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 msgr2=0x7f01780038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0190068160 0x7f01780038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 -- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0178003e10 msgr2=0x7f0178004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0178003e10 0x7f0178004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.383+0000 7f0194e45700 1 -- 172.21.15.92:0/3096662199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780033f0 con 0x7f0178004b80 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f0194e45700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178004b80 0x7f0178005000 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f018800e1f0 tx=0x7f018800b040).ready entity=mon.0 client_cookie=57c81876c45d580a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188009ac0 con 0x7f0178004b80 2023-05-16T07:36:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018800f040 con 0x7f0178004b80 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f018effd700 1 -- 172.21.15.92:0/3096662199 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800ecd0 con 0x7f0178004b80 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f01968a8700 1 -- 172.21.15.92:0/3096662199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178004b80 msgr2=0x7f0178005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f01968a8700 1 --2- 172.21.15.92:0/3096662199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178004b80 0x7f0178005000 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f018800e1f0 tx=0x7f018800b040).stop 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f0194e45700 1 -- 172.21.15.92:0/3096662199 reap_dead start 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f01968a8700 1 -- 172.21.15.92:0/3096662199 shutdown_connections 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f01968a8700 1 -- 172.21.15.92:0/3096662199 >> 172.21.15.92:0/3096662199 conn(0x7f0190067920 msgr2=0x7f01900cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f01968a8700 1 -- 172.21.15.92:0/3096662199 shutdown_connections 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.384+0000 7f01968a8700 1 -- 172.21.15.92:0/3096662199 wait complete. 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.385+0000 7f01968a8700 1 Processor -- start 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.385+0000 7f01968a8700 1 -- start start 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f01968a8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0190150e60 0x7f0190155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f01968a8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01901564b0 0x7f0190152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f01968a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 0x7f01900c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f01968a8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01901536d0 con 0x7f0190153190 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f01968a8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c3700 con 0x7f0190150e60 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f01968a8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c2dd0 con 0x7f01901564b0 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f0194e45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 0x7f01900c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f0194e45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 0x7f01900c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52002/0 (socket says 172.21.15.92:52002) 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f0194e45700 1 -- 172.21.15.92:0/2728367818 learned_addr learned my addr 172.21.15.92:0/2728367818 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f018f7fe700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01901564b0 0x7f0190152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.386+0000 7f018ffff700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0190150e60 0x7f0190155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0194e45700 1 -- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01901564b0 msgr2=0x7f0190152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0194e45700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01901564b0 0x7f0190152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0194e45700 1 -- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0190150e60 msgr2=0x7f0190155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0194e45700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0190150e60 0x7f0190155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0194e45700 1 -- 172.21.15.92:0/2728367818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900c30b0 con 0x7f0190153190 2023-05-16T07:36:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f018f7fe700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01901564b0 0x7f0190152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:36:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0194e45700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 0x7f01900c3b90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f01780045a0 tx=0x7f018801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800eaa0 con 0x7f0190153190 2023-05-16T07:36:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018800ec00 con 0x7f0190153190 2023-05-16T07:36:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01900c3390 con 0x7f0190153190 2023-05-16T07:36:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018801c640 con 0x7f0190153190 2023-05-16T07:36:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.387+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01901507a0 con 0x7f0190153190 2023-05-16T07:36:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.388+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0190150990 con 0x7f0190153190 2023-05-16T07:36:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.388+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 73) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f018802c070 con 0x7f0190153190 2023-05-16T07:36:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.389+0000 7f0177fff700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f017c05ed40 0x7f017c061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.389+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f018807b6b0 con 0x7f0190153190 2023-05-16T07:36:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.389+0000 7f018ffff700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f017c05ed40 0x7f017c061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.389+0000 7f018ffff700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f017c05ed40 0x7f017c061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0180000e10 tx=0x7f0180009040).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.393+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0190150990 con 0x7f0190153190 2023-05-16T07:36:47.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.581+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0190150990 con 0x7f017c05ed40 2023-05-16T07:36:47.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.587+0000 7f0177fff700 1 -- 172.21.15.92:0/2728367818 <== mgr.7127 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f0190150990 con 0x7f017c05ed40 2023-05-16T07:36:47.590 INFO:teuthology.orchestra.run.smithi092.stdout:Retention added to path snap_test_dir1 2023-05-16T07:36:47.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f017c05ed40 msgr2=0x7f017c061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f017c05ed40 0x7f017c061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0180000e10 tx=0x7f0180009040).stop 2023-05-16T07:36:47.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 msgr2=0x7f01900c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 0x7f01900c3b90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f01780045a0 tx=0x7f018801a000).stop 2023-05-16T07:36:47.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 shutdown_connections 2023-05-16T07:36:47.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0190153190 0x7f01900c3b90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0190150e60 0x7f0190155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f017c05ed40 0x7f017c061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 --2- 172.21.15.92:0/2728367818 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f01901564b0 0x7f0190152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 >> 172.21.15.92:0/2728367818 conn(0x7f0190067920 msgr2=0x7f019014f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:47.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 shutdown_connections 2023-05-16T07:36:47.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.589+0000 7f01968a8700 1 -- 172.21.15.92:0/2728367818 wait complete. 2023-05-16T07:36:47.604 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-05-16T07:36:47.604 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-05-16-07_37_00 in ~13s... 2023-05-16T07:36:47.604 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-05-16T07:36:47.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-05-16T07:36:47.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.715+0000 7f206177b700 1 Processor -- start 2023-05-16T07:36:47.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.715+0000 7f206177b700 1 -- start start 2023-05-16T07:36:47.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f206177b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f206177b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 0x7f205c068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f206177b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0c94d0 con 0x7f205c154470 2023-05-16T07:36:47.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f206177b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0cb7e0 con 0x7f205c068160 2023-05-16T07:36:47.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f206177b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c068a70 con 0x7f205c154090 2023-05-16T07:36:47.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f205affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.716+0000 7f205b7fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 0x7f205c068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:44582/0 (socket says 172.21.15.92:44582) 2023-05-16T07:36:47.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205b7fe700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 0x7f205c068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:55646/0 (socket says 172.21.15.92:55646) 2023-05-16T07:36:47.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 -- 172.21.15.92:0/1597346949 learned_addr learned my addr 172.21.15.92:0/1597346949 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:47.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205a7fc700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f205c154470 0x7f205c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:38126/0 (socket says 172.21.15.92:38126) 2023-05-16T07:36:47.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 -- 172.21.15.92:0/1597346949 >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 msgr2=0x7f205c068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:36:47.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 --2- 172.21.15.92:0/1597346949 >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 0x7f205c068580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 -- 172.21.15.92:0/1597346949 >> v1:172.21.15.158:6789/0 conn(0x7f205c154470 legacy=0x7f205c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f2059ffb700 1 -- 172.21.15.92:0/1597346949 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382533517 0 0) 0x7f205c0c94d0 con 0x7f205c154470 2023-05-16T07:36:47.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 -- 172.21.15.92:0/1597346949 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0c88c0 con 0x7f205c154090 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205b7fe700 1 --2- 172.21.15.92:0/1597346949 >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 0x7f205c068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f205affd700 1 --2- 172.21.15.92:0/1597346949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c0aaca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f204c009f20 tx=0x7f204c009840).ready entity=mon.2 client_cookie=c78f4f3104f6545a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f2059ffb700 1 -- 172.21.15.92:0/1597346949 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c00e070 con 0x7f205c154090 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f2059ffb700 1 -- 172.21.15.92:0/1597346949 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c00b9d0 con 0x7f205c154090 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.717+0000 7f2059ffb700 1 -- 172.21.15.92:0/1597346949 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c00bd50 con 0x7f205c154090 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 -- 172.21.15.92:0/1597346949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 msgr2=0x7f205c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 --2- 172.21.15.92:0/1597346949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c0aaca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f204c009f20 tx=0x7f204c009840).stop 2023-05-16T07:36:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 -- 172.21.15.92:0/1597346949 shutdown_connections 2023-05-16T07:36:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 --2- 172.21.15.92:0/1597346949 >> v2:172.21.15.158:3300/0 conn(0x7f205c068160 0x7f205c068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 --2- 172.21.15.92:0/1597346949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c0aaca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 -- 172.21.15.92:0/1597346949 >> 172.21.15.92:0/1597346949 conn(0x7f205c067920 msgr2=0x7f205c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 -- 172.21.15.92:0/1597346949 shutdown_connections 2023-05-16T07:36:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.718+0000 7f206177b700 1 -- 172.21.15.92:0/1597346949 wait complete. 2023-05-16T07:36:47.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.719+0000 7f206177b700 1 Processor -- start 2023-05-16T07:36:47.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.719+0000 7f206177b700 1 -- start start 2023-05-16T07:36:47.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f206177b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 0x7f205c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f206177b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f206177b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f205c154470 0x7f205c143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f206177b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c157050 con 0x7f205c068160 2023-05-16T07:36:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f206177b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c156440 con 0x7f205c154470 2023-05-16T07:36:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f206177b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c156720 con 0x7f205c154090 2023-05-16T07:36:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 0x7f205c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 0x7f205c1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52014/0 (socket says 172.21.15.92:52014) 2023-05-16T07:36:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 -- 172.21.15.92:0/3746800119 learned_addr learned my addr 172.21.15.92:0/3746800119 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:36:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205a7fc700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205b7fe700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f205c154470 0x7f205c143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 -- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 msgr2=0x7f205c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 -- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f205c154470 msgr2=0x7f205c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f205c154470 0x7f205c143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 -- 172.21.15.92:0/3746800119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540092a0 con 0x7f205c068160 2023-05-16T07:36:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f205affd700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 0x7f205c1574e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f204c00b610 tx=0x7f204c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.720+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c007170 con 0x7f205c068160 2023-05-16T07:36:47.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.721+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c013cc0 con 0x7f205c068160 2023-05-16T07:36:47.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.721+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c009ae0 con 0x7f205c068160 2023-05-16T07:36:47.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.721+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c01e6c0 con 0x7f205c068160 2023-05-16T07:36:47.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.721+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c156ce0 con 0x7f205c068160 2023-05-16T07:36:47.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.721+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c066730 con 0x7f205c068160 2023-05-16T07:36:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.722+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 73) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f204c025030 con 0x7f205c068160 2023-05-16T07:36:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.722+0000 7f2052ffd700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f203c05ed40 0x7f203c061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:36:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.722+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6933+0+0 (secure 0 0 0) 0x7f204c07b370 con 0x7f205c068160 2023-05-16T07:36:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.722+0000 7f205a7fc700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f203c05ed40 0x7f203c061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:36:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.723+0000 7f205a7fc700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f203c05ed40 0x7f203c061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2044007fd0 tx=0x7f2044009040).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:36:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.726+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f204c04c730 con 0x7f205c068160 2023-05-16T07:36:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.910+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f205c157190 con 0x7f203c05ed40 2023-05-16T07:36:47.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.914+0000 7f2052ffd700 1 -- 172.21.15.92:0/3746800119 <== mgr.7127 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f2044007110 con 0x7f203c05ed40 2023-05-16T07:36:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:36:47.917 INFO:teuthology.orchestra.run.smithi092.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-05-16T07:36:47.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f203c05ed40 msgr2=0x7f203c061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f203c05ed40 0x7f203c061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2044007fd0 tx=0x7f2044009040).stop 2023-05-16T07:36:47.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 msgr2=0x7f205c1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:36:47.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 0x7f205c1574e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f204c00b610 tx=0x7f204c009840).stop 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 shutdown_connections 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f205c154470 0x7f205c143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f205c068160 0x7f205c1574e0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f203c05ed40 0x7f203c061200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 --2- 172.21.15.92:0/3746800119 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f205c154090 0x7f205c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 >> 172.21.15.92:0/3746800119 conn(0x7f205c067920 msgr2=0x7f205c14f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:36:47.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 shutdown_connections 2023-05-16T07:36:47.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:36:47.915+0000 7f206177b700 1 -- 172.21.15.92:0/3746800119 wait complete. 2023-05-16T07:36:47.930 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-05-16T07:36:47.930 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-05-16T07:36:47.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:47.931 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:48.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:48.946 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:49.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:49.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:50.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:50.985 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:52.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:52.005 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:53.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:53.024 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:54.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:54.044 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:55.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:55.063 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:56.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:56.083 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:57.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:57.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:58.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:58.124 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:36:59.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:36:59.144 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:00.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:00.164 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:00.594 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:37:00.590+0000 7f8f08ca6700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:37:00.595 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:37:00.590+0000 7f8f08ca6700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:37:00.595 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:37:00.590+0000 7f8f08ca6700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:37:00.595 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:37:00.590+0000 7f8f08ca6700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:37:00.595 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:37:00.590+0000 7f8f08ca6700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:37:01.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:01.184 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:01.202 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:01.204 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-05-16-07_37_00_UTC'] 2023-05-16T07:37:01.204 DEBUG:tasks.cephfs.test_snap_schedules:exec=1684222607.6044936, now = 1684222621.2048137, timo = 13 2023-05-16T07:37:02.205 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-05-16-07_37_00 in ~73s... 2023-05-16T07:37:02.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:02.207 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:02.226 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:03.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:03.227 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:03.245 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:04.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:04.247 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:04.266 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:05.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:05.268 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:05.286 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:06.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:06.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:06.307 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:07.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:07.310 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:07.329 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:08.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:08.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:08.349 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:09.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:09.351 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:09.368 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:10.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:37:10.031 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:37:10.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:10.370 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:10.387 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:11.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:11.389 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:11.407 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:12.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:12.410 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:12.428 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:13.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:13.430 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:13.448 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:14.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:14.450 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:14.467 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:15.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:15.470 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:15.489 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:16.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:16.491 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:16.510 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:17.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:17.512 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:17.530 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:18.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:18.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:18.551 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:19.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:19.553 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:19.571 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:20.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:20.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:20.591 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:21.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:21.592 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:21.610 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:22.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:22.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:22.632 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:23.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:23.634 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:23.651 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:24.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:24.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:24.672 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:25.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:25.674 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:25.692 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:26.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:26.694 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:26.711 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:27.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:27.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:27.733 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:28.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:28.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:28.753 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:29.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:29.755 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:29.773 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:30.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:30.775 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:30.792 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:31.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:31.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:32.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:32.814 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:32.831 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:33.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:33.834 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:33.851 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:34.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:34.853 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:34.871 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:35.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:35.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:35.893 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:36.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:36.895 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:36.914 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:37.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:37.916 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:37.933 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:38.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:38.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:38.954 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:39.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:39.956 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:39.975 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:40.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:37:40.110 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:37:40.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:40.976 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:40.996 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:41.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:41.998 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:42.016 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:43.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:43.018 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:43.036 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:44.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:44.038 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:44.054 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:45.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:45.056 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:45.075 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:46.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:46.077 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:46.095 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:47.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:47.097 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:47.114 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:48.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:48.117 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:48.134 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:49.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:49.136 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:49.156 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:50.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:50.158 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:50.176 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:51.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:51.178 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:51.196 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:52.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:52.198 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:52.218 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:53.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:53.220 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:53.239 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:54.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:54.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:54.259 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:55.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:55.261 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:55.279 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:56.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:56.281 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:56.299 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:57.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:57.301 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:57.320 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:58.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:58.321 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:58.338 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:37:59.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:37:59.340 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:37:59.358 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:38:00.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:00.385 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:38:00.404 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_37_00_UTC 2023-05-16T07:38:01.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:01.407 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:38:01.426 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_38_00_UTC 2023-05-16T07:38:01.427 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-05-16-07_37_00_UTC'] 2023-05-16T07:38:01.427 DEBUG:tasks.cephfs.test_snap_schedules:exec=1684222607.6044936, now = 1684222681.4276655, timo = 73 2023-05-16T07:38:02.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-05-16T07:38:02.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.552+0000 7f67a46a7700 1 Processor -- start 2023-05-16T07:38:02.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.552+0000 7f67a46a7700 1 -- start start 2023-05-16T07:38:02.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a46a7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 0x7f679c063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a46a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c0bf950 con 0x7f679c1540e0 2023-05-16T07:38:02.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a46a7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c0a7c90 con 0x7f679c0622b0 2023-05-16T07:38:02.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a46a7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c067f60 con 0x7f679c153d00 2023-05-16T07:38:02.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a1c42700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f679c1540e0 0x7f679c0bf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34540/0 (socket says 172.21.15.92:34540) 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a1c42700 1 -- 172.21.15.92:0/2362007688 learned_addr learned my addr 172.21.15.92:0/2362007688 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.553+0000 7f67a2443700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 0x7f679c063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880785993 0 0) 0x7f679c0bf950 con 0x7f679c1540e0 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67840033f0 con 0x7f679c1540e0 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682133771 0 0) 0x7f679c0a7c90 con 0x7f679c0622b0 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f679c0bf950 con 0x7f679c0622b0 2023-05-16T07:38:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2801380782 0 0) 0x7f67840033f0 con 0x7f679c1540e0 2023-05-16T07:38:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 msgr2=0x7f679c063fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 0x7f679c063fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 >> v1:172.21.15.158:6789/0 conn(0x7f679c0622b0 legacy=0x7f679c0683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c0a7f00 con 0x7f679c1540e0 2023-05-16T07:38:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6790001470 con 0x7f679c1540e0 2023-05-16T07:38:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 >> v1:172.21.15.92:6789/0 conn(0x7f679c1540e0 legacy=0x7f679c0bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 0x7f6784003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784003fb0 0x7f6784004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784004b80 0x7f6784005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678400b990 con 0x7f6784003fb0 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678400bad0 con 0x7f6784004b80 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678400bc90 con 0x7f679c153d00 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784003fb0 0x7f6784004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a2443700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 0x7f6784003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a2c44700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784004b80 0x7f6784005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 -- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 msgr2=0x7f6784003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f679c153d00 0x7f6784003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 -- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784004b80 msgr2=0x7f6784005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6784004b80 0x7f6784005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 -- 172.21.15.92:0/2362007688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840033f0 con 0x7f6784003fb0 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1c42700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784003fb0 0x7f6784004690 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f679000c160 tx=0x7f6790001e10).ready entity=mon.0 client_cookie=85988246a8c21075 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000c9d0 con 0x7f6784003fb0 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.554+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679000cb30 con 0x7f6784003fb0 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a1441700 1 -- 172.21.15.92:0/2362007688 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900070d0 con 0x7f6784003fb0 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a46a7700 1 -- 172.21.15.92:0/2362007688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784003fb0 msgr2=0x7f6784004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a46a7700 1 --2- 172.21.15.92:0/2362007688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6784003fb0 0x7f6784004690 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f679000c160 tx=0x7f6790001e10).stop 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a2c44700 1 -- 172.21.15.92:0/2362007688 reap_dead start 2023-05-16T07:38:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a46a7700 1 -- 172.21.15.92:0/2362007688 shutdown_connections 2023-05-16T07:38:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a46a7700 1 -- 172.21.15.92:0/2362007688 >> 172.21.15.92:0/2362007688 conn(0x7f679c05e820 msgr2=0x7f679c0c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a46a7700 1 -- 172.21.15.92:0/2362007688 shutdown_connections 2023-05-16T07:38:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.555+0000 7f67a46a7700 1 -- 172.21.15.92:0/2362007688 wait complete. 2023-05-16T07:38:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.556+0000 7f67a46a7700 1 Processor -- start 2023-05-16T07:38:02.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.556+0000 7f67a46a7700 1 -- start start 2023-05-16T07:38:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a46a7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784003fb0 0x7f679c0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a46a7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f679c0a4d70 0x7f679c0a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a46a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 0x7f679c166050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a46a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c165bc0 con 0x7f679c0a9db0 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a46a7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0a51f0 con 0x7f679c0a4d70 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a46a7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0aaed0 con 0x7f6784003fb0 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 0x7f679c166050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 0x7f679c166050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52038/0 (socket says 172.21.15.92:52038) 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 -- 172.21.15.92:0/3306933713 learned_addr learned my addr 172.21.15.92:0/3306933713 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a1c42700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f679c0a4d70 0x7f679c0a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2443700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784003fb0 0x7f679c0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 -- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784003fb0 msgr2=0x7f679c0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784003fb0 0x7f679c0a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 -- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f679c0a4d70 msgr2=0x7f679c0a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f679c0a4d70 0x7f679c0a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 -- 172.21.15.92:0/3306933713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c0aa630 con 0x7f679c0a9db0 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.557+0000 7f67a2c44700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 0x7f679c166050 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f679c065a70 tx=0x7f6794009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.558+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679400ee40 con 0x7f679c0a9db0 2023-05-16T07:38:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.558+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6794007030 con 0x7f679c0a9db0 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.558+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c0aa910 con 0x7f679c0a9db0 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.558+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67940073f0 con 0x7f679c0a9db0 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.558+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c0aa230 con 0x7f679c0a9db0 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.559+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c158840 con 0x7f679c0a9db0 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.559+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 75) v1 ==== 75419+0+0 (secure 0 0 0) 0x7f6794006900 con 0x7f679c0a9db0 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.559+0000 7f679a7fc700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f678805ef00 0x7f67880613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.559+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7f6794079f90 con 0x7f679c0a9db0 2023-05-16T07:38:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.560+0000 7f67a2443700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f678805ef00 0x7f67880613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.560+0000 7f67a2443700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f678805ef00 0x7f67880613c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f678c0094d0 tx=0x7f678c00b040).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.564+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f679404b150 con 0x7f679c0a9db0 2023-05-16T07:38:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.755+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f679c0be8c0 con 0x7f678805ef00 2023-05-16T07:38:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.760+0000 7f679a7fc700 1 -- 172.21.15.92:0/3306933713 <== mgr.7127 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f679c0be8c0 con 0x7f678805ef00 2023-05-16T07:38:02.765 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path snap_test_dir1 2023-05-16T07:38:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f678805ef00 msgr2=0x7f67880613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f678805ef00 0x7f67880613c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f678c0094d0 tx=0x7f678c00b040).stop 2023-05-16T07:38:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 msgr2=0x7f679c166050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 0x7f679c166050 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f679c065a70 tx=0x7f6794009040).stop 2023-05-16T07:38:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 shutdown_connections 2023-05-16T07:38:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f678805ef00 0x7f67880613c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6784003fb0 0x7f679c0a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f679c0a9db0 0x7f679c166050 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 --2- 172.21.15.92:0/3306933713 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f679c0a4d70 0x7f679c0a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 >> 172.21.15.92:0/3306933713 conn(0x7f679c05e820 msgr2=0x7f679c0c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 shutdown_connections 2023-05-16T07:38:02.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:02.762+0000 7f67a46a7700 1 -- 172.21.15.92:0/3306933713 wait complete. 2023-05-16T07:38:02.778 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-05-16T07:38:02.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:02.779 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:38:02.795 INFO:teuthology.orchestra.run.smithi092.stdout:scheduled-2023-05-16-07_38_00_UTC 2023-05-16T07:38:02.795 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-05-16-07_38_00_UTC 2023-05-16T07:38:02.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:02.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-05-16-07_38_00_UTC) 2023-05-16T07:38:02.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:02.867 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-05-16T07:38:02.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:38:03.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.086+0000 7f0aec7e9700 1 Processor -- start 2023-05-16T07:38:03.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.086+0000 7f0aec7e9700 1 -- start start 2023-05-16T07:38:03.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aec7e9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 0x7f0ae40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:03.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aec7e9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 0x7f0ae40685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:03.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aec7e9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae40c94d0 con 0x7f0ae4154090 2023-05-16T07:38:03.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aec7e9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae40cb7e0 con 0x7f0ae4154470 2023-05-16T07:38:03.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aec7e9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4068aa0 con 0x7f0ae4068160 2023-05-16T07:38:03.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 0x7f0ae40685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:03.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0ae9d84700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 0x7f0ae40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:03.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 0x7f0ae40685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:44618/0 (socket says 172.21.15.92:44618) 2023-05-16T07:38:03.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0ae9d84700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 0x7f0ae40aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:55680/0 (socket says 172.21.15.92:55680) 2023-05-16T07:38:03.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 -- 172.21.15.92:0/1183554145 learned_addr learned my addr 172.21.15.92:0/1183554145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:03.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aea585700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f0ae4154090 0x7f0ae40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:38158/0 (socket says 172.21.15.92:38158) 2023-05-16T07:38:03.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 -- 172.21.15.92:0/1183554145 >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 msgr2=0x7f0ae40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 --2- 172.21.15.92:0/1183554145 >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 0x7f0ae40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 -- 172.21.15.92:0/1183554145 >> v1:172.21.15.158:6789/0 conn(0x7f0ae4154090 legacy=0x7f0ae40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0aead86700 1 -- 172.21.15.92:0/1183554145 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40c88c0 con 0x7f0ae4068160 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.087+0000 7f0ae9d84700 1 --2- 172.21.15.92:0/1183554145 >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 0x7f0ae40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aead86700 1 --2- 172.21.15.92:0/1183554145 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 0x7f0ae40685b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f0ae0006fd0 tx=0x7f0ae00090f0).ready entity=mon.2 client_cookie=1333976ff3686bb9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0ae9583700 1 -- 172.21.15.92:0/1183554145 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000b580 con 0x7f0ae4068160 2023-05-16T07:38:03.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0ae9583700 1 -- 172.21.15.92:0/1183554145 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae000b6e0 con 0x7f0ae4068160 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0ae9583700 1 -- 172.21.15.92:0/1183554145 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000ba60 con 0x7f0ae4068160 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 -- 172.21.15.92:0/1183554145 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 msgr2=0x7f0ae40685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/1183554145 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 0x7f0ae40685b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f0ae0006fd0 tx=0x7f0ae00090f0).stop 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 -- 172.21.15.92:0/1183554145 shutdown_connections 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/1183554145 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4068160 0x7f0ae40685b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/1183554145 >> v2:172.21.15.158:3300/0 conn(0x7f0ae4154470 0x7f0ae40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 -- 172.21.15.92:0/1183554145 >> 172.21.15.92:0/1183554145 conn(0x7f0ae4067920 msgr2=0x7f0ae40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:03.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 -- 172.21.15.92:0/1183554145 shutdown_connections 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.088+0000 7f0aec7e9700 1 -- 172.21.15.92:0/1183554145 wait complete. 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 Processor -- start 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 -- start start 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 0x7f0ae4145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4154090 0x7f0ae40c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ae4154470 0x7f0ae40c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:03.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40c3030 con 0x7f0ae4068160 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4151a50 con 0x7f0ae4154470 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aec7e9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40c2790 con 0x7f0ae4154090 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aea585700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 0x7f0ae4145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aea585700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 0x7f0ae4145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52050/0 (socket says 172.21.15.92:52050) 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.090+0000 7f0aea585700 1 -- 172.21.15.92:0/3771697542 learned_addr learned my addr 172.21.15.92:0/3771697542 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aead86700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ae4154470 0x7f0ae40c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0ae9d84700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4154090 0x7f0ae40c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aea585700 1 -- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4154090 msgr2=0x7f0ae40c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aea585700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4154090 0x7f0ae40c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aea585700 1 -- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ae4154470 msgr2=0x7f0ae40c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aea585700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ae4154470 0x7f0ae40c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aea585700 1 -- 172.21.15.92:0/3771697542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40092a0 con 0x7f0ae4068160 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aea585700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 0x7f0ae4145970 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f0adc00c870 tx=0x7f0adc0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc005c70 con 0x7f0ae4068160 2023-05-16T07:38:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc005dd0 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae0009370 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc007320 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.091+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae40c2d50 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.092+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae4151820 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.092+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 75) v1 ==== 75419+0+0 (secure 0 0 0) 0x7f0adc010030 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.093+0000 7f0ada7fc700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0ac805ef00 0x7f0ac80613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.093+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7f0adc079f00 con 0x7f0ae4068160 2023-05-16T07:38:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.093+0000 7f0ae9d84700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0ac805ef00 0x7f0ac80613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.093+0000 7f0ae9d84700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0ac805ef00 0x7f0ac80613c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0ae41500b0 tx=0x7f0ad4009060).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.097+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0ae4151820 con 0x7f0ae4068160 2023-05-16T07:38:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.300+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f0ae4151820 con 0x7f0ae4068160 2023-05-16T07:38:03.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.911+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f0adc077910 con 0x7f0ae4068160 2023-05-16T07:38:03.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.912+0000 7f0ada7fc700 1 -- 172.21.15.92:0/3771697542 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v76) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0adc073b40 con 0x7f0ae4068160 2023-05-16T07:38:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0ac805ef00 msgr2=0x7f0ac80613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0ac805ef00 0x7f0ac80613c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0ae41500b0 tx=0x7f0ad4009060).stop 2023-05-16T07:38:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 msgr2=0x7f0ae4145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 0x7f0ae4145970 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f0adc00c870 tx=0x7f0adc0015e0).stop 2023-05-16T07:38:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 shutdown_connections 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0ae4154470 0x7f0ae40c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4068160 0x7f0ae4145970 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0ac805ef00 0x7f0ac80613c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 --2- 172.21.15.92:0/3771697542 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0ae4154090 0x7f0ae40c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.915+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 >> 172.21.15.92:0/3771697542 conn(0x7f0ae4067920 msgr2=0x7f0ae40ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.916+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 shutdown_connections 2023-05-16T07:38:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:03.916+0000 7f0aec7e9700 1 -- 172.21.15.92:0/3771697542 wait complete. 2023-05-16T07:38:03.934 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:38:03.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:38:03.964 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:38:03.993 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:38:03.993 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:38:03.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:03.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:38:04.048 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:38:04.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:04.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:38:04.061 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:04.057+0000 7ff8e25c3680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:38:04.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:04.064 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:04.062+0000 7f11331ca680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:38:04.126 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:04.122+0000 7ff8e25c3680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:38:04.127 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:04.124+0000 7f11331ca680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:38:04.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.215+0000 7f7db439c700 1 Processor -- start 2023-05-16T07:38:04.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.215+0000 7f7db439c700 1 -- start start 2023-05-16T07:38:04.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.216+0000 7f7db439c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7dac154170 0x7f7dac0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.216+0000 7f7db439c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac069db0 0x7f7dac0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.216+0000 7f7db439c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac0c0660 con 0x7f7dac154550 2023-05-16T07:38:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.216+0000 7f7db439c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0c1b00 con 0x7f7dac154170 2023-05-16T07:38:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.216+0000 7f7db439c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac0bfd30 con 0x7f7dac069db0 2023-05-16T07:38:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7dac154170 0x7f7dac0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7dac154170 0x7f7dac0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52126/0 (socket says 172.21.15.92:52126) 2023-05-16T07:38:04.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 -- 172.21.15.92:0/3572964196 learned_addr learned my addr 172.21.15.92:0/3572964196 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2939700 1 --2- 172.21.15.92:0/3572964196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac069db0 0x7f7dac0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 -- 172.21.15.92:0/3572964196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac069db0 msgr2=0x7f7dac0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 --2- 172.21.15.92:0/3572964196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac069db0 0x7f7dac0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 -- 172.21.15.92:0/3572964196 >> v1:172.21.15.158:6789/0 conn(0x7f7dac154550 legacy=0x7f7dac0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 -- 172.21.15.92:0/3572964196 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0c0010 con 0x7f7dac154170 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db2138700 1 --2- 172.21.15.92:0/3572964196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac154170 0x7f7dac0c5c70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f7d9c006fd0 tx=0x7f7d9c0090f0).ready entity=mon.0 client_cookie=4213af7d2c7efb82 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db1136700 1 -- 172.21.15.92:0/3572964196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00b580 con 0x7f7dac154170 2023-05-16T07:38:04.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db1136700 1 -- 172.21.15.92:0/3572964196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00b6e0 con 0x7f7dac154170 2023-05-16T07:38:04.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.217+0000 7f7db1136700 1 -- 172.21.15.92:0/3572964196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00ba80 con 0x7f7dac154170 2023-05-16T07:38:04.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 -- 172.21.15.92:0/3572964196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac154170 msgr2=0x7f7dac0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 --2- 172.21.15.92:0/3572964196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac154170 0x7f7dac0c5c70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f7d9c006fd0 tx=0x7f7d9c0090f0).stop 2023-05-16T07:38:04.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 -- 172.21.15.92:0/3572964196 shutdown_connections 2023-05-16T07:38:04.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 --2- 172.21.15.92:0/3572964196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac154170 0x7f7dac0c5c70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 --2- 172.21.15.92:0/3572964196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac069db0 0x7f7dac0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 -- 172.21.15.92:0/3572964196 >> 172.21.15.92:0/3572964196 conn(0x7f7dac0cb700 msgr2=0x7f7dac0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 -- 172.21.15.92:0/3572964196 shutdown_connections 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.218+0000 7f7db439c700 1 -- 172.21.15.92:0/3572964196 wait complete. 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.219+0000 7f7db439c700 1 Processor -- start 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.219+0000 7f7db439c700 1 -- start start 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 0x7f7dac0c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dac154170 0x7f7dac145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac154550 0x7f7dac147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac0c31d0 con 0x7f7dac069db0 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac0c44d0 con 0x7f7dac154170 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac0c38c0 con 0x7f7dac154550 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 0x7f7dac0c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 0x7f7dac0c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52130/0 (socket says 172.21.15.92:52130) 2023-05-16T07:38:04.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 -- 172.21.15.92:0/446217334 learned_addr learned my addr 172.21.15.92:0/446217334 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2939700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac154550 0x7f7dac147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db1937700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dac154170 0x7f7dac145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 -- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac154550 msgr2=0x7f7dac147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac154550 0x7f7dac147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 -- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dac154170 msgr2=0x7f7dac145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dac154170 0x7f7dac145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 -- 172.21.15.92:0/446217334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c009370 con 0x7f7dac069db0 2023-05-16T07:38:04.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2939700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac154550 0x7f7dac147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db1937700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dac154170 0x7f7dac145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db2138700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 0x7f7dac0c4960 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f7d9c009ea0 tx=0x7f7d9c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00be40 con 0x7f7dac069db0 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00b850 con 0x7f7dac069db0 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac0c3ba0 con 0x7f7dac069db0 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c01c420 con 0x7f7dac069db0 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.220+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac0c4160 con 0x7f7dac069db0 2023-05-16T07:38:04.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.221+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac0666c0 con 0x7f7dac069db0 2023-05-16T07:38:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.222+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f7d9c02c070 con 0x7f7dac069db0 2023-05-16T07:38:04.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.222+0000 7f7da27fc700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 0x7f7d90061340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.222+0000 7f7db1937700 1 -- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 msgr2=0x7f7d90061340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.222+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7f7d9c07bc80 con 0x7f7dac069db0 2023-05-16T07:38:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.222+0000 7f7db1937700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 0x7f7d90061340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:04.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.227+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f7d9c04cd30 con 0x7f7dac069db0 2023-05-16T07:38:04.396 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:04.393+0000 7f11331ca680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:38:04.405 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:04.401+0000 7ff8e25c3680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:38:04.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.423+0000 7f7db1937700 1 -- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 msgr2=0x7f7d90061340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:04.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.423+0000 7f7db1937700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 0x7f7d90061340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:04.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.434+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dac06c350 con 0x7f7dac069db0 2023-05-16T07:38:04.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.435+0000 7f7da27fc700 1 -- 172.21.15.92:0/446217334 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f7d9c02a020 con 0x7f7dac069db0 2023-05-16T07:38:04.439 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:04.439 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:36:34.963007+0000","modified":"2023-05-16T07:36:36.970661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/1435424518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1435424518},{"type":"v1","addr":"172.21.15.158:6837","nonce":1435424518}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-05-16T07:38:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 msgr2=0x7f7d90061340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 0x7f7d90061340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:04.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 msgr2=0x7f7dac0c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 0x7f7dac0c4960 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f7d9c009ea0 tx=0x7f7d9c017000).stop 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 shutdown_connections 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f7d9005ee80 0x7f7d90061340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7dac154550 0x7f7dac147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dac069db0 0x7f7dac0c4960 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 --2- 172.21.15.92:0/446217334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7dac154170 0x7f7dac145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 >> 172.21.15.92:0/446217334 conn(0x7f7dac0cb700 msgr2=0x7f7dac0abbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:04.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 shutdown_connections 2023-05-16T07:38:04.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.437+0000 7f7db439c700 1 -- 172.21.15.92:0/446217334 wait complete. 2023-05-16T07:38:04.443 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-05-16T07:38:04.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:04.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.603+0000 7f259d1e7700 1 Processor -- start 2023-05-16T07:38:04.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.603+0000 7f259d1e7700 1 -- start start 2023-05-16T07:38:04.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259d1e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598154040 0x7f2598069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259d1e7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25980bf4f0 0x7f2598062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259d1e7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2598063720 0x7f2598063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259d1e7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2598154420 con 0x7f25980bf4f0 2023-05-16T07:38:04.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259d1e7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980640c0 con 0x7f2598063720 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259d1e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598062960 con 0x7f2598154040 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25980bf4f0 0x7f2598062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25980bf4f0 0x7f2598062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52184/0 (socket says 172.21.15.92:52184) 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 -- 172.21.15.92:0/3346423166 learned_addr learned my addr 172.21.15.92:0/3346423166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f2596d9d700 1 --2- 172.21.15.92:0/3346423166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598154040 0x7f2598069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259759e700 1 --2- 172.21.15.92:0/3346423166 >> v2:172.21.15.158:3300/0 conn(0x7f2598063720 0x7f2598063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 -- 172.21.15.92:0/3346423166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598154040 msgr2=0x7f2598069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 --2- 172.21.15.92:0/3346423166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598154040 0x7f2598069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 -- 172.21.15.92:0/3346423166 >> v2:172.21.15.158:3300/0 conn(0x7f2598063720 msgr2=0x7f2598063bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 --2- 172.21.15.92:0/3346423166 >> v2:172.21.15.158:3300/0 conn(0x7f2598063720 0x7f2598063bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259659c700 1 -- 172.21.15.92:0/3346423166 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598069fb0 con 0x7f25980bf4f0 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.604+0000 7f259759e700 1 --2- 172.21.15.92:0/3346423166 >> v2:172.21.15.158:3300/0 conn(0x7f2598063720 0x7f2598063bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259659c700 1 --2- 172.21.15.92:0/3346423166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25980bf4f0 0x7f2598062df0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f25980bf0c0 tx=0x7f25880090f0).ready entity=mon.0 client_cookie=220b2c808f0144dd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f2595d9b700 1 -- 172.21.15.92:0/3346423166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800b520 con 0x7f25980bf4f0 2023-05-16T07:38:04.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f2595d9b700 1 -- 172.21.15.92:0/3346423166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258800b680 con 0x7f25980bf4f0 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f2595d9b700 1 -- 172.21.15.92:0/3346423166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800ba00 con 0x7f25980bf4f0 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 -- 172.21.15.92:0/3346423166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25980bf4f0 msgr2=0x7f2598062df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 --2- 172.21.15.92:0/3346423166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25980bf4f0 0x7f2598062df0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f25980bf0c0 tx=0x7f25880090f0).stop 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 -- 172.21.15.92:0/3346423166 shutdown_connections 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 --2- 172.21.15.92:0/3346423166 >> v2:172.21.15.158:3300/0 conn(0x7f2598063720 0x7f2598063bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 --2- 172.21.15.92:0/3346423166 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598154040 0x7f2598069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 --2- 172.21.15.92:0/3346423166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25980bf4f0 0x7f2598062df0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 -- 172.21.15.92:0/3346423166 >> 172.21.15.92:0/3346423166 conn(0x7f25980c8aa0 msgr2=0x7f25980c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 -- 172.21.15.92:0/3346423166 shutdown_connections 2023-05-16T07:38:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.605+0000 7f259d1e7700 1 -- 172.21.15.92:0/3346423166 wait complete. 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 Processor -- start 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 -- start start 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598063720 0x7f25980a8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25980bf4f0 0x7f25980a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 0x7f259814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598146f00 con 0x7f2598154040 2023-05-16T07:38:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980aa5a0 con 0x7f25980bf4f0 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259d1e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25980a9990 con 0x7f2598063720 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 0x7f259814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 0x7f259814e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52192/0 (socket says 172.21.15.92:52192) 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259759e700 1 -- 172.21.15.92:0/600301561 learned_addr learned my addr 172.21.15.92:0/600301561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f2596d9d700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598063720 0x7f25980a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.607+0000 7f259659c700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25980bf4f0 0x7f25980a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259759e700 1 -- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598063720 msgr2=0x7f25980a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259759e700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598063720 0x7f25980a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259759e700 1 -- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25980bf4f0 msgr2=0x7f25980a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259759e700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25980bf4f0 0x7f25980a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259759e700 1 -- 172.21.15.92:0/600301561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588009370 con 0x7f2598154040 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259759e700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 0x7f259814e070 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f259000d100 tx=0x7f25900099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259000dc40 con 0x7f2598154040 2023-05-16T07:38:04.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259000f040 con 0x7f2598154040 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25980a9c70 con 0x7f2598154040 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900133f0 con 0x7f2598154040 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.608+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25980aa230 con 0x7f2598154040 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.609+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25980aa730 con 0x7f2598154040 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.609+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f25900137a0 con 0x7f2598154040 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.610+0000 7f257effd700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f258405eeb0 0x7f2584061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.610+0000 7f2596d9d700 1 -- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f258405eeb0 msgr2=0x7f2584061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.610+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7f259007a640 con 0x7f2598154040 2023-05-16T07:38:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.610+0000 7f2596d9d700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f258405eeb0 0x7f2584061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.614+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f259004b7a0 con 0x7f2598154040 2023-05-16T07:38:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.782+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f259806c350 con 0x7f2598154040 2023-05-16T07:38:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.783+0000 7f257effd700 1 -- 172.21.15.92:0/600301561 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+18308 (secure 0 0 0) 0x7f259004edc0 con 0x7f2598154040 2023-05-16T07:38:04.788 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:04.788 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":157,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:37:47.982726+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:36:33.491970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-05-16T07:36:33.931369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:6833/95623":"2023-05-17T07:36:46.188304+0000","172.21.15.158:6832/95623":"2023-05-17T07:36:46.188304+0000","172.21.15.158:0/2052480298":"2023-05-17T07:36:46.188304+0000","172.21.15.158:0/2894032393":"2023-05-17T07:36:46.188304+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f258405eeb0 msgr2=0x7f2584061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f258405eeb0 0x7f2584061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 msgr2=0x7f259814e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 0x7f259814e070 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f259000d100 tx=0x7f25900099b0).stop 2023-05-16T07:38:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 shutdown_connections 2023-05-16T07:38:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f258405eeb0 0x7f2584061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2598063720 0x7f25980a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2598154040 0x7f259814e070 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 --2- 172.21.15.92:0/600301561 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25980bf4f0 0x7f25980a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 >> 172.21.15.92:0/600301561 conn(0x7f25980c8aa0 msgr2=0x7f259814f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:04.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.785+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 shutdown_connections 2023-05-16T07:38:04.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.786+0000 7f259d1e7700 1 -- 172.21.15.92:0/600301561 wait complete. 2023-05-16T07:38:04.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:36:33.491970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:36:33.931369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:38:04.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:38:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.917+0000 7fc998228700 1 Processor -- start 2023-05-16T07:38:04.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.917+0000 7fc998228700 1 -- start start 2023-05-16T07:38:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc998228700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc990153ee0 0x7fc9901542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc998228700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9900be280 0x7fc990066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc998228700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc990155870 con 0x7fc990062cc0 2023-05-16T07:38:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc998228700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc990067340 con 0x7fc9900be280 2023-05-16T07:38:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc998228700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc990155670 con 0x7fc990153ee0 2023-05-16T07:38:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9900be280 0x7fc990066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc995fc4700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fc990153ee0 0x7fc9901542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9900be280 0x7fc990066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52200/0 (socket says 172.21.15.92:52200) 2023-05-16T07:38:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 -- 172.21.15.92:0/937217563 learned_addr learned my addr 172.21.15.92:0/937217563 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 -- 172.21.15.92:0/937217563 >> v2:172.21.15.158:3300/0 conn(0x7fc990153ee0 msgr2=0x7fc9901542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 --2- 172.21.15.92:0/937217563 >> v2:172.21.15.158:3300/0 conn(0x7fc990153ee0 0x7fc9901542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 -- 172.21.15.92:0/937217563 >> v1:172.21.15.158:6789/0 conn(0x7fc990062cc0 legacy=0x7fc9900bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 -- 172.21.15.92:0/937217563 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990063120 con 0x7fc9900be280 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.918+0000 7fc9967c5700 1 --2- 172.21.15.92:0/937217563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc990066e50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fc9900c7860 tx=0x7fc988009840).ready entity=mon.0 client_cookie=423ea9b8b6f7c361 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc994fc2700 1 -- 172.21.15.92:0/937217563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800e070 con 0x7fc9900be280 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc994fc2700 1 -- 172.21.15.92:0/937217563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800b990 con 0x7fc9900be280 2023-05-16T07:38:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc994fc2700 1 -- 172.21.15.92:0/937217563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800bd10 con 0x7fc9900be280 2023-05-16T07:38:04.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 -- 172.21.15.92:0/937217563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 msgr2=0x7fc990066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 --2- 172.21.15.92:0/937217563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc990066e50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fc9900c7860 tx=0x7fc988009840).stop 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 -- 172.21.15.92:0/937217563 shutdown_connections 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 --2- 172.21.15.92:0/937217563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc990066e50 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 --2- 172.21.15.92:0/937217563 >> v2:172.21.15.158:3300/0 conn(0x7fc990153ee0 0x7fc9901542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 -- 172.21.15.92:0/937217563 >> 172.21.15.92:0/937217563 conn(0x7fc9900c8420 msgr2=0x7fc9900c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 -- 172.21.15.92:0/937217563 shutdown_connections 2023-05-16T07:38:04.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.919+0000 7fc998228700 1 -- 172.21.15.92:0/937217563 wait complete. 2023-05-16T07:38:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.920+0000 7fc998228700 1 Processor -- start 2023-05-16T07:38:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.920+0000 7fc998228700 1 -- start start 2023-05-16T07:38:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc998228700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc990062cc0 0x7fc99014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc998228700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc9900aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc998228700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc990153ee0 0x7fc990152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc998228700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900bf950 con 0x7fc9900be280 2023-05-16T07:38:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc998228700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900ab350 con 0x7fc990153ee0 2023-05-16T07:38:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc998228700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900ab630 con 0x7fc990062cc0 2023-05-16T07:38:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc9957c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc9900aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc9957c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc9900aa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52204/0 (socket says 172.21.15.92:52204) 2023-05-16T07:38:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc9957c3700 1 -- 172.21.15.92:0/3925903309 learned_addr learned my addr 172.21.15.92:0/3925903309 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:04.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc995fc4700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc990062cc0 0x7fc99014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.921+0000 7fc9967c5700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc990153ee0 0x7fc990152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:04.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9957c3700 1 -- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc990062cc0 msgr2=0x7fc99014e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9957c3700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc990062cc0 0x7fc99014e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9957c3700 1 -- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc990153ee0 msgr2=0x7fc990152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:04.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9957c3700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc990153ee0 0x7fc990152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:04.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9957c3700 1 -- 172.21.15.92:0/3925903309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988009ae0 con 0x7fc9900be280 2023-05-16T07:38:04.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9967c5700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc990153ee0 0x7fc990152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:04.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc9957c3700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc9900aa900 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fc98000c4c0 tx=0x7fc980007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:04.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98000ce70 con 0x7fc9900be280 2023-05-16T07:38:04.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980001e90 con 0x7fc9900be280 2023-05-16T07:38:04.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9900ab910 con 0x7fc9900be280 2023-05-16T07:38:04.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980007250 con 0x7fc9900be280 2023-05-16T07:38:04.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.922+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9900bf470 con 0x7fc9900be280 2023-05-16T07:38:04.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.924+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99006c350 con 0x7fc9900be280 2023-05-16T07:38:04.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.924+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7fc980010070 con 0x7fc9900be280 2023-05-16T07:38:04.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.924+0000 7fc985ffb700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 0x7fc974061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:04.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.924+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7fc980079760 con 0x7fc9900be280 2023-05-16T07:38:04.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.924+0000 7fc995fc4700 1 -- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 msgr2=0x7fc974061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:04.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.924+0000 7fc995fc4700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 0x7fc974061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:04.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:04.928+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fc98004a920 con 0x7fc9900be280 2023-05-16T07:38:05.000 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:04.997+0000 7f11331ca680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:38:05.035 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:05.032+0000 7ff8e25c3680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:38:05.071 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:05.068+0000 7f11331ca680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:38:05.107 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:05.104+0000 7ff8e25c3680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:38:05.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.125+0000 7fc995fc4700 1 -- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 msgr2=0x7fc974061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:05.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.125+0000 7fc995fc4700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 0x7fc974061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:05.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.129+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc9901548b0 con 0x7fc9900be280 2023-05-16T07:38:05.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.129+0000 7fc985ffb700 1 -- 172.21.15.92:0/3925903309 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v157) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc98004df40 con 0x7fc9900be280 2023-05-16T07:38:05.133 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:38:05.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 msgr2=0x7fc974061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:05.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 0x7fc974061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:05.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 msgr2=0x7fc9900aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc9900aa900 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fc98000c4c0 tx=0x7fc980007e00).stop 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 shutdown_connections 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fc97405eeb0 0x7fc974061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fc990062cc0 0x7fc99014e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc990153ee0 0x7fc990152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 --2- 172.21.15.92:0/3925903309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900be280 0x7fc9900aa900 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 >> 172.21.15.92:0/3925903309 conn(0x7fc9900c8420 msgr2=0x7fc9900bc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:05.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 shutdown_connections 2023-05-16T07:38:05.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.132+0000 7fc998228700 1 -- 172.21.15.92:0/3925903309 wait complete. 2023-05-16T07:38:05.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:38:05.253 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:05.249+0000 7f11331ca680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:38:05.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.263+0000 7fcc9523c700 1 Processor -- start 2023-05-16T07:38:05.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.263+0000 7fcc9523c700 1 -- start start 2023-05-16T07:38:05.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.263+0000 7fcc9523c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcc90153bb0 0x7fcc900698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc9523c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc900bdf20 0x7fcc90067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc9523c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9005e760 con 0x7fcc90153f90 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc9523c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc90069ef0 con 0x7fcc900bdf20 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc9523c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc900bf540 con 0x7fcc90153bb0 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc94a3b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc900bdf20 0x7fcc90067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcc90153f90 0x7fcc900bd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34728/0 (socket says 172.21.15.92:34728) 2023-05-16T07:38:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8f7fe700 1 -- 172.21.15.92:0/3807209745 learned_addr learned my addr 172.21.15.92:0/3807209745 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8ffff700 1 --2- 172.21.15.92:0/3807209745 >> v2:172.21.15.158:3300/0 conn(0x7fcc90153bb0 0x7fcc900698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8effd700 1 -- 172.21.15.92:0/3807209745 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226706379 0 0) 0x7fcc9005e760 con 0x7fcc90153f90 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8effd700 1 -- 172.21.15.92:0/3807209745 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc780033f0 con 0x7fcc90153f90 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc94a3b700 1 -- 172.21.15.92:0/3807209745 >> v2:172.21.15.158:3300/0 conn(0x7fcc90153bb0 msgr2=0x7fcc900698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc94a3b700 1 --2- 172.21.15.92:0/3807209745 >> v2:172.21.15.158:3300/0 conn(0x7fcc90153bb0 0x7fcc900698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc94a3b700 1 -- 172.21.15.92:0/3807209745 >> v1:172.21.15.92:6789/0 conn(0x7fcc90153f90 legacy=0x7fcc900bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc94a3b700 1 -- 172.21.15.92:0/3807209745 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90068450 con 0x7fcc900bdf20 2023-05-16T07:38:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc94a3b700 1 --2- 172.21.15.92:0/3807209745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc900bdf20 0x7fcc90067f60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7fcc900c98c0 tx=0x7fcc88009780).ready entity=mon.0 client_cookie=961fda9832139f1e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8effd700 1 -- 172.21.15.92:0/3807209745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800e070 con 0x7fcc900bdf20 2023-05-16T07:38:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8effd700 1 -- 172.21.15.92:0/3807209745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8800bab0 con 0x7fcc900bdf20 2023-05-16T07:38:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.264+0000 7fcc8effd700 1 -- 172.21.15.92:0/3807209745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800be30 con 0x7fcc900bdf20 2023-05-16T07:38:05.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 -- 172.21.15.92:0/3807209745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc900bdf20 msgr2=0x7fcc90067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 --2- 172.21.15.92:0/3807209745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc900bdf20 0x7fcc90067f60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7fcc900c98c0 tx=0x7fcc88009780).stop 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 -- 172.21.15.92:0/3807209745 shutdown_connections 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 --2- 172.21.15.92:0/3807209745 >> v2:172.21.15.158:3300/0 conn(0x7fcc90153bb0 0x7fcc900698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 --2- 172.21.15.92:0/3807209745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc900bdf20 0x7fcc90067f60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 -- 172.21.15.92:0/3807209745 >> 172.21.15.92:0/3807209745 conn(0x7fcc90062d80 msgr2=0x7fcc9005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 -- 172.21.15.92:0/3807209745 shutdown_connections 2023-05-16T07:38:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.265+0000 7fcc9523c700 1 -- 172.21.15.92:0/3807209745 wait complete. 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.266+0000 7fcc9523c700 1 Processor -- start 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.266+0000 7fcc9523c700 1 -- start start 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc9523c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc900bdf20 0x7fcc900c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc9523c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc90153bb0 0x7fcc901574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc9523c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 0x7fcc901579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc9523c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90157eb0 con 0x7fcc90153f90 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc9523c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900c7d00 con 0x7fcc900bdf20 2023-05-16T07:38:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc9523c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901659b0 con 0x7fcc90153bb0 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 0x7fcc901579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 0x7fcc901579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52220/0 (socket says 172.21.15.92:52220) 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 -- 172.21.15.92:0/26572986 learned_addr learned my addr 172.21.15.92:0/26572986 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc8ffff700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc900bdf20 0x7fcc900c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc8f7fe700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc90153bb0 0x7fcc901574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 -- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc90153bb0 msgr2=0x7fcc901574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc90153bb0 0x7fcc901574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 -- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc900bdf20 msgr2=0x7fcc900c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc900bdf20 0x7fcc900c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 -- 172.21.15.92:0/26572986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88009a20 con 0x7fcc90153f90 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc8f7fe700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc90153bb0 0x7fcc901574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.267+0000 7fcc94a3b700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 0x7fcc901579c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fcc90065700 tx=0x7fcc88009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.268+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8801d630 con 0x7fcc90153f90 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.268+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8801d790 con 0x7fcc90153f90 2023-05-16T07:38:05.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.268+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc901653a0 con 0x7fcc90153f90 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.268+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8801db30 con 0x7fcc90153f90 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.268+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc90165640 con 0x7fcc90153f90 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.269+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc90047150 con 0x7fcc90153f90 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.269+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7fcc88026410 con 0x7fcc90153f90 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.269+0000 7fcc77fff700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcc7c05eeb0 0x7fcc7c061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.269+0000 7fcc8ffff700 1 -- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcc7c05eeb0 msgr2=0x7fcc7c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.269+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7fcc8807bed0 con 0x7fcc90153f90 2023-05-16T07:38:05.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.269+0000 7fcc8ffff700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcc7c05eeb0 0x7fcc7c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:05.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.273+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fcc8804d030 con 0x7fcc90153f90 2023-05-16T07:38:05.299 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:05.296+0000 7ff8e25c3680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:38:05.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.442+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcc90165af0 con 0x7fcc90153f90 2023-05-16T07:38:05.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.443+0000 7fcc77fff700 1 -- 172.21.15.92:0/26572986 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v157) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcc88050650 con 0x7fcc90153f90 2023-05-16T07:38:05.446 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:38:05.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcc7c05eeb0 msgr2=0x7fcc7c061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:05.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcc7c05eeb0 0x7fcc7c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 msgr2=0x7fcc901579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 0x7fcc901579c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fcc90065700 tx=0x7fcc88009780).stop 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 shutdown_connections 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc90153f90 0x7fcc901579c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcc900bdf20 0x7fcc900c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fcc7c05eeb0 0x7fcc7c061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:05.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 --2- 172.21.15.92:0/26572986 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcc90153bb0 0x7fcc901574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 >> 172.21.15.92:0/26572986 conn(0x7fcc90062d80 msgr2=0x7fcc9005dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:05.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 shutdown_connections 2023-05-16T07:38:05.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.445+0000 7fcc9523c700 1 -- 172.21.15.92:0/26572986 wait complete. 2023-05-16T07:38:05.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:38:05.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.575+0000 7f4667a82700 1 Processor -- start 2023-05-16T07:38:05.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.575+0000 7f4667a82700 1 -- start start 2023-05-16T07:38:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f4667a82700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46600c5d30 0x7f46600aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f4667a82700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4660153aa0 0x7f4660153f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f4667a82700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600c8830 con 0x7f46600c6110 2023-05-16T07:38:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f4667a82700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600ca600 con 0x7f46600c5d30 2023-05-16T07:38:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f4667a82700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600c8b10 con 0x7f4660153aa0 2023-05-16T07:38:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46600c5d30 0x7f46600aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46600c5d30 0x7f46600aa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52246/0 (socket says 172.21.15.92:52246) 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 -- 172.21.15.92:0/2561066900 learned_addr learned my addr 172.21.15.92:0/2561066900 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466601f700 1 --2- 172.21.15.92:0/2561066900 >> v2:172.21.15.158:3300/0 conn(0x7f4660153aa0 0x7f4660153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 -- 172.21.15.92:0/2561066900 >> v2:172.21.15.158:3300/0 conn(0x7f4660153aa0 msgr2=0x7f4660153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 --2- 172.21.15.92:0/2561066900 >> v2:172.21.15.158:3300/0 conn(0x7f4660153aa0 0x7f4660153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 -- 172.21.15.92:0/2561066900 >> v1:172.21.15.158:6789/0 conn(0x7f46600c6110 legacy=0x7f4660069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.576+0000 7f466581e700 1 -- 172.21.15.92:0/2561066900 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600c8df0 con 0x7f46600c5d30 2023-05-16T07:38:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f466581e700 1 --2- 172.21.15.92:0/2561066900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46600c5d30 0x7f46600aa210 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f466006bcd0 tx=0x7f46500090f0).ready entity=mon.0 client_cookie=b8cae39fbd2376fe server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f466481c700 1 -- 172.21.15.92:0/2561066900 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465000b5a0 con 0x7f46600c5d30 2023-05-16T07:38:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f466481c700 1 -- 172.21.15.92:0/2561066900 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465000b700 con 0x7f46600c5d30 2023-05-16T07:38:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f466481c700 1 -- 172.21.15.92:0/2561066900 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465000ba80 con 0x7f46600c5d30 2023-05-16T07:38:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 -- 172.21.15.92:0/2561066900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46600c5d30 msgr2=0x7f46600aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 --2- 172.21.15.92:0/2561066900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46600c5d30 0x7f46600aa210 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f466006bcd0 tx=0x7f46500090f0).stop 2023-05-16T07:38:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 -- 172.21.15.92:0/2561066900 shutdown_connections 2023-05-16T07:38:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 --2- 172.21.15.92:0/2561066900 >> v2:172.21.15.158:3300/0 conn(0x7f4660153aa0 0x7f4660153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 --2- 172.21.15.92:0/2561066900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46600c5d30 0x7f46600aa210 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 -- 172.21.15.92:0/2561066900 >> 172.21.15.92:0/2561066900 conn(0x7f46600cb700 msgr2=0x7f46600c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 -- 172.21.15.92:0/2561066900 shutdown_connections 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.577+0000 7f4667a82700 1 -- 172.21.15.92:0/2561066900 wait complete. 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.578+0000 7f4667a82700 1 Processor -- start 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.578+0000 7f4667a82700 1 -- start start 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f4667a82700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46600c5d30 0x7f46601400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f4667a82700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f46600c6110 0x7f46600a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f4667a82700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 0x7f4660150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f4667a82700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4660151720 con 0x7f4660153aa0 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f4667a82700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4660151a00 con 0x7f46600c6110 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f4667a82700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4660151ce0 con 0x7f46600c5d30 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f466601f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 0x7f4660150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f466601f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 0x7f4660150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52254/0 (socket says 172.21.15.92:52254) 2023-05-16T07:38:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f466601f700 1 -- 172.21.15.92:0/3492504845 learned_addr learned my addr 172.21.15.92:0/3492504845 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f466501d700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f46600c6110 0x7f46600a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.579+0000 7f466581e700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46600c5d30 0x7f46601400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f466601f700 1 -- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46600c5d30 msgr2=0x7f46601400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f466601f700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46600c5d30 0x7f46601400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f466601f700 1 -- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f46600c6110 msgr2=0x7f46600a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f466601f700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f46600c6110 0x7f46600a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f466601f700 1 -- 172.21.15.92:0/3492504845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650009370 con 0x7f4660153aa0 2023-05-16T07:38:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f466601f700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 0x7f4660150f10 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f4658007f20 tx=0x7f4658009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800eb90 con 0x7f4660153aa0 2023-05-16T07:38:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465800ecf0 con 0x7f4660153aa0 2023-05-16T07:38:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4660151fc0 con 0x7f4660153aa0 2023-05-16T07:38:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658014630 con 0x7f4660153aa0 2023-05-16T07:38:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.580+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4660151400 con 0x7f4660153aa0 2023-05-16T07:38:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.581+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f465801b030 con 0x7f4660153aa0 2023-05-16T07:38:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.581+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46601524e0 con 0x7f4660153aa0 2023-05-16T07:38:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.582+0000 7f4655ffb700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f464405eeb0 0x7f4644061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.582+0000 7f466581e700 1 -- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f464405eeb0 msgr2=0x7f4644061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.582+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7f465807a060 con 0x7f4660153aa0 2023-05-16T07:38:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.582+0000 7f466581e700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f464405eeb0 0x7f4644061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.586+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f46601524e0 con 0x7f4660153aa0 2023-05-16T07:38:05.669 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:05.665+0000 7f11331ca680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:38:05.733 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:05.730+0000 7ff8e25c3680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:38:05.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.763+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46601524e0 con 0x7f4660153aa0 2023-05-16T07:38:05.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.763+0000 7f4655ffb700 1 -- 172.21.15.92:0/3492504845 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46601524e0 con 0x7f4660153aa0 2023-05-16T07:38:05.767 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:38:05.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f464405eeb0 msgr2=0x7f4644061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:05.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f464405eeb0 0x7f4644061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:05.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 msgr2=0x7f4660150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 0x7f4660150f10 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f4658007f20 tx=0x7f4658009860).stop 2023-05-16T07:38:05.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 shutdown_connections 2023-05-16T07:38:05.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f464405eeb0 0x7f4644061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:05.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f46600c5d30 0x7f46601400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4660153aa0 0x7f4660150f10 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 --2- 172.21.15.92:0/3492504845 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f46600c6110 0x7f46600a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 >> 172.21.15.92:0/3492504845 conn(0x7f46600cb700 msgr2=0x7f466014f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:05.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 shutdown_connections 2023-05-16T07:38:05.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.765+0000 7f4667a82700 1 -- 172.21.15.92:0/3492504845 wait complete. 2023-05-16T07:38:05.771 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:05.765+0000 7f11331ca680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:05.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:38:05.840 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:05.837+0000 7ff8e25c3680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:05.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.893+0000 7fcef72e8700 1 Processor -- start 2023-05-16T07:38:05.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.893+0000 7fcef72e8700 1 -- start start 2023-05-16T07:38:05.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef72e8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcef0155140 0x7fcef00622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef72e8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fcef00627a0 0x7fcef0063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef72e8700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef014ead0 con 0x7fcef014e6f0 2023-05-16T07:38:05.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef72e8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef0154cb0 con 0x7fcef0155140 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef72e8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00c6020 con 0x7fcef00627a0 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef4883700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcef0155140 0x7fcef00622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef4883700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcef0155140 0x7fcef00622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52260/0 (socket says 172.21.15.92:52260) 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.894+0000 7fcef4883700 1 -- 172.21.15.92:0/2049634501 learned_addr learned my addr 172.21.15.92:0/2049634501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef5885700 1 --2- 172.21.15.92:0/2049634501 >> v2:172.21.15.158:3300/0 conn(0x7fcef00627a0 0x7fcef0063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef4883700 1 -- 172.21.15.92:0/2049634501 >> v2:172.21.15.158:3300/0 conn(0x7fcef00627a0 msgr2=0x7fcef0063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:05.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef4883700 1 --2- 172.21.15.92:0/2049634501 >> v2:172.21.15.158:3300/0 conn(0x7fcef00627a0 0x7fcef0063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef4883700 1 -- 172.21.15.92:0/2049634501 >> v1:172.21.15.158:6789/0 conn(0x7fcef014e6f0 legacy=0x7fcef00c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef4883700 1 -- 172.21.15.92:0/2049634501 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0154660 con 0x7fcef0155140 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef5885700 1 --2- 172.21.15.92:0/2049634501 >> v2:172.21.15.158:3300/0 conn(0x7fcef00627a0 0x7fcef0063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcef4883700 1 --2- 172.21.15.92:0/2049634501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef00622b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fcef00647e0 tx=0x7fcee00090f0).ready entity=mon.0 client_cookie=8d0113d04f75c7f3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcee7fff700 1 -- 172.21.15.92:0/2049634501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000b600 con 0x7fcef0155140 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcee7fff700 1 -- 172.21.15.92:0/2049634501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee000b760 con 0x7fcef0155140 2023-05-16T07:38:05.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.895+0000 7fcee7fff700 1 -- 172.21.15.92:0/2049634501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000bae0 con 0x7fcef0155140 2023-05-16T07:38:05.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 -- 172.21.15.92:0/2049634501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 msgr2=0x7fcef00622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 --2- 172.21.15.92:0/2049634501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef00622b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fcef00647e0 tx=0x7fcee00090f0).stop 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 -- 172.21.15.92:0/2049634501 shutdown_connections 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 --2- 172.21.15.92:0/2049634501 >> v2:172.21.15.158:3300/0 conn(0x7fcef00627a0 0x7fcef0063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 --2- 172.21.15.92:0/2049634501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef00622b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 -- 172.21.15.92:0/2049634501 >> 172.21.15.92:0/2049634501 conn(0x7fcef005e820 msgr2=0x7fcef00c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 -- 172.21.15.92:0/2049634501 shutdown_connections 2023-05-16T07:38:05.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.896+0000 7fcef72e8700 1 -- 172.21.15.92:0/2049634501 wait complete. 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.897+0000 7fcef72e8700 1 Processor -- start 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.897+0000 7fcef72e8700 1 -- start start 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef72e8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcef00627a0 0x7fcef00adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef72e8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcef014e6f0 0x7fcef01661b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef72e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef0167da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef72e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00adfc0 con 0x7fcef0155140 2023-05-16T07:38:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef72e8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00ae220 con 0x7fcef014e6f0 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef72e8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00a8610 con 0x7fcef00627a0 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef0167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef0167da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52268/0 (socket says 172.21.15.92:52268) 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 -- 172.21.15.92:0/474430468 learned_addr learned my addr 172.21.15.92:0/474430468 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef4883700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcef014e6f0 0x7fcef01661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5084700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcef00627a0 0x7fcef00adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 -- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcef00627a0 msgr2=0x7fcef00adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcef00627a0 0x7fcef00adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 -- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcef014e6f0 msgr2=0x7fcef01661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcef014e6f0 0x7fcef01661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 -- 172.21.15.92:0/474430468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0009370 con 0x7fcef0155140 2023-05-16T07:38:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcef5885700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef0167da0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fceec00d860 tx=0x7fceec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011590 con 0x7fcef0155140 2023-05-16T07:38:05.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.898+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0116f0 con 0x7fcef0155140 2023-05-16T07:38:05.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.899+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef00a88f0 con 0x7fcef0155140 2023-05-16T07:38:05.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.899+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011a90 con 0x7fcef0155140 2023-05-16T07:38:05.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.899+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef00a8eb0 con 0x7fcef0155140 2023-05-16T07:38:05.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.899+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef00a90a0 con 0x7fcef0155140 2023-05-16T07:38:05.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.900+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7fceec012420 con 0x7fcef0155140 2023-05-16T07:38:05.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.900+0000 7fcee57fa700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fced405eeb0 0x7fced4061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:05.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.900+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6991+0+0 (secure 0 0 0) 0x7fceec079b00 con 0x7fcef0155140 2023-05-16T07:38:05.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.900+0000 7fcef5084700 1 -- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fced405eeb0 msgr2=0x7fced4061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:05.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.900+0000 7fcef5084700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fced405eeb0 0x7fced4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:05.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:05.904+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fceec04ac60 con 0x7fcef0155140 2023-05-16T07:38:05.921 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:05.917+0000 7f11331ca680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:06.000 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:05.997+0000 7ff8e25c3680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:06.001 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:05.997+0000 7f11331ca680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:38:06.068 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:06.064+0000 7f11331ca680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:38:06.077 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:06.074+0000 7ff8e25c3680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:38:06.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.079+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcef00bf510 con 0x7fcef0155140 2023-05-16T07:38:06.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.079+0000 7fcee57fa700 1 -- 172.21.15.92:0/474430468 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+11 (secure 0 0 0) 0x7fceec04e280 con 0x7fcef0155140 2023-05-16T07:38:06.082 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:38:06.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fced405eeb0 msgr2=0x7fced4061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fced405eeb0 0x7fced4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 msgr2=0x7fcef0167da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef0167da0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fceec00d860 tx=0x7fceec008040).stop 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 shutdown_connections 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fced405eeb0 0x7fced4061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fcef00627a0 0x7fcef00adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcef0155140 0x7fcef0167da0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 --2- 172.21.15.92:0/474430468 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcef014e6f0 0x7fcef01661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 >> 172.21.15.92:0/474430468 conn(0x7fcef005e820 msgr2=0x7fcef0063600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 shutdown_connections 2023-05-16T07:38:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.081+0000 7fcef72e8700 1 -- 172.21.15.92:0/474430468 wait complete. 2023-05-16T07:38:06.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:06.149 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:06.146+0000 7ff8e25c3680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:38:06.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.207+0000 7fa2010d9700 1 Processor -- start 2023-05-16T07:38:06.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.207+0000 7fa2010d9700 1 -- start start 2023-05-16T07:38:06.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.208+0000 7fa2010d9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fa1fc1584a0 0x7fa1fc0646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.208+0000 7fa2010d9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc0c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.208+0000 7fa2010d9700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc158880 con 0x7fa1fc0c1fa0 2023-05-16T07:38:06.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.208+0000 7fa2010d9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c1570 con 0x7fa1fc1584a0 2023-05-16T07:38:06.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.208+0000 7fa2010d9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0629c0 con 0x7fa1fc05e1b0 2023-05-16T07:38:06.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fb59e700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fa1fc0c1fa0 0x7fa1fc067a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:38390/0 (socket says 172.21.15.92:38390) 2023-05-16T07:38:06.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fb59e700 1 -- 172.21.15.92:0/2179605188 learned_addr learned my addr 172.21.15.92:0/2179605188 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fad9d700 1 --2- 172.21.15.92:0/2179605188 >> v2:172.21.15.158:3300/0 conn(0x7fa1fc1584a0 0x7fa1fc0646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fa59c700 1 --2- 172.21.15.92:0/2179605188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc0c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fa59c700 1 -- 172.21.15.92:0/2179605188 >> v2:172.21.15.158:3300/0 conn(0x7fa1fc1584a0 msgr2=0x7fa1fc0646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fa59c700 1 --2- 172.21.15.92:0/2179605188 >> v2:172.21.15.158:3300/0 conn(0x7fa1fc1584a0 0x7fa1fc0646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fa59c700 1 -- 172.21.15.92:0/2179605188 >> v1:172.21.15.158:6789/0 conn(0x7fa1fc0c1fa0 legacy=0x7fa1fc067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.209+0000 7fa1fa59c700 1 -- 172.21.15.92:0/2179605188 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc062c10 con 0x7fa1fc05e1b0 2023-05-16T07:38:06.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.210+0000 7fa1fa59c700 1 --2- 172.21.15.92:0/2179605188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc0c1ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa1ec009e60 tx=0x7fa1ec009780).ready entity=mon.2 client_cookie=8308428b1a72af21 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.210+0000 7fa1f9d9b700 1 -- 172.21.15.92:0/2179605188 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00e070 con 0x7fa1fc05e1b0 2023-05-16T07:38:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.210+0000 7fa1f9d9b700 1 -- 172.21.15.92:0/2179605188 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec00bb60 con 0x7fa1fc05e1b0 2023-05-16T07:38:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.210+0000 7fa1f9d9b700 1 -- 172.21.15.92:0/2179605188 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec012410 con 0x7fa1fc05e1b0 2023-05-16T07:38:06.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 -- 172.21.15.92:0/2179605188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 msgr2=0x7fa1fc0c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 --2- 172.21.15.92:0/2179605188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc0c1ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa1ec009e60 tx=0x7fa1ec009780).stop 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 -- 172.21.15.92:0/2179605188 shutdown_connections 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 --2- 172.21.15.92:0/2179605188 >> v2:172.21.15.158:3300/0 conn(0x7fa1fc1584a0 0x7fa1fc0646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 --2- 172.21.15.92:0/2179605188 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc0c1ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 -- 172.21.15.92:0/2179605188 >> 172.21.15.92:0/2179605188 conn(0x7fa1fc0c4680 msgr2=0x7fa1fc0622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 -- 172.21.15.92:0/2179605188 shutdown_connections 2023-05-16T07:38:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.211+0000 7fa2010d9700 1 -- 172.21.15.92:0/2179605188 wait complete. 2023-05-16T07:38:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.213+0000 7fa2010d9700 1 Processor -- start 2023-05-16T07:38:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.213+0000 7fa2010d9700 1 -- start start 2023-05-16T07:38:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa2010d9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa2010d9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1fc0c1fa0 0x7fa1fc16c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa2010d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 0x7fa1fc16c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa2010d9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc169bb0 con 0x7fa1fc1584a0 2023-05-16T07:38:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa2010d9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0ab2c0 con 0x7fa1fc0c1fa0 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa2010d9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc15b470 con 0x7fa1fc05e1b0 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 0x7fa1fc16c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fad9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 0x7fa1fc16c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52280/0 (socket says 172.21.15.92:52280) 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fad9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:44850/0 (socket says 172.21.15.92:44850) 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 -- 172.21.15.92:0/3823922665 learned_addr learned my addr 172.21.15.92:0/3823922665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:06.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fa59c700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1fc0c1fa0 0x7fa1fc16c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 -- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 msgr2=0x7fa1fc158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 -- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1fc0c1fa0 msgr2=0x7fa1fc16c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1fc0c1fa0 0x7fa1fc16c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 -- 172.21.15.92:0/3823922665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009a20 con 0x7fa1fc1584a0 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fad9d700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:06.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fa59c700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1fc0c1fa0 0x7fa1fc16c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1fb59e700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 0x7fa1fc16c730 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fa1e400c410 tx=0x7fa1e4007d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.214+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40070d0 con 0x7fa1fc1584a0 2023-05-16T07:38:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.215+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4007230 con 0x7fa1fc1584a0 2023-05-16T07:38:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.215+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc15b750 con 0x7fa1fc1584a0 2023-05-16T07:38:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.215+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40075b0 con 0x7fa1fc1584a0 2023-05-16T07:38:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.215+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc15b070 con 0x7fa1fc1584a0 2023-05-16T07:38:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.216+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7fa1e401a060 con 0x7fa1fc1584a0 2023-05-16T07:38:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.216+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc15bc70 con 0x7fa1fc1584a0 2023-05-16T07:38:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.216+0000 7fa1f2ffd700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 0x7fa1dc061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.216+0000 7fa1fad9d700 1 -- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 msgr2=0x7fa1dc061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.216+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7047+0+0 (secure 0 0 0) 0x7fa1e40791c0 con 0x7fa1fc1584a0 2023-05-16T07:38:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.216+0000 7fa1fad9d700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 0x7fa1dc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.220+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa1fc15bc70 con 0x7fa1fc1584a0 2023-05-16T07:38:06.329 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:06.325+0000 7f11331ca680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:38:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.417+0000 7fa1fad9d700 1 -- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 msgr2=0x7fa1dc061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.417+0000 7fa1fad9d700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 0x7fa1dc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:06.428 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:06.425+0000 7ff8e25c3680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:38:06.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.447+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1fc0ab090 con 0x7fa1fc1584a0 2023-05-16T07:38:06.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.448+0000 7fa1f2ffd700 1 -- 172.21.15.92:0/3823922665 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fa1fc0ab090 con 0x7fa1fc1584a0 2023-05-16T07:38:06.451 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:06.452 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:36:34.963007+0000","modified":"2023-05-16T07:36:36.970661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/1435424518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1435424518},{"type":"v1","addr":"172.21.15.158:6837","nonce":1435424518}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-05-16T07:38:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 msgr2=0x7fa1dc061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:06.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 0x7fa1dc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 msgr2=0x7fa1fc16c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 0x7fa1fc16c730 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fa1e400c410 tx=0x7fa1e4007d30).stop 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 shutdown_connections 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa1dc05eeb0 0x7fa1dc061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa1fc05e1b0 0x7fa1fc158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1fc1584a0 0x7fa1fc16c730 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 --2- 172.21.15.92:0/3823922665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1fc0c1fa0 0x7fa1fc16c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 >> 172.21.15.92:0/3823922665 conn(0x7fa1fc0c4680 msgr2=0x7fa1fc0c5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:06.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 shutdown_connections 2023-05-16T07:38:06.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.450+0000 7fa2010d9700 1 -- 172.21.15.92:0/3823922665 wait complete. 2023-05-16T07:38:06.456 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-05-16T07:38:06.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:06.513 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:06.509+0000 7f11331ca680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.580+0000 7ff00c8c3700 1 Processor -- start 2023-05-16T07:38:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.580+0000 7ff00c8c3700 1 -- start start 2023-05-16T07:38:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00c8c3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff008150b50 0x7ff008150f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00c8c3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00c8c3700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0080a61d0 con 0x7ff008068f10 2023-05-16T07:38:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00c8c3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff008066330 con 0x7ff008150b50 2023-05-16T07:38:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00c8c3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080bd240 con 0x7ff0080bc8f0 2023-05-16T07:38:06.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff008150b50 0x7ff008150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff008150b50 0x7ff008150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52288/0 (socket says 172.21.15.92:52288) 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 -- 172.21.15.92:0/1998304205 learned_addr learned my addr 172.21.15.92:0/1998304205 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff006d9d700 1 --2- 172.21.15.92:0/1998304205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 -- 172.21.15.92:0/1998304205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 msgr2=0x7ff0080bcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 --2- 172.21.15.92:0/1998304205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080bcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 -- 172.21.15.92:0/1998304205 >> v1:172.21.15.158:6789/0 conn(0x7ff008068f10 legacy=0x7ff0080a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 -- 172.21.15.92:0/1998304205 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008069410 con 0x7ff008150b50 2023-05-16T07:38:06.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00659c700 1 --2- 172.21.15.92:0/1998304205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff008150f30 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7feff8006fd0 tx=0x7feff80090f0).ready entity=mon.0 client_cookie=90494c5b374502d9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00559a700 1 -- 172.21.15.92:0/1998304205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800b600 con 0x7ff008150b50 2023-05-16T07:38:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00559a700 1 -- 172.21.15.92:0/1998304205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff800b760 con 0x7ff008150b50 2023-05-16T07:38:06.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.581+0000 7ff00559a700 1 -- 172.21.15.92:0/1998304205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800bae0 con 0x7ff008150b50 2023-05-16T07:38:06.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 -- 172.21.15.92:0/1998304205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 msgr2=0x7ff008150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/1998304205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff008150f30 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7feff8006fd0 tx=0x7feff80090f0).stop 2023-05-16T07:38:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 -- 172.21.15.92:0/1998304205 shutdown_connections 2023-05-16T07:38:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/1998304205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff008150f30 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/1998304205 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 -- 172.21.15.92:0/1998304205 >> 172.21.15.92:0/1998304205 conn(0x7ff0080c7350 msgr2=0x7ff0080c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 -- 172.21.15.92:0/1998304205 shutdown_connections 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.582+0000 7ff00c8c3700 1 -- 172.21.15.92:0/1998304205 wait complete. 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.583+0000 7ff00c8c3700 1 Processor -- start 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.583+0000 7ff00c8c3700 1 -- start start 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00c8c3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff008068f10 0x7ff008141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00c8c3700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00c8c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff0080a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00c8c3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080a5260 con 0x7ff008150b50 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00c8c3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00814ca60 con 0x7ff008068f10 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00c8c3700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff008141540 con 0x7ff0080bc8f0 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff0080a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff0080a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52298/0 (socket says 172.21.15.92:52298) 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 -- 172.21.15.92:0/2997669770 learned_addr learned my addr 172.21.15.92:0/2997669770 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff00659c700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff008068f10 0x7ff008141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff005d9b700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 -- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 msgr2=0x7ff0080a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 -- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff008068f10 msgr2=0x7ff008141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff008068f10 0x7ff008141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 -- 172.21.15.92:0/2997669770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009370 con 0x7ff008150b50 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.584+0000 7ff006d9d700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff0080a4d70 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7ff000009fa0 tx=0x7ff000009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.585+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000ebc0 con 0x7ff008150b50 2023-05-16T07:38:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.585+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00000ed20 con 0x7ff008150b50 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.585+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000014610 con 0x7ff008150b50 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.585+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0080a73c0 con 0x7ff008150b50 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.585+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0080a7a00 con 0x7ff008150b50 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.586+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00806c350 con 0x7ff008150b50 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.586+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7ff00001b030 con 0x7ff008150b50 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.586+0000 7feff67fc700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fefe805eeb0 0x7fefe8061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.586+0000 7ff00659c700 1 -- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fefe805eeb0 msgr2=0x7fefe8061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.586+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7047+0+0 (secure 0 0 0) 0x7ff00007a300 con 0x7ff008150b50 2023-05-16T07:38:06.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.586+0000 7ff00659c700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fefe805eeb0 0x7fefe8061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:06.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.590+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff00004b420 con 0x7ff008150b50 2023-05-16T07:38:06.617 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:06.614+0000 7ff8e25c3680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.756+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0080c3e20 con 0x7ff008150b50 2023-05-16T07:38:06.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.757+0000 7feff67fc700 1 -- 172.21.15.92:0/2997669770 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+18420 (secure 0 0 0) 0x7ff00004ea40 con 0x7ff008150b50 2023-05-16T07:38:06.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:06.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":158,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:05.917821+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:36:33.491970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-05-16T07:36:33.931369+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:6833/95623":"2023-05-17T07:36:46.188304+0000","172.21.15.158:6832/95623":"2023-05-17T07:36:46.188304+0000","172.21.15.158:0/2052480298":"2023-05-17T07:36:46.188304+0000","172.21.15.158:0/2894032393":"2023-05-17T07:36:46.188304+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fefe805eeb0 msgr2=0x7fefe8061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:06.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fefe805eeb0 0x7fefe8061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 msgr2=0x7ff0080a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff0080a4d70 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7ff000009fa0 tx=0x7ff000009900).stop 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 shutdown_connections 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff008150b50 0x7ff0080a4d70 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff008068f10 0x7ff008141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fefe805eeb0 0x7fefe8061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:06.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 --2- 172.21.15.92:0/2997669770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff0080bc8f0 0x7ff0080a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 >> 172.21.15.92:0/2997669770 conn(0x7ff0080c7350 msgr2=0x7ff0080c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 shutdown_connections 2023-05-16T07:38:06.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.759+0000 7ff00c8c3700 1 -- 172.21.15.92:0/2997669770 wait complete. 2023-05-16T07:38:06.774 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:38:06.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.886+0000 7f770734c700 1 Processor -- start 2023-05-16T07:38:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.886+0000 7f770734c700 1 -- start start 2023-05-16T07:38:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.887+0000 7f770734c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7700153eb0 0x7f77000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.887+0000 7f770734c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7700154290 0x7f77000a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.887+0000 7f770734c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77000bb720 0x7f77000a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.887+0000 7f770734c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000a5260 con 0x7f77000bb720 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.887+0000 7f770734c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000c30d0 con 0x7f7700154290 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.887+0000 7f770734c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000a5d60 con 0x7f7700153eb0 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77000bb720 0x7f77000a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77000bb720 0x7f77000a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52306/0 (socket says 172.21.15.92:52306) 2023-05-16T07:38:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 -- 172.21.15.92:0/1461691167 learned_addr learned my addr 172.21.15.92:0/1461691167 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77048e7700 1 --2- 172.21.15.92:0/1461691167 >> v2:172.21.15.158:3300/0 conn(0x7f7700154290 0x7f77000a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77050e8700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7700153eb0 0x7f77000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 -- 172.21.15.92:0/1461691167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7700153eb0 msgr2=0x7f77000a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7700153eb0 0x7f77000a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 -- 172.21.15.92:0/1461691167 >> v2:172.21.15.158:3300/0 conn(0x7f7700154290 msgr2=0x7f77000a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 --2- 172.21.15.92:0/1461691167 >> v2:172.21.15.158:3300/0 conn(0x7f7700154290 0x7f77000a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 -- 172.21.15.92:0/1461691167 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000bbc70 con 0x7f77000bb720 2023-05-16T07:38:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77050e8700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7700153eb0 0x7f77000a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f77058e9700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77000bb720 0x7f77000a4d70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f770005ede0 tx=0x7f76f8009840).ready entity=mon.0 client_cookie=5cdcdbc1dc4d31ee server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f76f7fff700 1 -- 172.21.15.92:0/1461691167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800e070 con 0x7f77000bb720 2023-05-16T07:38:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f76f7fff700 1 -- 172.21.15.92:0/1461691167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f800ba20 con 0x7f77000bb720 2023-05-16T07:38:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.888+0000 7f76f7fff700 1 -- 172.21.15.92:0/1461691167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800bda0 con 0x7f77000bb720 2023-05-16T07:38:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 -- 172.21.15.92:0/1461691167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77000bb720 msgr2=0x7f77000a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77000bb720 0x7f77000a4d70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f770005ede0 tx=0x7f76f8009840).stop 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 -- 172.21.15.92:0/1461691167 shutdown_connections 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77000bb720 0x7f77000a4d70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 --2- 172.21.15.92:0/1461691167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7700153eb0 0x7f77000a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 --2- 172.21.15.92:0/1461691167 >> v2:172.21.15.158:3300/0 conn(0x7f7700154290 0x7f77000a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 -- 172.21.15.92:0/1461691167 >> 172.21.15.92:0/1461691167 conn(0x7f77000c7290 msgr2=0x7f77000c3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 -- 172.21.15.92:0/1461691167 shutdown_connections 2023-05-16T07:38:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.889+0000 7f770734c700 1 -- 172.21.15.92:0/1461691167 wait complete. 2023-05-16T07:38:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.891+0000 7f770734c700 1 Processor -- start 2023-05-16T07:38:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.891+0000 7f770734c700 1 -- start start 2023-05-16T07:38:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.892+0000 7f770734c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77000bb720 0x7f77000a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7700153eb0 0x7f77000a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 0x7f7700155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000a7c10 con 0x7f7700154290 2023-05-16T07:38:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000a7ef0 con 0x7f7700153eb0 2023-05-16T07:38:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000a81d0 con 0x7f77000bb720 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 0x7f7700155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 0x7f7700155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52310/0 (socket says 172.21.15.92:52310) 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 -- 172.21.15.92:0/4035144499 learned_addr learned my addr 172.21.15.92:0/4035144499 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77048e7700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7700153eb0 0x7f77000a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77050e8700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77000bb720 0x7f77000a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 -- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77000bb720 msgr2=0x7f77000a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77000bb720 0x7f77000a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 -- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7700153eb0 msgr2=0x7f77000a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7700153eb0 0x7f77000a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 -- 172.21.15.92:0/4035144499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8009ae0 con 0x7f7700154290 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f77058e9700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 0x7f7700155600 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f76f800b6c0 tx=0x7f76f8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800bda0 con 0x7f7700154290 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f800b6f0 con 0x7f7700154290 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77000a84b0 con 0x7f7700154290 2023-05-16T07:38:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f801d6b0 con 0x7f7700154290 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.893+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7700155af0 con 0x7f7700154290 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.894+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770006c350 con 0x7f7700154290 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.895+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f76f8024030 con 0x7f7700154290 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.895+0000 7f76f57fa700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 0x7f76e4061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.895+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f76f807b230 con 0x7f7700154290 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.895+0000 7f77050e8700 1 -- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 msgr2=0x7f76e4061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.895+0000 7f77050e8700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 0x7f76e4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:06.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:06.899+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f76f804c460 con 0x7f7700154290 2023-05-16T07:38:07.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.096+0000 7f77050e8700 1 -- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 msgr2=0x7f76e4061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.096+0000 7f77050e8700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 0x7f76e4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:07.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.108+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77000c4010 con 0x7f7700154290 2023-05-16T07:38:07.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.109+0000 7f76f57fa700 1 -- 172.21.15.92:0/4035144499 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f76f8022070 con 0x7f7700154290 2023-05-16T07:38:07.113 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:07.113 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:36:34.963007+0000","modified":"2023-05-16T07:36:36.970661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/1435424518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1435424518},{"type":"v1","addr":"172.21.15.158:6837","nonce":1435424518}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-05-16T07:38:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 msgr2=0x7f76e4061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 0x7f76e4061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 msgr2=0x7f7700155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 0x7f7700155600 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f76f800b6c0 tx=0x7f76f8009840).stop 2023-05-16T07:38:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 shutdown_connections 2023-05-16T07:38:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f76e405eeb0 0x7f76e4061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f77000bb720 0x7f77000a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7700154290 0x7f7700155600 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 --2- 172.21.15.92:0/4035144499 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7700153eb0 0x7f77000a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 >> 172.21.15.92:0/4035144499 conn(0x7f77000c7290 msgr2=0x7f77000c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:07.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 shutdown_connections 2023-05-16T07:38:07.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.111+0000 7f770734c700 1 -- 172.21.15.92:0/4035144499 wait complete. 2023-05-16T07:38:07.118 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-05-16T07:38:07.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:07.157 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.153+0000 7f11331ca680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:38:07.224 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.220+0000 7f11331ca680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:38:07.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.241+0000 7febf06e6700 1 Processor -- start 2023-05-16T07:38:07.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.241+0000 7febf06e6700 1 -- start start 2023-05-16T07:38:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febf06e6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febe813a5a0 0x7febe8068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febf06e6700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7febe80bd9a0 0x7febe80c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febf06e6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe8154b80 con 0x7febe8068060 2023-05-16T07:38:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febf06e6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe8154e60 con 0x7febe813a5a0 2023-05-16T07:38:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febf06e6700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe8155140 con 0x7febe80bd9a0 2023-05-16T07:38:07.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febe813a5a0 0x7febe8068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febe813a5a0 0x7febe8068600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52336/0 (socket says 172.21.15.92:52336) 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 -- 172.21.15.92:0/3416141223 learned_addr learned my addr 172.21.15.92:0/3416141223 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febedc81700 1 --1- 172.21.15.92:0/3416141223 >> v1:172.21.15.92:6789/0 conn(0x7febe8068060 0x7febe80bd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34852/0 (socket says 172.21.15.92:34852) 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febeec83700 1 --2- 172.21.15.92:0/3416141223 >> v2:172.21.15.158:3300/0 conn(0x7febe80bd9a0 0x7febe80c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 -- 172.21.15.92:0/3416141223 >> v2:172.21.15.158:3300/0 conn(0x7febe80bd9a0 msgr2=0x7febe80c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 --2- 172.21.15.92:0/3416141223 >> v2:172.21.15.158:3300/0 conn(0x7febe80bd9a0 0x7febe80c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 -- 172.21.15.92:0/3416141223 >> v1:172.21.15.92:6789/0 conn(0x7febe8068060 legacy=0x7febe80bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febed480700 1 -- 172.21.15.92:0/3416141223 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012650826 0 0) 0x7febe8154b80 con 0x7febe8068060 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 -- 172.21.15.92:0/3416141223 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8155420 con 0x7febe813a5a0 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febeec83700 1 --2- 172.21.15.92:0/3416141223 >> v2:172.21.15.158:3300/0 conn(0x7febe80bd9a0 0x7febe80c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.242+0000 7febee482700 1 --2- 172.21.15.92:0/3416141223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe813a5a0 0x7febe8068600 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7febe805eef0 tx=0x7febe400b3b0).ready entity=mon.0 client_cookie=cd8401f6752d180d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febed480700 1 -- 172.21.15.92:0/3416141223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe400db10 con 0x7febe813a5a0 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febed480700 1 -- 172.21.15.92:0/3416141223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe400dc70 con 0x7febe813a5a0 2023-05-16T07:38:07.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febed480700 1 -- 172.21.15.92:0/3416141223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4010470 con 0x7febe813a5a0 2023-05-16T07:38:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 -- 172.21.15.92:0/3416141223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe813a5a0 msgr2=0x7febe8068600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 --2- 172.21.15.92:0/3416141223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe813a5a0 0x7febe8068600 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7febe805eef0 tx=0x7febe400b3b0).stop 2023-05-16T07:38:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 -- 172.21.15.92:0/3416141223 shutdown_connections 2023-05-16T07:38:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 --2- 172.21.15.92:0/3416141223 >> v2:172.21.15.158:3300/0 conn(0x7febe80bd9a0 0x7febe80c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 --2- 172.21.15.92:0/3416141223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe813a5a0 0x7febe8068600 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 -- 172.21.15.92:0/3416141223 >> 172.21.15.92:0/3416141223 conn(0x7febe80c4430 msgr2=0x7febe80c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 -- 172.21.15.92:0/3416141223 shutdown_connections 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.243+0000 7febf06e6700 1 -- 172.21.15.92:0/3416141223 wait complete. 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.244+0000 7febf06e6700 1 Processor -- start 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.244+0000 7febf06e6700 1 -- start start 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febf06e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 0x7febe80ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febf06e6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febe80bd9a0 0x7febe80ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febf06e6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febe813a5a0 0x7febe81630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febf06e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80ac950 con 0x7febe8068060 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febf06e6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80ab9a0 con 0x7febe80bd9a0 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febf06e6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe814c4f0 con 0x7febe813a5a0 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febee482700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 0x7febe80ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febee482700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 0x7febe80ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52340/0 (socket says 172.21.15.92:52340) 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febee482700 1 -- 172.21.15.92:0/3105752196 learned_addr learned my addr 172.21.15.92:0/3105752196 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febedc81700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febe80bd9a0 0x7febe80ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.245+0000 7febeec83700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febe813a5a0 0x7febe81630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febee482700 1 -- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febe813a5a0 msgr2=0x7febe81630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febee482700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febe813a5a0 0x7febe81630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febee482700 1 -- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febe80bd9a0 msgr2=0x7febe80ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febee482700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febe80bd9a0 0x7febe80ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febee482700 1 -- 172.21.15.92:0/3105752196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe400b610 con 0x7febe8068060 2023-05-16T07:38:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febeec83700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febe813a5a0 0x7febe81630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febee482700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 0x7febe80ac460 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7febe4006df0 tx=0x7febe400b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe401f070 con 0x7febe8068060 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4000d10 con 0x7febe8068060 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe814bbc0 con 0x7febe8068060 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe40198b0 con 0x7febe8068060 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.246+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe814c180 con 0x7febe8068060 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.247+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe806c350 con 0x7febe8068060 2023-05-16T07:38:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.247+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7febe4016030 con 0x7febe8068060 2023-05-16T07:38:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.247+0000 7febde7fc700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 0x7febcc061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.248+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7047+0+0 (secure 0 0 0) 0x7febe407d980 con 0x7febe8068060 2023-05-16T07:38:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.248+0000 7febedc81700 1 -- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 msgr2=0x7febcc061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.248+0000 7febedc81700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 0x7febcc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:07.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.252+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7febe404eb00 con 0x7febe8068060 2023-05-16T07:38:07.287 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.284+0000 7ff8e25c3680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:38:07.298 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.294+0000 7f11331ca680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:38:07.356 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.353+0000 7ff8e25c3680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:38:07.431 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.428+0000 7ff8e25c3680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:38:07.442 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.438+0000 7f11331ca680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:38:07.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.449+0000 7febedc81700 1 -- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 msgr2=0x7febcc061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:07.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.449+0000 7febedc81700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 0x7febcc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:07.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.462+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febe8155700 con 0x7febe8068060 2023-05-16T07:38:07.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.462+0000 7febde7fc700 1 -- 172.21.15.92:0/3105752196 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4497 (secure 0 0 0) 0x7febe4052120 con 0x7febe8068060 2023-05-16T07:38:07.466 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:07.466 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:36:34.963007+0000","modified":"2023-05-16T07:36:36.970661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/1435424518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":1435424518},{"type":"v1","addr":"172.21.15.158:6837","nonce":1435424518}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-05-16T07:38:07.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.464+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 msgr2=0x7febcc061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:07.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 0x7febcc061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:07.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 msgr2=0x7febe80ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 0x7febe80ac460 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7febe4006df0 tx=0x7febe400b3b0).stop 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 shutdown_connections 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7febcc05eeb0 0x7febcc061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7febe813a5a0 0x7febe81630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe8068060 0x7febe80ac460 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 --2- 172.21.15.92:0/3105752196 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febe80bd9a0 0x7febe80ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 >> 172.21.15.92:0/3105752196 conn(0x7febe80c4430 msgr2=0x7febe80bd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 shutdown_connections 2023-05-16T07:38:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.465+0000 7febf06e6700 1 -- 172.21.15.92:0/3105752196 wait complete. 2023-05-16T07:38:07.470 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 55 2023-05-16T07:38:07.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:07.511 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.507+0000 7f11331ca680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:38:07.582 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.579+0000 7ff8e25c3680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:38:07.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.591+0000 7f2d30329700 1 Processor -- start 2023-05-16T07:38:07.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.591+0000 7f2d30329700 1 -- start start 2023-05-16T07:38:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d30329700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d2815f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d30329700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d280c0660 con 0x7f2d28154550 2023-05-16T07:38:07.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d30329700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d280c1b00 con 0x7f2d28154170 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d30329700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280bfd30 con 0x7f2d28069db0 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d2d8c4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d28154550 0x7f2d280696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34866/0 (socket says 172.21.15.92:34866) 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d2d8c4700 1 -- 172.21.15.92:0/2377204966 learned_addr learned my addr 172.21.15.92:0/2377204966 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.592+0000 7f2d2e8c6700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d2815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987533814 0 0) 0x7f2d280c0660 con 0x7f2d28154550 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d140033f0 con 0x7f2d28154550 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850372318 0 0) 0x7f2d280c1b00 con 0x7f2d28154170 2023-05-16T07:38:07.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d280c0660 con 0x7f2d28154170 2023-05-16T07:38:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284421740 0 0) 0x7f2d140033f0 con 0x7f2d28154550 2023-05-16T07:38:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 msgr2=0x7f2d2815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d2815f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 >> v1:172.21.15.158:6789/0 conn(0x7f2d28154170 legacy=0x7f2d280c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280c0010 con 0x7f2d28154550 2023-05-16T07:38:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f2d24001470 con 0x7f2d28154550 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e8c6700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d2815f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 >> v1:172.21.15.92:6789/0 conn(0x7f2d28154550 legacy=0x7f2d280696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d14003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d14004690 0x7f2d14004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d14005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280bfd30 con 0x7f2d14003aa0 2023-05-16T07:38:07.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400b990 con 0x7f2d14004690 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400bb20 con 0x7f2d28069db0 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d14003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e8c6700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d14005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2d8c4700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d14004690 0x7f2d14004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 msgr2=0x7f2d14005020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d28069db0 0x7f2d14005020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d14004690 msgr2=0x7f2d14004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d14004690 0x7f2d14004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/2377204966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d20009a20 con 0x7f2d14003aa0 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.593+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d14003ea0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f2d180018c0 tx=0x7f2d18007bb0).ready entity=mon.0 client_cookie=c40d2e2bea72f4a9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007020 con 0x7f2d14003aa0 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18007180 con 0x7f2d14003aa0 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d2d0c3700 1 -- 172.21.15.92:0/2377204966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007500 con 0x7f2d14003aa0 2023-05-16T07:38:07.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d30329700 1 -- 172.21.15.92:0/2377204966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 msgr2=0x7f2d14003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d30329700 1 --2- 172.21.15.92:0/2377204966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d14003ea0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f2d180018c0 tx=0x7f2d18007bb0).stop 2023-05-16T07:38:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d2e8c6700 1 -- 172.21.15.92:0/2377204966 reap_dead start 2023-05-16T07:38:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d30329700 1 -- 172.21.15.92:0/2377204966 shutdown_connections 2023-05-16T07:38:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d30329700 1 -- 172.21.15.92:0/2377204966 >> 172.21.15.92:0/2377204966 conn(0x7f2d280cb700 msgr2=0x7f2d280c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d30329700 1 -- 172.21.15.92:0/2377204966 shutdown_connections 2023-05-16T07:38:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.594+0000 7f2d30329700 1 -- 172.21.15.92:0/2377204966 wait complete. 2023-05-16T07:38:07.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.595+0000 7f2d30329700 1 Processor -- start 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.595+0000 7f2d30329700 1 -- start start 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d30329700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d281536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d30329700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2814c8b0 0x7f2d28141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d30329700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d281683b0 0x7f2d280ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d30329700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28141710 con 0x7f2d14003aa0 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d30329700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2814ce20 con 0x7f2d2814c8b0 2023-05-16T07:38:07.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d30329700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28152c00 con 0x7f2d281683b0 2023-05-16T07:38:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d281536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d281536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52362/0 (socket says 172.21.15.92:52362) 2023-05-16T07:38:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/3703187545 learned_addr learned my addr 172.21.15.92:0/3703187545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e8c6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d281683b0 0x7f2d280ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2d8c4700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2814c8b0 0x7f2d28141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d281683b0 msgr2=0x7f2d280ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d281683b0 0x7f2d280ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2814c8b0 msgr2=0x7f2d28141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2814c8b0 0x7f2d28141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.596+0000 7f2d2e0c5700 1 -- 172.21.15.92:0/3703187545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28152ee0 con 0x7f2d14003aa0 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.597+0000 7f2d2e0c5700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d281536e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f2d18001eb0 tx=0x7f2d18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.597+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18012610 con 0x7f2d14003aa0 2023-05-16T07:38:07.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.597+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18012770 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.597+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d28141710 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.597+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18012b10 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.597+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d281528b0 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.598+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28153530 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.598+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f2d18024030 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.599+0000 7f2d1e7fc700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2d0c05eeb0 0x7f2d0c061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.599+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f2d1807af90 con 0x7f2d14003aa0 2023-05-16T07:38:07.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.599+0000 7f2d2d8c4700 1 -- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2d0c05eeb0 msgr2=0x7f2d0c061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:07.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.599+0000 7f2d2d8c4700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2d0c05eeb0 0x7f2d0c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:07.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.606+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f2d1804c110 con 0x7f2d14003aa0 2023-05-16T07:38:07.625 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.621+0000 7f11331ca680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:38:07.659 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.656+0000 7ff8e25c3680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:38:07.731 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:07.727+0000 7f11331ca680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:07.774 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.771+0000 7ff8e25c3680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:38:07.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.774+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d280666c0 con 0x7f2d14003aa0 2023-05-16T07:38:07.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.776+0000 7f2d1e7fc700 1 -- 172.21.15.92:0/3703187545 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+18420 (secure 0 0 0) 0x7f2d18022020 con 0x7f2d14003aa0 2023-05-16T07:38:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:07.780 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":158,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:05.917821+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:36:33.491970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-05-16T07:36:33.931369+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:6833/95623":"2023-05-17T07:36:46.188304+0000","172.21.15.158:6832/95623":"2023-05-17T07:36:46.188304+0000","172.21.15.158:0/2052480298":"2023-05-17T07:36:46.188304+0000","172.21.15.158:0/2894032393":"2023-05-17T07:36:46.188304+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2d0c05eeb0 msgr2=0x7f2d0c061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:07.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2d0c05eeb0 0x7f2d0c061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 msgr2=0x7f2d281536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d281536e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f2d18001eb0 tx=0x7f2d18016000).stop 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 shutdown_connections 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2d0c05eeb0 0x7f2d0c061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2d281683b0 0x7f2d280ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d14003aa0 0x7f2d281536e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 --2- 172.21.15.92:0/3703187545 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2814c8b0 0x7f2d28141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 >> 172.21.15.92:0/3703187545 conn(0x7f2d280cb700 msgr2=0x7f2d280aac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 shutdown_connections 2023-05-16T07:38:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.778+0000 7f2d30329700 1 -- 172.21.15.92:0/3703187545 wait complete. 2023-05-16T07:38:07.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:38:07.887 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:07.884+0000 7ff8e25c3680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.904+0000 7f35b8ba8700 1 Processor -- start 2023-05-16T07:38:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.904+0000 7f35b8ba8700 1 -- start start 2023-05-16T07:38:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.905+0000 7f35b8ba8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35b414e200 0x7f35b414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.905+0000 7f35b8ba8700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f35b4066e50 0x7f35b4157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.905+0000 7f35b8ba8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b40672b0 0x7f35b4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.905+0000 7f35b8ba8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b4137470 con 0x7f35b414e200 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.905+0000 7f35b8ba8700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40c3520 con 0x7f35b4066e50 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.905+0000 7f35b8ba8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b4158430 con 0x7f35b40672b0 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35b414e200 0x7f35b414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35b414e200 0x7f35b414e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52370/0 (socket says 172.21.15.92:52370) 2023-05-16T07:38:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 -- 172.21.15.92:0/904375584 learned_addr learned my addr 172.21.15.92:0/904375584 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b1d9b700 1 --2- 172.21.15.92:0/904375584 >> v2:172.21.15.158:3300/0 conn(0x7f35b4066e50 0x7f35b4157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b2d9d700 1 --2- 172.21.15.92:0/904375584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b40672b0 0x7f35b4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 -- 172.21.15.92:0/904375584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b40672b0 msgr2=0x7f35b4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 --2- 172.21.15.92:0/904375584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b40672b0 0x7f35b4157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 -- 172.21.15.92:0/904375584 >> v2:172.21.15.158:3300/0 conn(0x7f35b4066e50 msgr2=0x7f35b4157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 --2- 172.21.15.92:0/904375584 >> v2:172.21.15.158:3300/0 conn(0x7f35b4066e50 0x7f35b4157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 -- 172.21.15.92:0/904375584 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b40683d0 con 0x7f35b414e200 2023-05-16T07:38:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b1d9b700 1 --2- 172.21.15.92:0/904375584 >> v2:172.21.15.158:3300/0 conn(0x7f35b4066e50 0x7f35b4157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b259c700 1 --2- 172.21.15.92:0/904375584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b414e5e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f35b40a6be0 tx=0x7f35a80090f0).ready entity=mon.0 client_cookie=31936112bf477ea5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b159a700 1 -- 172.21.15.92:0/904375584 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800b520 con 0x7f35b414e200 2023-05-16T07:38:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b159a700 1 -- 172.21.15.92:0/904375584 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a800b680 con 0x7f35b414e200 2023-05-16T07:38:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.906+0000 7f35b159a700 1 -- 172.21.15.92:0/904375584 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800ba00 con 0x7f35b414e200 2023-05-16T07:38:07.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 -- 172.21.15.92:0/904375584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 msgr2=0x7f35b414e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/904375584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b414e5e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f35b40a6be0 tx=0x7f35a80090f0).stop 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 -- 172.21.15.92:0/904375584 shutdown_connections 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/904375584 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b40672b0 0x7f35b4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/904375584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b414e5e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/904375584 >> v2:172.21.15.158:3300/0 conn(0x7f35b4066e50 0x7f35b4157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 -- 172.21.15.92:0/904375584 >> 172.21.15.92:0/904375584 conn(0x7f35b40c7bc0 msgr2=0x7f35b40c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 -- 172.21.15.92:0/904375584 shutdown_connections 2023-05-16T07:38:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.907+0000 7f35b8ba8700 1 -- 172.21.15.92:0/904375584 wait complete. 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.908+0000 7f35b8ba8700 1 Processor -- start 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.908+0000 7f35b8ba8700 1 -- start start 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b8ba8700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b4066e50 0x7f35b40a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b8ba8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b40672b0 0x7f35b4136460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b8ba8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b40649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b8ba8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40be9e0 con 0x7f35b414e200 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b8ba8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40a7460 con 0x7f35b40672b0 2023-05-16T07:38:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b8ba8700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40a7740 con 0x7f35b4066e50 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b40649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b40649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52378/0 (socket says 172.21.15.92:52378) 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 -- 172.21.15.92:0/2099354501 learned_addr learned my addr 172.21.15.92:0/2099354501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b259c700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b4066e50 0x7f35b40a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b1d9b700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b40672b0 0x7f35b4136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b4066e50 msgr2=0x7f35b40a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b4066e50 0x7f35b40a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b40672b0 msgr2=0x7f35b4136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b40672b0 0x7f35b4136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 -- 172.21.15.92:0/2099354501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8009370 con 0x7f35b414e200 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.909+0000 7f35b2d9d700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b40649e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f35ac009e90 tx=0x7f35ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.910+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00caa0 con 0x7f35b414e200 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.910+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac00f040 con 0x7f35b414e200 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.910+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00ce10 con 0x7f35b414e200 2023-05-16T07:38:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.910+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b40a7a20 con 0x7f35b414e200 2023-05-16T07:38:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.910+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b40a7100 con 0x7f35b414e200 2023-05-16T07:38:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.911+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b40beb90 con 0x7f35b414e200 2023-05-16T07:38:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.911+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 76) v1 ==== 75402+0+0 (secure 0 0 0) 0x7f35ac013610 con 0x7f35b414e200 2023-05-16T07:38:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.912+0000 7f35a27fc700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 0x7f3594061370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.912+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f35ac07b2b0 con 0x7f35b414e200 2023-05-16T07:38:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.912+0000 7f35b259c700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 msgr2=0x7f3594061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.912+0000 7f35b259c700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 0x7f3594061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:07.915+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f35ac04c450 con 0x7f35b414e200 2023-05-16T07:38:08.020 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.016+0000 7f11331ca680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:38:08.090 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.086+0000 7f11331ca680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:38:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.113+0000 7f35b259c700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 msgr2=0x7f3594061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.113+0000 7f35b259c700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 0x7f3594061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:08.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.124+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35b41372b0 con 0x7f35b414e200 2023-05-16T07:38:08.180 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:08.176+0000 7ff8e25c3680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:38:08.248 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:08.245+0000 7ff8e25c3680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:38:08.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.514+0000 7f35b259c700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 msgr2=0x7f3594061370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:08.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.514+0000 7f35b259c700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 0x7f3594061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:38:08.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.932+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 77) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f35ac078e10 con 0x7f35b414e200 2023-05-16T07:38:08.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.932+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 msgr2=0x7f3594061370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:08.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:08.932+0000 7f35a27fc700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 0x7f3594061370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:08.984 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b61b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.984 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b61b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b61b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b61b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b61b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b29ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b29ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b29ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.985 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b29ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.986 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:08.980+0000 7f10b29ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:08.986 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:38:08.982+0000 7f4581fb9700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:08.986 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:38:08.982+0000 7f4581fb9700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:08.987 INFO:tasks.ceph.mds.b.smithi158.stderr:2023-05-16T07:38:08.982+0000 7f4581fb9700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:38:09.001 INFO:tasks.ceph.mds.b.smithi158.stderr: -4> 2023-05-16T07:38:08.982+0000 7f4581fb9700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:09.001 INFO:tasks.ceph.mds.b.smithi158.stderr: -3> 2023-05-16T07:38:08.982+0000 7f4581fb9700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:09.001 INFO:tasks.ceph.mds.b.smithi158.stderr: -1> 2023-05-16T07:38:08.982+0000 7f4581fb9700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:38:09.019 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:38:09.104 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:38:09.101+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:38:09.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.103+0000 7f35a27fc700 1 -- 172.21.15.92:0/2099354501 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f35ac070a70 con 0x7f35b414e200 2023-05-16T07:38:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 msgr2=0x7f35b40649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b40649e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f35ac009e90 tx=0x7f35ac00b040).stop 2023-05-16T07:38:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 shutdown_connections 2023-05-16T07:38:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f359405eeb0 0x7f3594061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f35b4066e50 0x7f35b40a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35b414e200 0x7f35b40649e0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 --2- 172.21.15.92:0/2099354501 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b40672b0 0x7f35b4136460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 >> 172.21.15.92:0/2099354501 conn(0x7f35b40c7bc0 msgr2=0x7f35b40c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 shutdown_connections 2023-05-16T07:38:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.106+0000 7f35b8ba8700 1 -- 172.21.15.92:0/2099354501 wait complete. 2023-05-16T07:38:09.115 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:38:09.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:38:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.233+0000 7f8a8b533700 1 Processor -- start 2023-05-16T07:38:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.233+0000 7f8a8b533700 1 -- start start 2023-05-16T07:38:09.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a8b533700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8a84154470 0x7f8a840aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:09.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a8b533700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84068160 0x7f8a840685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a8b533700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a840c94d0 con 0x7f8a84154090 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a8b533700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a840cb7e0 con 0x7f8a84154470 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a8b533700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84068aa0 con 0x7f8a84068160 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a88ace700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8a84154470 0x7f8a840aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a892cf700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8a84154090 0x7f8a840a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34910/0 (socket says 172.21.15.92:34910) 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a892cf700 1 -- 172.21.15.92:0/1921042156 learned_addr learned my addr 172.21.15.92:0/1921042156 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:09.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a88ace700 1 -- 172.21.15.92:0/1921042156 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84068160 msgr2=0x7f8a840685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a89ad0700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84068160 0x7f8a840685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a88ace700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84068160 0x7f8a840685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a83fff700 1 -- 172.21.15.92:0/1921042156 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629539031 0 0) 0x7f8a840c94d0 con 0x7f8a84154090 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a88ace700 1 -- 172.21.15.92:0/1921042156 >> v1:172.21.15.92:6789/0 conn(0x7f8a84154090 legacy=0x7f8a840a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a88ace700 1 -- 172.21.15.92:0/1921042156 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840c88c0 con 0x7f8a84154470 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a89ad0700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84068160 0x7f8a840685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:09.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a88ace700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a840aaca0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f8a78009760 tx=0x7f8a78009060).ready entity=mon.0 client_cookie=e04c3a64d7ee06c2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a83fff700 1 -- 172.21.15.92:0/1921042156 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800b580 con 0x7f8a84154470 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a83fff700 1 -- 172.21.15.92:0/1921042156 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800b6e0 con 0x7f8a84154470 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.234+0000 7f8a83fff700 1 -- 172.21.15.92:0/1921042156 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800ba60 con 0x7f8a84154470 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 -- 172.21.15.92:0/1921042156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 msgr2=0x7f8a840aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a840aaca0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f8a78009760 tx=0x7f8a78009060).stop 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 -- 172.21.15.92:0/1921042156 shutdown_connections 2023-05-16T07:38:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84068160 0x7f8a840685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 --2- 172.21.15.92:0/1921042156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a840aaca0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 -- 172.21.15.92:0/1921042156 >> 172.21.15.92:0/1921042156 conn(0x7f8a84067920 msgr2=0x7f8a840cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 -- 172.21.15.92:0/1921042156 shutdown_connections 2023-05-16T07:38:09.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.235+0000 7f8a8b533700 1 -- 172.21.15.92:0/1921042156 wait complete. 2023-05-16T07:38:09.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.236+0000 7f8a8b533700 1 Processor -- start 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.236+0000 7f8a8b533700 1 -- start start 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a8b533700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8a84068160 0x7f8a84155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a8b533700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84154090 0x7f8a840c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a8b533700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a8414f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a8b533700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84168450 con 0x7f8a84154470 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a8b533700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84167840 con 0x7f8a84068160 2023-05-16T07:38:09.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a8b533700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84167b20 con 0x7f8a84154090 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a8414f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a8414f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52404/0 (socket says 172.21.15.92:52404) 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 -- 172.21.15.92:0/2312335334 learned_addr learned my addr 172.21.15.92:0/2312335334 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 -- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84154090 msgr2=0x7f8a840c11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84154090 0x7f8a840c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 -- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8a84068160 msgr2=0x7f8a84155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a892cf700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8a84068160 0x7f8a84155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:09.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8a84068160 0x7f8a84155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 -- 172.21.15.92:0/2312335334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a780092a0 con 0x7f8a84154470 2023-05-16T07:38:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a892cf700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8a84068160 0x7f8a84155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.237+0000 7f8a89ad0700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a8414f580 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f8a7c00c880 tx=0x7f8a7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.238+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c010580 con 0x7f8a84154470 2023-05-16T07:38:09.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.238+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c0106e0 con 0x7f8a84154470 2023-05-16T07:38:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.238+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a84167e00 con 0x7f8a84154470 2023-05-16T07:38:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.238+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c010a60 con 0x7f8a84154470 2023-05-16T07:38:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.238+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a84155750 con 0x7f8a84154470 2023-05-16T07:38:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.239+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a841559d0 con 0x7f8a84154470 2023-05-16T07:38:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.239+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 77) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f8a7c01b460 con 0x7f8a84154470 2023-05-16T07:38:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.239+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7363+0+0 (secure 0 0 0) 0x7f8a7c07a000 con 0x7f8a84154470 2023-05-16T07:38:09.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.243+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8a7c04b100 con 0x7f8a84154470 2023-05-16T07:38:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.446+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a84168600 con 0x7f8a84154470 2023-05-16T07:38:09.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.936+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 78) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f8a7c077ac0 con 0x7f8a84154470 2023-05-16T07:38:09.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.936+0000 7f8a817fa700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8a700301b0 0x7f8a70067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:09.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.939+0000 7f8a892cf700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8a700301b0 0x7f8a70067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:09.940+0000 7f8a892cf700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8a700301b0 0x7f8a70067fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f8a74001e40 tx=0x7f8a74009040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.107+0000 7f8a817fa700 1 -- 172.21.15.92:0/2312335334 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a7c06e370 con 0x7f8a84154470 2023-05-16T07:38:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8a700301b0 msgr2=0x7f8a70067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8a700301b0 0x7f8a70067fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f8a74001e40 tx=0x7f8a74009040).stop 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 msgr2=0x7f8a8414f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a8414f580 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f8a7c00c880 tx=0x7f8a7c008040).stop 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 shutdown_connections 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a84154470 0x7f8a8414f580 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8a84068160 0x7f8a84155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f8a700301b0 0x7f8a70067fb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 --2- 172.21.15.92:0/2312335334 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8a84154090 0x7f8a840c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:10.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 >> 172.21.15.92:0/2312335334 conn(0x7f8a84067920 msgr2=0x7f8a840afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:10.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 shutdown_connections 2023-05-16T07:38:10.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.110+0000 7f8a8b533700 1 -- 172.21.15.92:0/2312335334 wait complete. 2023-05-16T07:38:10.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:38:10.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:38:10.207 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:38:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.245+0000 7fac4d4d0700 1 Processor -- start 2023-05-16T07:38:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.245+0000 7fac4d4d0700 1 -- start start 2023-05-16T07:38:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.245+0000 7fac4d4d0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 0x7fac48156140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.245+0000 7fac4d4d0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac48155cb0 con 0x7fac48151b80 2023-05-16T07:38:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.245+0000 7fac4d4d0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac480c4c30 con 0x7fac480a7790 2023-05-16T07:38:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.245+0000 7fac4d4d0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac48155730 con 0x7fac480bf3a0 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac46ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fac48151b80 0x7fac480c2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34940/0 (socket says 172.21.15.92:34940) 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac46ffd700 1 -- 172.21.15.92:0/3089058907 learned_addr learned my addr 172.21.15.92:0/3089058907 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955775672 0 0) 0x7fac48155cb0 con 0x7fac48151b80 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 0x7fac48156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac340033f0 con 0x7fac48151b80 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472887203 0 0) 0x7fac340033f0 con 0x7fac48151b80 2023-05-16T07:38:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 msgr2=0x7fac48156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 0x7fac48156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 >> v1:172.21.15.158:6789/0 conn(0x7fac480a7790 legacy=0x7fac480634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480c4f10 con 0x7fac48151b80 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fac38001470 con 0x7fac48151b80 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 >> v1:172.21.15.92:6789/0 conn(0x7fac48151b80 legacy=0x7fac480c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34003910 0x7fac34003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac34004690 0x7fac34004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 0x7fac340050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48155730 con 0x7fac34004690 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3400b9e0 con 0x7fac34003910 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3400bba0 con 0x7fac480bf3a0 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac34004690 0x7fac34004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac46ffd700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34003910 0x7fac34003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.246+0000 7fac477fe700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 0x7fac340050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 msgr2=0x7fac340050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480bf3a0 0x7fac340050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34003910 msgr2=0x7fac34003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34003910 0x7fac34003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3089058907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340033f0 con 0x7fac34004690 2023-05-16T07:38:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac46ffd700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34003910 0x7fac34003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac34004690 0x7fac34004b10 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7fac480bf7c0 tx=0x7fac30009060).ready entity=mon.0 client_cookie=36860a09d9e2c01e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30009430 con 0x7fac34004690 2023-05-16T07:38:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30009e70 con 0x7fac34004690 2023-05-16T07:38:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac467fc700 1 -- 172.21.15.92:0/3089058907 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000e850 con 0x7fac34004690 2023-05-16T07:38:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3089058907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac34004690 msgr2=0x7fac34004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3089058907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac34004690 0x7fac34004b10 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7fac480bf7c0 tx=0x7fac30009060).stop 2023-05-16T07:38:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac477fe700 1 -- 172.21.15.92:0/3089058907 reap_dead start 2023-05-16T07:38:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3089058907 shutdown_connections 2023-05-16T07:38:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3089058907 >> 172.21.15.92:0/3089058907 conn(0x7fac48062d80 msgr2=0x7fac4805dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3089058907 shutdown_connections 2023-05-16T07:38:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.247+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3089058907 wait complete. 2023-05-16T07:38:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.248+0000 7fac4d4d0700 1 Processor -- start 2023-05-16T07:38:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 -- start start 2023-05-16T07:38:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34004690 0x7fac480c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 0x7fac48163a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480a5870 0x7fac480aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48067f60 con 0x7fac4814cca0 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4814bce0 con 0x7fac34004690 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac4d4d0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac48163f20 con 0x7fac480a5870 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac3e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 0x7fac48163a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac3e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 0x7fac48163a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52438/0 (socket says 172.21.15.92:52438) 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3609385061 learned_addr learned my addr 172.21.15.92:0/3609385061 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.249+0000 7fac46ffd700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34004690 0x7fac480c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac477fe700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480a5870 0x7fac480aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480a5870 msgr2=0x7fac480aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480a5870 0x7fac480aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34004690 msgr2=0x7fac480c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34004690 0x7fac480c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3e7fc700 1 -- 172.21.15.92:0/3609385061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48068170 con 0x7fac4814cca0 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac46ffd700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34004690 0x7fac480c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac477fe700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480a5870 0x7fac480aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3e7fc700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 0x7fac48163a30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fac3400b9b0 tx=0x7fac3001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30020070 con 0x7fac4814cca0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30009430 con 0x7fac4814cca0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3001be70 con 0x7fac4814cca0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac48068450 con 0x7fac4814cca0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.250+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac48068a10 con 0x7fac4814cca0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.251+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac48047150 con 0x7fac4814cca0 2023-05-16T07:38:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.251+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 78) v1 ==== 75162+0+0 (secure 0 0 0) 0x7fac30025670 con 0x7fac4814cca0 2023-05-16T07:38:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.251+0000 7fac3f7fe700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fac2805ecf0 0x7fac280611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.251+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7363+0+0 (secure 0 0 0) 0x7fac3007b760 con 0x7fac4814cca0 2023-05-16T07:38:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.252+0000 7fac46ffd700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fac2805ecf0 0x7fac280611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.252+0000 7fac46ffd700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fac2805ecf0 0x7fac280611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fac38001ce0 tx=0x7fac38000f60).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.255+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fac3004c8c0 con 0x7fac4814cca0 2023-05-16T07:38:10.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:10.423+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac48155940 con 0x7fac4814cca0 2023-05-16T07:38:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.113+0000 7fac3f7fe700 1 -- 172.21.15.92:0/3609385061 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v161) v1 ==== 181+0+0 (secure 0 0 0) 0x7fac3002a020 con 0x7fac4814cca0 2023-05-16T07:38:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fac2805ecf0 msgr2=0x7fac280611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fac2805ecf0 0x7fac280611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fac38001ce0 tx=0x7fac38000f60).stop 2023-05-16T07:38:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 msgr2=0x7fac48163a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 0x7fac48163a30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fac3400b9b0 tx=0x7fac3001a000).stop 2023-05-16T07:38:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 shutdown_connections 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fac2805ecf0 0x7fac280611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fac480a5870 0x7fac480aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fac34004690 0x7fac480c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 --2- 172.21.15.92:0/3609385061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4814cca0 0x7fac48163a30 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 >> 172.21.15.92:0/3609385061 conn(0x7fac48062d80 msgr2=0x7fac48063e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 shutdown_connections 2023-05-16T07:38:11.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.117+0000 7fac4d4d0700 1 -- 172.21.15.92:0/3609385061 wait complete. 2023-05-16T07:38:11.123 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:38:11.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:38:11.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.249+0000 7f4d3035b700 1 Processor -- start 2023-05-16T07:38:11.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.249+0000 7f4d3035b700 1 -- start start 2023-05-16T07:38:11.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.250+0000 7f4d3035b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d28154090 0x7f4d280aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:11.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.250+0000 7f4d3035b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d280c94d0 con 0x7f4d28068160 2023-05-16T07:38:11.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.250+0000 7f4d3035b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d280cb7e0 con 0x7f4d28154470 2023-05-16T07:38:11.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.250+0000 7f4d3035b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d280c88c0 con 0x7f4d28154090 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d28154090 0x7f4d280aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d28154090 0x7f4d280aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52444/0 (socket says 172.21.15.92:52444) 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/88215594 learned_addr learned my addr 172.21.15.92:0/88215594 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/88215594 >> v1:172.21.15.158:6789/0 conn(0x7f4d28154470 legacy=0x7f4d280a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/88215594 >> v1:172.21.15.92:6789/0 conn(0x7f4d28068160 legacy=0x7f4d280c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/88215594 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280c8ba0 con 0x7f4d28154090 2023-05-16T07:38:11.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2e0f7700 1 --2- 172.21.15.92:0/88215594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28154090 0x7f4d280aaca0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f4d18009f20 tx=0x7f4d18009840).ready entity=mon.0 client_cookie=1400e693be40b8f4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2d0f5700 1 -- 172.21.15.92:0/88215594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800e070 con 0x7f4d28154090 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2d0f5700 1 -- 172.21.15.92:0/88215594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1800ba80 con 0x7f4d28154090 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.251+0000 7f4d2d0f5700 1 -- 172.21.15.92:0/88215594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800be00 con 0x7f4d28154090 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 -- 172.21.15.92:0/88215594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28154090 msgr2=0x7f4d280aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 --2- 172.21.15.92:0/88215594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28154090 0x7f4d280aaca0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f4d18009f20 tx=0x7f4d18009840).stop 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 -- 172.21.15.92:0/88215594 shutdown_connections 2023-05-16T07:38:11.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 --2- 172.21.15.92:0/88215594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28154090 0x7f4d280aaca0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 -- 172.21.15.92:0/88215594 >> 172.21.15.92:0/88215594 conn(0x7f4d28067920 msgr2=0x7f4d280cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 -- 172.21.15.92:0/88215594 shutdown_connections 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.252+0000 7f4d3035b700 1 -- 172.21.15.92:0/88215594 wait complete. 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.253+0000 7f4d3035b700 1 Processor -- start 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.253+0000 7f4d3035b700 1 -- start start 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 0x7f4d28155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d28154090 0x7f4d280b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d28154470 0x7f4d280aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28155f00 con 0x7f4d28068160 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280b0bf0 con 0x7f4d28154090 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2814d640 con 0x7f4d28154470 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 0x7f4d28155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 0x7f4d28155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52450/0 (socket says 172.21.15.92:52450) 2023-05-16T07:38:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/3278237612 learned_addr learned my addr 172.21.15.92:0/3278237612 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:11.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2d8f6700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d28154090 0x7f4d280b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:11.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e8f8700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d28154470 0x7f4d280aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:11.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d28154470 msgr2=0x7f4d280aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:11.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d28154470 0x7f4d280aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d28154090 msgr2=0x7f4d280b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:11.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d28154090 0x7f4d280b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:11.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 -- 172.21.15.92:0/3278237612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18009ae0 con 0x7f4d28068160 2023-05-16T07:38:11.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d2e0f7700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 0x7f4d28155a10 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4d2806bb30 tx=0x7f4d18009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:11.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18013ce0 con 0x7f4d28068160 2023-05-16T07:38:11.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d18013e40 con 0x7f4d28068160 2023-05-16T07:38:11.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d28165440 con 0x7f4d28068160 2023-05-16T07:38:11.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801d640 con 0x7f4d28068160 2023-05-16T07:38:11.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.254+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d28164970 con 0x7f4d28068160 2023-05-16T07:38:11.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.255+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2806c350 con 0x7f4d28068160 2023-05-16T07:38:11.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.256+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4d1801b040 con 0x7f4d28068160 2023-05-16T07:38:11.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.256+0000 7f4d267fc700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4d1405ed90 0x7f4d14061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:11.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.256+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f4d1807b800 con 0x7f4d28068160 2023-05-16T07:38:11.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.257+0000 7f4d2d8f6700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4d1405ed90 0x7f4d14061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:11.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.258+0000 7f4d2d8f6700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4d1405ed90 0x7f4d14061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4d28157790 tx=0x7f4d1c009040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:11.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.260+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4d1804cab0 con 0x7f4d28068160 2023-05-16T07:38:11.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:11.422+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d28066930 con 0x7f4d28068160 2023-05-16T07:38:12.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.124+0000 7f4d267fc700 1 -- 172.21.15.92:0/3278237612 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v162) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4d18007320 con 0x7f4d28068160 2023-05-16T07:38:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4d1405ed90 msgr2=0x7f4d14061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4d1405ed90 0x7f4d14061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4d28157790 tx=0x7f4d1c009040).stop 2023-05-16T07:38:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 msgr2=0x7f4d28155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 0x7f4d28155a10 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4d2806bb30 tx=0x7f4d18009840).stop 2023-05-16T07:38:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 shutdown_connections 2023-05-16T07:38:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4d1405ed90 0x7f4d14061250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d28154470 0x7f4d280aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d28068160 0x7f4d28155a10 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 --2- 172.21.15.92:0/3278237612 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d28154090 0x7f4d280b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 >> 172.21.15.92:0/3278237612 conn(0x7f4d28067920 msgr2=0x7f4d281533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:12.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 shutdown_connections 2023-05-16T07:38:12.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.127+0000 7f4d3035b700 1 -- 172.21.15.92:0/3278237612 wait complete. 2023-05-16T07:38:12.133 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:38:12.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.254+0000 7fada971f700 1 Processor -- start 2023-05-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.254+0000 7fada971f700 1 -- start start 2023-05-16T07:38:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada971f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fada4154470 0x7fada40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada971f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4068160 0x7fada40685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada971f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada40c94d0 con 0x7fada4154090 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada971f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada40cb7e0 con 0x7fada4154470 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada971f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada4068aa0 con 0x7fada4068160 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fada4154470 0x7fada40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fada4154470 0x7fada40aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52460/0 (socket says 172.21.15.92:52460) 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada2ffd700 1 --1- 172.21.15.92:0/265579226 >> v1:172.21.15.92:6789/0 conn(0x7fada4154090 0x7fada40a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34974/0 (socket says 172.21.15.92:34974) 2023-05-16T07:38:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 -- 172.21.15.92:0/265579226 learned_addr learned my addr 172.21.15.92:0/265579226 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 -- 172.21.15.92:0/265579226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4068160 msgr2=0x7fada40685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 --2- 172.21.15.92:0/265579226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4068160 0x7fada40685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 -- 172.21.15.92:0/265579226 >> v1:172.21.15.92:6789/0 conn(0x7fada4154090 legacy=0x7fada40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada1ffb700 1 -- 172.21.15.92:0/265579226 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017712205 0 0) 0x7fada40c94d0 con 0x7fada4154090 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 -- 172.21.15.92:0/265579226 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40c88c0 con 0x7fada4154470 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada27fc700 1 --2- 172.21.15.92:0/265579226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154470 0x7fada40aaca0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fad8c006fd0 tx=0x7fad8c0090f0).ready entity=mon.0 client_cookie=9f5d68b65d981744 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.255+0000 7fada1ffb700 1 -- 172.21.15.92:0/265579226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00b600 con 0x7fada4154470 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada1ffb700 1 -- 172.21.15.92:0/265579226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c00b760 con 0x7fada4154470 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada1ffb700 1 -- 172.21.15.92:0/265579226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00bae0 con 0x7fada4154470 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 -- 172.21.15.92:0/265579226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154470 msgr2=0x7fada40aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 --2- 172.21.15.92:0/265579226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154470 0x7fada40aaca0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fad8c006fd0 tx=0x7fad8c0090f0).stop 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 -- 172.21.15.92:0/265579226 shutdown_connections 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 --2- 172.21.15.92:0/265579226 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4068160 0x7fada40685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 --2- 172.21.15.92:0/265579226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154470 0x7fada40aaca0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 -- 172.21.15.92:0/265579226 >> 172.21.15.92:0/265579226 conn(0x7fada4067920 msgr2=0x7fada40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 -- 172.21.15.92:0/265579226 shutdown_connections 2023-05-16T07:38:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.256+0000 7fada971f700 1 -- 172.21.15.92:0/265579226 wait complete. 2023-05-16T07:38:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.257+0000 7fada971f700 1 Processor -- start 2023-05-16T07:38:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.257+0000 7fada971f700 1 -- start start 2023-05-16T07:38:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada971f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada4068160 0x7fada40c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada971f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 0x7fada4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada971f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4154470 0x7fada4152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada971f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4155c40 con 0x7fada4154090 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada971f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4155030 con 0x7fada4068160 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada971f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada4155310 con 0x7fada4154470 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 0x7fada4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 0x7fada4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52466/0 (socket says 172.21.15.92:52466) 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 -- 172.21.15.92:0/2343636581 learned_addr learned my addr 172.21.15.92:0/2343636581 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 -- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4154470 msgr2=0x7fada4152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4154470 0x7fada4152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 -- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada4068160 msgr2=0x7fada40c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.258+0000 7fada27fc700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada4068160 0x7fada40c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fada27fc700 1 -- 172.21.15.92:0/2343636581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c009370 con 0x7fada4154090 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fada27fc700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 0x7fada4154850 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fada406bb30 tx=0x7fad8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00b8d0 con 0x7fada4154090 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c00ba30 con 0x7fada4154090 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada41555f0 con 0x7fada4154090 2023-05-16T07:38:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c01b660 con 0x7fada4154090 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.259+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada4154d40 con 0x7fada4154090 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.260+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada4066730 con 0x7fada4154090 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.260+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fad8c02a070 con 0x7fada4154090 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.260+0000 7fad9affd700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fad8405ed90 0x7fad84061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.261+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6453+0+0 (secure 0 0 0) 0x7fad8c07b2e0 con 0x7fada4154090 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.261+0000 7fada2ffd700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fad8405ed90 0x7fad84061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.261+0000 7fada2ffd700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fad8405ed90 0x7fad84061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fad94007fd0 tx=0x7fad94007bb0).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.264+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fad8c04c7b0 con 0x7fada4154090 2023-05-16T07:38:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.428+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fada406c350 con 0x7fada4154090 2023-05-16T07:38:12.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.432+0000 7fad9affd700 1 -- 172.21.15.92:0/2343636581 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fad8c01f0a0 con 0x7fada4154090 2023-05-16T07:38:12.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fad8405ed90 msgr2=0x7fad84061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fad8405ed90 0x7fad84061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fad94007fd0 tx=0x7fad94007bb0).stop 2023-05-16T07:38:12.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 msgr2=0x7fada4154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 0x7fada4154850 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fada406bb30 tx=0x7fad8c017000).stop 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 shutdown_connections 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fad8405ed90 0x7fad84061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fada4154470 0x7fada4152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada4068160 0x7fada40c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 --2- 172.21.15.92:0/2343636581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4154090 0x7fada4154850 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 >> 172.21.15.92:0/2343636581 conn(0x7fada4067920 msgr2=0x7fada4153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 shutdown_connections 2023-05-16T07:38:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.434+0000 7fada971f700 1 -- 172.21.15.92:0/2343636581 wait complete. 2023-05-16T07:38:12.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:38:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.557+0000 7f0d44f30700 1 Processor -- start 2023-05-16T07:38:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.557+0000 7f0d44f30700 1 -- start start 2023-05-16T07:38:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d44f30700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d4014e6f0 0x7f0d40155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d44f30700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d400c4d40 0x7f0d400622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d44f30700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f0d400627a0 0x7f0d40063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d44f30700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4014ead0 con 0x7f0d400c4d40 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d44f30700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d40154cb0 con 0x7f0d400627a0 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d44f30700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40062c30 con 0x7f0d4014e6f0 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.558+0000 7f0d3dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d400c4d40 0x7f0d400622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d400c4d40 0x7f0d400622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52476/0 (socket says 172.21.15.92:52476) 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/4088785920 learned_addr learned my addr 172.21.15.92:0/4088785920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3ed9d700 1 --2- 172.21.15.92:0/4088785920 >> v2:172.21.15.158:3300/0 conn(0x7f0d400627a0 0x7f0d40063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3e59c700 1 --2- 172.21.15.92:0/4088785920 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d4014e6f0 0x7f0d40155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/4088785920 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d4014e6f0 msgr2=0x7f0d40155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 --2- 172.21.15.92:0/4088785920 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d4014e6f0 0x7f0d40155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/4088785920 >> v2:172.21.15.158:3300/0 conn(0x7f0d400627a0 msgr2=0x7f0d40063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 --2- 172.21.15.92:0/4088785920 >> v2:172.21.15.158:3300/0 conn(0x7f0d400627a0 0x7f0d40063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/4088785920 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400c6020 con 0x7f0d400c4d40 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3dd9b700 1 --2- 172.21.15.92:0/4088785920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400622b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f0d400c3240 tx=0x7f0d280090f0).ready entity=mon.0 client_cookie=3ef76f7e5194b9f4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3d59a700 1 -- 172.21.15.92:0/4088785920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800b600 con 0x7f0d400c4d40 2023-05-16T07:38:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3d59a700 1 -- 172.21.15.92:0/4088785920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800b760 con 0x7f0d400c4d40 2023-05-16T07:38:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.559+0000 7f0d3d59a700 1 -- 172.21.15.92:0/4088785920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800bae0 con 0x7f0d400c4d40 2023-05-16T07:38:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 -- 172.21.15.92:0/4088785920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 msgr2=0x7f0d400622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 --2- 172.21.15.92:0/4088785920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400622b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f0d400c3240 tx=0x7f0d280090f0).stop 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 -- 172.21.15.92:0/4088785920 shutdown_connections 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 --2- 172.21.15.92:0/4088785920 >> v2:172.21.15.158:3300/0 conn(0x7f0d400627a0 0x7f0d40063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 --2- 172.21.15.92:0/4088785920 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d4014e6f0 0x7f0d40155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 --2- 172.21.15.92:0/4088785920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400622b0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 -- 172.21.15.92:0/4088785920 >> 172.21.15.92:0/4088785920 conn(0x7f0d4005e820 msgr2=0x7f0d400c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 -- 172.21.15.92:0/4088785920 shutdown_connections 2023-05-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.560+0000 7f0d44f30700 1 -- 172.21.15.92:0/4088785920 wait complete. 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.561+0000 7f0d44f30700 1 Processor -- start 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.561+0000 7f0d44f30700 1 -- start start 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d400627a0 0x7f0d401413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d4014e6f0 0x7f0d400c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400c3cb0 con 0x7f0d400c4d40 2023-05-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d401682f0 con 0x7f0d4014e6f0 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d401676e0 con 0x7f0d400627a0 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52480/0 (socket says 172.21.15.92:52480) 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/2960156311 learned_addr learned my addr 172.21.15.92:0/2960156311 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3e59c700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d400627a0 0x7f0d401413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3ed9d700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d4014e6f0 0x7f0d400c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d400627a0 msgr2=0x7f0d401413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d400627a0 0x7f0d401413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d4014e6f0 msgr2=0x7f0d400c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d4014e6f0 0x7f0d400c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 -- 172.21.15.92:0/2960156311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009370 con 0x7f0d400c4d40 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3e59c700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d400627a0 0x7f0d401413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d3dd9b700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400c32d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f0d28009700 tx=0x7f0d28017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800b8d0 con 0x7f0d400c4d40 2023-05-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800ba30 con 0x7f0d400c4d40 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d401679c0 con 0x7f0d400c4d40 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2801c670 con 0x7f0d400c4d40 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.562+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d40167f80 con 0x7f0d400c4d40 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.563+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4006c350 con 0x7f0d400c4d40 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.564+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f0d28022030 con 0x7f0d400c4d40 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.564+0000 7f0d367fc700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0d2005ed90 0x7f0d20061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.564+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f0d2807bc80 con 0x7f0d400c4d40 2023-05-16T07:38:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.564+0000 7f0d3e59c700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0d2005ed90 0x7f0d20061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.565+0000 7f0d3e59c700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0d2005ed90 0x7f0d20061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0d30007460 tx=0x7f0d3000b040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.568+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f0d28050b00 con 0x7f0d400c4d40 2023-05-16T07:38:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.731+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f0d40064830 con 0x7f0d400c4d40 2023-05-16T07:38:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.735+0000 7f0d367fc700 1 -- 172.21.15.92:0/2960156311 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0d2802a0b0 con 0x7f0d400c4d40 2023-05-16T07:38:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0d2005ed90 msgr2=0x7f0d20061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0d2005ed90 0x7f0d20061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0d30007460 tx=0x7f0d3000b040).stop 2023-05-16T07:38:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 msgr2=0x7f0d400c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400c32d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f0d28009700 tx=0x7f0d28017000).stop 2023-05-16T07:38:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 shutdown_connections 2023-05-16T07:38:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f0d2005ed90 0x7f0d20061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0d400627a0 0x7f0d401413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0d4014e6f0 0x7f0d400c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 --2- 172.21.15.92:0/2960156311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d400c4d40 0x7f0d400c32d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 >> 172.21.15.92:0/2960156311 conn(0x7f0d4005e820 msgr2=0x7f0d400c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 shutdown_connections 2023-05-16T07:38:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.737+0000 7f0d44f30700 1 -- 172.21.15.92:0/2960156311 wait complete. 2023-05-16T07:38:12.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-05-16T07:38:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.858+0000 7f4eb91b0700 1 Processor -- start 2023-05-16T07:38:12.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.858+0000 7f4eb91b0700 1 -- start start 2023-05-16T07:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb91b0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4eb4153da0 0x7f4eb4069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb91b0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4eb4154180 0x7f4eb4063720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb91b0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb4063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb91b0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb4064100 con 0x7f4eb4154180 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb91b0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb4068620 con 0x7f4eb4153da0 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb91b0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40622b0 con 0x7f4eb4062fa0 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4eb4154180 0x7f4eb4063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4eb4154180 0x7f4eb4063720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52484/0 (socket says 172.21.15.92:52484) 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.859+0000 7f4eb259c700 1 -- 172.21.15.92:0/3369112811 learned_addr learned my addr 172.21.15.92:0/3369112811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:12.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb2d9d700 1 --2- 172.21.15.92:0/3369112811 >> v2:172.21.15.158:3300/0 conn(0x7f4eb4153da0 0x7f4eb4069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb359e700 1 --2- 172.21.15.92:0/3369112811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb4063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb259c700 1 -- 172.21.15.92:0/3369112811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 msgr2=0x7f4eb4063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb259c700 1 --2- 172.21.15.92:0/3369112811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb4063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb259c700 1 -- 172.21.15.92:0/3369112811 >> v2:172.21.15.158:3300/0 conn(0x7f4eb4153da0 msgr2=0x7f4eb4069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb259c700 1 --2- 172.21.15.92:0/3369112811 >> v2:172.21.15.158:3300/0 conn(0x7f4eb4153da0 0x7f4eb4069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb259c700 1 -- 172.21.15.92:0/3369112811 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb4062b10 con 0x7f4eb4154180 2023-05-16T07:38:12.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb259c700 1 --2- 172.21.15.92:0/3369112811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4154180 0x7f4eb4063720 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f4eb4067ae0 tx=0x7f4e9c0090f0).ready entity=mon.0 client_cookie=34610311c89dd23b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb1d9b700 1 -- 172.21.15.92:0/3369112811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00b600 con 0x7f4eb4154180 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb1d9b700 1 -- 172.21.15.92:0/3369112811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c00b760 con 0x7f4eb4154180 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb1d9b700 1 -- 172.21.15.92:0/3369112811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00bae0 con 0x7f4eb4154180 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 -- 172.21.15.92:0/3369112811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4154180 msgr2=0x7f4eb4063720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/3369112811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4154180 0x7f4eb4063720 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f4eb4067ae0 tx=0x7f4e9c0090f0).stop 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 -- 172.21.15.92:0/3369112811 shutdown_connections 2023-05-16T07:38:12.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/3369112811 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb4063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/3369112811 >> v2:172.21.15.158:3300/0 conn(0x7f4eb4153da0 0x7f4eb4069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/3369112811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4154180 0x7f4eb4063720 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.860+0000 7f4eb91b0700 1 -- 172.21.15.92:0/3369112811 >> 172.21.15.92:0/3369112811 conn(0x7f4eb40c8aa0 msgr2=0x7f4eb40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.861+0000 7f4eb91b0700 1 -- 172.21.15.92:0/3369112811 shutdown_connections 2023-05-16T07:38:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.861+0000 7f4eb91b0700 1 -- 172.21.15.92:0/3369112811 wait complete. 2023-05-16T07:38:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.862+0000 7f4eb91b0700 1 Processor -- start 2023-05-16T07:38:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.862+0000 7f4eb91b0700 1 -- start start 2023-05-16T07:38:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.862+0000 7f4eb91b0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb41414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.862+0000 7f4eb91b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 0x7f4eb40c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb91b0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4eb4154180 0x7f4eb4152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb91b0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41503e0 con 0x7f4eb4153da0 2023-05-16T07:38:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb91b0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41506c0 con 0x7f4eb4154180 2023-05-16T07:38:12.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb91b0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41509a0 con 0x7f4eb4062fa0 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 0x7f4eb40c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 0x7f4eb40c1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52492/0 (socket says 172.21.15.92:52492) 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 -- 172.21.15.92:0/532825262 learned_addr learned my addr 172.21.15.92:0/532825262 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb2d9d700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb41414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb359e700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4eb4154180 0x7f4eb4152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 -- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 msgr2=0x7f4eb41414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb41414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 -- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4eb4154180 msgr2=0x7f4eb4152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4eb4154180 0x7f4eb4152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 -- 172.21.15.92:0/532825262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c009370 con 0x7f4eb4153da0 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb259c700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 0x7f4eb40c1e60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f4e9c009130 tx=0x7f4e9c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00b8d0 con 0x7f4eb4153da0 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c00ba30 con 0x7f4eb4153da0 2023-05-16T07:38:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c01c670 con 0x7f4eb4153da0 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb4150c80 con 0x7f4eb4153da0 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.863+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb4150020 con 0x7f4eb4153da0 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.864+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb406c350 con 0x7f4eb4153da0 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.865+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4e9c00f070 con 0x7f4eb4153da0 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.865+0000 7f4eaaffd700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4e9405ed90 0x7f4e94061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.865+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f4e9c07b7d0 con 0x7f4eb4153da0 2023-05-16T07:38:12.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.865+0000 7f4eb2d9d700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4e9405ed90 0x7f4e94061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.866+0000 7f4eb2d9d700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4e9405ed90 0x7f4e94061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4ea40075a0 tx=0x7f4ea40097a0).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:12.869+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4e9c04cc20 con 0x7f4eb4153da0 2023-05-16T07:38:13.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.096+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f4eb4068340 con 0x7f4eb4153da0 2023-05-16T07:38:13.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.127+0000 7f4eaaffd700 1 -- 172.21.15.92:0/532825262 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v1232) v1 ==== 163+0+0 (secure 0 0 0) 0x7f4e9c050240 con 0x7f4eb4153da0 2023-05-16T07:38:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4e9405ed90 msgr2=0x7f4e94061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4e9405ed90 0x7f4e94061250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4ea40075a0 tx=0x7f4ea40097a0).stop 2023-05-16T07:38:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 msgr2=0x7f4eb40c1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 0x7f4eb40c1e60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f4e9c009130 tx=0x7f4e9c017000).stop 2023-05-16T07:38:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 shutdown_connections 2023-05-16T07:38:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4e9405ed90 0x7f4e94061250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4eb4062fa0 0x7f4eb41414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4eb4154180 0x7f4eb4152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 --2- 172.21.15.92:0/532825262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eb4153da0 0x7f4eb40c1e60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 >> 172.21.15.92:0/532825262 conn(0x7f4eb40c8aa0 msgr2=0x7f4eb4063720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 shutdown_connections 2023-05-16T07:38:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.129+0000 7f4eb91b0700 1 -- 172.21.15.92:0/532825262 wait complete. 2023-05-16T07:38:13.144 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-05-16T07:38:13.145 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-05-16T07:38:13.145 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-05-16T07:38:13.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-05-16T07:38:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.253+0000 7f875d473700 1 Processor -- start 2023-05-16T07:38:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.253+0000 7f875d473700 1 -- start start 2023-05-16T07:38:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f875d473700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8758154170 0x7f87580c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f875d473700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f87580c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f875d473700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87580c0660 con 0x7f87580696a0 2023-05-16T07:38:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f875d473700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87580c1b00 con 0x7f8758154170 2023-05-16T07:38:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f875d473700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580bfd30 con 0x7f8758154550 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8758154170 0x7f87580c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8758154170 0x7f87580c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52500/0 (socket says 172.21.15.92:52500) 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 -- 172.21.15.92:0/212445462 learned_addr learned my addr 172.21.15.92:0/212445462 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f87567fc700 1 --2- 172.21.15.92:0/212445462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f87580c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 -- 172.21.15.92:0/212445462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 msgr2=0x7f87580c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 --2- 172.21.15.92:0/212445462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f87580c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 -- 172.21.15.92:0/212445462 >> v1:172.21.15.158:6789/0 conn(0x7f87580696a0 legacy=0x7f8758157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.254+0000 7f8756ffd700 1 -- 172.21.15.92:0/212445462 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580c0010 con 0x7f8758154170 2023-05-16T07:38:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f8756ffd700 1 --2- 172.21.15.92:0/212445462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f87580c5c70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f8748009f20 tx=0x7f8748009840).ready entity=mon.0 client_cookie=51130cb22f021b36 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f8755ffb700 1 -- 172.21.15.92:0/212445462 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800e070 con 0x7f8758154170 2023-05-16T07:38:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f8755ffb700 1 -- 172.21.15.92:0/212445462 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874800ba80 con 0x7f8758154170 2023-05-16T07:38:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f8755ffb700 1 -- 172.21.15.92:0/212445462 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800be00 con 0x7f8758154170 2023-05-16T07:38:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 -- 172.21.15.92:0/212445462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 msgr2=0x7f87580c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 --2- 172.21.15.92:0/212445462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f87580c5c70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f8748009f20 tx=0x7f8748009840).stop 2023-05-16T07:38:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 -- 172.21.15.92:0/212445462 shutdown_connections 2023-05-16T07:38:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 --2- 172.21.15.92:0/212445462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f87580c5c70 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 --2- 172.21.15.92:0/212445462 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f87580c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 -- 172.21.15.92:0/212445462 >> 172.21.15.92:0/212445462 conn(0x7f87580cb700 msgr2=0x7f87580c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 -- 172.21.15.92:0/212445462 shutdown_connections 2023-05-16T07:38:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.255+0000 7f875d473700 1 -- 172.21.15.92:0/212445462 wait complete. 2023-05-16T07:38:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.256+0000 7f875d473700 1 Processor -- start 2023-05-16T07:38:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 -- start start 2023-05-16T07:38:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87580696a0 0x7f8758167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f8758158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f875814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758165560 con 0x7f8758154170 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581649b0 con 0x7f87580696a0 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f875d473700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8758164c30 con 0x7f8758154550 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f87567fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f8758158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f87567fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f8758158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52504/0 (socket says 172.21.15.92:52504) 2023-05-16T07:38:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f87567fc700 1 -- 172.21.15.92:0/565408484 learned_addr learned my addr 172.21.15.92:0/565408484 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f8756ffd700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87580696a0 0x7f8758167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.257+0000 7f87577fe700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f875814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f87567fc700 1 -- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 msgr2=0x7f875814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f87567fc700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f875814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f87567fc700 1 -- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87580696a0 msgr2=0x7f8758167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f87567fc700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87580696a0 0x7f8758167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f87567fc700 1 -- 172.21.15.92:0/565408484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8748009ae0 con 0x7f8758154170 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f8756ffd700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87580696a0 0x7f8758167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f87567fc700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f8758158270 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f8740006980 tx=0x7f8740008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400125f0 con 0x7f8758154170 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740012750 con 0x7f8758154170 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740012ad0 con 0x7f8758154170 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8758164f10 con 0x7f8758154170 2023-05-16T07:38:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.258+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8758158760 con 0x7f8758154170 2023-05-16T07:38:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.259+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8758165710 con 0x7f8758154170 2023-05-16T07:38:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.259+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8740013480 con 0x7f8758154170 2023-05-16T07:38:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.259+0000 7f874effd700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f873805ed90 0x7f8738061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.259+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f8740079e40 con 0x7f8758154170 2023-05-16T07:38:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.260+0000 7f8756ffd700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f873805ed90 0x7f8738061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:13.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.260+0000 7f8756ffd700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f873805ed90 0x7f8738061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8748009930 tx=0x7f8748009840).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:13.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.264+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f8758165710 con 0x7f8758154170 2023-05-16T07:38:13.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:13.488+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f87580666c0 con 0x7f8758154170 2023-05-16T07:38:14.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.129+0000 7f874effd700 1 -- 172.21.15.92:0/565408484 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v1233) v1 ==== 158+0+0 (secure 0 0 0) 0x7f874004b230 con 0x7f8758154170 2023-05-16T07:38:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f873805ed90 msgr2=0x7f8738061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f873805ed90 0x7f8738061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8748009930 tx=0x7f8748009840).stop 2023-05-16T07:38:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 msgr2=0x7f8758158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f8758158270 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f8740006980 tx=0x7f8740008040).stop 2023-05-16T07:38:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 shutdown_connections 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f873805ed90 0x7f8738061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8758154550 0x7f875814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f87580696a0 0x7f8758167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 --2- 172.21.15.92:0/565408484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8758154170 0x7f8758158270 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 >> 172.21.15.92:0/565408484 conn(0x7f87580cb700 msgr2=0x7f87580abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 shutdown_connections 2023-05-16T07:38:14.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.131+0000 7f875d473700 1 -- 172.21.15.92:0/565408484 wait complete. 2023-05-16T07:38:14.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-16T07:38:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.262+0000 7fedc11ab700 1 Processor -- start 2023-05-16T07:38:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.262+0000 7fedc11ab700 1 -- start start 2023-05-16T07:38:14.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedc11ab700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedc11ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedbc068160 0x7fedbc0685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedc11ab700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedbc0c94d0 con 0x7fedbc154090 2023-05-16T07:38:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedc11ab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedbc0cb7e0 con 0x7fedbc068160 2023-05-16T07:38:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedc11ab700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc068ac0 con 0x7fedbc154470 2023-05-16T07:38:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedbc068160 0x7fedbc0685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedbc068160 0x7fedbc0685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52514/0 (socket says 172.21.15.92:52514) 2023-05-16T07:38:14.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 -- 172.21.15.92:0/2078733872 learned_addr learned my addr 172.21.15.92:0/2078733872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedba59c700 1 --2- 172.21.15.92:0/2078733872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 -- 172.21.15.92:0/2078733872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 msgr2=0x7fedbc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 --2- 172.21.15.92:0/2078733872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 -- 172.21.15.92:0/2078733872 >> v1:172.21.15.158:6789/0 conn(0x7fedbc154090 legacy=0x7fedbc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 -- 172.21.15.92:0/2078733872 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc0c88c0 con 0x7fedbc068160 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedbb59e700 1 --2- 172.21.15.92:0/2078733872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0685d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7fedb4006fd0 tx=0x7fedb40090f0).ready entity=mon.0 client_cookie=edda5c467e5a85fe server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedb9d9b700 1 -- 172.21.15.92:0/2078733872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400b600 con 0x7fedbc068160 2023-05-16T07:38:14.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.263+0000 7fedb9d9b700 1 -- 172.21.15.92:0/2078733872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb400b760 con 0x7fedbc068160 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedb9d9b700 1 -- 172.21.15.92:0/2078733872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400bae0 con 0x7fedbc068160 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 -- 172.21.15.92:0/2078733872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 msgr2=0x7fedbc0685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 --2- 172.21.15.92:0/2078733872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0685d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7fedb4006fd0 tx=0x7fedb40090f0).stop 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 -- 172.21.15.92:0/2078733872 shutdown_connections 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 --2- 172.21.15.92:0/2078733872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0685d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 --2- 172.21.15.92:0/2078733872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 -- 172.21.15.92:0/2078733872 >> 172.21.15.92:0/2078733872 conn(0x7fedbc067920 msgr2=0x7fedbc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:14.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 -- 172.21.15.92:0/2078733872 shutdown_connections 2023-05-16T07:38:14.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.264+0000 7fedc11ab700 1 -- 172.21.15.92:0/2078733872 wait complete. 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.265+0000 7fedc11ab700 1 Processor -- start 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.265+0000 7fedc11ab700 1 -- start start 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedc11ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedc11ab700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fedbc154090 0x7fedbc14f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedc11ab700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedc11ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc0c4ae0 con 0x7fedbc068160 2023-05-16T07:38:14.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedc11ab700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc0c3ed0 con 0x7fedbc154090 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedc11ab700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc0c41b0 con 0x7fedbc154470 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52516/0 (socket says 172.21.15.92:52516) 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 -- 172.21.15.92:0/1286510895 learned_addr learned my addr 172.21.15.92:0/1286510895 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbb59e700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedba59c700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fedbc154090 0x7fedbc14f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 -- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 msgr2=0x7fedbc0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 -- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fedbc154090 msgr2=0x7fedbc14f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fedbc154090 0x7fedbc14f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.266+0000 7fedbad9d700 1 -- 172.21.15.92:0/1286510895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4009370 con 0x7fedbc068160 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.267+0000 7fedbad9d700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0c4f70 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fedac00c810 tx=0x7fedac0025e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.267+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac010510 con 0x7fedbc068160 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.267+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac0106b0 con 0x7fedbc068160 2023-05-16T07:38:14.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.267+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc0c4490 con 0x7fedbc068160 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.267+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac010af0 con 0x7fedbc068160 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.267+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedbc14fdf0 con 0x7fedbc068160 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.268+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedbc0c4dc0 con 0x7fedbc068160 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.268+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fedac010070 con 0x7fedbc068160 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.268+0000 7fedb2ffd700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fed9c05ed90 0x7fed9c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.268+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6453+0+0 (secure 0 0 0) 0x7fedac079a00 con 0x7fedbc068160 2023-05-16T07:38:14.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.269+0000 7fedba59c700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fed9c05ed90 0x7fed9c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.269+0000 7fedba59c700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fed9c05ed90 0x7fed9c061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fedbc0c5460 tx=0x7feda4008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:14.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.273+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fedac04ae50 con 0x7fedbc068160 2023-05-16T07:38:14.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.436+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fedbc066730 con 0x7fedbc068160 2023-05-16T07:38:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.439+0000 7fedb2ffd700 1 -- 172.21.15.92:0/1286510895 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedac04e470 con 0x7fedbc068160 2023-05-16T07:38:14.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fed9c05ed90 msgr2=0x7fed9c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fed9c05ed90 0x7fed9c061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fedbc0c5460 tx=0x7feda4008040).stop 2023-05-16T07:38:14.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 msgr2=0x7fedbc0c4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0c4f70 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fedac00c810 tx=0x7fedac0025e0).stop 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 shutdown_connections 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fed9c05ed90 0x7fed9c061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fedbc154470 0x7fedbc0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedbc068160 0x7fedbc0c4f70 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 --2- 172.21.15.92:0/1286510895 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fedbc154090 0x7fedbc14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 >> 172.21.15.92:0/1286510895 conn(0x7fedbc067920 msgr2=0x7fedbc0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:14.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 shutdown_connections 2023-05-16T07:38:14.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.441+0000 7fedc11ab700 1 -- 172.21.15.92:0/1286510895 wait complete. 2023-05-16T07:38:14.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:38:14.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:38:14.485 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:38:14.521 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:38:14.522 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-16T07:38:14.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 Processor -- start 2023-05-16T07:38:14.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 -- start start 2023-05-16T07:38:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2154154180 0x7f2154067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f2154063720 0x7f2154063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2154063350 con 0x7f2154153da0 2023-05-16T07:38:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2154068450 con 0x7f2154063720 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.635+0000 7f215ad7a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540640c0 con 0x7f2154154180 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2158b16700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2154153da0 0x7f21540699e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35040/0 (socket says 172.21.15.92:35040) 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2158b16700 1 -- 172.21.15.92:0/4257472476 learned_addr learned my addr 172.21.15.92:0/4257472476 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2159317700 1 --2- 172.21.15.92:0/4257472476 >> v2:172.21.15.158:3300/0 conn(0x7f2154063720 0x7f2154063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2154154180 0x7f2154067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202480492 0 0) 0x7f2144001490 con 0x7f2154153da0 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f213c0033f0 con 0x7f2154153da0 2023-05-16T07:38:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3742216027 0 0) 0x7f213c0033f0 con 0x7f2154153da0 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2154154180 msgr2=0x7f2154067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2154154180 0x7f2154067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 >> v2:172.21.15.158:3300/0 conn(0x7f2154063720 msgr2=0x7f2154063bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 --2- 172.21.15.92:0/4257472476 >> v2:172.21.15.158:3300/0 conn(0x7f2154063720 0x7f2154063bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154062ec0 con 0x7f2154153da0 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f2144001980 con 0x7f2154153da0 2023-05-16T07:38:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 >> v1:172.21.15.92:6789/0 conn(0x7f2154153da0 legacy=0x7f21540699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f213c0038e0 0x7f213c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f213c004690 0x7f213c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c005000 0x7f213c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154068450 con 0x7f213c004690 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540640c0 con 0x7f213c0038e0 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0054c0 con 0x7f213c005000 2023-05-16T07:38:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f213c004690 0x7f213c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2158b16700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f213c0038e0 0x7f213c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2159317700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c005000 0x7f213c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 -- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c005000 msgr2=0x7f213c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c005000 0x7f213c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 -- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f213c0038e0 msgr2=0x7f213c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f213c0038e0 0x7f213c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.636+0000 7f2153fff700 1 -- 172.21.15.92:0/4257472476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0033f0 con 0x7f213c004690 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f2158b16700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f213c0038e0 0x7f213c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f2159317700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c005000 0x7f213c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f2153fff700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f213c004690 0x7f213c004b10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f2148009a90 tx=0x7f2148009060).ready entity=mon.0 client_cookie=c1fac4dbe53c9169 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148009430 con 0x7f213c004690 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214800e640 con 0x7f213c004690 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f21537fe700 1 -- 172.21.15.92:0/4257472476 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214800e9c0 con 0x7f213c004690 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f215ad7a700 1 -- 172.21.15.92:0/4257472476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f213c004690 msgr2=0x7f213c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f215ad7a700 1 --2- 172.21.15.92:0/4257472476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f213c004690 0x7f213c004b10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f2148009a90 tx=0x7f2148009060).stop 2023-05-16T07:38:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f2159317700 1 -- 172.21.15.92:0/4257472476 reap_dead start 2023-05-16T07:38:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f215ad7a700 1 -- 172.21.15.92:0/4257472476 shutdown_connections 2023-05-16T07:38:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f215ad7a700 1 -- 172.21.15.92:0/4257472476 >> 172.21.15.92:0/4257472476 conn(0x7f21540c8aa0 msgr2=0x7f21540c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f215ad7a700 1 -- 172.21.15.92:0/4257472476 shutdown_connections 2023-05-16T07:38:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.637+0000 7f215ad7a700 1 -- 172.21.15.92:0/4257472476 wait complete. 2023-05-16T07:38:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.638+0000 7f215ad7a700 1 Processor -- start 2023-05-16T07:38:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 -- start start 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c004690 0x7f21540bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 0x7f21540c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f21540c2690 0x7f21540c0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c0500 con 0x7f215414f900 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c2b10 con 0x7f21540c2690 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f215ad7a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21541556a0 con 0x7f213c004690 2023-05-16T07:38:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f2153fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 0x7f21540c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f2153fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 0x7f21540c21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52538/0 (socket says 172.21.15.92:52538) 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.639+0000 7f2153fff700 1 -- 172.21.15.92:0/1498191770 learned_addr learned my addr 172.21.15.92:0/1498191770 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2159317700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f21540c2690 0x7f21540c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2158b16700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c004690 0x7f21540bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2153fff700 1 -- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c004690 msgr2=0x7f21540bfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2153fff700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c004690 0x7f21540bfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2153fff700 1 -- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f21540c2690 msgr2=0x7f21540c0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2153fff700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f21540c2690 0x7f21540c0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2153fff700 1 -- 172.21.15.92:0/1498191770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540c51d0 con 0x7f215414f900 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2153fff700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 0x7f21540c21a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f214801bed0 tx=0x7f214801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148020070 con 0x7f215414f900 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2148009ed0 con 0x7f215414f900 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21540c54b0 con 0x7f215414f900 2023-05-16T07:38:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148024410 con 0x7f215414f900 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.640+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21540c0030 con 0x7f215414f900 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.641+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21540c59d0 con 0x7f215414f900 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.641+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f214802b030 con 0x7f215414f900 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.642+0000 7f2150ff9700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f214005ed90 0x7f2140061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.642+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f214807b1f0 con 0x7f215414f900 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.642+0000 7f2158b16700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f214005ed90 0x7f2140061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.642+0000 7f2158b16700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f214005ed90 0x7f2140061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2144001190 tx=0x7f2144009040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:14.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.646+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f21540c59d0 con 0x7f215414f900 2023-05-16T07:38:14.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.849+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21540c02d0 con 0x7f215414f900 2023-05-16T07:38:14.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.849+0000 7f2150ff9700 1 -- 172.21.15.92:0/1498191770 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f21540c02d0 con 0x7f215414f900 2023-05-16T07:38:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:14.853 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6836/2508205729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2508205729},{"type":"v1","addr":"172.21.15.158:6836","nonce":2508205729}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8089,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/746112960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6835","nonce":746112960},{"type":"v1","addr":"172.21.15.158:6837","nonce":746112960}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1498166107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1498166107},{"type":"v1","addr":"172.21.15.92:6833","nonce":1498166107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1098606551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1098606551},{"type":"v1","addr":"172.21.15.92:6835","nonce":1098606551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[]} 2023-05-16T07:38:14.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f214005ed90 msgr2=0x7f2140061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f214005ed90 0x7f2140061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2144001190 tx=0x7f2144009040).stop 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 msgr2=0x7f21540c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 0x7f21540c21a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f214801bed0 tx=0x7f214801a000).stop 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 shutdown_connections 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f214005ed90 0x7f2140061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f213c004690 0x7f21540bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f21540c2690 0x7f21540c0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 --2- 172.21.15.92:0/1498191770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f215414f900 0x7f21540c21a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 >> 172.21.15.92:0/1498191770 conn(0x7f21540c8aa0 msgr2=0x7f21540ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 shutdown_connections 2023-05-16T07:38:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:14.851+0000 7f215ad7a700 1 -- 172.21.15.92:0/1498191770 wait complete. 2023-05-16T07:38:14.856 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 57 2023-05-16T07:38:14.866 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-16T07:38:14.866 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-16T07:38:14.867 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:38:14.867 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:38:14.867 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-16T07:38:14.867 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-16T07:38:14.868 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:38:14.868 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:38:14.868 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-16T07:38:14.868 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-16T07:38:14.869 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:38:14.869 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:38:14.869 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-16T07:38:14.869 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-16T07:38:14.869 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:38:14.870 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:38:14.875 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:38:14.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-16T07:38:14.876 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:38:14.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-16T07:38:14.880 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:38:14.880 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-16T07:38:14.881 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:38:14.881 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-16T07:38:14.918 INFO:tasks.ceph.mds.a:Started 2023-05-16T07:38:14.918 DEBUG:teuthology.parallel:result is None 2023-05-16T07:38:14.919 INFO:tasks.ceph.mds.c:Started 2023-05-16T07:38:14.919 DEBUG:teuthology.parallel:result is None 2023-05-16T07:38:14.922 INFO:tasks.ceph.mds.b:Started 2023-05-16T07:38:14.922 DEBUG:teuthology.parallel:result is None 2023-05-16T07:38:14.923 INFO:tasks.ceph.mds.d:Started 2023-05-16T07:38:14.923 DEBUG:teuthology.parallel:result is None 2023-05-16T07:38:14.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-16T07:38:14.988 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-05-16T07:38:14.989 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-05-16T07:38:15.004 INFO:tasks.ceph.mds.b.smithi158.stdout:starting mds.b at 2023-05-16T07:38:15.005 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:38:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.039+0000 7f5a834df700 1 Processor -- start 2023-05-16T07:38:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.039+0000 7f5a834df700 1 -- start start 2023-05-16T07:38:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a834df700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f5a7c154170 0x7f5a7c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a834df700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a7c154550 0x7f5a7c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a834df700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c0c0660 con 0x7f5a7c0696a0 2023-05-16T07:38:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a834df700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c1b00 con 0x7f5a7c154550 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a834df700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c0bfd30 con 0x7f5a7c154170 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a80a7a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a7c154550 0x7f5a7c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a80a7a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a7c154550 0x7f5a7c0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52566/0 (socket says 172.21.15.92:52566) 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.040+0000 7f5a80a7a700 1 -- 172.21.15.92:0/2603586966 learned_addr learned my addr 172.21.15.92:0/2603586966 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a8127b700 1 --2- 172.21.15.92:0/2603586966 >> v2:172.21.15.158:3300/0 conn(0x7f5a7c154170 0x7f5a7c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a80a7a700 1 -- 172.21.15.92:0/2603586966 >> v2:172.21.15.158:3300/0 conn(0x7f5a7c154170 msgr2=0x7f5a7c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a80a7a700 1 --2- 172.21.15.92:0/2603586966 >> v2:172.21.15.158:3300/0 conn(0x7f5a7c154170 0x7f5a7c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a80a7a700 1 -- 172.21.15.92:0/2603586966 >> v1:172.21.15.158:6789/0 conn(0x7f5a7c0696a0 legacy=0x7f5a7c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a80a7a700 1 -- 172.21.15.92:0/2603586966 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0c0010 con 0x7f5a7c154550 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a8127b700 1 --2- 172.21.15.92:0/2603586966 >> v2:172.21.15.158:3300/0 conn(0x7f5a7c154170 0x7f5a7c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a80a7a700 1 --2- 172.21.15.92:0/2603586966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154550 0x7f5a7c0c6160 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f5a7c083030 tx=0x7f5a700090f0).ready entity=mon.0 client_cookie=48f7d5ba6a0388db server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a7bfff700 1 -- 172.21.15.92:0/2603586966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000b5a0 con 0x7f5a7c154550 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a7bfff700 1 -- 172.21.15.92:0/2603586966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7000b700 con 0x7f5a7c154550 2023-05-16T07:38:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a7bfff700 1 -- 172.21.15.92:0/2603586966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000baa0 con 0x7f5a7c154550 2023-05-16T07:38:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a834df700 1 -- 172.21.15.92:0/2603586966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154550 msgr2=0x7f5a7c0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a834df700 1 --2- 172.21.15.92:0/2603586966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154550 0x7f5a7c0c6160 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f5a7c083030 tx=0x7f5a700090f0).stop 2023-05-16T07:38:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a834df700 1 -- 172.21.15.92:0/2603586966 shutdown_connections 2023-05-16T07:38:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a834df700 1 --2- 172.21.15.92:0/2603586966 >> v2:172.21.15.158:3300/0 conn(0x7f5a7c154170 0x7f5a7c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a834df700 1 --2- 172.21.15.92:0/2603586966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154550 0x7f5a7c0c6160 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.041+0000 7f5a834df700 1 -- 172.21.15.92:0/2603586966 >> 172.21.15.92:0/2603586966 conn(0x7f5a7c0cb700 msgr2=0x7f5a7c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.042+0000 7f5a834df700 1 -- 172.21.15.92:0/2603586966 shutdown_connections 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.042+0000 7f5a834df700 1 -- 172.21.15.92:0/2603586966 wait complete. 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 Processor -- start 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 -- start start 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a7c0696a0 0x7f5a7c1642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:15.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 0x7f5a7c157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a7c154550 0x7f5a7c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c3460 con 0x7f5a7c154170 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1647e0 con 0x7f5a7c154550 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.043+0000 7f5a834df700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c158690 con 0x7f5a7c0696a0 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 0x7f5a7c157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 0x7f5a7c157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52572/0 (socket says 172.21.15.92:52572) 2023-05-16T07:38:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 -- 172.21.15.92:0/1814504510 learned_addr learned my addr 172.21.15.92:0/1814504510 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a8127b700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a7c0696a0 0x7f5a7c1642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a81a7c700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a7c154550 0x7f5a7c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 -- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a7c0696a0 msgr2=0x7f5a7c1642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a7c0696a0 0x7f5a7c1642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 -- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a7c154550 msgr2=0x7f5a7c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a7c154550 0x7f5a7c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 -- 172.21.15.92:0/1814504510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a70009370 con 0x7f5a7c154170 2023-05-16T07:38:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a80a7a700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 0x7f5a7c157d90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f5a70011000 tx=0x7f5a70015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000bee0 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7000b870 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c0c3670 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7001c420 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.044+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c0c3c30 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.045+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c0666c0 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.045+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5a7002b070 con 0x7f5a7c154170 2023-05-16T07:38:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.046+0000 7f5a797fa700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5a6805ed90 0x7f5a68061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.046+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f5a7007b610 con 0x7f5a7c154170 2023-05-16T07:38:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.046+0000 7f5a8127b700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5a6805ed90 0x7f5a68061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.046+0000 7f5a8127b700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5a6805ed90 0x7f5a68061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5a6c00d6e0 tx=0x7f5a6c007850).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.050+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f5a7004ca60 con 0x7f5a7c154170 2023-05-16T07:38:15.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:15.229+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5a7c06c350 con 0x7f5a7c154170 2023-05-16T07:38:16.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.136+0000 7f5a797fa700 1 -- 172.21.15.92:0/1814504510 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v167) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5a70020020 con 0x7f5a7c154170 2023-05-16T07:38:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5a6805ed90 msgr2=0x7f5a68061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5a6805ed90 0x7f5a68061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5a6c00d6e0 tx=0x7f5a6c007850).stop 2023-05-16T07:38:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 msgr2=0x7f5a7c157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 0x7f5a7c157d90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f5a70011000 tx=0x7f5a70015000).stop 2023-05-16T07:38:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 shutdown_connections 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f5a6805ed90 0x7f5a68061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5a7c0696a0 0x7f5a7c1642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5a7c154550 0x7f5a7c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 --2- 172.21.15.92:0/1814504510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c154170 0x7f5a7c157d90 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.138+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 >> 172.21.15.92:0/1814504510 conn(0x7f5a7c0cb700 msgr2=0x7f5a7c14da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.139+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 shutdown_connections 2023-05-16T07:38:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.139+0000 7f5a834df700 1 -- 172.21.15.92:0/1814504510 wait complete. 2023-05-16T07:38:16.144 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-05-16T07:38:16.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-16T07:38:16.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.269+0000 7fbfa3c7f700 1 Processor -- start 2023-05-16T07:38:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.269+0000 7fbfa3c7f700 1 -- start start 2023-05-16T07:38:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.270+0000 7fbfa3c7f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbf9c069db0 0x7fbf9c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.270+0000 7fbfa3c7f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0c0660 con 0x7fbf9c154170 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.270+0000 7fbfa3c7f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0c1b00 con 0x7fbf9c154550 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.270+0000 7fbfa3c7f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0bfd30 con 0x7fbf9c069db0 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.270+0000 7fbfa1a1b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbf9c154170 0x7fbf9c0c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35098/0 (socket says 172.21.15.92:35098) 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.270+0000 7fbfa1a1b700 1 -- 172.21.15.92:0/3666420422 learned_addr learned my addr 172.21.15.92:0/3666420422 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460770201 0 0) 0x7fbf9c0c0660 con 0x7fbf9c154170 2023-05-16T07:38:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf840033f0 con 0x7fbf9c154170 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa221c700 1 --2- 172.21.15.92:0/3666420422 >> v2:172.21.15.158:3300/0 conn(0x7fbf9c069db0 0x7fbf9c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382590656 0 0) 0x7fbf840033f0 con 0x7fbf9c154170 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 >> v2:172.21.15.158:3300/0 conn(0x7fbf9c069db0 msgr2=0x7fbf9c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 --2- 172.21.15.92:0/3666420422 >> v2:172.21.15.158:3300/0 conn(0x7fbf9c069db0 0x7fbf9c15f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 >> v1:172.21.15.158:6789/0 conn(0x7fbf9c154550 legacy=0x7fbf9c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0c0010 con 0x7fbf9c154170 2023-05-16T07:38:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fbf8c0017f0 con 0x7fbf9c154170 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 >> v1:172.21.15.92:6789/0 conn(0x7fbf9c154170 legacy=0x7fbf9c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf840038e0 0x7fbf84003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf84004690 0x7fbf84004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840050d0 0x7fbf8400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0bfd30 con 0x7fbf840038e0 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8400c060 con 0x7fbf840050d0 2023-05-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8400c220 con 0x7fbf84004690 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa221c700 1 --2- 172.21.15.92:0/3666420422 >> v2:172.21.15.158:3300/0 conn(0x7fbf9c069db0 0x7fbf9c15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf840038e0 0x7fbf84003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa121a700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf84004690 0x7fbf84004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa221c700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840050d0 0x7fbf8400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 -- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf84004690 msgr2=0x7fbf84004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf84004690 0x7fbf84004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 -- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840050d0 msgr2=0x7fbf8400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840050d0 0x7fbf8400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa221c700 1 -- 172.21.15.92:0/3666420422 reap_dead start 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 -- 172.21.15.92:0/3666420422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840033f0 con 0x7fbf840038e0 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.271+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf840038e0 0x7fbf84003ce0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fbf8c00c5c0 tx=0x7fbf8c001d90).ready entity=mon.0 client_cookie=9166209eca392ba6 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00ce90 con 0x7fbf840038e0 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c001fc0 con 0x7fbf840038e0 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa0a19700 1 -- 172.21.15.92:0/3666420422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c005d70 con 0x7fbf840038e0 2023-05-16T07:38:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/3666420422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf840038e0 msgr2=0x7fbf84003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf840038e0 0x7fbf84003ce0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fbf8c00c5c0 tx=0x7fbf8c001d90).stop 2023-05-16T07:38:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/3666420422 shutdown_connections 2023-05-16T07:38:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/3666420422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf840038e0 0x7fbf84003ce0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/3666420422 >> 172.21.15.92:0/3666420422 conn(0x7fbf9c0cb700 msgr2=0x7fbf9c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/3666420422 shutdown_connections 2023-05-16T07:38:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.272+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/3666420422 wait complete. 2023-05-16T07:38:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.273+0000 7fbfa3c7f700 1 Processor -- start 2023-05-16T07:38:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.273+0000 7fbfa3c7f700 1 -- start start 2023-05-16T07:38:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa3c7f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840038e0 0x7fbf9c0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa3c7f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf9c0aedb0 0x7fbf9c150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa3c7f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 0x7fbf9c167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa3c7f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c2bf0 con 0x7fbf9c150c20 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa3c7f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b0360 con 0x7fbf840038e0 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa3c7f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b0640 con 0x7fbf9c0aedb0 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 0x7fbf9c167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 0x7fbf9c167fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52598/0 (socket says 172.21.15.92:52598) 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 -- 172.21.15.92:0/2626194505 learned_addr learned my addr 172.21.15.92:0/2626194505 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840038e0 0x7fbf9c0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa121a700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf9c0aedb0 0x7fbf9c150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 -- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf9c0aedb0 msgr2=0x7fbf9c150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf9c0aedb0 0x7fbf9c150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 -- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840038e0 msgr2=0x7fbf9c0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840038e0 0x7fbf9c0afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 -- 172.21.15.92:0/2626194505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0c2ec0 con 0x7fbf9c150c20 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa121a700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf9c0aedb0 0x7fbf9c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.274+0000 7fbfa221c700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 0x7fbf9c167fe0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7fbf9c083aa0 tx=0x7fbf94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.275+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400fdf0 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.275+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94013400 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.275+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0c31a0 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.275+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94013780 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.275+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c164950 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.276+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c0666c0 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.276+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbf94013af0 con 0x7fbf9c150c20 2023-05-16T07:38:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.276+0000 7fbf99ffb700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf8805ed90 0x7fbf88061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.276+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5937+0+0 (secure 0 0 0) 0x7fbf94079780 con 0x7fbf9c150c20 2023-05-16T07:38:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.277+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf8805ed90 0x7fbf88061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.277+0000 7fbfa1a1b700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf8805ed90 0x7fbf88061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fbf8c001460 tx=0x7fbf8c015000).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.281+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbf9404add0 con 0x7fbf9c150c20 2023-05-16T07:38:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.503+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbf9c06c350 con 0x7fbf9c150c20 2023-05-16T07:38:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.504+0000 7fbf99ffb700 1 -- 172.21.15.92:0/2626194505 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fbf9404e3f0 con 0x7fbf9c150c20 2023-05-16T07:38:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:16.507 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:16.507 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk6YtJFhAA96xhRmacju6SoY5a9g9CfQ==", 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:38:16.508 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiXsnIRAAN4ARZQqIOvQ55UiLMY7FHQ==", 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.509 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.510 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk+8HpHBAAlDdqMRtB5mo+Z7aRkw8bcg==", 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkhxHCJxAA8EZhbZbGebkGur6TNWda6g==", 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:38:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBUKmNkuWKuHxAAjBynifRL41RiV9JV04rDBg==", 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.513 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBXKmNkClDjJBAA+MGnG5O5qJLzqJiiSk0reQ==", 2023-05-16T07:38:16.514 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBaKmNk0Re2JhAApnqU+mClcmgcB8EKR/dKgw==", 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBdKmNk5QzlKBAAnT6GS6Xy1Ta5b4BDXetUzw==", 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBhKmNkMV+7MhAAbchMfJidetp7TVlmgDzXnw==", 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBkKmNkiavqMhAAnmAMuu5pRkkIHN5dtDa3Rg==", 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.519 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBnKmNkirPlMBAAqwVAkOtjiN7ryd/NWcjmcg==", 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.520 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-05-16T07:38:16.521 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBqKmNkkrdZMBAActGd+Eu8d634tqFimKJ2fg==", 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.522 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNko0BrKxAAQRZW3HzsEKskzkPV3rQloQ==", 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1m+uMxAAebOrGzmiKkRKEtB+jfqrlQ==", 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:38:16.524 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkVti+MBAAsM2HPSgPpZ+8WgOAIMXICw==", 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.525 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-05-16T07:38:16.526 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkqc3pOBAAApUwarRJiMOI/9f971CZ6g==", 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBQKmNkvoK6NxAAplUutALw/Qyx5HaG6Wp+Dw==", 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-05-16T07:38:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNk1QZmExAAgfpZg37RiO5+QYMYH4QjXg==", 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:38:16.529 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBRKmNkiEMlEBAAOfJfO36Cmgpvy2jvlqzg7w==", 2023-05-16T07:38:16.530 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-05-16T07:38:16.531 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf8805ed90 msgr2=0x7fbf88061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf8805ed90 0x7fbf88061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fbf8c001460 tx=0x7fbf8c015000).stop 2023-05-16T07:38:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 msgr2=0x7fbf9c167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 0x7fbf9c167fe0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7fbf9c083aa0 tx=0x7fbf94008040).stop 2023-05-16T07:38:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 shutdown_connections 2023-05-16T07:38:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf9c150c20 0x7fbf9c167fe0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf840038e0 0x7fbf9c0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf8805ed90 0x7fbf88061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 --2- 172.21.15.92:0/2626194505 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf9c0aedb0 0x7fbf9c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 >> 172.21.15.92:0/2626194505 conn(0x7fbf9c0cb700 msgr2=0x7fbf9c0c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:16.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 shutdown_connections 2023-05-16T07:38:16.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.505+0000 7fbfa3c7f700 1 -- 172.21.15.92:0/2626194505 wait complete. 2023-05-16T07:38:16.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:16.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.642+0000 7f18f42bb700 1 Processor -- start 2023-05-16T07:38:16.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.642+0000 7f18f42bb700 1 -- start start 2023-05-16T07:38:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f42bb700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f18ec154170 0x7f18ec0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f42bb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18ec154550 0x7f18ec0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f42bb700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0c0660 con 0x7f18ec0696a0 2023-05-16T07:38:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f42bb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0c1b00 con 0x7f18ec154550 2023-05-16T07:38:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f42bb700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0bfd30 con 0x7f18ec154170 2023-05-16T07:38:16.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18ec154550 0x7f18ec0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18ec154550 0x7f18ec0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52604/0 (socket says 172.21.15.92:52604) 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 -- 172.21.15.92:0/3720813208 learned_addr learned my addr 172.21.15.92:0/3720813208 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f2057700 1 --2- 172.21.15.92:0/3720813208 >> v2:172.21.15.158:3300/0 conn(0x7f18ec154170 0x7f18ec0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 -- 172.21.15.92:0/3720813208 >> v2:172.21.15.158:3300/0 conn(0x7f18ec154170 msgr2=0x7f18ec0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 --2- 172.21.15.92:0/3720813208 >> v2:172.21.15.158:3300/0 conn(0x7f18ec154170 0x7f18ec0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 -- 172.21.15.92:0/3720813208 >> v1:172.21.15.158:6789/0 conn(0x7f18ec0696a0 legacy=0x7f18ec157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 -- 172.21.15.92:0/3720813208 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0c0010 con 0x7f18ec154550 2023-05-16T07:38:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.643+0000 7f18f1856700 1 --2- 172.21.15.92:0/3720813208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0c6160 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f18ec083030 tx=0x7f18dc0090f0).ready entity=mon.0 client_cookie=2c4d72656fedb295 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f1055700 1 -- 172.21.15.92:0/3720813208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00b5a0 con 0x7f18ec154550 2023-05-16T07:38:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f1055700 1 -- 172.21.15.92:0/3720813208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00b700 con 0x7f18ec154550 2023-05-16T07:38:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f1055700 1 -- 172.21.15.92:0/3720813208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00baa0 con 0x7f18ec154550 2023-05-16T07:38:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 -- 172.21.15.92:0/3720813208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 msgr2=0x7f18ec0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 --2- 172.21.15.92:0/3720813208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0c6160 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f18ec083030 tx=0x7f18dc0090f0).stop 2023-05-16T07:38:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 -- 172.21.15.92:0/3720813208 shutdown_connections 2023-05-16T07:38:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 --2- 172.21.15.92:0/3720813208 >> v2:172.21.15.158:3300/0 conn(0x7f18ec154170 0x7f18ec0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 --2- 172.21.15.92:0/3720813208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0c6160 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 -- 172.21.15.92:0/3720813208 >> 172.21.15.92:0/3720813208 conn(0x7f18ec0cb700 msgr2=0x7f18ec0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 -- 172.21.15.92:0/3720813208 shutdown_connections 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.644+0000 7f18f42bb700 1 -- 172.21.15.92:0/3720813208 wait complete. 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.645+0000 7f18f42bb700 1 Processor -- start 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.645+0000 7f18f42bb700 1 -- start start 2023-05-16T07:38:16.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f42bb700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18ec0696a0 0x7f18ec14d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f42bb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18ec154170 0x7f18ec0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f42bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f42bb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0afaf0 con 0x7f18ec154550 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f42bb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec168080 con 0x7f18ec154170 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f42bb700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec168360 con 0x7f18ec0696a0 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52610/0 (socket says 172.21.15.92:52610) 2023-05-16T07:38:16.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 -- 172.21.15.92:0/601082128 learned_addr learned my addr 172.21.15.92:0/601082128 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f1856700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18ec154170 0x7f18ec0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2057700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18ec0696a0 0x7f18ec14d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 -- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18ec0696a0 msgr2=0x7f18ec14d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18ec0696a0 0x7f18ec14d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 -- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18ec154170 msgr2=0x7f18ec0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18ec154170 0x7f18ec0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 -- 172.21.15.92:0/601082128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009370 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.646+0000 7f18f2858700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0af600 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f18e400c810 tx=0x7f18e4002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.647+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4010510 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.647+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4010690 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.647+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec168640 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.647+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4010a50 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.647+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec168c00 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.647+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec14fc70 con 0x7f18ec154550 2023-05-16T07:38:16.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.648+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f18e4019030 con 0x7f18ec154550 2023-05-16T07:38:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.648+0000 7f18e27fc700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f18d005ed60 0x7f18d0061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.648+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5937+0+0 (secure 0 0 0) 0x7f18e4079240 con 0x7f18ec154550 2023-05-16T07:38:16.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.648+0000 7f18f2057700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f18d005ed60 0x7f18d0061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.649+0000 7f18f2057700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f18d005ed60 0x7f18d0061220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f18e800bbc0 tx=0x7f18e8008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.652+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f18e40488f0 con 0x7f18ec154550 2023-05-16T07:38:16.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.816+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18ec0666c0 con 0x7f18ec154550 2023-05-16T07:38:16.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.817+0000 7f18e27fc700 1 -- 172.21.15.92:0/601082128 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+14789 (secure 0 0 0) 0x7f18ec14fc70 con 0x7f18ec154550 2023-05-16T07:38:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:16.820 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":167,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:16.130547+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:16.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f18d005ed60 msgr2=0x7f18d0061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f18d005ed60 0x7f18d0061220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f18e800bbc0 tx=0x7f18e8008040).stop 2023-05-16T07:38:16.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 msgr2=0x7f18ec0af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0af600 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f18e400c810 tx=0x7f18e4002560).stop 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 shutdown_connections 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f18d005ed60 0x7f18d0061220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f18ec0696a0 0x7f18ec14d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18ec154550 0x7f18ec0af600 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 --2- 172.21.15.92:0/601082128 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f18ec154170 0x7f18ec0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 >> 172.21.15.92:0/601082128 conn(0x7f18ec0cb700 msgr2=0x7f18ec153460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:16.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 shutdown_connections 2023-05-16T07:38:16.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.819+0000 7f18f42bb700 1 -- 172.21.15.92:0/601082128 wait complete. 2023-05-16T07:38:16.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-05-16T07:38:16.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:38:16.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.944+0000 7f6c70078700 1 Processor -- start 2023-05-16T07:38:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.945+0000 7f6c70078700 1 -- start start 2023-05-16T07:38:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.945+0000 7f6c70078700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c680aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.945+0000 7f6c70078700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680c94d0 con 0x7f6c68154090 2023-05-16T07:38:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.945+0000 7f6c70078700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680cb7e0 con 0x7f6c68154470 2023-05-16T07:38:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.945+0000 7f6c70078700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c88c0 con 0x7f6c68068870 2023-05-16T07:38:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.945+0000 7f6c6de14700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c68154090 0x7f6c680a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35132/0 (socket says 172.21.15.92:35132) 2023-05-16T07:38:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6de14700 1 -- 172.21.15.92:0/2021858256 learned_addr learned my addr 172.21.15.92:0/2021858256 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495661460 0 0) 0x7f6c680c94d0 con 0x7f6c68154090 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c500033f0 con 0x7f6c68154090 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6e615700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c680aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108150006 0 0) 0x7f6c500033f0 con 0x7f6c68154090 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 msgr2=0x7f6c680aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c680aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 >> v1:172.21.15.158:6789/0 conn(0x7f6c68154470 legacy=0x7f6c68068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680c8ba0 con 0x7f6c68154090 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f6c580017f0 con 0x7f6c68154090 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 >> v1:172.21.15.92:6789/0 conn(0x7f6c68154090 legacy=0x7f6c680a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c50003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c50003ef0 0x7f6c50004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c50004b80 0x7f6c50005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c88c0 con 0x7f6c50003ef0 2023-05-16T07:38:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5000ba90 con 0x7f6c50004b80 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5000bc50 con 0x7f6c68068870 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6d613700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c50003ef0 0x7f6c50004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6de14700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c50003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6e615700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c50004b80 0x7f6c50005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6d613700 1 -- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 msgr2=0x7f6c50003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6d613700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c50003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6d613700 1 -- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c50004b80 msgr2=0x7f6c50005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6d613700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c50004b80 0x7f6c50005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6d613700 1 -- 172.21.15.92:0/2021858256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c500033f0 con 0x7f6c50003ef0 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6de14700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c68068870 0x7f6c50003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.946+0000 7f6c6e615700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c50004b80 0x7f6c50005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c6d613700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c50003ef0 0x7f6c50004690 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f6c5c001730 tx=0x7f6c5c007bb0).ready entity=mon.0 client_cookie=bb490fbd862f5293 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c007020 con 0x7f6c50003ef0 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c007180 con 0x7f6c50003ef0 2023-05-16T07:38:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c6ce12700 1 -- 172.21.15.92:0/2021858256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c007500 con 0x7f6c50003ef0 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c70078700 1 -- 172.21.15.92:0/2021858256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c50003ef0 msgr2=0x7f6c50004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c70078700 1 --2- 172.21.15.92:0/2021858256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c50003ef0 0x7f6c50004690 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f6c5c001730 tx=0x7f6c5c007bb0).stop 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c6e615700 1 -- 172.21.15.92:0/2021858256 reap_dead start 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c70078700 1 -- 172.21.15.92:0/2021858256 shutdown_connections 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c70078700 1 -- 172.21.15.92:0/2021858256 >> 172.21.15.92:0/2021858256 conn(0x7f6c68067920 msgr2=0x7f6c680cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c70078700 1 -- 172.21.15.92:0/2021858256 shutdown_connections 2023-05-16T07:38:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.947+0000 7f6c70078700 1 -- 172.21.15.92:0/2021858256 wait complete. 2023-05-16T07:38:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.948+0000 7f6c70078700 1 Processor -- start 2023-05-16T07:38:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.948+0000 7f6c70078700 1 -- start start 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c70078700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c50003ef0 0x7f6c680c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c70078700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 0x7f6c680c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c70078700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c6814d4a0 0x7f6c680c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c70078700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680aef00 con 0x7f6c680adbe0 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c70078700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c4480 con 0x7f6c6814d4a0 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c70078700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680afee0 con 0x7f6c50003ef0 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 0x7f6c680c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 0x7f6c680c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52630/0 (socket says 172.21.15.92:52630) 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 -- 172.21.15.92:0/1338930590 learned_addr learned my addr 172.21.15.92:0/1338930590 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 -- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c50003ef0 msgr2=0x7f6c680c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c50003ef0 0x7f6c680c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 -- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c6814d4a0 msgr2=0x7f6c680c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c6814d4a0 0x7f6c680c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.949+0000 7f6c6d613700 1 -- 172.21.15.92:0/1338930590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68156720 con 0x7f6c680adbe0 2023-05-16T07:38:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.950+0000 7f6c6d613700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 0x7f6c680c3aa0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f6c5c001d20 tx=0x7f6c5c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.950+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c007000 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.950+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c013670 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.950+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680b0a30 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.950+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c013a10 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.950+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c680b0100 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.951+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c68066730 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.951+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6c5c01d070 con 0x7f6c680adbe0 2023-05-16T07:38:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.951+0000 7f6c65ffb700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6c5405ed90 0x7f6c54061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.952+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5937+0+0 (secure 0 0 0) 0x7f6c5c07a5b0 con 0x7f6c680adbe0 2023-05-16T07:38:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.952+0000 7f6c6de14700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6c5405ed90 0x7f6c54061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.952+0000 7f6c6de14700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6c5405ed90 0x7f6c54061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6c58007d30 tx=0x7f6c580013e0).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:16.956+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6c5c04bc00 con 0x7f6c680adbe0 2023-05-16T07:38:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.118+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6c6806c350 con 0x7f6c680adbe0 2023-05-16T07:38:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.118+0000 7f6c65ffb700 1 -- 172.21.15.92:0/1338930590 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v167) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6c5c01a070 con 0x7f6c680adbe0 2023-05-16T07:38:17.122 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:38:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6c5405ed90 msgr2=0x7f6c54061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6c5405ed90 0x7f6c54061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6c58007d30 tx=0x7f6c580013e0).stop 2023-05-16T07:38:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 msgr2=0x7f6c680c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 0x7f6c680c3aa0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f6c5c001d20 tx=0x7f6c5c015000).stop 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 shutdown_connections 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6c5405ed90 0x7f6c54061250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6c50003ef0 0x7f6c680c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6c6814d4a0 0x7f6c680c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 --2- 172.21.15.92:0/1338930590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c680adbe0 0x7f6c680c3aa0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 >> 172.21.15.92:0/1338930590 conn(0x7f6c68067920 msgr2=0x7f6c680ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 shutdown_connections 2023-05-16T07:38:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.120+0000 7f6c70078700 1 -- 172.21.15.92:0/1338930590 wait complete. 2023-05-16T07:38:17.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:38:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.244+0000 7fefbe476700 1 Processor -- start 2023-05-16T07:38:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.244+0000 7fefbe476700 1 -- start start 2023-05-16T07:38:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefbe476700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb814e6f0 0x7fefb8155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefbe476700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 0x7fefb80622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefbe476700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb814ead0 con 0x7fefb80627a0 2023-05-16T07:38:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefbe476700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb8154cb0 con 0x7fefb80c4d40 2023-05-16T07:38:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefbe476700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb8062c30 con 0x7fefb814e6f0 2023-05-16T07:38:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefaffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 0x7fefb80622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefb7fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb814e6f0 0x7fefb8155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefaffff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 0x7fefb80622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:56274/0 (socket says 172.21.15.92:56274) 2023-05-16T07:38:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefaffff700 1 -- 172.21.15.92:0/2952459425 learned_addr learned my addr 172.21.15.92:0/2952459425 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:17.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefb7fff700 1 -- 172.21.15.92:0/2952459425 >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 msgr2=0x7fefb80622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefb7fff700 1 --2- 172.21.15.92:0/2952459425 >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 0x7fefb80622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefb7fff700 1 -- 172.21.15.92:0/2952459425 >> v1:172.21.15.158:6789/0 conn(0x7fefb80627a0 legacy=0x7fefb8063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefb7fff700 1 -- 172.21.15.92:0/2952459425 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80c6020 con 0x7fefb814e6f0 2023-05-16T07:38:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.245+0000 7fefaffff700 1 --2- 172.21.15.92:0/2952459425 >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 0x7fefb80622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefb7fff700 1 --2- 172.21.15.92:0/2952459425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb814e6f0 0x7fefb8155140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fefa8009760 tx=0x7fefa8009060).ready entity=mon.2 client_cookie=c80a92173bd35d54 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefb77fe700 1 -- 172.21.15.92:0/2952459425 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800f510 con 0x7fefb814e6f0 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefb77fe700 1 -- 172.21.15.92:0/2952459425 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa800f670 con 0x7fefb814e6f0 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefb77fe700 1 -- 172.21.15.92:0/2952459425 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800f9f0 con 0x7fefb814e6f0 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 -- 172.21.15.92:0/2952459425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb814e6f0 msgr2=0x7fefb8155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 --2- 172.21.15.92:0/2952459425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb814e6f0 0x7fefb8155140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fefa8009760 tx=0x7fefa8009060).stop 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 -- 172.21.15.92:0/2952459425 shutdown_connections 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 --2- 172.21.15.92:0/2952459425 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb814e6f0 0x7fefb8155140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 --2- 172.21.15.92:0/2952459425 >> v2:172.21.15.158:3300/0 conn(0x7fefb80c4d40 0x7fefb80622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 -- 172.21.15.92:0/2952459425 >> 172.21.15.92:0/2952459425 conn(0x7fefb805e820 msgr2=0x7fefb80c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 -- 172.21.15.92:0/2952459425 shutdown_connections 2023-05-16T07:38:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.246+0000 7fefbe476700 1 -- 172.21.15.92:0/2952459425 wait complete. 2023-05-16T07:38:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.247+0000 7fefbe476700 1 Processor -- start 2023-05-16T07:38:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.247+0000 7fefbe476700 1 -- start start 2023-05-16T07:38:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbe476700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefb80627a0 0x7fefb80ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbe476700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb80c4d40 0x7fefb80aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbe476700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 0x7fefb814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbe476700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80ad910 con 0x7fefb814e6f0 2023-05-16T07:38:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbe476700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80ab3a0 con 0x7fefb80627a0 2023-05-16T07:38:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbe476700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80a5490 con 0x7fefb80c4d40 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbca13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 0x7fefb814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbca13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 0x7fefb814c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52646/0 (socket says 172.21.15.92:52646) 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefbca13700 1 -- 172.21.15.92:0/3661786453 learned_addr learned my addr 172.21.15.92:0/3661786453 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefb7fff700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefb80627a0 0x7fefb80ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.248+0000 7fefaffff700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb80c4d40 0x7fefb80aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbca13700 1 -- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb80c4d40 msgr2=0x7fefb80aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbca13700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb80c4d40 0x7fefb80aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbca13700 1 -- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefb80627a0 msgr2=0x7fefb80ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbca13700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefb80627a0 0x7fefb80ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbca13700 1 -- 172.21.15.92:0/3661786453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009370 con 0x7fefb814e6f0 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbca13700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 0x7fefb814c840 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fefb000c7f0 tx=0x7fefb0001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000e4a0 con 0x7fefb814e6f0 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb000e600 con 0x7fefb814e6f0 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa80092a0 con 0x7fefb814e6f0 2023-05-16T07:38:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000e980 con 0x7fefb814e6f0 2023-05-16T07:38:17.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.249+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb80ae3c0 con 0x7fefb814e6f0 2023-05-16T07:38:17.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.250+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb8065da0 con 0x7fefb814e6f0 2023-05-16T07:38:17.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.250+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fefb0010070 con 0x7fefb814e6f0 2023-05-16T07:38:17.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.251+0000 7fefb4ff9700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fef9805ed90 0x7fef98061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.251+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5937+0+0 (secure 0 0 0) 0x7fefb00791b0 con 0x7fefb814e6f0 2023-05-16T07:38:17.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.251+0000 7fefb7fff700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fef9805ed90 0x7fef98061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.251+0000 7fefb7fff700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fef9805ed90 0x7fef98061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fefa8009fd0 tx=0x7fefa8017000).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.255+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fefb00478f0 con 0x7fefb814e6f0 2023-05-16T07:38:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.414+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fefb80ae220 con 0x7fefb814e6f0 2023-05-16T07:38:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.415+0000 7fefb4ff9700 1 -- 172.21.15.92:0/3661786453 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v167) v1 ==== 92+0+10 (secure 0 0 0) 0x7fefb80ae220 con 0x7fefb814e6f0 2023-05-16T07:38:17.418 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:38:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fef9805ed90 msgr2=0x7fef98061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fef9805ed90 0x7fef98061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fefa8009fd0 tx=0x7fefa8017000).stop 2023-05-16T07:38:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 msgr2=0x7fefb814c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 0x7fefb814c840 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fefb000c7f0 tx=0x7fefb0001510).stop 2023-05-16T07:38:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 shutdown_connections 2023-05-16T07:38:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb814e6f0 0x7fefb814c840 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fefb80627a0 0x7fefb80ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fef9805ed90 0x7fef98061250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 --2- 172.21.15.92:0/3661786453 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fefb80c4d40 0x7fefb80aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 >> 172.21.15.92:0/3661786453 conn(0x7fefb805e820 msgr2=0x7fefb8063600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 shutdown_connections 2023-05-16T07:38:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.416+0000 7fefbe476700 1 -- 172.21.15.92:0/3661786453 wait complete. 2023-05-16T07:38:17.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-16T07:38:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.539+0000 7f55d9f2b700 1 Processor -- start 2023-05-16T07:38:17.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.539+0000 7f55d9f2b700 1 -- start start 2023-05-16T07:38:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.539+0000 7f55d9f2b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f55d4154e50 0x7f55d40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d9f2b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d4158970 con 0x7f55d40c7380 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d9f2b700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d4155300 con 0x7f55d40ad040 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d9f2b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d4157d60 con 0x7f55d4154e50 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d8f29700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f55d40c7380 0x7f55d40c7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35166/0 (socket says 172.21.15.92:35166) 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d8f29700 1 -- 172.21.15.92:0/1738047756 learned_addr learned my addr 172.21.15.92:0/1738047756 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d3fff700 1 --2- 172.21.15.92:0/1738047756 >> v2:172.21.15.158:3300/0 conn(0x7f55d4154e50 0x7f55d40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836536219 0 0) 0x7f55d4158970 con 0x7f55d40c7380 2023-05-16T07:38:17.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c00033f0 con 0x7f55d40c7380 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253651170 0 0) 0x7f55c00033f0 con 0x7f55d40c7380 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 >> v2:172.21.15.158:3300/0 conn(0x7f55d4154e50 msgr2=0x7f55d40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 --2- 172.21.15.92:0/1738047756 >> v2:172.21.15.158:3300/0 conn(0x7f55d4154e50 0x7f55d40acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 >> v1:172.21.15.158:6789/0 conn(0x7f55d40ad040 legacy=0x7f55d414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d4158040 con 0x7f55d40c7380 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f55bc001470 con 0x7f55d40c7380 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 >> v1:172.21.15.92:6789/0 conn(0x7f55d40c7380 legacy=0x7f55d40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c00038e0 0x7f55c0003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55c0004690 0x7f55c0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c0005000 0x7f55c000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d4157d60 con 0x7f55c00038e0 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c000be80 con 0x7f55c0004690 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c000c040 con 0x7f55c0005000 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c00038e0 0x7f55c0003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.540+0000 7f55d3fff700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55c0004690 0x7f55c0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d972a700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c0005000 0x7f55c000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d8f29700 1 -- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c0005000 msgr2=0x7f55c000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c0005000 0x7f55c000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d8f29700 1 -- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55c0004690 msgr2=0x7f55c0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55c0004690 0x7f55c0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d8f29700 1 -- 172.21.15.92:0/1738047756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00033f0 con 0x7f55c00038e0 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d972a700 1 -- 172.21.15.92:0/1738047756 reap_dead start 2023-05-16T07:38:17.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d3fff700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55c0004690 0x7f55c0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c00038e0 0x7f55c0003ce0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f55bc00c160 tx=0x7f55bc001e70).ready entity=mon.0 client_cookie=b84bb35582a96172 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc007a40 con 0x7f55c00038e0 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00ca90 con 0x7f55c00038e0 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d37fe700 1 -- 172.21.15.92:0/1738047756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00ce10 con 0x7f55c00038e0 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1738047756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c00038e0 msgr2=0x7f55c0003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c00038e0 0x7f55c0003ce0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f55bc00c160 tx=0x7f55bc001e70).stop 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1738047756 shutdown_connections 2023-05-16T07:38:17.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1738047756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c00038e0 0x7f55c0003ce0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1738047756 >> 172.21.15.92:0/1738047756 conn(0x7f55d4068330 msgr2=0x7f55d4068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1738047756 shutdown_connections 2023-05-16T07:38:17.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.541+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1738047756 wait complete. 2023-05-16T07:38:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.542+0000 7f55d9f2b700 1 Processor -- start 2023-05-16T07:38:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 -- start start 2023-05-16T07:38:17.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c00038e0 0x7f55d40ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 0x7f55d4142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55d4156210 0x7f55d4156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40adf70 con 0x7f55d414f520 2023-05-16T07:38:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40aff20 con 0x7f55d4156210 2023-05-16T07:38:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d9f2b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40c99e0 con 0x7f55c00038e0 2023-05-16T07:38:17.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.543+0000 7f55d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 0x7f55d4142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 0x7f55d4142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52664/0 (socket says 172.21.15.92:52664) 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 -- 172.21.15.92:0/1078756162 learned_addr learned my addr 172.21.15.92:0/1078756162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c00038e0 0x7f55d40ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 -- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c00038e0 msgr2=0x7f55d40ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c00038e0 0x7f55d40ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 -- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55d4156210 msgr2=0x7f55d4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d972a700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55d4156210 0x7f55d4156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55d4156210 0x7f55d4156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 -- 172.21.15.92:0/1078756162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d4156b80 con 0x7f55d414f520 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c00038e0 0x7f55d40ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d3fff700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 0x7f55d4142e80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f55c40101f0 tx=0x7f55c400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400e030 con 0x7f55d414f520 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c400f040 con 0x7f55d414f520 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d4143370 con 0x7f55d414f520 2023-05-16T07:38:17.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4015690 con 0x7f55d414f520 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.544+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d40ae460 con 0x7f55d414f520 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.545+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d4047150 con 0x7f55d414f520 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.545+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f55c4015ac0 con 0x7f55d414f520 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.546+0000 7f55d0ff9700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f55b405ed90 0x7f55b4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.546+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5937+0+0 (secure 0 0 0) 0x7f55c407a760 con 0x7f55d414f520 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.546+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f55b405ed90 0x7f55b4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.546+0000 7f55d8f29700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f55b405ed90 0x7f55b4061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f55bc00ca60 tx=0x7f55bc015000).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.550+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f55c404bd50 con 0x7f55d414f520 2023-05-16T07:38:17.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.714+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55d40c5410 con 0x7f55d414f520 2023-05-16T07:38:17.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.714+0000 7f55d0ff9700 1 -- 172.21.15.92:0/1078756162 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v167) v1 ==== 84+0+143 (secure 0 0 0) 0x7f55c404f370 con 0x7f55d414f520 2023-05-16T07:38:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:17.718 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:38:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "device_health_metrics" 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-05-16T07:38:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:17.720 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:38:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f55b405ed90 msgr2=0x7f55b4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f55b405ed90 0x7f55b4061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f55bc00ca60 tx=0x7f55bc015000).stop 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 msgr2=0x7f55d4142e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 0x7f55d4142e80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f55c40101f0 tx=0x7f55c400b040).stop 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 shutdown_connections 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f55b405ed90 0x7f55b4061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f55c00038e0 0x7f55d40ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f55d4156210 0x7f55d4156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 --2- 172.21.15.92:0/1078756162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d414f520 0x7f55d4142e80 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 >> 172.21.15.92:0/1078756162 conn(0x7f55d4068330 msgr2=0x7f55d4068710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 shutdown_connections 2023-05-16T07:38:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.716+0000 7f55d9f2b700 1 -- 172.21.15.92:0/1078756162 wait complete. 2023-05-16T07:38:17.731 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-16T07:38:17.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-05-16T07:38:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.843+0000 7fdc3f2cd700 1 Processor -- start 2023-05-16T07:38:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.843+0000 7fdc3f2cd700 1 -- start start 2023-05-16T07:38:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3f2cd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc38153f90 0x7fdc380a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3f2cd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38155780 con 0x7fdc380c5d00 2023-05-16T07:38:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3f2cd700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3806a140 con 0x7fdc38155380 2023-05-16T07:38:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3f2cd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc38069f60 con 0x7fdc38153f90 2023-05-16T07:38:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3e2cb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc38153f90 0x7fdc380a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3daca700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc380c5d00 0x7fdc380c0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35188/0 (socket says 172.21.15.92:35188) 2023-05-16T07:38:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.844+0000 7fdc3daca700 1 -- 172.21.15.92:0/1229962755 learned_addr learned my addr 172.21.15.92:0/1229962755 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3e2cb700 1 -- 172.21.15.92:0/1229962755 >> v1:172.21.15.158:6789/0 conn(0x7fdc38155380 legacy=0x7fdc380c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3e2cb700 1 -- 172.21.15.92:0/1229962755 >> v1:172.21.15.92:6789/0 conn(0x7fdc380c5d00 legacy=0x7fdc380c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3d2c9700 1 -- 172.21.15.92:0/1229962755 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251123445 0 0) 0x7fdc38155780 con 0x7fdc380c5d00 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3e2cb700 1 -- 172.21.15.92:0/1229962755 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3806a420 con 0x7fdc38153f90 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3e2cb700 1 --2- 172.21.15.92:0/1229962755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380a9d20 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fdc380aa7e0 tx=0x7fdc28009780).ready entity=mon.0 client_cookie=61e11b8d7c2d0c0c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3d2c9700 1 -- 172.21.15.92:0/1229962755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800e070 con 0x7fdc38153f90 2023-05-16T07:38:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3d2c9700 1 -- 172.21.15.92:0/1229962755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2800bc30 con 0x7fdc38153f90 2023-05-16T07:38:17.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3d2c9700 1 -- 172.21.15.92:0/1229962755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28012410 con 0x7fdc38153f90 2023-05-16T07:38:17.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1229962755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 msgr2=0x7fdc380a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1229962755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380a9d20 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fdc380aa7e0 tx=0x7fdc28009780).stop 2023-05-16T07:38:17.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1229962755 shutdown_connections 2023-05-16T07:38:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.845+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1229962755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380a9d20 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.846+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1229962755 >> 172.21.15.92:0/1229962755 conn(0x7fdc380cb600 msgr2=0x7fdc380c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.846+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1229962755 shutdown_connections 2023-05-16T07:38:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.846+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1229962755 wait complete. 2023-05-16T07:38:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 Processor -- start 2023-05-16T07:38:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 -- start start 2023-05-16T07:38:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc380c5d00 0x7fdc3814f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380c0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc38155380 0x7fdc380c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380c1300 con 0x7fdc38153f90 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380aa3b0 con 0x7fdc38155380 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3f2cd700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380c46e0 con 0x7fdc380c5d00 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.847+0000 7fdc3daca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380c0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52678/0 (socket says 172.21.15.92:52678) 2023-05-16T07:38:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 -- 172.21.15.92:0/1101336455 learned_addr learned my addr 172.21.15.92:0/1101336455 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3e2cb700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc380c5d00 0x7fdc3814f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3eacc700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc38155380 0x7fdc380c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 -- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc380c5d00 msgr2=0x7fdc3814f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc380c5d00 0x7fdc3814f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 -- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc38155380 msgr2=0x7fdc380c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc38155380 0x7fdc380c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 -- 172.21.15.92:0/1101336455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc28009a20 con 0x7fdc38153f90 2023-05-16T07:38:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3eacc700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc38155380 0x7fdc380c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3daca700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380c0920 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7fdc2c00c4c0 tx=0x7fdc2c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0072e0 con 0x7fdc38153f90 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c007440 con 0x7fdc38153f90 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3814fa60 con 0x7fdc38153f90 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0057a0 con 0x7fdc38153f90 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.848+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc380c4090 con 0x7fdc38153f90 2023-05-16T07:38:17.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.849+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc38083120 con 0x7fdc38153f90 2023-05-16T07:38:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.849+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fdc2c010070 con 0x7fdc38153f90 2023-05-16T07:38:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.850+0000 7fdc367fc700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdc2405ed90 0x7fdc24061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.850+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5937+0+0 (secure 0 0 0) 0x7fdc2c079100 con 0x7fdc38153f90 2023-05-16T07:38:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.850+0000 7fdc3e2cb700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdc2405ed90 0x7fdc24061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.850+0000 7fdc3e2cb700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdc2405ed90 0x7fdc24061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdc3815af80 tx=0x7fdc28009780).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:17.854+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdc2c04a750 con 0x7fdc38153f90 2023-05-16T07:38:18.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.016+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fdc38047150 con 0x7fdc38153f90 2023-05-16T07:38:18.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.144+0000 7fdc367fc700 1 -- 172.21.15.92:0/1101336455 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v168) v1 ==== 152+0+0 (secure 0 0 0) 0x7fdc2c04dd70 con 0x7fdc38153f90 2023-05-16T07:38:18.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.147+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdc2405ed90 msgr2=0x7fdc24061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:18.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.147+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdc2405ed90 0x7fdc24061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdc3815af80 tx=0x7fdc28009780).stop 2023-05-16T07:38:18.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.147+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 msgr2=0x7fdc380c0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:18.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.147+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380c0920 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7fdc2c00c4c0 tx=0x7fdc2c007c10).stop 2023-05-16T07:38:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 shutdown_connections 2023-05-16T07:38:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdc2405ed90 0x7fdc24061250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdc380c5d00 0x7fdc3814f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdc38155380 0x7fdc380c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 --2- 172.21.15.92:0/1101336455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc38153f90 0x7fdc380c0920 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 >> 172.21.15.92:0/1101336455 conn(0x7fdc380cb600 msgr2=0x7fdc38157080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.150+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 shutdown_connections 2023-05-16T07:38:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.151+0000 7fdc3f2cd700 1 -- 172.21.15.92:0/1101336455 wait complete. 2023-05-16T07:38:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-05-16T07:38:18.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-16T07:38:18.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.281+0000 7ff7068e7700 1 Processor -- start 2023-05-16T07:38:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.281+0000 7ff7068e7700 1 -- start start 2023-05-16T07:38:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7068e7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff7000c1160 0x7ff7000c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7068e7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7000c61f0 0x7ff700154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7068e7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70006a140 con 0x7ff700153f90 2023-05-16T07:38:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7068e7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff700069f60 con 0x7ff7000c61f0 2023-05-16T07:38:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7068e7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70006a9e0 con 0x7ff7000c1160 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7060e6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7000c61f0 0x7ff700154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7060e6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7000c61f0 0x7ff700154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52686/0 (socket says 172.21.15.92:52686) 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.282+0000 7ff7060e6700 1 -- 172.21.15.92:0/2451923874 learned_addr learned my addr 172.21.15.92:0/2451923874 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7050e4700 1 --2- 172.21.15.92:0/2451923874 >> v2:172.21.15.158:3300/0 conn(0x7ff7000c1160 0x7ff7000c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7060e6700 1 -- 172.21.15.92:0/2451923874 >> v2:172.21.15.158:3300/0 conn(0x7ff7000c1160 msgr2=0x7ff7000c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7060e6700 1 --2- 172.21.15.92:0/2451923874 >> v2:172.21.15.158:3300/0 conn(0x7ff7000c1160 0x7ff7000c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7060e6700 1 -- 172.21.15.92:0/2451923874 >> v1:172.21.15.158:6789/0 conn(0x7ff700153f90 legacy=0x7ff7000a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7060e6700 1 -- 172.21.15.92:0/2451923874 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70006a420 con 0x7ff7000c61f0 2023-05-16T07:38:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7060e6700 1 --2- 172.21.15.92:0/2451923874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700154370 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7ff7000c8240 tx=0x7ff6f80090f0).ready entity=mon.0 client_cookie=c35232f785dabae2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7048e3700 1 -- 172.21.15.92:0/2451923874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800b5a0 con 0x7ff7000c61f0 2023-05-16T07:38:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7048e3700 1 -- 172.21.15.92:0/2451923874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f800b700 con 0x7ff7000c61f0 2023-05-16T07:38:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7048e3700 1 -- 172.21.15.92:0/2451923874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800ba80 con 0x7ff7000c61f0 2023-05-16T07:38:18.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7068e7700 1 -- 172.21.15.92:0/2451923874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 msgr2=0x7ff700154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:18.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7068e7700 1 --2- 172.21.15.92:0/2451923874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700154370 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7ff7000c8240 tx=0x7ff6f80090f0).stop 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7068e7700 1 -- 172.21.15.92:0/2451923874 shutdown_connections 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7068e7700 1 --2- 172.21.15.92:0/2451923874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700154370 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7068e7700 1 --2- 172.21.15.92:0/2451923874 >> v2:172.21.15.158:3300/0 conn(0x7ff7000c1160 0x7ff7000c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.283+0000 7ff7068e7700 1 -- 172.21.15.92:0/2451923874 >> 172.21.15.92:0/2451923874 conn(0x7ff7000cb600 msgr2=0x7ff7000c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.284+0000 7ff7068e7700 1 -- 172.21.15.92:0/2451923874 shutdown_connections 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.284+0000 7ff7068e7700 1 -- 172.21.15.92:0/2451923874 wait complete. 2023-05-16T07:38:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 Processor -- start 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 -- start start 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7000c1160 0x7ff700145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff700153f90 0x7ff7000ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff700145630 con 0x7ff7000c61f0 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000af890 con 0x7ff7000c1160 2023-05-16T07:38:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7068e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000af060 con 0x7ff700153f90 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7050e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7050e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700163a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52692/0 (socket says 172.21.15.92:52692) 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7050e4700 1 -- 172.21.15.92:0/644789289 learned_addr learned my addr 172.21.15.92:0/644789289 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.285+0000 7ff7058e5700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7000c1160 0x7ff700145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7060e6700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff700153f90 0x7ff7000ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7050e4700 1 -- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff700153f90 msgr2=0x7ff7000ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7050e4700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff700153f90 0x7ff7000ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7050e4700 1 -- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7000c1160 msgr2=0x7ff700145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7050e4700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7000c1160 0x7ff700145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7050e4700 1 -- 172.21.15.92:0/644789289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8009370 con 0x7ff7000c61f0 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7050e4700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700163a10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7ff6f4006980 tx=0x7ff6f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011590 con 0x7ff7000c61f0 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f40116f0 con 0x7ff7000c61f0 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7000c1d00 con 0x7ff7000c61f0 2023-05-16T07:38:18.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011a70 con 0x7ff7000c61f0 2023-05-16T07:38:18.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.286+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000c2280 con 0x7ff7000c61f0 2023-05-16T07:38:18.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.287+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff700083120 con 0x7ff7000c61f0 2023-05-16T07:38:18.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.287+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7ff6f401c460 con 0x7ff7000c61f0 2023-05-16T07:38:18.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.288+0000 7ff6fdffb700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff6ec05ed60 0x7ff6ec061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:18.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.288+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6307+0+0 (secure 0 0 0) 0x7ff6f40799e0 con 0x7ff7000c61f0 2023-05-16T07:38:18.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.288+0000 7ff7058e5700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff6ec05ed60 0x7ff6ec061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:18.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.288+0000 7ff7058e5700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff6ec05ed60 0x7ff6ec061220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff6f0006fd0 tx=0x7ff6f0008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:18.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.292+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff6f404ae60 con 0x7ff7000c61f0 2023-05-16T07:38:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:18.455+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff700047150 con 0x7ff7000c61f0 2023-05-16T07:38:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.146+0000 7ff6fdffb700 1 -- 172.21.15.92:0/644789289 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v169) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff6f4011e60 con 0x7ff7000c61f0 2023-05-16T07:38:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.150+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff6ec05ed60 msgr2=0x7ff6ec061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.150+0000 7ff7068e7700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff6ec05ed60 0x7ff6ec061220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff6f0006fd0 tx=0x7ff6f0008040).stop 2023-05-16T07:38:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.150+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 msgr2=0x7ff700163a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.150+0000 7ff7068e7700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700163a10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7ff6f4006980 tx=0x7ff6f4008040).stop 2023-05-16T07:38:19.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.151+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 shutdown_connections 2023-05-16T07:38:19.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.151+0000 7ff7068e7700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff6ec05ed60 0x7ff6ec061220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.151+0000 7ff7068e7700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff700153f90 0x7ff7000ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.151+0000 7ff7068e7700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7000c1160 0x7ff700145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.151+0000 7ff7068e7700 1 --2- 172.21.15.92:0/644789289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7000c61f0 0x7ff700163a10 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.151+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 >> 172.21.15.92:0/644789289 conn(0x7ff7000cb600 msgr2=0x7ff7000acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.153+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 shutdown_connections 2023-05-16T07:38:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.153+0000 7ff7068e7700 1 -- 172.21.15.92:0/644789289 wait complete. 2023-05-16T07:38:19.158 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-05-16T07:38:19.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-16T07:38:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.296+0000 7f006aa0e700 1 Processor -- start 2023-05-16T07:38:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.296+0000 7f006aa0e700 1 -- start start 2023-05-16T07:38:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f006aa0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0064154090 0x7f00640aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f006aa0e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00640c94d0 con 0x7f0064068160 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f006aa0e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00640cb7e0 con 0x7f0064154470 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f006aa0e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00640c88c0 con 0x7f0064154090 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0063fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0064154090 0x7f00640aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0068fab700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0064068160 0x7f00640c9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35218/0 (socket says 172.21.15.92:35218) 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0068fab700 1 -- 172.21.15.92:0/2278117377 learned_addr learned my addr 172.21.15.92:0/2278117377 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0063fff700 1 -- 172.21.15.92:0/2278117377 >> v1:172.21.15.158:6789/0 conn(0x7f0064154470 legacy=0x7f00640a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0063fff700 1 -- 172.21.15.92:0/2278117377 >> v1:172.21.15.92:6789/0 conn(0x7f0064068160 legacy=0x7f00640c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0062ffd700 1 -- 172.21.15.92:0/2278117377 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478858431 0 0) 0x7f00640c94d0 con 0x7f0064068160 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0063fff700 1 -- 172.21.15.92:0/2278117377 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640c8ba0 con 0x7f0064154090 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.297+0000 7f0063fff700 1 --2- 172.21.15.92:0/2278117377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064154090 0x7f00640aaca0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f0054009760 tx=0x7f0054009060).ready entity=mon.0 client_cookie=d69d686615efcc1d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f0062ffd700 1 -- 172.21.15.92:0/2278117377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400b520 con 0x7f0064154090 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f0062ffd700 1 -- 172.21.15.92:0/2278117377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005400b680 con 0x7f0064154090 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f0062ffd700 1 -- 172.21.15.92:0/2278117377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400ba00 con 0x7f0064154090 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 -- 172.21.15.92:0/2278117377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064154090 msgr2=0x7f00640aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2278117377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064154090 0x7f00640aaca0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f0054009760 tx=0x7f0054009060).stop 2023-05-16T07:38:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 -- 172.21.15.92:0/2278117377 shutdown_connections 2023-05-16T07:38:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2278117377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064154090 0x7f00640aaca0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 -- 172.21.15.92:0/2278117377 >> 172.21.15.92:0/2278117377 conn(0x7f0064067920 msgr2=0x7f00640cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 -- 172.21.15.92:0/2278117377 shutdown_connections 2023-05-16T07:38:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.298+0000 7f006aa0e700 1 -- 172.21.15.92:0/2278117377 wait complete. 2023-05-16T07:38:19.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.299+0000 7f006aa0e700 1 Processor -- start 2023-05-16T07:38:19.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.299+0000 7f006aa0e700 1 -- start start 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f006aa0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 0x7f00641649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f006aa0e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0064154090 0x7f00641637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f006aa0e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0064154470 0x7f0064156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f006aa0e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c32d0 con 0x7f0064068160 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f006aa0e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00641502a0 con 0x7f0064154470 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f006aa0e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0064164e90 con 0x7f0064154090 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f0063fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 0x7f00641649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f0063fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 0x7f00641649a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52704/0 (socket says 172.21.15.92:52704) 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f0063fff700 1 -- 172.21.15.92:0/2389198067 learned_addr learned my addr 172.21.15.92:0/2389198067 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f0068fab700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0064154470 0x7f0064156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f0063fff700 1 -- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0064154090 msgr2=0x7f00641637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.300+0000 7f00637fe700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0064154090 0x7f00641637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f0063fff700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0064154090 0x7f00641637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f0063fff700 1 -- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0064154470 msgr2=0x7f0064156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f0063fff700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0064154470 0x7f0064156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f0063fff700 1 -- 172.21.15.92:0/2389198067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540092a0 con 0x7f0064068160 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f0068fab700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0064154470 0x7f0064156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f00637fe700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0064154090 0x7f00641637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f0063fff700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 0x7f00641649a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f0054009eb0 tx=0x7f0054017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400ba00 con 0x7f0064068160 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005401b410 con 0x7f0064068160 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00640c2ca0 con 0x7f0064068160 2023-05-16T07:38:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005401b7d0 con 0x7f0064068160 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.301+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0064154cd0 con 0x7f0064068160 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.302+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0064066730 con 0x7f0064068160 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.302+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f005400f070 con 0x7f0064068160 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.303+0000 7f004bfff700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f005005ed90 0x7f0050061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.303+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f005407bb30 con 0x7f0064068160 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.303+0000 7f00637fe700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f005005ed90 0x7f0050061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.303+0000 7f00637fe700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f005005ed90 0x7f0050061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0064167490 tx=0x7f0058008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.307+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f005404ce90 con 0x7f0064068160 2023-05-16T07:38:19.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.512+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f00640c8e80 con 0x7f0064068160 2023-05-16T07:38:19.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.520+0000 7f004bfff700 1 -- 172.21.15.92:0/2389198067 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7f00540504b0 con 0x7f0064068160 2023-05-16T07:38:19.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.529+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f005005ed90 msgr2=0x7f0050061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.529+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f005005ed90 0x7f0050061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0064167490 tx=0x7f0058008040).stop 2023-05-16T07:38:19.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.530+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 msgr2=0x7f00641649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.530+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 0x7f00641649a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f0054009eb0 tx=0x7f0054017000).stop 2023-05-16T07:38:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.536+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 shutdown_connections 2023-05-16T07:38:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.536+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0064154470 0x7f0064156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.536+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0064068160 0x7f00641649a0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.536+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f005005ed90 0x7f0050061250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.536+0000 7f006aa0e700 1 --2- 172.21.15.92:0/2389198067 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f0064154090 0x7f00641637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.536+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 >> 172.21.15.92:0/2389198067 conn(0x7f0064067920 msgr2=0x7f00640c4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:19.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.538+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 shutdown_connections 2023-05-16T07:38:19.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.539+0000 7f006aa0e700 1 -- 172.21.15.92:0/2389198067 wait complete. 2023-05-16T07:38:19.545 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 21 and data pool 22 2023-05-16T07:38:19.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:19.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.662+0000 7fb1532ab700 1 Processor -- start 2023-05-16T07:38:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.662+0000 7fb1532ab700 1 -- start start 2023-05-16T07:38:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb1532ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb14c154170 0x7fb14c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb1532ab700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c069db0 0x7fb14c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb1532ab700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0c0660 con 0x7fb14c154550 2023-05-16T07:38:19.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb1532ab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14c0c1b00 con 0x7fb14c154170 2023-05-16T07:38:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb1532ab700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0bfd30 con 0x7fb14c069db0 2023-05-16T07:38:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb151047700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb14c154170 0x7fb14c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb151047700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb14c154170 0x7fb14c0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52712/0 (socket says 172.21.15.92:52712) 2023-05-16T07:38:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.663+0000 7fb151047700 1 -- 172.21.15.92:0/1838681940 learned_addr learned my addr 172.21.15.92:0/1838681940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb151047700 1 -- 172.21.15.92:0/1838681940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c069db0 msgr2=0x7fb14c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb151047700 1 --2- 172.21.15.92:0/1838681940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c069db0 0x7fb14c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb151047700 1 -- 172.21.15.92:0/1838681940 >> v1:172.21.15.158:6789/0 conn(0x7fb14c154550 legacy=0x7fb14c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb151047700 1 -- 172.21.15.92:0/1838681940 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c0c0010 con 0x7fb14c154170 2023-05-16T07:38:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb151047700 1 --2- 172.21.15.92:0/1838681940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c154170 0x7fb14c0c5c70 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fb13c006fd0 tx=0x7fb13c0090f0).ready entity=mon.0 client_cookie=5ba43882866481d2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb14bfff700 1 -- 172.21.15.92:0/1838681940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00b600 con 0x7fb14c154170 2023-05-16T07:38:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb14bfff700 1 -- 172.21.15.92:0/1838681940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c00b760 con 0x7fb14c154170 2023-05-16T07:38:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb14bfff700 1 -- 172.21.15.92:0/1838681940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00bb00 con 0x7fb14c154170 2023-05-16T07:38:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb1532ab700 1 -- 172.21.15.92:0/1838681940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c154170 msgr2=0x7fb14c0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb1532ab700 1 --2- 172.21.15.92:0/1838681940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c154170 0x7fb14c0c5c70 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fb13c006fd0 tx=0x7fb13c0090f0).stop 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb1532ab700 1 -- 172.21.15.92:0/1838681940 shutdown_connections 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb1532ab700 1 --2- 172.21.15.92:0/1838681940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c154170 0x7fb14c0c5c70 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb1532ab700 1 --2- 172.21.15.92:0/1838681940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c069db0 0x7fb14c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.664+0000 7fb1532ab700 1 -- 172.21.15.92:0/1838681940 >> 172.21.15.92:0/1838681940 conn(0x7fb14c0cb700 msgr2=0x7fb14c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.665+0000 7fb1532ab700 1 -- 172.21.15.92:0/1838681940 shutdown_connections 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.665+0000 7fb1532ab700 1 -- 172.21.15.92:0/1838681940 wait complete. 2023-05-16T07:38:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 Processor -- start 2023-05-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 -- start start 2023-05-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 0x7fb14c14ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c154170 0x7fb14c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb14c154550 0x7fb14c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c15ab20 con 0x7fb14c069db0 2023-05-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0ae140 con 0x7fb14c154550 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.666+0000 7fb1532ab700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0ae420 con 0x7fb14c154170 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 0x7fb14c14ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 0x7fb14c14ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52718/0 (socket says 172.21.15.92:52718) 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 -- 172.21.15.92:0/904246931 learned_addr learned my addr 172.21.15.92:0/904246931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb150846700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c154170 0x7fb14c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 -- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c154170 msgr2=0x7fb14c15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c154170 0x7fb14c15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 -- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb14c154550 msgr2=0x7fb14c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151848700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb14c154550 0x7fb14c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb14c154550 0x7fb14c15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 -- 172.21.15.92:0/904246931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009370 con 0x7fb14c069db0 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb151047700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 0x7fb14c14ca30 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fb13c009f50 tx=0x7fb13c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c002f70 con 0x7fb14c069db0 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c0069b0 con 0x7fb14c069db0 2023-05-16T07:38:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c0ae700 con 0x7fb14c069db0 2023-05-16T07:38:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00bb00 con 0x7fb14c069db0 2023-05-16T07:38:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.667+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c164540 con 0x7fb14c069db0 2023-05-16T07:38:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.668+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c0666c0 con 0x7fb14c069db0 2023-05-16T07:38:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.668+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb13c00f070 con 0x7fb14c069db0 2023-05-16T07:38:19.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.669+0000 7fb1497fa700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb13805ed90 0x7fb138061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.669+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fb13c07bad0 con 0x7fb14c069db0 2023-05-16T07:38:19.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.669+0000 7fb150846700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb13805ed90 0x7fb138061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.669+0000 7fb150846700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb13805ed90 0x7fb138061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb14c167a40 tx=0x7fb140008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.673+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb13c04cdd0 con 0x7fb14c069db0 2023-05-16T07:38:19.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.835+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb14c06c350 con 0x7fb14c069db0 2023-05-16T07:38:19.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.836+0000 7fb1497fa700 1 -- 172.21.15.92:0/904246931 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fb13c0503f0 con 0x7fb14c069db0 2023-05-16T07:38:19.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:19.839 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":170,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:19.513565+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:38:18.041970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-05-16T07:38:18.481177+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:19.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb13805ed90 msgr2=0x7fb138061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb13805ed90 0x7fb138061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb14c167a40 tx=0x7fb140008040).stop 2023-05-16T07:38:19.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 msgr2=0x7fb14c14ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 0x7fb14c14ca30 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fb13c009f50 tx=0x7fb13c017000).stop 2023-05-16T07:38:19.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 shutdown_connections 2023-05-16T07:38:19.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb14c154550 0x7fb14c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb14c069db0 0x7fb14c14ca30 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb13805ed90 0x7fb138061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 --2- 172.21.15.92:0/904246931 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb14c154170 0x7fb14c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 >> 172.21.15.92:0/904246931 conn(0x7fb14c0cb700 msgr2=0x7fb14c14ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:19.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.837+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 shutdown_connections 2023-05-16T07:38:19.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.838+0000 7fb1532ab700 1 -- 172.21.15.92:0/904246931 wait complete. 2023-05-16T07:38:19.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:19.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.961+0000 7fa22ea37700 1 Processor -- start 2023-05-16T07:38:19.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.961+0000 7fa22ea37700 1 -- start start 2023-05-16T07:38:19.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22ea37700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa228063e30 0x7fa2280c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22ea37700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa22813b930 0x7fa2280abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22ea37700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2280bdc30 con 0x7fa22813b550 2023-05-16T07:38:19.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22ea37700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2280c5400 con 0x7fa22813b930 2023-05-16T07:38:19.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22ea37700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280c4e40 con 0x7fa228063e30 2023-05-16T07:38:19.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22cfd4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa22813b930 0x7fa2280abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22cfd4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa22813b930 0x7fa2280abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52728/0 (socket says 172.21.15.92:52728) 2023-05-16T07:38:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa22cfd4700 1 -- 172.21.15.92:0/1018701530 learned_addr learned my addr 172.21.15.92:0/1018701530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.962+0000 7fa2277fe700 1 --2- 172.21.15.92:0/1018701530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa228063e30 0x7fa2280c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22cfd4700 1 -- 172.21.15.92:0/1018701530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa228063e30 msgr2=0x7fa2280c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22cfd4700 1 --2- 172.21.15.92:0/1018701530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa228063e30 0x7fa2280c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22cfd4700 1 -- 172.21.15.92:0/1018701530 >> v1:172.21.15.158:6789/0 conn(0x7fa22813b550 legacy=0x7fa228063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22cfd4700 1 -- 172.21.15.92:0/1018701530 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280c5120 con 0x7fa22813b930 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22cfd4700 1 --2- 172.21.15.92:0/1018701530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa22813b930 0x7fa2280abb20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7fa220009f20 tx=0x7fa220009840).ready entity=mon.0 client_cookie=e4078a1d45de7f6b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa226ffd700 1 -- 172.21.15.92:0/1018701530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000e070 con 0x7fa22813b930 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa226ffd700 1 -- 172.21.15.92:0/1018701530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22000ba80 con 0x7fa22813b930 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa226ffd700 1 -- 172.21.15.92:0/1018701530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000be00 con 0x7fa22813b930 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 -- 172.21.15.92:0/1018701530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa22813b930 msgr2=0x7fa2280abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 --2- 172.21.15.92:0/1018701530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa22813b930 0x7fa2280abb20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7fa220009f20 tx=0x7fa220009840).stop 2023-05-16T07:38:19.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 -- 172.21.15.92:0/1018701530 shutdown_connections 2023-05-16T07:38:19.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 --2- 172.21.15.92:0/1018701530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa22813b930 0x7fa2280abb20 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 --2- 172.21.15.92:0/1018701530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa228063e30 0x7fa2280c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 -- 172.21.15.92:0/1018701530 >> 172.21.15.92:0/1018701530 conn(0x7fa2280c8aa0 msgr2=0x7fa2280c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:19.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 -- 172.21.15.92:0/1018701530 shutdown_connections 2023-05-16T07:38:19.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.963+0000 7fa22ea37700 1 -- 172.21.15.92:0/1018701530 wait complete. 2023-05-16T07:38:19.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.965+0000 7fa22ea37700 1 Processor -- start 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.965+0000 7fa22ea37700 1 -- start start 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.965+0000 7fa22ea37700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 0x7fa2280c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.965+0000 7fa22ea37700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa22813b550 0x7fa2280abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22ea37700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa22813b930 0x7fa2280ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22ea37700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228136380 con 0x7fa228063e30 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22ea37700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280ac6b0 con 0x7fa22813b930 2023-05-16T07:38:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22ea37700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280c5bc0 con 0x7fa22813b550 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 0x7fa2280c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 0x7fa2280c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52732/0 (socket says 172.21.15.92:52732) 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 -- 172.21.15.92:0/612060045 learned_addr learned my addr 172.21.15.92:0/612060045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22cfd4700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa22813b930 0x7fa2280ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa2277fe700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa22813b550 0x7fa2280abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 -- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa22813b550 msgr2=0x7fa2280abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa22813b550 0x7fa2280abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 -- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa22813b930 msgr2=0x7fa2280ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa22813b930 0x7fa2280ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 -- 172.21.15.92:0/612060045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220009ae0 con 0x7fa228063e30 2023-05-16T07:38:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22cfd4700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa22813b930 0x7fa2280ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa227fff700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 0x7fa2280c6050 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7fa21800c4c0 tx=0x7fa218007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218007020 con 0x7fa228063e30 2023-05-16T07:38:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa218001e90 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2280c55e0 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180073b0 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.966+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2280c5850 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.967+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2280c5ea0 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.967+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fa218010070 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.968+0000 7fa20ffff700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa21405ed60 0x7fa214061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.968+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa218079740 con 0x7fa228063e30 2023-05-16T07:38:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.968+0000 7fa2277fe700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa21405ed60 0x7fa214061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:19.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.968+0000 7fa2277fe700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa21405ed60 0x7fa214061220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa21c006fd0 tx=0x7fa21c008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:19.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:19.972+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa21804a930 con 0x7fa228063e30 2023-05-16T07:38:20.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.135+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa22806c350 con 0x7fa228063e30 2023-05-16T07:38:20.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.136+0000 7fa20ffff700 1 -- 172.21.15.92:0/612060045 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fa21804df50 con 0x7fa228063e30 2023-05-16T07:38:20.140 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:20.140 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":170,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:19.513565+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:38:18.041970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-05-16T07:38:18.481177+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:20.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa21405ed60 msgr2=0x7fa214061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa21405ed60 0x7fa214061220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa21c006fd0 tx=0x7fa21c008040).stop 2023-05-16T07:38:20.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 msgr2=0x7fa2280c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 0x7fa2280c6050 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7fa21800c4c0 tx=0x7fa218007e00).stop 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 shutdown_connections 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa22813b930 0x7fa2280ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa228063e30 0x7fa2280c6050 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa21405ed60 0x7fa214061220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 --2- 172.21.15.92:0/612060045 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa22813b550 0x7fa2280abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.138+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 >> 172.21.15.92:0/612060045 conn(0x7fa2280c8aa0 msgr2=0x7fa228154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:20.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.139+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 shutdown_connections 2023-05-16T07:38:20.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.139+0000 7fa22ea37700 1 -- 172.21.15.92:0/612060045 wait complete. 2023-05-16T07:38:20.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-16T07:38:20.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.263+0000 7f74f54d6700 1 Processor -- start 2023-05-16T07:38:20.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.264+0000 7f74f54d6700 1 -- start start 2023-05-16T07:38:20.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.264+0000 7f74f54d6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74f0155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.264+0000 7f74f54d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f014ead0 con 0x7f74f00629c0 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.264+0000 7f74f54d6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f0154cb0 con 0x7f74f00c4d40 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.264+0000 7f74f54d6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c6020 con 0x7f74f014e6f0 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74ef7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f74f00629c0 0x7f74f0063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35256/0 (socket says 172.21.15.92:35256) 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74ef7fe700 1 -- 172.21.15.92:0/4097794949 learned_addr learned my addr 172.21.15.92:0/4097794949 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74eeffd700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74f0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640228382 0 0) 0x7f74f014ead0 con 0x7f74f00629c0 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74dc0033f0 con 0x7f74f00629c0 2023-05-16T07:38:20.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240843898 0 0) 0x7f74f0154cb0 con 0x7f74f00c4d40 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f014ead0 con 0x7f74f00c4d40 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612635579 0 0) 0x7f74dc0033f0 con 0x7f74f00629c0 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 msgr2=0x7f74f0155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74f0155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 >> v1:172.21.15.158:6789/0 conn(0x7f74f00c4d40 legacy=0x7f74f00622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0154660 con 0x7f74f00629c0 2023-05-16T07:38:20.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f74e8001470 con 0x7f74f00629c0 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 >> v1:172.21.15.92:6789/0 conn(0x7f74f00629c0 legacy=0x7f74f0063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74dc003aa0 0x7f74dc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74dc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc004b80 0x7f74dc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c6020 con 0x7f74dc004b80 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc00b910 con 0x7f74dc003aa0 2023-05-16T07:38:20.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc00bad0 con 0x7f74f014e6f0 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74ef7fe700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc004b80 0x7f74dc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74eeffd700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74dc003aa0 0x7f74dc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.265+0000 7f74ee7fc700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74dc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 -- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 msgr2=0x7f74dc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74dc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 -- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74dc003aa0 msgr2=0x7f74dc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74dc003aa0 0x7f74dc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 -- 172.21.15.92:0/4097794949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0033f0 con 0x7f74dc004b80 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ee7fc700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f014e6f0 0x7f74dc004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc004b80 0x7f74dc005000 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f74f0064800 tx=0x7f74e8001e10).ready entity=mon.0 client_cookie=a78534e69695f727 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800c9a0 con 0x7f74dc004b80 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800cb00 con 0x7f74dc004b80 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74edffb700 1 -- 172.21.15.92:0/4097794949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e80070d0 con 0x7f74dc004b80 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74f54d6700 1 -- 172.21.15.92:0/4097794949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc004b80 msgr2=0x7f74dc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74f54d6700 1 --2- 172.21.15.92:0/4097794949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc004b80 0x7f74dc005000 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f74f0064800 tx=0x7f74e8001e10).stop 2023-05-16T07:38:20.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74ef7fe700 1 -- 172.21.15.92:0/4097794949 reap_dead start 2023-05-16T07:38:20.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74f54d6700 1 -- 172.21.15.92:0/4097794949 shutdown_connections 2023-05-16T07:38:20.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74f54d6700 1 -- 172.21.15.92:0/4097794949 >> 172.21.15.92:0/4097794949 conn(0x7f74f005e820 msgr2=0x7f74f00c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:20.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74f54d6700 1 -- 172.21.15.92:0/4097794949 shutdown_connections 2023-05-16T07:38:20.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.266+0000 7f74f54d6700 1 -- 172.21.15.92:0/4097794949 wait complete. 2023-05-16T07:38:20.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 Processor -- start 2023-05-16T07:38:20.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 -- start start 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 0x7f74f00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f00c0a30 0x7f74f0165610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74f0165b00 0x7f74f00ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00acd80 con 0x7f74f0165230 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c0eb0 con 0x7f74f0165b00 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74f54d6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c1130 con 0x7f74f00c0a30 2023-05-16T07:38:20.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74eeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 0x7f74f00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74eeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 0x7f74f00a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52750/0 (socket says 172.21.15.92:52750) 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74eeffd700 1 -- 172.21.15.92:0/3663399797 learned_addr learned my addr 172.21.15.92:0/3663399797 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74ee7fc700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f00c0a30 0x7f74f0165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.268+0000 7f74ef7fe700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74f0165b00 0x7f74f00ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74eeffd700 1 -- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f00c0a30 msgr2=0x7f74f0165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74eeffd700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f00c0a30 0x7f74f0165610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74eeffd700 1 -- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74f0165b00 msgr2=0x7f74f00ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74eeffd700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74f0165b00 0x7f74f00ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74eeffd700 1 -- 172.21.15.92:0/3663399797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00c1410 con 0x7f74f0165230 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74eeffd700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 0x7f74f00a48f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f74e0011f30 tx=0x7f74e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000e040 con 0x7f74f0165230 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000f040 con 0x7f74f0165230 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000fe70 con 0x7f74f0165230 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f00c16f0 con 0x7f74f0165230 2023-05-16T07:38:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.269+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f014cdf0 con 0x7f74f0165230 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.270+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f006c350 con 0x7f74f0165230 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.270+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f74e0015660 con 0x7f74f0165230 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.271+0000 7f74e6ffd700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f74d005ed90 0x7f74d0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.271+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f74e007adc0 con 0x7f74f0165230 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.271+0000 7f74ee7fc700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f74d005ed90 0x7f74d0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.271+0000 7f74ee7fc700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f74d005ed90 0x7f74d0061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f74d8007fd0 tx=0x7f74d800c040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.274+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f74e004c0c0 con 0x7f74f0165230 2023-05-16T07:38:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.480+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f74f00ad060 con 0x7f74f0165230 2023-05-16T07:38:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.517+0000 7f74e6ffd700 1 -- 172.21.15.92:0/3663399797 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7f74e004f6e0 con 0x7f74f0165230 2023-05-16T07:38:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f74d005ed90 msgr2=0x7f74d0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f74d005ed90 0x7f74d0061250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f74d8007fd0 tx=0x7f74d800c040).stop 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 msgr2=0x7f74f00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 0x7f74f00a48f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f74e0011f30 tx=0x7f74e000b040).stop 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 shutdown_connections 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f74f0165b00 0x7f74f00ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f0165230 0x7f74f00a48f0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f74d005ed90 0x7f74d0061250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 --2- 172.21.15.92:0/3663399797 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f74f00c0a30 0x7f74f0165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 >> 172.21.15.92:0/3663399797 conn(0x7f74f005e820 msgr2=0x7f74f00c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 shutdown_connections 2023-05-16T07:38:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.519+0000 7f74f54d6700 1 -- 172.21.15.92:0/3663399797 wait complete. 2023-05-16T07:38:20.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-16T07:38:20.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.647+0000 7f8397c43700 1 Processor -- start 2023-05-16T07:38:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.647+0000 7f8397c43700 1 -- start start 2023-05-16T07:38:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f8397c43700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 0x7f83900aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f8397c43700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8390068160 0x7f83900685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f8397c43700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83900c94d0 con 0x7f8390154090 2023-05-16T07:38:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f8397c43700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900cb7e0 con 0x7f8390068160 2023-05-16T07:38:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f8397c43700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8390068ac0 con 0x7f8390154470 2023-05-16T07:38:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83959df700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8390154090 0x7f83900a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35274/0 (socket says 172.21.15.92:35274) 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83959df700 1 -- 172.21.15.92:0/3851688143 learned_addr learned my addr 172.21.15.92:0/3851688143 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109133123 0 0) 0x7f83900c94d0 con 0x7f8390154090 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f837c0033f0 con 0x7f8390154090 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83951de700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 0x7f83900aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83961e0700 1 --2- 172.21.15.92:0/3851688143 >> v2:172.21.15.158:3300/0 conn(0x7f8390068160 0x7f83900685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3084534018 0 0) 0x7f837c0033f0 con 0x7f8390154090 2023-05-16T07:38:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 msgr2=0x7f83900aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 0x7f83900aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 >> v2:172.21.15.158:3300/0 conn(0x7f8390068160 msgr2=0x7f83900685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 --2- 172.21.15.92:0/3851688143 >> v2:172.21.15.158:3300/0 conn(0x7f8390068160 0x7f83900685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900c88c0 con 0x7f8390154090 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f838c001470 con 0x7f8390154090 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83961e0700 1 --2- 172.21.15.92:0/3851688143 >> v2:172.21.15.158:3300/0 conn(0x7f8390068160 0x7f83900685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 >> v1:172.21.15.92:6789/0 conn(0x7f8390154090 legacy=0x7f83900a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f837c0038e0 0x7f837c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 0x7f837c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f837c004b80 0x7f837c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900cb7e0 con 0x7f837c0038e0 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390068ac0 con 0x7f837c004b80 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.648+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c014140 con 0x7f8390154470 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f837c0038e0 0x7f837c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83951de700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 0x7f837c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83961e0700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f837c004b80 0x7f837c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 -- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 msgr2=0x7f837c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8390154470 0x7f837c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 -- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f837c004b80 msgr2=0x7f837c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f837c004b80 0x7f837c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 -- 172.21.15.92:0/3851688143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c0033f0 con 0x7f837c0038e0 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83959df700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f837c0038e0 0x7f837c003d00 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f838c00c210 tx=0x7f838c001e90).ready entity=mon.0 client_cookie=cec552133e0cd713 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c007a60 con 0x7f837c0038e0 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c00cae0 con 0x7f837c0038e0 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.649+0000 7f83949dd700 1 -- 172.21.15.92:0/3851688143 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c00ce60 con 0x7f837c0038e0 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f8397c43700 1 -- 172.21.15.92:0/3851688143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f837c0038e0 msgr2=0x7f837c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f8397c43700 1 --2- 172.21.15.92:0/3851688143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f837c0038e0 0x7f837c003d00 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f838c00c210 tx=0x7f838c001e90).stop 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f83961e0700 1 -- 172.21.15.92:0/3851688143 reap_dead start 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f8397c43700 1 -- 172.21.15.92:0/3851688143 shutdown_connections 2023-05-16T07:38:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f8397c43700 1 -- 172.21.15.92:0/3851688143 >> 172.21.15.92:0/3851688143 conn(0x7f8390067920 msgr2=0x7f83900cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:20.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f8397c43700 1 -- 172.21.15.92:0/3851688143 shutdown_connections 2023-05-16T07:38:20.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.650+0000 7f8397c43700 1 -- 172.21.15.92:0/3851688143 wait complete. 2023-05-16T07:38:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 Processor -- start 2023-05-16T07:38:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 -- start start 2023-05-16T07:38:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f837c0038e0 0x7f839014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8390154850 0x7f8390154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 0x7f83900ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900c2190 con 0x7f83900acc10 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900ad560 con 0x7f8390154850 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.651+0000 7f8397c43700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83901552f0 con 0x7f837c0038e0 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 0x7f83900ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 0x7f83900ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52774/0 (socket says 172.21.15.92:52774) 2023-05-16T07:38:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 -- 172.21.15.92:0/2551745821 learned_addr learned my addr 172.21.15.92:0/2551745821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83959df700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f837c0038e0 0x7f839014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83951de700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8390154850 0x7f8390154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 -- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f837c0038e0 msgr2=0x7f839014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f837c0038e0 0x7f839014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 -- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8390154850 msgr2=0x7f8390154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8390154850 0x7f8390154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 -- 172.21.15.92:0/2551745821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83901555d0 con 0x7f83900acc10 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f83961e0700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 0x7f83900ad070 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f839005efb0 tx=0x7f83880097d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388010b10 con 0x7f83900acc10 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388010c70 con 0x7f83900acc10 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83901558b0 con 0x7f83900acc10 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388015420 con 0x7f83900acc10 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.652+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83900affb0 con 0x7f83900acc10 2023-05-16T07:38:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.653+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f8388015790 con 0x7f83900acc10 2023-05-16T07:38:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.653+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8390066730 con 0x7f83900acc10 2023-05-16T07:38:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.654+0000 7f8385ffb700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f837405ed60 0x7f8374061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.654+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f838807a510 con 0x7f83900acc10 2023-05-16T07:38:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.654+0000 7f83959df700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f837405ed60 0x7f8374061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.654+0000 7f83959df700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f837405ed60 0x7f8374061220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f838c001ed0 tx=0x7f838c015000).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.658+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f838804b810 con 0x7f83900acc10 2023-05-16T07:38:20.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:20.862+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f839006c350 con 0x7f83900acc10 2023-05-16T07:38:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.524+0000 7f8385ffb700 1 -- 172.21.15.92:0/2551745821 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7f838804ee30 con 0x7f83900acc10 2023-05-16T07:38:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f837405ed60 msgr2=0x7f8374061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f837405ed60 0x7f8374061220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f838c001ed0 tx=0x7f838c015000).stop 2023-05-16T07:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 msgr2=0x7f83900ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 0x7f83900ad070 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f839005efb0 tx=0x7f83880097d0).stop 2023-05-16T07:38:21.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 shutdown_connections 2023-05-16T07:38:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f837405ed60 0x7f8374061220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f837c0038e0 0x7f839014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83900acc10 0x7f83900ad070 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 --2- 172.21.15.92:0/2551745821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8390154850 0x7f8390154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 >> 172.21.15.92:0/2551745821 conn(0x7f8390067920 msgr2=0x7f83900c12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 shutdown_connections 2023-05-16T07:38:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.527+0000 7f8397c43700 1 -- 172.21.15.92:0/2551745821 wait complete. 2023-05-16T07:38:21.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.658+0000 7fb31f9ed700 1 Processor -- start 2023-05-16T07:38:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.658+0000 7fb31f9ed700 1 -- start start 2023-05-16T07:38:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31f9ed700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb318154170 0x7fb3180c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31f9ed700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154550 0x7fb3180c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31f9ed700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180c0660 con 0x7fb3180696a0 2023-05-16T07:38:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31f9ed700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180c1b00 con 0x7fb318154170 2023-05-16T07:38:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31f9ed700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180bfd30 con 0x7fb318154550 2023-05-16T07:38:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31d789700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb318154170 0x7fb3180c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31df8a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3180696a0 0x7fb318157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35296/0 (socket says 172.21.15.92:35296) 2023-05-16T07:38:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31df8a700 1 -- 172.21.15.92:0/3105703851 learned_addr learned my addr 172.21.15.92:0/3105703851 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31cf88700 1 --2- 172.21.15.92:0/3105703851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154550 0x7fb3180c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31d789700 1 -- 172.21.15.92:0/3105703851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154550 msgr2=0x7fb3180c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31d789700 1 --2- 172.21.15.92:0/3105703851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154550 0x7fb3180c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31d789700 1 -- 172.21.15.92:0/3105703851 >> v1:172.21.15.92:6789/0 conn(0x7fb3180696a0 legacy=0x7fb318157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb30ffff700 1 -- 172.21.15.92:0/3105703851 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946578205 0 0) 0x7fb3180c0660 con 0x7fb3180696a0 2023-05-16T07:38:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31d789700 1 -- 172.21.15.92:0/3105703851 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180c0010 con 0x7fb318154170 2023-05-16T07:38:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.659+0000 7fb31d789700 1 --2- 172.21.15.92:0/3105703851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb318154170 0x7fb3180c5c70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7fb310009e60 tx=0x7fb310009780).ready entity=mon.0 client_cookie=46ab79a6acd2acb7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb30ffff700 1 -- 172.21.15.92:0/3105703851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000e070 con 0x7fb318154170 2023-05-16T07:38:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb30ffff700 1 -- 172.21.15.92:0/3105703851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31000bc30 con 0x7fb318154170 2023-05-16T07:38:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb30ffff700 1 -- 172.21.15.92:0/3105703851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310012410 con 0x7fb318154170 2023-05-16T07:38:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 -- 172.21.15.92:0/3105703851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb318154170 msgr2=0x7fb3180c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/3105703851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb318154170 0x7fb3180c5c70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7fb310009e60 tx=0x7fb310009780).stop 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 -- 172.21.15.92:0/3105703851 shutdown_connections 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/3105703851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb318154170 0x7fb3180c5c70 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/3105703851 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154550 0x7fb3180c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 -- 172.21.15.92:0/3105703851 >> 172.21.15.92:0/3105703851 conn(0x7fb3180cb750 msgr2=0x7fb3180c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 -- 172.21.15.92:0/3105703851 shutdown_connections 2023-05-16T07:38:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.660+0000 7fb31f9ed700 1 -- 172.21.15.92:0/3105703851 wait complete. 2023-05-16T07:38:21.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.661+0000 7fb31f9ed700 1 Processor -- start 2023-05-16T07:38:21.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.661+0000 7fb31f9ed700 1 -- start start 2023-05-16T07:38:21.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31f9ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 0x7fb3180c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:21.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31f9ed700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154170 0x7fb3181467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:21.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31f9ed700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb318154550 0x7fb3180abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:21.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31f9ed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c50d0 con 0x7fb3180696a0 2023-05-16T07:38:21.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31f9ed700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318156df0 con 0x7fb318154550 2023-05-16T07:38:21.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31f9ed700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181570d0 con 0x7fb318154170 2023-05-16T07:38:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31d789700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 0x7fb3180c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31d789700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 0x7fb3180c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52782/0 (socket says 172.21.15.92:52782) 2023-05-16T07:38:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31d789700 1 -- 172.21.15.92:0/1000727004 learned_addr learned my addr 172.21.15.92:0/1000727004 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31df8a700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb318154550 0x7fb3180abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31cf88700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154170 0x7fb3181467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31d789700 1 -- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154170 msgr2=0x7fb3181467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31d789700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154170 0x7fb3181467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.662+0000 7fb31d789700 1 -- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb318154550 msgr2=0x7fb3180abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb31d789700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb318154550 0x7fb3180abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb31d789700 1 -- 172.21.15.92:0/1000727004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310009a20 con 0x7fb3180696a0 2023-05-16T07:38:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb31df8a700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb318154550 0x7fb3180abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb31d789700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 0x7fb3180c4be0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7fb31000b6c0 tx=0x7fb310009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31001d630 con 0x7fb3180696a0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31001d790 con 0x7fb3180696a0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31001db50 con 0x7fb3180696a0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3181573b0 con 0x7fb3180696a0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.663+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb318157970 con 0x7fb3180696a0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.664+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3180666c0 con 0x7fb3180696a0 2023-05-16T07:38:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.664+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb3100083a0 con 0x7fb3180696a0 2023-05-16T07:38:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.664+0000 7fb30d7fa700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb2fc05ed90 0x7fb2fc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.665+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fb31007c1d0 con 0x7fb3180696a0 2023-05-16T07:38:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.665+0000 7fb31cf88700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb2fc05ed90 0x7fb2fc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.665+0000 7fb31cf88700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb2fc05ed90 0x7fb2fc061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb308006fd0 tx=0x7fb308008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:21.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.669+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb31004d360 con 0x7fb3180696a0 2023-05-16T07:38:21.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.876+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31806c350 con 0x7fb3180696a0 2023-05-16T07:38:21.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.876+0000 7fb30d7fa700 1 -- 172.21.15.92:0/1000727004 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fb310050980 con 0x7fb3180696a0 2023-05-16T07:38:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:21.881 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:38:19.513917+0000","modified":"2023-05-16T07:38:21.520884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/862724662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":862724662},{"type":"v1","addr":"172.21.15.158:6837","nonce":862724662}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-05-16T07:38:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb2fc05ed90 msgr2=0x7fb2fc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb2fc05ed90 0x7fb2fc061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb308006fd0 tx=0x7fb308008040).stop 2023-05-16T07:38:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 msgr2=0x7fb3180c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 0x7fb3180c4be0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7fb31000b6c0 tx=0x7fb310009780).stop 2023-05-16T07:38:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 shutdown_connections 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb318154550 0x7fb3180abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3180696a0 0x7fb3180c4be0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb2fc05ed90 0x7fb2fc061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 --2- 172.21.15.92:0/1000727004 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb318154170 0x7fb3181467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 >> 172.21.15.92:0/1000727004 conn(0x7fb3180cb750 msgr2=0x7fb3180c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 shutdown_connections 2023-05-16T07:38:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:21.878+0000 7fb31f9ed700 1 -- 172.21.15.92:0/1000727004 wait complete. 2023-05-16T07:38:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-05-16T07:38:21.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.001+0000 7f83ef05f700 1 Processor -- start 2023-05-16T07:38:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.001+0000 7f83ef05f700 1 -- start start 2023-05-16T07:38:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83ef05f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e81386a0 0x7f83e80c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83ef05f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83e80c30a0 0x7f83e80622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83ef05f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e80c37f0 con 0x7f83e80627a0 2023-05-16T07:38:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83ef05f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e8138a80 con 0x7f83e80c30a0 2023-05-16T07:38:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83ef05f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8062c50 con 0x7f83e81386a0 2023-05-16T07:38:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83e7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83e80c30a0 0x7f83e80622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83e7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83e80c30a0 0x7f83e80622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52790/0 (socket says 172.21.15.92:52790) 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.002+0000 7f83e7fff700 1 -- 172.21.15.92:0/1602934473 learned_addr learned my addr 172.21.15.92:0/1602934473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83ecdfb700 1 --2- 172.21.15.92:0/1602934473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e81386a0 0x7f83e80c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e7fff700 1 -- 172.21.15.92:0/1602934473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e81386a0 msgr2=0x7f83e80c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e7fff700 1 --2- 172.21.15.92:0/1602934473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e81386a0 0x7f83e80c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e7fff700 1 -- 172.21.15.92:0/1602934473 >> v1:172.21.15.158:6789/0 conn(0x7f83e80627a0 legacy=0x7f83e8063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e7fff700 1 -- 172.21.15.92:0/1602934473 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80bef90 con 0x7f83e80c30a0 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e7fff700 1 --2- 172.21.15.92:0/1602934473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e80622b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f83d0009f20 tx=0x7f83d0009840).ready entity=mon.0 client_cookie=d4d9390a2efe475b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e77fe700 1 -- 172.21.15.92:0/1602934473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000e070 con 0x7f83e80c30a0 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e77fe700 1 -- 172.21.15.92:0/1602934473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d000ba80 con 0x7f83e80c30a0 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83e77fe700 1 -- 172.21.15.92:0/1602934473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000be00 con 0x7f83e80c30a0 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83ef05f700 1 -- 172.21.15.92:0/1602934473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 msgr2=0x7f83e80622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.003+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1602934473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e80622b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f83d0009f20 tx=0x7f83d0009840).stop 2023-05-16T07:38:22.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.004+0000 7f83ef05f700 1 -- 172.21.15.92:0/1602934473 shutdown_connections 2023-05-16T07:38:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.004+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1602934473 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e81386a0 0x7f83e80c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.004+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1602934473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e80622b0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.004+0000 7f83ef05f700 1 -- 172.21.15.92:0/1602934473 >> 172.21.15.92:0/1602934473 conn(0x7f83e80c8aa0 msgr2=0x7f83e80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.004+0000 7f83ef05f700 1 -- 172.21.15.92:0/1602934473 shutdown_connections 2023-05-16T07:38:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.004+0000 7f83ef05f700 1 -- 172.21.15.92:0/1602934473 wait complete. 2023-05-16T07:38:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.005+0000 7f83ef05f700 1 Processor -- start 2023-05-16T07:38:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.005+0000 7f83ef05f700 1 -- start start 2023-05-16T07:38:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e80627a0 0x7f83e8150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e8067f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83e81386a0 0x7f83e8068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80a5b40 con 0x7f83e80c30a0 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8150600 con 0x7f83e81386a0 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8068a20 con 0x7f83e80627a0 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e8067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e8067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52798/0 (socket says 172.21.15.92:52798) 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 -- 172.21.15.92:0/1373216009 learned_addr learned my addr 172.21.15.92:0/1373216009 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ecdfb700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e80627a0 0x7f83e8150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ed5fc700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83e81386a0 0x7f83e8068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 -- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e80627a0 msgr2=0x7f83e8150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e80627a0 0x7f83e8150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 -- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83e81386a0 msgr2=0x7f83e8068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83e81386a0 0x7f83e8068450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 -- 172.21.15.92:0/1373216009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0009ae0 con 0x7f83e80c30a0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e7fff700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e8067f60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f83d00070b0 tx=0x7f83d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d00092e0 con 0x7f83e80c30a0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d0013d10 con 0x7f83e80c30a0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d001e680 con 0x7f83e80c30a0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e8068d00 con 0x7f83e80c30a0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.006+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e80692c0 con 0x7f83e80c30a0 2023-05-16T07:38:22.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.007+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e80697e0 con 0x7f83e80c30a0 2023-05-16T07:38:22.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.007+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f83d0025030 con 0x7f83e80c30a0 2023-05-16T07:38:22.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.008+0000 7f83e4ff9700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f83c805ed90 0x7f83c8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.008+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f83d007b310 con 0x7f83e80c30a0 2023-05-16T07:38:22.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.008+0000 7f83ecdfb700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f83c805ed90 0x7f83c8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.008+0000 7f83ecdfb700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f83c805ed90 0x7f83c8061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f83d800bb20 tx=0x7f83d8008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.012+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f83e80697e0 con 0x7f83e80c30a0 2023-05-16T07:38:22.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.176+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83e80697e0 con 0x7f83e80c30a0 2023-05-16T07:38:22.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.177+0000 7f83e4ff9700 1 -- 172.21.15.92:0/1373216009 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18009 (secure 0 0 0) 0x7f83e80697e0 con 0x7f83e80c30a0 2023-05-16T07:38:22.181 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:22.181 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":171,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:20.515576+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:38:18.041970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-05-16T07:38:18.481177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f83c805ed90 msgr2=0x7f83c8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f83c805ed90 0x7f83c8061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f83d800bb20 tx=0x7f83d8008040).stop 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 msgr2=0x7f83e8067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e8067f60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f83d00070b0 tx=0x7f83d0009840).stop 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 shutdown_connections 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f83c805ed90 0x7f83c8061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83e80627a0 0x7f83e8150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83e81386a0 0x7f83e8068450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 --2- 172.21.15.92:0/1373216009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e80c30a0 0x7f83e8067f60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 >> 172.21.15.92:0/1373216009 conn(0x7f83e80c8aa0 msgr2=0x7f83e81373b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 shutdown_connections 2023-05-16T07:38:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.179+0000 7f83ef05f700 1 -- 172.21.15.92:0/1373216009 wait complete. 2023-05-16T07:38:22.195 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-16T07:38:22.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:38:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.302+0000 7fd064a41700 1 Processor -- start 2023-05-16T07:38:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.302+0000 7fd064a41700 1 -- start start 2023-05-16T07:38:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd064a41700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd060154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd064a41700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd060065000 0x7fd060065490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd064a41700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd060067560 con 0x7fd060068c70 2023-05-16T07:38:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd064a41700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd060065980 con 0x7fd060065000 2023-05-16T07:38:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd064a41700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600bc9b0 con 0x7fd060153eb0 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd05ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd060065000 0x7fd060065490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd05dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd060068c70 0x7fd060066e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35320/0 (socket says 172.21.15.92:35320) 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.303+0000 7fd05dd9b700 1 -- 172.21.15.92:0/4257728821 learned_addr learned my addr 172.21.15.92:0/4257728821 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05e59c700 1 --2- 172.21.15.92:0/4257728821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd060154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05ed9d700 1 -- 172.21.15.92:0/4257728821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 msgr2=0x7fd060154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05ed9d700 1 --2- 172.21.15.92:0/4257728821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd060154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05ed9d700 1 -- 172.21.15.92:0/4257728821 >> v1:172.21.15.92:6789/0 conn(0x7fd060068c70 legacy=0x7fd060066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05ed9d700 1 -- 172.21.15.92:0/4257728821 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600bcc90 con 0x7fd060065000 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05d59a700 1 -- 172.21.15.92:0/4257728821 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327242902 0 0) 0x7fd048001110 con 0x7fd060068c70 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05ed9d700 1 --2- 172.21.15.92:0/4257728821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060065000 0x7fd060065490 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7fd058009760 tx=0x7fd058009060).ready entity=mon.0 client_cookie=93081a7925e5f456 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05d59a700 1 -- 172.21.15.92:0/4257728821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800b580 con 0x7fd060065000 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05d59a700 1 -- 172.21.15.92:0/4257728821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05800b6e0 con 0x7fd060065000 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd05d59a700 1 -- 172.21.15.92:0/4257728821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800ba60 con 0x7fd060065000 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd064a41700 1 -- 172.21.15.92:0/4257728821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060065000 msgr2=0x7fd060065490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.304+0000 7fd064a41700 1 --2- 172.21.15.92:0/4257728821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060065000 0x7fd060065490 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7fd058009760 tx=0x7fd058009060).stop 2023-05-16T07:38:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.305+0000 7fd064a41700 1 -- 172.21.15.92:0/4257728821 shutdown_connections 2023-05-16T07:38:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.305+0000 7fd064a41700 1 --2- 172.21.15.92:0/4257728821 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd060154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.305+0000 7fd064a41700 1 --2- 172.21.15.92:0/4257728821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060065000 0x7fd060065490 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.305+0000 7fd064a41700 1 -- 172.21.15.92:0/4257728821 >> 172.21.15.92:0/4257728821 conn(0x7fd0600c4430 msgr2=0x7fd0600c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.305+0000 7fd064a41700 1 -- 172.21.15.92:0/4257728821 shutdown_connections 2023-05-16T07:38:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.305+0000 7fd064a41700 1 -- 172.21.15.92:0/4257728821 wait complete. 2023-05-16T07:38:22.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 Processor -- start 2023-05-16T07:38:22.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 -- start start 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd060065000 0x7fd0600ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd0600aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 0x7fd060153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ab410 con 0x7fd060150710 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ab6f0 con 0x7fd060065000 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd064a41700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ab9d0 con 0x7fd060153eb0 2023-05-16T07:38:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.306+0000 7fd05ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 0x7fd060153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 0x7fd060153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52810/0 (socket says 172.21.15.92:52810) 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 -- 172.21.15.92:0/3362833515 learned_addr learned my addr 172.21.15.92:0/3362833515 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05e59c700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd060065000 0x7fd0600ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05dd9b700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd0600aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 -- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 msgr2=0x7fd0600aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd0600aa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 -- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd060065000 msgr2=0x7fd0600ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd060065000 0x7fd0600ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 -- 172.21.15.92:0/3362833515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580092a0 con 0x7fd060150710 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05e59c700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd060065000 0x7fd0600ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05dd9b700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd0600aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd05ed9d700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 0x7fd060153210 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7fd060064390 tx=0x7fd058017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800ba60 con 0x7fd060150710 2023-05-16T07:38:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05801b410 con 0x7fd060150710 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600abcb0 con 0x7fd060150710 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05801b790 con 0x7fd060150710 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.307+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd060062b60 con 0x7fd060150710 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.308+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06006c350 con 0x7fd060150710 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.308+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd05802c070 con 0x7fd060150710 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.309+0000 7fd0567fc700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd04005ed90 0x7fd040061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.309+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fd05807bc00 con 0x7fd060150710 2023-05-16T07:38:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.309+0000 7fd05e59c700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd04005ed90 0x7fd040061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.310+0000 7fd05e59c700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd04005ed90 0x7fd040061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd05000e420 tx=0x7fd050007670).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.313+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd05804cf00 con 0x7fd060150710 2023-05-16T07:38:22.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.472+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0600a6d30 con 0x7fd04005ed90 2023-05-16T07:38:22.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.484+0000 7fd0567fc700 1 -- 172.21.15.92:0/3362833515 <== mgr.7208 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361496 (secure 0 0 0) 0x7fd0600a6d30 con 0x7fd04005ed90 2023-05-16T07:38:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:22.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd04005ed90 msgr2=0x7fd040061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd04005ed90 0x7fd040061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd05000e420 tx=0x7fd050007670).stop 2023-05-16T07:38:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 msgr2=0x7fd060153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 0x7fd060153210 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7fd060064390 tx=0x7fd058017000).stop 2023-05-16T07:38:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 shutdown_connections 2023-05-16T07:38:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd060150710 0x7fd060153210 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd060065000 0x7fd0600ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd04005ed90 0x7fd040061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 --2- 172.21.15.92:0/3362833515 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd060153eb0 0x7fd0600aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 >> 172.21.15.92:0/3362833515 conn(0x7fd0600c4430 msgr2=0x7fd060066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 shutdown_connections 2023-05-16T07:38:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.487+0000 7fd064a41700 1 -- 172.21.15.92:0/3362833515 wait complete. 2023-05-16T07:38:22.497 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:38:22.502 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:38:20.942207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":193684,"kb_used_data":16532,"kb_used_omap":0,"kb_used_meta":177088,"kb_avail":754781036,"statfs":{"total":773094113280,"available":772895780864,"internally_reserved":0,"allocated":16928768,"data_stored":9047064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":181338112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28002,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-77,"num_write_kb":-59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":-1,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.587081"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529432+0000","last_change":"2023-05-16T07:38:19.529432+0000","last_active":"2023-05-16T07:38:19.529432+0000","last_peered":"2023-05-16T07:38:19.529432+0000","last_clean":"2023-05-16T07:38:19.529432+0000","last_became_active":"2023-05-16T07:38:19.529104+0000","last_became_peered":"2023-05-16T07:38:19.529104+0000","last_unstale":"2023-05-16T07:38:19.529432+0000","last_undegraded":"2023-05-16T07:38:19.529432+0000","last_fullsized":"2023-05-16T07:38:19.529432+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3f","version":"170'3","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538943+0000","last_change":"2023-05-16T07:38:19.169222+0000","last_active":"2023-05-16T07:38:19.538943+0000","last_peered":"2023-05-16T07:38:19.538943+0000","last_clean":"2023-05-16T07:38:19.538943+0000","last_became_active":"2023-05-16T07:38:19.168992+0000","last_became_peered":"2023-05-16T07:38:19.168992+0000","last_unstale":"2023-05-16T07:38:19.538943+0000","last_undegraded":"2023-05-16T07:38:19.538943+0000","last_fullsized":"2023-05-16T07:38:19.538943+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539968+0000","last_change":"2023-05-16T07:38:19.539968+0000","last_active":"2023-05-16T07:38:19.539968+0000","last_peered":"2023-05-16T07:38:19.539968+0000","last_clean":"2023-05-16T07:38:19.539968+0000","last_became_active":"2023-05-16T07:38:19.539715+0000","last_became_peered":"2023-05-16T07:38:19.539715+0000","last_unstale":"2023-05-16T07:38:19.539968+0000","last_undegraded":"2023-05-16T07:38:19.539968+0000","last_fullsized":"2023-05-16T07:38:19.539968+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168219+0000","last_change":"2023-05-16T07:38:19.164572+0000","last_active":"2023-05-16T07:38:19.168219+0000","last_peered":"2023-05-16T07:38:19.168219+0000","last_clean":"2023-05-16T07:38:19.168219+0000","last_became_active":"2023-05-16T07:38:19.164316+0000","last_became_peered":"2023-05-16T07:38:19.164316+0000","last_unstale":"2023-05-16T07:38:19.168219+0000","last_undegraded":"2023-05-16T07:38:19.168219+0000","last_fullsized":"2023-05-16T07:38:19.168219+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533168+0000","last_change":"2023-05-16T07:38:19.533168+0000","last_active":"2023-05-16T07:38:19.533168+0000","last_peered":"2023-05-16T07:38:19.533168+0000","last_clean":"2023-05-16T07:38:19.533168+0000","last_became_active":"2023-05-16T07:38:19.530640+0000","last_became_peered":"2023-05-16T07:38:19.530640+0000","last_unstale":"2023-05-16T07:38:19.533168+0000","last_undegraded":"2023-05-16T07:38:19.533168+0000","last_fullsized":"2023-05-16T07:38:19.533168+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174939+0000","last_change":"2023-05-16T07:38:19.170667+0000","last_active":"2023-05-16T07:38:19.174939+0000","last_peered":"2023-05-16T07:38:19.174939+0000","last_clean":"2023-05-16T07:38:19.174939+0000","last_became_active":"2023-05-16T07:38:19.170247+0000","last_became_peered":"2023-05-16T07:38:19.170247+0000","last_unstale":"2023-05-16T07:38:19.174939+0000","last_undegraded":"2023-05-16T07:38:19.174939+0000","last_fullsized":"2023-05-16T07:38:19.174939+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.545057+0000","last_change":"2023-05-16T07:38:19.545057+0000","last_active":"2023-05-16T07:38:19.545057+0000","last_peered":"2023-05-16T07:38:19.545057+0000","last_clean":"2023-05-16T07:38:19.545057+0000","last_became_active":"2023-05-16T07:38:19.544804+0000","last_became_peered":"2023-05-16T07:38:19.544804+0000","last_unstale":"2023-05-16T07:38:19.545057+0000","last_undegraded":"2023-05-16T07:38:19.545057+0000","last_fullsized":"2023-05-16T07:38:19.545057+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3c","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539613+0000","last_change":"2023-05-16T07:38:19.163223+0000","last_active":"2023-05-16T07:38:19.539613+0000","last_peered":"2023-05-16T07:38:19.539613+0000","last_clean":"2023-05-16T07:38:19.539613+0000","last_became_active":"2023-05-16T07:38:19.162988+0000","last_became_peered":"2023-05-16T07:38:19.162988+0000","last_unstale":"2023-05-16T07:38:19.539613+0000","last_undegraded":"2023-05-16T07:38:19.539613+0000","last_fullsized":"2023-05-16T07:38:19.539613+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537238+0000","last_change":"2023-05-16T07:38:19.537238+0000","last_active":"2023-05-16T07:38:19.537238+0000","last_peered":"2023-05-16T07:38:19.537238+0000","last_clean":"2023-05-16T07:38:19.537238+0000","last_became_active":"2023-05-16T07:38:19.536977+0000","last_became_peered":"2023-05-16T07:38:19.536977+0000","last_unstale":"2023-05-16T07:38:19.537238+0000","last_undegraded":"2023-05-16T07:38:19.537238+0000","last_fullsized":"2023-05-16T07:38:19.537238+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.527727+0000","last_change":"2023-05-16T07:38:19.172261+0000","last_active":"2023-05-16T07:38:19.527727+0000","last_peered":"2023-05-16T07:38:19.527727+0000","last_clean":"2023-05-16T07:38:19.527727+0000","last_became_active":"2023-05-16T07:38:19.171980+0000","last_became_peered":"2023-05-16T07:38:19.171980+0000","last_unstale":"2023-05-16T07:38:19.527727+0000","last_undegraded":"2023-05-16T07:38:19.527727+0000","last_fullsized":"2023-05-16T07:38:19.527727+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539205+0000","last_change":"2023-05-16T07:38:19.539205+0000","last_active":"2023-05-16T07:38:19.539205+0000","last_peered":"2023-05-16T07:38:19.539205+0000","last_clean":"2023-05-16T07:38:19.539205+0000","last_became_active":"2023-05-16T07:38:19.538925+0000","last_became_peered":"2023-05-16T07:38:19.538925+0000","last_unstale":"2023-05-16T07:38:19.539205+0000","last_undegraded":"2023-05-16T07:38:19.539205+0000","last_fullsized":"2023-05-16T07:38:19.539205+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.519780+0000","last_change":"2023-05-16T07:38:19.166200+0000","last_active":"2023-05-16T07:38:19.519780+0000","last_peered":"2023-05-16T07:38:19.519780+0000","last_clean":"2023-05-16T07:38:19.519780+0000","last_became_active":"2023-05-16T07:38:19.163888+0000","last_became_peered":"2023-05-16T07:38:19.163888+0000","last_unstale":"2023-05-16T07:38:19.519780+0000","last_undegraded":"2023-05-16T07:38:19.519780+0000","last_fullsized":"2023-05-16T07:38:19.519780+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541792+0000","last_change":"2023-05-16T07:38:19.541792+0000","last_active":"2023-05-16T07:38:19.541792+0000","last_peered":"2023-05-16T07:38:19.541792+0000","last_clean":"2023-05-16T07:38:19.541792+0000","last_became_active":"2023-05-16T07:38:19.541516+0000","last_became_peered":"2023-05-16T07:38:19.541516+0000","last_unstale":"2023-05-16T07:38:19.541792+0000","last_undegraded":"2023-05-16T07:38:19.541792+0000","last_fullsized":"2023-05-16T07:38:19.541792+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175800+0000","last_change":"2023-05-16T07:38:19.171841+0000","last_active":"2023-05-16T07:38:19.175800+0000","last_peered":"2023-05-16T07:38:19.175800+0000","last_clean":"2023-05-16T07:38:19.175800+0000","last_became_active":"2023-05-16T07:38:19.171486+0000","last_became_peered":"2023-05-16T07:38:19.171486+0000","last_unstale":"2023-05-16T07:38:19.175800+0000","last_undegraded":"2023-05-16T07:38:19.175800+0000","last_fullsized":"2023-05-16T07:38:19.175800+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541783+0000","last_change":"2023-05-16T07:38:19.541783+0000","last_active":"2023-05-16T07:38:19.541783+0000","last_peered":"2023-05-16T07:38:19.541783+0000","last_clean":"2023-05-16T07:38:19.541783+0000","last_became_active":"2023-05-16T07:38:19.541556+0000","last_became_peered":"2023-05-16T07:38:19.541556+0000","last_unstale":"2023-05-16T07:38:19.541783+0000","last_undegraded":"2023-05-16T07:38:19.541783+0000","last_fullsized":"2023-05-16T07:38:19.541783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173124+0000","last_change":"2023-05-16T07:38:19.163119+0000","last_active":"2023-05-16T07:38:19.173124+0000","last_peered":"2023-05-16T07:38:19.173124+0000","last_clean":"2023-05-16T07:38:19.173124+0000","last_became_active":"2023-05-16T07:38:19.160693+0000","last_became_peered":"2023-05-16T07:38:19.160693+0000","last_unstale":"2023-05-16T07:38:19.173124+0000","last_undegraded":"2023-05-16T07:38:19.173124+0000","last_fullsized":"2023-05-16T07:38:19.173124+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533171+0000","last_change":"2023-05-16T07:38:19.533171+0000","last_active":"2023-05-16T07:38:19.533171+0000","last_peered":"2023-05-16T07:38:19.533171+0000","last_clean":"2023-05-16T07:38:19.533171+0000","last_became_active":"2023-05-16T07:38:19.532677+0000","last_became_peered":"2023-05-16T07:38:19.532677+0000","last_unstale":"2023-05-16T07:38:19.533171+0000","last_undegraded":"2023-05-16T07:38:19.533171+0000","last_fullsized":"2023-05-16T07:38:19.533171+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168211+0000","last_change":"2023-05-16T07:38:19.164038+0000","last_active":"2023-05-16T07:38:19.168211+0000","last_peered":"2023-05-16T07:38:19.168211+0000","last_clean":"2023-05-16T07:38:19.168211+0000","last_became_active":"2023-05-16T07:38:19.163560+0000","last_became_peered":"2023-05-16T07:38:19.163560+0000","last_unstale":"2023-05-16T07:38:19.168211+0000","last_undegraded":"2023-05-16T07:38:19.168211+0000","last_fullsized":"2023-05-16T07:38:19.168211+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539791+0000","last_change":"2023-05-16T07:38:19.539791+0000","last_active":"2023-05-16T07:38:19.539791+0000","last_peered":"2023-05-16T07:38:19.539791+0000","last_clean":"2023-05-16T07:38:19.539791+0000","last_became_active":"2023-05-16T07:38:19.539557+0000","last_became_peered":"2023-05-16T07:38:19.539557+0000","last_unstale":"2023-05-16T07:38:19.539791+0000","last_undegraded":"2023-05-16T07:38:19.539791+0000","last_fullsized":"2023-05-16T07:38:19.539791+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174695+0000","last_change":"2023-05-16T07:38:19.171996+0000","last_active":"2023-05-16T07:38:19.174695+0000","last_peered":"2023-05-16T07:38:19.174695+0000","last_clean":"2023-05-16T07:38:19.174695+0000","last_became_active":"2023-05-16T07:38:19.171721+0000","last_became_peered":"2023-05-16T07:38:19.171721+0000","last_unstale":"2023-05-16T07:38:19.174695+0000","last_undegraded":"2023-05-16T07:38:19.174695+0000","last_fullsized":"2023-05-16T07:38:19.174695+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533922+0000","last_change":"2023-05-16T07:38:19.533922+0000","last_active":"2023-05-16T07:38:19.533922+0000","last_peered":"2023-05-16T07:38:19.533922+0000","last_clean":"2023-05-16T07:38:19.533922+0000","last_became_active":"2023-05-16T07:38:19.532776+0000","last_became_peered":"2023-05-16T07:38:19.532776+0000","last_unstale":"2023-05-16T07:38:19.533922+0000","last_undegraded":"2023-05-16T07:38:19.533922+0000","last_fullsized":"2023-05-16T07:38:19.533922+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173173+0000","last_change":"2023-05-16T07:38:19.164985+0000","last_active":"2023-05-16T07:38:19.173173+0000","last_peered":"2023-05-16T07:38:19.173173+0000","last_clean":"2023-05-16T07:38:19.173173+0000","last_became_active":"2023-05-16T07:38:19.162021+0000","last_became_peered":"2023-05-16T07:38:19.162021+0000","last_unstale":"2023-05-16T07:38:19.173173+0000","last_undegraded":"2023-05-16T07:38:19.173173+0000","last_fullsized":"2023-05-16T07:38:19.173173+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538004+0000","last_change":"2023-05-16T07:38:19.538004+0000","last_active":"2023-05-16T07:38:19.538004+0000","last_peered":"2023-05-16T07:38:19.538004+0000","last_clean":"2023-05-16T07:38:19.538004+0000","last_became_active":"2023-05-16T07:38:19.537686+0000","last_became_peered":"2023-05-16T07:38:19.537686+0000","last_unstale":"2023-05-16T07:38:19.538004+0000","last_undegraded":"2023-05-16T07:38:19.538004+0000","last_fullsized":"2023-05-16T07:38:19.538004+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.34","version":"170'4","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.546133+0000","last_change":"2023-05-16T07:38:19.164116+0000","last_active":"2023-05-16T07:38:19.546133+0000","last_peered":"2023-05-16T07:38:19.546133+0000","last_clean":"2023-05-16T07:38:19.546133+0000","last_became_active":"2023-05-16T07:38:19.163844+0000","last_became_peered":"2023-05-16T07:38:19.163844+0000","last_unstale":"2023-05-16T07:38:19.546133+0000","last_undegraded":"2023-05-16T07:38:19.546133+0000","last_fullsized":"2023-05-16T07:38:19.546133+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541633+0000","last_change":"2023-05-16T07:38:19.541633+0000","last_active":"2023-05-16T07:38:19.541633+0000","last_peered":"2023-05-16T07:38:19.541633+0000","last_clean":"2023-05-16T07:38:19.541633+0000","last_became_active":"2023-05-16T07:38:19.541202+0000","last_became_peered":"2023-05-16T07:38:19.541202+0000","last_unstale":"2023-05-16T07:38:19.541633+0000","last_undegraded":"2023-05-16T07:38:19.541633+0000","last_fullsized":"2023-05-16T07:38:19.541633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.33","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.527851+0000","last_change":"2023-05-16T07:38:19.165851+0000","last_active":"2023-05-16T07:38:19.527851+0000","last_peered":"2023-05-16T07:38:19.527851+0000","last_clean":"2023-05-16T07:38:19.527851+0000","last_became_active":"2023-05-16T07:38:19.165548+0000","last_became_peered":"2023-05-16T07:38:19.165548+0000","last_unstale":"2023-05-16T07:38:19.527851+0000","last_undegraded":"2023-05-16T07:38:19.527851+0000","last_fullsized":"2023-05-16T07:38:19.527851+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529948+0000","last_change":"2023-05-16T07:38:19.529948+0000","last_active":"2023-05-16T07:38:19.529948+0000","last_peered":"2023-05-16T07:38:19.529948+0000","last_clean":"2023-05-16T07:38:19.529948+0000","last_became_active":"2023-05-16T07:38:19.529659+0000","last_became_peered":"2023-05-16T07:38:19.529659+0000","last_unstale":"2023-05-16T07:38:19.529948+0000","last_undegraded":"2023-05-16T07:38:19.529948+0000","last_fullsized":"2023-05-16T07:38:19.529948+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170386+0000","last_change":"2023-05-16T07:38:19.165230+0000","last_active":"2023-05-16T07:38:19.170386+0000","last_peered":"2023-05-16T07:38:19.170386+0000","last_clean":"2023-05-16T07:38:19.170386+0000","last_became_active":"2023-05-16T07:38:19.164978+0000","last_became_peered":"2023-05-16T07:38:19.164978+0000","last_unstale":"2023-05-16T07:38:19.170386+0000","last_undegraded":"2023-05-16T07:38:19.170386+0000","last_fullsized":"2023-05-16T07:38:19.170386+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.532977+0000","last_change":"2023-05-16T07:38:19.532977+0000","last_active":"2023-05-16T07:38:19.532977+0000","last_peered":"2023-05-16T07:38:19.532977+0000","last_clean":"2023-05-16T07:38:19.532977+0000","last_became_active":"2023-05-16T07:38:19.532691+0000","last_became_peered":"2023-05-16T07:38:19.532691+0000","last_unstale":"2023-05-16T07:38:19.532977+0000","last_undegraded":"2023-05-16T07:38:19.532977+0000","last_fullsized":"2023-05-16T07:38:19.532977+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.166346+0000","last_change":"2023-05-16T07:38:19.163439+0000","last_active":"2023-05-16T07:38:19.166346+0000","last_peered":"2023-05-16T07:38:19.166346+0000","last_clean":"2023-05-16T07:38:19.166346+0000","last_became_active":"2023-05-16T07:38:19.163155+0000","last_became_peered":"2023-05-16T07:38:19.163155+0000","last_unstale":"2023-05-16T07:38:19.166346+0000","last_undegraded":"2023-05-16T07:38:19.166346+0000","last_fullsized":"2023-05-16T07:38:19.166346+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533079+0000","last_change":"2023-05-16T07:38:19.533079+0000","last_active":"2023-05-16T07:38:19.533079+0000","last_peered":"2023-05-16T07:38:19.533079+0000","last_clean":"2023-05-16T07:38:19.533079+0000","last_became_active":"2023-05-16T07:38:19.532807+0000","last_became_peered":"2023-05-16T07:38:19.532807+0000","last_unstale":"2023-05-16T07:38:19.533079+0000","last_undegraded":"2023-05-16T07:38:19.533079+0000","last_fullsized":"2023-05-16T07:38:19.533079+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.30","version":"170'4","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539002+0000","last_change":"2023-05-16T07:38:19.170157+0000","last_active":"2023-05-16T07:38:19.539002+0000","last_peered":"2023-05-16T07:38:19.539002+0000","last_clean":"2023-05-16T07:38:19.539002+0000","last_became_active":"2023-05-16T07:38:19.169889+0000","last_became_peered":"2023-05-16T07:38:19.169889+0000","last_unstale":"2023-05-16T07:38:19.539002+0000","last_undegraded":"2023-05-16T07:38:19.539002+0000","last_fullsized":"2023-05-16T07:38:19.539002+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529235+0000","last_change":"2023-05-16T07:38:19.529235+0000","last_active":"2023-05-16T07:38:19.529235+0000","last_peered":"2023-05-16T07:38:19.529235+0000","last_clean":"2023-05-16T07:38:19.529235+0000","last_became_active":"2023-05-16T07:38:19.528983+0000","last_became_peered":"2023-05-16T07:38:19.528983+0000","last_unstale":"2023-05-16T07:38:19.529235+0000","last_undegraded":"2023-05-16T07:38:19.529235+0000","last_fullsized":"2023-05-16T07:38:19.529235+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.172781+0000","last_change":"2023-05-16T07:38:19.169280+0000","last_active":"2023-05-16T07:38:19.172781+0000","last_peered":"2023-05-16T07:38:19.172781+0000","last_clean":"2023-05-16T07:38:19.172781+0000","last_became_active":"2023-05-16T07:38:19.168999+0000","last_became_peered":"2023-05-16T07:38:19.168999+0000","last_unstale":"2023-05-16T07:38:19.172781+0000","last_undegraded":"2023-05-16T07:38:19.172781+0000","last_fullsized":"2023-05-16T07:38:19.172781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544366+0000","last_change":"2023-05-16T07:38:19.544366+0000","last_active":"2023-05-16T07:38:19.544366+0000","last_peered":"2023-05-16T07:38:19.544366+0000","last_clean":"2023-05-16T07:38:19.544366+0000","last_became_active":"2023-05-16T07:38:19.544185+0000","last_became_peered":"2023-05-16T07:38:19.544185+0000","last_unstale":"2023-05-16T07:38:19.544366+0000","last_undegraded":"2023-05-16T07:38:19.544366+0000","last_fullsized":"2023-05-16T07:38:19.544366+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175054+0000","last_change":"2023-05-16T07:38:19.171517+0000","last_active":"2023-05-16T07:38:19.175054+0000","last_peered":"2023-05-16T07:38:19.175054+0000","last_clean":"2023-05-16T07:38:19.175054+0000","last_became_active":"2023-05-16T07:38:19.170701+0000","last_became_peered":"2023-05-16T07:38:19.170701+0000","last_unstale":"2023-05-16T07:38:19.175054+0000","last_undegraded":"2023-05-16T07:38:19.175054+0000","last_fullsized":"2023-05-16T07:38:19.175054+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.542546+0000","last_change":"2023-05-16T07:38:19.542546+0000","last_active":"2023-05-16T07:38:19.542546+0000","last_peered":"2023-05-16T07:38:19.542546+0000","last_clean":"2023-05-16T07:38:19.542546+0000","last_became_active":"2023-05-16T07:38:19.542182+0000","last_became_peered":"2023-05-16T07:38:19.542182+0000","last_unstale":"2023-05-16T07:38:19.542546+0000","last_undegraded":"2023-05-16T07:38:19.542546+0000","last_fullsized":"2023-05-16T07:38:19.542546+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2d","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.520263+0000","last_change":"2023-05-16T07:38:19.167516+0000","last_active":"2023-05-16T07:38:19.520263+0000","last_peered":"2023-05-16T07:38:19.520263+0000","last_clean":"2023-05-16T07:38:19.520263+0000","last_became_active":"2023-05-16T07:38:19.167249+0000","last_became_peered":"2023-05-16T07:38:19.167249+0000","last_unstale":"2023-05-16T07:38:19.520263+0000","last_undegraded":"2023-05-16T07:38:19.520263+0000","last_fullsized":"2023-05-16T07:38:19.520263+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.531180+0000","last_change":"2023-05-16T07:38:19.531180+0000","last_active":"2023-05-16T07:38:19.531180+0000","last_peered":"2023-05-16T07:38:19.531180+0000","last_clean":"2023-05-16T07:38:19.531180+0000","last_became_active":"2023-05-16T07:38:19.530903+0000","last_became_peered":"2023-05-16T07:38:19.530903+0000","last_unstale":"2023-05-16T07:38:19.531180+0000","last_undegraded":"2023-05-16T07:38:19.531180+0000","last_fullsized":"2023-05-16T07:38:19.531180+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170692+0000","last_change":"2023-05-16T07:38:19.168112+0000","last_active":"2023-05-16T07:38:19.170692+0000","last_peered":"2023-05-16T07:38:19.170692+0000","last_clean":"2023-05-16T07:38:19.170692+0000","last_became_active":"2023-05-16T07:38:19.167857+0000","last_became_peered":"2023-05-16T07:38:19.167857+0000","last_unstale":"2023-05-16T07:38:19.170692+0000","last_undegraded":"2023-05-16T07:38:19.170692+0000","last_fullsized":"2023-05-16T07:38:19.170692+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530050+0000","last_change":"2023-05-16T07:38:19.530050+0000","last_active":"2023-05-16T07:38:19.530050+0000","last_peered":"2023-05-16T07:38:19.530050+0000","last_clean":"2023-05-16T07:38:19.530050+0000","last_became_active":"2023-05-16T07:38:19.529858+0000","last_became_peered":"2023-05-16T07:38:19.529858+0000","last_unstale":"2023-05-16T07:38:19.530050+0000","last_undegraded":"2023-05-16T07:38:19.530050+0000","last_fullsized":"2023-05-16T07:38:19.530050+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.524840+0000","last_change":"2023-05-16T07:38:19.171823+0000","last_active":"2023-05-16T07:38:19.524840+0000","last_peered":"2023-05-16T07:38:19.524840+0000","last_clean":"2023-05-16T07:38:19.524840+0000","last_became_active":"2023-05-16T07:38:19.171527+0000","last_became_peered":"2023-05-16T07:38:19.171527+0000","last_unstale":"2023-05-16T07:38:19.524840+0000","last_undegraded":"2023-05-16T07:38:19.524840+0000","last_fullsized":"2023-05-16T07:38:19.524840+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537287+0000","last_change":"2023-05-16T07:38:19.537287+0000","last_active":"2023-05-16T07:38:19.537287+0000","last_peered":"2023-05-16T07:38:19.537287+0000","last_clean":"2023-05-16T07:38:19.537287+0000","last_became_active":"2023-05-16T07:38:19.537002+0000","last_became_peered":"2023-05-16T07:38:19.537002+0000","last_unstale":"2023-05-16T07:38:19.537287+0000","last_undegraded":"2023-05-16T07:38:19.537287+0000","last_fullsized":"2023-05-16T07:38:19.537287+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170016+0000","last_change":"2023-05-16T07:38:19.160298+0000","last_active":"2023-05-16T07:38:19.170016+0000","last_peered":"2023-05-16T07:38:19.170016+0000","last_clean":"2023-05-16T07:38:19.170016+0000","last_became_active":"2023-05-16T07:38:19.159649+0000","last_became_peered":"2023-05-16T07:38:19.159649+0000","last_unstale":"2023-05-16T07:38:19.170016+0000","last_undegraded":"2023-05-16T07:38:19.170016+0000","last_fullsized":"2023-05-16T07:38:19.170016+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539406+0000","last_change":"2023-05-16T07:38:19.539406+0000","last_active":"2023-05-16T07:38:19.539406+0000","last_peered":"2023-05-16T07:38:19.539406+0000","last_clean":"2023-05-16T07:38:19.539406+0000","last_became_active":"2023-05-16T07:38:19.539124+0000","last_became_peered":"2023-05-16T07:38:19.539124+0000","last_unstale":"2023-05-16T07:38:19.539406+0000","last_undegraded":"2023-05-16T07:38:19.539406+0000","last_fullsized":"2023-05-16T07:38:19.539406+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174770+0000","last_change":"2023-05-16T07:38:19.172878+0000","last_active":"2023-05-16T07:38:19.174770+0000","last_peered":"2023-05-16T07:38:19.174770+0000","last_clean":"2023-05-16T07:38:19.174770+0000","last_became_active":"2023-05-16T07:38:19.172144+0000","last_became_peered":"2023-05-16T07:38:19.172144+0000","last_unstale":"2023-05-16T07:38:19.174770+0000","last_undegraded":"2023-05-16T07:38:19.174770+0000","last_fullsized":"2023-05-16T07:38:19.174770+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537911+0000","last_change":"2023-05-16T07:38:19.537911+0000","last_active":"2023-05-16T07:38:19.537911+0000","last_peered":"2023-05-16T07:38:19.537911+0000","last_clean":"2023-05-16T07:38:19.537911+0000","last_became_active":"2023-05-16T07:38:19.537701+0000","last_became_peered":"2023-05-16T07:38:19.537701+0000","last_unstale":"2023-05-16T07:38:19.537911+0000","last_undegraded":"2023-05-16T07:38:19.537911+0000","last_fullsized":"2023-05-16T07:38:19.537911+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174291+0000","last_change":"2023-05-16T07:38:19.171498+0000","last_active":"2023-05-16T07:38:19.174291+0000","last_peered":"2023-05-16T07:38:19.174291+0000","last_clean":"2023-05-16T07:38:19.174291+0000","last_became_active":"2023-05-16T07:38:19.171216+0000","last_became_peered":"2023-05-16T07:38:19.171216+0000","last_unstale":"2023-05-16T07:38:19.174291+0000","last_undegraded":"2023-05-16T07:38:19.174291+0000","last_fullsized":"2023-05-16T07:38:19.174291+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.531209+0000","last_change":"2023-05-16T07:38:19.531209+0000","last_active":"2023-05-16T07:38:19.531209+0000","last_peered":"2023-05-16T07:38:19.531209+0000","last_clean":"2023-05-16T07:38:19.531209+0000","last_became_active":"2023-05-16T07:38:19.530796+0000","last_became_peered":"2023-05-16T07:38:19.530796+0000","last_unstale":"2023-05-16T07:38:19.531209+0000","last_undegraded":"2023-05-16T07:38:19.531209+0000","last_fullsized":"2023-05-16T07:38:19.531209+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167002+0000","last_change":"2023-05-16T07:38:19.163587+0000","last_active":"2023-05-16T07:38:19.167002+0000","last_peered":"2023-05-16T07:38:19.167002+0000","last_clean":"2023-05-16T07:38:19.167002+0000","last_became_active":"2023-05-16T07:38:19.163304+0000","last_became_peered":"2023-05-16T07:38:19.163304+0000","last_unstale":"2023-05-16T07:38:19.167002+0000","last_undegraded":"2023-05-16T07:38:19.167002+0000","last_fullsized":"2023-05-16T07:38:19.167002+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.540666+0000","last_change":"2023-05-16T07:38:19.540666+0000","last_active":"2023-05-16T07:38:19.540666+0000","last_peered":"2023-05-16T07:38:19.540666+0000","last_clean":"2023-05-16T07:38:19.540666+0000","last_became_active":"2023-05-16T07:38:19.540404+0000","last_became_peered":"2023-05-16T07:38:19.540404+0000","last_unstale":"2023-05-16T07:38:19.540666+0000","last_undegraded":"2023-05-16T07:38:19.540666+0000","last_fullsized":"2023-05-16T07:38:19.540666+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174026+0000","last_change":"2023-05-16T07:38:19.167379+0000","last_active":"2023-05-16T07:38:19.174026+0000","last_peered":"2023-05-16T07:38:19.174026+0000","last_clean":"2023-05-16T07:38:19.174026+0000","last_became_active":"2023-05-16T07:38:19.167100+0000","last_became_peered":"2023-05-16T07:38:19.167100+0000","last_unstale":"2023-05-16T07:38:19.174026+0000","last_undegraded":"2023-05-16T07:38:19.174026+0000","last_fullsized":"2023-05-16T07:38:19.174026+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538069+0000","last_change":"2023-05-16T07:38:19.538069+0000","last_active":"2023-05-16T07:38:19.538069+0000","last_peered":"2023-05-16T07:38:19.538069+0000","last_clean":"2023-05-16T07:38:19.538069+0000","last_became_active":"2023-05-16T07:38:19.537729+0000","last_became_peered":"2023-05-16T07:38:19.537729+0000","last_unstale":"2023-05-16T07:38:19.538069+0000","last_undegraded":"2023-05-16T07:38:19.538069+0000","last_fullsized":"2023-05-16T07:38:19.538069+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167636+0000","last_change":"2023-05-16T07:38:19.165776+0000","last_active":"2023-05-16T07:38:19.167636+0000","last_peered":"2023-05-16T07:38:19.167636+0000","last_clean":"2023-05-16T07:38:19.167636+0000","last_became_active":"2023-05-16T07:38:19.165529+0000","last_became_peered":"2023-05-16T07:38:19.165529+0000","last_unstale":"2023-05-16T07:38:19.167636+0000","last_undegraded":"2023-05-16T07:38:19.167636+0000","last_fullsized":"2023-05-16T07:38:19.167636+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534985+0000","last_change":"2023-05-16T07:38:19.534985+0000","last_active":"2023-05-16T07:38:19.534985+0000","last_peered":"2023-05-16T07:38:19.534985+0000","last_clean":"2023-05-16T07:38:19.534985+0000","last_became_active":"2023-05-16T07:38:19.534698+0000","last_became_peered":"2023-05-16T07:38:19.534698+0000","last_unstale":"2023-05-16T07:38:19.534985+0000","last_undegraded":"2023-05-16T07:38:19.534985+0000","last_fullsized":"2023-05-16T07:38:19.534985+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.169637+0000","last_change":"2023-05-16T07:38:19.164818+0000","last_active":"2023-05-16T07:38:19.169637+0000","last_peered":"2023-05-16T07:38:19.169637+0000","last_clean":"2023-05-16T07:38:19.169637+0000","last_became_active":"2023-05-16T07:38:19.164479+0000","last_became_peered":"2023-05-16T07:38:19.164479+0000","last_unstale":"2023-05-16T07:38:19.169637+0000","last_undegraded":"2023-05-16T07:38:19.169637+0000","last_fullsized":"2023-05-16T07:38:19.169637+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534139+0000","last_change":"2023-05-16T07:38:19.534139+0000","last_active":"2023-05-16T07:38:19.534139+0000","last_peered":"2023-05-16T07:38:19.534139+0000","last_clean":"2023-05-16T07:38:19.534139+0000","last_became_active":"2023-05-16T07:38:19.533066+0000","last_became_peered":"2023-05-16T07:38:19.533066+0000","last_unstale":"2023-05-16T07:38:19.534139+0000","last_undegraded":"2023-05-16T07:38:19.534139+0000","last_fullsized":"2023-05-16T07:38:19.534139+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173037+0000","last_change":"2023-05-16T07:38:19.162695+0000","last_active":"2023-05-16T07:38:19.173037+0000","last_peered":"2023-05-16T07:38:19.173037+0000","last_clean":"2023-05-16T07:38:19.173037+0000","last_became_active":"2023-05-16T07:38:19.162417+0000","last_became_peered":"2023-05-16T07:38:19.162417+0000","last_unstale":"2023-05-16T07:38:19.173037+0000","last_undegraded":"2023-05-16T07:38:19.173037+0000","last_fullsized":"2023-05-16T07:38:19.173037+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530706+0000","last_change":"2023-05-16T07:38:19.530706+0000","last_active":"2023-05-16T07:38:19.530706+0000","last_peered":"2023-05-16T07:38:19.530706+0000","last_clean":"2023-05-16T07:38:19.530706+0000","last_became_active":"2023-05-16T07:38:19.530450+0000","last_became_peered":"2023-05-16T07:38:19.530450+0000","last_unstale":"2023-05-16T07:38:19.530706+0000","last_undegraded":"2023-05-16T07:38:19.530706+0000","last_fullsized":"2023-05-16T07:38:19.530706+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170594+0000","last_change":"2023-05-16T07:38:19.169576+0000","last_active":"2023-05-16T07:38:19.170594+0000","last_peered":"2023-05-16T07:38:19.170594+0000","last_clean":"2023-05-16T07:38:19.170594+0000","last_became_active":"2023-05-16T07:38:19.169141+0000","last_became_peered":"2023-05-16T07:38:19.169141+0000","last_unstale":"2023-05-16T07:38:19.170594+0000","last_undegraded":"2023-05-16T07:38:19.170594+0000","last_fullsized":"2023-05-16T07:38:19.170594+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534104+0000","last_change":"2023-05-16T07:38:19.534104+0000","last_active":"2023-05-16T07:38:19.534104+0000","last_peered":"2023-05-16T07:38:19.534104+0000","last_clean":"2023-05-16T07:38:19.534104+0000","last_became_active":"2023-05-16T07:38:19.533716+0000","last_became_peered":"2023-05-16T07:38:19.533716+0000","last_unstale":"2023-05-16T07:38:19.534104+0000","last_undegraded":"2023-05-16T07:38:19.534104+0000","last_fullsized":"2023-05-16T07:38:19.534104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167999+0000","last_change":"2023-05-16T07:38:19.161601+0000","last_active":"2023-05-16T07:38:19.167999+0000","last_peered":"2023-05-16T07:38:19.167999+0000","last_clean":"2023-05-16T07:38:19.167999+0000","last_became_active":"2023-05-16T07:38:19.161110+0000","last_became_peered":"2023-05-16T07:38:19.161110+0000","last_unstale":"2023-05-16T07:38:19.167999+0000","last_undegraded":"2023-05-16T07:38:19.167999+0000","last_fullsized":"2023-05-16T07:38:19.167999+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537801+0000","last_change":"2023-05-16T07:38:19.537801+0000","last_active":"2023-05-16T07:38:19.537801+0000","last_peered":"2023-05-16T07:38:19.537801+0000","last_clean":"2023-05-16T07:38:19.537801+0000","last_became_active":"2023-05-16T07:38:19.537487+0000","last_became_peered":"2023-05-16T07:38:19.537487+0000","last_unstale":"2023-05-16T07:38:19.537801+0000","last_undegraded":"2023-05-16T07:38:19.537801+0000","last_fullsized":"2023-05-16T07:38:19.537801+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.166757+0000","last_change":"2023-05-16T07:38:19.162979+0000","last_active":"2023-05-16T07:38:19.166757+0000","last_peered":"2023-05-16T07:38:19.166757+0000","last_clean":"2023-05-16T07:38:19.166757+0000","last_became_active":"2023-05-16T07:38:19.162683+0000","last_became_peered":"2023-05-16T07:38:19.162683+0000","last_unstale":"2023-05-16T07:38:19.166757+0000","last_undegraded":"2023-05-16T07:38:19.166757+0000","last_fullsized":"2023-05-16T07:38:19.166757+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536047+0000","last_change":"2023-05-16T07:38:19.536047+0000","last_active":"2023-05-16T07:38:19.536047+0000","last_peered":"2023-05-16T07:38:19.536047+0000","last_clean":"2023-05-16T07:38:19.536047+0000","last_became_active":"2023-05-16T07:38:19.534937+0000","last_became_peered":"2023-05-16T07:38:19.534937+0000","last_unstale":"2023-05-16T07:38:19.536047+0000","last_undegraded":"2023-05-16T07:38:19.536047+0000","last_fullsized":"2023-05-16T07:38:19.536047+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.528215+0000","last_change":"2023-05-16T07:38:19.152409+0000","last_active":"2023-05-16T07:38:19.528215+0000","last_peered":"2023-05-16T07:38:19.528215+0000","last_clean":"2023-05-16T07:38:19.528215+0000","last_became_active":"2023-05-16T07:38:19.151297+0000","last_became_peered":"2023-05-16T07:38:19.151297+0000","last_unstale":"2023-05-16T07:38:19.528215+0000","last_undegraded":"2023-05-16T07:38:19.528215+0000","last_fullsized":"2023-05-16T07:38:19.528215+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529767+0000","last_change":"2023-05-16T07:38:19.529767+0000","last_active":"2023-05-16T07:38:19.529767+0000","last_peered":"2023-05-16T07:38:19.529767+0000","last_clean":"2023-05-16T07:38:19.529767+0000","last_became_active":"2023-05-16T07:38:19.529496+0000","last_became_peered":"2023-05-16T07:38:19.529496+0000","last_unstale":"2023-05-16T07:38:19.529767+0000","last_undegraded":"2023-05-16T07:38:19.529767+0000","last_fullsized":"2023-05-16T07:38:19.529767+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.524937+0000","last_change":"2023-05-16T07:38:19.163250+0000","last_active":"2023-05-16T07:38:19.524937+0000","last_peered":"2023-05-16T07:38:19.524937+0000","last_clean":"2023-05-16T07:38:19.524937+0000","last_became_active":"2023-05-16T07:38:19.162947+0000","last_became_peered":"2023-05-16T07:38:19.162947+0000","last_unstale":"2023-05-16T07:38:19.524937+0000","last_undegraded":"2023-05-16T07:38:19.524937+0000","last_fullsized":"2023-05-16T07:38:19.524937+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533912+0000","last_change":"2023-05-16T07:38:19.533912+0000","last_active":"2023-05-16T07:38:19.533912+0000","last_peered":"2023-05-16T07:38:19.533912+0000","last_clean":"2023-05-16T07:38:19.533912+0000","last_became_active":"2023-05-16T07:38:19.532383+0000","last_became_peered":"2023-05-16T07:38:19.532383+0000","last_unstale":"2023-05-16T07:38:19.533912+0000","last_undegraded":"2023-05-16T07:38:19.533912+0000","last_fullsized":"2023-05-16T07:38:19.533912+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.546953+0000","last_change":"2023-05-16T07:38:19.165008+0000","last_active":"2023-05-16T07:38:19.546953+0000","last_peered":"2023-05-16T07:38:19.546953+0000","last_clean":"2023-05-16T07:38:19.546953+0000","last_became_active":"2023-05-16T07:38:19.164735+0000","last_became_peered":"2023-05-16T07:38:19.164735+0000","last_unstale":"2023-05-16T07:38:19.546953+0000","last_undegraded":"2023-05-16T07:38:19.546953+0000","last_fullsized":"2023-05-16T07:38:19.546953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.543367+0000","last_change":"2023-05-16T07:38:19.543367+0000","last_active":"2023-05-16T07:38:19.543367+0000","last_peered":"2023-05-16T07:38:19.543367+0000","last_clean":"2023-05-16T07:38:19.543367+0000","last_became_active":"2023-05-16T07:38:19.543102+0000","last_became_peered":"2023-05-16T07:38:19.543102+0000","last_unstale":"2023-05-16T07:38:19.543367+0000","last_undegraded":"2023-05-16T07:38:19.543367+0000","last_fullsized":"2023-05-16T07:38:19.543367+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.178766+0000","last_change":"2023-05-16T07:38:19.175866+0000","last_active":"2023-05-16T07:38:19.178766+0000","last_peered":"2023-05-16T07:38:19.178766+0000","last_clean":"2023-05-16T07:38:19.178766+0000","last_became_active":"2023-05-16T07:38:19.175575+0000","last_became_peered":"2023-05-16T07:38:19.175575+0000","last_unstale":"2023-05-16T07:38:19.178766+0000","last_undegraded":"2023-05-16T07:38:19.178766+0000","last_fullsized":"2023-05-16T07:38:19.178766+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530070+0000","last_change":"2023-05-16T07:38:19.530070+0000","last_active":"2023-05-16T07:38:19.530070+0000","last_peered":"2023-05-16T07:38:19.530070+0000","last_clean":"2023-05-16T07:38:19.530070+0000","last_became_active":"2023-05-16T07:38:19.529855+0000","last_became_peered":"2023-05-16T07:38:19.529855+0000","last_unstale":"2023-05-16T07:38:19.530070+0000","last_undegraded":"2023-05-16T07:38:19.530070+0000","last_fullsized":"2023-05-16T07:38:19.530070+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167107+0000","last_change":"2023-05-16T07:38:19.162618+0000","last_active":"2023-05-16T07:38:19.167107+0000","last_peered":"2023-05-16T07:38:19.167107+0000","last_clean":"2023-05-16T07:38:19.167107+0000","last_became_active":"2023-05-16T07:38:19.162061+0000","last_became_peered":"2023-05-16T07:38:19.162061+0000","last_unstale":"2023-05-16T07:38:19.167107+0000","last_undegraded":"2023-05-16T07:38:19.167107+0000","last_fullsized":"2023-05-16T07:38:19.167107+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530811+0000","last_change":"2023-05-16T07:38:19.530811+0000","last_active":"2023-05-16T07:38:19.530811+0000","last_peered":"2023-05-16T07:38:19.530811+0000","last_clean":"2023-05-16T07:38:19.530811+0000","last_became_active":"2023-05-16T07:38:19.530506+0000","last_became_peered":"2023-05-16T07:38:19.530506+0000","last_unstale":"2023-05-16T07:38:19.530811+0000","last_undegraded":"2023-05-16T07:38:19.530811+0000","last_fullsized":"2023-05-16T07:38:19.530811+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168007+0000","last_change":"2023-05-16T07:38:19.161158+0000","last_active":"2023-05-16T07:38:19.168007+0000","last_peered":"2023-05-16T07:38:19.168007+0000","last_clean":"2023-05-16T07:38:19.168007+0000","last_became_active":"2023-05-16T07:38:19.160661+0000","last_became_peered":"2023-05-16T07:38:19.160661+0000","last_unstale":"2023-05-16T07:38:19.168007+0000","last_undegraded":"2023-05-16T07:38:19.168007+0000","last_fullsized":"2023-05-16T07:38:19.168007+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.528209+0000","last_change":"2023-05-16T07:38:19.528209+0000","last_active":"2023-05-16T07:38:19.528209+0000","last_peered":"2023-05-16T07:38:19.528209+0000","last_clean":"2023-05-16T07:38:19.528209+0000","last_became_active":"2023-05-16T07:38:19.527939+0000","last_became_peered":"2023-05-16T07:38:19.527939+0000","last_unstale":"2023-05-16T07:38:19.528209+0000","last_undegraded":"2023-05-16T07:38:19.528209+0000","last_fullsized":"2023-05-16T07:38:19.528209+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170201+0000","last_change":"2023-05-16T07:38:19.168860+0000","last_active":"2023-05-16T07:38:19.170201+0000","last_peered":"2023-05-16T07:38:19.170201+0000","last_clean":"2023-05-16T07:38:19.170201+0000","last_became_active":"2023-05-16T07:38:19.168593+0000","last_became_peered":"2023-05-16T07:38:19.168593+0000","last_unstale":"2023-05-16T07:38:19.170201+0000","last_undegraded":"2023-05-16T07:38:19.170201+0000","last_fullsized":"2023-05-16T07:38:19.170201+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544375+0000","last_change":"2023-05-16T07:38:19.544375+0000","last_active":"2023-05-16T07:38:19.544375+0000","last_peered":"2023-05-16T07:38:19.544375+0000","last_clean":"2023-05-16T07:38:19.544375+0000","last_became_active":"2023-05-16T07:38:19.544092+0000","last_became_peered":"2023-05-16T07:38:19.544092+0000","last_unstale":"2023-05-16T07:38:19.544375+0000","last_undegraded":"2023-05-16T07:38:19.544375+0000","last_fullsized":"2023-05-16T07:38:19.544375+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167611+0000","last_change":"2023-05-16T07:38:19.157775+0000","last_active":"2023-05-16T07:38:19.167611+0000","last_peered":"2023-05-16T07:38:19.167611+0000","last_clean":"2023-05-16T07:38:19.167611+0000","last_became_active":"2023-05-16T07:38:19.157496+0000","last_became_peered":"2023-05-16T07:38:19.157496+0000","last_unstale":"2023-05-16T07:38:19.167611+0000","last_undegraded":"2023-05-16T07:38:19.167611+0000","last_fullsized":"2023-05-16T07:38:19.167611+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538641+0000","last_change":"2023-05-16T07:38:19.538641+0000","last_active":"2023-05-16T07:38:19.538641+0000","last_peered":"2023-05-16T07:38:19.538641+0000","last_clean":"2023-05-16T07:38:19.538641+0000","last_became_active":"2023-05-16T07:38:19.538365+0000","last_became_peered":"2023-05-16T07:38:19.538365+0000","last_unstale":"2023-05-16T07:38:19.538641+0000","last_undegraded":"2023-05-16T07:38:19.538641+0000","last_fullsized":"2023-05-16T07:38:19.538641+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.523167+0000","last_change":"2023-05-16T07:38:19.165083+0000","last_active":"2023-05-16T07:38:19.523167+0000","last_peered":"2023-05-16T07:38:19.523167+0000","last_clean":"2023-05-16T07:38:19.523167+0000","last_became_active":"2023-05-16T07:38:19.164486+0000","last_became_peered":"2023-05-16T07:38:19.164486+0000","last_unstale":"2023-05-16T07:38:19.523167+0000","last_undegraded":"2023-05-16T07:38:19.523167+0000","last_fullsized":"2023-05-16T07:38:19.523167+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544385+0000","last_change":"2023-05-16T07:38:19.544385+0000","last_active":"2023-05-16T07:38:19.544385+0000","last_peered":"2023-05-16T07:38:19.544385+0000","last_clean":"2023-05-16T07:38:19.544385+0000","last_became_active":"2023-05-16T07:38:19.544070+0000","last_became_peered":"2023-05-16T07:38:19.544070+0000","last_unstale":"2023-05-16T07:38:19.544385+0000","last_undegraded":"2023-05-16T07:38:19.544385+0000","last_fullsized":"2023-05-16T07:38:19.544385+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.540376+0000","last_change":"2023-05-16T07:38:19.161915+0000","last_active":"2023-05-16T07:38:19.540376+0000","last_peered":"2023-05-16T07:38:19.540376+0000","last_clean":"2023-05-16T07:38:19.540376+0000","last_became_active":"2023-05-16T07:38:19.158881+0000","last_became_peered":"2023-05-16T07:38:19.158881+0000","last_unstale":"2023-05-16T07:38:19.540376+0000","last_undegraded":"2023-05-16T07:38:19.540376+0000","last_fullsized":"2023-05-16T07:38:19.540376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534991+0000","last_change":"2023-05-16T07:38:19.534991+0000","last_active":"2023-05-16T07:38:19.534991+0000","last_peered":"2023-05-16T07:38:19.534991+0000","last_clean":"2023-05-16T07:38:19.534991+0000","last_became_active":"2023-05-16T07:38:19.534657+0000","last_became_peered":"2023-05-16T07:38:19.534657+0000","last_unstale":"2023-05-16T07:38:19.534991+0000","last_undegraded":"2023-05-16T07:38:19.534991+0000","last_fullsized":"2023-05-16T07:38:19.534991+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173984+0000","last_change":"2023-05-16T07:38:19.166573+0000","last_active":"2023-05-16T07:38:19.173984+0000","last_peered":"2023-05-16T07:38:19.173984+0000","last_clean":"2023-05-16T07:38:19.173984+0000","last_became_active":"2023-05-16T07:38:19.166316+0000","last_became_peered":"2023-05-16T07:38:19.166316+0000","last_unstale":"2023-05-16T07:38:19.173984+0000","last_undegraded":"2023-05-16T07:38:19.173984+0000","last_fullsized":"2023-05-16T07:38:19.173984+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541725+0000","last_change":"2023-05-16T07:38:19.541725+0000","last_active":"2023-05-16T07:38:19.541725+0000","last_peered":"2023-05-16T07:38:19.541725+0000","last_clean":"2023-05-16T07:38:19.541725+0000","last_became_active":"2023-05-16T07:38:19.539585+0000","last_became_peered":"2023-05-16T07:38:19.539585+0000","last_unstale":"2023-05-16T07:38:19.541725+0000","last_undegraded":"2023-05-16T07:38:19.541725+0000","last_fullsized":"2023-05-16T07:38:19.541725+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174995+0000","last_change":"2023-05-16T07:38:19.171306+0000","last_active":"2023-05-16T07:38:19.174995+0000","last_peered":"2023-05-16T07:38:19.174995+0000","last_clean":"2023-05-16T07:38:19.174995+0000","last_became_active":"2023-05-16T07:38:19.171025+0000","last_became_peered":"2023-05-16T07:38:19.171025+0000","last_unstale":"2023-05-16T07:38:19.174995+0000","last_undegraded":"2023-05-16T07:38:19.174995+0000","last_fullsized":"2023-05-16T07:38:19.174995+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544423+0000","last_change":"2023-05-16T07:38:19.544423+0000","last_active":"2023-05-16T07:38:19.544423+0000","last_peered":"2023-05-16T07:38:19.544423+0000","last_clean":"2023-05-16T07:38:19.544423+0000","last_became_active":"2023-05-16T07:38:19.544061+0000","last_became_peered":"2023-05-16T07:38:19.544061+0000","last_unstale":"2023-05-16T07:38:19.544423+0000","last_undegraded":"2023-05-16T07:38:19.544423+0000","last_fullsized":"2023-05-16T07:38:19.544423+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.523885+0000","last_change":"2023-05-16T07:38:19.172332+0000","last_active":"2023-05-16T07:38:19.523885+0000","last_peered":"2023-05-16T07:38:19.523885+0000","last_clean":"2023-05-16T07:38:19.523885+0000","last_became_active":"2023-05-16T07:38:19.172170+0000","last_became_peered":"2023-05-16T07:38:19.172170+0000","last_unstale":"2023-05-16T07:38:19.523885+0000","last_undegraded":"2023-05-16T07:38:19.523885+0000","last_fullsized":"2023-05-16T07:38:19.523885+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.535734+0000","last_change":"2023-05-16T07:38:19.535734+0000","last_active":"2023-05-16T07:38:19.535734+0000","last_peered":"2023-05-16T07:38:19.535734+0000","last_clean":"2023-05-16T07:38:19.535734+0000","last_became_active":"2023-05-16T07:38:19.535471+0000","last_became_peered":"2023-05-16T07:38:19.535471+0000","last_unstale":"2023-05-16T07:38:19.535734+0000","last_undegraded":"2023-05-16T07:38:19.535734+0000","last_fullsized":"2023-05-16T07:38:19.535734+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170569+0000","last_change":"2023-05-16T07:38:19.169407+0000","last_active":"2023-05-16T07:38:19.170569+0000","last_peered":"2023-05-16T07:38:19.170569+0000","last_clean":"2023-05-16T07:38:19.170569+0000","last_became_active":"2023-05-16T07:38:19.166523+0000","last_became_peered":"2023-05-16T07:38:19.166523+0000","last_unstale":"2023-05-16T07:38:19.170569+0000","last_undegraded":"2023-05-16T07:38:19.170569+0000","last_fullsized":"2023-05-16T07:38:19.170569+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537079+0000","last_change":"2023-05-16T07:38:19.537079+0000","last_active":"2023-05-16T07:38:19.537079+0000","last_peered":"2023-05-16T07:38:19.537079+0000","last_clean":"2023-05-16T07:38:19.537079+0000","last_became_active":"2023-05-16T07:38:19.535678+0000","last_became_peered":"2023-05-16T07:38:19.535678+0000","last_unstale":"2023-05-16T07:38:19.537079+0000","last_undegraded":"2023-05-16T07:38:19.537079+0000","last_fullsized":"2023-05-16T07:38:19.537079+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175781+0000","last_change":"2023-05-16T07:38:19.171657+0000","last_active":"2023-05-16T07:38:19.175781+0000","last_peered":"2023-05-16T07:38:19.175781+0000","last_clean":"2023-05-16T07:38:19.175781+0000","last_became_active":"2023-05-16T07:38:19.171371+0000","last_became_peered":"2023-05-16T07:38:19.171371+0000","last_unstale":"2023-05-16T07:38:19.175781+0000","last_undegraded":"2023-05-16T07:38:19.175781+0000","last_fullsized":"2023-05-16T07:38:19.175781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533872+0000","last_change":"2023-05-16T07:38:19.533872+0000","last_active":"2023-05-16T07:38:19.533872+0000","last_peered":"2023-05-16T07:38:19.533872+0000","last_clean":"2023-05-16T07:38:19.533872+0000","last_became_active":"2023-05-16T07:38:19.533622+0000","last_became_peered":"2023-05-16T07:38:19.533622+0000","last_unstale":"2023-05-16T07:38:19.533872+0000","last_undegraded":"2023-05-16T07:38:19.533872+0000","last_fullsized":"2023-05-16T07:38:19.533872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174282+0000","last_change":"2023-05-16T07:38:19.171432+0000","last_active":"2023-05-16T07:38:19.174282+0000","last_peered":"2023-05-16T07:38:19.174282+0000","last_clean":"2023-05-16T07:38:19.174282+0000","last_became_active":"2023-05-16T07:38:19.171247+0000","last_became_peered":"2023-05-16T07:38:19.171247+0000","last_unstale":"2023-05-16T07:38:19.174282+0000","last_undegraded":"2023-05-16T07:38:19.174282+0000","last_fullsized":"2023-05-16T07:38:19.174282+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541806+0000","last_change":"2023-05-16T07:38:19.541806+0000","last_active":"2023-05-16T07:38:19.541806+0000","last_peered":"2023-05-16T07:38:19.541806+0000","last_clean":"2023-05-16T07:38:19.541806+0000","last_became_active":"2023-05-16T07:38:19.541536+0000","last_became_peered":"2023-05-16T07:38:19.541536+0000","last_unstale":"2023-05-16T07:38:19.541806+0000","last_undegraded":"2023-05-16T07:38:19.541806+0000","last_fullsized":"2023-05-16T07:38:19.541806+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539976+0000","last_change":"2023-05-16T07:38:19.167397+0000","last_active":"2023-05-16T07:38:19.539976+0000","last_peered":"2023-05-16T07:38:19.539976+0000","last_clean":"2023-05-16T07:38:19.539976+0000","last_became_active":"2023-05-16T07:38:19.167126+0000","last_became_peered":"2023-05-16T07:38:19.167126+0000","last_unstale":"2023-05-16T07:38:19.539976+0000","last_undegraded":"2023-05-16T07:38:19.539976+0000","last_fullsized":"2023-05-16T07:38:19.539976+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533710+0000","last_change":"2023-05-16T07:38:19.533710+0000","last_active":"2023-05-16T07:38:19.533710+0000","last_peered":"2023-05-16T07:38:19.533710+0000","last_clean":"2023-05-16T07:38:19.533710+0000","last_became_active":"2023-05-16T07:38:19.533420+0000","last_became_peered":"2023-05-16T07:38:19.533420+0000","last_unstale":"2023-05-16T07:38:19.533710+0000","last_undegraded":"2023-05-16T07:38:19.533710+0000","last_fullsized":"2023-05-16T07:38:19.533710+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174104+0000","last_change":"2023-05-16T07:38:19.172674+0000","last_active":"2023-05-16T07:38:19.174104+0000","last_peered":"2023-05-16T07:38:19.174104+0000","last_clean":"2023-05-16T07:38:19.174104+0000","last_became_active":"2023-05-16T07:38:19.172017+0000","last_became_peered":"2023-05-16T07:38:19.172017+0000","last_unstale":"2023-05-16T07:38:19.174104+0000","last_undegraded":"2023-05-16T07:38:19.174104+0000","last_fullsized":"2023-05-16T07:38:19.174104+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544441+0000","last_change":"2023-05-16T07:38:19.544441+0000","last_active":"2023-05-16T07:38:19.544441+0000","last_peered":"2023-05-16T07:38:19.544441+0000","last_clean":"2023-05-16T07:38:19.544441+0000","last_became_active":"2023-05-16T07:38:19.544174+0000","last_became_peered":"2023-05-16T07:38:19.544174+0000","last_unstale":"2023-05-16T07:38:19.544441+0000","last_undegraded":"2023-05-16T07:38:19.544441+0000","last_fullsized":"2023-05-16T07:38:19.544441+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.176422+0000","last_change":"2023-05-16T07:38:19.175532+0000","last_active":"2023-05-16T07:38:19.176422+0000","last_peered":"2023-05-16T07:38:19.176422+0000","last_clean":"2023-05-16T07:38:19.176422+0000","last_became_active":"2023-05-16T07:38:19.175252+0000","last_became_peered":"2023-05-16T07:38:19.175252+0000","last_unstale":"2023-05-16T07:38:19.176422+0000","last_undegraded":"2023-05-16T07:38:19.176422+0000","last_fullsized":"2023-05-16T07:38:19.176422+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533936+0000","last_change":"2023-05-16T07:38:19.533936+0000","last_active":"2023-05-16T07:38:19.533936+0000","last_peered":"2023-05-16T07:38:19.533936+0000","last_clean":"2023-05-16T07:38:19.533936+0000","last_became_active":"2023-05-16T07:38:19.533674+0000","last_became_peered":"2023-05-16T07:38:19.533674+0000","last_unstale":"2023-05-16T07:38:19.533936+0000","last_undegraded":"2023-05-16T07:38:19.533936+0000","last_fullsized":"2023-05-16T07:38:19.533936+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168010+0000","last_change":"2023-05-16T07:38:19.160936+0000","last_active":"2023-05-16T07:38:19.168010+0000","last_peered":"2023-05-16T07:38:19.168010+0000","last_clean":"2023-05-16T07:38:19.168010+0000","last_became_active":"2023-05-16T07:38:19.160578+0000","last_became_peered":"2023-05-16T07:38:19.160578+0000","last_unstale":"2023-05-16T07:38:19.168010+0000","last_undegraded":"2023-05-16T07:38:19.168010+0000","last_fullsized":"2023-05-16T07:38:19.168010+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536663+0000","last_change":"2023-05-16T07:38:19.536663+0000","last_active":"2023-05-16T07:38:19.536663+0000","last_peered":"2023-05-16T07:38:19.536663+0000","last_clean":"2023-05-16T07:38:19.536663+0000","last_became_active":"2023-05-16T07:38:19.535219+0000","last_became_peered":"2023-05-16T07:38:19.535219+0000","last_unstale":"2023-05-16T07:38:19.536663+0000","last_undegraded":"2023-05-16T07:38:19.536663+0000","last_fullsized":"2023-05-16T07:38:19.536663+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167286+0000","last_change":"2023-05-16T07:38:19.162725+0000","last_active":"2023-05-16T07:38:19.167286+0000","last_peered":"2023-05-16T07:38:19.167286+0000","last_clean":"2023-05-16T07:38:19.167286+0000","last_became_active":"2023-05-16T07:38:19.162449+0000","last_became_peered":"2023-05-16T07:38:19.162449+0000","last_unstale":"2023-05-16T07:38:19.167286+0000","last_undegraded":"2023-05-16T07:38:19.167286+0000","last_fullsized":"2023-05-16T07:38:19.167286+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.531948+0000","last_change":"2023-05-16T07:38:19.531948+0000","last_active":"2023-05-16T07:38:19.531948+0000","last_peered":"2023-05-16T07:38:19.531948+0000","last_clean":"2023-05-16T07:38:19.531948+0000","last_became_active":"2023-05-16T07:38:19.531663+0000","last_became_peered":"2023-05-16T07:38:19.531663+0000","last_unstale":"2023-05-16T07:38:19.531948+0000","last_undegraded":"2023-05-16T07:38:19.531948+0000","last_fullsized":"2023-05-16T07:38:19.531948+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.176254+0000","last_change":"2023-05-16T07:38:19.171302+0000","last_active":"2023-05-16T07:38:19.176254+0000","last_peered":"2023-05-16T07:38:19.176254+0000","last_clean":"2023-05-16T07:38:19.176254+0000","last_became_active":"2023-05-16T07:38:19.171017+0000","last_became_peered":"2023-05-16T07:38:19.171017+0000","last_unstale":"2023-05-16T07:38:19.176254+0000","last_undegraded":"2023-05-16T07:38:19.176254+0000","last_fullsized":"2023-05-16T07:38:19.176254+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536567+0000","last_change":"2023-05-16T07:38:19.536567+0000","last_active":"2023-05-16T07:38:19.536567+0000","last_peered":"2023-05-16T07:38:19.536567+0000","last_clean":"2023-05-16T07:38:19.536567+0000","last_became_active":"2023-05-16T07:38:19.536299+0000","last_became_peered":"2023-05-16T07:38:19.536299+0000","last_unstale":"2023-05-16T07:38:19.536567+0000","last_undegraded":"2023-05-16T07:38:19.536567+0000","last_fullsized":"2023-05-16T07:38:19.536567+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.169744+0000","last_change":"2023-05-16T07:38:19.167585+0000","last_active":"2023-05-16T07:38:19.169744+0000","last_peered":"2023-05-16T07:38:19.169744+0000","last_clean":"2023-05-16T07:38:19.169744+0000","last_became_active":"2023-05-16T07:38:19.167197+0000","last_became_peered":"2023-05-16T07:38:19.167197+0000","last_unstale":"2023-05-16T07:38:19.169744+0000","last_undegraded":"2023-05-16T07:38:19.169744+0000","last_fullsized":"2023-05-16T07:38:19.169744+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534798+0000","last_change":"2023-05-16T07:38:19.534798+0000","last_active":"2023-05-16T07:38:19.534798+0000","last_peered":"2023-05-16T07:38:19.534798+0000","last_clean":"2023-05-16T07:38:19.534798+0000","last_became_active":"2023-05-16T07:38:19.534521+0000","last_became_peered":"2023-05-16T07:38:19.534521+0000","last_unstale":"2023-05-16T07:38:19.534798+0000","last_undegraded":"2023-05-16T07:38:19.534798+0000","last_fullsized":"2023-05-16T07:38:19.534798+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174272+0000","last_change":"2023-05-16T07:38:19.170260+0000","last_active":"2023-05-16T07:38:19.174272+0000","last_peered":"2023-05-16T07:38:19.174272+0000","last_clean":"2023-05-16T07:38:19.174272+0000","last_became_active":"2023-05-16T07:38:19.169979+0000","last_became_peered":"2023-05-16T07:38:19.169979+0000","last_unstale":"2023-05-16T07:38:19.174272+0000","last_undegraded":"2023-05-16T07:38:19.174272+0000","last_fullsized":"2023-05-16T07:38:19.174272+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.525635+0000","last_change":"2023-05-16T07:38:19.525635+0000","last_active":"2023-05-16T07:38:19.525635+0000","last_peered":"2023-05-16T07:38:19.525635+0000","last_clean":"2023-05-16T07:38:19.525635+0000","last_became_active":"2023-05-16T07:38:19.525279+0000","last_became_peered":"2023-05-16T07:38:19.525279+0000","last_unstale":"2023-05-16T07:38:19.525635+0000","last_undegraded":"2023-05-16T07:38:19.525635+0000","last_fullsized":"2023-05-16T07:38:19.525635+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.528621+0000","last_change":"2023-05-16T07:38:19.163935+0000","last_active":"2023-05-16T07:38:19.528621+0000","last_peered":"2023-05-16T07:38:19.528621+0000","last_clean":"2023-05-16T07:38:19.528621+0000","last_became_active":"2023-05-16T07:38:19.163457+0000","last_became_peered":"2023-05-16T07:38:19.163457+0000","last_unstale":"2023-05-16T07:38:19.528621+0000","last_undegraded":"2023-05-16T07:38:19.528621+0000","last_fullsized":"2023-05-16T07:38:19.528621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.154048+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:38:19.154048+0000","last_peered":"2023-05-16T07:38:19.154048+0000","last_clean":"2023-05-16T07:38:19.154048+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:19.154048+0000","last_undegraded":"2023-05-16T07:38:19.154048+0000","last_fullsized":"2023-05-16T07:38:19.154048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.542456+0000","last_change":"2023-05-16T07:38:19.542456+0000","last_active":"2023-05-16T07:38:19.542456+0000","last_peered":"2023-05-16T07:38:19.542456+0000","last_clean":"2023-05-16T07:38:19.542456+0000","last_became_active":"2023-05-16T07:38:19.542159+0000","last_became_peered":"2023-05-16T07:38:19.542159+0000","last_unstale":"2023-05-16T07:38:19.542456+0000","last_undegraded":"2023-05-16T07:38:19.542456+0000","last_fullsized":"2023-05-16T07:38:19.542456+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"170'3","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537203+0000","last_change":"2023-05-16T07:38:19.165203+0000","last_active":"2023-05-16T07:38:19.537203+0000","last_peered":"2023-05-16T07:38:19.537203+0000","last_clean":"2023-05-16T07:38:19.537203+0000","last_became_active":"2023-05-16T07:38:19.164447+0000","last_became_peered":"2023-05-16T07:38:19.164447+0000","last_unstale":"2023-05-16T07:38:19.537203+0000","last_undegraded":"2023-05-16T07:38:19.537203+0000","last_fullsized":"2023-05-16T07:38:19.537203+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.156185+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:38:19.156185+0000","last_peered":"2023-05-16T07:38:19.156185+0000","last_clean":"2023-05-16T07:38:19.156185+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:19.156185+0000","last_undegraded":"2023-05-16T07:38:19.156185+0000","last_fullsized":"2023-05-16T07:38:19.156185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539802+0000","last_change":"2023-05-16T07:38:19.539802+0000","last_active":"2023-05-16T07:38:19.539802+0000","last_peered":"2023-05-16T07:38:19.539802+0000","last_clean":"2023-05-16T07:38:19.539802+0000","last_became_active":"2023-05-16T07:38:19.539584+0000","last_became_peered":"2023-05-16T07:38:19.539584+0000","last_unstale":"2023-05-16T07:38:19.539802+0000","last_undegraded":"2023-05-16T07:38:19.539802+0000","last_fullsized":"2023-05-16T07:38:19.539802+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.169713+0000","last_change":"2023-05-16T07:38:19.167975+0000","last_active":"2023-05-16T07:38:19.169713+0000","last_peered":"2023-05-16T07:38:19.169713+0000","last_clean":"2023-05-16T07:38:19.169713+0000","last_became_active":"2023-05-16T07:38:19.167683+0000","last_became_peered":"2023-05-16T07:38:19.167683+0000","last_unstale":"2023-05-16T07:38:19.169713+0000","last_undegraded":"2023-05-16T07:38:19.169713+0000","last_fullsized":"2023-05-16T07:38:19.169713+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.163756+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:38:19.163756+0000","last_peered":"2023-05-16T07:38:19.163756+0000","last_clean":"2023-05-16T07:38:19.163756+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:19.163756+0000","last_undegraded":"2023-05-16T07:38:19.163756+0000","last_fullsized":"2023-05-16T07:38:19.163756+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536285+0000","last_change":"2023-05-16T07:38:19.536285+0000","last_active":"2023-05-16T07:38:19.536285+0000","last_peered":"2023-05-16T07:38:19.536285+0000","last_clean":"2023-05-16T07:38:19.536285+0000","last_became_active":"2023-05-16T07:38:19.535906+0000","last_became_peered":"2023-05-16T07:38:19.535906+0000","last_unstale":"2023-05-16T07:38:19.536285+0000","last_undegraded":"2023-05-16T07:38:19.536285+0000","last_fullsized":"2023-05-16T07:38:19.536285+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541009+0000","last_change":"2023-05-16T07:38:19.172800+0000","last_active":"2023-05-16T07:38:19.541009+0000","last_peered":"2023-05-16T07:38:19.541009+0000","last_clean":"2023-05-16T07:38:19.541009+0000","last_became_active":"2023-05-16T07:38:19.172550+0000","last_became_peered":"2023-05-16T07:38:19.172550+0000","last_unstale":"2023-05-16T07:38:19.541009+0000","last_undegraded":"2023-05-16T07:38:19.541009+0000","last_fullsized":"2023-05-16T07:38:19.541009+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.156307+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:38:19.156307+0000","last_peered":"2023-05-16T07:38:19.156307+0000","last_clean":"2023-05-16T07:38:19.156307+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:19.156307+0000","last_undegraded":"2023-05-16T07:38:19.156307+0000","last_fullsized":"2023-05-16T07:38:19.156307+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":135,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.155152+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:38:19.155152+0000","last_peered":"2023-05-16T07:38:19.155152+0000","last_clean":"2023-05-16T07:38:19.155152+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:19.155152+0000","last_undegraded":"2023-05-16T07:38:19.155152+0000","last_fullsized":"2023-05-16T07:38:19.155152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536747+0000","last_change":"2023-05-16T07:38:19.536747+0000","last_active":"2023-05-16T07:38:19.536747+0000","last_peered":"2023-05-16T07:38:19.536747+0000","last_clean":"2023-05-16T07:38:19.536747+0000","last_became_active":"2023-05-16T07:38:19.533150+0000","last_became_peered":"2023-05-16T07:38:19.533150+0000","last_unstale":"2023-05-16T07:38:19.536747+0000","last_undegraded":"2023-05-16T07:38:19.536747+0000","last_fullsized":"2023-05-16T07:38:19.536747+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174143+0000","last_change":"2023-05-16T07:38:19.171534+0000","last_active":"2023-05-16T07:38:19.174143+0000","last_peered":"2023-05-16T07:38:19.174143+0000","last_clean":"2023-05-16T07:38:19.174143+0000","last_became_active":"2023-05-16T07:38:19.171245+0000","last_became_peered":"2023-05-16T07:38:19.171245+0000","last_unstale":"2023-05-16T07:38:19.174143+0000","last_undegraded":"2023-05-16T07:38:19.174143+0000","last_fullsized":"2023-05-16T07:38:19.174143+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.155239+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:38:19.155239+0000","last_peered":"2023-05-16T07:38:19.155239+0000","last_clean":"2023-05-16T07:38:19.155239+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:19.155239+0000","last_undegraded":"2023-05-16T07:38:19.155239+0000","last_fullsized":"2023-05-16T07:38:19.155239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534724+0000","last_change":"2023-05-16T07:38:19.534724+0000","last_active":"2023-05-16T07:38:19.534724+0000","last_peered":"2023-05-16T07:38:19.534724+0000","last_clean":"2023-05-16T07:38:19.534724+0000","last_became_active":"2023-05-16T07:38:19.534443+0000","last_became_peered":"2023-05-16T07:38:19.534443+0000","last_unstale":"2023-05-16T07:38:19.534724+0000","last_undegraded":"2023-05-16T07:38:19.534724+0000","last_fullsized":"2023-05-16T07:38:19.534724+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175084+0000","last_change":"2023-05-16T07:38:19.171763+0000","last_active":"2023-05-16T07:38:19.175084+0000","last_peered":"2023-05-16T07:38:19.175084+0000","last_clean":"2023-05-16T07:38:19.175084+0000","last_became_active":"2023-05-16T07:38:19.171125+0000","last_became_peered":"2023-05-16T07:38:19.171125+0000","last_unstale":"2023-05-16T07:38:19.175084+0000","last_undegraded":"2023-05-16T07:38:19.175084+0000","last_fullsized":"2023-05-16T07:38:19.175084+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.153981+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:38:19.153981+0000","last_peered":"2023-05-16T07:38:19.153981+0000","last_clean":"2023-05-16T07:38:19.153981+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:19.153981+0000","last_undegraded":"2023-05-16T07:38:19.153981+0000","last_fullsized":"2023-05-16T07:38:19.153981+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.532926+0000","last_change":"2023-05-16T07:38:19.532926+0000","last_active":"2023-05-16T07:38:19.532926+0000","last_peered":"2023-05-16T07:38:19.532926+0000","last_clean":"2023-05-16T07:38:19.532926+0000","last_became_active":"2023-05-16T07:38:19.532656+0000","last_became_peered":"2023-05-16T07:38:19.532656+0000","last_unstale":"2023-05-16T07:38:19.532926+0000","last_undegraded":"2023-05-16T07:38:19.532926+0000","last_fullsized":"2023-05-16T07:38:19.532926+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.166189+0000","last_change":"2023-05-16T07:38:19.165001+0000","last_active":"2023-05-16T07:38:19.166189+0000","last_peered":"2023-05-16T07:38:19.166189+0000","last_clean":"2023-05-16T07:38:19.166189+0000","last_became_active":"2023-05-16T07:38:19.164667+0000","last_became_peered":"2023-05-16T07:38:19.164667+0000","last_unstale":"2023-05-16T07:38:19.166189+0000","last_undegraded":"2023-05-16T07:38:19.166189+0000","last_fullsized":"2023-05-16T07:38:19.166189+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.153076+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:38:19.153076+0000","last_peered":"2023-05-16T07:38:19.153076+0000","last_clean":"2023-05-16T07:38:19.153076+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:19.153076+0000","last_undegraded":"2023-05-16T07:38:19.153076+0000","last_fullsized":"2023-05-16T07:38:19.153076+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.535416+0000","last_change":"2023-05-16T07:38:19.535416+0000","last_active":"2023-05-16T07:38:19.535416+0000","last_peered":"2023-05-16T07:38:19.535416+0000","last_clean":"2023-05-16T07:38:19.535416+0000","last_became_active":"2023-05-16T07:38:19.535132+0000","last_became_peered":"2023-05-16T07:38:19.535132+0000","last_unstale":"2023-05-16T07:38:19.535416+0000","last_undegraded":"2023-05-16T07:38:19.535416+0000","last_fullsized":"2023-05-16T07:38:19.535416+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"170'4","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.543207+0000","last_change":"2023-05-16T07:38:19.172524+0000","last_active":"2023-05-16T07:38:19.543207+0000","last_peered":"2023-05-16T07:38:19.543207+0000","last_clean":"2023-05-16T07:38:19.543207+0000","last_became_active":"2023-05-16T07:38:19.172262+0000","last_became_peered":"2023-05-16T07:38:19.172262+0000","last_unstale":"2023-05-16T07:38:19.543207+0000","last_undegraded":"2023-05-16T07:38:19.543207+0000","last_fullsized":"2023-05-16T07:38:19.543207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":161,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.153250+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:38:19.153250+0000","last_peered":"2023-05-16T07:38:19.153250+0000","last_clean":"2023-05-16T07:38:19.153250+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:19.153250+0000","last_undegraded":"2023-05-16T07:38:19.153250+0000","last_fullsized":"2023-05-16T07:38:19.153250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575273,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24980,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":22912,"kb_avail":94346860,"statfs":{"total":96636764160,"available":96611184640,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23461888},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.77300000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.505}]},{"osd":1,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.34899999999999998},"max":{"1min":0.747,"5min":0.77600000000000002,"15min":0.79800000000000004},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.38900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72999999999999998,"5min":0.77500000000000002,"15min":0.81999999999999995},"last":0.57799999999999996}]},{"osd":3,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.45400000000000001,"5min":0.436,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68000000000000005}]},{"osd":4,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46300000000000002,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.71499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.75},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.71199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575273,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24992,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":22912,"kb_avail":94346848,"statfs":{"total":96636764160,"available":96611172352,"internally_reserved":0,"allocated":2121728,"data_stored":1136587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23461888},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.26700000000000002},"max":{"1min":0.68700000000000006,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.27900000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.505}]},{"osd":1,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.5,"15min":1.5},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.35599999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60399999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.373},"max":{"1min":0.747,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.57999999999999996},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.32900000000000001},"max":{"1min":0.71299999999999997,"5min":1.486,"15min":1.486},"last":0.44900000000000001}]},{"osd":3,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.373},"max":{"1min":0.77400000000000002,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78300000000000003,"5min":1.532,"15min":1.532},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76500000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.499},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.30199999999999999},"max":{"1min":0.76500000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52600000000000002,"15min":0.497},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.79900000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81299999999999994},"last":0.753}]},{"osd":7,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58899999999999997,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.316},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575274,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20360,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":18304,"kb_avail":94351480,"statfs":{"total":96636764160,"available":96615915520,"internally_reserved":0,"allocated":2097152,"data_stored":1121393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18743296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.374,"15min":0.372},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.47799999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60299999999999998,"5min":0.63,"15min":0.64100000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.42499999999999999,"15min":0.433},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40100000000000002,"15min":0.40799999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.58599999999999997,"15min":0.67900000000000005},"last":0.505}]},{"osd":4,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.872,"5min":0.873,"15min":0.877},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.62}]},{"osd":5,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.83699999999999997,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.42699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.998},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.437,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.67000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575274,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20208,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":18112,"kb_avail":94351632,"statfs":{"total":96636764160,"available":96616071168,"internally_reserved":0,"allocated":2138112,"data_stored":1137355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18546688},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.36499999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.222},"max":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.58899999999999997},"last":0.311},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.24199999999999999},"max":{"1min":0.48199999999999998,"5min":0.59799999999999998,"15min":0.627},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63100000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.26300000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.64600000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.70199999999999996},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.92700000000000005,"15min":0.98699999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.46700000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":1.0549999999999999},"last":0.496}]},{"osd":5,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":0.94899999999999995,"15min":1.0680000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.68300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.872,"5min":0.93500000000000005,"15min":1.095},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92100000000000004,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.42699999999999999,"15min":0.40899999999999997},"max":{"1min":0.89100000000000001,"5min":0.92900000000000005,"15min":1},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.03},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.44400000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":1.056},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21932,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94349908,"statfs":{"total":96636764160,"available":96614305792,"internally_reserved":0,"allocated":2134016,"data_stored":1136769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.28000000000000003}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.375}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.53700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575273,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.5,"5min":0.51200000000000001,"15min":0.61899999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28299999999999997,"5min":0.223,"15min":0.223},"max":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.308}]},{"osd":1,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39000000000000001,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48499999999999999,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40999999999999998,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.72999999999999998},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.432},"min":{"1min":0.34699999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.68400000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.441,"15min":0.442},"min":{"1min":0.29599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.63900000000000001,"5min":0.67300000000000004,"15min":0.70099999999999996},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":1.014,"15min":1.014},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.94399999999999995,"15min":1.024},"last":0.41699999999999998}]},{"osd":5,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84799999999999998,"5min":1.05,"15min":1.05},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.65400000000000003},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.41699999999999998},"max":{"1min":0.84799999999999998,"5min":0.96599999999999997,"15min":0.97299999999999998},"last":0.40600000000000003}]},{"osd":6,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83399999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.432}]},{"osd":7,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.48799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88100000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.89300000000000002,"5min":0.91300000000000003,"15min":1.012},"last":0.504}]}]},{"osd":4,"up_from":13,"seq":55834575273,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24788,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":22720,"kb_avail":94347052,"statfs":{"total":96636764160,"available":96611381248,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23265280},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.439,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.503}]},{"osd":1,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78600000000000003,"15min":0.81899999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38},"max":{"1min":0.74299999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.58099999999999996},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.68400000000000005}]},{"osd":3,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.373},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.624}]},{"osd":5,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.29099999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"max":{"1min":0.72499999999999998,"5min":0.755,"15min":0.755},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.33500000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.754},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.70599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575273,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80400000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.80800000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.39700000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.626},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.85599999999999998},"last":0.61399999999999999}]},{"osd":4,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.71299999999999997,"5min":0.74199999999999999,"15min":0.84899999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.497,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.83299999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.504,"15min":0.496},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.872},"last":0.626}]},{"osd":7,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":0.86299999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81100000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:38:22.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:38:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.649+0000 7fe22a921700 1 Processor -- start 2023-05-16T07:38:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.649+0000 7fe22a921700 1 -- start start 2023-05-16T07:38:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22a921700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe224150b50 0x7fe224154870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22a921700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2240bd390 con 0x7fe224155470 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22a921700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2240bd820 con 0x7fe224150f30 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22a921700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2240bc880 con 0x7fe224150b50 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe224150b50 0x7fe224154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe224150b50 0x7fe224154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52816/0 (socket says 172.21.15.92:52816) 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22a120700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe224155470 0x7fe224065090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35336/0 (socket says 172.21.15.92:35336) 2023-05-16T07:38:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 -- 172.21.15.92:0/2823236976 learned_addr learned my addr 172.21.15.92:0/2823236976 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 -- 172.21.15.92:0/2823236976 >> v1:172.21.15.158:6789/0 conn(0x7fe224150f30 legacy=0x7fe224154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 -- 172.21.15.92:0/2823236976 >> v1:172.21.15.92:6789/0 conn(0x7fe224155470 legacy=0x7fe224065090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22891d700 1 -- 172.21.15.92:0/2823236976 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499288384 0 0) 0x7fe2240bd390 con 0x7fe224155470 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 -- 172.21.15.92:0/2823236976 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2240bca60 con 0x7fe224150b50 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22991f700 1 --2- 172.21.15.92:0/2823236976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150b50 0x7fe224154870 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7fe21c009f70 tx=0x7fe21c009840).ready entity=mon.0 client_cookie=53ce9aee73023101 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22891d700 1 -- 172.21.15.92:0/2823236976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00e070 con 0x7fe224150b50 2023-05-16T07:38:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22891d700 1 -- 172.21.15.92:0/2823236976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c00ba90 con 0x7fe224150b50 2023-05-16T07:38:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.650+0000 7fe22891d700 1 -- 172.21.15.92:0/2823236976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00be10 con 0x7fe224150b50 2023-05-16T07:38:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 -- 172.21.15.92:0/2823236976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150b50 msgr2=0x7fe224154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 --2- 172.21.15.92:0/2823236976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150b50 0x7fe224154870 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7fe21c009f70 tx=0x7fe21c009840).stop 2023-05-16T07:38:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 -- 172.21.15.92:0/2823236976 shutdown_connections 2023-05-16T07:38:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 --2- 172.21.15.92:0/2823236976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150b50 0x7fe224154870 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 -- 172.21.15.92:0/2823236976 >> 172.21.15.92:0/2823236976 conn(0x7fe224069940 msgr2=0x7fe2240c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 -- 172.21.15.92:0/2823236976 shutdown_connections 2023-05-16T07:38:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.651+0000 7fe22a921700 1 -- 172.21.15.92:0/2823236976 wait complete. 2023-05-16T07:38:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.652+0000 7fe22a921700 1 Processor -- start 2023-05-16T07:38:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.652+0000 7fe22a921700 1 -- start start 2023-05-16T07:38:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe224150b50 0x7fe2240ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 0x7fe2240be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe224155470 0x7fe22413e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22414d590 con 0x7fe224150f30 2023-05-16T07:38:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240bf8f0 con 0x7fe224155470 2023-05-16T07:38:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240bf050 con 0x7fe224150b50 2023-05-16T07:38:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 0x7fe2240be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 0x7fe2240be9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52824/0 (socket says 172.21.15.92:52824) 2023-05-16T07:38:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 -- 172.21.15.92:0/2850479758 learned_addr learned my addr 172.21.15.92:0/2850479758 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a120700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe224155470 0x7fe22413e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 -- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe224150b50 msgr2=0x7fe2240ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe224150b50 0x7fe2240ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 -- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe224155470 msgr2=0x7fe22413e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe224155470 0x7fe22413e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 -- 172.21.15.92:0/2850479758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c009ae0 con 0x7fe224150f30 2023-05-16T07:38:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22911e700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 0x7fe2240be9a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fe21400b490 tx=0x7fe214009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214012410 con 0x7fe224150f30 2023-05-16T07:38:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2140125b0 con 0x7fe224150f30 2023-05-16T07:38:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2240bf330 con 0x7fe224150f30 2023-05-16T07:38:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.653+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214012970 con 0x7fe224150f30 2023-05-16T07:38:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.654+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2240aa060 con 0x7fe224150f30 2023-05-16T07:38:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.654+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2240aa280 con 0x7fe224150f30 2023-05-16T07:38:22.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.655+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe21400f030 con 0x7fe224150f30 2023-05-16T07:38:22.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.655+0000 7fe219ffb700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe20805ed90 0x7fe208061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:22.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.655+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fe214079ad0 con 0x7fe224150f30 2023-05-16T07:38:22.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.655+0000 7fe22991f700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe20805ed90 0x7fe208061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:22.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.656+0000 7fe22991f700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe20805ed90 0x7fe208061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fe21c00ba60 tx=0x7fe21c009840).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:22.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.659+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe21404add0 con 0x7fe224150f30 2023-05-16T07:38:22.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.816+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe22405e160 con 0x7fe20805ed90 2023-05-16T07:38:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.827+0000 7fe219ffb700 1 -- 172.21.15.92:0/2850479758 <== mgr.7208 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361496 (secure 0 0 0) 0x7fe22405e160 con 0x7fe20805ed90 2023-05-16T07:38:22.836 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:22.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe20805ed90 msgr2=0x7fe208061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe20805ed90 0x7fe208061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fe21c00ba60 tx=0x7fe21c009840).stop 2023-05-16T07:38:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 msgr2=0x7fe2240be9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 0x7fe2240be9a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fe21400b490 tx=0x7fe214009060).stop 2023-05-16T07:38:22.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 shutdown_connections 2023-05-16T07:38:22.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe20805ed90 0x7fe208061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe224150b50 0x7fe2240ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:22.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe224155470 0x7fe22413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 --2- 172.21.15.92:0/2850479758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe224150f30 0x7fe2240be9a0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:22.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 >> 172.21.15.92:0/2850479758 conn(0x7fe224069940 msgr2=0x7fe22414dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:22.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 shutdown_connections 2023-05-16T07:38:22.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:22.830+0000 7fe22a921700 1 -- 172.21.15.92:0/2850479758 wait complete. 2023-05-16T07:38:22.843 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:38:22.846 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:38:20.942207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":193684,"kb_used_data":16532,"kb_used_omap":0,"kb_used_meta":177088,"kb_avail":754781036,"statfs":{"total":773094113280,"available":772895780864,"internally_reserved":0,"allocated":16928768,"data_stored":9047064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":181338112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28002,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-77,"num_write_kb":-59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":-1,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.587081"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529432+0000","last_change":"2023-05-16T07:38:19.529432+0000","last_active":"2023-05-16T07:38:19.529432+0000","last_peered":"2023-05-16T07:38:19.529432+0000","last_clean":"2023-05-16T07:38:19.529432+0000","last_became_active":"2023-05-16T07:38:19.529104+0000","last_became_peered":"2023-05-16T07:38:19.529104+0000","last_unstale":"2023-05-16T07:38:19.529432+0000","last_undegraded":"2023-05-16T07:38:19.529432+0000","last_fullsized":"2023-05-16T07:38:19.529432+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3f","version":"170'3","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538943+0000","last_change":"2023-05-16T07:38:19.169222+0000","last_active":"2023-05-16T07:38:19.538943+0000","last_peered":"2023-05-16T07:38:19.538943+0000","last_clean":"2023-05-16T07:38:19.538943+0000","last_became_active":"2023-05-16T07:38:19.168992+0000","last_became_peered":"2023-05-16T07:38:19.168992+0000","last_unstale":"2023-05-16T07:38:19.538943+0000","last_undegraded":"2023-05-16T07:38:19.538943+0000","last_fullsized":"2023-05-16T07:38:19.538943+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539968+0000","last_change":"2023-05-16T07:38:19.539968+0000","last_active":"2023-05-16T07:38:19.539968+0000","last_peered":"2023-05-16T07:38:19.539968+0000","last_clean":"2023-05-16T07:38:19.539968+0000","last_became_active":"2023-05-16T07:38:19.539715+0000","last_became_peered":"2023-05-16T07:38:19.539715+0000","last_unstale":"2023-05-16T07:38:19.539968+0000","last_undegraded":"2023-05-16T07:38:19.539968+0000","last_fullsized":"2023-05-16T07:38:19.539968+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168219+0000","last_change":"2023-05-16T07:38:19.164572+0000","last_active":"2023-05-16T07:38:19.168219+0000","last_peered":"2023-05-16T07:38:19.168219+0000","last_clean":"2023-05-16T07:38:19.168219+0000","last_became_active":"2023-05-16T07:38:19.164316+0000","last_became_peered":"2023-05-16T07:38:19.164316+0000","last_unstale":"2023-05-16T07:38:19.168219+0000","last_undegraded":"2023-05-16T07:38:19.168219+0000","last_fullsized":"2023-05-16T07:38:19.168219+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533168+0000","last_change":"2023-05-16T07:38:19.533168+0000","last_active":"2023-05-16T07:38:19.533168+0000","last_peered":"2023-05-16T07:38:19.533168+0000","last_clean":"2023-05-16T07:38:19.533168+0000","last_became_active":"2023-05-16T07:38:19.530640+0000","last_became_peered":"2023-05-16T07:38:19.530640+0000","last_unstale":"2023-05-16T07:38:19.533168+0000","last_undegraded":"2023-05-16T07:38:19.533168+0000","last_fullsized":"2023-05-16T07:38:19.533168+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174939+0000","last_change":"2023-05-16T07:38:19.170667+0000","last_active":"2023-05-16T07:38:19.174939+0000","last_peered":"2023-05-16T07:38:19.174939+0000","last_clean":"2023-05-16T07:38:19.174939+0000","last_became_active":"2023-05-16T07:38:19.170247+0000","last_became_peered":"2023-05-16T07:38:19.170247+0000","last_unstale":"2023-05-16T07:38:19.174939+0000","last_undegraded":"2023-05-16T07:38:19.174939+0000","last_fullsized":"2023-05-16T07:38:19.174939+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.545057+0000","last_change":"2023-05-16T07:38:19.545057+0000","last_active":"2023-05-16T07:38:19.545057+0000","last_peered":"2023-05-16T07:38:19.545057+0000","last_clean":"2023-05-16T07:38:19.545057+0000","last_became_active":"2023-05-16T07:38:19.544804+0000","last_became_peered":"2023-05-16T07:38:19.544804+0000","last_unstale":"2023-05-16T07:38:19.545057+0000","last_undegraded":"2023-05-16T07:38:19.545057+0000","last_fullsized":"2023-05-16T07:38:19.545057+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3c","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539613+0000","last_change":"2023-05-16T07:38:19.163223+0000","last_active":"2023-05-16T07:38:19.539613+0000","last_peered":"2023-05-16T07:38:19.539613+0000","last_clean":"2023-05-16T07:38:19.539613+0000","last_became_active":"2023-05-16T07:38:19.162988+0000","last_became_peered":"2023-05-16T07:38:19.162988+0000","last_unstale":"2023-05-16T07:38:19.539613+0000","last_undegraded":"2023-05-16T07:38:19.539613+0000","last_fullsized":"2023-05-16T07:38:19.539613+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537238+0000","last_change":"2023-05-16T07:38:19.537238+0000","last_active":"2023-05-16T07:38:19.537238+0000","last_peered":"2023-05-16T07:38:19.537238+0000","last_clean":"2023-05-16T07:38:19.537238+0000","last_became_active":"2023-05-16T07:38:19.536977+0000","last_became_peered":"2023-05-16T07:38:19.536977+0000","last_unstale":"2023-05-16T07:38:19.537238+0000","last_undegraded":"2023-05-16T07:38:19.537238+0000","last_fullsized":"2023-05-16T07:38:19.537238+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.527727+0000","last_change":"2023-05-16T07:38:19.172261+0000","last_active":"2023-05-16T07:38:19.527727+0000","last_peered":"2023-05-16T07:38:19.527727+0000","last_clean":"2023-05-16T07:38:19.527727+0000","last_became_active":"2023-05-16T07:38:19.171980+0000","last_became_peered":"2023-05-16T07:38:19.171980+0000","last_unstale":"2023-05-16T07:38:19.527727+0000","last_undegraded":"2023-05-16T07:38:19.527727+0000","last_fullsized":"2023-05-16T07:38:19.527727+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539205+0000","last_change":"2023-05-16T07:38:19.539205+0000","last_active":"2023-05-16T07:38:19.539205+0000","last_peered":"2023-05-16T07:38:19.539205+0000","last_clean":"2023-05-16T07:38:19.539205+0000","last_became_active":"2023-05-16T07:38:19.538925+0000","last_became_peered":"2023-05-16T07:38:19.538925+0000","last_unstale":"2023-05-16T07:38:19.539205+0000","last_undegraded":"2023-05-16T07:38:19.539205+0000","last_fullsized":"2023-05-16T07:38:19.539205+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.519780+0000","last_change":"2023-05-16T07:38:19.166200+0000","last_active":"2023-05-16T07:38:19.519780+0000","last_peered":"2023-05-16T07:38:19.519780+0000","last_clean":"2023-05-16T07:38:19.519780+0000","last_became_active":"2023-05-16T07:38:19.163888+0000","last_became_peered":"2023-05-16T07:38:19.163888+0000","last_unstale":"2023-05-16T07:38:19.519780+0000","last_undegraded":"2023-05-16T07:38:19.519780+0000","last_fullsized":"2023-05-16T07:38:19.519780+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541792+0000","last_change":"2023-05-16T07:38:19.541792+0000","last_active":"2023-05-16T07:38:19.541792+0000","last_peered":"2023-05-16T07:38:19.541792+0000","last_clean":"2023-05-16T07:38:19.541792+0000","last_became_active":"2023-05-16T07:38:19.541516+0000","last_became_peered":"2023-05-16T07:38:19.541516+0000","last_unstale":"2023-05-16T07:38:19.541792+0000","last_undegraded":"2023-05-16T07:38:19.541792+0000","last_fullsized":"2023-05-16T07:38:19.541792+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175800+0000","last_change":"2023-05-16T07:38:19.171841+0000","last_active":"2023-05-16T07:38:19.175800+0000","last_peered":"2023-05-16T07:38:19.175800+0000","last_clean":"2023-05-16T07:38:19.175800+0000","last_became_active":"2023-05-16T07:38:19.171486+0000","last_became_peered":"2023-05-16T07:38:19.171486+0000","last_unstale":"2023-05-16T07:38:19.175800+0000","last_undegraded":"2023-05-16T07:38:19.175800+0000","last_fullsized":"2023-05-16T07:38:19.175800+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541783+0000","last_change":"2023-05-16T07:38:19.541783+0000","last_active":"2023-05-16T07:38:19.541783+0000","last_peered":"2023-05-16T07:38:19.541783+0000","last_clean":"2023-05-16T07:38:19.541783+0000","last_became_active":"2023-05-16T07:38:19.541556+0000","last_became_peered":"2023-05-16T07:38:19.541556+0000","last_unstale":"2023-05-16T07:38:19.541783+0000","last_undegraded":"2023-05-16T07:38:19.541783+0000","last_fullsized":"2023-05-16T07:38:19.541783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173124+0000","last_change":"2023-05-16T07:38:19.163119+0000","last_active":"2023-05-16T07:38:19.173124+0000","last_peered":"2023-05-16T07:38:19.173124+0000","last_clean":"2023-05-16T07:38:19.173124+0000","last_became_active":"2023-05-16T07:38:19.160693+0000","last_became_peered":"2023-05-16T07:38:19.160693+0000","last_unstale":"2023-05-16T07:38:19.173124+0000","last_undegraded":"2023-05-16T07:38:19.173124+0000","last_fullsized":"2023-05-16T07:38:19.173124+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533171+0000","last_change":"2023-05-16T07:38:19.533171+0000","last_active":"2023-05-16T07:38:19.533171+0000","last_peered":"2023-05-16T07:38:19.533171+0000","last_clean":"2023-05-16T07:38:19.533171+0000","last_became_active":"2023-05-16T07:38:19.532677+0000","last_became_peered":"2023-05-16T07:38:19.532677+0000","last_unstale":"2023-05-16T07:38:19.533171+0000","last_undegraded":"2023-05-16T07:38:19.533171+0000","last_fullsized":"2023-05-16T07:38:19.533171+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168211+0000","last_change":"2023-05-16T07:38:19.164038+0000","last_active":"2023-05-16T07:38:19.168211+0000","last_peered":"2023-05-16T07:38:19.168211+0000","last_clean":"2023-05-16T07:38:19.168211+0000","last_became_active":"2023-05-16T07:38:19.163560+0000","last_became_peered":"2023-05-16T07:38:19.163560+0000","last_unstale":"2023-05-16T07:38:19.168211+0000","last_undegraded":"2023-05-16T07:38:19.168211+0000","last_fullsized":"2023-05-16T07:38:19.168211+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539791+0000","last_change":"2023-05-16T07:38:19.539791+0000","last_active":"2023-05-16T07:38:19.539791+0000","last_peered":"2023-05-16T07:38:19.539791+0000","last_clean":"2023-05-16T07:38:19.539791+0000","last_became_active":"2023-05-16T07:38:19.539557+0000","last_became_peered":"2023-05-16T07:38:19.539557+0000","last_unstale":"2023-05-16T07:38:19.539791+0000","last_undegraded":"2023-05-16T07:38:19.539791+0000","last_fullsized":"2023-05-16T07:38:19.539791+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174695+0000","last_change":"2023-05-16T07:38:19.171996+0000","last_active":"2023-05-16T07:38:19.174695+0000","last_peered":"2023-05-16T07:38:19.174695+0000","last_clean":"2023-05-16T07:38:19.174695+0000","last_became_active":"2023-05-16T07:38:19.171721+0000","last_became_peered":"2023-05-16T07:38:19.171721+0000","last_unstale":"2023-05-16T07:38:19.174695+0000","last_undegraded":"2023-05-16T07:38:19.174695+0000","last_fullsized":"2023-05-16T07:38:19.174695+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533922+0000","last_change":"2023-05-16T07:38:19.533922+0000","last_active":"2023-05-16T07:38:19.533922+0000","last_peered":"2023-05-16T07:38:19.533922+0000","last_clean":"2023-05-16T07:38:19.533922+0000","last_became_active":"2023-05-16T07:38:19.532776+0000","last_became_peered":"2023-05-16T07:38:19.532776+0000","last_unstale":"2023-05-16T07:38:19.533922+0000","last_undegraded":"2023-05-16T07:38:19.533922+0000","last_fullsized":"2023-05-16T07:38:19.533922+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173173+0000","last_change":"2023-05-16T07:38:19.164985+0000","last_active":"2023-05-16T07:38:19.173173+0000","last_peered":"2023-05-16T07:38:19.173173+0000","last_clean":"2023-05-16T07:38:19.173173+0000","last_became_active":"2023-05-16T07:38:19.162021+0000","last_became_peered":"2023-05-16T07:38:19.162021+0000","last_unstale":"2023-05-16T07:38:19.173173+0000","last_undegraded":"2023-05-16T07:38:19.173173+0000","last_fullsized":"2023-05-16T07:38:19.173173+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538004+0000","last_change":"2023-05-16T07:38:19.538004+0000","last_active":"2023-05-16T07:38:19.538004+0000","last_peered":"2023-05-16T07:38:19.538004+0000","last_clean":"2023-05-16T07:38:19.538004+0000","last_became_active":"2023-05-16T07:38:19.537686+0000","last_became_peered":"2023-05-16T07:38:19.537686+0000","last_unstale":"2023-05-16T07:38:19.538004+0000","last_undegraded":"2023-05-16T07:38:19.538004+0000","last_fullsized":"2023-05-16T07:38:19.538004+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.34","version":"170'4","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.546133+0000","last_change":"2023-05-16T07:38:19.164116+0000","last_active":"2023-05-16T07:38:19.546133+0000","last_peered":"2023-05-16T07:38:19.546133+0000","last_clean":"2023-05-16T07:38:19.546133+0000","last_became_active":"2023-05-16T07:38:19.163844+0000","last_became_peered":"2023-05-16T07:38:19.163844+0000","last_unstale":"2023-05-16T07:38:19.546133+0000","last_undegraded":"2023-05-16T07:38:19.546133+0000","last_fullsized":"2023-05-16T07:38:19.546133+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541633+0000","last_change":"2023-05-16T07:38:19.541633+0000","last_active":"2023-05-16T07:38:19.541633+0000","last_peered":"2023-05-16T07:38:19.541633+0000","last_clean":"2023-05-16T07:38:19.541633+0000","last_became_active":"2023-05-16T07:38:19.541202+0000","last_became_peered":"2023-05-16T07:38:19.541202+0000","last_unstale":"2023-05-16T07:38:19.541633+0000","last_undegraded":"2023-05-16T07:38:19.541633+0000","last_fullsized":"2023-05-16T07:38:19.541633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.33","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.527851+0000","last_change":"2023-05-16T07:38:19.165851+0000","last_active":"2023-05-16T07:38:19.527851+0000","last_peered":"2023-05-16T07:38:19.527851+0000","last_clean":"2023-05-16T07:38:19.527851+0000","last_became_active":"2023-05-16T07:38:19.165548+0000","last_became_peered":"2023-05-16T07:38:19.165548+0000","last_unstale":"2023-05-16T07:38:19.527851+0000","last_undegraded":"2023-05-16T07:38:19.527851+0000","last_fullsized":"2023-05-16T07:38:19.527851+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529948+0000","last_change":"2023-05-16T07:38:19.529948+0000","last_active":"2023-05-16T07:38:19.529948+0000","last_peered":"2023-05-16T07:38:19.529948+0000","last_clean":"2023-05-16T07:38:19.529948+0000","last_became_active":"2023-05-16T07:38:19.529659+0000","last_became_peered":"2023-05-16T07:38:19.529659+0000","last_unstale":"2023-05-16T07:38:19.529948+0000","last_undegraded":"2023-05-16T07:38:19.529948+0000","last_fullsized":"2023-05-16T07:38:19.529948+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170386+0000","last_change":"2023-05-16T07:38:19.165230+0000","last_active":"2023-05-16T07:38:19.170386+0000","last_peered":"2023-05-16T07:38:19.170386+0000","last_clean":"2023-05-16T07:38:19.170386+0000","last_became_active":"2023-05-16T07:38:19.164978+0000","last_became_peered":"2023-05-16T07:38:19.164978+0000","last_unstale":"2023-05-16T07:38:19.170386+0000","last_undegraded":"2023-05-16T07:38:19.170386+0000","last_fullsized":"2023-05-16T07:38:19.170386+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.532977+0000","last_change":"2023-05-16T07:38:19.532977+0000","last_active":"2023-05-16T07:38:19.532977+0000","last_peered":"2023-05-16T07:38:19.532977+0000","last_clean":"2023-05-16T07:38:19.532977+0000","last_became_active":"2023-05-16T07:38:19.532691+0000","last_became_peered":"2023-05-16T07:38:19.532691+0000","last_unstale":"2023-05-16T07:38:19.532977+0000","last_undegraded":"2023-05-16T07:38:19.532977+0000","last_fullsized":"2023-05-16T07:38:19.532977+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.166346+0000","last_change":"2023-05-16T07:38:19.163439+0000","last_active":"2023-05-16T07:38:19.166346+0000","last_peered":"2023-05-16T07:38:19.166346+0000","last_clean":"2023-05-16T07:38:19.166346+0000","last_became_active":"2023-05-16T07:38:19.163155+0000","last_became_peered":"2023-05-16T07:38:19.163155+0000","last_unstale":"2023-05-16T07:38:19.166346+0000","last_undegraded":"2023-05-16T07:38:19.166346+0000","last_fullsized":"2023-05-16T07:38:19.166346+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533079+0000","last_change":"2023-05-16T07:38:19.533079+0000","last_active":"2023-05-16T07:38:19.533079+0000","last_peered":"2023-05-16T07:38:19.533079+0000","last_clean":"2023-05-16T07:38:19.533079+0000","last_became_active":"2023-05-16T07:38:19.532807+0000","last_became_peered":"2023-05-16T07:38:19.532807+0000","last_unstale":"2023-05-16T07:38:19.533079+0000","last_undegraded":"2023-05-16T07:38:19.533079+0000","last_fullsized":"2023-05-16T07:38:19.533079+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.30","version":"170'4","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539002+0000","last_change":"2023-05-16T07:38:19.170157+0000","last_active":"2023-05-16T07:38:19.539002+0000","last_peered":"2023-05-16T07:38:19.539002+0000","last_clean":"2023-05-16T07:38:19.539002+0000","last_became_active":"2023-05-16T07:38:19.169889+0000","last_became_peered":"2023-05-16T07:38:19.169889+0000","last_unstale":"2023-05-16T07:38:19.539002+0000","last_undegraded":"2023-05-16T07:38:19.539002+0000","last_fullsized":"2023-05-16T07:38:19.539002+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529235+0000","last_change":"2023-05-16T07:38:19.529235+0000","last_active":"2023-05-16T07:38:19.529235+0000","last_peered":"2023-05-16T07:38:19.529235+0000","last_clean":"2023-05-16T07:38:19.529235+0000","last_became_active":"2023-05-16T07:38:19.528983+0000","last_became_peered":"2023-05-16T07:38:19.528983+0000","last_unstale":"2023-05-16T07:38:19.529235+0000","last_undegraded":"2023-05-16T07:38:19.529235+0000","last_fullsized":"2023-05-16T07:38:19.529235+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.172781+0000","last_change":"2023-05-16T07:38:19.169280+0000","last_active":"2023-05-16T07:38:19.172781+0000","last_peered":"2023-05-16T07:38:19.172781+0000","last_clean":"2023-05-16T07:38:19.172781+0000","last_became_active":"2023-05-16T07:38:19.168999+0000","last_became_peered":"2023-05-16T07:38:19.168999+0000","last_unstale":"2023-05-16T07:38:19.172781+0000","last_undegraded":"2023-05-16T07:38:19.172781+0000","last_fullsized":"2023-05-16T07:38:19.172781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544366+0000","last_change":"2023-05-16T07:38:19.544366+0000","last_active":"2023-05-16T07:38:19.544366+0000","last_peered":"2023-05-16T07:38:19.544366+0000","last_clean":"2023-05-16T07:38:19.544366+0000","last_became_active":"2023-05-16T07:38:19.544185+0000","last_became_peered":"2023-05-16T07:38:19.544185+0000","last_unstale":"2023-05-16T07:38:19.544366+0000","last_undegraded":"2023-05-16T07:38:19.544366+0000","last_fullsized":"2023-05-16T07:38:19.544366+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175054+0000","last_change":"2023-05-16T07:38:19.171517+0000","last_active":"2023-05-16T07:38:19.175054+0000","last_peered":"2023-05-16T07:38:19.175054+0000","last_clean":"2023-05-16T07:38:19.175054+0000","last_became_active":"2023-05-16T07:38:19.170701+0000","last_became_peered":"2023-05-16T07:38:19.170701+0000","last_unstale":"2023-05-16T07:38:19.175054+0000","last_undegraded":"2023-05-16T07:38:19.175054+0000","last_fullsized":"2023-05-16T07:38:19.175054+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.542546+0000","last_change":"2023-05-16T07:38:19.542546+0000","last_active":"2023-05-16T07:38:19.542546+0000","last_peered":"2023-05-16T07:38:19.542546+0000","last_clean":"2023-05-16T07:38:19.542546+0000","last_became_active":"2023-05-16T07:38:19.542182+0000","last_became_peered":"2023-05-16T07:38:19.542182+0000","last_unstale":"2023-05-16T07:38:19.542546+0000","last_undegraded":"2023-05-16T07:38:19.542546+0000","last_fullsized":"2023-05-16T07:38:19.542546+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2d","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.520263+0000","last_change":"2023-05-16T07:38:19.167516+0000","last_active":"2023-05-16T07:38:19.520263+0000","last_peered":"2023-05-16T07:38:19.520263+0000","last_clean":"2023-05-16T07:38:19.520263+0000","last_became_active":"2023-05-16T07:38:19.167249+0000","last_became_peered":"2023-05-16T07:38:19.167249+0000","last_unstale":"2023-05-16T07:38:19.520263+0000","last_undegraded":"2023-05-16T07:38:19.520263+0000","last_fullsized":"2023-05-16T07:38:19.520263+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.531180+0000","last_change":"2023-05-16T07:38:19.531180+0000","last_active":"2023-05-16T07:38:19.531180+0000","last_peered":"2023-05-16T07:38:19.531180+0000","last_clean":"2023-05-16T07:38:19.531180+0000","last_became_active":"2023-05-16T07:38:19.530903+0000","last_became_peered":"2023-05-16T07:38:19.530903+0000","last_unstale":"2023-05-16T07:38:19.531180+0000","last_undegraded":"2023-05-16T07:38:19.531180+0000","last_fullsized":"2023-05-16T07:38:19.531180+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170692+0000","last_change":"2023-05-16T07:38:19.168112+0000","last_active":"2023-05-16T07:38:19.170692+0000","last_peered":"2023-05-16T07:38:19.170692+0000","last_clean":"2023-05-16T07:38:19.170692+0000","last_became_active":"2023-05-16T07:38:19.167857+0000","last_became_peered":"2023-05-16T07:38:19.167857+0000","last_unstale":"2023-05-16T07:38:19.170692+0000","last_undegraded":"2023-05-16T07:38:19.170692+0000","last_fullsized":"2023-05-16T07:38:19.170692+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530050+0000","last_change":"2023-05-16T07:38:19.530050+0000","last_active":"2023-05-16T07:38:19.530050+0000","last_peered":"2023-05-16T07:38:19.530050+0000","last_clean":"2023-05-16T07:38:19.530050+0000","last_became_active":"2023-05-16T07:38:19.529858+0000","last_became_peered":"2023-05-16T07:38:19.529858+0000","last_unstale":"2023-05-16T07:38:19.530050+0000","last_undegraded":"2023-05-16T07:38:19.530050+0000","last_fullsized":"2023-05-16T07:38:19.530050+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.524840+0000","last_change":"2023-05-16T07:38:19.171823+0000","last_active":"2023-05-16T07:38:19.524840+0000","last_peered":"2023-05-16T07:38:19.524840+0000","last_clean":"2023-05-16T07:38:19.524840+0000","last_became_active":"2023-05-16T07:38:19.171527+0000","last_became_peered":"2023-05-16T07:38:19.171527+0000","last_unstale":"2023-05-16T07:38:19.524840+0000","last_undegraded":"2023-05-16T07:38:19.524840+0000","last_fullsized":"2023-05-16T07:38:19.524840+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537287+0000","last_change":"2023-05-16T07:38:19.537287+0000","last_active":"2023-05-16T07:38:19.537287+0000","last_peered":"2023-05-16T07:38:19.537287+0000","last_clean":"2023-05-16T07:38:19.537287+0000","last_became_active":"2023-05-16T07:38:19.537002+0000","last_became_peered":"2023-05-16T07:38:19.537002+0000","last_unstale":"2023-05-16T07:38:19.537287+0000","last_undegraded":"2023-05-16T07:38:19.537287+0000","last_fullsized":"2023-05-16T07:38:19.537287+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170016+0000","last_change":"2023-05-16T07:38:19.160298+0000","last_active":"2023-05-16T07:38:19.170016+0000","last_peered":"2023-05-16T07:38:19.170016+0000","last_clean":"2023-05-16T07:38:19.170016+0000","last_became_active":"2023-05-16T07:38:19.159649+0000","last_became_peered":"2023-05-16T07:38:19.159649+0000","last_unstale":"2023-05-16T07:38:19.170016+0000","last_undegraded":"2023-05-16T07:38:19.170016+0000","last_fullsized":"2023-05-16T07:38:19.170016+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539406+0000","last_change":"2023-05-16T07:38:19.539406+0000","last_active":"2023-05-16T07:38:19.539406+0000","last_peered":"2023-05-16T07:38:19.539406+0000","last_clean":"2023-05-16T07:38:19.539406+0000","last_became_active":"2023-05-16T07:38:19.539124+0000","last_became_peered":"2023-05-16T07:38:19.539124+0000","last_unstale":"2023-05-16T07:38:19.539406+0000","last_undegraded":"2023-05-16T07:38:19.539406+0000","last_fullsized":"2023-05-16T07:38:19.539406+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174770+0000","last_change":"2023-05-16T07:38:19.172878+0000","last_active":"2023-05-16T07:38:19.174770+0000","last_peered":"2023-05-16T07:38:19.174770+0000","last_clean":"2023-05-16T07:38:19.174770+0000","last_became_active":"2023-05-16T07:38:19.172144+0000","last_became_peered":"2023-05-16T07:38:19.172144+0000","last_unstale":"2023-05-16T07:38:19.174770+0000","last_undegraded":"2023-05-16T07:38:19.174770+0000","last_fullsized":"2023-05-16T07:38:19.174770+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537911+0000","last_change":"2023-05-16T07:38:19.537911+0000","last_active":"2023-05-16T07:38:19.537911+0000","last_peered":"2023-05-16T07:38:19.537911+0000","last_clean":"2023-05-16T07:38:19.537911+0000","last_became_active":"2023-05-16T07:38:19.537701+0000","last_became_peered":"2023-05-16T07:38:19.537701+0000","last_unstale":"2023-05-16T07:38:19.537911+0000","last_undegraded":"2023-05-16T07:38:19.537911+0000","last_fullsized":"2023-05-16T07:38:19.537911+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174291+0000","last_change":"2023-05-16T07:38:19.171498+0000","last_active":"2023-05-16T07:38:19.174291+0000","last_peered":"2023-05-16T07:38:19.174291+0000","last_clean":"2023-05-16T07:38:19.174291+0000","last_became_active":"2023-05-16T07:38:19.171216+0000","last_became_peered":"2023-05-16T07:38:19.171216+0000","last_unstale":"2023-05-16T07:38:19.174291+0000","last_undegraded":"2023-05-16T07:38:19.174291+0000","last_fullsized":"2023-05-16T07:38:19.174291+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.531209+0000","last_change":"2023-05-16T07:38:19.531209+0000","last_active":"2023-05-16T07:38:19.531209+0000","last_peered":"2023-05-16T07:38:19.531209+0000","last_clean":"2023-05-16T07:38:19.531209+0000","last_became_active":"2023-05-16T07:38:19.530796+0000","last_became_peered":"2023-05-16T07:38:19.530796+0000","last_unstale":"2023-05-16T07:38:19.531209+0000","last_undegraded":"2023-05-16T07:38:19.531209+0000","last_fullsized":"2023-05-16T07:38:19.531209+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167002+0000","last_change":"2023-05-16T07:38:19.163587+0000","last_active":"2023-05-16T07:38:19.167002+0000","last_peered":"2023-05-16T07:38:19.167002+0000","last_clean":"2023-05-16T07:38:19.167002+0000","last_became_active":"2023-05-16T07:38:19.163304+0000","last_became_peered":"2023-05-16T07:38:19.163304+0000","last_unstale":"2023-05-16T07:38:19.167002+0000","last_undegraded":"2023-05-16T07:38:19.167002+0000","last_fullsized":"2023-05-16T07:38:19.167002+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.540666+0000","last_change":"2023-05-16T07:38:19.540666+0000","last_active":"2023-05-16T07:38:19.540666+0000","last_peered":"2023-05-16T07:38:19.540666+0000","last_clean":"2023-05-16T07:38:19.540666+0000","last_became_active":"2023-05-16T07:38:19.540404+0000","last_became_peered":"2023-05-16T07:38:19.540404+0000","last_unstale":"2023-05-16T07:38:19.540666+0000","last_undegraded":"2023-05-16T07:38:19.540666+0000","last_fullsized":"2023-05-16T07:38:19.540666+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174026+0000","last_change":"2023-05-16T07:38:19.167379+0000","last_active":"2023-05-16T07:38:19.174026+0000","last_peered":"2023-05-16T07:38:19.174026+0000","last_clean":"2023-05-16T07:38:19.174026+0000","last_became_active":"2023-05-16T07:38:19.167100+0000","last_became_peered":"2023-05-16T07:38:19.167100+0000","last_unstale":"2023-05-16T07:38:19.174026+0000","last_undegraded":"2023-05-16T07:38:19.174026+0000","last_fullsized":"2023-05-16T07:38:19.174026+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538069+0000","last_change":"2023-05-16T07:38:19.538069+0000","last_active":"2023-05-16T07:38:19.538069+0000","last_peered":"2023-05-16T07:38:19.538069+0000","last_clean":"2023-05-16T07:38:19.538069+0000","last_became_active":"2023-05-16T07:38:19.537729+0000","last_became_peered":"2023-05-16T07:38:19.537729+0000","last_unstale":"2023-05-16T07:38:19.538069+0000","last_undegraded":"2023-05-16T07:38:19.538069+0000","last_fullsized":"2023-05-16T07:38:19.538069+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167636+0000","last_change":"2023-05-16T07:38:19.165776+0000","last_active":"2023-05-16T07:38:19.167636+0000","last_peered":"2023-05-16T07:38:19.167636+0000","last_clean":"2023-05-16T07:38:19.167636+0000","last_became_active":"2023-05-16T07:38:19.165529+0000","last_became_peered":"2023-05-16T07:38:19.165529+0000","last_unstale":"2023-05-16T07:38:19.167636+0000","last_undegraded":"2023-05-16T07:38:19.167636+0000","last_fullsized":"2023-05-16T07:38:19.167636+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534985+0000","last_change":"2023-05-16T07:38:19.534985+0000","last_active":"2023-05-16T07:38:19.534985+0000","last_peered":"2023-05-16T07:38:19.534985+0000","last_clean":"2023-05-16T07:38:19.534985+0000","last_became_active":"2023-05-16T07:38:19.534698+0000","last_became_peered":"2023-05-16T07:38:19.534698+0000","last_unstale":"2023-05-16T07:38:19.534985+0000","last_undegraded":"2023-05-16T07:38:19.534985+0000","last_fullsized":"2023-05-16T07:38:19.534985+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.169637+0000","last_change":"2023-05-16T07:38:19.164818+0000","last_active":"2023-05-16T07:38:19.169637+0000","last_peered":"2023-05-16T07:38:19.169637+0000","last_clean":"2023-05-16T07:38:19.169637+0000","last_became_active":"2023-05-16T07:38:19.164479+0000","last_became_peered":"2023-05-16T07:38:19.164479+0000","last_unstale":"2023-05-16T07:38:19.169637+0000","last_undegraded":"2023-05-16T07:38:19.169637+0000","last_fullsized":"2023-05-16T07:38:19.169637+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534139+0000","last_change":"2023-05-16T07:38:19.534139+0000","last_active":"2023-05-16T07:38:19.534139+0000","last_peered":"2023-05-16T07:38:19.534139+0000","last_clean":"2023-05-16T07:38:19.534139+0000","last_became_active":"2023-05-16T07:38:19.533066+0000","last_became_peered":"2023-05-16T07:38:19.533066+0000","last_unstale":"2023-05-16T07:38:19.534139+0000","last_undegraded":"2023-05-16T07:38:19.534139+0000","last_fullsized":"2023-05-16T07:38:19.534139+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173037+0000","last_change":"2023-05-16T07:38:19.162695+0000","last_active":"2023-05-16T07:38:19.173037+0000","last_peered":"2023-05-16T07:38:19.173037+0000","last_clean":"2023-05-16T07:38:19.173037+0000","last_became_active":"2023-05-16T07:38:19.162417+0000","last_became_peered":"2023-05-16T07:38:19.162417+0000","last_unstale":"2023-05-16T07:38:19.173037+0000","last_undegraded":"2023-05-16T07:38:19.173037+0000","last_fullsized":"2023-05-16T07:38:19.173037+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530706+0000","last_change":"2023-05-16T07:38:19.530706+0000","last_active":"2023-05-16T07:38:19.530706+0000","last_peered":"2023-05-16T07:38:19.530706+0000","last_clean":"2023-05-16T07:38:19.530706+0000","last_became_active":"2023-05-16T07:38:19.530450+0000","last_became_peered":"2023-05-16T07:38:19.530450+0000","last_unstale":"2023-05-16T07:38:19.530706+0000","last_undegraded":"2023-05-16T07:38:19.530706+0000","last_fullsized":"2023-05-16T07:38:19.530706+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170594+0000","last_change":"2023-05-16T07:38:19.169576+0000","last_active":"2023-05-16T07:38:19.170594+0000","last_peered":"2023-05-16T07:38:19.170594+0000","last_clean":"2023-05-16T07:38:19.170594+0000","last_became_active":"2023-05-16T07:38:19.169141+0000","last_became_peered":"2023-05-16T07:38:19.169141+0000","last_unstale":"2023-05-16T07:38:19.170594+0000","last_undegraded":"2023-05-16T07:38:19.170594+0000","last_fullsized":"2023-05-16T07:38:19.170594+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534104+0000","last_change":"2023-05-16T07:38:19.534104+0000","last_active":"2023-05-16T07:38:19.534104+0000","last_peered":"2023-05-16T07:38:19.534104+0000","last_clean":"2023-05-16T07:38:19.534104+0000","last_became_active":"2023-05-16T07:38:19.533716+0000","last_became_peered":"2023-05-16T07:38:19.533716+0000","last_unstale":"2023-05-16T07:38:19.534104+0000","last_undegraded":"2023-05-16T07:38:19.534104+0000","last_fullsized":"2023-05-16T07:38:19.534104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167999+0000","last_change":"2023-05-16T07:38:19.161601+0000","last_active":"2023-05-16T07:38:19.167999+0000","last_peered":"2023-05-16T07:38:19.167999+0000","last_clean":"2023-05-16T07:38:19.167999+0000","last_became_active":"2023-05-16T07:38:19.161110+0000","last_became_peered":"2023-05-16T07:38:19.161110+0000","last_unstale":"2023-05-16T07:38:19.167999+0000","last_undegraded":"2023-05-16T07:38:19.167999+0000","last_fullsized":"2023-05-16T07:38:19.167999+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537801+0000","last_change":"2023-05-16T07:38:19.537801+0000","last_active":"2023-05-16T07:38:19.537801+0000","last_peered":"2023-05-16T07:38:19.537801+0000","last_clean":"2023-05-16T07:38:19.537801+0000","last_became_active":"2023-05-16T07:38:19.537487+0000","last_became_peered":"2023-05-16T07:38:19.537487+0000","last_unstale":"2023-05-16T07:38:19.537801+0000","last_undegraded":"2023-05-16T07:38:19.537801+0000","last_fullsized":"2023-05-16T07:38:19.537801+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.166757+0000","last_change":"2023-05-16T07:38:19.162979+0000","last_active":"2023-05-16T07:38:19.166757+0000","last_peered":"2023-05-16T07:38:19.166757+0000","last_clean":"2023-05-16T07:38:19.166757+0000","last_became_active":"2023-05-16T07:38:19.162683+0000","last_became_peered":"2023-05-16T07:38:19.162683+0000","last_unstale":"2023-05-16T07:38:19.166757+0000","last_undegraded":"2023-05-16T07:38:19.166757+0000","last_fullsized":"2023-05-16T07:38:19.166757+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536047+0000","last_change":"2023-05-16T07:38:19.536047+0000","last_active":"2023-05-16T07:38:19.536047+0000","last_peered":"2023-05-16T07:38:19.536047+0000","last_clean":"2023-05-16T07:38:19.536047+0000","last_became_active":"2023-05-16T07:38:19.534937+0000","last_became_peered":"2023-05-16T07:38:19.534937+0000","last_unstale":"2023-05-16T07:38:19.536047+0000","last_undegraded":"2023-05-16T07:38:19.536047+0000","last_fullsized":"2023-05-16T07:38:19.536047+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.528215+0000","last_change":"2023-05-16T07:38:19.152409+0000","last_active":"2023-05-16T07:38:19.528215+0000","last_peered":"2023-05-16T07:38:19.528215+0000","last_clean":"2023-05-16T07:38:19.528215+0000","last_became_active":"2023-05-16T07:38:19.151297+0000","last_became_peered":"2023-05-16T07:38:19.151297+0000","last_unstale":"2023-05-16T07:38:19.528215+0000","last_undegraded":"2023-05-16T07:38:19.528215+0000","last_fullsized":"2023-05-16T07:38:19.528215+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.529767+0000","last_change":"2023-05-16T07:38:19.529767+0000","last_active":"2023-05-16T07:38:19.529767+0000","last_peered":"2023-05-16T07:38:19.529767+0000","last_clean":"2023-05-16T07:38:19.529767+0000","last_became_active":"2023-05-16T07:38:19.529496+0000","last_became_peered":"2023-05-16T07:38:19.529496+0000","last_unstale":"2023-05-16T07:38:19.529767+0000","last_undegraded":"2023-05-16T07:38:19.529767+0000","last_fullsized":"2023-05-16T07:38:19.529767+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.524937+0000","last_change":"2023-05-16T07:38:19.163250+0000","last_active":"2023-05-16T07:38:19.524937+0000","last_peered":"2023-05-16T07:38:19.524937+0000","last_clean":"2023-05-16T07:38:19.524937+0000","last_became_active":"2023-05-16T07:38:19.162947+0000","last_became_peered":"2023-05-16T07:38:19.162947+0000","last_unstale":"2023-05-16T07:38:19.524937+0000","last_undegraded":"2023-05-16T07:38:19.524937+0000","last_fullsized":"2023-05-16T07:38:19.524937+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533912+0000","last_change":"2023-05-16T07:38:19.533912+0000","last_active":"2023-05-16T07:38:19.533912+0000","last_peered":"2023-05-16T07:38:19.533912+0000","last_clean":"2023-05-16T07:38:19.533912+0000","last_became_active":"2023-05-16T07:38:19.532383+0000","last_became_peered":"2023-05-16T07:38:19.532383+0000","last_unstale":"2023-05-16T07:38:19.533912+0000","last_undegraded":"2023-05-16T07:38:19.533912+0000","last_fullsized":"2023-05-16T07:38:19.533912+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.546953+0000","last_change":"2023-05-16T07:38:19.165008+0000","last_active":"2023-05-16T07:38:19.546953+0000","last_peered":"2023-05-16T07:38:19.546953+0000","last_clean":"2023-05-16T07:38:19.546953+0000","last_became_active":"2023-05-16T07:38:19.164735+0000","last_became_peered":"2023-05-16T07:38:19.164735+0000","last_unstale":"2023-05-16T07:38:19.546953+0000","last_undegraded":"2023-05-16T07:38:19.546953+0000","last_fullsized":"2023-05-16T07:38:19.546953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.543367+0000","last_change":"2023-05-16T07:38:19.543367+0000","last_active":"2023-05-16T07:38:19.543367+0000","last_peered":"2023-05-16T07:38:19.543367+0000","last_clean":"2023-05-16T07:38:19.543367+0000","last_became_active":"2023-05-16T07:38:19.543102+0000","last_became_peered":"2023-05-16T07:38:19.543102+0000","last_unstale":"2023-05-16T07:38:19.543367+0000","last_undegraded":"2023-05-16T07:38:19.543367+0000","last_fullsized":"2023-05-16T07:38:19.543367+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.178766+0000","last_change":"2023-05-16T07:38:19.175866+0000","last_active":"2023-05-16T07:38:19.178766+0000","last_peered":"2023-05-16T07:38:19.178766+0000","last_clean":"2023-05-16T07:38:19.178766+0000","last_became_active":"2023-05-16T07:38:19.175575+0000","last_became_peered":"2023-05-16T07:38:19.175575+0000","last_unstale":"2023-05-16T07:38:19.178766+0000","last_undegraded":"2023-05-16T07:38:19.178766+0000","last_fullsized":"2023-05-16T07:38:19.178766+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530070+0000","last_change":"2023-05-16T07:38:19.530070+0000","last_active":"2023-05-16T07:38:19.530070+0000","last_peered":"2023-05-16T07:38:19.530070+0000","last_clean":"2023-05-16T07:38:19.530070+0000","last_became_active":"2023-05-16T07:38:19.529855+0000","last_became_peered":"2023-05-16T07:38:19.529855+0000","last_unstale":"2023-05-16T07:38:19.530070+0000","last_undegraded":"2023-05-16T07:38:19.530070+0000","last_fullsized":"2023-05-16T07:38:19.530070+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167107+0000","last_change":"2023-05-16T07:38:19.162618+0000","last_active":"2023-05-16T07:38:19.167107+0000","last_peered":"2023-05-16T07:38:19.167107+0000","last_clean":"2023-05-16T07:38:19.167107+0000","last_became_active":"2023-05-16T07:38:19.162061+0000","last_became_peered":"2023-05-16T07:38:19.162061+0000","last_unstale":"2023-05-16T07:38:19.167107+0000","last_undegraded":"2023-05-16T07:38:19.167107+0000","last_fullsized":"2023-05-16T07:38:19.167107+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.530811+0000","last_change":"2023-05-16T07:38:19.530811+0000","last_active":"2023-05-16T07:38:19.530811+0000","last_peered":"2023-05-16T07:38:19.530811+0000","last_clean":"2023-05-16T07:38:19.530811+0000","last_became_active":"2023-05-16T07:38:19.530506+0000","last_became_peered":"2023-05-16T07:38:19.530506+0000","last_unstale":"2023-05-16T07:38:19.530811+0000","last_undegraded":"2023-05-16T07:38:19.530811+0000","last_fullsized":"2023-05-16T07:38:19.530811+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168007+0000","last_change":"2023-05-16T07:38:19.161158+0000","last_active":"2023-05-16T07:38:19.168007+0000","last_peered":"2023-05-16T07:38:19.168007+0000","last_clean":"2023-05-16T07:38:19.168007+0000","last_became_active":"2023-05-16T07:38:19.160661+0000","last_became_peered":"2023-05-16T07:38:19.160661+0000","last_unstale":"2023-05-16T07:38:19.168007+0000","last_undegraded":"2023-05-16T07:38:19.168007+0000","last_fullsized":"2023-05-16T07:38:19.168007+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.528209+0000","last_change":"2023-05-16T07:38:19.528209+0000","last_active":"2023-05-16T07:38:19.528209+0000","last_peered":"2023-05-16T07:38:19.528209+0000","last_clean":"2023-05-16T07:38:19.528209+0000","last_became_active":"2023-05-16T07:38:19.527939+0000","last_became_peered":"2023-05-16T07:38:19.527939+0000","last_unstale":"2023-05-16T07:38:19.528209+0000","last_undegraded":"2023-05-16T07:38:19.528209+0000","last_fullsized":"2023-05-16T07:38:19.528209+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170201+0000","last_change":"2023-05-16T07:38:19.168860+0000","last_active":"2023-05-16T07:38:19.170201+0000","last_peered":"2023-05-16T07:38:19.170201+0000","last_clean":"2023-05-16T07:38:19.170201+0000","last_became_active":"2023-05-16T07:38:19.168593+0000","last_became_peered":"2023-05-16T07:38:19.168593+0000","last_unstale":"2023-05-16T07:38:19.170201+0000","last_undegraded":"2023-05-16T07:38:19.170201+0000","last_fullsized":"2023-05-16T07:38:19.170201+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544375+0000","last_change":"2023-05-16T07:38:19.544375+0000","last_active":"2023-05-16T07:38:19.544375+0000","last_peered":"2023-05-16T07:38:19.544375+0000","last_clean":"2023-05-16T07:38:19.544375+0000","last_became_active":"2023-05-16T07:38:19.544092+0000","last_became_peered":"2023-05-16T07:38:19.544092+0000","last_unstale":"2023-05-16T07:38:19.544375+0000","last_undegraded":"2023-05-16T07:38:19.544375+0000","last_fullsized":"2023-05-16T07:38:19.544375+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167611+0000","last_change":"2023-05-16T07:38:19.157775+0000","last_active":"2023-05-16T07:38:19.167611+0000","last_peered":"2023-05-16T07:38:19.167611+0000","last_clean":"2023-05-16T07:38:19.167611+0000","last_became_active":"2023-05-16T07:38:19.157496+0000","last_became_peered":"2023-05-16T07:38:19.157496+0000","last_unstale":"2023-05-16T07:38:19.167611+0000","last_undegraded":"2023-05-16T07:38:19.167611+0000","last_fullsized":"2023-05-16T07:38:19.167611+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.538641+0000","last_change":"2023-05-16T07:38:19.538641+0000","last_active":"2023-05-16T07:38:19.538641+0000","last_peered":"2023-05-16T07:38:19.538641+0000","last_clean":"2023-05-16T07:38:19.538641+0000","last_became_active":"2023-05-16T07:38:19.538365+0000","last_became_peered":"2023-05-16T07:38:19.538365+0000","last_unstale":"2023-05-16T07:38:19.538641+0000","last_undegraded":"2023-05-16T07:38:19.538641+0000","last_fullsized":"2023-05-16T07:38:19.538641+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.523167+0000","last_change":"2023-05-16T07:38:19.165083+0000","last_active":"2023-05-16T07:38:19.523167+0000","last_peered":"2023-05-16T07:38:19.523167+0000","last_clean":"2023-05-16T07:38:19.523167+0000","last_became_active":"2023-05-16T07:38:19.164486+0000","last_became_peered":"2023-05-16T07:38:19.164486+0000","last_unstale":"2023-05-16T07:38:19.523167+0000","last_undegraded":"2023-05-16T07:38:19.523167+0000","last_fullsized":"2023-05-16T07:38:19.523167+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544385+0000","last_change":"2023-05-16T07:38:19.544385+0000","last_active":"2023-05-16T07:38:19.544385+0000","last_peered":"2023-05-16T07:38:19.544385+0000","last_clean":"2023-05-16T07:38:19.544385+0000","last_became_active":"2023-05-16T07:38:19.544070+0000","last_became_peered":"2023-05-16T07:38:19.544070+0000","last_unstale":"2023-05-16T07:38:19.544385+0000","last_undegraded":"2023-05-16T07:38:19.544385+0000","last_fullsized":"2023-05-16T07:38:19.544385+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.540376+0000","last_change":"2023-05-16T07:38:19.161915+0000","last_active":"2023-05-16T07:38:19.540376+0000","last_peered":"2023-05-16T07:38:19.540376+0000","last_clean":"2023-05-16T07:38:19.540376+0000","last_became_active":"2023-05-16T07:38:19.158881+0000","last_became_peered":"2023-05-16T07:38:19.158881+0000","last_unstale":"2023-05-16T07:38:19.540376+0000","last_undegraded":"2023-05-16T07:38:19.540376+0000","last_fullsized":"2023-05-16T07:38:19.540376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534991+0000","last_change":"2023-05-16T07:38:19.534991+0000","last_active":"2023-05-16T07:38:19.534991+0000","last_peered":"2023-05-16T07:38:19.534991+0000","last_clean":"2023-05-16T07:38:19.534991+0000","last_became_active":"2023-05-16T07:38:19.534657+0000","last_became_peered":"2023-05-16T07:38:19.534657+0000","last_unstale":"2023-05-16T07:38:19.534991+0000","last_undegraded":"2023-05-16T07:38:19.534991+0000","last_fullsized":"2023-05-16T07:38:19.534991+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.173984+0000","last_change":"2023-05-16T07:38:19.166573+0000","last_active":"2023-05-16T07:38:19.173984+0000","last_peered":"2023-05-16T07:38:19.173984+0000","last_clean":"2023-05-16T07:38:19.173984+0000","last_became_active":"2023-05-16T07:38:19.166316+0000","last_became_peered":"2023-05-16T07:38:19.166316+0000","last_unstale":"2023-05-16T07:38:19.173984+0000","last_undegraded":"2023-05-16T07:38:19.173984+0000","last_fullsized":"2023-05-16T07:38:19.173984+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541725+0000","last_change":"2023-05-16T07:38:19.541725+0000","last_active":"2023-05-16T07:38:19.541725+0000","last_peered":"2023-05-16T07:38:19.541725+0000","last_clean":"2023-05-16T07:38:19.541725+0000","last_became_active":"2023-05-16T07:38:19.539585+0000","last_became_peered":"2023-05-16T07:38:19.539585+0000","last_unstale":"2023-05-16T07:38:19.541725+0000","last_undegraded":"2023-05-16T07:38:19.541725+0000","last_fullsized":"2023-05-16T07:38:19.541725+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174995+0000","last_change":"2023-05-16T07:38:19.171306+0000","last_active":"2023-05-16T07:38:19.174995+0000","last_peered":"2023-05-16T07:38:19.174995+0000","last_clean":"2023-05-16T07:38:19.174995+0000","last_became_active":"2023-05-16T07:38:19.171025+0000","last_became_peered":"2023-05-16T07:38:19.171025+0000","last_unstale":"2023-05-16T07:38:19.174995+0000","last_undegraded":"2023-05-16T07:38:19.174995+0000","last_fullsized":"2023-05-16T07:38:19.174995+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544423+0000","last_change":"2023-05-16T07:38:19.544423+0000","last_active":"2023-05-16T07:38:19.544423+0000","last_peered":"2023-05-16T07:38:19.544423+0000","last_clean":"2023-05-16T07:38:19.544423+0000","last_became_active":"2023-05-16T07:38:19.544061+0000","last_became_peered":"2023-05-16T07:38:19.544061+0000","last_unstale":"2023-05-16T07:38:19.544423+0000","last_undegraded":"2023-05-16T07:38:19.544423+0000","last_fullsized":"2023-05-16T07:38:19.544423+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.523885+0000","last_change":"2023-05-16T07:38:19.172332+0000","last_active":"2023-05-16T07:38:19.523885+0000","last_peered":"2023-05-16T07:38:19.523885+0000","last_clean":"2023-05-16T07:38:19.523885+0000","last_became_active":"2023-05-16T07:38:19.172170+0000","last_became_peered":"2023-05-16T07:38:19.172170+0000","last_unstale":"2023-05-16T07:38:19.523885+0000","last_undegraded":"2023-05-16T07:38:19.523885+0000","last_fullsized":"2023-05-16T07:38:19.523885+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.535734+0000","last_change":"2023-05-16T07:38:19.535734+0000","last_active":"2023-05-16T07:38:19.535734+0000","last_peered":"2023-05-16T07:38:19.535734+0000","last_clean":"2023-05-16T07:38:19.535734+0000","last_became_active":"2023-05-16T07:38:19.535471+0000","last_became_peered":"2023-05-16T07:38:19.535471+0000","last_unstale":"2023-05-16T07:38:19.535734+0000","last_undegraded":"2023-05-16T07:38:19.535734+0000","last_fullsized":"2023-05-16T07:38:19.535734+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.170569+0000","last_change":"2023-05-16T07:38:19.169407+0000","last_active":"2023-05-16T07:38:19.170569+0000","last_peered":"2023-05-16T07:38:19.170569+0000","last_clean":"2023-05-16T07:38:19.170569+0000","last_became_active":"2023-05-16T07:38:19.166523+0000","last_became_peered":"2023-05-16T07:38:19.166523+0000","last_unstale":"2023-05-16T07:38:19.170569+0000","last_undegraded":"2023-05-16T07:38:19.170569+0000","last_fullsized":"2023-05-16T07:38:19.170569+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537079+0000","last_change":"2023-05-16T07:38:19.537079+0000","last_active":"2023-05-16T07:38:19.537079+0000","last_peered":"2023-05-16T07:38:19.537079+0000","last_clean":"2023-05-16T07:38:19.537079+0000","last_became_active":"2023-05-16T07:38:19.535678+0000","last_became_peered":"2023-05-16T07:38:19.535678+0000","last_unstale":"2023-05-16T07:38:19.537079+0000","last_undegraded":"2023-05-16T07:38:19.537079+0000","last_fullsized":"2023-05-16T07:38:19.537079+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175781+0000","last_change":"2023-05-16T07:38:19.171657+0000","last_active":"2023-05-16T07:38:19.175781+0000","last_peered":"2023-05-16T07:38:19.175781+0000","last_clean":"2023-05-16T07:38:19.175781+0000","last_became_active":"2023-05-16T07:38:19.171371+0000","last_became_peered":"2023-05-16T07:38:19.171371+0000","last_unstale":"2023-05-16T07:38:19.175781+0000","last_undegraded":"2023-05-16T07:38:19.175781+0000","last_fullsized":"2023-05-16T07:38:19.175781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533872+0000","last_change":"2023-05-16T07:38:19.533872+0000","last_active":"2023-05-16T07:38:19.533872+0000","last_peered":"2023-05-16T07:38:19.533872+0000","last_clean":"2023-05-16T07:38:19.533872+0000","last_became_active":"2023-05-16T07:38:19.533622+0000","last_became_peered":"2023-05-16T07:38:19.533622+0000","last_unstale":"2023-05-16T07:38:19.533872+0000","last_undegraded":"2023-05-16T07:38:19.533872+0000","last_fullsized":"2023-05-16T07:38:19.533872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174282+0000","last_change":"2023-05-16T07:38:19.171432+0000","last_active":"2023-05-16T07:38:19.174282+0000","last_peered":"2023-05-16T07:38:19.174282+0000","last_clean":"2023-05-16T07:38:19.174282+0000","last_became_active":"2023-05-16T07:38:19.171247+0000","last_became_peered":"2023-05-16T07:38:19.171247+0000","last_unstale":"2023-05-16T07:38:19.174282+0000","last_undegraded":"2023-05-16T07:38:19.174282+0000","last_fullsized":"2023-05-16T07:38:19.174282+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541806+0000","last_change":"2023-05-16T07:38:19.541806+0000","last_active":"2023-05-16T07:38:19.541806+0000","last_peered":"2023-05-16T07:38:19.541806+0000","last_clean":"2023-05-16T07:38:19.541806+0000","last_became_active":"2023-05-16T07:38:19.541536+0000","last_became_peered":"2023-05-16T07:38:19.541536+0000","last_unstale":"2023-05-16T07:38:19.541806+0000","last_undegraded":"2023-05-16T07:38:19.541806+0000","last_fullsized":"2023-05-16T07:38:19.541806+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539976+0000","last_change":"2023-05-16T07:38:19.167397+0000","last_active":"2023-05-16T07:38:19.539976+0000","last_peered":"2023-05-16T07:38:19.539976+0000","last_clean":"2023-05-16T07:38:19.539976+0000","last_became_active":"2023-05-16T07:38:19.167126+0000","last_became_peered":"2023-05-16T07:38:19.167126+0000","last_unstale":"2023-05-16T07:38:19.539976+0000","last_undegraded":"2023-05-16T07:38:19.539976+0000","last_fullsized":"2023-05-16T07:38:19.539976+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533710+0000","last_change":"2023-05-16T07:38:19.533710+0000","last_active":"2023-05-16T07:38:19.533710+0000","last_peered":"2023-05-16T07:38:19.533710+0000","last_clean":"2023-05-16T07:38:19.533710+0000","last_became_active":"2023-05-16T07:38:19.533420+0000","last_became_peered":"2023-05-16T07:38:19.533420+0000","last_unstale":"2023-05-16T07:38:19.533710+0000","last_undegraded":"2023-05-16T07:38:19.533710+0000","last_fullsized":"2023-05-16T07:38:19.533710+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174104+0000","last_change":"2023-05-16T07:38:19.172674+0000","last_active":"2023-05-16T07:38:19.174104+0000","last_peered":"2023-05-16T07:38:19.174104+0000","last_clean":"2023-05-16T07:38:19.174104+0000","last_became_active":"2023-05-16T07:38:19.172017+0000","last_became_peered":"2023-05-16T07:38:19.172017+0000","last_unstale":"2023-05-16T07:38:19.174104+0000","last_undegraded":"2023-05-16T07:38:19.174104+0000","last_fullsized":"2023-05-16T07:38:19.174104+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.544441+0000","last_change":"2023-05-16T07:38:19.544441+0000","last_active":"2023-05-16T07:38:19.544441+0000","last_peered":"2023-05-16T07:38:19.544441+0000","last_clean":"2023-05-16T07:38:19.544441+0000","last_became_active":"2023-05-16T07:38:19.544174+0000","last_became_peered":"2023-05-16T07:38:19.544174+0000","last_unstale":"2023-05-16T07:38:19.544441+0000","last_undegraded":"2023-05-16T07:38:19.544441+0000","last_fullsized":"2023-05-16T07:38:19.544441+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.176422+0000","last_change":"2023-05-16T07:38:19.175532+0000","last_active":"2023-05-16T07:38:19.176422+0000","last_peered":"2023-05-16T07:38:19.176422+0000","last_clean":"2023-05-16T07:38:19.176422+0000","last_became_active":"2023-05-16T07:38:19.175252+0000","last_became_peered":"2023-05-16T07:38:19.175252+0000","last_unstale":"2023-05-16T07:38:19.176422+0000","last_undegraded":"2023-05-16T07:38:19.176422+0000","last_fullsized":"2023-05-16T07:38:19.176422+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.533936+0000","last_change":"2023-05-16T07:38:19.533936+0000","last_active":"2023-05-16T07:38:19.533936+0000","last_peered":"2023-05-16T07:38:19.533936+0000","last_clean":"2023-05-16T07:38:19.533936+0000","last_became_active":"2023-05-16T07:38:19.533674+0000","last_became_peered":"2023-05-16T07:38:19.533674+0000","last_unstale":"2023-05-16T07:38:19.533936+0000","last_undegraded":"2023-05-16T07:38:19.533936+0000","last_fullsized":"2023-05-16T07:38:19.533936+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.168010+0000","last_change":"2023-05-16T07:38:19.160936+0000","last_active":"2023-05-16T07:38:19.168010+0000","last_peered":"2023-05-16T07:38:19.168010+0000","last_clean":"2023-05-16T07:38:19.168010+0000","last_became_active":"2023-05-16T07:38:19.160578+0000","last_became_peered":"2023-05-16T07:38:19.160578+0000","last_unstale":"2023-05-16T07:38:19.168010+0000","last_undegraded":"2023-05-16T07:38:19.168010+0000","last_fullsized":"2023-05-16T07:38:19.168010+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536663+0000","last_change":"2023-05-16T07:38:19.536663+0000","last_active":"2023-05-16T07:38:19.536663+0000","last_peered":"2023-05-16T07:38:19.536663+0000","last_clean":"2023-05-16T07:38:19.536663+0000","last_became_active":"2023-05-16T07:38:19.535219+0000","last_became_peered":"2023-05-16T07:38:19.535219+0000","last_unstale":"2023-05-16T07:38:19.536663+0000","last_undegraded":"2023-05-16T07:38:19.536663+0000","last_fullsized":"2023-05-16T07:38:19.536663+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.167286+0000","last_change":"2023-05-16T07:38:19.162725+0000","last_active":"2023-05-16T07:38:19.167286+0000","last_peered":"2023-05-16T07:38:19.167286+0000","last_clean":"2023-05-16T07:38:19.167286+0000","last_became_active":"2023-05-16T07:38:19.162449+0000","last_became_peered":"2023-05-16T07:38:19.162449+0000","last_unstale":"2023-05-16T07:38:19.167286+0000","last_undegraded":"2023-05-16T07:38:19.167286+0000","last_fullsized":"2023-05-16T07:38:19.167286+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.531948+0000","last_change":"2023-05-16T07:38:19.531948+0000","last_active":"2023-05-16T07:38:19.531948+0000","last_peered":"2023-05-16T07:38:19.531948+0000","last_clean":"2023-05-16T07:38:19.531948+0000","last_became_active":"2023-05-16T07:38:19.531663+0000","last_became_peered":"2023-05-16T07:38:19.531663+0000","last_unstale":"2023-05-16T07:38:19.531948+0000","last_undegraded":"2023-05-16T07:38:19.531948+0000","last_fullsized":"2023-05-16T07:38:19.531948+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.176254+0000","last_change":"2023-05-16T07:38:19.171302+0000","last_active":"2023-05-16T07:38:19.176254+0000","last_peered":"2023-05-16T07:38:19.176254+0000","last_clean":"2023-05-16T07:38:19.176254+0000","last_became_active":"2023-05-16T07:38:19.171017+0000","last_became_peered":"2023-05-16T07:38:19.171017+0000","last_unstale":"2023-05-16T07:38:19.176254+0000","last_undegraded":"2023-05-16T07:38:19.176254+0000","last_fullsized":"2023-05-16T07:38:19.176254+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536567+0000","last_change":"2023-05-16T07:38:19.536567+0000","last_active":"2023-05-16T07:38:19.536567+0000","last_peered":"2023-05-16T07:38:19.536567+0000","last_clean":"2023-05-16T07:38:19.536567+0000","last_became_active":"2023-05-16T07:38:19.536299+0000","last_became_peered":"2023-05-16T07:38:19.536299+0000","last_unstale":"2023-05-16T07:38:19.536567+0000","last_undegraded":"2023-05-16T07:38:19.536567+0000","last_fullsized":"2023-05-16T07:38:19.536567+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.169744+0000","last_change":"2023-05-16T07:38:19.167585+0000","last_active":"2023-05-16T07:38:19.169744+0000","last_peered":"2023-05-16T07:38:19.169744+0000","last_clean":"2023-05-16T07:38:19.169744+0000","last_became_active":"2023-05-16T07:38:19.167197+0000","last_became_peered":"2023-05-16T07:38:19.167197+0000","last_unstale":"2023-05-16T07:38:19.169744+0000","last_undegraded":"2023-05-16T07:38:19.169744+0000","last_fullsized":"2023-05-16T07:38:19.169744+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534798+0000","last_change":"2023-05-16T07:38:19.534798+0000","last_active":"2023-05-16T07:38:19.534798+0000","last_peered":"2023-05-16T07:38:19.534798+0000","last_clean":"2023-05-16T07:38:19.534798+0000","last_became_active":"2023-05-16T07:38:19.534521+0000","last_became_peered":"2023-05-16T07:38:19.534521+0000","last_unstale":"2023-05-16T07:38:19.534798+0000","last_undegraded":"2023-05-16T07:38:19.534798+0000","last_fullsized":"2023-05-16T07:38:19.534798+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174272+0000","last_change":"2023-05-16T07:38:19.170260+0000","last_active":"2023-05-16T07:38:19.174272+0000","last_peered":"2023-05-16T07:38:19.174272+0000","last_clean":"2023-05-16T07:38:19.174272+0000","last_became_active":"2023-05-16T07:38:19.169979+0000","last_became_peered":"2023-05-16T07:38:19.169979+0000","last_unstale":"2023-05-16T07:38:19.174272+0000","last_undegraded":"2023-05-16T07:38:19.174272+0000","last_fullsized":"2023-05-16T07:38:19.174272+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.525635+0000","last_change":"2023-05-16T07:38:19.525635+0000","last_active":"2023-05-16T07:38:19.525635+0000","last_peered":"2023-05-16T07:38:19.525635+0000","last_clean":"2023-05-16T07:38:19.525635+0000","last_became_active":"2023-05-16T07:38:19.525279+0000","last_became_peered":"2023-05-16T07:38:19.525279+0000","last_unstale":"2023-05-16T07:38:19.525635+0000","last_undegraded":"2023-05-16T07:38:19.525635+0000","last_fullsized":"2023-05-16T07:38:19.525635+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"170'1","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.528621+0000","last_change":"2023-05-16T07:38:19.163935+0000","last_active":"2023-05-16T07:38:19.528621+0000","last_peered":"2023-05-16T07:38:19.528621+0000","last_clean":"2023-05-16T07:38:19.528621+0000","last_became_active":"2023-05-16T07:38:19.163457+0000","last_became_peered":"2023-05-16T07:38:19.163457+0000","last_unstale":"2023-05-16T07:38:19.528621+0000","last_undegraded":"2023-05-16T07:38:19.528621+0000","last_fullsized":"2023-05-16T07:38:19.528621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.154048+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:38:19.154048+0000","last_peered":"2023-05-16T07:38:19.154048+0000","last_clean":"2023-05-16T07:38:19.154048+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:19.154048+0000","last_undegraded":"2023-05-16T07:38:19.154048+0000","last_fullsized":"2023-05-16T07:38:19.154048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.542456+0000","last_change":"2023-05-16T07:38:19.542456+0000","last_active":"2023-05-16T07:38:19.542456+0000","last_peered":"2023-05-16T07:38:19.542456+0000","last_clean":"2023-05-16T07:38:19.542456+0000","last_became_active":"2023-05-16T07:38:19.542159+0000","last_became_peered":"2023-05-16T07:38:19.542159+0000","last_unstale":"2023-05-16T07:38:19.542456+0000","last_undegraded":"2023-05-16T07:38:19.542456+0000","last_fullsized":"2023-05-16T07:38:19.542456+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"170'3","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.537203+0000","last_change":"2023-05-16T07:38:19.165203+0000","last_active":"2023-05-16T07:38:19.537203+0000","last_peered":"2023-05-16T07:38:19.537203+0000","last_clean":"2023-05-16T07:38:19.537203+0000","last_became_active":"2023-05-16T07:38:19.164447+0000","last_became_peered":"2023-05-16T07:38:19.164447+0000","last_unstale":"2023-05-16T07:38:19.537203+0000","last_undegraded":"2023-05-16T07:38:19.537203+0000","last_fullsized":"2023-05-16T07:38:19.537203+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.156185+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:38:19.156185+0000","last_peered":"2023-05-16T07:38:19.156185+0000","last_clean":"2023-05-16T07:38:19.156185+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:19.156185+0000","last_undegraded":"2023-05-16T07:38:19.156185+0000","last_fullsized":"2023-05-16T07:38:19.156185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.539802+0000","last_change":"2023-05-16T07:38:19.539802+0000","last_active":"2023-05-16T07:38:19.539802+0000","last_peered":"2023-05-16T07:38:19.539802+0000","last_clean":"2023-05-16T07:38:19.539802+0000","last_became_active":"2023-05-16T07:38:19.539584+0000","last_became_peered":"2023-05-16T07:38:19.539584+0000","last_unstale":"2023-05-16T07:38:19.539802+0000","last_undegraded":"2023-05-16T07:38:19.539802+0000","last_fullsized":"2023-05-16T07:38:19.539802+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.169713+0000","last_change":"2023-05-16T07:38:19.167975+0000","last_active":"2023-05-16T07:38:19.169713+0000","last_peered":"2023-05-16T07:38:19.169713+0000","last_clean":"2023-05-16T07:38:19.169713+0000","last_became_active":"2023-05-16T07:38:19.167683+0000","last_became_peered":"2023-05-16T07:38:19.167683+0000","last_unstale":"2023-05-16T07:38:19.169713+0000","last_undegraded":"2023-05-16T07:38:19.169713+0000","last_fullsized":"2023-05-16T07:38:19.169713+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.163756+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:38:19.163756+0000","last_peered":"2023-05-16T07:38:19.163756+0000","last_clean":"2023-05-16T07:38:19.163756+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:19.163756+0000","last_undegraded":"2023-05-16T07:38:19.163756+0000","last_fullsized":"2023-05-16T07:38:19.163756+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536285+0000","last_change":"2023-05-16T07:38:19.536285+0000","last_active":"2023-05-16T07:38:19.536285+0000","last_peered":"2023-05-16T07:38:19.536285+0000","last_clean":"2023-05-16T07:38:19.536285+0000","last_became_active":"2023-05-16T07:38:19.535906+0000","last_became_peered":"2023-05-16T07:38:19.535906+0000","last_unstale":"2023-05-16T07:38:19.536285+0000","last_undegraded":"2023-05-16T07:38:19.536285+0000","last_fullsized":"2023-05-16T07:38:19.536285+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"170'2","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.541009+0000","last_change":"2023-05-16T07:38:19.172800+0000","last_active":"2023-05-16T07:38:19.541009+0000","last_peered":"2023-05-16T07:38:19.541009+0000","last_clean":"2023-05-16T07:38:19.541009+0000","last_became_active":"2023-05-16T07:38:19.172550+0000","last_became_peered":"2023-05-16T07:38:19.172550+0000","last_unstale":"2023-05-16T07:38:19.541009+0000","last_undegraded":"2023-05-16T07:38:19.541009+0000","last_fullsized":"2023-05-16T07:38:19.541009+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.156307+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:38:19.156307+0000","last_peered":"2023-05-16T07:38:19.156307+0000","last_clean":"2023-05-16T07:38:19.156307+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:19.156307+0000","last_undegraded":"2023-05-16T07:38:19.156307+0000","last_fullsized":"2023-05-16T07:38:19.156307+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":135,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.155152+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:38:19.155152+0000","last_peered":"2023-05-16T07:38:19.155152+0000","last_clean":"2023-05-16T07:38:19.155152+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:19.155152+0000","last_undegraded":"2023-05-16T07:38:19.155152+0000","last_fullsized":"2023-05-16T07:38:19.155152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.536747+0000","last_change":"2023-05-16T07:38:19.536747+0000","last_active":"2023-05-16T07:38:19.536747+0000","last_peered":"2023-05-16T07:38:19.536747+0000","last_clean":"2023-05-16T07:38:19.536747+0000","last_became_active":"2023-05-16T07:38:19.533150+0000","last_became_peered":"2023-05-16T07:38:19.533150+0000","last_unstale":"2023-05-16T07:38:19.536747+0000","last_undegraded":"2023-05-16T07:38:19.536747+0000","last_fullsized":"2023-05-16T07:38:19.536747+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.174143+0000","last_change":"2023-05-16T07:38:19.171534+0000","last_active":"2023-05-16T07:38:19.174143+0000","last_peered":"2023-05-16T07:38:19.174143+0000","last_clean":"2023-05-16T07:38:19.174143+0000","last_became_active":"2023-05-16T07:38:19.171245+0000","last_became_peered":"2023-05-16T07:38:19.171245+0000","last_unstale":"2023-05-16T07:38:19.174143+0000","last_undegraded":"2023-05-16T07:38:19.174143+0000","last_fullsized":"2023-05-16T07:38:19.174143+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.155239+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:38:19.155239+0000","last_peered":"2023-05-16T07:38:19.155239+0000","last_clean":"2023-05-16T07:38:19.155239+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:19.155239+0000","last_undegraded":"2023-05-16T07:38:19.155239+0000","last_fullsized":"2023-05-16T07:38:19.155239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.534724+0000","last_change":"2023-05-16T07:38:19.534724+0000","last_active":"2023-05-16T07:38:19.534724+0000","last_peered":"2023-05-16T07:38:19.534724+0000","last_clean":"2023-05-16T07:38:19.534724+0000","last_became_active":"2023-05-16T07:38:19.534443+0000","last_became_peered":"2023-05-16T07:38:19.534443+0000","last_unstale":"2023-05-16T07:38:19.534724+0000","last_undegraded":"2023-05-16T07:38:19.534724+0000","last_fullsized":"2023-05-16T07:38:19.534724+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.175084+0000","last_change":"2023-05-16T07:38:19.171763+0000","last_active":"2023-05-16T07:38:19.175084+0000","last_peered":"2023-05-16T07:38:19.175084+0000","last_clean":"2023-05-16T07:38:19.175084+0000","last_became_active":"2023-05-16T07:38:19.171125+0000","last_became_peered":"2023-05-16T07:38:19.171125+0000","last_unstale":"2023-05-16T07:38:19.175084+0000","last_undegraded":"2023-05-16T07:38:19.175084+0000","last_fullsized":"2023-05-16T07:38:19.175084+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.153981+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:38:19.153981+0000","last_peered":"2023-05-16T07:38:19.153981+0000","last_clean":"2023-05-16T07:38:19.153981+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:19.153981+0000","last_undegraded":"2023-05-16T07:38:19.153981+0000","last_fullsized":"2023-05-16T07:38:19.153981+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.532926+0000","last_change":"2023-05-16T07:38:19.532926+0000","last_active":"2023-05-16T07:38:19.532926+0000","last_peered":"2023-05-16T07:38:19.532926+0000","last_clean":"2023-05-16T07:38:19.532926+0000","last_became_active":"2023-05-16T07:38:19.532656+0000","last_became_peered":"2023-05-16T07:38:19.532656+0000","last_unstale":"2023-05-16T07:38:19.532926+0000","last_undegraded":"2023-05-16T07:38:19.532926+0000","last_fullsized":"2023-05-16T07:38:19.532926+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.166189+0000","last_change":"2023-05-16T07:38:19.165001+0000","last_active":"2023-05-16T07:38:19.166189+0000","last_peered":"2023-05-16T07:38:19.166189+0000","last_clean":"2023-05-16T07:38:19.166189+0000","last_became_active":"2023-05-16T07:38:19.164667+0000","last_became_peered":"2023-05-16T07:38:19.164667+0000","last_unstale":"2023-05-16T07:38:19.166189+0000","last_undegraded":"2023-05-16T07:38:19.166189+0000","last_fullsized":"2023-05-16T07:38:19.166189+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.153076+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:38:19.153076+0000","last_peered":"2023-05-16T07:38:19.153076+0000","last_clean":"2023-05-16T07:38:19.153076+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:19.153076+0000","last_undegraded":"2023-05-16T07:38:19.153076+0000","last_fullsized":"2023-05-16T07:38:19.153076+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":8,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.535416+0000","last_change":"2023-05-16T07:38:19.535416+0000","last_active":"2023-05-16T07:38:19.535416+0000","last_peered":"2023-05-16T07:38:19.535416+0000","last_clean":"2023-05-16T07:38:19.535416+0000","last_became_active":"2023-05-16T07:38:19.535132+0000","last_became_peered":"2023-05-16T07:38:19.535132+0000","last_unstale":"2023-05-16T07:38:19.535416+0000","last_undegraded":"2023-05-16T07:38:19.535416+0000","last_fullsized":"2023-05-16T07:38:19.535416+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:19.142228+0000","last_clean_scrub_stamp":"2023-05-16T07:38:19.142228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"170'4","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.543207+0000","last_change":"2023-05-16T07:38:19.172524+0000","last_active":"2023-05-16T07:38:19.543207+0000","last_peered":"2023-05-16T07:38:19.543207+0000","last_clean":"2023-05-16T07:38:19.543207+0000","last_became_active":"2023-05-16T07:38:19.172262+0000","last_became_peered":"2023-05-16T07:38:19.172262+0000","last_unstale":"2023-05-16T07:38:19.543207+0000","last_undegraded":"2023-05-16T07:38:19.543207+0000","last_fullsized":"2023-05-16T07:38:19.543207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:18.140065+0000","last_clean_scrub_stamp":"2023-05-16T07:38:18.140065+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":161,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-05-16T07:38:19.153250+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:38:19.153250+0000","last_peered":"2023-05-16T07:38:19.153250+0000","last_clean":"2023-05-16T07:38:19.153250+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:19.153250+0000","last_undegraded":"2023-05-16T07:38:19.153250+0000","last_fullsized":"2023-05-16T07:38:19.153250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575273,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24980,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":22912,"kb_avail":94346860,"statfs":{"total":96636764160,"available":96611184640,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23461888},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.77300000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.505}]},{"osd":1,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.34899999999999998},"max":{"1min":0.747,"5min":0.77600000000000002,"15min":0.79800000000000004},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.38900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72999999999999998,"5min":0.77500000000000002,"15min":0.81999999999999995},"last":0.57799999999999996}]},{"osd":3,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.45400000000000001,"5min":0.436,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68000000000000005}]},{"osd":4,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46300000000000002,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.71499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.75},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue May 16 07:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.71199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575273,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24992,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":22912,"kb_avail":94346848,"statfs":{"total":96636764160,"available":96611172352,"internally_reserved":0,"allocated":2121728,"data_stored":1136587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23461888},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.26700000000000002},"max":{"1min":0.68700000000000006,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.27900000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.505}]},{"osd":1,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":1.5,"15min":1.5},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.35599999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60399999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.373},"max":{"1min":0.747,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.57999999999999996},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.32900000000000001},"max":{"1min":0.71299999999999997,"5min":1.486,"15min":1.486},"last":0.44900000000000001}]},{"osd":3,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.373},"max":{"1min":0.77400000000000002,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78300000000000003,"5min":1.532,"15min":1.532},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76500000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.499},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.30199999999999999},"max":{"1min":0.76500000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52600000000000002,"15min":0.497},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.79900000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81299999999999994},"last":0.753}]},{"osd":7,"last update":"Tue May 16 07:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58899999999999997,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.316},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575274,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20360,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":18304,"kb_avail":94351480,"statfs":{"total":96636764160,"available":96615915520,"internally_reserved":0,"allocated":2097152,"data_stored":1121393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18743296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.374,"15min":0.372},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.47799999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60299999999999998,"5min":0.63,"15min":0.64100000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.42499999999999999,"15min":0.433},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40100000000000002,"15min":0.40799999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.58599999999999997,"15min":0.67900000000000005},"last":0.505}]},{"osd":4,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.872,"5min":0.873,"15min":0.877},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.62}]},{"osd":5,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.83699999999999997,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.42699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.998},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue May 16 07:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.437,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.67000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575274,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20208,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":18112,"kb_avail":94351632,"statfs":{"total":96636764160,"available":96616071168,"internally_reserved":0,"allocated":2138112,"data_stored":1137355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18546688},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36599999999999999,"15min":0.36499999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.222},"max":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.58899999999999997},"last":0.311},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.24199999999999999},"max":{"1min":0.48199999999999998,"5min":0.59799999999999998,"15min":0.627},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63100000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.26300000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.64600000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.70199999999999996},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.92700000000000005,"15min":0.98699999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.46700000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":1.0549999999999999},"last":0.496}]},{"osd":5,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":0.94899999999999995,"15min":1.0680000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67600000000000005,"15min":0.68300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.872,"5min":0.93500000000000005,"15min":1.095},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92100000000000004,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.42699999999999999,"15min":0.40899999999999997},"max":{"1min":0.89100000000000001,"5min":0.92900000000000005,"15min":1},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.03},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.44400000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":1.056},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21932,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94349908,"statfs":{"total":96636764160,"available":96614305792,"internally_reserved":0,"allocated":2134016,"data_stored":1136769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.28000000000000003}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.375}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.498}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.53700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575273,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.5,"5min":0.51200000000000001,"15min":0.61899999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28299999999999997,"5min":0.223,"15min":0.223},"max":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.308}]},{"osd":1,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39000000000000001,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48499999999999999,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40999999999999998,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.72999999999999998},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.432},"min":{"1min":0.34699999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.68400000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.441,"15min":0.442},"min":{"1min":0.29599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.63900000000000001,"5min":0.67300000000000004,"15min":0.70099999999999996},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":1.014,"15min":1.014},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.94399999999999995,"15min":1.024},"last":0.41699999999999998}]},{"osd":5,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84799999999999998,"5min":1.05,"15min":1.05},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.65400000000000003},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.41699999999999998},"max":{"1min":0.84799999999999998,"5min":0.96599999999999997,"15min":0.97299999999999998},"last":0.40600000000000003}]},{"osd":6,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83399999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.432}]},{"osd":7,"last update":"Tue May 16 07:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.48799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88100000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.89300000000000002,"5min":0.91300000000000003,"15min":1.012},"last":0.504}]}]},{"osd":4,"up_from":13,"seq":55834575273,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24788,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":22720,"kb_avail":94347052,"statfs":{"total":96636764160,"available":96611381248,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23265280},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.439,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.503}]},{"osd":1,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78600000000000003,"15min":0.81899999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38},"max":{"1min":0.74299999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.58099999999999996},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.68400000000000005}]},{"osd":3,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.373},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.624}]},{"osd":5,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.29099999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"max":{"1min":0.72499999999999998,"5min":0.755,"15min":0.755},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Tue May 16 07:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.33500000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.754},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.70599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575273,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":2109440,"data_stored":1128740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80400000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.80800000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.39700000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.626},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.85599999999999998},"last":0.61399999999999999}]},{"osd":4,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.71299999999999997,"5min":0.74199999999999999,"15min":0.84899999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.497,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.82699999999999996,"15min":0.83299999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.504,"15min":0.496},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.872},"last":0.626}]},{"osd":7,"last update":"Tue May 16 07:37:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":0.86299999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81100000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:38:22.859 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-16T07:38:22.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-05-16T07:38:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.000+0000 7fd1cdc81700 1 Processor -- start 2023-05-16T07:38:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.000+0000 7fd1cdc81700 1 -- start start 2023-05-16T07:38:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cdc81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c80a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cdc81700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1c8155380 0x7fd1c80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cdc81700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c80c6670 con 0x7fd1c80c5d00 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cdc81700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c8069f60 con 0x7fd1c8155380 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cdc81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c806a9e0 con 0x7fd1c8153f90 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1c8155380 0x7fd1c80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1c8155380 0x7fd1c80c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52832/0 (socket says 172.21.15.92:52832) 2023-05-16T07:38:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 -- 172.21.15.92:0/3676482971 learned_addr learned my addr 172.21.15.92:0/3676482971 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 -- 172.21.15.92:0/3676482971 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 msgr2=0x7fd1c80a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 --2- 172.21.15.92:0/3676482971 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c80a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 -- 172.21.15.92:0/3676482971 >> v1:172.21.15.158:6789/0 conn(0x7fd1c80c5d00 legacy=0x7fd1c80c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 -- 172.21.15.92:0/3676482971 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c806a140 con 0x7fd1c8155380 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.001+0000 7fd1cd480700 1 --2- 172.21.15.92:0/3676482971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c8155380 0x7fd1c80c6180 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7fd1c0006fd0 tx=0x7fd1c00090f0).ready entity=mon.0 client_cookie=501e3e2d229058c9 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1c77fe700 1 -- 172.21.15.92:0/3676482971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000b600 con 0x7fd1c8155380 2023-05-16T07:38:23.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1c77fe700 1 -- 172.21.15.92:0/3676482971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c000b760 con 0x7fd1c8155380 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1c77fe700 1 -- 172.21.15.92:0/3676482971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000bae0 con 0x7fd1c8155380 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3676482971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c8155380 msgr2=0x7fd1c80c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3676482971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c8155380 0x7fd1c80c6180 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7fd1c0006fd0 tx=0x7fd1c00090f0).stop 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3676482971 shutdown_connections 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3676482971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c8155380 0x7fd1c80c6180 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3676482971 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c80a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3676482971 >> 172.21.15.92:0/3676482971 conn(0x7fd1c80cb600 msgr2=0x7fd1c80c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3676482971 shutdown_connections 2023-05-16T07:38:23.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.002+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3676482971 wait complete. 2023-05-16T07:38:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.003+0000 7fd1cdc81700 1 Processor -- start 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.003+0000 7fd1cdc81700 1 -- start start 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cdc81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 0x7fd1c80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cdc81700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c8151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cdc81700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1c8155380 0x7fd1c814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cdc81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80adae0 con 0x7fd1c80c5d00 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cdc81700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80addc0 con 0x7fd1c8155380 2023-05-16T07:38:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cdc81700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8151ab0 con 0x7fd1c8153f90 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1ccc7f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 0x7fd1c80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1ccc7f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 0x7fd1c80b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52836/0 (socket says 172.21.15.92:52836) 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1ccc7f700 1 -- 172.21.15.92:0/3696861538 learned_addr learned my addr 172.21.15.92:0/3696861538 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1cd480700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1c8155380 0x7fd1c814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.004+0000 7fd1c7fff700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c8151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1ccc7f700 1 -- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 msgr2=0x7fd1c8151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1ccc7f700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c8151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1ccc7f700 1 -- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1c8155380 msgr2=0x7fd1c814d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1ccc7f700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1c8155380 0x7fd1c814d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1ccc7f700 1 -- 172.21.15.92:0/3696861538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0009370 con 0x7fd1c80c5d00 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1cd480700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1c8155380 0x7fd1c814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1ccc7f700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 0x7fd1c80b08a0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fd1b800bc30 tx=0x7fd1b8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800c7a0 con 0x7fd1c80c5d00 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b8010070 con 0x7fd1c80c5d00 2023-05-16T07:38:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c8151d90 con 0x7fd1c80c5d00 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800c960 con 0x7fd1c80c5d00 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.005+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8151720 con 0x7fd1c80c5d00 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.006+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c8083120 con 0x7fd1c80c5d00 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.006+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fd1b801c070 con 0x7fd1c80c5d00 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.006+0000 7fd1c4ff9700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd1a805ed90 0x7fd1a8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.007+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fd1b80794c0 con 0x7fd1c80c5d00 2023-05-16T07:38:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.007+0000 7fd1c7fff700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd1a805ed90 0x7fd1a8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.007+0000 7fd1c7fff700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd1a805ed90 0x7fd1a8061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd1c80b0d90 tx=0x7fd1b0008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:23.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.011+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd1b804a7c0 con 0x7fd1c80c5d00 2023-05-16T07:38:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.234+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fd1c8047150 con 0x7fd1c80c5d00 2023-05-16T07:38:23.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.237+0000 7fd1c4ff9700 1 -- 172.21.15.92:0/3696861538 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd1b801a070 con 0x7fd1c80c5d00 2023-05-16T07:38:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.239+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd1a805ed90 msgr2=0x7fd1a8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.239+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd1a805ed90 0x7fd1a8061250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd1c80b0d90 tx=0x7fd1b0008040).stop 2023-05-16T07:38:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.239+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 msgr2=0x7fd1c80b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.239+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 0x7fd1c80b08a0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fd1b800bc30 tx=0x7fd1b8009840).stop 2023-05-16T07:38:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 shutdown_connections 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1c8155380 0x7fd1c814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1c80c5d00 0x7fd1c80b08a0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd1a805ed90 0x7fd1a8061250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 --2- 172.21.15.92:0/3696861538 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd1c8153f90 0x7fd1c8151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 >> 172.21.15.92:0/3696861538 conn(0x7fd1c80cb600 msgr2=0x7fd1c80ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 shutdown_connections 2023-05-16T07:38:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.240+0000 7fd1cdc81700 1 -- 172.21.15.92:0/3696861538 wait complete. 2023-05-16T07:38:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-05-16T07:38:23.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:23.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.360+0000 7f5263be6700 1 Processor -- start 2023-05-16T07:38:23.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.360+0000 7f5263be6700 1 -- start start 2023-05-16T07:38:23.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5263be6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f525c154170 0x7f525c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5263be6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0c0660 con 0x7f525c154550 2023-05-16T07:38:23.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5263be6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0bfd30 con 0x7f525c069db0 2023-05-16T07:38:23.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5263be6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c0c0010 con 0x7f525c154170 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5261982700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f525c154170 0x7f525c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5261982700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f525c154170 0x7f525c0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52844/0 (socket says 172.21.15.92:52844) 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5261982700 1 -- 172.21.15.92:0/3009903158 learned_addr learned my addr 172.21.15.92:0/3009903158 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.361+0000 7f5261181700 1 --1- 172.21.15.92:0/3009903158 >> v1:172.21.15.92:6789/0 conn(0x7f525c154550 0x7f525c0696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35360/0 (socket says 172.21.15.92:35360) 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5261982700 1 -- 172.21.15.92:0/3009903158 >> v1:172.21.15.158:6789/0 conn(0x7f525c069db0 legacy=0x7f525c0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5261982700 1 -- 172.21.15.92:0/3009903158 >> v1:172.21.15.92:6789/0 conn(0x7f525c154550 legacy=0x7f525c0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5260980700 1 -- 172.21.15.92:0/3009903158 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049175069 0 0) 0x7f525c0c0660 con 0x7f525c154550 2023-05-16T07:38:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5261982700 1 -- 172.21.15.92:0/3009903158 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0c02f0 con 0x7f525c154170 2023-05-16T07:38:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5261982700 1 --2- 172.21.15.92:0/3009903158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c154170 0x7f525c0c5c70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f524c009f20 tx=0x7f524c009840).ready entity=mon.0 client_cookie=7b25e00f7a9c798b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5260980700 1 -- 172.21.15.92:0/3009903158 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e070 con 0x7f525c154170 2023-05-16T07:38:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5260980700 1 -- 172.21.15.92:0/3009903158 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00ba40 con 0x7f525c154170 2023-05-16T07:38:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5260980700 1 -- 172.21.15.92:0/3009903158 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00bdc0 con 0x7f525c154170 2023-05-16T07:38:23.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5263be6700 1 -- 172.21.15.92:0/3009903158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c154170 msgr2=0x7f525c0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.362+0000 7f5263be6700 1 --2- 172.21.15.92:0/3009903158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c154170 0x7f525c0c5c70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f524c009f20 tx=0x7f524c009840).stop 2023-05-16T07:38:23.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.363+0000 7f5263be6700 1 -- 172.21.15.92:0/3009903158 shutdown_connections 2023-05-16T07:38:23.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.363+0000 7f5263be6700 1 --2- 172.21.15.92:0/3009903158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c154170 0x7f525c0c5c70 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.363+0000 7f5263be6700 1 -- 172.21.15.92:0/3009903158 >> 172.21.15.92:0/3009903158 conn(0x7f525c0cb700 msgr2=0x7f525c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:23.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.363+0000 7f5263be6700 1 -- 172.21.15.92:0/3009903158 shutdown_connections 2023-05-16T07:38:23.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.363+0000 7f5263be6700 1 -- 172.21.15.92:0/3009903158 wait complete. 2023-05-16T07:38:23.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 Processor -- start 2023-05-16T07:38:23.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 -- start start 2023-05-16T07:38:23.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 0x7f525c0c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f525c154170 0x7f525c14dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f525c154550 0x7f525c0c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c14dfd0 con 0x7f525c069db0 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0c3320 con 0x7f525c154170 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.364+0000 7f5263be6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0c29f0 con 0x7f525c154550 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 0x7f525c0c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 0x7f525c0c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52850/0 (socket says 172.21.15.92:52850) 2023-05-16T07:38:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 -- 172.21.15.92:0/3514256551 learned_addr learned my addr 172.21.15.92:0/3514256551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 -- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f525c154550 msgr2=0x7f525c0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f525c154550 0x7f525c0c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 -- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f525c154170 msgr2=0x7f525c14dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f525c154170 0x7f525c14dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 -- 172.21.15.92:0/3514256551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009ae0 con 0x7f525c069db0 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5261982700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 0x7f525c0c4cf0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f525c083060 tx=0x7f524c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00bc80 con 0x7f525c069db0 2023-05-16T07:38:23.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c01d410 con 0x7f525c069db0 2023-05-16T07:38:23.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c01d7d0 con 0x7f525c069db0 2023-05-16T07:38:23.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c0c2cd0 con 0x7f525c069db0 2023-05-16T07:38:23.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.365+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c0c2700 con 0x7f525c069db0 2023-05-16T07:38:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.366+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c0c3600 con 0x7f525c069db0 2023-05-16T07:38:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.367+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f524c01b040 con 0x7f525c069db0 2023-05-16T07:38:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.367+0000 7f5259ffb700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f524805ed90 0x7f5248061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.367+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f524c07b810 con 0x7f525c069db0 2023-05-16T07:38:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.367+0000 7f5261181700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f524805ed90 0x7f5248061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.368+0000 7f5261181700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f524805ed90 0x7f5248061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f525c0c3b10 tx=0x7f52500079f0).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:23.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.371+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f524c04cb90 con 0x7f525c069db0 2023-05-16T07:38:23.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.577+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f525c0c3600 con 0x7f525c069db0 2023-05-16T07:38:23.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.578+0000 7f5259ffb700 1 -- 172.21.15.92:0/3514256551 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f524c0501b0 con 0x7f525c069db0 2023-05-16T07:38:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:23.582 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:38:19.513917+0000","modified":"2023-05-16T07:38:21.520884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/862724662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":862724662},{"type":"v1","addr":"172.21.15.158:6837","nonce":862724662}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-05-16T07:38:23.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f524805ed90 msgr2=0x7f5248061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f524805ed90 0x7f5248061250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f525c0c3b10 tx=0x7f52500079f0).stop 2023-05-16T07:38:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 msgr2=0x7f525c0c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 0x7f525c0c4cf0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f525c083060 tx=0x7f524c009840).stop 2023-05-16T07:38:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 shutdown_connections 2023-05-16T07:38:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f524805ed90 0x7f5248061250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f525c154550 0x7f525c0c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525c069db0 0x7f525c0c4cf0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 --2- 172.21.15.92:0/3514256551 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f525c154170 0x7f525c14dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 >> 172.21.15.92:0/3514256551 conn(0x7f525c0cb700 msgr2=0x7f525c0aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 shutdown_connections 2023-05-16T07:38:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:23.580+0000 7f5263be6700 1 -- 172.21.15.92:0/3514256551 wait complete. 2023-05-16T07:38:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-05-16T07:38:23.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-16T07:38:19.513917+0000', 'modified': '2023-05-16T07:38:21.520884+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8835}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8835': {'gid': 8835, 'name': 'd', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.158:6837/862724662', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6836', 'nonce': 862724662}, {'type': 'v1', 'addr': '172.21.15.158:6837', 'nonce': 862724662}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-05-16T07:38:23.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-16T07:38:23.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-05-16T07:38:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.107+0000 7f59b359e700 1 Processor -- start 2023-05-16T07:38:24.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.107+0000 7f59b359e700 1 -- start start 2023-05-16T07:38:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.108+0000 7f59b359e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f59b4153ca0 0x7f59b4067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.108+0000 7f59b359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b40691c0 con 0x7f59b40bf270 2023-05-16T07:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.108+0000 7f59b359e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b4069d60 con 0x7f59b40c8800 2023-05-16T07:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.108+0000 7f59b359e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b40685b0 con 0x7f59b4153ca0 2023-05-16T07:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b1d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f59b40bf270 0x7f59b4069650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35376/0 (socket says 172.21.15.92:35376) 2023-05-16T07:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b1d9b700 1 -- 172.21.15.92:0/3307324925 learned_addr learned my addr 172.21.15.92:0/3307324925 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b259c700 1 --2- 172.21.15.92:0/3307324925 >> v2:172.21.15.158:3300/0 conn(0x7f59b4153ca0 0x7f59b4067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747469529 0 0) 0x7f59b40691c0 con 0x7f59b40bf270 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a00033f0 con 0x7f59b40bf270 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3567295691 0 0) 0x7f59a00033f0 con 0x7f59b40bf270 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 >> v2:172.21.15.158:3300/0 conn(0x7f59b4153ca0 msgr2=0x7f59b4067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 --2- 172.21.15.92:0/3307324925 >> v2:172.21.15.158:3300/0 conn(0x7f59b4153ca0 0x7f59b4067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 >> v1:172.21.15.158:6789/0 conn(0x7f59b40c8800 legacy=0x7f59b40c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4068890 con 0x7f59b40bf270 2023-05-16T07:38:24.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f599c001470 con 0x7f59b40bf270 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 >> v1:172.21.15.92:6789/0 conn(0x7f59b40bf270 legacy=0x7f59b4069650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59a00038e0 0x7f59a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0004690 0x7f59a0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a0005000 0x7f59a000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40685b0 con 0x7f59a0005000 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a000beb0 con 0x7f59a0004690 2023-05-16T07:38:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a000c070 con 0x7f59a00038e0 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a0005000 0x7f59a000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b259c700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59a00038e0 0x7f59a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.109+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0004690 0x7f59a0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 -- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59a00038e0 msgr2=0x7f59a0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59a00038e0 0x7f59a0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 -- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0004690 msgr2=0x7f59a0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0004690 0x7f59a0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 -- 172.21.15.92:0/3307324925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00033f0 con 0x7f59a0005000 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0004690 0x7f59a0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 -- 172.21.15.92:0/3307324925 reap_dead start 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a0005000 0x7f59a000b8e0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f59ac001730 tx=0x7f59ac007bb0).ready entity=mon.0 client_cookie=de8ea1d1d20cb17b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac007020 con 0x7f59a0005000 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac007180 con 0x7f59a0005000 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b159a700 1 -- 172.21.15.92:0/3307324925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac007520 con 0x7f59a0005000 2023-05-16T07:38:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 -- 172.21.15.92:0/3307324925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a0005000 msgr2=0x7f59a000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a0005000 0x7f59a000b8e0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f59ac001730 tx=0x7f59ac007bb0).stop 2023-05-16T07:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 -- 172.21.15.92:0/3307324925 shutdown_connections 2023-05-16T07:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 --2- 172.21.15.92:0/3307324925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a0005000 0x7f59a000b8e0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 -- 172.21.15.92:0/3307324925 >> 172.21.15.92:0/3307324925 conn(0x7f59b40622b0 msgr2=0x7f59b40626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 -- 172.21.15.92:0/3307324925 shutdown_connections 2023-05-16T07:38:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.110+0000 7f59b359e700 1 -- 172.21.15.92:0/3307324925 wait complete. 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.111+0000 7f59b359e700 1 Processor -- start 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.111+0000 7f59b359e700 1 -- start start 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b359e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0005000 0x7f59b414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b359e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59b40a8de0 0x7f59b40aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 0x7f59b40a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40a9ef0 con 0x7f59b414d7c0 2023-05-16T07:38:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b359e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40a9270 con 0x7f59a0005000 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b359e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40a9550 con 0x7f59b40a8de0 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 0x7f59b40a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 0x7f59b40a5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52874/0 (socket says 172.21.15.92:52874) 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 -- 172.21.15.92:0/2248583870 learned_addr learned my addr 172.21.15.92:0/2248583870 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b259c700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0005000 0x7f59b414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59b40a8de0 0x7f59b40aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 -- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59b40a8de0 msgr2=0x7f59b40aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59b40a8de0 0x7f59b40aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 -- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0005000 msgr2=0x7f59b414d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0005000 0x7f59b414d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b2d9d700 1 -- 172.21.15.92:0/2248583870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40a9830 con 0x7f59b414d7c0 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.112+0000 7f59b259c700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0005000 0x7f59b414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.113+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 0x7f59b40a5f30 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f59b4065e10 tx=0x7f59ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.113+0000 7f59aa7fc700 1 -- 172.21.15.92:0/2248583870 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac013670 con 0x7f59b414d7c0 2023-05-16T07:38:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.113+0000 7f59aa7fc700 1 -- 172.21.15.92:0/2248583870 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac0137d0 con 0x7f59b414d7c0 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.113+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b40a9b10 con 0x7f59b414d7c0 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.113+0000 7f59aa7fc700 1 -- 172.21.15.92:0/2248583870 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac013b70 con 0x7f59b414d7c0 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.113+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b40aa7f0 con 0x7f59b414d7c0 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.114+0000 7f59aa7fc700 1 -- 172.21.15.92:0/2248583870 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f59ac027070 con 0x7f59b414d7c0 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.114+0000 7f59aa7fc700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f599405ed90 0x7f5994061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.114+0000 7f59aa7fc700 1 -- 172.21.15.92:0/2248583870 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f59ac07a6c0 con 0x7f59b414d7c0 2023-05-16T07:38:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.114+0000 7f59b259c700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f599405ed90 0x7f5994061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.115+0000 7f59b259c700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f599405ed90 0x7f5994061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f59a40094a0 tx=0x7f59a400b040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.116+0000 7f59ca4feb80 1 Processor -- start 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.116+0000 7f59ca4feb80 1 -- start start 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a26496b0 0x5584a2671a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 0x5584a2649a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a2644660 0x5584a2644ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a260f5e0 con 0x5584a2692350 2023-05-16T07:38:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a25868a0 con 0x5584a2644660 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a259b900 con 0x5584a26496b0 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59b1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 0x5584a2649a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59b1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 0x5584a2649a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52880/0 (socket says 172.21.15.92:52880) 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.117+0000 7f59b1d9b700 1 -- 172.21.15.92:0/2116775822 learned_addr learned my addr 172.21.15.92:0/2116775822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b259c700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a26496b0 0x5584a2671a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a2644660 0x5584a2644ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b1d9b700 1 -- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a26496b0 msgr2=0x5584a2671a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a26496b0 0x5584a2671a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b1d9b700 1 -- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a2644660 msgr2=0x5584a2644ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a2644660 0x5584a2644ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b1d9b700 1 -- 172.21.15.92:0/2116775822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a243e310 con 0x5584a2692350 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b259c700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a26496b0 0x5584a2671a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a2644660 0x5584a2644ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 0x5584a2649a90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f599c005810 tx=0x7f599c009040).ready entity=mon.0 client_cookie=23f57e5d04a0c611 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59937fe700 1 -- 172.21.15.92:0/2116775822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0105b0 con 0x5584a2692350 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59937fe700 1 -- 172.21.15.92:0/2116775822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c010710 con 0x5584a2692350 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59937fe700 1 -- 172.21.15.92:0/2116775822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c010ab0 con 0x5584a2692350 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59ca4feb80 1 -- 172.21.15.92:0/2116775822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 msgr2=0x5584a2649a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 0x5584a2649a90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f599c005810 tx=0x7f599c009040).stop 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59ca4feb80 1 -- 172.21.15.92:0/2116775822 shutdown_connections 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a26496b0 0x5584a2671a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.118+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a2644660 0x5584a2644ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.119+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/2116775822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2692350 0x5584a2649a90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.119+0000 7f59ca4feb80 1 -- 172.21.15.92:0/2116775822 >> 172.21.15.92:0/2116775822 conn(0x5584a2406e40 msgr2=0x5584a26927e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.119+0000 7f59ca4feb80 1 -- 172.21.15.92:0/2116775822 shutdown_connections 2023-05-16T07:38:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.119+0000 7f59ca4feb80 1 -- 172.21.15.92:0/2116775822 wait complete. 2023-05-16T07:38:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.119+0000 7f59ca4feb80 1 -- start start 2023-05-16T07:38:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.119+0000 7f59ca4feb80 1 Processor -- start 2023-05-16T07:38:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2671a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2674860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2692350 0x5584a2697420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a2562ed0 con 0x5584a2644660 2023-05-16T07:38:24.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a26942a0 con 0x5584a26496b0 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a2673b70 con 0x5584a2692350 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59ca4feb80 10 client.0 ll_register_callbacks cb 0x5584a26093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2671a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2671a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52884/0 (socket says 172.21.15.92:52884) 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 -- 172.21.15.92:0/1471687221 learned_addr learned my addr 172.21.15.92:0/1471687221 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2692350 0x5584a2697420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2674860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 -- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2692350 msgr2=0x5584a2697420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2692350 0x5584a2697420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 -- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 msgr2=0x5584a2674860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2674860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b259c700 1 -- 172.21.15.92:0/1471687221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b4068890 con 0x5584a2644660 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.120+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2674860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59b259c700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2671a50 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f59a4009b10 tx=0x7f59a4018000).ready entity=mon.0 client_cookie=44af389d84a04314 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f5990ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f5990ff9700 1 -- 172.21.15.92:0/1471687221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400e040 con 0x5584a2644660 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f5990ff9700 1 -- 172.21.15.92:0/1471687221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f59a4015dc0 con 0x5584a2644660 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584a2641a40 con 0x5584a2644660 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f5990ff9700 1 -- 172.21.15.92:0/1471687221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a4021620 con 0x5584a2644660 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 10 client.8976 fetch_fsmap learned FSMap version 61 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5584a2594880 con 0x5584a2644660 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f5990ff9700 1 -- 172.21.15.92:0/1471687221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5584a2641a40 con 0x5584a2644660 2023-05-16T07:38:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f5990ff9700 1 -- 172.21.15.92:0/1471687221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 61) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f59a4021780 con 0x5584a2644660 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 10 client.8976 fetch_fsmap finished waiting for FSMap version 61 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 10 client.8976 resolve_mds: resolved name 'd' to daemon mds.d 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 20 client.8976 populate_metadata read hostname 'smithi092' 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2676890 0x5584a2695d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 4 client.8976 mds_command: new command op to 8835 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 --> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584a2402ca0 con 0x5584a2676890 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.121+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2676890 0x5584a2695d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.122+0000 7f5990ff9700 10 client.8976 ms_handle_connect on v2:172.21.15.158:6836/862724662 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.122+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2676890 0x5584a2695d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1c5cfdfd48da8286 server_cookie=bec75571265a9831 in_seq=0 out_seq=0 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.122+0000 7f5990ff9700 1 -- 172.21.15.92:0/1471687221 <== mds.0 v2:172.21.15.158:6836/862724662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5584a2402ca0 con 0x5584a2676890 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.122+0000 7f5990ff9700 10 client.8976 handle_command_reply: tid=0 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.122+0000 7f59ca4feb80 1 client.8976 shutdown 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 msgr2=0x5584a2671a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2671a50 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f59a4009b10 tx=0x7f59a4018000).stop 2023-05-16T07:38:24.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 shutdown_connections 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2676890 0x5584a2695d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2692350 0x5584a2697420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f5990ff9700 0 client.8976 ms_handle_reset on v2:172.21.15.158:6836/862724662 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2671a50 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1471687221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2674860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 >> 172.21.15.92:0/1471687221 conn(0x5584a2406e40 msgr2=0x5584a2643560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 shutdown_connections 2023-05-16T07:38:24.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1471687221 wait complete. 2023-05-16T07:38:24.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.123+0000 7f59ca4feb80 20 client.8976 trim_cache size 0 max 16384 2023-05-16T07:38:24.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.139+0000 7f59ca4feb80 1 Processor -- start 2023-05-16T07:38:24.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.139+0000 7f59ca4feb80 1 -- start start 2023-05-16T07:38:24.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2643cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2673f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a2693a60 con 0x5584a2644660 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a2675740 con 0x5584a26496b0 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a2645f10 con 0x5584a2676890 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2643cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2643cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52892/0 (socket says 172.21.15.92:52892) 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 -- 172.21.15.92:0/1806214142 learned_addr learned my addr 172.21.15.92:0/1806214142 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2673f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 -- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 msgr2=0x5584a26463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 -- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 msgr2=0x5584a2673f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2673f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 -- 172.21.15.92:0/1806214142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a2590070 con 0x5584a2644660 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f59b259c700 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2643cc0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f59a40071c0 tx=0x7f59a4023000).ready entity=mon.0 client_cookie=3925eaa2ec07fa09 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f5993fff700 1 -- 172.21.15.92:0/1806214142 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400e030 con 0x5584a2644660 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f5993fff700 1 -- 172.21.15.92:0/1806214142 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400f040 con 0x5584a2644660 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.140+0000 7f5993fff700 1 -- 172.21.15.92:0/1806214142 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4021620 con 0x5584a2644660 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1806214142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 msgr2=0x5584a2643cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2643cc0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f59a40071c0 tx=0x7f59a4023000).stop 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1806214142 shutdown_connections 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a2643cc0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/1806214142 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a2673f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1806214142 >> 172.21.15.92:0/1806214142 conn(0x5584a2406e40 msgr2=0x5584a26758a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1806214142 shutdown_connections 2023-05-16T07:38:24.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 -- 172.21.15.92:0/1806214142 wait complete. 2023-05-16T07:38:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 -- start start 2023-05-16T07:38:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.141+0000 7f59ca4feb80 1 Processor -- start 2023-05-16T07:38:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a268c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a26f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a268bb40 con 0x5584a2644660 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a26707c0 con 0x5584a26496b0 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a26a0f70 con 0x5584a2676890 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59ca4feb80 10 client.0 ll_register_callbacks cb 0x7f59a4016760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a268c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a268c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52898/0 (socket says 172.21.15.92:52898) 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59b259c700 1 -- 172.21.15.92:0/923199451 learned_addr learned my addr 172.21.15.92:0/923199451 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.142+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a26f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b2d9d700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b259c700 1 -- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 msgr2=0x5584a26a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b259c700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b259c700 1 -- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 msgr2=0x5584a26f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b259c700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a26f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b259c700 1 -- 172.21.15.92:0/923199451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584a2594880 con 0x5584a2644660 2023-05-16T07:38:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59b259c700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a268c540 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f59a4009cd0 tx=0x7f59a4035070).ready entity=mon.0 client_cookie=ec5e4ea31583e8b5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f5990ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f5990ff9700 1 -- 172.21.15.92:0/923199451 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4021620 con 0x5584a2644660 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f5990ff9700 1 -- 172.21.15.92:0/923199451 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f59a4029560 con 0x5584a2644660 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f599c010710 con 0x5584a2644660 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f5990ff9700 1 -- 172.21.15.92:0/923199451 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a4029930 con 0x5584a2644660 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 10 client.8982 fetch_fsmap learned FSMap version 61 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5584a258fdc0 con 0x5584a2644660 2023-05-16T07:38:24.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f5990ff9700 1 -- 172.21.15.92:0/923199451 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c010710 con 0x5584a2644660 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f5990ff9700 1 -- 172.21.15.92:0/923199451 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 61) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f59a4029b10 con 0x5584a2644660 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 10 client.8982 fetch_fsmap finished waiting for FSMap version 61 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 10 client.8982 resolve_mds: resolved name 'd' to daemon mds.d 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 20 client.8982 populate_metadata read hostname 'smithi092' 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2692350 0x5584a268ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 4 client.8982 mds_command: new command op to 8835 tid=0[{"prefix": "status"}] 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.143+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 --> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5584a2439ca0 con 0x5584a2692350 2023-05-16T07:38:24.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2692350 0x5584a268ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f5990ff9700 10 client.8982 ms_handle_connect on v2:172.21.15.158:6836/862724662 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59b1d9b700 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2692350 0x5584a268ad60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c4a92b39ea084555 server_cookie=9b8825492071ced4 in_seq=0 out_seq=0 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f5990ff9700 1 -- 172.21.15.92:0/923199451 <== mds.0 v2:172.21.15.158:6836/862724662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5584a2439ca0 con 0x5584a2692350 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f5990ff9700 10 client.8982 handle_command_reply: tid=0 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 client.8982 shutdown 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 msgr2=0x5584a268c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a268c540 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f59a4009cd0 tx=0x7f59a4035070).stop 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 shutdown_connections 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:6836/862724662,v1:172.21.15.158:6837/862724662] conn(0x5584a2692350 0x5584a268ad60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x5584a2676890 0x5584a26a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f5990ff9700 0 client.8982 ms_handle_reset on v2:172.21.15.158:6836/862724662 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5584a2644660 0x5584a268c540 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 --2- 172.21.15.92:0/923199451 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x5584a26496b0 0x5584a26f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 >> 172.21.15.92:0/923199451 conn(0x5584a2406e40 msgr2=0x5584a2643770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 shutdown_connections 2023-05-16T07:38:24.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.144+0000 7f59ca4feb80 1 -- 172.21.15.92:0/923199451 wait complete. 2023-05-16T07:38:24.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59ca4feb80 20 client.8982 trim_cache size 0 max 16384 2023-05-16T07:38:24.158 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:24.158 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "7dc962fc-4181-4aa6-8ae9-07b8e654738a", 2023-05-16T07:38:24.158 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-05-16T07:38:24.158 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8835, 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 4.6294823860000003, 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 61, 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 171, 2023-05-16T07:38:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 171, 2023-05-16T07:38:24.160 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 9.1409250719999999 2023-05-16T07:38:24.160 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f599405ed90 msgr2=0x7f5994061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f599405ed90 0x7f5994061250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f59a40094a0 tx=0x7f59a400b040).stop 2023-05-16T07:38:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 msgr2=0x7f59b40a5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 0x7f59b40a5f30 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f59b4065e10 tx=0x7f59ac015000).stop 2023-05-16T07:38:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 shutdown_connections 2023-05-16T07:38:24.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59b414d7c0 0x7f59b40a5f30 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59a0005000 0x7f59b414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f599405ed90 0x7f5994061250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 --2- 172.21.15.92:0/2248583870 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f59b40a8de0 0x7f59b40aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 >> 172.21.15.92:0/2248583870 conn(0x7f59b40622b0 msgr2=0x7f59b4063ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 shutdown_connections 2023-05-16T07:38:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.145+0000 7f59b359e700 1 -- 172.21.15.92:0/2248583870 wait complete. 2023-05-16T07:38:24.163 DEBUG:teuthology.orchestra.run.smithi092:> ceph fs ls 2023-05-16T07:38:24.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.258+0000 7f3ca0f9c700 1 Processor -- start 2023-05-16T07:38:24.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.258+0000 7f3ca0f9c700 1 -- start start 2023-05-16T07:38:24.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.259+0000 7f3ca0f9c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 0x7f3c9c0c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.259+0000 7f3ca0f9c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c0aaf40 con 0x7f3c9c063d00 2023-05-16T07:38:24.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.259+0000 7f3ca0f9c700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c0c1450 con 0x7f3c9c1541a0 2023-05-16T07:38:24.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.259+0000 7f3ca0f9c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0c1730 con 0x7f3c9c153dc0 2023-05-16T07:38:24.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.259+0000 7f3c9ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3c9c063d00 0x7f3c9c0ab3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35422/0 (socket says 172.21.15.92:35422) 2023-05-16T07:38:24.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.259+0000 7f3c9ad9d700 1 -- 172.21.15.92:0/2431511292 learned_addr learned my addr 172.21.15.92:0/2431511292 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:24.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9a59c700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 0x7f3c9c0c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833694172 0 0) 0x7f3c9c0aaf40 con 0x7f3c9c063d00 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c880033f0 con 0x7f3c9c063d00 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283191757 0 0) 0x7f3c880033f0 con 0x7f3c9c063d00 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 msgr2=0x7f3c9c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 0x7f3c9c0c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 >> v1:172.21.15.158:6789/0 conn(0x7f3c9c1541a0 legacy=0x7f3c9c0635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0c1a10 con 0x7f3c9c063d00 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f3c94001470 con 0x7f3c9c063d00 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 >> v1:172.21.15.92:6789/0 conn(0x7f3c9c063d00 legacy=0x7f3c9c0ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 0x7f3c88003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c88003e40 0x7f3c88004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88004b80 0x7f3c88005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0c1730 con 0x7f3c88003e40 2023-05-16T07:38:24.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8800ba10 con 0x7f3c88004b80 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8800bbd0 con 0x7f3c9c153dc0 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c88003e40 0x7f3c88004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9a59c700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 0x7f3c88003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c9ad9d700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88004b80 0x7f3c88005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c99d9b700 1 -- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 msgr2=0x7f3c88003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c153dc0 0x7f3c88003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c99d9b700 1 -- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88004b80 msgr2=0x7f3c88005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:24.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88004b80 0x7f3c88005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.260+0000 7f3c99d9b700 1 -- 172.21.15.92:0/2431511292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c880033f0 con 0x7f3c88003e40 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c88003e40 0x7f3c88004690 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f3c9c05eb90 tx=0x7f3c84007bb0).ready entity=mon.0 client_cookie=b5f5ae0fd1194876 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84007020 con 0x7f3c88003e40 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c84007180 con 0x7f3c88003e40 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3c9959a700 1 -- 172.21.15.92:0/2431511292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84007500 con 0x7f3c88003e40 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/2431511292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c88003e40 msgr2=0x7f3c88004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/2431511292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c88003e40 0x7f3c88004690 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f3c9c05eb90 tx=0x7f3c84007bb0).stop 2023-05-16T07:38:24.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3c9ad9d700 1 -- 172.21.15.92:0/2431511292 reap_dead start 2023-05-16T07:38:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/2431511292 shutdown_connections 2023-05-16T07:38:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/2431511292 >> 172.21.15.92:0/2431511292 conn(0x7f3c9c0caca0 msgr2=0x7f3c9c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/2431511292 shutdown_connections 2023-05-16T07:38:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.261+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/2431511292 wait complete. 2023-05-16T07:38:24.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.262+0000 7f3ca0f9c700 1 Processor -- start 2023-05-16T07:38:24.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.262+0000 7f3ca0f9c700 1 -- start start 2023-05-16T07:38:24.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3ca0f9c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88003e40 0x7f3c9c149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3ca0f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 0x7f3c9c14b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3ca0f9c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c14cd30 0x7f3c9c0c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3ca0f9c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c14c8a0 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3ca0f9c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c14bc90 con 0x7f3c88003e40 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3ca0f9c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c14bf70 con 0x7f3c9c14cd30 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 0x7f3c9c14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 0x7f3c9c14b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52920/0 (socket says 172.21.15.92:52920) 2023-05-16T07:38:24.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 -- 172.21.15.92:0/4178370866 learned_addr learned my addr 172.21.15.92:0/4178370866 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c9a59c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88003e40 0x7f3c9c149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c9ad9d700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c14cd30 0x7f3c9c0c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 -- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c14cd30 msgr2=0x7f3c9c0c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c14cd30 0x7f3c9c0c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 -- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88003e40 msgr2=0x7f3c9c149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88003e40 0x7f3c9c149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c99d9b700 1 -- 172.21.15.92:0/4178370866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c14c250 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.263+0000 7f3c9ad9d700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c14cd30 0x7f3c9c0c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.264+0000 7f3c99d9b700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 0x7f3c9c14b360 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f3c9c0b0e70 tx=0x7f3c84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.264+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84018030 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.264+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8401f3f0 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.264+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c14c530 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.264+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8401f7b0 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.264+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9c0c2230 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.265+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9c14ca50 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.265+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f3c8401d070 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.265+0000 7f3c927fc700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3c7c05edc0 0x7f3c7c061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:24.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.266+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f3c8407a6d0 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.266+0000 7f3c9a59c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3c7c05edc0 0x7f3c7c061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:24.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.266+0000 7f3c9a59c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3c7c05edc0 0x7f3c7c061280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3c8c0094d0 tx=0x7f3c8c00b040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:24.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.270+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3c9c14ca50 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.474+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c9c05efa0 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.474+0000 7f3c927fc700 1 -- 172.21.15.92:0/4178370866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c84024030 con 0x7f3c9c0b07a0 2023-05-16T07:38:24.478 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-16T07:38:24.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3c7c05edc0 msgr2=0x7f3c7c061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3c7c05edc0 0x7f3c7c061280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3c8c0094d0 tx=0x7f3c8c00b040).stop 2023-05-16T07:38:24.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 msgr2=0x7f3c9c14b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:24.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 0x7f3c9c14b360 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f3c9c0b0e70 tx=0x7f3c84015000).stop 2023-05-16T07:38:24.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 shutdown_connections 2023-05-16T07:38:24.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3c7c05edc0 0x7f3c7c061280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3c9c14cd30 0x7f3c9c0c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3c88003e40 0x7f3c9c149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 --2- 172.21.15.92:0/4178370866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9c0b07a0 0x7f3c9c14b360 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 >> 172.21.15.92:0/4178370866 conn(0x7f3c9c0caca0 msgr2=0x7f3c9c152830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 shutdown_connections 2023-05-16T07:38:24.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:24.476+0000 7f3ca0f9c700 1 -- 172.21.15.92:0/4178370866 wait complete. 2023-05-16T07:38:24.490 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-16T07:38:24.490 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-16T07:38:24.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-05-16T07:38:24.491 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-16T07:38:24.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-16T07:38:24.492 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-16T07:38:24.492 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-16T07:38:24.492 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-16T07:38:24.492 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:38:24.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.493 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-05-16T07:38:24.509 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-16T07:38:24.509 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-16T07:38:24.509 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-05-16T07:38:24.510 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.510 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-05-16T07:38:24.510 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.511 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:38:24.511 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:38:24.511 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:38:24.512 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:38:24.512 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-16T07:38:24.512 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:38:24.513 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-16T07:38:24.513 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:38:24.513 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-16T07:38:24.514 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 39850sec preferred_lft 39850sec 2023-05-16T07:38:24.514 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-05-16T07:38:24.514 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.515 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-16T07:38:24.515 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff 2023-05-16T07:38:24.515 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-16T07:38:24.516 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.516 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::6c5c:21ff:febf:ae94/64 scope link 2023-05-16T07:38:24.516 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.516 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:38:24.517 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8e:68:b4:65:71:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:38:24.517 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::8c68:b4ff:fe65:71a6/64 scope link 2023-05-16T07:38:24.517 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.518 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-16T07:38:24.518 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 12:b4:20:44:d1:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:38:24.518 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::10b4:20ff:fe44:d133/64 scope link 2023-05-16T07:38:24.519 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-05-16T07:38:24.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.520 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-16T07:38:24.578 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-05-16T07:38:24.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.581 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-05-16T07:38:24.641 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-05-16T07:38:24.642 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-16T07:38:24.642 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-16T07:38:24.642 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-16T07:38:24.643 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-16T07:38:24.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.644 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-05-16T07:38:24.645 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-05-16T07:38:24.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-16T07:38:24.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-16T07:38:24.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-16T07:38:24.647 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-05-16T07:38:24.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.744 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:38:24.762 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:38:24.762 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:38:24.763 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-16T07:38:24.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-05-16T07:38:24.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:24.828 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-05-16T07:38:24.883 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-05-16T07:38:24.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:38:24.961 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:38:24.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:38:25.004 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2023-05-16T07:38:25.005 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2023-05-16T07:38:25.007 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-16T07:38:25.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:25.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-16T07:38:25.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-05-16T07:38:25.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.219+0000 7f49586c5700 1 Processor -- start 2023-05-16T07:38:25.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.219+0000 7f49586c5700 1 -- start start 2023-05-16T07:38:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f49586c5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154170 0x7f49500c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f49586c5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4950154550 0x7f49500c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f49586c5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49500696a0 0x7f4950069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f49586c5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49500c0660 con 0x7f49500696a0 2023-05-16T07:38:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f49586c5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495006a020 con 0x7f4950154550 2023-05-16T07:38:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f49586c5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49500bfd30 con 0x7f4950154170 2023-05-16T07:38:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f4956c62700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49500696a0 0x7f4950069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f4956c62700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49500696a0 0x7f4950069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52932/0 (socket says 172.21.15.92:52932) 2023-05-16T07:38:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f4956c62700 1 -- 172.21.15.92:0/2354257176 learned_addr learned my addr 172.21.15.92:0/2354257176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:25.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f4955c60700 1 --2- 172.21.15.92:0/2354257176 >> v2:172.21.15.158:3300/0 conn(0x7f4950154550 0x7f49500c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f4956c62700 1 -- 172.21.15.92:0/2354257176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154170 msgr2=0x7f49500c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:25.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.220+0000 7f4956461700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154170 0x7f49500c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4956c62700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154170 0x7f49500c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4956c62700 1 -- 172.21.15.92:0/2354257176 >> v2:172.21.15.158:3300/0 conn(0x7f4950154550 msgr2=0x7f49500c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4956c62700 1 --2- 172.21.15.92:0/2354257176 >> v2:172.21.15.158:3300/0 conn(0x7f4950154550 0x7f49500c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4956c62700 1 -- 172.21.15.92:0/2354257176 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500c0010 con 0x7f49500696a0 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4955c60700 1 --2- 172.21.15.92:0/2354257176 >> v2:172.21.15.158:3300/0 conn(0x7f4950154550 0x7f49500c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4956461700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154170 0x7f49500c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f4956c62700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950069b30 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f4950083fe0 tx=0x7f49480090f0).ready entity=mon.0 client_cookie=30ab61924478e112 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f495545f700 1 -- 172.21.15.92:0/2354257176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800b600 con 0x7f49500696a0 2023-05-16T07:38:25.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f495545f700 1 -- 172.21.15.92:0/2354257176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494800b760 con 0x7f49500696a0 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f495545f700 1 -- 172.21.15.92:0/2354257176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800bb00 con 0x7f49500696a0 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 -- 172.21.15.92:0/2354257176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 msgr2=0x7f4950069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950069b30 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f4950083fe0 tx=0x7f49480090f0).stop 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 -- 172.21.15.92:0/2354257176 shutdown_connections 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950069b30 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 --2- 172.21.15.92:0/2354257176 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154170 0x7f49500c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 --2- 172.21.15.92:0/2354257176 >> v2:172.21.15.158:3300/0 conn(0x7f4950154550 0x7f49500c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 -- 172.21.15.92:0/2354257176 >> 172.21.15.92:0/2354257176 conn(0x7f49500cb700 msgr2=0x7f49500c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:25.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 -- 172.21.15.92:0/2354257176 shutdown_connections 2023-05-16T07:38:25.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.221+0000 7f49586c5700 1 -- 172.21.15.92:0/2354257176 wait complete. 2023-05-16T07:38:25.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.222+0000 7f49586c5700 1 Processor -- start 2023-05-16T07:38:25.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 -- start start 2023-05-16T07:38:25.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950132430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4950154170 0x7f49500aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154550 0x7f49500ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49500ab7b0 con 0x7f49500696a0 2023-05-16T07:38:25.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950145260 con 0x7f4950154170 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f49586c5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4950145540 con 0x7f4950154550 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f4956461700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950132430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f4956461700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950132430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52938/0 (socket says 172.21.15.92:52938) 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f4956461700 1 -- 172.21.15.92:0/3676946834 learned_addr learned my addr 172.21.15.92:0/3676946834 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f4956c62700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154550 0x7f49500ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.223+0000 7f4955c60700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4950154170 0x7f49500aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f4956461700 1 -- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154550 msgr2=0x7f49500ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f4956461700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154550 0x7f49500ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f4956461700 1 -- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4950154170 msgr2=0x7f49500aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f4956461700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4950154170 0x7f49500aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f4956461700 1 -- 172.21.15.92:0/3676946834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4948009370 con 0x7f49500696a0 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f4956461700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950132430 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f494c00a400 tx=0x7f494c007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c016470 con 0x7f49500696a0 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c0165d0 con 0x7f49500696a0 2023-05-16T07:38:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4950145820 con 0x7f49500696a0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c016950 con 0x7f49500696a0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.224+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950145bb0 con 0x7f49500696a0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.225+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49500666c0 con 0x7f49500696a0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.225+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f494c01d060 con 0x7f49500696a0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.226+0000 7f49467fc700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f493405ed90 0x7f4934061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.226+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f494c07aa50 con 0x7f49500696a0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.226+0000 7f4955c60700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f493405ed90 0x7f4934061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.226+0000 7f4955c60700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f493405ed90 0x7f4934061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f49500cab20 tx=0x7f4940008040).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:25.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.230+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f494c04bd50 con 0x7f49500696a0 2023-05-16T07:38:25.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.416+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f495006c350 con 0x7f493405ed90 2023-05-16T07:38:25.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.419+0000 7f49467fc700 1 -- 172.21.15.92:0/3676946834 <== mgr.7208 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f495006c350 con 0x7f493405ed90 2023-05-16T07:38:25.421 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-05-16T07:38:25.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-05-16T07:38:25.422 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-05-16T07:38:25.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-05-16T07:38:25.422 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-05-16T07:38:25.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f493405ed90 msgr2=0x7f4934061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f493405ed90 0x7f4934061250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f49500cab20 tx=0x7f4940008040).stop 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 msgr2=0x7f4950132430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950132430 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f494c00a400 tx=0x7f494c007910).stop 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 shutdown_connections 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f493405ed90 0x7f4934061250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4950154550 0x7f49500ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49500696a0 0x7f4950132430 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 --2- 172.21.15.92:0/3676946834 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4950154170 0x7f49500aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 >> 172.21.15.92:0/3676946834 conn(0x7f49500cb700 msgr2=0x7f49500ac6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:25.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 shutdown_connections 2023-05-16T07:38:25.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.420+0000 7f49586c5700 1 -- 172.21.15.92:0/3676946834 wait complete. 2023-05-16T07:38:25.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-05-16T07:38:25.449 INFO:tasks.ceph.mds.d.smithi158.stderr:2023-05-16T07:38:25.445+0000 7f5af700f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:38:25.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.545+0000 7f31da1f5700 1 Processor -- start 2023-05-16T07:38:25.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.545+0000 7f31da1f5700 1 -- start start 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31da1f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f31d40c1160 0x7f31d40c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31da1f5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f31d40c61f0 0x7f31d4154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31da1f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d406a140 con 0x7f31d4153f90 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31da1f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d4069f60 con 0x7f31d40c1160 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31da1f5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d406a9e0 con 0x7f31d40c61f0 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f31d40c1160 0x7f31d40c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d91f3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f31d4153f90 0x7f31d40a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35460/0 (socket says 172.21.15.92:35460) 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f31d40c1160 0x7f31d40c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52946/0 (socket says 172.21.15.92:52946) 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d91f3700 1 -- 172.21.15.92:0/21431215 learned_addr learned my addr 172.21.15.92:0/21431215 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 -- 172.21.15.92:0/21431215 >> v2:172.21.15.158:3300/0 conn(0x7f31d40c61f0 msgr2=0x7f31d4154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 --2- 172.21.15.92:0/21431215 >> v2:172.21.15.158:3300/0 conn(0x7f31d40c61f0 0x7f31d4154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 -- 172.21.15.92:0/21431215 >> v1:172.21.15.92:6789/0 conn(0x7f31d4153f90 legacy=0x7f31d40a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 -- 172.21.15.92:0/21431215 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d406a420 con 0x7f31d40c1160 2023-05-16T07:38:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31cbfff700 1 -- 172.21.15.92:0/21431215 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704098140 0 0) 0x7f31d406a140 con 0x7f31d4153f90 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31d89f2700 1 --2- 172.21.15.92:0/21431215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d40c1160 0x7f31d40c5d00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f31d40c1b50 tx=0x7f31c4009060).ready entity=mon.0 client_cookie=947bbf27be991d6a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31cbfff700 1 -- 172.21.15.92:0/21431215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400b580 con 0x7f31d40c1160 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.546+0000 7f31cbfff700 1 -- 172.21.15.92:0/21431215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c400b6e0 con 0x7f31d40c1160 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31cbfff700 1 -- 172.21.15.92:0/21431215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400ba60 con 0x7f31d40c1160 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 -- 172.21.15.92:0/21431215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d40c1160 msgr2=0x7f31d40c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 --2- 172.21.15.92:0/21431215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d40c1160 0x7f31d40c5d00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f31d40c1b50 tx=0x7f31c4009060).stop 2023-05-16T07:38:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 -- 172.21.15.92:0/21431215 shutdown_connections 2023-05-16T07:38:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 --2- 172.21.15.92:0/21431215 >> v2:172.21.15.158:3300/0 conn(0x7f31d40c61f0 0x7f31d4154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 --2- 172.21.15.92:0/21431215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d40c1160 0x7f31d40c5d00 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 -- 172.21.15.92:0/21431215 >> 172.21.15.92:0/21431215 conn(0x7f31d40cb600 msgr2=0x7f31d40c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 -- 172.21.15.92:0/21431215 shutdown_connections 2023-05-16T07:38:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.547+0000 7f31da1f5700 1 -- 172.21.15.92:0/21431215 wait complete. 2023-05-16T07:38:25.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.548+0000 7f31da1f5700 1 Processor -- start 2023-05-16T07:38:25.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.548+0000 7f31da1f5700 1 -- start start 2023-05-16T07:38:25.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31da1f5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31d40c1160 0x7f31d40b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31da1f5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f31d40c61f0 0x7f31d40c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31da1f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 0x7f31d41498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31da1f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40c2420 con 0x7f31d4153f90 2023-05-16T07:38:25.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31da1f5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40c2700 con 0x7f31d40c1160 2023-05-16T07:38:25.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31da1f5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40c29e0 con 0x7f31d40c61f0 2023-05-16T07:38:25.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31d99f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 0x7f31d41498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31d99f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 0x7f31d41498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52954/0 (socket says 172.21.15.92:52954) 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31d99f4700 1 -- 172.21.15.92:0/2871356200 learned_addr learned my addr 172.21.15.92:0/2871356200 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31d91f3700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31d40c1160 0x7f31d40b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.549+0000 7f31d89f2700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f31d40c61f0 0x7f31d40c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d99f4700 1 -- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f31d40c61f0 msgr2=0x7f31d40c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d99f4700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f31d40c61f0 0x7f31d40c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d99f4700 1 -- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31d40c1160 msgr2=0x7f31d40b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d99f4700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31d40c1160 0x7f31d40b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d99f4700 1 -- 172.21.15.92:0/2871356200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c40092a0 con 0x7f31d4153f90 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d89f2700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f31d40c61f0 0x7f31d40c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31d99f4700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 0x7f31d41498b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f31d000b5e0 tx=0x7f31d0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0002320 con 0x7f31d4153f90 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d0002480 con 0x7f31d4153f90 2023-05-16T07:38:25.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d40c2cc0 con 0x7f31d4153f90 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0002840 con 0x7f31d4153f90 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.550+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d4149da0 con 0x7f31d4153f90 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.551+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d4083120 con 0x7f31d4153f90 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.551+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 79) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f31d000f030 con 0x7f31d4153f90 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.551+0000 7f31c97fa700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f31b805ed60 0x7f31b8061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.551+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f31d0079b30 con 0x7f31d4153f90 2023-05-16T07:38:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.552+0000 7f31d91f3700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f31b805ed60 0x7f31b8061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.552+0000 7f31d91f3700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f31b805ed60 0x7f31b8061220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f31cc001560 tx=0x7f31cc007f30).ready entity=mgr.7208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.556+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f31d004add0 con 0x7f31d4153f90 2023-05-16T07:38:25.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:25.759+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f31d4047150 con 0x7f31d4153f90 2023-05-16T07:38:26.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.545+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 80) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f31d0002bf0 con 0x7f31d4153f90 2023-05-16T07:38:26.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.546+0000 7f31c97fa700 1 -- 172.21.15.92:0/2871356200 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v80) v1 ==== 92+0+0 (secure 0 0 0) 0x7f31d0073480 con 0x7f31d4153f90 2023-05-16T07:38:26.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f31b805ed60 msgr2=0x7f31b8061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f31b805ed60 0x7f31b8061220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f31cc001560 tx=0x7f31cc007f30).stop 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 msgr2=0x7f31d41498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 0x7f31d41498b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f31d000b5e0 tx=0x7f31d0009060).stop 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 shutdown_connections 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31d4153f90 0x7f31d41498b0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31d40c1160 0x7f31d40b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f31b805ed60 0x7f31b8061220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 --2- 172.21.15.92:0/2871356200 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f31d40c61f0 0x7f31d40c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 >> 172.21.15.92:0/2871356200 conn(0x7f31d40cb600 msgr2=0x7f31d40c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:26.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 shutdown_connections 2023-05-16T07:38:26.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.548+0000 7f31da1f5700 1 -- 172.21.15.92:0/2871356200 wait complete. 2023-05-16T07:38:26.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-05-16T07:38:26.684 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:26.680+0000 7f04baca4680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:38:26.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.681+0000 7f3f42086700 1 Processor -- start 2023-05-16T07:38:26.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f42086700 1 -- start start 2023-05-16T07:38:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f42086700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f3c154470 0x7f3f3c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f42086700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c068160 0x7f3f3c0685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f42086700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0c94d0 con 0x7f3f3c154090 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f42086700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c0cb7e0 con 0x7f3f3c154470 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f42086700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c068aa0 con 0x7f3f3c068160 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.682+0000 7f3f32ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f3c154470 0x7f3f3c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f3c154090 0x7f3f3c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35500/0 (socket says 172.21.15.92:35500) 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f32ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f3c154470 0x7f3f3c0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52984/0 (socket says 172.21.15.92:52984) 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3b7fe700 1 -- 172.21.15.92:0/558421586 learned_addr learned my addr 172.21.15.92:0/558421586 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:26.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3bfff700 1 --2- 172.21.15.92:0/558421586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c068160 0x7f3f3c0685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f32ffd700 1 -- 172.21.15.92:0/558421586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c068160 msgr2=0x7f3f3c0685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f32ffd700 1 --2- 172.21.15.92:0/558421586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c068160 0x7f3f3c0685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f32ffd700 1 -- 172.21.15.92:0/558421586 >> v1:172.21.15.92:6789/0 conn(0x7f3f3c154090 legacy=0x7f3f3c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3affd700 1 -- 172.21.15.92:0/558421586 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280617235 0 0) 0x7f3f3c0c94d0 con 0x7f3f3c154090 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f32ffd700 1 -- 172.21.15.92:0/558421586 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0c88c0 con 0x7f3f3c154470 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f32ffd700 1 --2- 172.21.15.92:0/558421586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c154470 0x7f3f3c0aaca0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f3f24009760 tx=0x7f3f24009060).ready entity=mon.0 client_cookie=db564b690f8bf8d3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3affd700 1 -- 172.21.15.92:0/558421586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400f510 con 0x7f3f3c154470 2023-05-16T07:38:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3affd700 1 -- 172.21.15.92:0/558421586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2400f670 con 0x7f3f3c154470 2023-05-16T07:38:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.683+0000 7f3f3affd700 1 -- 172.21.15.92:0/558421586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400f9f0 con 0x7f3f3c154470 2023-05-16T07:38:26.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 -- 172.21.15.92:0/558421586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c154470 msgr2=0x7f3f3c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 --2- 172.21.15.92:0/558421586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c154470 0x7f3f3c0aaca0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f3f24009760 tx=0x7f3f24009060).stop 2023-05-16T07:38:26.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 -- 172.21.15.92:0/558421586 shutdown_connections 2023-05-16T07:38:26.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 --2- 172.21.15.92:0/558421586 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c068160 0x7f3f3c0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 --2- 172.21.15.92:0/558421586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c154470 0x7f3f3c0aaca0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 -- 172.21.15.92:0/558421586 >> 172.21.15.92:0/558421586 conn(0x7f3f3c067920 msgr2=0x7f3f3c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 -- 172.21.15.92:0/558421586 shutdown_connections 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.684+0000 7f3f42086700 1 -- 172.21.15.92:0/558421586 wait complete. 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.685+0000 7f3f42086700 1 Processor -- start 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.685+0000 7f3f42086700 1 -- start start 2023-05-16T07:38:26.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 0x7f3f3c14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c154090 0x7f3f3c156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f3c154470 0x7f3f3c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c1574c0 con 0x7f3f3c068160 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c164610 con 0x7f3f3c154470 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c1648f0 con 0x7f3f3c154090 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 0x7f3f3c14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:26.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 0x7f3f3c14e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52988/0 (socket says 172.21.15.92:52988) 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 -- 172.21.15.92:0/1882397244 learned_addr learned my addr 172.21.15.92:0/1882397244 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f32ffd700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c154090 0x7f3f3c156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3bfff700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f3c154470 0x7f3f3c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 -- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c154090 msgr2=0x7f3f3c156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c154090 0x7f3f3c156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 -- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f3c154470 msgr2=0x7f3f3c156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f3c154470 0x7f3f3c156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 -- 172.21.15.92:0/1882397244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240092a0 con 0x7f3f3c068160 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f32ffd700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c154090 0x7f3f3c156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f3b7fe700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 0x7f3f3c14e7b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f3f2c00c4e0 tx=0x7f3f2c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0072e0 con 0x7f3f3c068160 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c007440 con 0x7f3f3c068160 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c164bd0 con 0x7f3f3c068160 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0057a0 con 0x7f3f3c068160 2023-05-16T07:38:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.686+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c14ef00 con 0x7f3f3c068160 2023-05-16T07:38:26.696 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:26.687+0000 7fa751f76680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.687+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c1650f0 con 0x7f3f3c068160 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.688+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 80) v1 ==== 75259+0+0 (secure 0 0 0) 0x7f3f2c01a060 con 0x7f3f3c068160 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.688+0000 7f3f33fff700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f1c05ede0 0x7f3f1c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.688+0000 7f3f32ffd700 1 -- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f1c05ede0 msgr2=0x7f3f1c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.688+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f3f2c079730 con 0x7f3f3c068160 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.688+0000 7f3f32ffd700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f1c05ede0 0x7f3f1c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.692+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f3f3c1650f0 con 0x7f3f3c068160 2023-05-16T07:38:26.746 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:26.742+0000 7f04baca4680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:38:26.755 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:26.752+0000 7fa751f76680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:38:26.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.865+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3f3c066730 con 0x7f3f3c068160 2023-05-16T07:38:26.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.870+0000 7f3f33fff700 1 -- 172.21.15.92:0/1882397244 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f2c04aa30 con 0x7f3f3c068160 2023-05-16T07:38:26.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f1c05ede0 msgr2=0x7f3f1c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:26.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f1c05ede0 0x7f3f1c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:26.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 msgr2=0x7f3f3c14e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 0x7f3f3c14e7b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f3f2c00c4e0 tx=0x7f3f2c007c70).stop 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 shutdown_connections 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f3c154470 0x7f3f3c156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c068160 0x7f3f3c14e7b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f3f1c05ede0 0x7f3f1c0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 --2- 172.21.15.92:0/1882397244 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3f3c154090 0x7f3f3c156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 >> 172.21.15.92:0/1882397244 conn(0x7f3f3c067920 msgr2=0x7f3f3c0acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:26.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 shutdown_connections 2023-05-16T07:38:26.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:26.872+0000 7f3f42086700 1 -- 172.21.15.92:0/1882397244 wait complete. 2023-05-16T07:38:26.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:26.888 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-05-16T07:38:26.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-05-16T07:38:27.018 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:27.014+0000 7f04baca4680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:38:27.039 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:27.035+0000 7fa751f76680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:38:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.064+0000 7fe64f9ef700 1 Processor -- start 2023-05-16T07:38:27.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.064+0000 7fe64f9ef700 1 -- start start 2023-05-16T07:38:27.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64f9ef700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fe64814ffa0 0x7fe648150380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:27.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64f9ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bd810 0x7fe648153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:27.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64f9ef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6480bc770 con 0x7fe6480bc290 2023-05-16T07:38:27.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64f9ef700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64806a510 con 0x7fe64814ffa0 2023-05-16T07:38:27.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64f9ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480bca50 con 0x7fe6480bd810 2023-05-16T07:38:27.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64cf8a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe6480bc290 0x7fe648069e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35556/0 (socket says 172.21.15.92:35556) 2023-05-16T07:38:27.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64cf8a700 1 -- 172.21.15.92:0/3302623070 learned_addr learned my addr 172.21.15.92:0/3302623070 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64d78b700 1 --2- 172.21.15.92:0/3302623070 >> v2:172.21.15.158:3300/0 conn(0x7fe64814ffa0 0x7fe648150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64df8c700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bd810 0x7fe648153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe63ffff700 1 -- 172.21.15.92:0/3302623070 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318063680 0 0) 0x7fe6480bc770 con 0x7fe6480bc290 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe63ffff700 1 -- 172.21.15.92:0/3302623070 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6340033f0 con 0x7fe6480bc290 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64d78b700 1 -- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bd810 msgr2=0x7fe648153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64d78b700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bd810 0x7fe648153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64d78b700 1 -- 172.21.15.92:0/3302623070 >> v1:172.21.15.92:6789/0 conn(0x7fe6480bc290 legacy=0x7fe648069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe63ffff700 1 -- 172.21.15.92:0/3302623070 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3849579192 0 0) 0x7fe6340033f0 con 0x7fe6480bc290 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64d78b700 1 -- 172.21.15.92:0/3302623070 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480bcd30 con 0x7fe64814ffa0 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.065+0000 7fe64df8c700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bd810 0x7fe648153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64d78b700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe64814ffa0 0x7fe648150380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fe644009760 tx=0x7fe644009060).ready entity=mon.1 client_cookie=133715273dbfa948 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe63ffff700 1 -- 172.21.15.92:0/3302623070 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638001470 con 0x7fe64814ffa0 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe63ffff700 1 -- 172.21.15.92:0/3302623070 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe644011400 con 0x7fe64814ffa0 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe63ffff700 1 -- 172.21.15.92:0/3302623070 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644011780 con 0x7fe64814ffa0 2023-05-16T07:38:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 -- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe64814ffa0 msgr2=0x7fe648150380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe64814ffa0 0x7fe648150380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fe644009760 tx=0x7fe644009060).stop 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 -- 172.21.15.92:0/3302623070 shutdown_connections 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bd810 0x7fe648153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/3302623070 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe64814ffa0 0x7fe648150380 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 -- 172.21.15.92:0/3302623070 >> 172.21.15.92:0/3302623070 conn(0x7fe6480c4620 msgr2=0x7fe648066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 -- 172.21.15.92:0/3302623070 shutdown_connections 2023-05-16T07:38:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.066+0000 7fe64f9ef700 1 -- 172.21.15.92:0/3302623070 wait complete. 2023-05-16T07:38:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.067+0000 7fe64f9ef700 1 Processor -- start 2023-05-16T07:38:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.067+0000 7fe64f9ef700 1 -- start start 2023-05-16T07:38:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.068+0000 7fe64f9ef700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 0x7fe6480aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.068+0000 7fe64f9ef700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 0x7fe648151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.068+0000 7fe64f9ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64814ffa0 0x7fe6481523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.068+0000 7fe64f9ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe648151a90 con 0x7fe64814ffa0 2023-05-16T07:38:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.068+0000 7fe64f9ef700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64814f890 con 0x7fe6480bd810 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.068+0000 7fe64f9ef700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480abef0 con 0x7fe6480bc290 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64cf8a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 0x7fe648151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64d78b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 0x7fe6480aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64d78b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 0x7fe6480aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:45618/0 (socket says 172.21.15.92:45618) 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64cf8a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 0x7fe648151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:56684/0 (socket says 172.21.15.92:56684) 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64814ffa0 0x7fe6481523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64d78b700 1 -- 172.21.15.92:0/1731762216 learned_addr learned my addr 172.21.15.92:0/1731762216 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:27.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 msgr2=0x7fe6480aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 0x7fe6480aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 msgr2=0x7fe648151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 0x7fe648151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 -- 172.21.15.92:0/1731762216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe640009a20 con 0x7fe64814ffa0 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 0x7fe6480aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64cf8a700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 0x7fe648151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:27.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64df8c700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64814ffa0 0x7fe6481523d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fe640009f40 tx=0x7fe64000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000cb70 con 0x7fe64814ffa0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64000f040 con 0x7fe64814ffa0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6440092a0 con 0x7fe64814ffa0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000cee0 con 0x7fe64814ffa0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.069+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6480ac420 con 0x7fe64814ffa0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.070+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe648151860 con 0x7fe64814ffa0 2023-05-16T07:38:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.071+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 80) v1 ==== 75259+0+0 (secure 0 0 0) 0x7fe640013610 con 0x7fe64814ffa0 2023-05-16T07:38:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.072+0000 7fe63d7fa700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.072+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fe64007b0c0 con 0x7fe64814ffa0 2023-05-16T07:38:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.072+0000 7fe64d78b700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.072+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.075+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe64004c3e0 con 0x7fe64814ffa0 2023-05-16T07:38:27.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.273+0000 7fe64d78b700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:27.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.273+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.273+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe648151bd0 con 0x7fe62c05ede0 2023-05-16T07:38:27.635 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:27.624+0000 7f04baca4680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:38:27.655 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:27.652+0000 7fa751f76680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:38:27.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.674+0000 7fe64d78b700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:27.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:27.674+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:38:27.699 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:27.695+0000 7f04baca4680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:38:27.727 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:27.724+0000 7fa751f76680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:38:27.882 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:27.878+0000 7f04baca4680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:38:27.910 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:27.906+0000 7fa751f76680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:38:28.295 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:28.291+0000 7f04baca4680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:38:28.421 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:28.317+0000 7fa751f76680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:38:28.421 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:28.392+0000 7f04baca4680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:28.422 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:28.418+0000 7fa751f76680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:28.475+0000 7fe64d78b700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:28.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:28.475+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-05-16T07:38:28.548 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:28.544+0000 7f04baca4680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:28.572 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:28.569+0000 7fa751f76680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:28.621 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:28.617+0000 7f04baca4680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:38:28.645 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:28.642+0000 7fa751f76680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:38:28.687 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:28.683+0000 7f04baca4680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:38:28.711 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:28.708+0000 7fa751f76680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:38:28.940 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:28.936+0000 7f04baca4680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:38:28.964 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:28.961+0000 7fa751f76680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:38:29.120 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:29.116+0000 7f04baca4680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:29.144 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:29.141+0000 7fa751f76680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:29.760 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:29.756+0000 7f04baca4680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:38:29.779 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:29.775+0000 7fa751f76680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:38:29.827 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:29.823+0000 7f04baca4680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:38:29.846 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:29.842+0000 7fa751f76680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:38:29.900 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:29.897+0000 7f04baca4680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:38:29.919 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:29.916+0000 7fa751f76680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:38:30.044 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.040+0000 7f04baca4680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:38:30.062 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:30.059+0000 7fa751f76680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:38:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:30.077+0000 7fe64d78b700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:30.077+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-05-16T07:38:30.113 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.109+0000 7f04baca4680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:38:30.131 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:30.128+0000 7fa751f76680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:38:30.225 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.221+0000 7f04baca4680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:38:30.243 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:30.240+0000 7fa751f76680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:38:30.332 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.328+0000 7f04baca4680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:30.349 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:30.346+0000 7fa751f76680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:30.622 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.618+0000 7f04baca4680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:38:30.634 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:30.631+0000 7fa751f76680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:38:30.691 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.688+0000 7f04baca4680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:38:30.703 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:30.700+0000 7fa751f76680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:38:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:30.758+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 81) v1 ==== 75109+0+0 (secure 0 0 0) 0x7fe640078da0 con 0x7fe64814ffa0 2023-05-16T07:38:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:30.759+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:30.759+0000 7fe63d7fa700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:30.824 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0436480700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.824 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0436480700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0436480700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0436480700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0436480700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0439486700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0439486700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.825 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0439486700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.826 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0439486700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:30.826 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:30.820+0000 7f0439486700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.776+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 82) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fe640079120 con 0x7fe64814ffa0 2023-05-16T07:38:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.776+0000 7fe63d7fa700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.776+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe648151bd0 con 0x7fe62c05ede0 2023-05-16T07:38:31.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.779+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.781+0000 7fe64d78b700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c061220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fe6480665c0 tx=0x7fe644016000).ready entity=mgr.8242 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:31.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.797+0000 7fe63d7fa700 1 -- 172.21.15.92:0/1731762216 <== mgr.8242 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe648151bd0 con 0x7fe62c05ede0 2023-05-16T07:38:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule set for path snap_test_dir1 2023-05-16T07:38:31.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 msgr2=0x7fe62c061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c061220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fe6480665c0 tx=0x7fe644016000).stop 2023-05-16T07:38:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64814ffa0 msgr2=0x7fe6481523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64814ffa0 0x7fe6481523d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fe640009f40 tx=0x7fe64000b040).stop 2023-05-16T07:38:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 shutdown_connections 2023-05-16T07:38:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe62c05ede0 0x7fe62c061220 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe6480bc290 0x7fe6480aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64814ffa0 0x7fe6481523d0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 --2- 172.21.15.92:0/1731762216 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe6480bd810 0x7fe648151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 >> 172.21.15.92:0/1731762216 conn(0x7fe6480c4620 msgr2=0x7fe648066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 shutdown_connections 2023-05-16T07:38:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.799+0000 7fe64f9ef700 1 -- 172.21.15.92:0/1731762216 wait complete. 2023-05-16T07:38:31.817 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-05-16T07:38:31.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-05-16T07:38:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.934+0000 7f5721e61700 1 Processor -- start 2023-05-16T07:38:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.934+0000 7f5721e61700 1 -- start start 2023-05-16T07:38:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5721e61700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 0x7f571c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5721e61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c06a140 con 0x7f571c155380 2023-05-16T07:38:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5721e61700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c06a420 con 0x7f571c0c5d00 2023-05-16T07:38:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5721e61700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c069f60 con 0x7f571c153f90 2023-05-16T07:38:31.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5721660700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f571c155380 0x7f571c0c0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35700/0 (socket says 172.21.15.92:35700) 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5721660700 1 -- 172.21.15.92:0/3727620115 learned_addr learned my addr 172.21.15.92:0/3727620115 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f5720e5f700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 0x7f571c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205451138 0 0) 0x7f571c06a140 con 0x7f571c155380 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.935+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57080033f0 con 0x7f571c155380 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3404733342 0 0) 0x7f57080033f0 con 0x7f571c155380 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 msgr2=0x7f571c0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 0x7f571c0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 >> v1:172.21.15.158:6789/0 conn(0x7f571c0c5d00 legacy=0x7f571c0c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c06a700 con 0x7f571c155380 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f5714001470 con 0x7f571c155380 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 >> v1:172.21.15.92:6789/0 conn(0x7f571c155380 legacy=0x7f571c0c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 0x7f5708003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5708003e40 0x7f5708004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5708004b80 0x7f5708005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c069f60 con 0x7f5708003e40 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570800b990 con 0x7f5708004b80 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570800bb50 con 0x7f571c153f90 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5708003e40 0x7f5708004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f5720e5f700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 0x7f5708003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f5721660700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5708004b80 0x7f5708005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 -- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 msgr2=0x7f5708003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f571c153f90 0x7f5708003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 -- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5708004b80 msgr2=0x7f5708005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5708004b80 0x7f5708005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 -- 172.21.15.92:0/3727620115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080033f0 con 0x7f5708003e40 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f5721660700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5708004b80 0x7f5708005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571bfff700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5708003e40 0x7f5708004690 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7f5704001700 tx=0x7f5704007bb0).ready entity=mon.0 client_cookie=32dd04d0cd5b0ee3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704007020 con 0x7f5708003e40 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704007180 con 0x7f5708003e40 2023-05-16T07:38:31.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.936+0000 7f571b7fe700 1 -- 172.21.15.92:0/3727620115 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704007500 con 0x7f5708003e40 2023-05-16T07:38:31.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721e61700 1 -- 172.21.15.92:0/3727620115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5708003e40 msgr2=0x7f5708004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721e61700 1 --2- 172.21.15.92:0/3727620115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5708003e40 0x7f5708004690 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7f5704001700 tx=0x7f5704007bb0).stop 2023-05-16T07:38:31.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721660700 1 -- 172.21.15.92:0/3727620115 reap_dead start 2023-05-16T07:38:31.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721e61700 1 -- 172.21.15.92:0/3727620115 shutdown_connections 2023-05-16T07:38:31.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721e61700 1 -- 172.21.15.92:0/3727620115 >> 172.21.15.92:0/3727620115 conn(0x7f571c0cb600 msgr2=0x7f571c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:31.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721e61700 1 -- 172.21.15.92:0/3727620115 shutdown_connections 2023-05-16T07:38:31.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.937+0000 7f5721e61700 1 -- 172.21.15.92:0/3727620115 wait complete. 2023-05-16T07:38:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.938+0000 7f5721e61700 1 Processor -- start 2023-05-16T07:38:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.938+0000 7f5721e61700 1 -- start start 2023-05-16T07:38:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721e61700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5708003e40 0x7f571c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721e61700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f571c0c42f0 0x7f571c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721e61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 0x7f571c0682c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721e61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0aed20 con 0x7f571c0b0120 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721e61700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ae110 con 0x7f571c0c42f0 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721e61700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ae3f0 con 0x7f5708003e40 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 0x7f571c0682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 0x7f571c0682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53198/0 (socket says 172.21.15.92:53198) 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 -- 172.21.15.92:0/3610310815 learned_addr learned my addr 172.21.15.92:0/3610310815 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:31.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5720e5f700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5708003e40 0x7f571c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 -- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5708003e40 msgr2=0x7f571c0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5708003e40 0x7f571c0adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 -- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f571c0c42f0 msgr2=0x7f571c0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f571bfff700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f571c0c42f0 0x7f571c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f571c0c42f0 0x7f571c0afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 -- 172.21.15.92:0/3610310815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0ae6d0 con 0x7f571c0b0120 2023-05-16T07:38:31.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5720e5f700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5708003e40 0x7f571c0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5721660700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 0x7f571c0682c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f5714000c00 tx=0x7f571400eba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.939+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714001d90 con 0x7f571c0b0120 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.940+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714010520 con 0x7f571c0b0120 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.940+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140108c0 con 0x7f571c0b0120 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.940+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c0ae9b0 con 0x7f571c0b0120 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.940+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c0687b0 con 0x7f571c0b0120 2023-05-16T07:38:31.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.940+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c0aeed0 con 0x7f571c0b0120 2023-05-16T07:38:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.941+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 82) v1 ==== 75179+0+0 (secure 0 0 0) 0x7f5714010c30 con 0x7f571c0b0120 2023-05-16T07:38:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.941+0000 7f5718ff9700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f56fc05ed40 0x7f56fc061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.941+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f5714079930 con 0x7f571c0b0120 2023-05-16T07:38:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.941+0000 7f5720e5f700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f56fc05ed40 0x7f56fc061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.942+0000 7f5720e5f700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f56fc05ed40 0x7f56fc061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f570c0099f0 tx=0x7f570c00b040).ready entity=mgr.8242 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:31.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:31.945+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f57140498f0 con 0x7f571c0b0120 2023-05-16T07:38:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.130+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f571c083120 con 0x7f56fc05ed40 2023-05-16T07:38:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.136+0000 7f5718ff9700 1 -- 172.21.15.92:0/3610310815 <== mgr.8242 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f571c083120 con 0x7f56fc05ed40 2023-05-16T07:38:32.139 INFO:teuthology.orchestra.run.smithi092.stdout:Retention added to path snap_test_dir1 2023-05-16T07:38:32.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f56fc05ed40 msgr2=0x7f56fc061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f56fc05ed40 0x7f56fc061200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f570c0099f0 tx=0x7f570c00b040).stop 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 msgr2=0x7f571c0682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 0x7f571c0682c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f5714000c00 tx=0x7f571400eba0).stop 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 shutdown_connections 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f56fc05ed40 0x7f56fc061200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5708003e40 0x7f571c0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f571c0b0120 0x7f571c0682c0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 --2- 172.21.15.92:0/3610310815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f571c0c42f0 0x7f571c0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.137+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 >> 172.21.15.92:0/3610310815 conn(0x7f571c0cb600 msgr2=0x7f571c153610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:32.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.138+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 shutdown_connections 2023-05-16T07:38:32.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.138+0000 7f5721e61700 1 -- 172.21.15.92:0/3610310815 wait complete. 2023-05-16T07:38:32.153 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-05-16T07:38:32.157 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-05-16-07_39_00 in ~28s... 2023-05-16T07:38:32.157 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-05-16T07:38:32.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-05-16T07:38:32.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.277+0000 7fd57f948700 1 Processor -- start 2023-05-16T07:38:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.277+0000 7fd57f948700 1 -- start start 2023-05-16T07:38:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57f948700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fd578068870 0x7fd5780aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57f948700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5780c94d0 con 0x7fd578154090 2023-05-16T07:38:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57f948700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5780cb7e0 con 0x7fd578154470 2023-05-16T07:38:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57f948700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780c88c0 con 0x7fd578068870 2023-05-16T07:38:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57d6e4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd578154090 0x7fd5780a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35718/0 (socket says 172.21.15.92:35718) 2023-05-16T07:38:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57d6e4700 1 -- 172.21.15.92:0/918238645 learned_addr learned my addr 172.21.15.92:0/918238645 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd57dee5700 1 --2- 172.21.15.92:0/918238645 >> v2:172.21.15.158:3300/0 conn(0x7fd578068870 0x7fd5780aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.278+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970082497 0 0) 0x7fd5780c94d0 con 0x7fd578154090 2023-05-16T07:38:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5600033f0 con 0x7fd578154090 2023-05-16T07:38:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54861456 0 0) 0x7fd5600033f0 con 0x7fd578154090 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 >> v2:172.21.15.158:3300/0 conn(0x7fd578068870 msgr2=0x7fd5780aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 --2- 172.21.15.92:0/918238645 >> v2:172.21.15.158:3300/0 conn(0x7fd578068870 0x7fd5780aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 >> v1:172.21.15.158:6789/0 conn(0x7fd578154470 legacy=0x7fd578068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780c8ba0 con 0x7fd578154090 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fd5680017f0 con 0x7fd578154090 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 >> v1:172.21.15.92:6789/0 conn(0x7fd578154090 legacy=0x7fd5780a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5600038e0 0x7fd560003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd560004690 0x7fd560004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd560005000 0x7fd56000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780c88c0 con 0x7fd560005000 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56000bf60 con 0x7fd560004690 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56000c0f0 con 0x7fd5600038e0 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd560005000 0x7fd56000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57d6e4700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5600038e0 0x7fd560003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 -- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5600038e0 msgr2=0x7fd560003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd5600038e0 0x7fd560003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 -- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd560004690 msgr2=0x7fd560004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd560004690 0x7fd560004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 -- 172.21.15.92:0/918238645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600033f0 con 0x7fd560005000 2023-05-16T07:38:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 -- 172.21.15.92:0/918238645 reap_dead start 2023-05-16T07:38:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd57dee5700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd560005000 0x7fd56000b8e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7fd570009a80 tx=0x7fd570009060).ready entity=mon.0 client_cookie=1f42878c149875d5 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570009430 con 0x7fd560005000 2023-05-16T07:38:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000e640 con 0x7fd560005000 2023-05-16T07:38:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.279+0000 7fd577fff700 1 -- 172.21.15.92:0/918238645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000ea00 con 0x7fd560005000 2023-05-16T07:38:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 -- 172.21.15.92:0/918238645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd560005000 msgr2=0x7fd56000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd560005000 0x7fd56000b8e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7fd570009a80 tx=0x7fd570009060).stop 2023-05-16T07:38:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 -- 172.21.15.92:0/918238645 shutdown_connections 2023-05-16T07:38:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 --2- 172.21.15.92:0/918238645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd560005000 0x7fd56000b8e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 -- 172.21.15.92:0/918238645 >> 172.21.15.92:0/918238645 conn(0x7fd578067920 msgr2=0x7fd5780cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 -- 172.21.15.92:0/918238645 shutdown_connections 2023-05-16T07:38:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.280+0000 7fd57f948700 1 -- 172.21.15.92:0/918238645 wait complete. 2023-05-16T07:38:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.281+0000 7fd57f948700 1 Processor -- start 2023-05-16T07:38:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.281+0000 7fd57f948700 1 -- start start 2023-05-16T07:38:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57f948700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd560005000 0x7fd578154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57f948700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5780aeb00 0x7fd578157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57f948700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 0x7fd57815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57f948700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578156a00 con 0x7fd578164850 2023-05-16T07:38:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57f948700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578156f80 con 0x7fd5780aeb00 2023-05-16T07:38:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57f948700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781657f0 con 0x7fd560005000 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57dee5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 0x7fd57815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57dee5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 0x7fd57815a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53218/0 (socket says 172.21.15.92:53218) 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57dee5700 1 -- 172.21.15.92:0/1350082661 learned_addr learned my addr 172.21.15.92:0/1350082661 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57d6e4700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd560005000 0x7fd578154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.282+0000 7fd57cee3700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5780aeb00 0x7fd578157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57dee5700 1 -- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd560005000 msgr2=0x7fd578154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57dee5700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd560005000 0x7fd578154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57dee5700 1 -- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5780aeb00 msgr2=0x7fd578157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57dee5700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5780aeb00 0x7fd578157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57dee5700 1 -- 172.21.15.92:0/1350082661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578165230 con 0x7fd578164850 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57dee5700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 0x7fd57815a250 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fd570008000 tx=0x7fd570019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570009ec0 con 0x7fd578164850 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570009430 con 0x7fd578164850 2023-05-16T07:38:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570025670 con 0x7fd578164850 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd578165510 con 0x7fd578164850 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.283+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578155430 con 0x7fd578164850 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.284+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd578157260 con 0x7fd578164850 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.284+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 82) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fd57002d070 con 0x7fd578164850 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.285+0000 7fd5757fa700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd56405ed40 0x7fd564061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.285+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fd57007afc0 con 0x7fd578164850 2023-05-16T07:38:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.285+0000 7fd57d6e4700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd56405ed40 0x7fd564061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.285+0000 7fd57d6e4700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd56405ed40 0x7fd564061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd578167d40 tx=0x7fd568001370).ready entity=mgr.8242 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:32.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.289+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fd57004c2b0 con 0x7fd578164850 2023-05-16T07:38:32.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.475+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd578066730 con 0x7fd56405ed40 2023-05-16T07:38:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.477+0000 7fd5757fa700 1 -- 172.21.15.92:0/1350082661 <== mgr.8242 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fd578066730 con 0x7fd56405ed40 2023-05-16T07:38:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:32.481 INFO:teuthology.orchestra.run.smithi092.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd56405ed40 msgr2=0x7fd564061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd56405ed40 0x7fd564061200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd578167d40 tx=0x7fd568001370).stop 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 msgr2=0x7fd57815a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 0x7fd57815a250 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fd570008000 tx=0x7fd570019000).stop 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 shutdown_connections 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fd56405ed40 0x7fd564061200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fd560005000 0x7fd578154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd578164850 0x7fd57815a250 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 --2- 172.21.15.92:0/1350082661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd5780aeb00 0x7fd578157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 >> 172.21.15.92:0/1350082661 conn(0x7fd578067920 msgr2=0x7fd5780c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 shutdown_connections 2023-05-16T07:38:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.479+0000 7fd57f948700 1 -- 172.21.15.92:0/1350082661 wait complete. 2023-05-16T07:38:32.494 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-05-16T07:38:32.495 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-05-16T07:38:32.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-05-16T07:38:32.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.607+0000 7fa4a1671700 1 Processor -- start 2023-05-16T07:38:32.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.607+0000 7fa4a1671700 1 -- start start 2023-05-16T07:38:32.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa4a1671700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa49c14e620 0x7fa49c0c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa4a1671700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c0c4610 0x7fa49c0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa4a1671700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c065bf0 con 0x7fa49c14ea00 2023-05-16T07:38:32.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa4a1671700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c065ed0 con 0x7fa49c14e620 2023-05-16T07:38:32.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa4a1671700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0661b0 con 0x7fa49c0c4610 2023-05-16T07:38:32.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa49bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa49c14e620 0x7fa49c0c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa49bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa49c14e620 0x7fa49c0c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53222/0 (socket says 172.21.15.92:53222) 2023-05-16T07:38:32.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa49bfff700 1 -- 172.21.15.92:0/4163892739 learned_addr learned my addr 172.21.15.92:0/4163892739 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:32.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.608+0000 7fa49b7fe700 1 --1- 172.21.15.92:0/4163892739 >> v1:172.21.15.92:6789/0 conn(0x7fa49c14ea00 0x7fa49c154760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35742/0 (socket says 172.21.15.92:35742) 2023-05-16T07:38:32.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a0e70700 1 --2- 172.21.15.92:0/4163892739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c0c4610 0x7fa49c0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49bfff700 1 -- 172.21.15.92:0/4163892739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c0c4610 msgr2=0x7fa49c0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:32.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49bfff700 1 --2- 172.21.15.92:0/4163892739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c0c4610 0x7fa49c0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49bfff700 1 -- 172.21.15.92:0/4163892739 >> v1:172.21.15.92:6789/0 conn(0x7fa49c14ea00 legacy=0x7fa49c154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49bfff700 1 -- 172.21.15.92:0/4163892739 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c066490 con 0x7fa49c14e620 2023-05-16T07:38:32.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49bfff700 1 --2- 172.21.15.92:0/4163892739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c0c4120 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fa48c006fd0 tx=0x7fa48c00b3b0).ready entity=mon.0 client_cookie=a3201f5e97592155 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:32.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49affd700 1 -- 172.21.15.92:0/4163892739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00db40 con 0x7fa49c14e620 2023-05-16T07:38:32.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49affd700 1 -- 172.21.15.92:0/4163892739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c00dca0 con 0x7fa49c14e620 2023-05-16T07:38:32.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa49affd700 1 -- 172.21.15.92:0/4163892739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c010680 con 0x7fa49c14e620 2023-05-16T07:38:32.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 -- 172.21.15.92:0/4163892739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 msgr2=0x7fa49c0c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 --2- 172.21.15.92:0/4163892739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c0c4120 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fa48c006fd0 tx=0x7fa48c00b3b0).stop 2023-05-16T07:38:32.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 -- 172.21.15.92:0/4163892739 shutdown_connections 2023-05-16T07:38:32.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 --2- 172.21.15.92:0/4163892739 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c0c4610 0x7fa49c0c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 --2- 172.21.15.92:0/4163892739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c0c4120 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 -- 172.21.15.92:0/4163892739 >> 172.21.15.92:0/4163892739 conn(0x7fa49c0c7a80 msgr2=0x7fa49c0c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:32.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 -- 172.21.15.92:0/4163892739 shutdown_connections 2023-05-16T07:38:32.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.609+0000 7fa4a1671700 1 -- 172.21.15.92:0/4163892739 wait complete. 2023-05-16T07:38:32.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.610+0000 7fa4a1671700 1 Processor -- start 2023-05-16T07:38:32.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 -- start start 2023-05-16T07:38:32.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa49c0c4610 0x7fa49c13bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c14a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c14ea00 0x7fa49c14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c13bfb0 con 0x7fa49c14e620 2023-05-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c14cbe0 con 0x7fa49c0c4610 2023-05-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.611+0000 7fa4a1671700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c147030 con 0x7fa49c14ea00 2023-05-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c14a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c14a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53230/0 (socket says 172.21.15.92:53230) 2023-05-16T07:38:32.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2799405065 learned_addr learned my addr 172.21.15.92:0/2799405065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:32.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49bfff700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa49c0c4610 0x7fa49c13bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa4a0e70700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c14ea00 0x7fa49c14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c14ea00 msgr2=0x7fa49c14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c14ea00 0x7fa49c14d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa49c0c4610 msgr2=0x7fa49c13bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa49c0c4610 0x7fa49c13bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 -- 172.21.15.92:0/2799405065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c00b610 con 0x7fa49c14e620 2023-05-16T07:38:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa49b7fe700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c14a210 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7fa490007fd0 tx=0x7fa490007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900072e0 con 0x7fa49c14e620 2023-05-16T07:38:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490007440 con 0x7fa49c14e620 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c147310 con 0x7fa49c14e620 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900058f0 con 0x7fa49c14e620 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.612+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c14a700 con 0x7fa49c14e620 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.613+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c14cd70 con 0x7fa49c14e620 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.613+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 82) v1 ==== 75179+0+0 (secure 0 0 0) 0x7fa49001a060 con 0x7fa49c14e620 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.614+0000 7fa483fff700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa48805ed40 0x7fa488061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:32.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.614+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fa4900794b0 con 0x7fa49c14e620 2023-05-16T07:38:32.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.614+0000 7fa49bfff700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa48805ed40 0x7fa488061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:32.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.615+0000 7fa49bfff700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa48805ed40 0x7fa488061200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa48c00bf90 tx=0x7fa48c00b3b0).ready entity=mgr.8242 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:32.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.618+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fa49004a7a0 con 0x7fa49c14e620 2023-05-16T07:38:32.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.780+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 83) v1 ==== 75379+0+0 (secure 0 0 0) 0x7fa490010070 con 0x7fa49c14e620 2023-05-16T07:38:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.810+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa49c05e180 con 0x7fa48805ed40 2023-05-16T07:38:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.815+0000 7fa483fff700 1 -- 172.21.15.92:0/2799405065 <== mgr.8242 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa49c05e180 con 0x7fa48805ed40 2023-05-16T07:38:32.818 INFO:teuthology.orchestra.run.smithi092.stdout:Schedule removed for path snap_test_dir1 2023-05-16T07:38:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa48805ed40 msgr2=0x7fa488061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa48805ed40 0x7fa488061200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa48c00bf90 tx=0x7fa48c00b3b0).stop 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 msgr2=0x7fa49c14a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c14a210 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7fa490007fd0 tx=0x7fa490007df0).stop 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 shutdown_connections 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fa48805ed40 0x7fa488061200 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fa49c14ea00 0x7fa49c14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa49c0c4610 0x7fa49c13bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 --2- 172.21.15.92:0/2799405065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa49c14e620 0x7fa49c14a210 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 >> 172.21.15.92:0/2799405065 conn(0x7fa49c0c7a80 msgr2=0x7fa49c068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 shutdown_connections 2023-05-16T07:38:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:32.817+0000 7fa4a1671700 1 -- 172.21.15.92:0/2799405065 wait complete. 2023-05-16T07:38:32.832 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-05-16T07:38:32.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:32.833 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-05-16T07:38:32.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:32.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-05-16T07:38:32.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-05-16T07:38:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.054+0000 7ff34b44a700 1 Processor -- start 2023-05-16T07:38:33.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.055+0000 7ff34b44a700 1 -- start start 2023-05-16T07:38:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34b44a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff344153f90 0x7ff3440a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34b44a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff3440c5d00 0x7ff3440c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34b44a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3440c6670 con 0x7ff3440c0150 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34b44a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff344069f60 con 0x7ff3440c5d00 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34b44a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406a9e0 con 0x7ff344153f90 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff3440c5d00 0x7ff3440c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff3440c5d00 0x7ff3440c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53238/0 (socket says 172.21.15.92:53238) 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 -- 172.21.15.92:0/596362288 learned_addr learned my addr 172.21.15.92:0/596362288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:33.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34a448700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff344153f90 0x7ff3440a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 -- 172.21.15.92:0/596362288 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff344153f90 msgr2=0x7ff3440a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff344153f90 0x7ff3440a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 -- 172.21.15.92:0/596362288 >> v1:172.21.15.158:6789/0 conn(0x7ff3440c0150 legacy=0x7ff3440c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 -- 172.21.15.92:0/596362288 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34406a140 con 0x7ff3440c5d00 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff34a448700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff344153f90 0x7ff3440a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349c47700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3440c5d00 0x7ff3440c6180 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7ff338006fd0 tx=0x7ff3380090f0).ready entity=mon.0 client_cookie=c08df3e4ccd9c38d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:33.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349446700 1 -- 172.21.15.92:0/596362288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800b600 con 0x7ff3440c5d00 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349446700 1 -- 172.21.15.92:0/596362288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33800b760 con 0x7ff3440c5d00 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.056+0000 7ff349446700 1 -- 172.21.15.92:0/596362288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800bae0 con 0x7ff3440c5d00 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 -- 172.21.15.92:0/596362288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3440c5d00 msgr2=0x7ff3440c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3440c5d00 0x7ff3440c6180 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7ff338006fd0 tx=0x7ff3380090f0).stop 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 -- 172.21.15.92:0/596362288 shutdown_connections 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff344153f90 0x7ff3440a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 --2- 172.21.15.92:0/596362288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3440c5d00 0x7ff3440c6180 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 -- 172.21.15.92:0/596362288 >> 172.21.15.92:0/596362288 conn(0x7ff3440cb600 msgr2=0x7ff3440c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:33.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 -- 172.21.15.92:0/596362288 shutdown_connections 2023-05-16T07:38:33.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.057+0000 7ff34b44a700 1 -- 172.21.15.92:0/596362288 wait complete. 2023-05-16T07:38:33.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.058+0000 7ff34b44a700 1 Processor -- start 2023-05-16T07:38:33.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.058+0000 7ff34b44a700 1 -- start start 2023-05-16T07:38:33.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3440c0150 0x7ff3440c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:33.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3440c5d00 0x7ff3440a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:33.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 0x7ff344163f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34414e160 con 0x7ff344153f90 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c3840 con 0x7ff3440c0150 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c2c30 con 0x7ff3440c5d00 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 0x7ff344163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 0x7ff344163f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53246/0 (socket says 172.21.15.92:53246) 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 -- 172.21.15.92:0/2653377691 learned_addr learned my addr 172.21.15.92:0/2653377691 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:33.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff349c47700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3440c5d00 0x7ff3440a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34a448700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3440c0150 0x7ff3440c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 -- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3440c5d00 msgr2=0x7ff3440a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3440c5d00 0x7ff3440a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 -- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3440c0150 msgr2=0x7ff3440c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3440c0150 0x7ff3440c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 -- 172.21.15.92:0/2653377691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338009370 con 0x7ff344153f90 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34ac49700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 0x7ff344163f40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7ff33c00c490 tx=0x7ff33c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c002ea0 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c00f470 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3440c2f10 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00f810 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.059+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3440c34d0 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.060+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3440adae0 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.061+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 83) v1 ==== 75379+0+0 (secure 0 0 0) 0x7ff33c010070 con 0x7ff344153f90 2023-05-16T07:38:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.061+0000 7ff3427fc700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff33005ee30 0x7ff3300612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.061+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7ff33c079370 con 0x7ff344153f90 2023-05-16T07:38:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.061+0000 7ff34a448700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff33005ee30 0x7ff3300612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.062+0000 7ff34a448700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff33005ee30 0x7ff3300612f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff33400e4b0 tx=0x7ff334007670).ready entity=mgr.8242 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.065+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff33c0478f0 con 0x7ff344153f90 2023-05-16T07:38:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.271+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff344083120 con 0x7ff344153f90 2023-05-16T07:38:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.786+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7ff33c079080 con 0x7ff344153f90 2023-05-16T07:38:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.787+0000 7ff3427fc700 1 -- 172.21.15.92:0/2653377691 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v84) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff33c073c50 con 0x7ff344153f90 2023-05-16T07:38:33.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff33005ee30 msgr2=0x7ff3300612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:33.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff33005ee30 0x7ff3300612f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff33400e4b0 tx=0x7ff334007670).stop 2023-05-16T07:38:33.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 msgr2=0x7ff344163f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 0x7ff344163f40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7ff33c00c490 tx=0x7ff33c007de0).stop 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 shutdown_connections 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff344153f90 0x7ff344163f40 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3440c0150 0x7ff3440c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff33005ee30 0x7ff3300612f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 --2- 172.21.15.92:0/2653377691 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff3440c5d00 0x7ff3440a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 >> 172.21.15.92:0/2653377691 conn(0x7ff3440cb600 msgr2=0x7ff344151480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:33.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 shutdown_connections 2023-05-16T07:38:33.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:33.790+0000 7ff34b44a700 1 -- 172.21.15.92:0/2653377691 wait complete. 2023-05-16T07:38:33.807 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-16T07:38:33.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:38:33.836 DEBUG:teuthology.orchestra.run.smithi158:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-16T07:38:33.864 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-16T07:38:33.865 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-16T07:38:33.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-05-16T07:38:33.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-16T07:38:33.918 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:38:33.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:33.919 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-16T07:38:33.931 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:33.928+0000 7f5a6751c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:38:33.935 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:33.931+0000 7fb223f5b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-16T07:38:33.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:33.995 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:33.992+0000 7f5a6751c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:38:33.996 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:33.992+0000 7fb223f5b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-16T07:38:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.097+0000 7f426c83f700 1 Processor -- start 2023-05-16T07:38:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.097+0000 7f426c83f700 1 -- start start 2023-05-16T07:38:34.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f426c83f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4268150ac0 0x7f42680c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f426c83f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f42681547b0 0x7f4268154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f426c83f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42680c2240 con 0x7f42680bc050 2023-05-16T07:38:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f426c83f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42680c31a0 con 0x7f4268150ac0 2023-05-16T07:38:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f426c83f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4268155120 con 0x7f42681547b0 2023-05-16T07:38:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f426659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4268150ac0 0x7f42680c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f4266d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42680bc050 0x7f42680bc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35820/0 (socket says 172.21.15.92:35820) 2023-05-16T07:38:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f4266d9d700 1 -- 172.21.15.92:0/2709425794 learned_addr learned my addr 172.21.15.92:0/2709425794 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:34.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.098+0000 7f4265d9b700 1 --2- 172.21.15.92:0/2709425794 >> v2:172.21.15.158:3300/0 conn(0x7f42681547b0 0x7f4268154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426659c700 1 -- 172.21.15.92:0/2709425794 >> v2:172.21.15.158:3300/0 conn(0x7f42681547b0 msgr2=0x7f4268154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426659c700 1 --2- 172.21.15.92:0/2709425794 >> v2:172.21.15.158:3300/0 conn(0x7f42681547b0 0x7f4268154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426659c700 1 -- 172.21.15.92:0/2709425794 >> v1:172.21.15.92:6789/0 conn(0x7f42680bc050 legacy=0x7f42680bc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426659c700 1 -- 172.21.15.92:0/2709425794 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680c4680 con 0x7f4268150ac0 2023-05-16T07:38:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426659c700 1 --2- 172.21.15.92:0/2709425794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268150ac0 0x7f42680c2cb0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f4268063dd0 tx=0x7f425c00b300).ready entity=mon.0 client_cookie=59ffafed5af0c611 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:34.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426559a700 1 -- 172.21.15.92:0/2709425794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00d970 con 0x7f4268150ac0 2023-05-16T07:38:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426559a700 1 -- 172.21.15.92:0/2709425794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00dad0 con 0x7f4268150ac0 2023-05-16T07:38:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.099+0000 7f426559a700 1 -- 172.21.15.92:0/2709425794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00de50 con 0x7f4268150ac0 2023-05-16T07:38:34.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 -- 172.21.15.92:0/2709425794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268150ac0 msgr2=0x7f42680c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 --2- 172.21.15.92:0/2709425794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268150ac0 0x7f42680c2cb0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f4268063dd0 tx=0x7f425c00b300).stop 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 -- 172.21.15.92:0/2709425794 shutdown_connections 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 --2- 172.21.15.92:0/2709425794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268150ac0 0x7f42680c2cb0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 --2- 172.21.15.92:0/2709425794 >> v2:172.21.15.158:3300/0 conn(0x7f42681547b0 0x7f4268154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 -- 172.21.15.92:0/2709425794 >> 172.21.15.92:0/2709425794 conn(0x7f42680c7bc0 msgr2=0x7f42680c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 -- 172.21.15.92:0/2709425794 shutdown_connections 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.100+0000 7f426c83f700 1 -- 172.21.15.92:0/2709425794 wait complete. 2023-05-16T07:38:34.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.101+0000 7f426c83f700 1 Processor -- start 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.101+0000 7f426c83f700 1 -- start start 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 0x7f42680a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4268150ac0 0x7f4268153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42681547b0 0x7f4268160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680a87c0 con 0x7f42680bc050 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680a8aa0 con 0x7f4268150ac0 2023-05-16T07:38:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680a8d80 con 0x7f42681547b0 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 0x7f42680a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 0x7f42680a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53306/0 (socket says 172.21.15.92:53306) 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 -- 172.21.15.92:0/3136247865 learned_addr learned my addr 172.21.15.92:0/3136247865 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f4265d9b700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4268150ac0 0x7f4268153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f4266d9d700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42681547b0 0x7f4268160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 -- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42681547b0 msgr2=0x7f4268160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42681547b0 0x7f4268160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 -- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4268150ac0 msgr2=0x7f4268153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4268150ac0 0x7f4268153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 -- 172.21.15.92:0/3136247865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c00b540 con 0x7f42680bc050 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426659c700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 0x7f42680a9860 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f425c00d5d0 tx=0x7f425c00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c020070 con 0x7f42680bc050 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c010dc0 con 0x7f42680bc050 2023-05-16T07:38:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42680a9060 con 0x7f42680bc050 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0196a0 con 0x7f42680bc050 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.102+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4268153af0 con 0x7f42680bc050 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.103+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426806c350 con 0x7f42680bc050 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.104+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f425c02c070 con 0x7f42680bc050 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.104+0000 7f42567fc700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 0x7f42480612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.104+0000 7f4265d9b700 1 -- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 msgr2=0x7f42480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.104+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f425c07d510 con 0x7f42680bc050 2023-05-16T07:38:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.104+0000 7f4265d9b700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 0x7f42480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.108+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f425c04e690 con 0x7f42680bc050 2023-05-16T07:38:34.266 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:34.262+0000 7fb223f5b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:38:34.276 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:34.272+0000 7f5a6751c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-16T07:38:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.305+0000 7f4265d9b700 1 -- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 msgr2=0x7f42480612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.305+0000 7f4265d9b700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 0x7f42480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:34.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.317+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42680c24b0 con 0x7f42680bc050 2023-05-16T07:38:34.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.318+0000 7f42567fc700 1 -- 172.21.15.92:0/3136247865 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f425c02a070 con 0x7f42680bc050 2023-05-16T07:38:34.322 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:34.323 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:38:19.513917+0000","modified":"2023-05-16T07:38:21.520884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/862724662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":862724662},{"type":"v1","addr":"172.21.15.158:6837","nonce":862724662}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-05-16T07:38:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 msgr2=0x7f42480612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 0x7f42480612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 msgr2=0x7f42680a9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 0x7f42680a9860 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f425c00d5d0 tx=0x7f425c00b300).stop 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 shutdown_connections 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f424805ede0 0x7f42480612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f42681547b0 0x7f4268160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42680bc050 0x7f42680a9860 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 --2- 172.21.15.92:0/3136247865 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4268150ac0 0x7f4268153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 >> 172.21.15.92:0/3136247865 conn(0x7f42680c7bc0 msgr2=0x7f426814eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 shutdown_connections 2023-05-16T07:38:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.320+0000 7f426c83f700 1 -- 172.21.15.92:0/3136247865 wait complete. 2023-05-16T07:38:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-05-16T07:38:34.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:34.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.490+0000 7f86c4ad7700 1 Processor -- start 2023-05-16T07:38:34.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.490+0000 7f86c4ad7700 1 -- start start 2023-05-16T07:38:34.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86c4ad7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f86c0151ff0 0x7f86c01555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86c4ad7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86c0068870 0x7f86c00bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86c4ad7700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c0155130 con 0x7f86c00a9ee0 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86c4ad7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c00c55d0 con 0x7f86c0068870 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86c4ad7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c00cb740 con 0x7f86c0151ff0 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86bed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86c0068870 0x7f86c00bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86bed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86c0068870 0x7f86c00bfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53360/0 (socket says 172.21.15.92:53360) 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.491+0000 7f86bed9d700 1 -- 172.21.15.92:0/103208738 learned_addr learned my addr 172.21.15.92:0/103208738 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86be59c700 1 --2- 172.21.15.92:0/103208738 >> v2:172.21.15.158:3300/0 conn(0x7f86c0151ff0 0x7f86c01555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bed9d700 1 -- 172.21.15.92:0/103208738 >> v2:172.21.15.158:3300/0 conn(0x7f86c0151ff0 msgr2=0x7f86c01555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bed9d700 1 --2- 172.21.15.92:0/103208738 >> v2:172.21.15.158:3300/0 conn(0x7f86c0151ff0 0x7f86c01555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bed9d700 1 -- 172.21.15.92:0/103208738 >> v1:172.21.15.158:6789/0 conn(0x7f86c00a9ee0 legacy=0x7f86c0068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bed9d700 1 -- 172.21.15.92:0/103208738 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00cba20 con 0x7f86c0068870 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86be59c700 1 --2- 172.21.15.92:0/103208738 >> v2:172.21.15.158:3300/0 conn(0x7f86c0151ff0 0x7f86c01555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bed9d700 1 --2- 172.21.15.92:0/103208738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0068870 0x7f86c00bfc20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f86c00c8ac0 tx=0x7f86b8009840).ready entity=mon.0 client_cookie=9cbcacbddaf1953d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bd59a700 1 -- 172.21.15.92:0/103208738 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b800e070 con 0x7f86c0068870 2023-05-16T07:38:34.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bd59a700 1 -- 172.21.15.92:0/103208738 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b800ba50 con 0x7f86c0068870 2023-05-16T07:38:34.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86bd59a700 1 -- 172.21.15.92:0/103208738 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b800bdd0 con 0x7f86c0068870 2023-05-16T07:38:34.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86c4ad7700 1 -- 172.21.15.92:0/103208738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0068870 msgr2=0x7f86c00bfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.492+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/103208738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0068870 0x7f86c00bfc20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f86c00c8ac0 tx=0x7f86b8009840).stop 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.493+0000 7f86c4ad7700 1 -- 172.21.15.92:0/103208738 shutdown_connections 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.493+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/103208738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0068870 0x7f86c00bfc20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.493+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/103208738 >> v2:172.21.15.158:3300/0 conn(0x7f86c0151ff0 0x7f86c01555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.493+0000 7f86c4ad7700 1 -- 172.21.15.92:0/103208738 >> 172.21.15.92:0/103208738 conn(0x7f86c0067920 msgr2=0x7f86c00cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.493+0000 7f86c4ad7700 1 -- 172.21.15.92:0/103208738 shutdown_connections 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.493+0000 7f86c4ad7700 1 -- 172.21.15.92:0/103208738 wait complete. 2023-05-16T07:38:34.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 Processor -- start 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 -- start start 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86c0068870 0x7f86c00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 0x7f86c00b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86c0151ff0 0x7f86c01533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0154850 con 0x7f86c00a9ee0 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c2140 con 0x7f86c0151ff0 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.494+0000 7f86c4ad7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00c2420 con 0x7f86c0068870 2023-05-16T07:38:34.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 0x7f86c00b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86be59c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86c0068870 0x7f86c00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86c0151ff0 0x7f86c01533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 0x7f86c00b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53364/0 (socket says 172.21.15.92:53364) 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bdd9b700 1 -- 172.21.15.92:0/1918527535 learned_addr learned my addr 172.21.15.92:0/1918527535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 -- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86c0068870 msgr2=0x7f86c00ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86c0068870 0x7f86c00ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 -- 172.21.15.92:0/1918527535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 msgr2=0x7f86c00b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 0x7f86c00b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 -- 172.21.15.92:0/1918527535 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b8009ae0 con 0x7f86c0151ff0 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bdd9b700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 0x7f86c00b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86bed9d700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86c0151ff0 0x7f86c01533a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f86b8013000 tx=0x7f86b8009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80032d0 con 0x7f86c0151ff0 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b800b6c0 con 0x7f86c0151ff0 2023-05-16T07:38:34.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b801d620 con 0x7f86c0151ff0 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c00c2700 con 0x7f86c0151ff0 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.495+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c00c2cc0 con 0x7f86c0151ff0 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.496+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c00666c0 con 0x7f86c0151ff0 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.497+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f86b801b040 con 0x7f86c0151ff0 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.497+0000 7f86ae7fc700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f86a005edb0 0x7f86a0061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.497+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f86b807b910 con 0x7f86c0151ff0 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.497+0000 7f86be59c700 1 -- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f86a005edb0 msgr2=0x7f86a0061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.497+0000 7f86be59c700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f86a005edb0 0x7f86a0061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:34.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.501+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f86b804ca90 con 0x7f86c0151ff0 2023-05-16T07:38:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.671+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c00c2eb0 con 0x7f86c0151ff0 2023-05-16T07:38:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.673+0000 7f86ae7fc700 1 -- 172.21.15.92:0/1918527535 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f86b80500b0 con 0x7f86c0151ff0 2023-05-16T07:38:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:34.678 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":172,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:30.703341+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:38:18.041970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-05-16T07:38:18.481177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3460186346":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6833/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6832/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:0/240635450":"2023-05-17T07:38:30.703311+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f86a005edb0 msgr2=0x7f86a0061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f86a005edb0 0x7f86a0061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86c0151ff0 msgr2=0x7f86c01533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86c0151ff0 0x7f86c01533a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f86b8013000 tx=0x7f86b8009840).stop 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 shutdown_connections 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f86a005edb0 0x7f86a0061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f86c0068870 0x7f86c00ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86c0151ff0 0x7f86c01533a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 --2- 172.21.15.92:0/1918527535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c00a9ee0 0x7f86c00b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 >> 172.21.15.92:0/1918527535 conn(0x7f86c0067920 msgr2=0x7f86c014d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 shutdown_connections 2023-05-16T07:38:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.675+0000 7f86c4ad7700 1 -- 172.21.15.92:0/1918527535 wait complete. 2023-05-16T07:38:34.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-05-16T07:02:17.752010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-05-16T07:02:19.800493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-16T07:38:18.041970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-05-16T07:38:18.481177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-16T07:38:34.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-05-16T07:38:34.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.807+0000 7f443e719700 1 Processor -- start 2023-05-16T07:38:34.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.807+0000 7f443e719700 1 -- start start 2023-05-16T07:38:34.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f443e719700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44380aa210 0x7f44380c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f443e719700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44380ca600 con 0x7f4438153aa0 2023-05-16T07:38:34.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f443e719700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4438154600 con 0x7f44380c5d30 2023-05-16T07:38:34.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f443e719700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44380c8830 con 0x7f44380aa210 2023-05-16T07:38:34.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44380aa210 0x7f44380c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44380aa210 0x7f44380c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53372/0 (socket says 172.21.15.92:53372) 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f443ccb6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4438153aa0 0x7f4438153ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35890/0 (socket says 172.21.15.92:35890) 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 -- 172.21.15.92:0/121601573 learned_addr learned my addr 172.21.15.92:0/121601573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 -- 172.21.15.92:0/121601573 >> v1:172.21.15.158:6789/0 conn(0x7f44380c5d30 legacy=0x7f4438069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 -- 172.21.15.92:0/121601573 >> v1:172.21.15.92:6789/0 conn(0x7f4438153aa0 legacy=0x7f4438153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 -- 172.21.15.92:0/121601573 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380c8b10 con 0x7f44380aa210 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f4436ffd700 1 -- 172.21.15.92:0/121601573 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690458784 0 0) 0x7f44380ca600 con 0x7f4438153aa0 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f44377fe700 1 --2- 172.21.15.92:0/121601573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44380aa210 0x7f44380c6110 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f4428006fd0 tx=0x7f44280090f0).ready entity=mon.0 client_cookie=d7925d4cb610ccdf server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:34.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.808+0000 7f4436ffd700 1 -- 172.21.15.92:0/121601573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800b600 con 0x7f44380aa210 2023-05-16T07:38:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f4436ffd700 1 -- 172.21.15.92:0/121601573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442800b760 con 0x7f44380aa210 2023-05-16T07:38:34.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f4436ffd700 1 -- 172.21.15.92:0/121601573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800bae0 con 0x7f44380aa210 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 -- 172.21.15.92:0/121601573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44380aa210 msgr2=0x7f44380c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 --2- 172.21.15.92:0/121601573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44380aa210 0x7f44380c6110 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f4428006fd0 tx=0x7f44280090f0).stop 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 -- 172.21.15.92:0/121601573 shutdown_connections 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 --2- 172.21.15.92:0/121601573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44380aa210 0x7f44380c6110 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 -- 172.21.15.92:0/121601573 >> 172.21.15.92:0/121601573 conn(0x7f44380cb700 msgr2=0x7f44380c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 -- 172.21.15.92:0/121601573 shutdown_connections 2023-05-16T07:38:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.809+0000 7f443e719700 1 -- 172.21.15.92:0/121601573 wait complete. 2023-05-16T07:38:34.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.811+0000 7f443e719700 1 Processor -- start 2023-05-16T07:38:34.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.811+0000 7f443e719700 1 -- start start 2023-05-16T07:38:34.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f443e719700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44380c5d30 0x7f44380c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f443e719700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 0x7f44381547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f443e719700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4438154ce0 0x7f44381649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f443e719700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c4220 con 0x7f4438153aa0 2023-05-16T07:38:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f443e719700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438155160 con 0x7f44380c5d30 2023-05-16T07:38:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f443e719700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c3340 con 0x7f4438154ce0 2023-05-16T07:38:34.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.812+0000 7f44377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 0x7f44381547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 0x7f44381547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53378/0 (socket says 172.21.15.92:53378) 2023-05-16T07:38:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 -- 172.21.15.92:0/4130080640 learned_addr learned my addr 172.21.15.92:0/4130080640 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f4437fff700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44380c5d30 0x7f44380c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f443ccb6700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4438154ce0 0x7f44381649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 -- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4438154ce0 msgr2=0x7f44381649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4438154ce0 0x7f44381649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 -- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44380c5d30 msgr2=0x7f44380c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44380c5d30 0x7f44380c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 -- 172.21.15.92:0/4130080640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428009370 con 0x7f4438153aa0 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f443ccb6700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4438154ce0 0x7f44381649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f44377fe700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 0x7f44381547f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f4438083c50 tx=0x7f4428017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800b8d0 con 0x7f4438153aa0 2023-05-16T07:38:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442800ba30 con 0x7f4438153aa0 2023-05-16T07:38:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44380c3610 con 0x7f4438153aa0 2023-05-16T07:38:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442801c670 con 0x7f4438153aa0 2023-05-16T07:38:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.813+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44380c3bd0 con 0x7f4438153aa0 2023-05-16T07:38:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.814+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44380c43d0 con 0x7f4438153aa0 2023-05-16T07:38:34.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.814+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f442800f070 con 0x7f4438153aa0 2023-05-16T07:38:34.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.815+0000 7f441ffff700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 0x7f44240612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:34.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.815+0000 7f4437fff700 1 -- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 msgr2=0x7f44240612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:34.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.815+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f442807bc50 con 0x7f4438153aa0 2023-05-16T07:38:34.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.815+0000 7f4437fff700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 0x7f44240612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:34.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:34.819+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f44380c43d0 con 0x7f4438153aa0 2023-05-16T07:38:34.870 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:34.867+0000 7fb223f5b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:38:34.915 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:34.910+0000 7f5a6751c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-16T07:38:34.941 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:34.937+0000 7fb223f5b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:38:34.986 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:34.982+0000 7f5a6751c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-16T07:38:35.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.016+0000 7f4437fff700 1 -- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 msgr2=0x7f44240612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:35.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.016+0000 7f4437fff700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 0x7f44240612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.021+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f44380c43d0 con 0x7f4438153aa0 2023-05-16T07:38:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.021+0000 7f441ffff700 1 -- 172.21.15.92:0/4130080640 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v172) v1 ==== 110+0+10 (secure 0 0 0) 0x7f44380c43d0 con 0x7f4438153aa0 2023-05-16T07:38:35.025 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-05-16T07:38:35.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 msgr2=0x7f44240612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:35.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 0x7f44240612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 msgr2=0x7f44381547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 0x7f44381547f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f4438083c50 tx=0x7f4428017000).stop 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 shutdown_connections 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f442405ede0 0x7f44240612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4438154ce0 0x7f44381649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44380c5d30 0x7f44380c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 --2- 172.21.15.92:0/4130080640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4438153aa0 0x7f44381547f0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 >> 172.21.15.92:0/4130080640 conn(0x7f44380cb700 msgr2=0x7f44380c1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 shutdown_connections 2023-05-16T07:38:35.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.023+0000 7f443e719700 1 -- 172.21.15.92:0/4130080640 wait complete. 2023-05-16T07:38:35.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-16T07:38:35.126 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:35.122+0000 7fb223f5b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:38:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.154+0000 7f1f74241700 1 Processor -- start 2023-05-16T07:38:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.154+0000 7f1f74241700 1 -- start start 2023-05-16T07:38:35.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f74241700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1f6c0cb420 0x7f1f6c154500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f74241700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c0c6320 con 0x7f1f6c155580 2023-05-16T07:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f74241700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c1550f0 con 0x7f1f6c154120 2023-05-16T07:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f74241700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c0c6840 con 0x7f1f6c0cb420 2023-05-16T07:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f727de700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1f6c155580 0x7f1f6c068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35906/0 (socket says 172.21.15.92:35906) 2023-05-16T07:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f727de700 1 -- 172.21.15.92:0/3430794435 learned_addr learned my addr 172.21.15.92:0/3430794435 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:35.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f717dc700 1 --2- 172.21.15.92:0/3430794435 >> v2:172.21.15.158:3300/0 conn(0x7f1f6c0cb420 0x7f1f6c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616265867 0 0) 0x7f1f64001490 con 0x7f1f6c155580 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f580033f0 con 0x7f1f6c155580 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101983697 0 0) 0x7f1f6c1550f0 con 0x7f1f6c154120 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f64001490 con 0x7f1f6c154120 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f717dc700 1 -- 172.21.15.92:0/3430794435 >> v1:172.21.15.158:6789/0 conn(0x7f1f6c154120 legacy=0x7f1f6c0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f717dc700 1 -- 172.21.15.92:0/3430794435 >> v1:172.21.15.92:6789/0 conn(0x7f1f6c155580 legacy=0x7f1f6c068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f717dc700 1 -- 172.21.15.92:0/3430794435 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0c6560 con 0x7f1f6c0cb420 2023-05-16T07:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.156+0000 7f1f717dc700 1 --2- 172.21.15.92:0/3430794435 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c154500 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f1f68009760 tx=0x7f1f68009060).ready entity=mon.1 client_cookie=4c7c654643c96d49 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800d4a0 con 0x7f1f6c0cb420 2023-05-16T07:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6800d600 con 0x7f1f6c0cb420 2023-05-16T07:38:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f70fdb700 1 -- 172.21.15.92:0/3430794435 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800d980 con 0x7f1f6c0cb420 2023-05-16T07:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f74241700 1 -- 172.21.15.92:0/3430794435 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 msgr2=0x7f1f6c154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f74241700 1 --2- 172.21.15.92:0/3430794435 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c154500 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f1f68009760 tx=0x7f1f68009060).stop 2023-05-16T07:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f74241700 1 -- 172.21.15.92:0/3430794435 shutdown_connections 2023-05-16T07:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f74241700 1 --2- 172.21.15.92:0/3430794435 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c154500 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.157+0000 7f1f74241700 1 -- 172.21.15.92:0/3430794435 >> 172.21.15.92:0/3430794435 conn(0x7f1f6c0678e0 msgr2=0x7f1f6c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.158+0000 7f1f74241700 1 -- 172.21.15.92:0/3430794435 shutdown_connections 2023-05-16T07:38:35.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.158+0000 7f1f74241700 1 -- 172.21.15.92:0/3430794435 wait complete. 2023-05-16T07:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.159+0000 7f1f74241700 1 Processor -- start 2023-05-16T07:38:35.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.159+0000 7f1f74241700 1 -- start start 2023-05-16T07:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f74241700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c0c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f74241700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 0x7f1f6c13d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f74241700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f6c155580 0x7f1f6c167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f74241700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c158ae0 con 0x7f1f6c154120 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f74241700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c158790 con 0x7f1f6c0cb420 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f74241700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c168770 con 0x7f1f6c155580 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f717dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 0x7f1f6c13d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f71fdd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c0c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f727de700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f6c155580 0x7f1f6c167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f717dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 0x7f1f6c13d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53396/0 (socket says 172.21.15.92:53396) 2023-05-16T07:38:35.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f717dc700 1 -- 172.21.15.92:0/2526054989 learned_addr learned my addr 172.21.15.92:0/2526054989 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.160+0000 7f1f71fdd700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c0c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:57028/0 (socket says 172.21.15.92:57028) 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f717dc700 1 -- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f6c155580 msgr2=0x7f1f6c167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f717dc700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f6c155580 0x7f1f6c167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f717dc700 1 -- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 msgr2=0x7f1f6c0c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f717dc700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c0c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f717dc700 1 -- 172.21.15.92:0/2526054989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680092a0 con 0x7f1f6c154120 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f717dc700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 0x7f1f6c13d800 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f1f6c083c50 tx=0x7f1f68016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:35.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800dcf0 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6800de50 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c168a50 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800d980 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.161+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c168280 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.162+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c168530 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.163+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f1f6800f070 con 0x7f1f6c154120 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.164+0000 7f1f61ffb700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1f5005ede0 0x7f1f500612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.164+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f1f6807bf80 con 0x7f1f6c154120 2023-05-16T07:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.164+0000 7f1f71fdd700 1 -- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1f5005ede0 msgr2=0x7f1f500612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.164+0000 7f1f71fdd700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1f5005ede0 0x7f1f500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:35.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.168+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f1f6804d010 con 0x7f1f6c154120 2023-05-16T07:38:35.185 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:35.181+0000 7f5a6751c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-16T07:38:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.336+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1f6c168530 con 0x7f1f6c154120 2023-05-16T07:38:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.337+0000 7f1f61ffb700 1 -- 172.21.15.92:0/2526054989 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v172) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1f6c168530 con 0x7f1f6c154120 2023-05-16T07:38:35.341 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-05-16T07:38:35.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1f5005ede0 msgr2=0x7f1f500612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:35.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1f5005ede0 0x7f1f500612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 msgr2=0x7f1f6c13d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 0x7f1f6c13d800 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f1f6c083c50 tx=0x7f1f68016000).stop 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 shutdown_connections 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1f5005ede0 0x7f1f500612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1f6c155580 0x7f1f6c167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1f6c0cb420 0x7f1f6c0c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 --2- 172.21.15.92:0/2526054989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f6c154120 0x7f1f6c13d800 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 >> 172.21.15.92:0/2526054989 conn(0x7f1f6c0678e0 msgr2=0x7f1f6c1535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 shutdown_connections 2023-05-16T07:38:35.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.339+0000 7f1f74241700 1 -- 172.21.15.92:0/2526054989 wait complete. 2023-05-16T07:38:35.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-16T07:38:35.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.470+0000 7f9f79843700 1 Processor -- start 2023-05-16T07:38:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.470+0000 7f9f79843700 1 -- start start 2023-05-16T07:38:35.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f79843700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f7414f990 0x7f9f740c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f79843700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f74152e40 0x7f9f740c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f79843700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f9f740622b0 0x7f9f74062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f79843700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f74063690 con 0x7f9f7414f990 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f79843700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f74062c30 con 0x7f9f740622b0 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f79843700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7414fd70 con 0x7f9f74152e40 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f7414f990 0x7f9f740c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f7414f990 0x7f9f740c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53444/0 (socket says 172.21.15.92:53444) 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 -- 172.21.15.92:0/4212137289 learned_addr learned my addr 172.21.15.92:0/4212137289 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f737fe700 1 --2- 172.21.15.92:0/4212137289 >> v2:172.21.15.158:3300/0 conn(0x7f9f740622b0 0x7f9f74062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f727fc700 1 --2- 172.21.15.92:0/4212137289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f74152e40 0x7f9f740c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 -- 172.21.15.92:0/4212137289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f74152e40 msgr2=0x7f9f740c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 --2- 172.21.15.92:0/4212137289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f74152e40 0x7f9f740c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 -- 172.21.15.92:0/4212137289 >> v2:172.21.15.158:3300/0 conn(0x7f9f740622b0 msgr2=0x7f9f74062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 --2- 172.21.15.92:0/4212137289 >> v2:172.21.15.158:3300/0 conn(0x7f9f740622b0 0x7f9f74062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 -- 172.21.15.92:0/4212137289 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740c52f0 con 0x7f9f7414f990 2023-05-16T07:38:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f737fe700 1 --2- 172.21.15.92:0/4212137289 >> v2:172.21.15.158:3300/0 conn(0x7f9f740622b0 0x7f9f74062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f72ffd700 1 --2- 172.21.15.92:0/4212137289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f7414f990 0x7f9f740c4c80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f9f740bfae0 tx=0x7f9f680090f0).ready entity=mon.0 client_cookie=6dd1a9200827fc24 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f71ffb700 1 -- 172.21.15.92:0/4212137289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680108a0 con 0x7f9f7414f990 2023-05-16T07:38:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f71ffb700 1 -- 172.21.15.92:0/4212137289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f68010a00 con 0x7f9f7414f990 2023-05-16T07:38:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.472+0000 7f9f71ffb700 1 -- 172.21.15.92:0/4212137289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68010d80 con 0x7f9f7414f990 2023-05-16T07:38:35.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 -- 172.21.15.92:0/4212137289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f7414f990 msgr2=0x7f9f740c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 --2- 172.21.15.92:0/4212137289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f7414f990 0x7f9f740c4c80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f9f740bfae0 tx=0x7f9f680090f0).stop 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 -- 172.21.15.92:0/4212137289 shutdown_connections 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 --2- 172.21.15.92:0/4212137289 >> v2:172.21.15.158:3300/0 conn(0x7f9f740622b0 0x7f9f74062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 --2- 172.21.15.92:0/4212137289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f7414f990 0x7f9f740c4c80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 --2- 172.21.15.92:0/4212137289 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f74152e40 0x7f9f740c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 -- 172.21.15.92:0/4212137289 >> 172.21.15.92:0/4212137289 conn(0x7f9f7405e820 msgr2=0x7f9f740c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 -- 172.21.15.92:0/4212137289 shutdown_connections 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.473+0000 7f9f79843700 1 -- 172.21.15.92:0/4212137289 wait complete. 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 Processor -- start 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 -- start start 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 0x7f9f7413a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f7414f990 0x7f9f7414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f74152e40 0x7f9f74139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740c16c0 con 0x7f9f740622b0 2023-05-16T07:38:35.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74133e90 con 0x7f9f74152e40 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f79843700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740bfb30 con 0x7f9f7414f990 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 0x7f9f7413a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 0x7f9f7413a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53450/0 (socket says 172.21.15.92:53450) 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.475+0000 7f9f72ffd700 1 -- 172.21.15.92:0/204701444 learned_addr learned my addr 172.21.15.92:0/204701444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f727fc700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f7414f990 0x7f9f7414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f737fe700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f74152e40 0x7f9f74139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f72ffd700 1 -- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f7414f990 msgr2=0x7f9f7414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f72ffd700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f7414f990 0x7f9f7414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f72ffd700 1 -- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f74152e40 msgr2=0x7f9f74139ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f72ffd700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f74152e40 0x7f9f74139ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f72ffd700 1 -- 172.21.15.92:0/204701444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f68009370 con 0x7f9f740622b0 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f72ffd700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 0x7f9f7413a9b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f9f68010fd0 tx=0x7f9f68019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6801ad80 con 0x7f9f740622b0 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f68023440 con 0x7f9f740622b0 2023-05-16T07:38:35.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7413aea0 con 0x7f9f740622b0 2023-05-16T07:38:35.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68023800 con 0x7f9f740622b0 2023-05-16T07:38:35.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.476+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f740c00d0 con 0x7f9f740622b0 2023-05-16T07:38:35.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.477+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f740c2d70 con 0x7f9f740622b0 2023-05-16T07:38:35.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.477+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f9f68029020 con 0x7f9f740622b0 2023-05-16T07:38:35.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.478+0000 7f9f5affd700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f6005ede0 0x7f9f600612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.478+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f9f6807f830 con 0x7f9f740622b0 2023-05-16T07:38:35.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.478+0000 7f9f727fc700 1 -- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f6005ede0 msgr2=0x7f9f600612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:35.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.478+0000 7f9f727fc700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f6005ede0 0x7f9f600612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:35.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.483+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9f680509b0 con 0x7f9f740622b0 2023-05-16T07:38:35.548 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:35.544+0000 7fb223f5b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:38:35.621 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:35.617+0000 7f5a6751c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-16T07:38:35.651 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:35.647+0000 7fb223f5b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:35.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.655+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f740c5ce0 con 0x7f9f740622b0 2023-05-16T07:38:35.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.656+0000 7f9f5affd700 1 -- 172.21.15.92:0/204701444 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f68053fd0 con 0x7f9f740622b0 2023-05-16T07:38:35.660 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:38:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f6005ede0 msgr2=0x7f9f600612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f6005ede0 0x7f9f600612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 msgr2=0x7f9f7413a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 0x7f9f7413a9b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f9f68010fd0 tx=0x7f9f68019000).stop 2023-05-16T07:38:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 shutdown_connections 2023-05-16T07:38:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f74152e40 0x7f9f74139ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f740622b0 0x7f9f7413a9b0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f6005ede0 0x7f9f600612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 --2- 172.21.15.92:0/204701444 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f7414f990 0x7f9f7414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 >> 172.21.15.92:0/204701444 conn(0x7f9f7405e820 msgr2=0x7f9f74152240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 shutdown_connections 2023-05-16T07:38:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.658+0000 7f9f79843700 1 -- 172.21.15.92:0/204701444 wait complete. 2023-05-16T07:38:35.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-16T07:38:35.731 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:35.727+0000 7f5a6751c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:35.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.789+0000 7f90f9470700 1 Processor -- start 2023-05-16T07:38:35.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.789+0000 7f90f9470700 1 -- start start 2023-05-16T07:38:35.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f9470700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f90f4154090 0x7f90f40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f9470700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90f4068160 0x7f90f40685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f9470700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40c94d0 con 0x7f90f4154470 2023-05-16T07:38:35.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f9470700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f40cb7e0 con 0x7f90f4068160 2023-05-16T07:38:35.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f9470700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f4068aa0 con 0x7f90f4154090 2023-05-16T07:38:35.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90f4068160 0x7f90f40685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90f4068160 0x7f90f40685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53462/0 (socket says 172.21.15.92:53462) 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 -- 172.21.15.92:0/3138454038 learned_addr learned my addr 172.21.15.92:0/3138454038 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f2ffd700 1 --2- 172.21.15.92:0/3138454038 >> v2:172.21.15.158:3300/0 conn(0x7f90f4154090 0x7f90f40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 -- 172.21.15.92:0/3138454038 >> v2:172.21.15.158:3300/0 conn(0x7f90f4154090 msgr2=0x7f90f40aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 --2- 172.21.15.92:0/3138454038 >> v2:172.21.15.158:3300/0 conn(0x7f90f4154090 0x7f90f40aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 -- 172.21.15.92:0/3138454038 >> v1:172.21.15.158:6789/0 conn(0x7f90f4154470 legacy=0x7f90f40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 -- 172.21.15.92:0/3138454038 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40c88c0 con 0x7f90f4068160 2023-05-16T07:38:35.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.790+0000 7f90f37fe700 1 --2- 172.21.15.92:0/3138454038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4068160 0x7f90f40685b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f90f406bd10 tx=0x7f90ec009840).ready entity=mon.0 client_cookie=f6345a2bf5499e8 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:35.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f1ffb700 1 -- 172.21.15.92:0/3138454038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00e070 con 0x7f90f4068160 2023-05-16T07:38:35.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f1ffb700 1 -- 172.21.15.92:0/3138454038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00ba20 con 0x7f90f4068160 2023-05-16T07:38:35.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f1ffb700 1 -- 172.21.15.92:0/3138454038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00bda0 con 0x7f90f4068160 2023-05-16T07:38:35.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 -- 172.21.15.92:0/3138454038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4068160 msgr2=0x7f90f40685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 --2- 172.21.15.92:0/3138454038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4068160 0x7f90f40685b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f90f406bd10 tx=0x7f90ec009840).stop 2023-05-16T07:38:35.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 -- 172.21.15.92:0/3138454038 shutdown_connections 2023-05-16T07:38:35.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 --2- 172.21.15.92:0/3138454038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4068160 0x7f90f40685b0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 --2- 172.21.15.92:0/3138454038 >> v2:172.21.15.158:3300/0 conn(0x7f90f4154090 0x7f90f40aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 -- 172.21.15.92:0/3138454038 >> 172.21.15.92:0/3138454038 conn(0x7f90f4067920 msgr2=0x7f90f40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 -- 172.21.15.92:0/3138454038 shutdown_connections 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.791+0000 7f90f9470700 1 -- 172.21.15.92:0/3138454038 wait complete. 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 Processor -- start 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 -- start start 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 0x7f90f4152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4154090 0x7f90f4157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90f4154470 0x7f90f40c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40c2d10 con 0x7f90f4154090 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40c2100 con 0x7f90f4068160 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f9470700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40c23e0 con 0x7f90f4154470 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4154090 0x7f90f4157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f37fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90f4154470 0x7f90f40c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.793+0000 7f90f2ffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 0x7f90f4152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4154090 0x7f90f4157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53466/0 (socket says 172.21.15.92:53466) 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f37fe700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90f4154470 0x7f90f40c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:46040/0 (socket says 172.21.15.92:46040) 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 0x7f90f4152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:57100/0 (socket says 172.21.15.92:57100) 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f27fc700 1 -- 172.21.15.92:0/2289394546 learned_addr learned my addr 172.21.15.92:0/2289394546 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 -- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90f4154470 msgr2=0x7f90f40c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90f4154470 0x7f90f40c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 -- 172.21.15.92:0/2289394546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4154090 msgr2=0x7f90f4157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4154090 0x7f90f4157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 -- 172.21.15.92:0/2289394546 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec009ae0 con 0x7f90f4068160 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f2ffd700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 0x7f90f4152170 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f90e800e850 tx=0x7f90e8009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8009430 con 0x7f90f4068160 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e8012600 con 0x7f90f4068160 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40c26c0 con 0x7f90f4068160 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8012980 con 0x7f90f4068160 2023-05-16T07:38:35.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.794+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f4152660 con 0x7f90f4068160 2023-05-16T07:38:35.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.795+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f90e8022070 con 0x7f90f4068160 2023-05-16T07:38:35.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.795+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f4066730 con 0x7f90f4068160 2023-05-16T07:38:35.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.796+0000 7f90daffd700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f90e005ede0 0x7f90e00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:35.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.796+0000 7f90f27fc700 1 -- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f90e005ede0 msgr2=0x7f90e00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:35.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.796+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f90e807a330 con 0x7f90f4068160 2023-05-16T07:38:35.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.796+0000 7f90f27fc700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f90e005ede0 0x7f90e00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:35.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.800+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f90e804b3c0 con 0x7f90f4068160 2023-05-16T07:38:35.805 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:35.801+0000 7fb223f5b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:35.878 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:35.874+0000 7fb223f5b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:38:35.898 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:35.894+0000 7f5a6751c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:35.944 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:35.940+0000 7fb223f5b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:38:35.973 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:35.969+0000 7f5a6751c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-16T07:38:35.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.982+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90f40c8ba0 con 0x7f90f4068160 2023-05-16T07:38:35.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.983+0000 7f90daffd700 1 -- 172.21.15.92:0/2289394546 <== mon.1 v2:172.21.15.158:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90e804e9e0 con 0x7f90f4068160 2023-05-16T07:38:35.987 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-05-16T07:38:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f90e005ede0 msgr2=0x7f90e00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f90e005ede0 0x7f90e00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 msgr2=0x7f90f4152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 0x7f90f4152170 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f90e800e850 tx=0x7f90e8009060).stop 2023-05-16T07:38:35.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 shutdown_connections 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f90e005ede0 0x7f90e00612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f90f4154470 0x7f90f40c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f90f4068160 0x7f90f4152170 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 --2- 172.21.15.92:0/2289394546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4154090 0x7f90f4157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 >> 172.21.15.92:0/2289394546 conn(0x7f90f4067920 msgr2=0x7f90f4153000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 shutdown_connections 2023-05-16T07:38:35.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:35.985+0000 7f90f9470700 1 -- 172.21.15.92:0/2289394546 wait complete. 2023-05-16T07:38:36.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:36.044 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:36.040+0000 7f5a6751c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-16T07:38:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.112+0000 7f6a25648700 1 Processor -- start 2023-05-16T07:38:36.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.112+0000 7f6a25648700 1 -- start start 2023-05-16T07:38:36.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a25648700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6a20069db0 0x7f6a2015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a25648700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a200c0660 con 0x7f6a20154170 2023-05-16T07:38:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a25648700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a200c1b00 con 0x7f6a20154550 2023-05-16T07:38:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a25648700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a200bfd30 con 0x7f6a20069db0 2023-05-16T07:38:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6a20154170 0x7f6a200c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35988/0 (socket says 172.21.15.92:35988) 2023-05-16T07:38:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1effd700 1 -- 172.21.15.92:0/11150274 learned_addr learned my addr 172.21.15.92:0/11150274 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:36.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1f7fe700 1 --2- 172.21.15.92:0/11150274 >> v2:172.21.15.158:3300/0 conn(0x7f6a20069db0 0x7f6a2015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112150724 0 0) 0x7f6a200c0660 con 0x7f6a20154170 2023-05-16T07:38:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a080033f0 con 0x7f6a20154170 2023-05-16T07:38:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491412862 0 0) 0x7f6a200c1b00 con 0x7f6a20154550 2023-05-16T07:38:36.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a200c0660 con 0x7f6a20154550 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1f7fe700 1 -- 172.21.15.92:0/11150274 >> v1:172.21.15.158:6789/0 conn(0x7f6a20154550 legacy=0x7f6a200696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1f7fe700 1 -- 172.21.15.92:0/11150274 >> v1:172.21.15.92:6789/0 conn(0x7f6a20154170 legacy=0x7f6a200c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1f7fe700 1 -- 172.21.15.92:0/11150274 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200c0010 con 0x7f6a20069db0 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.113+0000 7f6a1f7fe700 1 --2- 172.21.15.92:0/11150274 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20069db0 0x7f6a2015f200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f6a18009e60 tx=0x7f6a18009780).ready entity=mon.1 client_cookie=4d1c331ab84eaada server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800e070 con 0x7f6a20069db0 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1800bab0 con 0x7f6a20069db0 2023-05-16T07:38:36.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a1dffb700 1 -- 172.21.15.92:0/11150274 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800be30 con 0x7f6a20069db0 2023-05-16T07:38:36.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 -- 172.21.15.92:0/11150274 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20069db0 msgr2=0x7f6a2015f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 --2- 172.21.15.92:0/11150274 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20069db0 0x7f6a2015f200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f6a18009e60 tx=0x7f6a18009780).stop 2023-05-16T07:38:36.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 -- 172.21.15.92:0/11150274 shutdown_connections 2023-05-16T07:38:36.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 --2- 172.21.15.92:0/11150274 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20069db0 0x7f6a2015f200 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 -- 172.21.15.92:0/11150274 >> 172.21.15.92:0/11150274 conn(0x7f6a200cb700 msgr2=0x7f6a200c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:36.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 -- 172.21.15.92:0/11150274 shutdown_connections 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.114+0000 7f6a25648700 1 -- 172.21.15.92:0/11150274 wait complete. 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.115+0000 7f6a25648700 1 Processor -- start 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.115+0000 7f6a25648700 1 -- start start 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a25648700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a20069db0 0x7f6a201649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a25648700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 0x7f6a20147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a25648700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20154550 0x7f6a20150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a25648700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20147a00 con 0x7f6a20154170 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a25648700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20150de0 con 0x7f6a20154550 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a25648700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a201510c0 con 0x7f6a20069db0 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a1e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 0x7f6a20147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a1e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 0x7f6a20147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53480/0 (socket says 172.21.15.92:53480) 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a1effd700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a20069db0 0x7f6a201649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a1e7fc700 1 -- 172.21.15.92:0/2097049979 learned_addr learned my addr 172.21.15.92:0/2097049979 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.116+0000 7f6a1f7fe700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20154550 0x7f6a20150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1e7fc700 1 -- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a20069db0 msgr2=0x7f6a201649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1e7fc700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a20069db0 0x7f6a201649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1e7fc700 1 -- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20154550 msgr2=0x7f6a20150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1e7fc700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20154550 0x7f6a20150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1e7fc700 1 -- 172.21.15.92:0/2097049979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a18009a20 con 0x7f6a20154170 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1f7fe700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20154550 0x7f6a20150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1effd700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a20069db0 0x7f6a201649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:36.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a1e7fc700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 0x7f6a20147510 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f6a1400c420 tx=0x7f6a14007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a140123f0 con 0x7f6a20154170 2023-05-16T07:38:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a14012550 con 0x7f6a20154170 2023-05-16T07:38:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a201513a0 con 0x7f6a20154170 2023-05-16T07:38:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14012700 con 0x7f6a20154170 2023-05-16T07:38:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.117+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a20151960 con 0x7f6a20154170 2023-05-16T07:38:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.118+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a200666c0 con 0x7f6a20154170 2023-05-16T07:38:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.119+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f6a14010070 con 0x7f6a20154170 2023-05-16T07:38:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.119+0000 7f6a06ffd700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 0x7f6a0c0612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.119+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f6a14079180 con 0x7f6a20154170 2023-05-16T07:38:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.119+0000 7f6a1effd700 1 -- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 msgr2=0x7f6a0c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:36.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.119+0000 7f6a1effd700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 0x7f6a0c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:36.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.123+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f6a140478f0 con 0x7f6a20154170 2023-05-16T07:38:36.197 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:36.193+0000 7fb223f5b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:38:36.309 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:36.305+0000 7f5a6751c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-16T07:38:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.320+0000 7f6a1effd700 1 -- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 msgr2=0x7f6a0c0612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.320+0000 7f6a1effd700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 0x7f6a0c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.335+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a20151b50 con 0x7f6a20154170 2023-05-16T07:38:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.336+0000 7f6a06ffd700 1 -- 172.21.15.92:0/2097049979 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f6a1404a300 con 0x7f6a20154170 2023-05-16T07:38:36.340 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:36.340 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:38:19.513917+0000","modified":"2023-05-16T07:38:21.520884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/862724662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":862724662},{"type":"v1","addr":"172.21.15.158:6837","nonce":862724662}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-05-16T07:38:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 msgr2=0x7f6a0c0612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 0x7f6a0c0612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 msgr2=0x7f6a20147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 0x7f6a20147510 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f6a1400c420 tx=0x7f6a14007c10).stop 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 shutdown_connections 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f6a0c05ede0 0x7f6a0c0612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6a20069db0 0x7f6a201649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a20154550 0x7f6a20150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 --2- 172.21.15.92:0/2097049979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a20154170 0x7f6a20147510 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 >> 172.21.15.92:0/2097049979 conn(0x7f6a200cb700 msgr2=0x7f6a200696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 shutdown_connections 2023-05-16T07:38:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.338+0000 7f6a25648700 1 -- 172.21.15.92:0/2097049979 wait complete. 2023-05-16T07:38:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-05-16T07:38:36.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:36.380 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:36.376+0000 7fb223f5b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:36.502 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:36.498+0000 7f5a6751c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-16T07:38:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.508+0000 7ff53346a700 1 Processor -- start 2023-05-16T07:38:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.508+0000 7ff53346a700 1 -- start start 2023-05-16T07:38:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff53346a700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7ff52c0c10a0 0x7ff52c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff53346a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c158900 con 0x7ff52c0c7ac0 2023-05-16T07:38:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff53346a700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c157cf0 con 0x7ff52c0c76e0 2023-05-16T07:38:36.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff53346a700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c157fd0 con 0x7ff52c0c10a0 2023-05-16T07:38:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531c67700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff52c0c7ac0 0x7ff52c153820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36004/0 (socket says 172.21.15.92:36004) 2023-05-16T07:38:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531c67700 1 -- 172.21.15.92:0/635345375 learned_addr learned my addr 172.21.15.92:0/635345375 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff532c69700 1 --2- 172.21.15.92:0/635345375 >> v2:172.21.15.158:3300/0 conn(0x7ff52c0c10a0 0x7ff52c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94204283 0 0) 0x7ff52c158900 con 0x7ff52c0c7ac0 2023-05-16T07:38:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5180033f0 con 0x7ff52c0c7ac0 2023-05-16T07:38:36.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974857651 0 0) 0x7ff52c157cf0 con 0x7ff52c0c76e0 2023-05-16T07:38:36.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c158900 con 0x7ff52c0c76e0 2023-05-16T07:38:36.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff532c69700 1 -- 172.21.15.92:0/635345375 >> v1:172.21.15.158:6789/0 conn(0x7ff52c0c76e0 legacy=0x7ff52c0a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612786982 0 0) 0x7ff5180033f0 con 0x7ff52c0c7ac0 2023-05-16T07:38:36.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff532c69700 1 -- 172.21.15.92:0/635345375 >> v1:172.21.15.92:6789/0 conn(0x7ff52c0c7ac0 legacy=0x7ff52c153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff532c69700 1 -- 172.21.15.92:0/635345375 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c1582b0 con 0x7ff52c0c10a0 2023-05-16T07:38:36.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.509+0000 7ff532c69700 1 --2- 172.21.15.92:0/635345375 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c10a0 0x7ff52c153f30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7ff524009e60 tx=0x7ff524009780).ready entity=mon.1 client_cookie=4b45f82990384c4e server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52400e070 con 0x7ff52c0c10a0 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52400b7a0 con 0x7ff52c0c10a0 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff531466700 1 -- 172.21.15.92:0/635345375 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528001470 con 0x7ff52c0c10a0 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 -- 172.21.15.92:0/635345375 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c10a0 msgr2=0x7ff52c153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 --2- 172.21.15.92:0/635345375 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c10a0 0x7ff52c153f30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7ff524009e60 tx=0x7ff524009780).stop 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 -- 172.21.15.92:0/635345375 shutdown_connections 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 --2- 172.21.15.92:0/635345375 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c10a0 0x7ff52c153f30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 -- 172.21.15.92:0/635345375 >> 172.21.15.92:0/635345375 conn(0x7ff52c068330 msgr2=0x7ff52c068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 -- 172.21.15.92:0/635345375 shutdown_connections 2023-05-16T07:38:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.510+0000 7ff53346a700 1 -- 172.21.15.92:0/635345375 wait complete. 2023-05-16T07:38:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.511+0000 7ff53346a700 1 Processor -- start 2023-05-16T07:38:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.511+0000 7ff53346a700 1 -- start start 2023-05-16T07:38:36.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff53346a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff52c0c10a0 0x7ff52c147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff53346a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 0x7ff52c0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff53346a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c7ac0 0x7ff52c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff53346a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c148460 con 0x7ff52c0c76e0 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff53346a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c2ad0 con 0x7ff52c0c7ac0 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff53346a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c1ec0 con 0x7ff52c0c10a0 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff531c67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 0x7ff52c0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff531c67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 0x7ff52c0c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53496/0 (socket says 172.21.15.92:53496) 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff531c67700 1 -- 172.21.15.92:0/4255349043 learned_addr learned my addr 172.21.15.92:0/4255349043 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:36.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff532c69700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c7ac0 0x7ff52c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff532468700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff52c0c10a0 0x7ff52c147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff531c67700 1 -- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff52c0c10a0 msgr2=0x7ff52c147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.512+0000 7ff531c67700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff52c0c10a0 0x7ff52c147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff531c67700 1 -- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c7ac0 msgr2=0x7ff52c0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff531c67700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c7ac0 0x7ff52c0ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff531c67700 1 -- 172.21.15.92:0/4255349043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524009a20 con 0x7ff52c0c76e0 2023-05-16T07:38:36.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff531c67700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 0x7ff52c0c4290 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7ff52800c830 tx=0x7ff528001a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528007170 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5280072f0 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c0c21a0 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280048c0 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.513+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c0c2760 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.514+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c0c2c80 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.514+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7ff52801b070 con 0x7ff52c0c76e0 2023-05-16T07:38:36.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.515+0000 7ff5227fc700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff51005edb0 0x7ff510061270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.515+0000 7ff532468700 1 -- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff51005edb0 msgr2=0x7ff510061270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:36.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.515+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7ff528079710 con 0x7ff52c0c76e0 2023-05-16T07:38:36.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.515+0000 7ff532468700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff51005edb0 0x7ff510061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:36.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.519+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7ff52804a890 con 0x7ff52c0c76e0 2023-05-16T07:38:36.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.689+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff52c0bfb90 con 0x7ff52c0c76e0 2023-05-16T07:38:36.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.690+0000 7ff5227fc700 1 -- 172.21.15.92:0/4255349043 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18255 (secure 0 0 0) 0x7ff528019020 con 0x7ff52c0c76e0 2023-05-16T07:38:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:36.695 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":172,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:30.703341+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:38:18.041970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-05-16T07:38:18.481177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3460186346":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6833/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6832/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:0/240635450":"2023-05-17T07:38:30.703311+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:36.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.691+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff51005edb0 msgr2=0x7ff510061270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:36.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.691+0000 7ff53346a700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff51005edb0 0x7ff510061270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:36.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 msgr2=0x7ff52c0c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 0x7ff52c0c4290 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7ff52800c830 tx=0x7ff528001a20).stop 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 shutdown_connections 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7ff51005edb0 0x7ff510061270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7ff52c0c10a0 0x7ff52c147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff52c0c7ac0 0x7ff52c0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 --2- 172.21.15.92:0/4255349043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff52c0c76e0 0x7ff52c0c4290 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 >> 172.21.15.92:0/4255349043 conn(0x7ff52c068330 msgr2=0x7ff52c14f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 shutdown_connections 2023-05-16T07:38:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.692+0000 7ff53346a700 1 -- 172.21.15.92:0/4255349043 wait complete. 2023-05-16T07:38:36.708 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:38:36.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:36.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.818+0000 7fdfba56f700 1 Processor -- start 2023-05-16T07:38:36.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.818+0000 7fdfba56f700 1 -- start start 2023-05-16T07:38:36.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfba56f700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdfb4150230 0x7fdfb4066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfba56f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150610 0x7fdfb4065430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfba56f700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb4066e50 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfba56f700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb40676f0 con 0x7fdfb4150230 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfba56f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb4065920 con 0x7fdfb4150610 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfb956d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fdfb4150230 0x7fdfb4066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfb8d6c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150610 0x7fdfb4065430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfb9d6e700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fdfb40bc3d0 0x7fdfb40c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:39632/0 (socket says 172.21.15.92:39632) 2023-05-16T07:38:36.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.819+0000 7fdfb9d6e700 1 -- 172.21.15.92:0/1462235630 learned_addr learned my addr 172.21.15.92:0/1462235630 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:36.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfb956d700 1 -- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150610 msgr2=0x7fdfb4065430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:36.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfb956d700 1 --2- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150610 0x7fdfb4065430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfb956d700 1 -- 172.21.15.92:0/1462235630 >> v1:172.21.15.158:6789/0 conn(0x7fdfb40bc3d0 legacy=0x7fdfb40c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfb956d700 1 -- 172.21.15.92:0/1462235630 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4065be0 con 0x7fdfb4150230 2023-05-16T07:38:36.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfb8d6c700 1 --2- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150610 0x7fdfb4065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:36.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfb956d700 1 --2- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150230 0x7fdfb4066420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fdfa4009760 tx=0x7fdfa4009060).ready entity=mon.1 client_cookie=e43fdf2aa9acfe0c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfabfff700 1 -- 172.21.15.92:0/1462235630 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400d4a0 con 0x7fdfb4150230 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfabfff700 1 -- 172.21.15.92:0/1462235630 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400d600 con 0x7fdfb4150230 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfabfff700 1 -- 172.21.15.92:0/1462235630 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400d980 con 0x7fdfb4150230 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfba56f700 1 -- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150230 msgr2=0x7fdfb4066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150230 0x7fdfb4066420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fdfa4009760 tx=0x7fdfa4009060).stop 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfba56f700 1 -- 172.21.15.92:0/1462235630 shutdown_connections 2023-05-16T07:38:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150230 0x7fdfb4066420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1462235630 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150610 0x7fdfb4065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.820+0000 7fdfba56f700 1 -- 172.21.15.92:0/1462235630 >> 172.21.15.92:0/1462235630 conn(0x7fdfb4069940 msgr2=0x7fdfb40c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:36.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.821+0000 7fdfba56f700 1 -- 172.21.15.92:0/1462235630 shutdown_connections 2023-05-16T07:38:36.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.821+0000 7fdfba56f700 1 -- 172.21.15.92:0/1462235630 wait complete. 2023-05-16T07:38:36.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 Processor -- start 2023-05-16T07:38:36.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 -- start start 2023-05-16T07:38:36.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 0x7fdfb4142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150230 0x7fdfb4144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150610 0x7fdfb40ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb41436c0 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb41439a0 con 0x7fdfb4150610 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.822+0000 7fdfba56f700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb4143c80 con 0x7fdfb4150230 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 0x7fdfb4142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 0x7fdfb4142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53518/0 (socket says 172.21.15.92:53518) 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 -- 172.21.15.92:0/1616481571 learned_addr learned my addr 172.21.15.92:0/1616481571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb8d6c700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150230 0x7fdfb4144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb9d6e700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150610 0x7fdfb40ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 -- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150230 msgr2=0x7fdfb4144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150230 0x7fdfb4144f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 -- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150610 msgr2=0x7fdfb40ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150610 0x7fdfb40ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 -- 172.21.15.92:0/1616481571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb00092a0 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfb956d700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 0x7fdfb4142ee0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7fdfa4009e00 tx=0x7fdfa4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4000c90 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400d770 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa40092a0 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa401d430 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.823+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb41433d0 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.824+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb4047150 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.824+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7fdfa400f070 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.825+0000 7fdfa97fa700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 0x7fdf980612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.825+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fdfa407b4a0 con 0x7fdfb40bc3d0 2023-05-16T07:38:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.825+0000 7fdfb8d6c700 1 -- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 msgr2=0x7fdf980612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.825+0000 7fdfb8d6c700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 0x7fdf980612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:36.829+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fdfa404c6a0 con 0x7fdfb40bc3d0 2023-05-16T07:38:37.023 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.019+0000 7fb223f5b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:38:37.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.026+0000 7fdfb8d6c700 1 -- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 msgr2=0x7fdf980612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:37.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.026+0000 7fdfb8d6c700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 0x7fdf980612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:37.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.037+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfb405ede0 con 0x7fdfb40bc3d0 2023-05-16T07:38:37.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.037+0000 7fdfa97fa700 1 -- 172.21.15.92:0/1616481571 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fdfa4016070 con 0x7fdfb40bc3d0 2023-05-16T07:38:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:37.042 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:38:19.513917+0000","modified":"2023-05-16T07:38:21.520884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/862724662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":862724662},{"type":"v1","addr":"172.21.15.158:6837","nonce":862724662}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-05-16T07:38:37.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 msgr2=0x7fdf980612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:37.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 0x7fdf980612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 msgr2=0x7fdfb4142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 0x7fdfb4142ee0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7fdfa4009e00 tx=0x7fdfa4014000).stop 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 shutdown_connections 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdfb4150610 0x7fdfb40ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfb40bc3d0 0x7fdfb4142ee0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fdf9805ede0 0x7fdf980612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:37.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 --2- 172.21.15.92:0/1616481571 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fdfb4150230 0x7fdfb4144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 >> 172.21.15.92:0/1616481571 conn(0x7fdfb4069940 msgr2=0x7fdfb40c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:37.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 shutdown_connections 2023-05-16T07:38:37.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.039+0000 7fdfba56f700 1 -- 172.21.15.92:0/1616481571 wait complete. 2023-05-16T07:38:37.045 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-05-16T07:38:37.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:38:37.090 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.086+0000 7fb223f5b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:38:37.163 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.159+0000 7fb223f5b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:38:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.168+0000 7f7106f0a700 1 Processor -- start 2023-05-16T07:38:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.168+0000 7f7106f0a700 1 -- start start 2023-05-16T07:38:37.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7106f0a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f710013a5a0 0x7f7100068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7106f0a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7100068060 0x7f71000bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7106f0a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000bdbb0 con 0x7f71000bd780 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7106f0a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7100154b80 con 0x7f710013a5a0 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7106f0a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7100154e60 con 0x7f7100068060 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f710013a5a0 0x7f7100068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f71054a7700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f71000bd780 0x7f7100065000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36048/0 (socket says 172.21.15.92:36048) 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f710013a5a0 0x7f7100068600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53528/0 (socket says 172.21.15.92:53528) 2023-05-16T07:38:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f71054a7700 1 -- 172.21.15.92:0/3494990167 learned_addr learned my addr 172.21.15.92:0/3494990167 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:37.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f70fffff700 1 --2- 172.21.15.92:0/3494990167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7100068060 0x7f71000bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 -- 172.21.15.92:0/3494990167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7100068060 msgr2=0x7f71000bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:37.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 --2- 172.21.15.92:0/3494990167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7100068060 0x7f71000bd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 -- 172.21.15.92:0/3494990167 >> v1:172.21.15.92:6789/0 conn(0x7f71000bd780 legacy=0x7f7100065000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 -- 172.21.15.92:0/3494990167 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100155140 con 0x7f710013a5a0 2023-05-16T07:38:37.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f70ff7fe700 1 -- 172.21.15.92:0/3494990167 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587893119 0 0) 0x7f71000bdbb0 con 0x7f71000bd780 2023-05-16T07:38:37.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.169+0000 7f7104ca6700 1 --2- 172.21.15.92:0/3494990167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710013a5a0 0x7f7100068600 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f70f4009760 tx=0x7f70f4009060).ready entity=mon.0 client_cookie=28ef7d98e2804684 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:37.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f70ff7fe700 1 -- 172.21.15.92:0/3494990167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400f510 con 0x7f710013a5a0 2023-05-16T07:38:37.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f70ff7fe700 1 -- 172.21.15.92:0/3494990167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f400f670 con 0x7f710013a5a0 2023-05-16T07:38:37.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f70ff7fe700 1 -- 172.21.15.92:0/3494990167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400f9f0 con 0x7f710013a5a0 2023-05-16T07:38:37.177 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f5a6751c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-16T07:38:37.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 -- 172.21.15.92:0/3494990167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710013a5a0 msgr2=0x7f7100068600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 --2- 172.21.15.92:0/3494990167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710013a5a0 0x7f7100068600 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f70f4009760 tx=0x7f70f4009060).stop 2023-05-16T07:38:37.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 -- 172.21.15.92:0/3494990167 shutdown_connections 2023-05-16T07:38:37.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 --2- 172.21.15.92:0/3494990167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f710013a5a0 0x7f7100068600 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 --2- 172.21.15.92:0/3494990167 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7100068060 0x7f71000bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 -- 172.21.15.92:0/3494990167 >> 172.21.15.92:0/3494990167 conn(0x7f71000c4430 msgr2=0x7f71000c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:37.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 -- 172.21.15.92:0/3494990167 shutdown_connections 2023-05-16T07:38:37.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.170+0000 7f7106f0a700 1 -- 172.21.15.92:0/3494990167 wait complete. 2023-05-16T07:38:37.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.171+0000 7f7106f0a700 1 Processor -- start 2023-05-16T07:38:37.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.171+0000 7f7106f0a700 1 -- start start 2023-05-16T07:38:37.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7106f0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 0x7f710014c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7106f0a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71000bd780 0x7f710014bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7106f0a700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f710013a5a0 0x7f7100165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7106f0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000c6140 con 0x7f7100068060 2023-05-16T07:38:37.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7106f0a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000c6420 con 0x7f71000bd780 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7106f0a700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000c6700 con 0x7f710013a5a0 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7104ca6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 0x7f710014c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7104ca6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 0x7f710014c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53534/0 (socket says 172.21.15.92:53534) 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f7104ca6700 1 -- 172.21.15.92:0/1344396659 learned_addr learned my addr 172.21.15.92:0/1344396659 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f71054a7700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f710013a5a0 0x7f7100165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.172+0000 7f70fffff700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71000bd780 0x7f710014bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7104ca6700 1 -- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f710013a5a0 msgr2=0x7f7100165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7104ca6700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f710013a5a0 0x7f7100165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7104ca6700 1 -- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71000bd780 msgr2=0x7f710014bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7104ca6700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71000bd780 0x7f710014bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7104ca6700 1 -- 172.21.15.92:0/1344396659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40092a0 con 0x7f7100068060 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7104ca6700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 0x7f710014c990 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f7100064cf0 tx=0x7f70f4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400f7e0 con 0x7f7100068060 2023-05-16T07:38:37.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f400f960 con 0x7f7100068060 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71001655c0 con 0x7f7100068060 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f401b690 con 0x7f7100068060 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.173+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71000c5cb0 con 0x7f7100068060 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.174+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7100165780 con 0x7f7100068060 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.174+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f70f402a070 con 0x7f7100068060 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.175+0000 7f70fcff9700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 0x7f70e40612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.175+0000 7f70fffff700 1 -- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 msgr2=0x7f70e40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:37.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.175+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f70f407b450 con 0x7f7100068060 2023-05-16T07:38:37.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.175+0000 7f70fffff700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 0x7f70e40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:37.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.179+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f70f404c5d0 con 0x7f7100068060 2023-05-16T07:38:37.246 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.242+0000 7f5a6751c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-16T07:38:37.308 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.304+0000 7fb223f5b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:38:37.321 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.317+0000 7f5a6751c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-16T07:38:37.376 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.372+0000 7fb223f5b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:38:37.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.376+0000 7f70fffff700 1 -- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 msgr2=0x7f70e40612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:37.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.376+0000 7f70fffff700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 0x7f70e40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:37.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.390+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7100165930 con 0x7f7100068060 2023-05-16T07:38:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.390+0000 7f70fcff9700 1 -- 172.21.15.92:0/1344396659 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f70f4028090 con 0x7f7100068060 2023-05-16T07:38:37.395 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:37.395 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":61,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-16T07:38:19.513917+0000","modified":"2023-05-16T07:38:21.520884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.158:6837/862724662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":862724662},{"type":"v1","addr":"172.21.15.158:6837","nonce":862724662}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-05-16T07:38:37.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.392+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 msgr2=0x7f70e40612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:37.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.392+0000 7f7106f0a700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 0x7f70e40612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:37.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.392+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 msgr2=0x7f710014c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.392+0000 7f7106f0a700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 0x7f710014c990 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f7100064cf0 tx=0x7f70f4017000).stop 2023-05-16T07:38:37.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 shutdown_connections 2023-05-16T07:38:37.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f70e405ede0 0x7f70e40612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:37.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f710013a5a0 0x7f7100165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7100068060 0x7f710014c990 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 --2- 172.21.15.92:0/1344396659 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f71000bd780 0x7f710014bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 >> 172.21.15.92:0/1344396659 conn(0x7f71000c4430 msgr2=0x7f71000c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:37.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 shutdown_connections 2023-05-16T07:38:37.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.393+0000 7f7106f0a700 1 -- 172.21.15.92:0/1344396659 wait complete. 2023-05-16T07:38:37.399 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 61 2023-05-16T07:38:37.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:37.472 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.468+0000 7f5a6751c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-16T07:38:37.489 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.485+0000 7fb223f5b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:38:37.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.519+0000 7f50b925b700 1 Processor -- start 2023-05-16T07:38:37.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.519+0000 7f50b925b700 1 -- start start 2023-05-16T07:38:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b925b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f50b4154170 0x7f50b40aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b925b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b40c5cb0 0x7f50b40c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b925b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c6650 con 0x7f50b4154550 2023-05-16T07:38:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b925b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b40ab200 con 0x7f50b4154170 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b925b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b40c08b0 con 0x7f50b40c5cb0 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f50b4154550 0x7f50b40a9d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36060/0 (socket says 172.21.15.92:36060) 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b259c700 1 -- 172.21.15.92:0/1349421502 learned_addr learned my addr 172.21.15.92:0/1349421502 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b2d9d700 1 --2- 172.21.15.92:0/1349421502 >> v2:172.21.15.158:3300/0 conn(0x7f50b4154170 0x7f50b40aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.520+0000 7f50b359e700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b40c5cb0 0x7f50b40c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b1d9b700 1 -- 172.21.15.92:0/1349421502 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801880403 0 0) 0x7f50a8001110 con 0x7f50b4154550 2023-05-16T07:38:37.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b1d9b700 1 -- 172.21.15.92:0/1349421502 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f509c0033f0 con 0x7f50b4154550 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b2d9d700 1 -- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b40c5cb0 msgr2=0x7f50b40c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b2d9d700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b40c5cb0 0x7f50b40c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b2d9d700 1 -- 172.21.15.92:0/1349421502 >> v1:172.21.15.92:6789/0 conn(0x7f50b4154550 legacy=0x7f50b40a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b2d9d700 1 -- 172.21.15.92:0/1349421502 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40bfca0 con 0x7f50b4154170 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b359e700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b40c5cb0 0x7f50b40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b2d9d700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b4154170 0x7f50b40aad10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f50a4009760 tx=0x7f50a4009060).ready entity=mon.1 client_cookie=2093de0d155f7b79 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:37.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b1d9b700 1 -- 172.21.15.92:0/1349421502 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400d4a0 con 0x7f50b4154170 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b1d9b700 1 -- 172.21.15.92:0/1349421502 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a400d600 con 0x7f50b4154170 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.521+0000 7f50b1d9b700 1 -- 172.21.15.92:0/1349421502 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400d980 con 0x7f50b4154170 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 -- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b4154170 msgr2=0x7f50b40aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b4154170 0x7f50b40aad10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f50a4009760 tx=0x7f50a4009060).stop 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 -- 172.21.15.92:0/1349421502 shutdown_connections 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b40c5cb0 0x7f50b40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 --2- 172.21.15.92:0/1349421502 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b4154170 0x7f50b40aad10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 -- 172.21.15.92:0/1349421502 >> 172.21.15.92:0/1349421502 conn(0x7f50b40cb6d0 msgr2=0x7f50b40c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 -- 172.21.15.92:0/1349421502 shutdown_connections 2023-05-16T07:38:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.522+0000 7f50b925b700 1 -- 172.21.15.92:0/1349421502 wait complete. 2023-05-16T07:38:37.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.523+0000 7f50b925b700 1 Processor -- start 2023-05-16T07:38:37.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.523+0000 7f50b925b700 1 -- start start 2023-05-16T07:38:37.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b40c5cb0 0x7f50b40c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 0x7f50b41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b4154550 0x7f50b40c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41505f0 con 0x7f50b4154170 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41682e0 con 0x7f50b40c5cb0 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41685c0 con 0x7f50b4154550 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 0x7f50b41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 0x7f50b41517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53552/0 (socket says 172.21.15.92:53552) 2023-05-16T07:38:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 -- 172.21.15.92:0/514456455 learned_addr learned my addr 172.21.15.92:0/514456455 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b2d9d700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b40c5cb0 0x7f50b40c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b359e700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b4154550 0x7f50b40c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 -- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b4154550 msgr2=0x7f50b40c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b4154550 0x7f50b40c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 -- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b40c5cb0 msgr2=0x7f50b40c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b40c5cb0 0x7f50b40c2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 -- 172.21.15.92:0/514456455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac009a20 con 0x7f50b4154170 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b259c700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 0x7f50b41517b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7f50a800c410 tx=0x7f50a8007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800ce30 con 0x7f50b4154170 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a40092a0 con 0x7f50b4154170 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a80072e0 con 0x7f50b4154170 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.524+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80057a0 con 0x7f50b4154170 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.525+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b4168b80 con 0x7f50b4154170 2023-05-16T07:38:37.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.526+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b40666c0 con 0x7f50b4154170 2023-05-16T07:38:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.526+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f50a8010070 con 0x7f50b4154170 2023-05-16T07:38:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.526+0000 7f509affd700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f50a005ede0 0x7f50a00612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.526+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f50a8079470 con 0x7f50b4154170 2023-05-16T07:38:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.526+0000 7f50b2d9d700 1 -- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f50a005ede0 msgr2=0x7f50a00612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:37.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.526+0000 7f50b2d9d700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f50a005ede0 0x7f50a00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:37.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.530+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f50a804a670 con 0x7f50b4154170 2023-05-16T07:38:37.545 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.541+0000 7f5a6751c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-16T07:38:37.595 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.591+0000 7fb223f5b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:37.661 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.657+0000 7f5a6751c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-16T07:38:37.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.698+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50b40c6650 con 0x7f50b4154170 2023-05-16T07:38:37.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.699+0000 7f509affd700 1 -- 172.21.15.92:0/514456455 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f50b40c6650 con 0x7f50b4154170 2023-05-16T07:38:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:37.704 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":172,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:30.703341+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-05-16T07:38:18.041970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-05-16T07:38:18.481177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:0/3460186346":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6833/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6832/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:0/240635450":"2023-05-17T07:38:30.703311+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f50a005ede0 msgr2=0x7f50a00612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f50a005ede0 0x7f50a00612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 msgr2=0x7f50b41517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 0x7f50b41517b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7f50a800c410 tx=0x7f50a8007ad0).stop 2023-05-16T07:38:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 shutdown_connections 2023-05-16T07:38:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f50a005ede0 0x7f50a00612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f50b4154550 0x7f50b40c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f50b40c5cb0 0x7f50b40c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 --2- 172.21.15.92:0/514456455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50b4154170 0x7f50b41517b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 >> 172.21.15.92:0/514456455 conn(0x7f50b40cb6d0 msgr2=0x7f50b4068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 shutdown_connections 2023-05-16T07:38:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.701+0000 7f50b925b700 1 -- 172.21.15.92:0/514456455 wait complete. 2023-05-16T07:38:37.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-16T07:38:37.773 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:37.769+0000 7f5a6751c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-16T07:38:37.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.829+0000 7f6089bec700 1 Processor -- start 2023-05-16T07:38:37.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.830+0000 7f6089bec700 1 -- start start 2023-05-16T07:38:37.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.830+0000 7f6089bec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f6084069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.830+0000 7f6089bec700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f6084063e30 0x7f60840bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.830+0000 7f6089bec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6084062b10 con 0x7f6084154180 2023-05-16T07:38:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.830+0000 7f6089bec700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6084068620 con 0x7f6084063e30 2023-05-16T07:38:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.830+0000 7f6089bec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60840622b0 con 0x7f6084153da0 2023-05-16T07:38:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f607affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6084154180 0x7f6084063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36076/0 (socket says 172.21.15.92:36076) 2023-05-16T07:38:37.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f607affd700 1 -- 172.21.15.92:0/1298096982 learned_addr learned my addr 172.21.15.92:0/1298096982 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6083fff700 1 --2- 172.21.15.92:0/1298096982 >> v2:172.21.15.158:3300/0 conn(0x7f6084063e30 0x7f60840bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f60837fe700 1 --2- 172.21.15.92:0/1298096982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f6084069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6082ffd700 1 -- 172.21.15.92:0/1298096982 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082129461 0 0) 0x7f6084062b10 con 0x7f6084154180 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6082ffd700 1 -- 172.21.15.92:0/1298096982 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60700033f0 con 0x7f6084154180 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f60837fe700 1 -- 172.21.15.92:0/1298096982 >> v2:172.21.15.158:3300/0 conn(0x7f6084063e30 msgr2=0x7f60840bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f60837fe700 1 --2- 172.21.15.92:0/1298096982 >> v2:172.21.15.158:3300/0 conn(0x7f6084063e30 0x7f60840bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f60837fe700 1 -- 172.21.15.92:0/1298096982 >> v1:172.21.15.92:6789/0 conn(0x7f6084154180 legacy=0x7f6084063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f60837fe700 1 -- 172.21.15.92:0/1298096982 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084068340 con 0x7f6084153da0 2023-05-16T07:38:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6083fff700 1 --2- 172.21.15.92:0/1298096982 >> v2:172.21.15.158:3300/0 conn(0x7f6084063e30 0x7f60840bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f60837fe700 1 --2- 172.21.15.92:0/1298096982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f6084069a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f6074009760 tx=0x7f6074009060).ready entity=mon.2 client_cookie=a04e3bc15f33ba69 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6082ffd700 1 -- 172.21.15.92:0/1298096982 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400d4a0 con 0x7f6084153da0 2023-05-16T07:38:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6082ffd700 1 -- 172.21.15.92:0/1298096982 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400d600 con 0x7f6084153da0 2023-05-16T07:38:37.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.831+0000 7f6082ffd700 1 -- 172.21.15.92:0/1298096982 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400d980 con 0x7f6084153da0 2023-05-16T07:38:37.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 -- 172.21.15.92:0/1298096982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 msgr2=0x7f6084069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 --2- 172.21.15.92:0/1298096982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f6084069a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f6074009760 tx=0x7f6074009060).stop 2023-05-16T07:38:37.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 -- 172.21.15.92:0/1298096982 shutdown_connections 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 --2- 172.21.15.92:0/1298096982 >> v2:172.21.15.158:3300/0 conn(0x7f6084063e30 0x7f60840bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 --2- 172.21.15.92:0/1298096982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f6084069a10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 -- 172.21.15.92:0/1298096982 >> 172.21.15.92:0/1298096982 conn(0x7f60840c8aa0 msgr2=0x7f60840c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 -- 172.21.15.92:0/1298096982 shutdown_connections 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.832+0000 7f6089bec700 1 -- 172.21.15.92:0/1298096982 wait complete. 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.833+0000 7f6089bec700 1 Processor -- start 2023-05-16T07:38:37.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.833+0000 7f6089bec700 1 -- start start 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6089bec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 0x7f6084150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6089bec700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f60840ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6089bec700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6084154180 0x7f60840ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6089bec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60840adca0 con 0x7f6084063e30 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6089bec700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60841651a0 con 0x7f6084154180 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6089bec700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60840c5ea0 con 0x7f6084153da0 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f60837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 0x7f6084150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f60837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 0x7f6084150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53564/0 (socket says 172.21.15.92:53564) 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f60837fe700 1 -- 172.21.15.92:0/1698398257 learned_addr learned my addr 172.21.15.92:0/1698398257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f607affd700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f60840ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.834+0000 7f6083fff700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6084154180 0x7f60840ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f60837fe700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 msgr2=0x7f60840ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f60837fe700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f60840ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f60837fe700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6084154180 msgr2=0x7f60840ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:37.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f60837fe700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6084154180 0x7f60840ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f60837fe700 1 -- 172.21.15.92:0/1698398257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c009a20 con 0x7f6084063e30 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f60837fe700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 0x7f6084150820 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7f6074009260 tx=0x7f6074016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400dd40 con 0x7f6084063e30 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400d770 con 0x7f6084063e30 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60740092a0 con 0x7f6084063e30 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400dea0 con 0x7f6084063e30 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.835+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60840c5850 con 0x7f6084063e30 2023-05-16T07:38:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.836+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608406c350 con 0x7f6084063e30 2023-05-16T07:38:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.836+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 84) v1 ==== 75362+0+0 (secure 0 0 0) 0x7f6074023020 con 0x7f6084063e30 2023-05-16T07:38:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.837+0000 7f607bfff700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 0x7f60640612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:37.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.837+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f607407ba00 con 0x7f6084063e30 2023-05-16T07:38:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.837+0000 7f607affd700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 msgr2=0x7f60640612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.837+0000 7f607affd700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 0x7f60640612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-05-16T07:38:37.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:37.841+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f607404cc80 con 0x7f6084063e30 2023-05-16T07:38:37.884 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.881+0000 7fb223f5b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:38:37.954 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:37.950+0000 7fb223f5b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:38:38.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.038+0000 7f607affd700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 msgr2=0x7f60640612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:38.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.038+0000 7f607affd700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 0x7f60640612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-05-16T07:38:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.051+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f60840624c0 con 0x7f6084063e30 2023-05-16T07:38:38.070 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:38.066+0000 7f5a6751c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-16T07:38:38.140 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-05-16T07:38:38.136+0000 7f5a6751c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-16T07:38:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.439+0000 7f607affd700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 msgr2=0x7f60640612a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.158:6832/95623 2023-05-16T07:38:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.439+0000 7f607affd700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 0x7f60640612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-05-16T07:38:38.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.796+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 85) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f607400f070 con 0x7f6084063e30 2023-05-16T07:38:38.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.796+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 msgr2=0x7f60640612a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-16T07:38:38.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.796+0000 7f607bfff700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 0x7f60640612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:38.849 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a3f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a3f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a3f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a3f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a3f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a6f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a6f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.850 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a6f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.851 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a6f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.851 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-05-16T07:38:38.846+0000 7fb1a6f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-05-16T07:38:38.852 INFO:tasks.ceph.mds.d.smithi158.stderr:2023-05-16T07:38:38.849+0000 7f5af3808700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:38.852 INFO:tasks.ceph.mds.d.smithi158.stderr:2023-05-16T07:38:38.849+0000 7f5af3808700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:38.852 INFO:tasks.ceph.mds.d.smithi158.stderr:2023-05-16T07:38:38.849+0000 7f5af3808700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:38:38.855 INFO:tasks.ceph.mds.d.smithi158.stderr: -2233> 2023-05-16T07:38:25.445+0000 7f5af700f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-16T07:38:38.858 INFO:tasks.ceph.mds.d.smithi158.stderr: -4> 2023-05-16T07:38:38.849+0000 7f5af3808700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:38.858 INFO:tasks.ceph.mds.d.smithi158.stderr: -3> 2023-05-16T07:38:38.849+0000 7f5af3808700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-05-16T07:38:38.858 INFO:tasks.ceph.mds.d.smithi158.stderr: -1> 2023-05-16T07:38:38.849+0000 7f5af3808700 -1 MDSIOContextBase: failed with -108, restarting... 2023-05-16T07:38:38.877 INFO:tasks.ceph.mds.d.smithi158.stdout:starting mds.d at 2023-05-16T07:38:38.972 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-05-16T07:38:38.968+0000 7f7b715c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-16T07:38:38.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.976+0000 7f607bfff700 1 -- 172.21.15.92:0/1698398257 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6074042b00 con 0x7f6084063e30 2023-05-16T07:38:38.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 msgr2=0x7f6084150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 0x7f6084150820 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7f6074009260 tx=0x7f6074016000).stop 2023-05-16T07:38:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 shutdown_connections 2023-05-16T07:38:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6084154180 0x7f60840ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084063e30 0x7f6084150820 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f606405ede0 0x7f60640612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:38.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 --2- 172.21.15.92:0/1698398257 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6084153da0 0x7f60840ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:38.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 >> 172.21.15.92:0/1698398257 conn(0x7f60840c8aa0 msgr2=0x7f6084063720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:38.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 shutdown_connections 2023-05-16T07:38:38.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:38.978+0000 7f6089bec700 1 -- 172.21.15.92:0/1698398257 wait complete. 2023-05-16T07:38:38.985 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-16T07:38:38.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-16T07:38:39.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.109+0000 7f9131ee9700 1 Processor -- start 2023-05-16T07:38:39.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.109+0000 7f9131ee9700 1 -- start start 2023-05-16T07:38:39.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f9131ee9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f912c154470 0x7f912c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:39.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f9131ee9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f912c068160 0x7f912c0685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f9131ee9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0c94d0 con 0x7f912c154090 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f9131ee9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c0cb7e0 con 0x7f912c154470 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f9131ee9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c068aa0 con 0x7f912c068160 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f912c154470 0x7f912c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f912c154090 0x7f912c0a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36098/0 (socket says 172.21.15.92:36098) 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912b7fe700 1 -- 172.21.15.92:0/2593662333 learned_addr learned my addr 172.21.15.92:0/2593662333 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:39.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912bfff700 1 --2- 172.21.15.92:0/2593662333 >> v2:172.21.15.158:3300/0 conn(0x7f912c068160 0x7f912c0685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912affd700 1 -- 172.21.15.92:0/2593662333 >> v2:172.21.15.158:3300/0 conn(0x7f912c068160 msgr2=0x7f912c0685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912affd700 1 --2- 172.21.15.92:0/2593662333 >> v2:172.21.15.158:3300/0 conn(0x7f912c068160 0x7f912c0685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912affd700 1 -- 172.21.15.92:0/2593662333 >> v1:172.21.15.92:6789/0 conn(0x7f912c154090 legacy=0x7f912c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912a7fc700 1 -- 172.21.15.92:0/2593662333 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268057615 0 0) 0x7f912c0c94d0 con 0x7f912c154090 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.110+0000 7f912affd700 1 -- 172.21.15.92:0/2593662333 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0c88c0 con 0x7f912c154470 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f912affd700 1 --2- 172.21.15.92:0/2593662333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c154470 0x7f912c0aaca0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f912c05f9e0 tx=0x7f9120009060).ready entity=mon.0 client_cookie=db5fbd9d67b7d98a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:39.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f912a7fc700 1 -- 172.21.15.92:0/2593662333 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000b580 con 0x7f912c154470 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f912a7fc700 1 -- 172.21.15.92:0/2593662333 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912000b6e0 con 0x7f912c154470 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f912a7fc700 1 -- 172.21.15.92:0/2593662333 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000ba60 con 0x7f912c154470 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 -- 172.21.15.92:0/2593662333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c154470 msgr2=0x7f912c0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 --2- 172.21.15.92:0/2593662333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c154470 0x7f912c0aaca0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f912c05f9e0 tx=0x7f9120009060).stop 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 -- 172.21.15.92:0/2593662333 shutdown_connections 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 --2- 172.21.15.92:0/2593662333 >> v2:172.21.15.158:3300/0 conn(0x7f912c068160 0x7f912c0685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 --2- 172.21.15.92:0/2593662333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c154470 0x7f912c0aaca0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 -- 172.21.15.92:0/2593662333 >> 172.21.15.92:0/2593662333 conn(0x7f912c067920 msgr2=0x7f912c0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 -- 172.21.15.92:0/2593662333 shutdown_connections 2023-05-16T07:38:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.111+0000 7f9131ee9700 1 -- 172.21.15.92:0/2593662333 wait complete. 2023-05-16T07:38:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.112+0000 7f9131ee9700 1 Processor -- start 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 -- start start 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 0x7f912c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c154090 0x7f912c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c154470 0x7f912c0c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c152c00 con 0x7f912c068160 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c153090 con 0x7f912c154470 2023-05-16T07:38:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f9131ee9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c14cf50 con 0x7f912c154090 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 0x7f912c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 0x7f912c151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53588/0 (socket says 172.21.15.92:53588) 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f912b7fe700 1 -- 172.21.15.92:0/3129971872 learned_addr learned my addr 172.21.15.92:0/3129971872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f912affd700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c154090 0x7f912c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.113+0000 7f912bfff700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c154470 0x7f912c0c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912b7fe700 1 -- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c154090 msgr2=0x7f912c14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912b7fe700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c154090 0x7f912c14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912b7fe700 1 -- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c154470 msgr2=0x7f912c0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912b7fe700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c154470 0x7f912c0c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912b7fe700 1 -- 172.21.15.92:0/3129971872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91200092a0 con 0x7f912c068160 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912bfff700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c154470 0x7f912c0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f912b7fe700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 0x7f912c151a10 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7f911c00c960 tx=0x7f911c001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c005d30 con 0x7f912c068160 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c005e90 con 0x7f912c068160 2023-05-16T07:38:39.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c151ff0 con 0x7f912c068160 2023-05-16T07:38:39.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c007320 con 0x7f912c068160 2023-05-16T07:38:39.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.114+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c1525b0 con 0x7f912c068160 2023-05-16T07:38:39.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.115+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c152ee0 con 0x7f912c068160 2023-05-16T07:38:39.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.115+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 85) v1 ==== 75092+0+0 (secure 0 0 0) 0x7f911c010030 con 0x7f912c068160 2023-05-16T07:38:39.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.115+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7262+0+0 (secure 0 0 0) 0x7f911c079e50 con 0x7f912c068160 2023-05-16T07:38:39.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.120+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f911c04b020 con 0x7f912c068160 2023-05-16T07:38:39.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.321+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f912c06c3b0 con 0x7f912c068160 2023-05-16T07:38:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.799+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 86) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f911c077980 con 0x7f912c068160 2023-05-16T07:38:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.799+0000 7f91137fe700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f91180301b0 0x7f9118067e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.801+0000 7f912affd700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f91180301b0 0x7f9118067e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.803+0000 7f912affd700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f91180301b0 0x7f9118067e60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f912000b340 tx=0x7f9120017000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.978+0000 7f91137fe700 1 -- 172.21.15.92:0/3129971872 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f911c06f7c0 con 0x7f912c068160 2023-05-16T07:38:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f91180301b0 msgr2=0x7f9118067e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f91180301b0 0x7f9118067e60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f912000b340 tx=0x7f9120017000).stop 2023-05-16T07:38:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 msgr2=0x7f912c151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 0x7f912c151a10 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7f911c00c960 tx=0x7f911c001630).stop 2023-05-16T07:38:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 shutdown_connections 2023-05-16T07:38:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f912c154470 0x7f912c0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f912c068160 0x7f912c151a10 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f91180301b0 0x7f9118067e60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 --2- 172.21.15.92:0/3129971872 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f912c154090 0x7f912c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 >> 172.21.15.92:0/3129971872 conn(0x7f912c067920 msgr2=0x7f912c14fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 shutdown_connections 2023-05-16T07:38:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:39.980+0000 7f9131ee9700 1 -- 172.21.15.92:0/3129971872 wait complete. 2023-05-16T07:38:39.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-16T07:38:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.108+0000 7f385e9e7700 1 Processor -- start 2023-05-16T07:38:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.108+0000 7f385e9e7700 1 -- start start 2023-05-16T07:38:40.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385e9e7700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f3858154470 0x7f38580aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385e9e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 0x7f38580685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385e9e7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38580c94d0 con 0x7f3858154090 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385e9e7700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38580cb7e0 con 0x7f3858154470 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385e9e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3858068aa0 con 0x7f3858068160 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3857fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3858154090 0x7f38580a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36128/0 (socket says 172.21.15.92:36128) 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3857fff700 1 -- 172.21.15.92:0/307690793 learned_addr learned my addr 172.21.15.92:0/307690793 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f38577fe700 1 --2- 172.21.15.92:0/307690793 >> v2:172.21.15.158:3300/0 conn(0x7f3858154470 0x7f38580aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385cf84700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 0x7f38580685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552741847 0 0) 0x7f38580c94d0 con 0x7f3858154090 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38400033f0 con 0x7f3858154090 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3459541561 0 0) 0x7f38400033f0 con 0x7f3858154090 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 msgr2=0x7f38580685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 0x7f38580685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 >> v2:172.21.15.158:3300/0 conn(0x7f3858154470 msgr2=0x7f38580aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:40.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 --2- 172.21.15.92:0/307690793 >> v2:172.21.15.158:3300/0 conn(0x7f3858154470 0x7f38580aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38580c88c0 con 0x7f3858154090 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f384c001470 con 0x7f3858154090 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 >> v1:172.21.15.92:6789/0 conn(0x7f3858154090 legacy=0x7f38580a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 0x7f38400038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3840003e10 0x7f3840004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3840005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580cb7e0 con 0x7f3840004b80 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3858068aa0 con 0x7f3840003e10 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384000b8e0 con 0x7f3858068160 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.109+0000 7f385cf84700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3840005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f3857fff700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 0x7f38400038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f38577fe700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3840003e10 0x7f3840004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 -- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 msgr2=0x7f38400038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858068160 0x7f38400038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 -- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3840003e10 msgr2=0x7f3840004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3840003e10 0x7f3840004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 -- 172.21.15.92:0/307690793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400033f0 con 0x7f3840004b80 2023-05-16T07:38:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f38577fe700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3840003e10 0x7f3840004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3840005000 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f385000e240 tx=0x7f385000b040).ready entity=mon.0 client_cookie=112f84d9c8b8484c server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:40.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850009ac0 con 0x7f3840004b80 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385000f040 con 0x7f3840004b80 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f3856ffd700 1 -- 172.21.15.92:0/307690793 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385000ed00 con 0x7f3840004b80 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385e9e7700 1 -- 172.21.15.92:0/307690793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 msgr2=0x7f3840005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385e9e7700 1 --2- 172.21.15.92:0/307690793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3840005000 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f385000e240 tx=0x7f385000b040).stop 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385cf84700 1 -- 172.21.15.92:0/307690793 reap_dead start 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.110+0000 7f385e9e7700 1 -- 172.21.15.92:0/307690793 shutdown_connections 2023-05-16T07:38:40.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.111+0000 7f385e9e7700 1 -- 172.21.15.92:0/307690793 >> 172.21.15.92:0/307690793 conn(0x7f3858067920 msgr2=0x7f38580cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.111+0000 7f385e9e7700 1 -- 172.21.15.92:0/307690793 shutdown_connections 2023-05-16T07:38:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.111+0000 7f385e9e7700 1 -- 172.21.15.92:0/307690793 wait complete. 2023-05-16T07:38:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.112+0000 7f385e9e7700 1 Processor -- start 2023-05-16T07:38:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.112+0000 7f385e9e7700 1 -- start start 2023-05-16T07:38:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.112+0000 7f385e9e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3858159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.112+0000 7f385e9e7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f385814fc30 0x7f3858164360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385e9e7700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858159f60 0x7f3858164850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385e9e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3858164d40 con 0x7f3840004b80 2023-05-16T07:38:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385e9e7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385815a470 con 0x7f385814fc30 2023-05-16T07:38:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385e9e7700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385815a750 con 0x7f3858159f60 2023-05-16T07:38:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3858159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3858159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53624/0 (socket says 172.21.15.92:53624) 2023-05-16T07:38:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 -- 172.21.15.92:0/2202469878 learned_addr learned my addr 172.21.15.92:0/2202469878 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385cf84700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858159f60 0x7f3858164850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f38577fe700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f385814fc30 0x7f3858164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 -- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858159f60 msgr2=0x7f3858164850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858159f60 0x7f3858164850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 -- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f385814fc30 msgr2=0x7f3858164360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f385814fc30 0x7f3858164360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 -- 172.21.15.92:0/2202469878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385815aa30 con 0x7f3840004b80 2023-05-16T07:38:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f3857fff700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3858159a70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f384c007fd0 tx=0x7f384c001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00f6c0 con 0x7f3840004b80 2023-05-16T07:38:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00f820 con 0x7f3840004b80 2023-05-16T07:38:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385815ad10 con 0x7f3840004b80 2023-05-16T07:38:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00fbc0 con 0x7f3840004b80 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.113+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3858158520 con 0x7f3840004b80 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.114+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3858155b90 con 0x7f3840004b80 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.114+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 86) v1 ==== 75162+0+0 (secure 0 0 0) 0x7f384c01b030 con 0x7f3840004b80 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.115+0000 7f383ffff700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f384405ecf0 0x7f38440611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.115+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7262+0+0 (secure 0 0 0) 0x7f384c0798a0 con 0x7f3840004b80 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.115+0000 7f38577fe700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f384405ecf0 0x7f38440611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.115+0000 7f38577fe700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f384405ecf0 0x7f38440611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3848009270 tx=0x7f3848008040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.119+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f384c04aa70 con 0x7f3840004b80 2023-05-16T07:38:40.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.282+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3858066730 con 0x7f3840004b80 2023-05-16T07:38:40.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:38:40.292 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:38:40.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.979+0000 7f383ffff700 1 -- 172.21.15.92:0/2202469878 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f384c019020 con 0x7f3840004b80 2023-05-16T07:38:40.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f384405ecf0 msgr2=0x7f38440611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:40.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f384405ecf0 0x7f38440611b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3848009270 tx=0x7f3848008040).stop 2023-05-16T07:38:40.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 msgr2=0x7f3858159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3858159a70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f384c007fd0 tx=0x7f384c001e90).stop 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 shutdown_connections 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f384405ecf0 0x7f38440611b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f3858159f60 0x7f3858164850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3840004b80 0x7f3858159a70 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 --2- 172.21.15.92:0/2202469878 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f385814fc30 0x7f3858164360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 >> 172.21.15.92:0/2202469878 conn(0x7f3858067920 msgr2=0x7f38580ab670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:40.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 shutdown_connections 2023-05-16T07:38:40.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:40.982+0000 7f385e9e7700 1 -- 172.21.15.92:0/2202469878 wait complete. 2023-05-16T07:38:40.987 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-05-16T07:38:40.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-16T07:38:41.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.114+0000 7f15aac8e700 1 Processor -- start 2023-05-16T07:38:41.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.114+0000 7f15aac8e700 1 -- start start 2023-05-16T07:38:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.114+0000 7f15aac8e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15a4154470 0x7f15a40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.114+0000 7f15aac8e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a40685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15aac8e700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a40c94d0 con 0x7f15a4154090 2023-05-16T07:38:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15aac8e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40cb7e0 con 0x7f15a4154470 2023-05-16T07:38:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15aac8e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4068aa0 con 0x7f15a4068160 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15a4154470 0x7f15a40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f15a4154470 0x7f15a40aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53636/0 (socket says 172.21.15.92:53636) 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 -- 172.21.15.92:0/2543758256 learned_addr learned my addr 172.21.15.92:0/2543758256 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 -- 172.21.15.92:0/2543758256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 msgr2=0x7f15a40685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 --2- 172.21.15.92:0/2543758256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a40685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 -- 172.21.15.92:0/2543758256 >> v1:172.21.15.158:6789/0 conn(0x7f15a4154090 legacy=0x7f15a40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:41.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 -- 172.21.15.92:0/2543758256 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40c88c0 con 0x7f15a4154470 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a3fff700 1 --2- 172.21.15.92:0/2543758256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154470 0x7f15a40aaca0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7f1598006fd0 tx=0x7f15980090f0).ready entity=mon.0 client_cookie=328b36a5d9357521 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a37fe700 1 -- 172.21.15.92:0/2543758256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800b600 con 0x7f15a4154470 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a37fe700 1 -- 172.21.15.92:0/2543758256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800b760 con 0x7f15a4154470 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.115+0000 7f15a37fe700 1 -- 172.21.15.92:0/2543758256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800bae0 con 0x7f15a4154470 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 -- 172.21.15.92:0/2543758256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154470 msgr2=0x7f15a40aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2543758256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154470 0x7f15a40aaca0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7f1598006fd0 tx=0x7f15980090f0).stop 2023-05-16T07:38:41.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 -- 172.21.15.92:0/2543758256 shutdown_connections 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2543758256 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a40685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2543758256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154470 0x7f15a40aaca0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 -- 172.21.15.92:0/2543758256 >> 172.21.15.92:0/2543758256 conn(0x7f15a4067920 msgr2=0x7f15a40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 -- 172.21.15.92:0/2543758256 shutdown_connections 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.116+0000 7f15aac8e700 1 -- 172.21.15.92:0/2543758256 wait complete. 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.117+0000 7f15aac8e700 1 Processor -- start 2023-05-16T07:38:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.117+0000 7f15aac8e700 1 -- start start 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15aac8e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a4167740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15aac8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 0x7f15a40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15aac8e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a4154470 0x7f15a41569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15aac8e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4164d20 con 0x7f15a4154090 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15aac8e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4165000 con 0x7f15a4154470 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15aac8e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41652e0 con 0x7f15a4068160 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 0x7f15a40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 0x7f15a40ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53642/0 (socket says 172.21.15.92:53642) 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 -- 172.21.15.92:0/2886947841 learned_addr learned my addr 172.21.15.92:0/2886947841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a922b700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a4154470 0x7f15a41569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a8a2a700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a4167740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 -- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 msgr2=0x7f15a4167740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a4167740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 -- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a4154470 msgr2=0x7f15a41569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a4154470 0x7f15a41569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 -- 172.21.15.92:0/2886947841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009370 con 0x7f15a4154090 2023-05-16T07:38:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a922b700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a4154470 0x7f15a41569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a8a2a700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a4167740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.118+0000 7f15a3fff700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 0x7f15a40ca040 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f15a406bb30 tx=0x7f1598017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.119+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800b8d0 con 0x7f15a4154090 2023-05-16T07:38:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.119+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800ba30 con 0x7f15a4154090 2023-05-16T07:38:41.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.119+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a41655c0 con 0x7f15a4154090 2023-05-16T07:38:41.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.119+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159801b660 con 0x7f15a4154090 2023-05-16T07:38:41.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.119+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a40ca530 con 0x7f15a4154090 2023-05-16T07:38:41.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.119+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a4066730 con 0x7f15a4154090 2023-05-16T07:38:41.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.120+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f159802b070 con 0x7f15a4154090 2023-05-16T07:38:41.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.120+0000 7f15a0ff9700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f159005ed90 0x7f1590061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:41.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.120+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f159807b600 con 0x7f15a4154090 2023-05-16T07:38:41.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.120+0000 7f15a8a2a700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f159005ed90 0x7f1590061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:41.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.121+0000 7f15a8a2a700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f159005ed90 0x7f1590061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f15a4165950 tx=0x7f1594009040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:41.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.124+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f159804c8e0 con 0x7f15a4154090 2023-05-16T07:38:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.292+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15a4165750 con 0x7f15a4154090 2023-05-16T07:38:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:41.999+0000 7f15a0ff9700 1 -- 172.21.15.92:0/2886947841 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f159801f020 con 0x7f15a4154090 2023-05-16T07:38:42.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f159005ed90 msgr2=0x7f1590061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f159005ed90 0x7f1590061250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f15a4165950 tx=0x7f1594009040).stop 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 msgr2=0x7f15a40ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 0x7f15a40ca040 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f15a406bb30 tx=0x7f1598017000).stop 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 shutdown_connections 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f159005ed90 0x7f1590061250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f15a4068160 0x7f15a4167740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f15a4154470 0x7f15a41569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 --2- 172.21.15.92:0/2886947841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15a4154090 0x7f15a40ca040 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 >> 172.21.15.92:0/2886947841 conn(0x7f15a4067920 msgr2=0x7f15a414f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 shutdown_connections 2023-05-16T07:38:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.002+0000 7f15aac8e700 1 -- 172.21.15.92:0/2886947841 wait complete. 2023-05-16T07:38:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-05-16T07:38:42.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-16T07:38:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.137+0000 7f196dd67700 1 Processor -- start 2023-05-16T07:38:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.137+0000 7f196dd67700 1 -- start start 2023-05-16T07:38:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196dd67700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1968150920 0x7f19680a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196dd67700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f19681519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196dd67700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1968151520 con 0x7f1968151ea0 2023-05-16T07:38:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196dd67700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19680a7aa0 con 0x7f1968150920 2023-05-16T07:38:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196dd67700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680a8480 con 0x7f1968150d00 2023-05-16T07:38:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196d566700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1968151ea0 0x7f1968062980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36162/0 (socket says 172.21.15.92:36162) 2023-05-16T07:38:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196d566700 1 -- 172.21.15.92:0/48515599 learned_addr learned my addr 172.21.15.92:0/48515599 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f196cd65700 1 --2- 172.21.15.92:0/48515599 >> v2:172.21.15.92:3300/0 conn(0x7f1968150920 0x7f19680a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.138+0000 7f1967fff700 1 --2- 172.21.15.92:0/48515599 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f19681519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f19677fe700 1 -- 172.21.15.92:0/48515599 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488028553 0 0) 0x7f1968151520 con 0x7f1968151ea0 2023-05-16T07:38:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f19677fe700 1 -- 172.21.15.92:0/48515599 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19580033f0 con 0x7f1968151ea0 2023-05-16T07:38:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f196cd65700 1 -- 172.21.15.92:0/48515599 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 msgr2=0x7f19681519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f196cd65700 1 --2- 172.21.15.92:0/48515599 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f19681519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f196cd65700 1 -- 172.21.15.92:0/48515599 >> v1:172.21.15.92:6789/0 conn(0x7f1968151ea0 legacy=0x7f1968062980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f196cd65700 1 -- 172.21.15.92:0/48515599 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680a8760 con 0x7f1968150920 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f196cd65700 1 --2- 172.21.15.92:0/48515599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968150920 0x7f19680a75b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f195c009760 tx=0x7f195c009060).ready entity=mon.0 client_cookie=73c4fe32c50752bd server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f19677fe700 1 -- 172.21.15.92:0/48515599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00d4a0 con 0x7f1968150920 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f19677fe700 1 -- 172.21.15.92:0/48515599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c00d600 con 0x7f1968150920 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.139+0000 7f19677fe700 1 -- 172.21.15.92:0/48515599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00d980 con 0x7f1968150920 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 -- 172.21.15.92:0/48515599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968150920 msgr2=0x7f19680a75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 --2- 172.21.15.92:0/48515599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968150920 0x7f19680a75b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f195c009760 tx=0x7f195c009060).stop 2023-05-16T07:38:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 -- 172.21.15.92:0/48515599 shutdown_connections 2023-05-16T07:38:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 --2- 172.21.15.92:0/48515599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968150920 0x7f19680a75b0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 --2- 172.21.15.92:0/48515599 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f19681519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 -- 172.21.15.92:0/48515599 >> 172.21.15.92:0/48515599 conn(0x7f1968068a30 msgr2=0x7f19680c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 -- 172.21.15.92:0/48515599 shutdown_connections 2023-05-16T07:38:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.140+0000 7f196dd67700 1 -- 172.21.15.92:0/48515599 wait complete. 2023-05-16T07:38:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.141+0000 7f196dd67700 1 Processor -- start 2023-05-16T07:38:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.141+0000 7f196dd67700 1 -- start start 2023-05-16T07:38:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196dd67700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1968150920 0x7f1968136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196dd67700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f196813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196dd67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 0x7f1968161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196dd67700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680ad760 con 0x7f1968151ea0 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196dd67700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680aa950 con 0x7f1968150920 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196dd67700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680aaf10 con 0x7f1968150d00 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 0x7f1968161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 0x7f1968161de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53656/0 (socket says 172.21.15.92:53656) 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 -- 172.21.15.92:0/4275663234 learned_addr learned my addr 172.21.15.92:0/4275663234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f1967fff700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f196813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196cd65700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1968150920 0x7f1968136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 -- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 msgr2=0x7f196813df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f196813df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 -- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1968150920 msgr2=0x7f1968136b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1968150920 0x7f1968136b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196d566700 1 -- 172.21.15.92:0/4275663234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0092a0 con 0x7f1968151ea0 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f1967fff700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f196813df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:42.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.142+0000 7f196cd65700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1968150920 0x7f1968136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.143+0000 7f196d566700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 0x7f1968161de0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f1960005ad0 tx=0x7f19600015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.143+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960005d30 con 0x7f1968151ea0 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.143+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960005e90 con 0x7f1968151ea0 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.143+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960007320 con 0x7f1968151ea0 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.143+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19681628f0 con 0x7f1968151ea0 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.143+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19680aaba0 con 0x7f1968151ea0 2023-05-16T07:38:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.144+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968047150 con 0x7f1968151ea0 2023-05-16T07:38:42.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.144+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1960010030 con 0x7f1968151ea0 2023-05-16T07:38:42.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.144+0000 7f1964ff9700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f194c05ed90 0x7f194c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.145+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1960079b00 con 0x7f1968151ea0 2023-05-16T07:38:42.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.145+0000 7f196cd65700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f194c05ed90 0x7f194c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.145+0000 7f196cd65700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f194c05ed90 0x7f194c061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f195c00d260 tx=0x7f195c016000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.149+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f196004af80 con 0x7f1968151ea0 2023-05-16T07:38:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.315+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f19680ab050 con 0x7f1968151ea0 2023-05-16T07:38:42.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.319+0000 7f1964ff9700 1 -- 172.21.15.92:0/4275663234 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f196004e5a0 con 0x7f1968151ea0 2023-05-16T07:38:42.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f194c05ed90 msgr2=0x7f194c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f194c05ed90 0x7f194c061250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f195c00d260 tx=0x7f195c016000).stop 2023-05-16T07:38:42.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 msgr2=0x7f1968161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 0x7f1968161de0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f1960005ad0 tx=0x7f19600015b0).stop 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 shutdown_connections 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1968151ea0 0x7f1968161de0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1968150920 0x7f1968136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f194c05ed90 0x7f194c061250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 --2- 172.21.15.92:0/4275663234 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1968150d00 0x7f196813df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 >> 172.21.15.92:0/4275663234 conn(0x7f1968068a30 msgr2=0x7f1968069980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:42.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 shutdown_connections 2023-05-16T07:38:42.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.321+0000 7f196dd67700 1 -- 172.21.15.92:0/4275663234 wait complete. 2023-05-16T07:38:42.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-05-16T07:38:42.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.445+0000 7efe6edf2700 1 Processor -- start 2023-05-16T07:38:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.445+0000 7efe6edf2700 1 -- start start 2023-05-16T07:38:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6edf2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe68154090 0x7efe680aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6edf2700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7efe68068160 0x7efe680685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6edf2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe680c94d0 con 0x7efe68154470 2023-05-16T07:38:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6edf2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe680cb7e0 con 0x7efe68154090 2023-05-16T07:38:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6edf2700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe68068aa0 con 0x7efe68068160 2023-05-16T07:38:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe68154090 0x7efe680aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe67fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efe68154470 0x7efe680a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36178/0 (socket says 172.21.15.92:36178) 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe67fff700 1 -- 172.21.15.92:0/2265509257 learned_addr learned my addr 172.21.15.92:0/2265509257 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 --2- 172.21.15.92:0/2265509257 >> v2:172.21.15.92:3300/0 conn(0x7efe68154090 0x7efe680aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53660/0 (socket says 172.21.15.92:53660) 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6d38f700 1 --2- 172.21.15.92:0/2265509257 >> v2:172.21.15.158:3300/0 conn(0x7efe68068160 0x7efe680685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 -- 172.21.15.92:0/2265509257 >> v2:172.21.15.158:3300/0 conn(0x7efe68068160 msgr2=0x7efe680685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 --2- 172.21.15.92:0/2265509257 >> v2:172.21.15.158:3300/0 conn(0x7efe68068160 0x7efe680685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe677fe700 1 -- 172.21.15.92:0/2265509257 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848316455 0 0) 0x7efe680c94d0 con 0x7efe68154470 2023-05-16T07:38:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 -- 172.21.15.92:0/2265509257 >> v1:172.21.15.92:6789/0 conn(0x7efe68154470 legacy=0x7efe680a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 -- 172.21.15.92:0/2265509257 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe680c88c0 con 0x7efe68154090 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6d38f700 1 --2- 172.21.15.92:0/2265509257 >> v2:172.21.15.158:3300/0 conn(0x7efe68068160 0x7efe680685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.446+0000 7efe6cb8e700 1 --2- 172.21.15.92:0/2265509257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68154090 0x7efe680aaca0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7efe6805f9e0 tx=0x7efe60009060).ready entity=mon.0 client_cookie=6a2c2cce4bb412b2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe677fe700 1 -- 172.21.15.92:0/2265509257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6000b580 con 0x7efe68154090 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe677fe700 1 -- 172.21.15.92:0/2265509257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6000b6e0 con 0x7efe68154090 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe677fe700 1 -- 172.21.15.92:0/2265509257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6000ba60 con 0x7efe68154090 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 -- 172.21.15.92:0/2265509257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68154090 msgr2=0x7efe680aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 --2- 172.21.15.92:0/2265509257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68154090 0x7efe680aaca0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7efe6805f9e0 tx=0x7efe60009060).stop 2023-05-16T07:38:42.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 -- 172.21.15.92:0/2265509257 shutdown_connections 2023-05-16T07:38:42.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 --2- 172.21.15.92:0/2265509257 >> v2:172.21.15.158:3300/0 conn(0x7efe68068160 0x7efe680685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 --2- 172.21.15.92:0/2265509257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68154090 0x7efe680aaca0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 -- 172.21.15.92:0/2265509257 >> 172.21.15.92:0/2265509257 conn(0x7efe68067920 msgr2=0x7efe680cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:42.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 -- 172.21.15.92:0/2265509257 shutdown_connections 2023-05-16T07:38:42.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.447+0000 7efe6edf2700 1 -- 172.21.15.92:0/2265509257 wait complete. 2023-05-16T07:38:42.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.448+0000 7efe6edf2700 1 Processor -- start 2023-05-16T07:38:42.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.448+0000 7efe6edf2700 1 -- start start 2023-05-16T07:38:42.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6edf2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 0x7efe68164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6edf2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efe68154090 0x7efe680bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6edf2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe68154470 0x7efe680c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6edf2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe680c0570 con 0x7efe68068160 2023-05-16T07:38:42.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6edf2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68164750 con 0x7efe68154470 2023-05-16T07:38:42.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6edf2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe68165200 con 0x7efe68154090 2023-05-16T07:38:42.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 0x7efe68164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 0x7efe68164260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53668/0 (socket says 172.21.15.92:53668) 2023-05-16T07:38:42.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 -- 172.21.15.92:0/3704075940 learned_addr learned my addr 172.21.15.92:0/3704075940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:42.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe67fff700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efe68154090 0x7efe680bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 -- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efe68154090 msgr2=0x7efe680bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efe68154090 0x7efe680bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6d38f700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe68154470 0x7efe680c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 -- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe68154470 msgr2=0x7efe680c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe68154470 0x7efe680c0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.449+0000 7efe6cb8e700 1 -- 172.21.15.92:0/3704075940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600092a0 con 0x7efe68068160 2023-05-16T07:38:42.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.450+0000 7efe6cb8e700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 0x7efe68164260 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7efe6806bb30 tx=0x7efe60017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.450+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6000b850 con 0x7efe68068160 2023-05-16T07:38:42.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.450+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6000b9b0 con 0x7efe68068160 2023-05-16T07:38:42.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.450+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe681655f0 con 0x7efe68068160 2023-05-16T07:38:42.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.450+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6001b660 con 0x7efe68068160 2023-05-16T07:38:42.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.450+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68164960 con 0x7efe68068160 2023-05-16T07:38:42.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.451+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe681653c0 con 0x7efe68068160 2023-05-16T07:38:42.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.451+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7efe6002b070 con 0x7efe68068160 2023-05-16T07:38:42.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.451+0000 7efe64ff9700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efe4805ed90 0x7efe48061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.451+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efe6007b710 con 0x7efe68068160 2023-05-16T07:38:42.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.452+0000 7efe67fff700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efe4805ed90 0x7efe48061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.452+0000 7efe67fff700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efe4805ed90 0x7efe48061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7efe58007fd0 tx=0x7efe58009040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.455+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7efe681653c0 con 0x7efe68068160 2023-05-16T07:38:42.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.618+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7efe681653c0 con 0x7efe68068160 2023-05-16T07:38:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.622+0000 7efe64ff9700 1 -- 172.21.15.92:0/3704075940 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v37) v1 ==== 120+0+0 (secure 0 0 0) 0x7efe681653c0 con 0x7efe68068160 2023-05-16T07:38:42.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.623+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efe4805ed90 msgr2=0x7efe48061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.623+0000 7efe6edf2700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efe4805ed90 0x7efe48061250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7efe58007fd0 tx=0x7efe58009040).stop 2023-05-16T07:38:42.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.623+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 msgr2=0x7efe68164260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.623+0000 7efe6edf2700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 0x7efe68164260 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7efe6806bb30 tx=0x7efe60017000).stop 2023-05-16T07:38:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 shutdown_connections 2023-05-16T07:38:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe68154470 0x7efe680c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe68068160 0x7efe68164260 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efe4805ed90 0x7efe48061250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 --2- 172.21.15.92:0/3704075940 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efe68154090 0x7efe680bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 >> 172.21.15.92:0/3704075940 conn(0x7efe68067920 msgr2=0x7efe680ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 shutdown_connections 2023-05-16T07:38:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.624+0000 7efe6edf2700 1 -- 172.21.15.92:0/3704075940 wait complete. 2023-05-16T07:38:42.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-05-16T07:38:42.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.748+0000 7f486d6f1700 1 Processor -- start 2023-05-16T07:38:42.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.748+0000 7f486d6f1700 1 -- start start 2023-05-16T07:38:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.749+0000 7f486d6f1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 0x7f48680c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.749+0000 7f486d6f1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680c0660 con 0x7f4868154550 2023-05-16T07:38:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.749+0000 7f486d6f1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680bfd30 con 0x7f4868069db0 2023-05-16T07:38:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.749+0000 7f486d6f1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c0010 con 0x7f4868154170 2023-05-16T07:38:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.749+0000 7f48667fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4868154550 0x7f48680696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36194/0 (socket says 172.21.15.92:36194) 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.749+0000 7f48667fc700 1 -- 172.21.15.92:0/2250716989 learned_addr learned my addr 172.21.15.92:0/2250716989 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4866ffd700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 0x7f48680c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625201078 0 0) 0x7f48680c0660 con 0x7f4868154550 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48540033f0 con 0x7f4868154550 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 125238660 0 0) 0x7f48540033f0 con 0x7f4868154550 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 msgr2=0x7f48680c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 0x7f48680c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 >> v1:172.21.15.158:6789/0 conn(0x7f4868069db0 legacy=0x7f48680c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680c02f0 con 0x7f4868154550 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f4850001470 con 0x7f4868154550 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 >> v1:172.21.15.92:6789/0 conn(0x7f4868154550 legacy=0x7f48680696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 0x7f4854003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4854003e00 0x7f4854004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4854004b80 0x7f4854005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c0010 con 0x7f4854004b80 2023-05-16T07:38:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485400b990 con 0x7f4854003e00 2023-05-16T07:38:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485400bb50 con 0x7f4868154170 2023-05-16T07:38:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48677fe700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4854004b80 0x7f4854005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f4866ffd700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 0x7f4854003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48667fc700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4854003e00 0x7f4854004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48677fe700 1 -- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 msgr2=0x7f4854003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48677fe700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868154170 0x7f4854003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48677fe700 1 -- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4854003e00 msgr2=0x7f4854004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48677fe700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4854003e00 0x7f4854004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.750+0000 7f48677fe700 1 -- 172.21.15.92:0/2250716989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540033f0 con 0x7f4854004b80 2023-05-16T07:38:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f48677fe700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4854004b80 0x7f4854005000 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 rx=0x7f48600017f0 tx=0x7f4860007b90).ready entity=mon.0 client_cookie=d9a7d9c54ef87357 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860007ea0 con 0x7f4854004b80 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4860012400 con 0x7f4854004b80 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f4865ffb700 1 -- 172.21.15.92:0/2250716989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860012780 con 0x7f4854004b80 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f486d6f1700 1 -- 172.21.15.92:0/2250716989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4854004b80 msgr2=0x7f4854005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f486d6f1700 1 --2- 172.21.15.92:0/2250716989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4854004b80 0x7f4854005000 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 rx=0x7f48600017f0 tx=0x7f4860007b90).stop 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f48677fe700 1 -- 172.21.15.92:0/2250716989 reap_dead start 2023-05-16T07:38:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f486d6f1700 1 -- 172.21.15.92:0/2250716989 shutdown_connections 2023-05-16T07:38:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f486d6f1700 1 -- 172.21.15.92:0/2250716989 >> 172.21.15.92:0/2250716989 conn(0x7f48680cb700 msgr2=0x7f48680c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f486d6f1700 1 -- 172.21.15.92:0/2250716989 shutdown_connections 2023-05-16T07:38:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.751+0000 7f486d6f1700 1 -- 172.21.15.92:0/2250716989 wait complete. 2023-05-16T07:38:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.752+0000 7f486d6f1700 1 Processor -- start 2023-05-16T07:38:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.752+0000 7f486d6f1700 1 -- start start 2023-05-16T07:38:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f486d6f1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4868147880 0x7f48680ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f486d6f1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868147c60 0x7f48680c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f486d6f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 0x7f48680c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f486d6f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48680ca480 con 0x7f48681490d0 2023-05-16T07:38:42.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f486d6f1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486814a250 con 0x7f4868147880 2023-05-16T07:38:42.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f486d6f1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4868149640 con 0x7f4868147c60 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f48677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 0x7f48680c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f48677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 0x7f48680c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53690/0 (socket says 172.21.15.92:53690) 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f48677fe700 1 -- 172.21.15.92:0/663787689 learned_addr learned my addr 172.21.15.92:0/663787689 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f4866ffd700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4868147880 0x7f48680ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.753+0000 7f48667fc700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868147c60 0x7f48680c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f48677fe700 1 -- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868147c60 msgr2=0x7f48680c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f48677fe700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868147c60 0x7f48680c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f48677fe700 1 -- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4868147880 msgr2=0x7f48680ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f48677fe700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4868147880 0x7f48680ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f48677fe700 1 -- 172.21.15.92:0/663787689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4868149920 con 0x7f48681490d0 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f48677fe700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 0x7f48680c9f90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f4854005670 tx=0x7f4860015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860007ea0 con 0x7f48681490d0 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48600075a0 con 0x7f48681490d0 2023-05-16T07:38:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4868149c00 con 0x7f48681490d0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860006ac0 con 0x7f48681490d0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.754+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48680c1eb0 con 0x7f48681490d0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.755+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48680b0cf0 con 0x7f48681490d0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.755+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4860023030 con 0x7f48681490d0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.756+0000 7f485effd700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f484805ed60 0x7f4848061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.756+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f486007a230 con 0x7f48681490d0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.756+0000 7f4866ffd700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f484805ed60 0x7f4848061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.756+0000 7f4866ffd700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f484805ed60 0x7f4848061220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f48680ca660 tx=0x7f485800b040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.760+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f486004b760 con 0x7f48681490d0 2023-05-16T07:38:42.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.984+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f48680666c0 con 0x7f48681490d0 2023-05-16T07:38:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:42.999+0000 7f485effd700 1 -- 172.21.15.92:0/663787689 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v1259) v1 ==== 155+0+0 (secure 0 0 0) 0x7f4860017020 con 0x7f48681490d0 2023-05-16T07:38:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.000+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f484805ed60 msgr2=0x7f4848061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:43.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.000+0000 7f486d6f1700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f484805ed60 0x7f4848061220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f48680ca660 tx=0x7f485800b040).stop 2023-05-16T07:38:43.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.000+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 msgr2=0x7f48680c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:43.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.000+0000 7f486d6f1700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 0x7f48680c9f90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f4854005670 tx=0x7f4860015000).stop 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 shutdown_connections 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48681490d0 0x7f48680c9f90 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4868147880 0x7f48680ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f484805ed60 0x7f4848061220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 --2- 172.21.15.92:0/663787689 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4868147c60 0x7f48680c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 >> 172.21.15.92:0/663787689 conn(0x7f48680cb700 msgr2=0x7f48680696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 shutdown_connections 2023-05-16T07:38:43.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.001+0000 7f486d6f1700 1 -- 172.21.15.92:0/663787689 wait complete. 2023-05-16T07:38:43.016 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-05-16T07:38:43.016 INFO:tasks.cephfs_test_runner: 2023-05-16T07:38:43.016 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-05-16T07:38:43.017 INFO:tasks.cephfs_test_runner:Ran 9 tests in 2149.248s 2023-05-16T07:38:43.017 INFO:tasks.cephfs_test_runner: 2023-05-16T07:38:43.017 INFO:tasks.cephfs_test_runner:OK 2023-05-16T07:38:43.017 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-05-16T07:38:43.028 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-05-16T07:38:43.088 INFO:tasks.kclient:Unmounting kernel clients... 2023-05-16T07:38:43.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.089 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:38:43.103 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-16T07:38:43.103 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-16T07:38:43.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-16T07:38:43.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-16T07:38:43.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-05-16T07:38:43.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.348 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-05-16T07:38:43.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-05-16T07:38:43.428 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-05-16T07:38:43.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:38:43.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.431 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list 2023-05-16T07:38:43.449 INFO:teuthology.orchestra.run.smithi158.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-05-16T07:38:43.449 INFO:teuthology.orchestra.run.smithi158.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-05-16T07:38:43.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.449 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-05-16T07:38:43.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.534 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-05-16T07:38:43.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.585 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link delete ceph-brx 2023-05-16T07:38:43.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.662 DEBUG:teuthology.orchestra.run.smithi158:> ip netns list 2023-05-16T07:38:43.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-16T07:38:43.678 DEBUG:teuthology.orchestra.run.smithi158:> sudo ip link delete ceph-brx 2023-05-16T07:38:43.745 INFO:teuthology.orchestra.run.smithi158.stderr:Cannot find device "ceph-brx" 2023-05-16T07:38:43.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:38:43.747 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-05-16T07:38:43.756 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-05-16T07:38:43.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.865+0000 7f8357fff700 1 Processor -- start 2023-05-16T07:38:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.865+0000 7f8357fff700 1 -- start start 2023-05-16T07:38:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f8357fff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f8358150bb0 0x7f8358068760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f8357fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 0x7f8358151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f8357fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8358064100 con 0x7f8358068fb0 2023-05-16T07:38:43.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f8357fff700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580bd1c0 con 0x7f8358150bb0 2023-05-16T07:38:43.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f8357fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83580c2700 con 0x7f8358150f90 2023-05-16T07:38:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f83577fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8358068fb0 0x7f8358153f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36210/0 (socket says 172.21.15.92:36210) 2023-05-16T07:38:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.866+0000 7f83577fe700 1 -- 172.21.15.92:0/209822841 learned_addr learned my addr 172.21.15.92:0/209822841 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140623475 0 0) 0x7f8358064100 con 0x7f8358068fb0 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f834ffff700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 0x7f8358151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83440033f0 con 0x7f8358068fb0 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 --2- 172.21.15.92:0/209822841 >> v2:172.21.15.158:3300/0 conn(0x7f8358150bb0 0x7f8358068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3340233877 0 0) 0x7f83440033f0 con 0x7f8358068fb0 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 msgr2=0x7f8358151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 0x7f8358151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 >> v2:172.21.15.158:3300/0 conn(0x7f8358150bb0 msgr2=0x7f8358068760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 --2- 172.21.15.92:0/209822841 >> v2:172.21.15.158:3300/0 conn(0x7f8358150bb0 0x7f8358068760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8358151900 con 0x7f8358068fb0 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f83500017f0 con 0x7f8358068fb0 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 >> v1:172.21.15.92:6789/0 conn(0x7f8358068fb0 legacy=0x7f8358153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f8344003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 0x7f8344004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8344004b80 0x7f8344005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580bd1c0 con 0x7f83440038e0 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580c2700 con 0x7f8344004b80 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834400b8e0 con 0x7f8358150f90 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f8344003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f834ffff700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 0x7f8344004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83577fe700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8344004b80 0x7f8344005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 -- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 msgr2=0x7f8344004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:43.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f8358150f90 0x7f8344004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 -- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8344004b80 msgr2=0x7f8344005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8344004b80 0x7f8344005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 -- 172.21.15.92:0/209822841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83440033f0 con 0x7f83440038e0 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f83577fe700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8344004b80 0x7f8344005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.867+0000 7f8356ffd700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f8344003d00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 rx=0x7f834800ac20 tx=0x7f834800a200).ready entity=mon.0 client_cookie=3e9d7859ae1b9b8b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800a5d0 con 0x7f83440038e0 2023-05-16T07:38:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800f820 con 0x7f83440038e0 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f83567fc700 1 -- 172.21.15.92:0/209822841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800f990 con 0x7f83440038e0 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f8357fff700 1 -- 172.21.15.92:0/209822841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 msgr2=0x7f8344003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f8357fff700 1 --2- 172.21.15.92:0/209822841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f8344003d00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 rx=0x7f834800ac20 tx=0x7f834800a200).stop 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f83577fe700 1 -- 172.21.15.92:0/209822841 reap_dead start 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f8357fff700 1 -- 172.21.15.92:0/209822841 shutdown_connections 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f8357fff700 1 -- 172.21.15.92:0/209822841 >> 172.21.15.92:0/209822841 conn(0x7f83580678d0 msgr2=0x7f83580c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f8357fff700 1 -- 172.21.15.92:0/209822841 shutdown_connections 2023-05-16T07:38:43.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.868+0000 7f8357fff700 1 -- 172.21.15.92:0/209822841 wait complete. 2023-05-16T07:38:43.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.869+0000 7f8357fff700 1 Processor -- start 2023-05-16T07:38:43.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.869+0000 7f8357fff700 1 -- start start 2023-05-16T07:38:43.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8357fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f83580aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8357fff700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83580a9a50 0x7f83581501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8357fff700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83580abc80 0x7f83580ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8357fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8358154960 con 0x7f83440038e0 2023-05-16T07:38:43.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8357fff700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8358154030 con 0x7f83580abc80 2023-05-16T07:38:43.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8357fff700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8358154310 con 0x7f83580a9a50 2023-05-16T07:38:43.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f83580aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f83580aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53704/0 (socket says 172.21.15.92:53704) 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 -- 172.21.15.92:0/1745422312 learned_addr learned my addr 172.21.15.92:0/1745422312 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f83577fe700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83580abc80 0x7f83580ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f834ffff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83580a9a50 0x7f83581501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 -- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83580a9a50 msgr2=0x7f83581501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83580a9a50 0x7f83581501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 -- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83580abc80 msgr2=0x7f83580ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83580abc80 0x7f83580ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.870+0000 7f8356ffd700 1 -- 172.21.15.92:0/1745422312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83581545f0 con 0x7f83440038e0 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.871+0000 7f8356ffd700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f83580aa000 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f8344004490 tx=0x7f834800a200).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.871+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348021070 con 0x7f83440038e0 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.871+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800fd00 con 0x7f83440038e0 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.871+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8358154df0 con 0x7f83440038e0 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.871+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480126b0 con 0x7f83440038e0 2023-05-16T07:38:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.871+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83580ac5f0 con 0x7f83440038e0 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.872+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8358154c40 con 0x7f83440038e0 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.872+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f834802d070 con 0x7f83440038e0 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.872+0000 7f834effd700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f833805ed90 0x7f8338061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.872+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f834807b160 con 0x7f83440038e0 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.872+0000 7f834ffff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f833805ed90 0x7f8338061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.873+0000 7f834ffff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f833805ed90 0x7f8338061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f83580a5750 tx=0x7f8340009060).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:43.876+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f834804c5e0 con 0x7f83440038e0 2023-05-16T07:38:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.043+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8358047150 con 0x7f83440038e0 2023-05-16T07:38:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.044+0000 7f834effd700 1 -- 172.21.15.92:0/1745422312 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+15477 (secure 0 0 0) 0x7f834804fc00 con 0x7f83440038e0 2023-05-16T07:38:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:44.048 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":176,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:41.995555+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:6837/862724662":"2023-05-17T07:38:38.053205+0000","172.21.15.158:6836/862724662":"2023-05-17T07:38:38.053205+0000","172.21.15.158:0/1117121967":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3339950":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3223099484":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/1153790295":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3551364644":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3460186346":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6833/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6832/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:0/240635450":"2023-05-17T07:38:30.703311+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f833805ed90 msgr2=0x7f8338061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f833805ed90 0x7f8338061250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f83580a5750 tx=0x7f8340009060).stop 2023-05-16T07:38:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 msgr2=0x7f83580aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f83580aa000 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f8344004490 tx=0x7f834800a200).stop 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 shutdown_connections 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f83580abc80 0x7f83580ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83440038e0 0x7f83580aa000 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f833805ed90 0x7f8338061250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 --2- 172.21.15.92:0/1745422312 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f83580a9a50 0x7f83581501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 >> 172.21.15.92:0/1745422312 conn(0x7f83580678d0 msgr2=0x7f83581521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 shutdown_connections 2023-05-16T07:38:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.046+0000 7f8357fff700 1 -- 172.21.15.92:0/1745422312 wait complete. 2023-05-16T07:38:44.061 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-05-16T07:38:44.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:38:44.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.170+0000 7f4d0eaf9700 1 Processor -- start 2023-05-16T07:38:44.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.170+0000 7f4d0eaf9700 1 -- start start 2023-05-16T07:38:44.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0eaf9700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f4d08154170 0x7f4d080c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0eaf9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d08154550 0x7f4d080c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0eaf9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d080c0660 con 0x7f4d080696a0 2023-05-16T07:38:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0eaf9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d080c1b00 con 0x7f4d08154550 2023-05-16T07:38:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0eaf9700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d080bfd30 con 0x7f4d08154170 2023-05-16T07:38:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d08154550 0x7f4d080c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d08154550 0x7f4d080c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53712/0 (socket says 172.21.15.92:53712) 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0d096700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4d080696a0 0x7f4d08157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36228/0 (socket says 172.21.15.92:36228) 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d07fff700 1 -- 172.21.15.92:0/983157295 learned_addr learned my addr 172.21.15.92:0/983157295 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d0c895700 1 --2- 172.21.15.92:0/983157295 >> v2:172.21.15.158:3300/0 conn(0x7f4d08154170 0x7f4d080c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d07fff700 1 -- 172.21.15.92:0/983157295 >> v2:172.21.15.158:3300/0 conn(0x7f4d08154170 msgr2=0x7f4d080c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d07fff700 1 --2- 172.21.15.92:0/983157295 >> v2:172.21.15.158:3300/0 conn(0x7f4d08154170 0x7f4d080c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d07fff700 1 -- 172.21.15.92:0/983157295 >> v1:172.21.15.92:6789/0 conn(0x7f4d080696a0 legacy=0x7f4d08157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d07fff700 1 -- 172.21.15.92:0/983157295 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d080c0010 con 0x7f4d08154550 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.171+0000 7f4d077fe700 1 -- 172.21.15.92:0/983157295 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337351281 0 0) 0x7f4d080c0660 con 0x7f4d080696a0 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0c895700 1 --2- 172.21.15.92:0/983157295 >> v2:172.21.15.158:3300/0 conn(0x7f4d08154170 0x7f4d080c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d07fff700 1 --2- 172.21.15.92:0/983157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154550 0x7f4d080c6160 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7f4d08083030 tx=0x7f4cfc0090f0).ready entity=mon.0 client_cookie=85eef03ffc06d65 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d077fe700 1 -- 172.21.15.92:0/983157295 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00b600 con 0x7f4d08154550 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d077fe700 1 -- 172.21.15.92:0/983157295 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00b760 con 0x7f4d08154550 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d077fe700 1 -- 172.21.15.92:0/983157295 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00bb00 con 0x7f4d08154550 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/983157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154550 msgr2=0x7f4d080c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/983157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154550 0x7f4d080c6160 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7f4d08083030 tx=0x7f4cfc0090f0).stop 2023-05-16T07:38:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/983157295 shutdown_connections 2023-05-16T07:38:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/983157295 >> v2:172.21.15.158:3300/0 conn(0x7f4d08154170 0x7f4d080c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/983157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154550 0x7f4d080c6160 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/983157295 >> 172.21.15.92:0/983157295 conn(0x7f4d080cb700 msgr2=0x7f4d080c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/983157295 shutdown_connections 2023-05-16T07:38:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.172+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/983157295 wait complete. 2023-05-16T07:38:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.173+0000 7f4d0eaf9700 1 Processor -- start 2023-05-16T07:38:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 -- start start 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d080696a0 0x7f4d0814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 0x7f4d081679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d08154550 0x7f4d08167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d080c4e00 con 0x7f4d08154170 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d080c4210 con 0x7f4d08154550 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d0eaf9700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d080c4460 con 0x7f4d080696a0 2023-05-16T07:38:44.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 0x7f4d081679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 0x7f4d081679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53722/0 (socket says 172.21.15.92:53722) 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.174+0000 7f4d07fff700 1 -- 172.21.15.92:0/2412140399 learned_addr learned my addr 172.21.15.92:0/2412140399 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d0d096700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d08154550 0x7f4d08167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d0c895700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d080696a0 0x7f4d0814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d07fff700 1 -- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d080696a0 msgr2=0x7f4d0814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d07fff700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d080696a0 0x7f4d0814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d07fff700 1 -- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d08154550 msgr2=0x7f4d08167ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d07fff700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d08154550 0x7f4d08167ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d07fff700 1 -- 172.21.15.92:0/2412140399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009370 con 0x7f4d08154170 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d0c895700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d080696a0 0x7f4d0814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d07fff700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 0x7f4d081679f0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f4cfc011000 tx=0x7f4cfc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc006970 con 0x7f4d08154170 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00b8d0 con 0x7f4d08154170 2023-05-16T07:38:44.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d080c4740 con 0x7f4d08154170 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc024470 con 0x7f4d08154170 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.175+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0814c600 con 0x7f4d08154170 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.176+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0814c850 con 0x7f4d08154170 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.176+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f4cfc00f070 con 0x7f4d08154170 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.177+0000 7f4d04ff9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4cf405ed90 0x7f4cf4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.177+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4cfc07b860 con 0x7f4d08154170 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.177+0000 7f4d0c895700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4cf405ed90 0x7f4cf4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.177+0000 7f4d0c895700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4cf405ed90 0x7f4cf4061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f4cf8008990 tx=0x7f4cf8008040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.181+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f4d0814c850 con 0x7f4d08154170 2023-05-16T07:38:44.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.339+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d0814c850 con 0x7f4cf405ed90 2023-05-16T07:38:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.342+0000 7f4d04ff9700 1 -- 172.21.15.92:0/2412140399 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70872 (secure 0 0 0) 0x7f4d0814c850 con 0x7f4cf405ed90 2023-05-16T07:38:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:44.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4cf405ed90 msgr2=0x7f4cf4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4cf405ed90 0x7f4cf4061250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f4cf8008990 tx=0x7f4cf8008040).stop 2023-05-16T07:38:44.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 msgr2=0x7f4d081679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 0x7f4d081679f0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f4cfc011000 tx=0x7f4cfc017000).stop 2023-05-16T07:38:44.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 shutdown_connections 2023-05-16T07:38:44.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f4cf405ed90 0x7f4cf4061250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f4d080696a0 0x7f4d0814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4d08154550 0x7f4d08167ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 --2- 172.21.15.92:0/2412140399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d08154170 0x7f4d081679f0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 >> 172.21.15.92:0/2412140399 conn(0x7f4d080cb700 msgr2=0x7f4d080ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 shutdown_connections 2023-05-16T07:38:44.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.344+0000 7f4d0eaf9700 1 -- 172.21.15.92:0/2412140399 wait complete. 2023-05-16T07:38:44.354 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:38:44.359 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2023-05-16T07:38:42.804462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":195296,"kb_used_data":17056,"kb_used_omap":0,"kb_used_meta":178176,"kb_avail":754779424,"statfs":{"total":773094113280,"available":772894130176,"internally_reserved":0,"allocated":17465344,"data_stored":9356428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":182452224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12242,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.170733"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.798611+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:38:30.798611+0000","last_peered":"2023-05-16T07:38:30.798611+0000","last_clean":"2023-05-16T07:38:30.798611+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:30.798611+0000","last_undegraded":"2023-05-16T07:38:30.798611+0000","last_fullsized":"2023-05-16T07:38:30.798611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.977198+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:38:37.977198+0000","last_peered":"2023-05-16T07:38:37.977198+0000","last_clean":"2023-05-16T07:38:37.977198+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:37.977198+0000","last_undegraded":"2023-05-16T07:38:37.977198+0000","last_fullsized":"2023-05-16T07:38:37.977198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.795679+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:38:30.795679+0000","last_peered":"2023-05-16T07:38:30.795679+0000","last_clean":"2023-05-16T07:38:30.795679+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:30.795679+0000","last_undegraded":"2023-05-16T07:38:30.795679+0000","last_fullsized":"2023-05-16T07:38:30.795679+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.977463+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:38:37.977463+0000","last_peered":"2023-05-16T07:38:37.977463+0000","last_clean":"2023-05-16T07:38:37.977463+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:37.977463+0000","last_undegraded":"2023-05-16T07:38:37.977463+0000","last_fullsized":"2023-05-16T07:38:37.977463+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":139,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.974775+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:38:37.974775+0000","last_peered":"2023-05-16T07:38:37.974775+0000","last_clean":"2023-05-16T07:38:37.974775+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:37.974775+0000","last_undegraded":"2023-05-16T07:38:37.974775+0000","last_fullsized":"2023-05-16T07:38:37.974775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.974856+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:38:37.974856+0000","last_peered":"2023-05-16T07:38:37.974856+0000","last_clean":"2023-05-16T07:38:37.974856+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:37.974856+0000","last_undegraded":"2023-05-16T07:38:37.974856+0000","last_fullsized":"2023-05-16T07:38:37.974856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.975157+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:38:37.975157+0000","last_peered":"2023-05-16T07:38:37.975157+0000","last_clean":"2023-05-16T07:38:37.975157+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:37.975157+0000","last_undegraded":"2023-05-16T07:38:37.975157+0000","last_fullsized":"2023-05-16T07:38:37.975157+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.798768+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:38:30.798768+0000","last_peered":"2023-05-16T07:38:30.798768+0000","last_clean":"2023-05-16T07:38:30.798768+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:30.798768+0000","last_undegraded":"2023-05-16T07:38:30.798768+0000","last_fullsized":"2023-05-16T07:38:30.798768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":165,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.972469+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:38:37.972469+0000","last_peered":"2023-05-16T07:38:37.972469+0000","last_clean":"2023-05-16T07:38:37.972469+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:37.972469+0000","last_undegraded":"2023-05-16T07:38:37.972469+0000","last_fullsized":"2023-05-16T07:38:37.972469+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575276,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25320,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":23168,"kb_avail":94346520,"statfs":{"total":96636764160,"available":96610836480,"internally_reserved":0,"allocated":2195456,"data_stored":1177796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23724032},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.82399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.755,"15min":0.79800000000000004},"last":0.44500000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.495}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73399999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52100000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.71499999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.315},"max":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.75},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.82699999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575276,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25108,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":22976,"kb_avail":94346732,"statfs":{"total":96636764160,"available":96611053568,"internally_reserved":0,"allocated":2174976,"data_stored":1166594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23527424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64900000000000002,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.622},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.77500000000000002,"5min":1.5,"15min":1.5},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73599999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":1.486,"15min":1.486},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":1.532,"15min":1.532},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.377}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38500000000000001}]},{"osd":7,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.878,"15min":0.878},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.316},"max":{"1min":0.873,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575277,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20632,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":18496,"kb_avail":94351208,"statfs":{"total":96636764160,"available":96615636992,"internally_reserved":0,"allocated":2179072,"data_stored":1167117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18939904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58899999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.64100000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41899999999999998},"min":{"1min":0.26100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.309}]},{"osd":3,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.67900000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.872,"15min":0.877},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.054,"15min":1.054},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.998},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575277,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20324,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":18176,"kb_avail":94351516,"statfs":{"total":96636764160,"available":96615952384,"internally_reserved":0,"allocated":2191360,"data_stored":1167984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18612224},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.223,"15min":0.223},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.435},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.433,"15min":0.432},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.98699999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.90400000000000003,"15min":1.0549999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":1.0680000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.68000000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.095},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":1.0369999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90300000000000002,"5min":0.91500000000000004,"15min":1.03},"last":0.626},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90400000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575277,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22044,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94349796,"statfs":{"total":96636764160,"available":96614191104,"internally_reserved":0,"allocated":2183168,"data_stored":1166675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.378}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.48099999999999998}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.50800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575276,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":26688,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":2179072,"data_stored":1166628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27328512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.58999999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.315},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.72999999999999998},"last":0.26300000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.94299999999999995,"15min":1.014},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":0.84099999999999997,"15min":1.024},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.97799999999999998,"15min":1.05},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.97299999999999998},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.91600000000000004,"15min":1.0389999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.505}]},{"osd":7,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":1.0640000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":1.012},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575276,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24984,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":22848,"kb_avail":94346856,"statfs":{"total":96636764160,"available":96611180544,"internally_reserved":0,"allocated":2179072,"data_stored":1167137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23396352},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.71799999999999997,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.753,"5min":1.359,"15min":1.359},"last":0.64500000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":2,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.81399999999999995},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.755,"15min":0.755},"last":0.623},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575276,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":25920,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":2183168,"data_stored":1176497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26542080},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67800000000000005}]},{"osd":4,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.442},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:38:44.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:38:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.480+0000 7f455109b700 1 Processor -- start 2023-05-16T07:38:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.480+0000 7f455109b700 1 -- start start 2023-05-16T07:38:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f455109b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c154170 0x7f454c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f455109b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f454c154550 0x7f454c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f455109b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f454c0696a0 0x7f454c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f455109b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0c0660 con 0x7f454c0696a0 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f455109b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c06a020 con 0x7f454c154550 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f455109b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0bfd30 con 0x7f454c154170 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f454c0696a0 0x7f454c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f454c0696a0 0x7f454c069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53730/0 (socket says 172.21.15.92:53730) 2023-05-16T07:38:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 -- 172.21.15.92:0/1226665707 learned_addr learned my addr 172.21.15.92:0/1226665707 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454a59c700 1 --2- 172.21.15.92:0/1226665707 >> v2:172.21.15.158:3300/0 conn(0x7f454c154550 0x7f454c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454ad9d700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c154170 0x7f454c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 -- 172.21.15.92:0/1226665707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c154170 msgr2=0x7f454c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c154170 0x7f454c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 -- 172.21.15.92:0/1226665707 >> v2:172.21.15.158:3300/0 conn(0x7f454c154550 msgr2=0x7f454c0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 --2- 172.21.15.92:0/1226665707 >> v2:172.21.15.158:3300/0 conn(0x7f454c154550 0x7f454c0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 -- 172.21.15.92:0/1226665707 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0c0010 con 0x7f454c0696a0 2023-05-16T07:38:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454ad9d700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c154170 0x7f454c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.481+0000 7f454b59e700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c0696a0 0x7f454c069b30 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f454c083fe0 tx=0x7f45440090f0).ready entity=mon.0 client_cookie=e9f55781acb6520d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f4549d9b700 1 -- 172.21.15.92:0/1226665707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400b600 con 0x7f454c0696a0 2023-05-16T07:38:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f4549d9b700 1 -- 172.21.15.92:0/1226665707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400b760 con 0x7f454c0696a0 2023-05-16T07:38:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f4549d9b700 1 -- 172.21.15.92:0/1226665707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400bb00 con 0x7f454c0696a0 2023-05-16T07:38:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 -- 172.21.15.92:0/1226665707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c0696a0 msgr2=0x7f454c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c0696a0 0x7f454c069b30 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f454c083fe0 tx=0x7f45440090f0).stop 2023-05-16T07:38:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 -- 172.21.15.92:0/1226665707 shutdown_connections 2023-05-16T07:38:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c0696a0 0x7f454c069b30 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 --2- 172.21.15.92:0/1226665707 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c154170 0x7f454c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 --2- 172.21.15.92:0/1226665707 >> v2:172.21.15.158:3300/0 conn(0x7f454c154550 0x7f454c0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 -- 172.21.15.92:0/1226665707 >> 172.21.15.92:0/1226665707 conn(0x7f454c0cb700 msgr2=0x7f454c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 -- 172.21.15.92:0/1226665707 shutdown_connections 2023-05-16T07:38:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.482+0000 7f455109b700 1 -- 172.21.15.92:0/1226665707 wait complete. 2023-05-16T07:38:44.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.483+0000 7f455109b700 1 Processor -- start 2023-05-16T07:38:44.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.483+0000 7f455109b700 1 -- start start 2023-05-16T07:38:44.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f455109b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c0696a0 0x7f454c145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f455109b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f454c154170 0x7f454c0ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f455109b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 0x7f454c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f455109b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b0c80 con 0x7f454c154550 2023-05-16T07:38:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f455109b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c145ca0 con 0x7f454c154170 2023-05-16T07:38:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f455109b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b0460 con 0x7f454c0696a0 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f454b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 0x7f454c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f454b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 0x7f454c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53736/0 (socket says 172.21.15.92:53736) 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f454b59e700 1 -- 172.21.15.92:0/3195228228 learned_addr learned my addr 172.21.15.92:0/3195228228 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f454ad9d700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c0696a0 0x7f454c145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.484+0000 7f454a59c700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f454c154170 0x7f454c0ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454b59e700 1 -- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c0696a0 msgr2=0x7f454c145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454b59e700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c0696a0 0x7f454c145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454b59e700 1 -- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f454c154170 msgr2=0x7f454c0ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454b59e700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f454c154170 0x7f454c0ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454b59e700 1 -- 172.21.15.92:0/3195228228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544009370 con 0x7f454c154550 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454a59c700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f454c154170 0x7f454c0ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f454b59e700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 0x7f454c0aece0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f4544006fd0 tx=0x7f4544017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400b600 con 0x7f454c154550 2023-05-16T07:38:44.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400bec0 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c0b0700 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440246a0 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.485+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c145930 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.486+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c0666c0 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.486+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f454400f070 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.486+0000 7f4542ffd700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f452c05ed90 0x7f452c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.487+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f454407b950 con 0x7f454c154550 2023-05-16T07:38:44.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.487+0000 7f454ad9d700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f452c05ed90 0x7f452c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.487+0000 7f454ad9d700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f452c05ed90 0x7f452c061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f453c00d6e0 tx=0x7f453c007850).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.490+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f454404cdd0 con 0x7f454c154550 2023-05-16T07:38:44.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.650+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f454c145e10 con 0x7f452c05ed90 2023-05-16T07:38:44.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.653+0000 7f4542ffd700 1 -- 172.21.15.92:0/3195228228 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70872 (secure 0 0 0) 0x7f454c145e10 con 0x7f452c05ed90 2023-05-16T07:38:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:44.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f452c05ed90 msgr2=0x7f452c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f452c05ed90 0x7f452c061250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f453c00d6e0 tx=0x7f453c007850).stop 2023-05-16T07:38:44.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 msgr2=0x7f454c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 0x7f454c0aece0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f4544006fd0 tx=0x7f4544017000).stop 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 shutdown_connections 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f452c05ed90 0x7f452c061250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f454c0696a0 0x7f454c145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c154550 0x7f454c0aece0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 --2- 172.21.15.92:0/3195228228 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f454c154170 0x7f454c0ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 >> 172.21.15.92:0/3195228228 conn(0x7f454c0cb700 msgr2=0x7f454c0aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 shutdown_connections 2023-05-16T07:38:44.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.655+0000 7f455109b700 1 -- 172.21.15.92:0/3195228228 wait complete. 2023-05-16T07:38:44.660 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:38:44.669 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2023-05-16T07:38:42.804462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":195296,"kb_used_data":17056,"kb_used_omap":0,"kb_used_meta":178176,"kb_avail":754779424,"statfs":{"total":773094113280,"available":772894130176,"internally_reserved":0,"allocated":17465344,"data_stored":9356428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":182452224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12242,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.170733"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.798611+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:38:30.798611+0000","last_peered":"2023-05-16T07:38:30.798611+0000","last_clean":"2023-05-16T07:38:30.798611+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:30.798611+0000","last_undegraded":"2023-05-16T07:38:30.798611+0000","last_fullsized":"2023-05-16T07:38:30.798611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.977198+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:38:37.977198+0000","last_peered":"2023-05-16T07:38:37.977198+0000","last_clean":"2023-05-16T07:38:37.977198+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:37.977198+0000","last_undegraded":"2023-05-16T07:38:37.977198+0000","last_fullsized":"2023-05-16T07:38:37.977198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.795679+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:38:30.795679+0000","last_peered":"2023-05-16T07:38:30.795679+0000","last_clean":"2023-05-16T07:38:30.795679+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:30.795679+0000","last_undegraded":"2023-05-16T07:38:30.795679+0000","last_fullsized":"2023-05-16T07:38:30.795679+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.977463+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:38:37.977463+0000","last_peered":"2023-05-16T07:38:37.977463+0000","last_clean":"2023-05-16T07:38:37.977463+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:37.977463+0000","last_undegraded":"2023-05-16T07:38:37.977463+0000","last_fullsized":"2023-05-16T07:38:37.977463+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":139,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.974775+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:38:37.974775+0000","last_peered":"2023-05-16T07:38:37.974775+0000","last_clean":"2023-05-16T07:38:37.974775+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:37.974775+0000","last_undegraded":"2023-05-16T07:38:37.974775+0000","last_fullsized":"2023-05-16T07:38:37.974775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.974856+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:38:37.974856+0000","last_peered":"2023-05-16T07:38:37.974856+0000","last_clean":"2023-05-16T07:38:37.974856+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:37.974856+0000","last_undegraded":"2023-05-16T07:38:37.974856+0000","last_fullsized":"2023-05-16T07:38:37.974856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.975157+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:38:37.975157+0000","last_peered":"2023-05-16T07:38:37.975157+0000","last_clean":"2023-05-16T07:38:37.975157+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:37.975157+0000","last_undegraded":"2023-05-16T07:38:37.975157+0000","last_fullsized":"2023-05-16T07:38:37.975157+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.798768+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:38:30.798768+0000","last_peered":"2023-05-16T07:38:30.798768+0000","last_clean":"2023-05-16T07:38:30.798768+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:30.798768+0000","last_undegraded":"2023-05-16T07:38:30.798768+0000","last_fullsized":"2023-05-16T07:38:30.798768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":165,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.972469+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:38:37.972469+0000","last_peered":"2023-05-16T07:38:37.972469+0000","last_clean":"2023-05-16T07:38:37.972469+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:37.972469+0000","last_undegraded":"2023-05-16T07:38:37.972469+0000","last_fullsized":"2023-05-16T07:38:37.972469+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575276,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25320,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":23168,"kb_avail":94346520,"statfs":{"total":96636764160,"available":96610836480,"internally_reserved":0,"allocated":2195456,"data_stored":1177796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23724032},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.82399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.755,"15min":0.79800000000000004},"last":0.44500000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.495}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73399999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52100000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.71499999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.315},"max":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.75},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.82699999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575276,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25108,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":22976,"kb_avail":94346732,"statfs":{"total":96636764160,"available":96611053568,"internally_reserved":0,"allocated":2174976,"data_stored":1166594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23527424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64900000000000002,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.622},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.77500000000000002,"5min":1.5,"15min":1.5},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73599999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":1.486,"15min":1.486},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":1.532,"15min":1.532},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.377}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38500000000000001}]},{"osd":7,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.878,"15min":0.878},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.316},"max":{"1min":0.873,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575277,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20632,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":18496,"kb_avail":94351208,"statfs":{"total":96636764160,"available":96615636992,"internally_reserved":0,"allocated":2179072,"data_stored":1167117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18939904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58899999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.64100000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41899999999999998},"min":{"1min":0.26100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.309}]},{"osd":3,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.67900000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.872,"15min":0.877},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.054,"15min":1.054},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.998},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575277,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20324,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":18176,"kb_avail":94351516,"statfs":{"total":96636764160,"available":96615952384,"internally_reserved":0,"allocated":2191360,"data_stored":1167984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18612224},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.223,"15min":0.223},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.435},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.433,"15min":0.432},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.98699999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.90400000000000003,"15min":1.0549999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":1.0680000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.68000000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.095},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":1.0369999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90300000000000002,"5min":0.91500000000000004,"15min":1.03},"last":0.626},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90400000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575277,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22044,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94349796,"statfs":{"total":96636764160,"available":96614191104,"internally_reserved":0,"allocated":2183168,"data_stored":1166675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.378}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.48099999999999998}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.50800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575276,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":26688,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":2179072,"data_stored":1166628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27328512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.58999999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.315},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.72999999999999998},"last":0.26300000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.94299999999999995,"15min":1.014},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":0.84099999999999997,"15min":1.024},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.97799999999999998,"15min":1.05},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.97299999999999998},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.91600000000000004,"15min":1.0389999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.505}]},{"osd":7,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":1.0640000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":1.012},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575276,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24984,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":22848,"kb_avail":94346856,"statfs":{"total":96636764160,"available":96611180544,"internally_reserved":0,"allocated":2179072,"data_stored":1167137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23396352},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.71799999999999997,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.753,"5min":1.359,"15min":1.359},"last":0.64500000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":2,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.81399999999999995},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.755,"15min":0.755},"last":0.623},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575276,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":25920,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":2183168,"data_stored":1176497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26542080},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67800000000000005}]},{"osd":4,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.442},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:38:44.672 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-05-16T07:38:44.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:38:44.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.787+0000 7f9426c5b700 1 Processor -- start 2023-05-16T07:38:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.787+0000 7f9426c5b700 1 -- start start 2023-05-16T07:38:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9426c5b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f94200c7380 0x7f9420154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9426c5b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94200c7760 0x7f94200acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9426c5b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420158970 con 0x7f94200ad040 2023-05-16T07:38:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9426c5b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9420155340 con 0x7f94200c7760 2023-05-16T07:38:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9426c5b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9420157d60 con 0x7f94200c7380 2023-05-16T07:38:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425458700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94200c7760 0x7f94200acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f942645a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f94200ad040 0x7f942014dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36260/0 (socket says 172.21.15.92:36260) 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f942645a700 1 -- 172.21.15.92:0/1327898324 learned_addr learned my addr 172.21.15.92:0/1327898324 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425c59700 1 --2- 172.21.15.92:0/1327898324 >> v2:172.21.15.158:3300/0 conn(0x7f94200c7380 0x7f9420154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425458700 1 -- 172.21.15.92:0/1327898324 >> v2:172.21.15.158:3300/0 conn(0x7f94200c7380 msgr2=0x7f9420154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425458700 1 --2- 172.21.15.92:0/1327898324 >> v2:172.21.15.158:3300/0 conn(0x7f94200c7380 0x7f9420154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425458700 1 -- 172.21.15.92:0/1327898324 >> v1:172.21.15.92:6789/0 conn(0x7f94200ad040 legacy=0x7f942014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9424c57700 1 -- 172.21.15.92:0/1327898324 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878607637 0 0) 0x7f9420158970 con 0x7f94200ad040 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425458700 1 -- 172.21.15.92:0/1327898324 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9420158040 con 0x7f94200c7760 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425c59700 1 --2- 172.21.15.92:0/1327898324 >> v2:172.21.15.158:3300/0 conn(0x7f94200c7380 0x7f9420154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9425458700 1 --2- 172.21.15.92:0/1327898324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7760 0x7f94200acb50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7f9420083ba0 tx=0x7f9410009060).ready entity=mon.0 client_cookie=388ef476806c0b11 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9424c57700 1 -- 172.21.15.92:0/1327898324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000b580 con 0x7f94200c7760 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9424c57700 1 -- 172.21.15.92:0/1327898324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941000b6e0 con 0x7f94200c7760 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.788+0000 7f9424c57700 1 -- 172.21.15.92:0/1327898324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000ba60 con 0x7f94200c7760 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 -- 172.21.15.92:0/1327898324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7760 msgr2=0x7f94200acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 --2- 172.21.15.92:0/1327898324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7760 0x7f94200acb50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7f9420083ba0 tx=0x7f9410009060).stop 2023-05-16T07:38:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 -- 172.21.15.92:0/1327898324 shutdown_connections 2023-05-16T07:38:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 --2- 172.21.15.92:0/1327898324 >> v2:172.21.15.158:3300/0 conn(0x7f94200c7380 0x7f9420154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 --2- 172.21.15.92:0/1327898324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7760 0x7f94200acb50 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 -- 172.21.15.92:0/1327898324 >> 172.21.15.92:0/1327898324 conn(0x7f9420068330 msgr2=0x7f9420068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 -- 172.21.15.92:0/1327898324 shutdown_connections 2023-05-16T07:38:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.789+0000 7f9426c5b700 1 -- 172.21.15.92:0/1327898324 wait complete. 2023-05-16T07:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.790+0000 7f9426c5b700 1 Processor -- start 2023-05-16T07:38:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.790+0000 7f9426c5b700 1 -- start start 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94200ad040 0x7f94200c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 0x7f9420143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94200c7760 0x7f942014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200c2c50 con 0x7f94200c7380 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200c8dd0 con 0x7f94200ad040 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94200c2040 con 0x7f94200c7760 2023-05-16T07:38:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 0x7f9420143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 0x7f9420143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53746/0 (socket says 172.21.15.92:53746) 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 -- 172.21.15.92:0/890416935 learned_addr learned my addr 172.21.15.92:0/890416935 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 -- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94200c7760 msgr2=0x7f942014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94200c7760 0x7f942014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 -- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94200ad040 msgr2=0x7f94200c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94200ad040 0x7f94200c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 -- 172.21.15.92:0/890416935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100092a0 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9425458700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 0x7f9420143700 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7f9420083bd0 tx=0x7f9410017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000ba60 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410000c90 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94200c2320 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.791+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941001b640 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.792+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94200c28e0 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.793+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420047150 con 0x7f94200c7380 2023-05-16T07:38:44.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.793+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f941000f070 con 0x7f94200c7380 2023-05-16T07:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.793+0000 7f9415ffb700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f940405ed90 0x7f9404061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.793+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f941007b2b0 con 0x7f94200c7380 2023-05-16T07:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.793+0000 7f9425c59700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f940405ed90 0x7f9404061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.794+0000 7f9425c59700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f940405ed90 0x7f9404061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f941c006fd0 tx=0x7f941c008040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:44.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.797+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f941004c730 con 0x7f94200c7380 2023-05-16T07:38:44.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.956+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94200bfb90 con 0x7f940405ed90 2023-05-16T07:38:44.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.959+0000 7f9415ffb700 1 -- 172.21.15.92:0/890416935 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70879 (secure 0 0 0) 0x7f94200bfb90 con 0x7f940405ed90 2023-05-16T07:38:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:44.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f940405ed90 msgr2=0x7f9404061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f940405ed90 0x7f9404061250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f941c006fd0 tx=0x7f941c008040).stop 2023-05-16T07:38:44.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 msgr2=0x7f9420143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:44.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 0x7f9420143700 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7f9420083bd0 tx=0x7f9410017000).stop 2023-05-16T07:38:44.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 shutdown_connections 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f940405ed90 0x7f9404061250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f94200c7760 0x7f942014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94200ad040 0x7f94200c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 --2- 172.21.15.92:0/890416935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94200c7380 0x7f9420143700 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 >> 172.21.15.92:0/890416935 conn(0x7f9420068330 msgr2=0x7f94201537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 shutdown_connections 2023-05-16T07:38:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:44.961+0000 7f9426c5b700 1 -- 172.21.15.92:0/890416935 wait complete. 2023-05-16T07:38:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:38:44.977 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-05-16T07:38:44.804976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":237,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":195520,"kb_used_data":17088,"kb_used_omap":0,"kb_used_meta":178368,"kb_avail":754779200,"statfs":{"total":773094113280,"available":772893900800,"internally_reserved":0,"allocated":17498112,"data_stored":9384493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":182648832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12242,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.170733"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.798611+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:38:30.798611+0000","last_peered":"2023-05-16T07:38:30.798611+0000","last_clean":"2023-05-16T07:38:30.798611+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:30.798611+0000","last_undegraded":"2023-05-16T07:38:30.798611+0000","last_fullsized":"2023-05-16T07:38:30.798611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.977198+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:38:37.977198+0000","last_peered":"2023-05-16T07:38:37.977198+0000","last_clean":"2023-05-16T07:38:37.977198+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:37.977198+0000","last_undegraded":"2023-05-16T07:38:37.977198+0000","last_fullsized":"2023-05-16T07:38:37.977198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.214226+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:38:41.214226+0000","last_peered":"2023-05-16T07:38:41.214226+0000","last_clean":"2023-05-16T07:38:41.214226+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:41.214226+0000","last_undegraded":"2023-05-16T07:38:41.214226+0000","last_fullsized":"2023-05-16T07:38:41.214226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.977463+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:38:37.977463+0000","last_peered":"2023-05-16T07:38:37.977463+0000","last_clean":"2023-05-16T07:38:37.977463+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:37.977463+0000","last_undegraded":"2023-05-16T07:38:37.977463+0000","last_fullsized":"2023-05-16T07:38:37.977463+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":139,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.974775+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:38:37.974775+0000","last_peered":"2023-05-16T07:38:37.974775+0000","last_clean":"2023-05-16T07:38:37.974775+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:37.974775+0000","last_undegraded":"2023-05-16T07:38:37.974775+0000","last_fullsized":"2023-05-16T07:38:37.974775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.974856+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:38:37.974856+0000","last_peered":"2023-05-16T07:38:37.974856+0000","last_clean":"2023-05-16T07:38:37.974856+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:37.974856+0000","last_undegraded":"2023-05-16T07:38:37.974856+0000","last_fullsized":"2023-05-16T07:38:37.974856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.975157+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:38:37.975157+0000","last_peered":"2023-05-16T07:38:37.975157+0000","last_clean":"2023-05-16T07:38:37.975157+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:37.975157+0000","last_undegraded":"2023-05-16T07:38:37.975157+0000","last_fullsized":"2023-05-16T07:38:37.975157+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-05-16T07:38:30.798768+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:38:30.798768+0000","last_peered":"2023-05-16T07:38:30.798768+0000","last_clean":"2023-05-16T07:38:30.798768+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:30.798768+0000","last_undegraded":"2023-05-16T07:38:30.798768+0000","last_fullsized":"2023-05-16T07:38:30.798768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":165,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-05-16T07:38:37.972469+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:38:37.972469+0000","last_peered":"2023-05-16T07:38:37.972469+0000","last_clean":"2023-05-16T07:38:37.972469+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:37.972469+0000","last_undegraded":"2023-05-16T07:38:37.972469+0000","last_fullsized":"2023-05-16T07:38:37.972469+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575276,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25320,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":23168,"kb_avail":94346520,"statfs":{"total":96636764160,"available":96610836480,"internally_reserved":0,"allocated":2195456,"data_stored":1177796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23724032},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.82399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.755,"15min":0.79800000000000004},"last":0.44500000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.495}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73399999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52100000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.71499999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.315},"max":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.75},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.82699999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575276,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25108,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":22976,"kb_avail":94346732,"statfs":{"total":96636764160,"available":96611053568,"internally_reserved":0,"allocated":2174976,"data_stored":1166594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23527424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64900000000000002,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.622},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.77500000000000002,"5min":1.5,"15min":1.5},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73599999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":1.486,"15min":1.486},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":1.532,"15min":1.532},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.377}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38500000000000001}]},{"osd":7,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.878,"15min":0.878},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.316},"max":{"1min":0.873,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575277,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20632,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":18496,"kb_avail":94351208,"statfs":{"total":96636764160,"available":96615636992,"internally_reserved":0,"allocated":2179072,"data_stored":1167117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18939904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58899999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.64100000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41899999999999998},"min":{"1min":0.26100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.309}]},{"osd":3,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.67900000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.872,"15min":0.877},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.054,"15min":1.054},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.998},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575278,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20464,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":18304,"kb_avail":94351376,"statfs":{"total":96636764160,"available":96615809024,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18743296},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.313},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.223,"15min":0.223},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.30399999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.433,"15min":0.432},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.246}]},{"osd":4,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.98699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.90400000000000003,"15min":1.0549999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":1.0680000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.68000000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.095},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":1.0369999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90300000000000002,"5min":0.91500000000000004,"15min":1.03},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90400000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.71799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575278,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22128,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94349712,"statfs":{"total":96636764160,"available":96614105088,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.314},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.378}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.51500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575276,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":26688,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":2179072,"data_stored":1166628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27328512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.58999999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.315},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.72999999999999998},"last":0.26300000000000001}]},{"osd":2,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.94299999999999995,"15min":1.014},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":0.84099999999999997,"15min":1.024},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.97799999999999998,"15min":1.05},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.97299999999999998},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.91600000000000004,"15min":1.0389999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.505}]},{"osd":7,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":1.0640000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":1.012},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575276,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24984,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":22848,"kb_avail":94346856,"statfs":{"total":96636764160,"available":96611180544,"internally_reserved":0,"allocated":2179072,"data_stored":1167137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23396352},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.71799999999999997,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.753,"5min":1.359,"15min":1.359},"last":0.64500000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":2,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.81399999999999995},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.755,"15min":0.755},"last":0.623},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575276,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":25920,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":2183168,"data_stored":1176497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26542080},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67800000000000005}]},{"osd":4,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.442},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:38:44.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-16T07:38:45.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.090+0000 7f39dadda700 1 Processor -- start 2023-05-16T07:38:45.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.090+0000 7f39dadda700 1 -- start start 2023-05-16T07:38:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.091+0000 7f39dadda700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f39d41555c0 0x7f39d4068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.091+0000 7f39dadda700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d4155a40 con 0x7f39d4068650 2023-05-16T07:38:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.091+0000 7f39dadda700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d4155130 con 0x7f39d4151ff0 2023-05-16T07:38:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.091+0000 7f39dadda700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d40c55d0 con 0x7f39d41555c0 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.091+0000 7f39d9377700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f39d4068650 0x7f39d40bfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36272/0 (socket says 172.21.15.92:36272) 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.091+0000 7f39d9377700 1 -- 172.21.15.92:0/2843271042 learned_addr learned my addr 172.21.15.92:0/2843271042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685011555 0 0) 0x7f39cc001110 con 0x7f39d4068650 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39bc0033f0 con 0x7f39d4068650 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cbfff700 1 --2- 172.21.15.92:0/2843271042 >> v2:172.21.15.158:3300/0 conn(0x7f39d41555c0 0x7f39d4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496294727 0 0) 0x7f39bc0033f0 con 0x7f39d4068650 2023-05-16T07:38:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 >> v2:172.21.15.158:3300/0 conn(0x7f39d41555c0 msgr2=0x7f39d4068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 --2- 172.21.15.92:0/2843271042 >> v2:172.21.15.158:3300/0 conn(0x7f39d41555c0 0x7f39d4068160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 >> v1:172.21.15.158:6789/0 conn(0x7f39d4151ff0 legacy=0x7f39d40a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40aa5f0 con 0x7f39d4068650 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f39cc001600 con 0x7f39d4068650 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 >> v1:172.21.15.92:6789/0 conn(0x7f39d4068650 legacy=0x7f39d40bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39bc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39bc004690 0x7f39bc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39bc005050 0x7f39bc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c55d0 con 0x7f39bc0038e0 2023-05-16T07:38:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc00bf30 con 0x7f39bc005050 2023-05-16T07:38:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc00c0f0 con 0x7f39bc004690 2023-05-16T07:38:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39bc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d9377700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39bc005050 0x7f39bc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39cbfff700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39bc004690 0x7f39bc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 -- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39bc004690 msgr2=0x7f39bc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39bc004690 0x7f39bc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 -- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39bc005050 msgr2=0x7f39bc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39bc005050 0x7f39bc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 -- 172.21.15.92:0/2843271042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0033f0 con 0x7f39bc0038e0 2023-05-16T07:38:45.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d9377700 1 -- 172.21.15.92:0/2843271042 reap_dead start 2023-05-16T07:38:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.092+0000 7f39d8b76700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39bc003d00 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7f39d0001e30 tx=0x7f39d0007f30).ready entity=mon.0 client_cookie=89ae31d5efdd42a1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0006a20 con 0x7f39bc0038e0 2023-05-16T07:38:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0006b80 con 0x7f39bc0038e0 2023-05-16T07:38:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39cb7fe700 1 -- 172.21.15.92:0/2843271042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000c5d0 con 0x7f39bc0038e0 2023-05-16T07:38:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 -- 172.21.15.92:0/2843271042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 msgr2=0x7f39bc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39bc003d00 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7f39d0001e30 tx=0x7f39d0007f30).stop 2023-05-16T07:38:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 -- 172.21.15.92:0/2843271042 shutdown_connections 2023-05-16T07:38:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 --2- 172.21.15.92:0/2843271042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39bc003d00 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 -- 172.21.15.92:0/2843271042 >> 172.21.15.92:0/2843271042 conn(0x7f39d4067920 msgr2=0x7f39d40cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 -- 172.21.15.92:0/2843271042 shutdown_connections 2023-05-16T07:38:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.093+0000 7f39dadda700 1 -- 172.21.15.92:0/2843271042 wait complete. 2023-05-16T07:38:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.094+0000 7f39dadda700 1 Processor -- start 2023-05-16T07:38:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.094+0000 7f39dadda700 1 -- start start 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39dadda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39d413da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39dadda700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39d40cade0 0x7f39d40c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39dadda700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39d40c3db0 0x7f39d40c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39dadda700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d413def0 con 0x7f39bc0038e0 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39dadda700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c4210 con 0x7f39d40cade0 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39dadda700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c1ee0 con 0x7f39d40c3db0 2023-05-16T07:38:45.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39d413da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39d413da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53766/0 (socket says 172.21.15.92:53766) 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 -- 172.21.15.92:0/1050077835 learned_addr learned my addr 172.21.15.92:0/1050077835 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 -- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39d40c3db0 msgr2=0x7f39d40c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39cbfff700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39d40cade0 0x7f39d40c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39d40c3db0 0x7f39d40c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 -- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39d40cade0 msgr2=0x7f39d40c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39d40cade0 0x7f39d40c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 -- 172.21.15.92:0/1050077835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40c21c0 con 0x7f39bc0038e0 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.095+0000 7f39d8b76700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39d413da00 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f39d0001600 tx=0x7f39d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.096+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0012610 con 0x7f39bc0038e0 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.096+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0012790 con 0x7f39bc0038e0 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.096+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d40c24a0 con 0x7f39bc0038e0 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.096+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0012b50 con 0x7f39bc0038e0 2023-05-16T07:38:45.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.096+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40c2a60 con 0x7f39bc0038e0 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.097+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d40666c0 con 0x7f39bc0038e0 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.097+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f39d0025030 con 0x7f39bc0038e0 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.097+0000 7f39c8ff9700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f39c005ed90 0x7f39c0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.097+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f39d007aa30 con 0x7f39bc0038e0 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.097+0000 7f39cbfff700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f39c005ed90 0x7f39c0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.098+0000 7f39cbfff700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f39c005ed90 0x7f39c0061250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f39c4006fd0 tx=0x7f39c4008040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:45.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.101+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f39d004beb0 con 0x7f39bc0038e0 2023-05-16T07:38:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.264+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39d406c350 con 0x7f39bc0038e0 2023-05-16T07:38:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.265+0000 7f39c8ff9700 1 -- 172.21.15.92:0/1050077835 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+15477 (secure 0 0 0) 0x7f39d0023020 con 0x7f39bc0038e0 2023-05-16T07:38:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:45.269 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":176,"fsid":"7dc962fc-4181-4aa6-8ae9-07b8e654738a","created":"2023-05-16T07:02:08.692555+0000","modified":"2023-05-16T07:38:41.995555+0000","last_up_change":"2023-05-16T07:02:17.717687+0000","last_in_change":"2023-05-16T07:02:11.115703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-05-16T07:02:17.752010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-05-16T07:02:19.800493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7a2cf1f9-d953-439c-a6b2-9aaa74344bdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":116543},{"type":"v1","addr":"172.21.15.92:6809","nonce":116543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":116543},{"type":"v1","addr":"172.21.15.92:6811","nonce":116543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":116543},{"type":"v1","addr":"172.21.15.92:6815","nonce":116543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":116543},{"type":"v1","addr":"172.21.15.92:6813","nonce":116543}]},"public_addr":"172.21.15.92:6809/116543","cluster_addr":"172.21.15.92:6811/116543","heartbeat_back_addr":"172.21.15.92:6815/116543","heartbeat_front_addr":"172.21.15.92:6813/116543","state":["exists","up"]},{"osd":1,"uuid":"9997c8eb-1fdd-4bfe-ae2d-b08544f22d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":116552},{"type":"v1","addr":"172.21.15.92:6822","nonce":116552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":116552},{"type":"v1","addr":"172.21.15.92:6826","nonce":116552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":116552},{"type":"v1","addr":"172.21.15.92:6831","nonce":116552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":116552},{"type":"v1","addr":"172.21.15.92:6829","nonce":116552}]},"public_addr":"172.21.15.92:6822/116552","cluster_addr":"172.21.15.92:6826/116552","heartbeat_back_addr":"172.21.15.92:6831/116552","heartbeat_front_addr":"172.21.15.92:6829/116552","state":["exists","up"]},{"osd":2,"uuid":"637f304d-b1f2-4217-8b73-1f40aa9115f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":116553},{"type":"v1","addr":"172.21.15.92:6817","nonce":116553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":116553},{"type":"v1","addr":"172.21.15.92:6819","nonce":116553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6825","nonce":116553},{"type":"v1","addr":"172.21.15.92:6827","nonce":116553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":116553},{"type":"v1","addr":"172.21.15.92:6823","nonce":116553}]},"public_addr":"172.21.15.92:6817/116553","cluster_addr":"172.21.15.92:6819/116553","heartbeat_back_addr":"172.21.15.92:6827/116553","heartbeat_front_addr":"172.21.15.92:6823/116553","state":["exists","up"]},{"osd":3,"uuid":"f061ab26-f80b-471b-93b4-a2c5ed339458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":116542},{"type":"v1","addr":"172.21.15.92:6801","nonce":116542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":116542},{"type":"v1","addr":"172.21.15.92:6803","nonce":116542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":116542},{"type":"v1","addr":"172.21.15.92:6807","nonce":116542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":116542},{"type":"v1","addr":"172.21.15.92:6805","nonce":116542}]},"public_addr":"172.21.15.92:6801/116542","cluster_addr":"172.21.15.92:6803/116542","heartbeat_back_addr":"172.21.15.92:6807/116542","heartbeat_front_addr":"172.21.15.92:6805/116542","state":["exists","up"]},{"osd":4,"uuid":"c2795304-ad42-4a2d-bd4a-bf8ad7aec574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":86891},{"type":"v1","addr":"172.21.15.158:6801","nonce":86891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":86891},{"type":"v1","addr":"172.21.15.158:6803","nonce":86891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":86891},{"type":"v1","addr":"172.21.15.158:6807","nonce":86891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":86891},{"type":"v1","addr":"172.21.15.158:6805","nonce":86891}]},"public_addr":"172.21.15.158:6801/86891","cluster_addr":"172.21.15.158:6803/86891","heartbeat_back_addr":"172.21.15.158:6807/86891","heartbeat_front_addr":"172.21.15.158:6805/86891","state":["exists","up"]},{"osd":5,"uuid":"0cd9fd10-239c-442c-a3fc-f6ec09b33c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":86917},{"type":"v1","addr":"172.21.15.158:6817","nonce":86917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":86917},{"type":"v1","addr":"172.21.15.158:6819","nonce":86917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":86917},{"type":"v1","addr":"172.21.15.158:6823","nonce":86917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":86917},{"type":"v1","addr":"172.21.15.158:6821","nonce":86917}]},"public_addr":"172.21.15.158:6817/86917","cluster_addr":"172.21.15.158:6819/86917","heartbeat_back_addr":"172.21.15.158:6823/86917","heartbeat_front_addr":"172.21.15.158:6821/86917","state":["exists","up"]},{"osd":6,"uuid":"f3047d3d-d21e-4666-a4b1-72e17b0cd879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":86922},{"type":"v1","addr":"172.21.15.158:6825","nonce":86922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":86922},{"type":"v1","addr":"172.21.15.158:6827","nonce":86922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":86922},{"type":"v1","addr":"172.21.15.158:6831","nonce":86922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":86922},{"type":"v1","addr":"172.21.15.158:6829","nonce":86922}]},"public_addr":"172.21.15.158:6825/86922","cluster_addr":"172.21.15.158:6827/86922","heartbeat_back_addr":"172.21.15.158:6831/86922","heartbeat_front_addr":"172.21.15.158:6829/86922","state":["exists","up"]},{"osd":7,"uuid":"09acff50-ea9d-4909-b83f-269422145185","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":86892},{"type":"v1","addr":"172.21.15.158:6809","nonce":86892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":86892},{"type":"v1","addr":"172.21.15.158:6811","nonce":86892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":86892},{"type":"v1","addr":"172.21.15.158:6815","nonce":86892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":86892},{"type":"v1","addr":"172.21.15.158:6813","nonce":86892}]},"public_addr":"172.21.15.158:6809/86892","cluster_addr":"172.21.15.158:6811/86892","heartbeat_back_addr":"172.21.15.158:6815/86892","heartbeat_front_addr":"172.21.15.158:6813/86892","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.252929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.315693+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.334821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.230619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.299478+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.263797+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.330764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-05-16T07:02:16.223005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.158:6837/862724662":"2023-05-17T07:38:38.053205+0000","172.21.15.158:6836/862724662":"2023-05-17T07:38:38.053205+0000","172.21.15.158:0/1117121967":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3339950":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3223099484":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/1153790295":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3551364644":"2023-05-17T07:38:37.966236+0000","172.21.15.158:0/3460186346":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6833/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:6832/95623":"2023-05-17T07:38:30.703311+0000","172.21.15.158:0/240635450":"2023-05-17T07:38:30.703311+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":19,"length":1},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":32,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":45,"length":1},{"begin":49,"length":1},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":82,"length":1},{"begin":85,"length":1},{"begin":88,"length":2},{"begin":92,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":186,"length":36}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":2,"length":32}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-16T07:38:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f39c005ed90 msgr2=0x7f39c0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f39c005ed90 0x7f39c0061250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f39c4006fd0 tx=0x7f39c4008040).stop 2023-05-16T07:38:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 msgr2=0x7f39d413da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39d413da00 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f39d0001600 tx=0x7f39d0016000).stop 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 shutdown_connections 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f39c005ed90 0x7f39c0061250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f39d40c3db0 0x7f39d40c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39bc0038e0 0x7f39d413da00 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.266+0000 7f39dadda700 1 --2- 172.21.15.92:0/1050077835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f39d40cade0 0x7f39d40c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.267+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 >> 172.21.15.92:0/1050077835 conn(0x7f39d4067920 msgr2=0x7f39d40ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.267+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 shutdown_connections 2023-05-16T07:38:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:45.267+0000 7f39dadda700 1 -- 172.21.15.92:0/1050077835 wait complete. 2023-05-16T07:38:46.282 INFO:tasks.ceph:Scrubbing osd.0 2023-05-16T07:38:46.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.399+0000 7f66719c4700 1 Processor -- start 2023-05-16T07:38:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.399+0000 7f66719c4700 1 -- start start 2023-05-16T07:38:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f66719c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f666c1509f0 0x7f666c0a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f66719c4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c151780 con 0x7f666c069820 2023-05-16T07:38:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f66719c4700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c151a60 con 0x7f666c150dd0 2023-05-16T07:38:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f66719c4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666c065570 con 0x7f666c1509f0 2023-05-16T07:38:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f666affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f666c1509f0 0x7f666c0a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f666affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f666c1509f0 0x7f666c0a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53774/0 (socket says 172.21.15.92:53774) 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f666b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f666c069820 0x7f666c0659c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36294/0 (socket says 172.21.15.92:36294) 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.400+0000 7f666affd700 1 -- 172.21.15.92:0/1387910771 learned_addr learned my addr 172.21.15.92:0/1387910771 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f666affd700 1 -- 172.21.15.92:0/1387910771 >> v1:172.21.15.158:6789/0 conn(0x7f666c150dd0 legacy=0x7f666c069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f666affd700 1 -- 172.21.15.92:0/1387910771 >> v1:172.21.15.92:6789/0 conn(0x7f666c069820 legacy=0x7f666c0659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f666affd700 1 -- 172.21.15.92:0/1387910771 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c151d40 con 0x7f666c1509f0 2023-05-16T07:38:46.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f6669ffb700 1 -- 172.21.15.92:0/1387910771 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806935828 0 0) 0x7f666c151780 con 0x7f666c069820 2023-05-16T07:38:46.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f666affd700 1 --2- 172.21.15.92:0/1387910771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0a6db0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f665c009f20 tx=0x7f665c009840).ready entity=mon.0 client_cookie=c77819351da0c56a server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f6669ffb700 1 -- 172.21.15.92:0/1387910771 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00e070 con 0x7f666c1509f0 2023-05-16T07:38:46.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f6669ffb700 1 -- 172.21.15.92:0/1387910771 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00ba80 con 0x7f666c1509f0 2023-05-16T07:38:46.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f6669ffb700 1 -- 172.21.15.92:0/1387910771 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00be00 con 0x7f666c1509f0 2023-05-16T07:38:46.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 -- 172.21.15.92:0/1387910771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 msgr2=0x7f666c0a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 --2- 172.21.15.92:0/1387910771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0a6db0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f665c009f20 tx=0x7f665c009840).stop 2023-05-16T07:38:46.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 -- 172.21.15.92:0/1387910771 shutdown_connections 2023-05-16T07:38:46.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 --2- 172.21.15.92:0/1387910771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0a6db0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 -- 172.21.15.92:0/1387910771 >> 172.21.15.92:0/1387910771 conn(0x7f666c0c8270 msgr2=0x7f666c0c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 -- 172.21.15.92:0/1387910771 shutdown_connections 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.401+0000 7f66719c4700 1 -- 172.21.15.92:0/1387910771 wait complete. 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 Processor -- start 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 -- start start 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f666c069820 0x7f666c0bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f666c150dd0 0x7f666c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c161bd0 con 0x7f666c1509f0 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c164460 con 0x7f666c069820 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.403+0000 7f66719c4700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c164740 con 0x7f666c150dd0 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0c0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53780/0 (socket says 172.21.15.92:53780) 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 -- 172.21.15.92:0/3695685081 learned_addr learned my addr 172.21.15.92:0/3695685081 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666affd700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f666c069820 0x7f666c0bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666b7fe700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f666c150dd0 0x7f666c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 -- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f666c150dd0 msgr2=0x7f666c14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f666c150dd0 0x7f666c14ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 -- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f666c069820 msgr2=0x7f666c0bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f666c069820 0x7f666c0bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 -- 172.21.15.92:0/3695685081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c009ae0 con 0x7f666c1509f0 2023-05-16T07:38:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f666a7fc700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0c0a20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7f665400c4c0 tx=0x7f6654007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400ce70 con 0x7f666c1509f0 2023-05-16T07:38:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654001e90 con 0x7f666c1509f0 2023-05-16T07:38:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c164a20 con 0x7f666c1509f0 2023-05-16T07:38:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654007250 con 0x7f666c1509f0 2023-05-16T07:38:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.404+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c0aa9d0 con 0x7f666c1509f0 2023-05-16T07:38:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.405+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f666c047150 con 0x7f666c1509f0 2023-05-16T07:38:46.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.405+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f6654010070 con 0x7f666c1509f0 2023-05-16T07:38:46.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f6662ffd700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f664c05ed90 0x7f664c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f66540793a0 con 0x7f666c1509f0 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f6662ffd700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7f664c065c40 0x7f664c0680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 --> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f664c068690 con 0x7f664c065c40 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665407ba10 con 0x7f666c1509f0 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f666affd700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f664c05ed90 0x7f664c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f666b7fe700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7f664c065c40 0x7f664c0680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f666b7fe700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7f664c065c40 0x7f664c0680b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.406+0000 7f666affd700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f664c05ed90 0x7f664c061250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f665c009880 tx=0x7f666c0654e0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.407+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== osd.0 v2:172.21.15.92:6808/116543 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f664c068690 con 0x7f664c065c40 2023-05-16T07:38:46.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.420+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 --> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f666c161d10 con 0x7f664c065c40 2023-05-16T07:38:46.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.420+0000 7f6662ffd700 1 -- 172.21.15.92:0/3695685081 <== osd.0 v2:172.21.15.92:6808/116543 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f666c161d10 con 0x7f664c065c40 2023-05-16T07:38:46.424 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:46.424 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:46.424 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7f664c065c40 msgr2=0x7f664c0680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:6808/116543,v1:172.21.15.92:6809/116543] conn(0x7f664c065c40 0x7f664c0680b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f664c05ed90 msgr2=0x7f664c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f664c05ed90 0x7f664c061250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f665c009880 tx=0x7f666c0654e0).stop 2023-05-16T07:38:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 msgr2=0x7f666c0c0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 --2- 172.21.15.92:0/3695685081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f666c1509f0 0x7f666c0c0a20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7f665400c4c0 tx=0x7f6654007e00).stop 2023-05-16T07:38:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f666b7fe700 1 -- 172.21.15.92:0/3695685081 reap_dead start 2023-05-16T07:38:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 shutdown_connections 2023-05-16T07:38:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 >> 172.21.15.92:0/3695685081 conn(0x7f666c0c8270 msgr2=0x7f666c0c5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 shutdown_connections 2023-05-16T07:38:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.421+0000 7f66719c4700 1 -- 172.21.15.92:0/3695685081 wait complete. 2023-05-16T07:38:46.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-05-16T07:38:46.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.542+0000 7f9f24ad5700 1 Processor -- start 2023-05-16T07:38:46.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.542+0000 7f9f24ad5700 1 -- start start 2023-05-16T07:38:46.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f24ad5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f24ad5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f20154550 0x7f9f200c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f24ad5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200c0660 con 0x7f9f200696a0 2023-05-16T07:38:46.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f24ad5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f200c1b00 con 0x7f9f20154550 2023-05-16T07:38:46.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f24ad5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200bfd30 con 0x7f9f20154170 2023-05-16T07:38:46.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f1dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f20154550 0x7f9f200c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f1dd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f20154550 0x7f9f200c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53790/0 (socket says 172.21.15.92:53790) 2023-05-16T07:38:46.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.543+0000 7f9f1dd9b700 1 -- 172.21.15.92:0/1857822341 learned_addr learned my addr 172.21.15.92:0/1857822341 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:46.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1dd9b700 1 -- 172.21.15.92:0/1857822341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 msgr2=0x7f9f200c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:46.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1e59c700 1 --2- 172.21.15.92:0/1857822341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1dd9b700 1 --2- 172.21.15.92:0/1857822341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1dd9b700 1 -- 172.21.15.92:0/1857822341 >> v1:172.21.15.158:6789/0 conn(0x7f9f200696a0 legacy=0x7f9f20157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:46.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1dd9b700 1 -- 172.21.15.92:0/1857822341 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f200c0010 con 0x7f9f20154550 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1dd9b700 1 --2- 172.21.15.92:0/1857822341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200c6160 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7f9f14006fd0 tx=0x7f9f140090f0).ready entity=mon.0 client_cookie=35fb563937c713cb server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1d59a700 1 -- 172.21.15.92:0/1857822341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400b600 con 0x7f9f20154550 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1d59a700 1 -- 172.21.15.92:0/1857822341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1400b760 con 0x7f9f20154550 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f1d59a700 1 -- 172.21.15.92:0/1857822341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400bb00 con 0x7f9f20154550 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 -- 172.21.15.92:0/1857822341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 msgr2=0x7f9f200c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/1857822341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200c6160 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7f9f14006fd0 tx=0x7f9f140090f0).stop 2023-05-16T07:38:46.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 -- 172.21.15.92:0/1857822341 shutdown_connections 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/1857822341 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/1857822341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200c6160 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 -- 172.21.15.92:0/1857822341 >> 172.21.15.92:0/1857822341 conn(0x7f9f200cb700 msgr2=0x7f9f200c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 -- 172.21.15.92:0/1857822341 shutdown_connections 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.544+0000 7f9f24ad5700 1 -- 172.21.15.92:0/1857822341 wait complete. 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.545+0000 7f9f24ad5700 1 Processor -- start 2023-05-16T07:38:46.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 -- start start 2023-05-16T07:38:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f200696a0 0x7f9f2014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20164f20 con 0x7f9f20154550 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f201500f0 con 0x7f9f200696a0 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f24ad5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200ab9b0 con 0x7f9f20154170 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f1ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53800/0 (socket says 172.21.15.92:53800) 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.546+0000 7f9f1ed9d700 1 -- 172.21.15.92:0/3476512517 learned_addr learned my addr 172.21.15.92:0/3476512517 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:46.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1e59c700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f200696a0 0x7f9f2014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1dd9b700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1ed9d700 1 -- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 msgr2=0x7f9f200aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1ed9d700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1ed9d700 1 -- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f200696a0 msgr2=0x7f9f2014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1ed9d700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f200696a0 0x7f9f2014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1ed9d700 1 -- 172.21.15.92:0/3476512517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009370 con 0x7f9f20154550 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f1ed9d700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200ab4c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f9f1800c490 tx=0x7f9f18007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18002ea0 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800f470 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f20152500 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800f810 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.547+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f20151bd0 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.548+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f200666c0 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.548+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f9f18010070 con 0x7f9f20154550 2023-05-16T07:38:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.548+0000 7f9f0e7fc700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f0005ed90 0x7f9f00061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.549+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9f18078f90 con 0x7f9f20154550 2023-05-16T07:38:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.549+0000 7f9f1e59c700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f0005ed90 0x7f9f00061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.549+0000 7f9f1e59c700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f0005ed90 0x7f9f00061250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9f1000e4c0 tx=0x7f9f100076f0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.553+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f9f180478f0 con 0x7f9f20154550 2023-05-16T07:38:46.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.715+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f20152690 con 0x7f9f0005ed90 2023-05-16T07:38:46.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.716+0000 7f9f0e7fc700 1 -- 172.21.15.92:0/3476512517 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9f20152690 con 0x7f9f0005ed90 2023-05-16T07:38:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f0005ed90 msgr2=0x7f9f00061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f0005ed90 0x7f9f00061250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9f1000e4c0 tx=0x7f9f100076f0).stop 2023-05-16T07:38:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 msgr2=0x7f9f200ab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200ab4c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f9f1800c490 tx=0x7f9f18007de0).stop 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 shutdown_connections 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f20154550 0x7f9f200ab4c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9f200696a0 0x7f9f2014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f9f0005ed90 0x7f9f00061250 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 --2- 172.21.15.92:0/3476512517 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f9f20154170 0x7f9f200aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 >> 172.21.15.92:0/3476512517 conn(0x7f9f200cb700 msgr2=0x7f9f200abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:46.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 shutdown_connections 2023-05-16T07:38:46.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.718+0000 7f9f24ad5700 1 -- 172.21.15.92:0/3476512517 wait complete. 2023-05-16T07:38:46.723 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2023-05-16T07:38:46.732 INFO:tasks.ceph:Scrubbing osd.1 2023-05-16T07:38:46.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 Processor -- start 2023-05-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 -- start start 2023-05-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f84c0154090 0x7f84c00aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84c0154470 0x7f84c00a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c00c94d0 con 0x7f84c00aa150 2023-05-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c00aa5a0 con 0x7f84c0154470 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.844+0000 7f84c6a98700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c00cb7e0 con 0x7f84c0154090 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84c0154470 0x7f84c00a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84c5035700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f84c00aa150 0x7f84c0068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36320/0 (socket says 172.21.15.92:36320) 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84c0154470 0x7f84c00a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53804/0 (socket says 172.21.15.92:53804) 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84c5035700 1 -- 172.21.15.92:0/3020799582 learned_addr learned my addr 172.21.15.92:0/3020799582 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84c4834700 1 --2- 172.21.15.92:0/3020799582 >> v2:172.21.15.158:3300/0 conn(0x7f84c0154090 0x7f84c00aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 -- 172.21.15.92:0/3020799582 >> v2:172.21.15.158:3300/0 conn(0x7f84c0154090 msgr2=0x7f84c00aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 --2- 172.21.15.92:0/3020799582 >> v2:172.21.15.158:3300/0 conn(0x7f84c0154090 0x7f84c00aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 -- 172.21.15.92:0/3020799582 >> v1:172.21.15.92:6789/0 conn(0x7f84c00aa150 legacy=0x7f84c0068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bf7fe700 1 -- 172.21.15.92:0/3020799582 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034205024 0 0) 0x7f84c00c94d0 con 0x7f84c00aa150 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 -- 172.21.15.92:0/3020799582 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00c88c0 con 0x7f84c0154470 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bffff700 1 --2- 172.21.15.92:0/3020799582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00a9c60 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f84c006bd10 tx=0x7f84a8009060).ready entity=mon.0 client_cookie=37ee92a03fff4091 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bf7fe700 1 -- 172.21.15.92:0/3020799582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800f510 con 0x7f84c0154470 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bf7fe700 1 -- 172.21.15.92:0/3020799582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a800f670 con 0x7f84c0154470 2023-05-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.845+0000 7f84bf7fe700 1 -- 172.21.15.92:0/3020799582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800f9f0 con 0x7f84c0154470 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 -- 172.21.15.92:0/3020799582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 msgr2=0x7f84c00a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 --2- 172.21.15.92:0/3020799582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00a9c60 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f84c006bd10 tx=0x7f84a8009060).stop 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 -- 172.21.15.92:0/3020799582 shutdown_connections 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 --2- 172.21.15.92:0/3020799582 >> v2:172.21.15.158:3300/0 conn(0x7f84c0154090 0x7f84c00aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 --2- 172.21.15.92:0/3020799582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00a9c60 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 -- 172.21.15.92:0/3020799582 >> 172.21.15.92:0/3020799582 conn(0x7f84c0067920 msgr2=0x7f84c00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 -- 172.21.15.92:0/3020799582 shutdown_connections 2023-05-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.846+0000 7f84c6a98700 1 -- 172.21.15.92:0/3020799582 wait complete. 2023-05-16T07:38:46.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.847+0000 7f84c6a98700 1 Processor -- start 2023-05-16T07:38:46.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.847+0000 7f84c6a98700 1 -- start start 2023-05-16T07:38:46.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f84c00aa150 0x7f84c00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84c0154090 0x7f84c00c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00b0c90 con 0x7f84c0154470 2023-05-16T07:38:46.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00c2b70 con 0x7f84c0154090 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00c4530 con 0x7f84c00aa150 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53812/0 (socket says 172.21.15.92:53812) 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 -- 172.21.15.92:0/4150057370 learned_addr learned my addr 172.21.15.92:0/4150057370 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84bffff700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84c0154090 0x7f84c00c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 -- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f84c00aa150 msgr2=0x7f84c00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f84c00aa150 0x7f84c00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 -- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84c0154090 msgr2=0x7f84c00c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84c0154090 0x7f84c00c5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 -- 172.21.15.92:0/4150057370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80092a0 con 0x7f84c0154470 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c5035700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00afa60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f84b800c7e0 tx=0x7f84b8007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8005d30 con 0x7f84c0154470 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b8005e90 con 0x7f84c0154470 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c00c4810 con 0x7f84c0154470 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8007320 con 0x7f84c0154470 2023-05-16T07:38:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.848+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c00c5500 con 0x7f84c0154470 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.849+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f84c0066730 con 0x7f84c0154470 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f84b8010030 con 0x7f84c0154470 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bcff9700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f84a005ed90 0x7f84a0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f84b8079ba0 con 0x7f84c0154470 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bcff9700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f84a0065c40 0x7f84a00680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 --> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84a0068690 con 0x7f84a0065c40 2023-05-16T07:38:46.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b807c210 con 0x7f84c0154470 2023-05-16T07:38:46.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84c4834700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f84a005ed90 0x7f84a0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.850+0000 7f84bffff700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f84a0065c40 0x7f84a00680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.851+0000 7f84bffff700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f84a0065c40 0x7f84a00680b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.851+0000 7f84c4834700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f84a005ed90 0x7f84a0061250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f84b0009a30 tx=0x7f84b0009060).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.851+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== osd.1 v2:172.21.15.92:6820/116552 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f84a0068690 con 0x7f84a0065c40 2023-05-16T07:38:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.864+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 --> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f84c006c350 con 0x7f84a0065c40 2023-05-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.864+0000 7f84bcff9700 1 -- 172.21.15.92:0/4150057370 <== osd.1 v2:172.21.15.92:6820/116552 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f84c006c350 con 0x7f84a0065c40 2023-05-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f84a0065c40 msgr2=0x7f84a00680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:6820/116552,v1:172.21.15.92:6822/116552] conn(0x7f84a0065c40 0x7f84a00680b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f84a005ed90 msgr2=0x7f84a0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f84a005ed90 0x7f84a0061250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f84b0009a30 tx=0x7f84b0009060).stop 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 msgr2=0x7f84c00afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 --2- 172.21.15.92:0/4150057370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c0154470 0x7f84c00afa60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f84b800c7e0 tx=0x7f84b8007f90).stop 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c5035700 1 -- 172.21.15.92:0/4150057370 reap_dead start 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 shutdown_connections 2023-05-16T07:38:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 >> 172.21.15.92:0/4150057370 conn(0x7f84c0067920 msgr2=0x7f84c014da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 shutdown_connections 2023-05-16T07:38:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.865+0000 7f84c6a98700 1 -- 172.21.15.92:0/4150057370 wait complete. 2023-05-16T07:38:46.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-05-16T07:38:46.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.988+0000 7f6586f39700 1 Processor -- start 2023-05-16T07:38:46.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.988+0000 7f6586f39700 1 -- start start 2023-05-16T07:38:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f6586f39700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6580069db0 0x7f658015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f6586f39700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800c0660 con 0x7f6580154550 2023-05-16T07:38:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f6586f39700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800c1b00 con 0x7f6580154170 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f6586f39700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65800bfd30 con 0x7f6580069db0 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6580154550 0x7f65800696a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36336/0 (socket says 172.21.15.92:36336) 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657ffff700 1 -- 172.21.15.92:0/3670472879 learned_addr learned my addr 172.21.15.92:0/3670472879 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f65854d6700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6580069db0 0x7f658015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176025561 0 0) 0x7f65800c0660 con 0x7f6580154550 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65680033f0 con 0x7f6580154550 2023-05-16T07:38:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580638022 0 0) 0x7f65800c1b00 con 0x7f6580154170 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65800c0660 con 0x7f6580154170 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349479082 0 0) 0x7f65680033f0 con 0x7f6580154550 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6580069db0 msgr2=0x7f658015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.989+0000 7f657f7fe700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6580069db0 0x7f658015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 >> v1:172.21.15.158:6789/0 conn(0x7f6580154170 legacy=0x7f65800c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65800c0010 con 0x7f6580154550 2023-05-16T07:38:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f65740013f0 con 0x7f6580154550 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f65854d6700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6580069db0 0x7f658015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 >> v1:172.21.15.92:6789/0 conn(0x7f6580154550 legacy=0x7f65800696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6568003aa0 0x7f6568003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6568004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6568005000 0x7f656800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800bfd30 con 0x7f6568004690 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656800bf60 con 0x7f6568005000 2023-05-16T07:38:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656800c120 con 0x7f6568003aa0 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6568004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f65854d6700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6568005000 0x7f656800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f6584cd5700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6568003aa0 0x7f6568003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 -- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6568003aa0 msgr2=0x7f6568003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f6568003aa0 0x7f6568003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 -- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6568005000 msgr2=0x7f656800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6568005000 0x7f656800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 -- 172.21.15.92:0/3670472879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680033f0 con 0x7f6568004690 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657ffff700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6568004b10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7f6580154930 tx=0x7f6574001cc0).ready entity=mon.0 client_cookie=d98a57acacde4ea4 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740070d0 con 0x7f6568004690 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6574007230 con 0x7f6568004690 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.990+0000 7f657f7fe700 1 -- 172.21.15.92:0/3670472879 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740075f0 con 0x7f6568004690 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f6586f39700 1 -- 172.21.15.92:0/3670472879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 msgr2=0x7f6568004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f6586f39700 1 --2- 172.21.15.92:0/3670472879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6568004b10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7f6580154930 tx=0x7f6574001cc0).stop 2023-05-16T07:38:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f65854d6700 1 -- 172.21.15.92:0/3670472879 reap_dead start 2023-05-16T07:38:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f6586f39700 1 -- 172.21.15.92:0/3670472879 shutdown_connections 2023-05-16T07:38:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f6586f39700 1 -- 172.21.15.92:0/3670472879 >> 172.21.15.92:0/3670472879 conn(0x7f65800cb700 msgr2=0x7f65800c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f6586f39700 1 -- 172.21.15.92:0/3670472879 shutdown_connections 2023-05-16T07:38:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.991+0000 7f6586f39700 1 -- 172.21.15.92:0/3670472879 wait complete. 2023-05-16T07:38:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.992+0000 7f6586f39700 1 Processor -- start 2023-05-16T07:38:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.992+0000 7f6586f39700 1 -- start start 2023-05-16T07:38:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6586f39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6580152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6586f39700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65800c9ef0 0x7f65800ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6586f39700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6580150610 0x7f6580150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6586f39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6580151f50 con 0x7f6568004690 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6586f39700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800ca860 con 0x7f6580150610 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6586f39700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6580151340 con 0x7f65800c9ef0 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6580152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6580152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53830/0 (socket says 172.21.15.92:53830) 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 -- 172.21.15.92:0/71441768 learned_addr learned my addr 172.21.15.92:0/71441768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f657ffff700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65800c9ef0 0x7f65800ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f65854d6700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6580150610 0x7f6580150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 -- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65800c9ef0 msgr2=0x7f65800ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65800c9ef0 0x7f65800ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 -- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6580150610 msgr2=0x7f6580150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6580150610 0x7f6580150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 -- 172.21.15.92:0/71441768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580151620 con 0x7f6568004690 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f657ffff700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65800c9ef0 0x7f65800ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f65854d6700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6580150610 0x7f6580150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f6584cd5700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6580152b20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7f657000eb90 tx=0x7f6570007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.993+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570001b70 con 0x7f6568004690 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.994+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570001cf0 con 0x7f6568004690 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.994+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6580151900 con 0x7f6568004690 2023-05-16T07:38:47.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.994+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000c8f0 con 0x7f6568004690 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.994+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580150f80 con 0x7f6568004690 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.994+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65800666c0 con 0x7f6568004690 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.995+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f657000cc60 con 0x7f6568004690 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.995+0000 7f657cff9700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f656c05ed90 0x7f656c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.995+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f657007a120 con 0x7f6568004690 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.995+0000 7f657ffff700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f656c05ed90 0x7f656c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.996+0000 7f657ffff700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f656c05ed90 0x7f656c061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f6574007cd0 tx=0x7f6574015000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:46.999+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f657004b5a0 con 0x7f6568004690 2023-05-16T07:38:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.164+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6580152090 con 0x7f656c05ed90 2023-05-16T07:38:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.164+0000 7f657cff9700 1 -- 172.21.15.92:0/71441768 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6574000f50 con 0x7f656c05ed90 2023-05-16T07:38:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f656c05ed90 msgr2=0x7f656c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f656c05ed90 0x7f656c061250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f6574007cd0 tx=0x7f6574015000).stop 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 msgr2=0x7f6580152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6580152b20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7f657000eb90 tx=0x7f6570007f30).stop 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 shutdown_connections 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6580150610 0x7f6580150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6568004690 0x7f6580152b20 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f656c05ed90 0x7f656c061250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 --2- 172.21.15.92:0/71441768 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65800c9ef0 0x7f65800ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 >> 172.21.15.92:0/71441768 conn(0x7f65800cb700 msgr2=0x7f65800ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 shutdown_connections 2023-05-16T07:38:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.166+0000 7f6586f39700 1 -- 172.21.15.92:0/71441768 wait complete. 2023-05-16T07:38:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2023-05-16T07:38:47.181 INFO:tasks.ceph:Scrubbing osd.2 2023-05-16T07:38:47.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:47.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.291+0000 7fba34c53700 1 Processor -- start 2023-05-16T07:38:47.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.291+0000 7fba34c53700 1 -- start start 2023-05-16T07:38:47.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba34c53700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fba30155140 0x7fba300622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba34c53700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba30063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba34c53700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3014ead0 con 0x7fba3014e6f0 2023-05-16T07:38:47.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba34c53700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba30154cb0 con 0x7fba30155140 2023-05-16T07:38:47.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba34c53700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba30062bf0 con 0x7fba300627a0 2023-05-16T07:38:47.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba2dd9b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fba30155140 0x7fba300622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba2ed9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba30063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba2dd9b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fba30155140 0x7fba300622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:57478/0 (socket says 172.21.15.92:57478) 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3131970140 learned_addr learned my addr 172.21.15.92:0/3131970140 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba2e59c700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fba3014e6f0 0x7fba300c4d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:39954/0 (socket says 172.21.15.92:39954) 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.292+0000 7fba2ed9d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba30063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:46412/0 (socket says 172.21.15.92:46412) 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 msgr2=0x7fba30063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2dd9b700 1 --2- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba30063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3131970140 >> v1:172.21.15.158:6789/0 conn(0x7fba3014e6f0 legacy=0x7fba300c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2d59a700 1 -- 172.21.15.92:0/3131970140 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483294442 0 0) 0x7fba3014ead0 con 0x7fba3014e6f0 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3131970140 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300c6020 con 0x7fba30155140 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2ed9d700 1 --2- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba30063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2dd9b700 1 --2- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 0x7fba300622b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fba18006fd0 tx=0x7fba180090f0).ready entity=mon.1 client_cookie=f0f5f7e084f551b2 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2d59a700 1 -- 172.21.15.92:0/3131970140 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800b580 con 0x7fba30155140 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2d59a700 1 -- 172.21.15.92:0/3131970140 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800b6e0 con 0x7fba30155140 2023-05-16T07:38:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.293+0000 7fba2d59a700 1 -- 172.21.15.92:0/3131970140 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800ba60 con 0x7fba30155140 2023-05-16T07:38:47.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 -- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 msgr2=0x7fba300622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 --2- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 0x7fba300622b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fba18006fd0 tx=0x7fba180090f0).stop 2023-05-16T07:38:47.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 -- 172.21.15.92:0/3131970140 shutdown_connections 2023-05-16T07:38:47.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 --2- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba30063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 --2- 172.21.15.92:0/3131970140 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 0x7fba300622b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 -- 172.21.15.92:0/3131970140 >> 172.21.15.92:0/3131970140 conn(0x7fba3005e820 msgr2=0x7fba300c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 -- 172.21.15.92:0/3131970140 shutdown_connections 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.294+0000 7fba34c53700 1 -- 172.21.15.92:0/3131970140 wait complete. 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.295+0000 7fba34c53700 1 Processor -- start 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.295+0000 7fba34c53700 1 -- start start 2023-05-16T07:38:47.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3014e6f0 0x7fba3014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 0x7fba300aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3014d810 con 0x7fba3014e6f0 2023-05-16T07:38:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300ab4e0 con 0x7fba30155140 2023-05-16T07:38:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba30164b00 con 0x7fba300627a0 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3014e6f0 0x7fba3014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3014e6f0 0x7fba3014d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53846/0 (socket says 172.21.15.92:53846) 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3067175273 learned_addr learned my addr 172.21.15.92:0/3067175273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2e59c700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2ed9d700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 0x7fba300aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 msgr2=0x7fba300a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fba300627a0 0x7fba300a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 msgr2=0x7fba300aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fba30155140 0x7fba300aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 -- 172.21.15.92:0/3067175273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba280092a0 con 0x7fba3014e6f0 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba2dd9b700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3014e6f0 0x7fba3014d320 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7fba30064800 tx=0x7fba18017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18006a40 con 0x7fba3014e6f0 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800b850 con 0x7fba3014e6f0 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1801d680 con 0x7fba3014e6f0 2023-05-16T07:38:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba18009370 con 0x7fba3014e6f0 2023-05-16T07:38:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.296+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba300a4a20 con 0x7fba3014e6f0 2023-05-16T07:38:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.297+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba3006c350 con 0x7fba3014e6f0 2023-05-16T07:38:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fba1800f070 con 0x7fba3014e6f0 2023-05-16T07:38:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba267fc700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fba1005ed90 0x7fba10061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fba1807bdf0 con 0x7fba3014e6f0 2023-05-16T07:38:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba267fc700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7fba10065c40 0x7fba100680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 --> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba10068690 con 0x7fba10065c40 2023-05-16T07:38:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba1807c1f0 con 0x7fba3014e6f0 2023-05-16T07:38:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.298+0000 7fba2e59c700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fba1005ed90 0x7fba10061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.299+0000 7fba2ed9d700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7fba10065c40 0x7fba100680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.299+0000 7fba2ed9d700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7fba10065c40 0x7fba100680b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.299+0000 7fba2e59c700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fba1005ed90 0x7fba10061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fba300aaaa0 tx=0x7fba20001500).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.299+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== osd.2 v2:172.21.15.92:6816/116553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fba10068690 con 0x7fba10065c40 2023-05-16T07:38:47.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.312+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 --> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fba30164c40 con 0x7fba10065c40 2023-05-16T07:38:47.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba267fc700 1 -- 172.21.15.92:0/3067175273 <== osd.2 v2:172.21.15.92:6816/116553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fba30164c40 con 0x7fba10065c40 2023-05-16T07:38:47.316 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:47.317 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7fba10065c40 msgr2=0x7fba100680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:6816/116553,v1:172.21.15.92:6817/116553] conn(0x7fba10065c40 0x7fba100680b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fba1005ed90 msgr2=0x7fba10061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fba1005ed90 0x7fba10061250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fba300aaaa0 tx=0x7fba20001500).stop 2023-05-16T07:38:47.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3014e6f0 msgr2=0x7fba3014d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 --2- 172.21.15.92:0/3067175273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3014e6f0 0x7fba3014d320 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7fba30064800 tx=0x7fba18017000).stop 2023-05-16T07:38:47.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba2ed9d700 1 -- 172.21.15.92:0/3067175273 reap_dead start 2023-05-16T07:38:47.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 shutdown_connections 2023-05-16T07:38:47.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 >> 172.21.15.92:0/3067175273 conn(0x7fba3005e820 msgr2=0x7fba300c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 shutdown_connections 2023-05-16T07:38:47.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.313+0000 7fba34c53700 1 -- 172.21.15.92:0/3067175273 wait complete. 2023-05-16T07:38:47.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-05-16T07:38:47.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.436+0000 7f65bddb6700 1 Processor -- start 2023-05-16T07:38:47.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.436+0000 7f65bddb6700 1 -- start start 2023-05-16T07:38:47.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65bddb6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65b8154170 0x7f65b80c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65bddb6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154550 0x7f65b80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65bddb6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b80c0660 con 0x7f65b80696a0 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65bddb6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b80c1b00 con 0x7f65b8154170 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65bddb6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b80bfd30 con 0x7f65b8154550 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65b8154170 0x7f65b80c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65b8154170 0x7f65b80c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53854/0 (socket says 172.21.15.92:53854) 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 -- 172.21.15.92:0/2153803246 learned_addr learned my addr 172.21.15.92:0/2153803246 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b6ffd700 1 --2- 172.21.15.92:0/2153803246 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154550 0x7f65b80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 -- 172.21.15.92:0/2153803246 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154550 msgr2=0x7f65b80c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 --2- 172.21.15.92:0/2153803246 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154550 0x7f65b80c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 -- 172.21.15.92:0/2153803246 >> v1:172.21.15.158:6789/0 conn(0x7f65b80696a0 legacy=0x7f65b8157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 -- 172.21.15.92:0/2153803246 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b80c0010 con 0x7f65b8154170 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b77fe700 1 --2- 172.21.15.92:0/2153803246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b8154170 0x7f65b80c5c70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f65a8009f20 tx=0x7f65a8009840).ready entity=mon.0 client_cookie=d01741900d490990 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b67fc700 1 -- 172.21.15.92:0/2153803246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a800e070 con 0x7f65b8154170 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b67fc700 1 -- 172.21.15.92:0/2153803246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a800ba80 con 0x7f65b8154170 2023-05-16T07:38:47.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.437+0000 7f65b67fc700 1 -- 172.21.15.92:0/2153803246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a800be00 con 0x7f65b8154170 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 -- 172.21.15.92:0/2153803246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b8154170 msgr2=0x7f65b80c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 --2- 172.21.15.92:0/2153803246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b8154170 0x7f65b80c5c70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f65a8009f20 tx=0x7f65a8009840).stop 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 -- 172.21.15.92:0/2153803246 shutdown_connections 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 --2- 172.21.15.92:0/2153803246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b8154170 0x7f65b80c5c70 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 --2- 172.21.15.92:0/2153803246 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154550 0x7f65b80c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 -- 172.21.15.92:0/2153803246 >> 172.21.15.92:0/2153803246 conn(0x7f65b80cb700 msgr2=0x7f65b80c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 -- 172.21.15.92:0/2153803246 shutdown_connections 2023-05-16T07:38:47.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.438+0000 7f65bddb6700 1 -- 172.21.15.92:0/2153803246 wait complete. 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.439+0000 7f65bddb6700 1 Processor -- start 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.439+0000 7f65bddb6700 1 -- start start 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65bddb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 0x7f65b80c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65bddb6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154170 0x7f65b8167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65bddb6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65b8154550 0x7f65b80c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65bddb6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8168240 con 0x7f65b80696a0 2023-05-16T07:38:47.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65bddb6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b814de80 con 0x7f65b8154550 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65bddb6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8154b90 con 0x7f65b8154170 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 0x7f65b80c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 0x7f65b80c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53860/0 (socket says 172.21.15.92:53860) 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 -- 172.21.15.92:0/1227098026 learned_addr learned my addr 172.21.15.92:0/1227098026 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b6ffd700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154170 0x7f65b8167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b7fff700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65b8154550 0x7f65b80c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 -- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154170 msgr2=0x7f65b8167d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154170 0x7f65b8167d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 -- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65b8154550 msgr2=0x7f65b80c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65b8154550 0x7f65b80c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b77fe700 1 -- 172.21.15.92:0/1227098026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a8009ae0 con 0x7f65b80696a0 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.440+0000 7f65b7fff700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65b8154550 0x7f65b80c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.441+0000 7f65b77fe700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 0x7f65b80c4980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f65a800b6c0 tx=0x7f65a8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.441+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8007080 con 0x7f65b80696a0 2023-05-16T07:38:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.441+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a80092e0 con 0x7f65b80696a0 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.441+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b80c2230 con 0x7f65b80696a0 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.441+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8013e50 con 0x7f65b80696a0 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.441+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b8168730 con 0x7f65b80696a0 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.442+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b8154e70 con 0x7f65b80696a0 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.442+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f65a801b040 con 0x7f65b80696a0 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.442+0000 7f659f7fe700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65a405ed90 0x7f65a4061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.442+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f65a807b490 con 0x7f65b80696a0 2023-05-16T07:38:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.442+0000 7f65b6ffd700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65a405ed90 0x7f65a4061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.443+0000 7f65b6ffd700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65a405ed90 0x7f65a4061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f65b80cab80 tx=0x7f65ac008040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.446+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f65a804c910 con 0x7f65b80696a0 2023-05-16T07:38:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.611+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f65b8154cd0 con 0x7f65a405ed90 2023-05-16T07:38:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.611+0000 7f659f7fe700 1 -- 172.21.15.92:0/1227098026 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65ac00b750 con 0x7f65a405ed90 2023-05-16T07:38:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65a405ed90 msgr2=0x7f65a4061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65a405ed90 0x7f65a4061250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f65b80cab80 tx=0x7f65ac008040).stop 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 msgr2=0x7f65b80c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 0x7f65b80c4980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f65a800b6c0 tx=0x7f65a8009840).stop 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 shutdown_connections 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f65b8154550 0x7f65b80c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b80696a0 0x7f65b80c4980 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f65a405ed90 0x7f65a4061250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 --2- 172.21.15.92:0/1227098026 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f65b8154170 0x7f65b8167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 >> 172.21.15.92:0/1227098026 conn(0x7f65b80cb700 msgr2=0x7f65b80afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 shutdown_connections 2023-05-16T07:38:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.613+0000 7f65bddb6700 1 -- 172.21.15.92:0/1227098026 wait complete. 2023-05-16T07:38:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2023-05-16T07:38:47.628 INFO:tasks.ceph:Scrubbing osd.3 2023-05-16T07:38:47.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.739+0000 7f2ce32da700 1 Processor -- start 2023-05-16T07:38:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.739+0000 7f2ce32da700 1 -- start start 2023-05-16T07:38:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce32da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc154470 0x7f2cdc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce32da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2cdc068160 0x7f2cdc0685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce32da700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0c94d0 con 0x7f2cdc154090 2023-05-16T07:38:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce32da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0cb7e0 con 0x7f2cdc068160 2023-05-16T07:38:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce32da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc068ac0 con 0x7f2cdc154470 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2cdc068160 0x7f2cdc0685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2cdc068160 0x7f2cdc0685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53870/0 (socket says 172.21.15.92:53870) 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 -- 172.21.15.92:0/193106303 learned_addr learned my addr 172.21.15.92:0/193106303 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce0875700 1 --2- 172.21.15.92:0/193106303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc154470 0x7f2cdc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 -- 172.21.15.92:0/193106303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc154470 msgr2=0x7f2cdc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 --2- 172.21.15.92:0/193106303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc154470 0x7f2cdc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 -- 172.21.15.92:0/193106303 >> v1:172.21.15.158:6789/0 conn(0x7f2cdc154090 legacy=0x7f2cdc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.740+0000 7f2ce1877700 1 -- 172.21.15.92:0/193106303 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0c88c0 con 0x7f2cdc068160 2023-05-16T07:38:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce1877700 1 --2- 172.21.15.92:0/193106303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc068160 0x7f2cdc0685d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7f2cd4006fd0 tx=0x7f2cd40090f0).ready entity=mon.0 client_cookie=6a2558554b00bcb1 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2cd3fff700 1 -- 172.21.15.92:0/193106303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400b600 con 0x7f2cdc068160 2023-05-16T07:38:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2cd3fff700 1 -- 172.21.15.92:0/193106303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd400b760 con 0x7f2cdc068160 2023-05-16T07:38:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2cd3fff700 1 -- 172.21.15.92:0/193106303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400bae0 con 0x7f2cdc068160 2023-05-16T07:38:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 -- 172.21.15.92:0/193106303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc068160 msgr2=0x7f2cdc0685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 --2- 172.21.15.92:0/193106303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc068160 0x7f2cdc0685d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7f2cd4006fd0 tx=0x7f2cd40090f0).stop 2023-05-16T07:38:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 -- 172.21.15.92:0/193106303 shutdown_connections 2023-05-16T07:38:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 --2- 172.21.15.92:0/193106303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc068160 0x7f2cdc0685d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 --2- 172.21.15.92:0/193106303 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc154470 0x7f2cdc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 -- 172.21.15.92:0/193106303 >> 172.21.15.92:0/193106303 conn(0x7f2cdc067920 msgr2=0x7f2cdc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 -- 172.21.15.92:0/193106303 shutdown_connections 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.741+0000 7f2ce32da700 1 -- 172.21.15.92:0/193106303 wait complete. 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.742+0000 7f2ce32da700 1 Processor -- start 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.742+0000 7f2ce32da700 1 -- start start 2023-05-16T07:38:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce32da700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc068160 0x7f2cdc14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce32da700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc154090 0x7f2cdc0aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce32da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc154470 0x7f2cdc0acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce32da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0ad460 con 0x7f2cdc154470 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce32da700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c2c30 con 0x7f2cdc154090 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce32da700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1512b0 con 0x7f2cdc068160 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.743+0000 7f2ce1877700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc154470 0x7f2cdc0acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc154470 0x7f2cdc0acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53878/0 (socket says 172.21.15.92:53878) 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 -- 172.21.15.92:0/4106742714 learned_addr learned my addr 172.21.15.92:0/4106742714 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1076700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc068160 0x7f2cdc14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce0875700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc154090 0x7f2cdc0aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 -- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc068160 msgr2=0x7f2cdc14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f2cdc068160 0x7f2cdc14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 -- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc154090 msgr2=0x7f2cdc0aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2cdc154090 0x7f2cdc0aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 -- 172.21.15.92:0/4106742714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4009370 con 0x7f2cdc154470 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce1877700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc154470 0x7f2cdc0acf70 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f2cdc06bb30 tx=0x7f2cd4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400bae0 con 0x7f2cdc154470 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd400bea0 con 0x7f2cdc154470 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc151590 con 0x7f2cdc154470 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd401c670 con 0x7f2cdc154470 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.744+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc0c2890 con 0x7f2cdc154470 2023-05-16T07:38:47.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.745+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cdc066730 con 0x7f2cdc154470 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.745+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2cd402c070 con 0x7f2cdc154470 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2cd17fa700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2cc005ed90 0x7f2cc0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2cd407bbc0 con 0x7f2cdc154470 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2cd17fa700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f2cc0065c40 0x7f2cc00680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 --> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2cc0068690 con 0x7f2cc0065c40 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd4080220 con 0x7f2cdc154470 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2ce1076700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2cc005ed90 0x7f2cc0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2ce0875700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f2cc0065c40 0x7f2cc00680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2ce0875700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f2cc0065c40 0x7f2cc00680b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.746+0000 7f2ce1076700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2cc005ed90 0x7f2cc0061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2cd8004290 tx=0x7f2cd8001560).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.747+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== osd.3 v2:172.21.15.92:6800/116542 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f2cc0068690 con 0x7f2cc0065c40 2023-05-16T07:38:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 --> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2cdc0c2da0 con 0x7f2cc0065c40 2023-05-16T07:38:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2cd17fa700 1 -- 172.21.15.92:0/4106742714 <== osd.3 v2:172.21.15.92:6800/116542 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2cdc0c2da0 con 0x7f2cc0065c40 2023-05-16T07:38:47.763 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:47.763 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:47.764 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f2cc0065c40 msgr2=0x7f2cc00680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:6800/116542,v1:172.21.15.92:6801/116542] conn(0x7f2cc0065c40 0x7f2cc00680b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2cc005ed90 msgr2=0x7f2cc0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f2cc005ed90 0x7f2cc0061250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2cd8004290 tx=0x7f2cd8001560).stop 2023-05-16T07:38:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc154470 msgr2=0x7f2cdc0acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 --2- 172.21.15.92:0/4106742714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cdc154470 0x7f2cdc0acf70 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f2cdc06bb30 tx=0x7f2cd4017000).stop 2023-05-16T07:38:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce1877700 1 -- 172.21.15.92:0/4106742714 reap_dead start 2023-05-16T07:38:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 shutdown_connections 2023-05-16T07:38:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 >> 172.21.15.92:0/4106742714 conn(0x7f2cdc067920 msgr2=0x7f2cdc0c1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 shutdown_connections 2023-05-16T07:38:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.760+0000 7f2ce32da700 1 -- 172.21.15.92:0/4106742714 wait complete. 2023-05-16T07:38:47.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-05-16T07:38:47.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.883+0000 7f7521caa700 1 Processor -- start 2023-05-16T07:38:47.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.883+0000 7f7521caa700 1 -- start start 2023-05-16T07:38:47.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.883+0000 7f7521caa700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f751c13a880 0x7f751c066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.883+0000 7f7521caa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0674c0 con 0x7f751c069c90 2023-05-16T07:38:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.883+0000 7f7521caa700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0bc910 con 0x7f751c0698b0 2023-05-16T07:38:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.883+0000 7f7521caa700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c1558f0 con 0x7f751c13a880 2023-05-16T07:38:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f751c069c90 0x7f751c13a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36402/0 (socket says 172.21.15.92:36402) 2023-05-16T07:38:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751affd700 1 -- 172.21.15.92:0/1712032968 learned_addr learned my addr 172.21.15.92:0/1712032968 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:47.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751bfff700 1 --2- 172.21.15.92:0/1712032968 >> v2:172.21.15.158:3300/0 conn(0x7f751c13a880 0x7f751c066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518259595 0 0) 0x7f751c0674c0 con 0x7f751c069c90 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75040033f0 con 0x7f751c069c90 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833477366 0 0) 0x7f751c0bc910 con 0x7f751c0698b0 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f751c0674c0 con 0x7f751c0698b0 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751bfff700 1 -- 172.21.15.92:0/1712032968 >> v1:172.21.15.158:6789/0 conn(0x7f751c0698b0 legacy=0x7f751c0659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751bfff700 1 -- 172.21.15.92:0/1712032968 >> v1:172.21.15.92:6789/0 conn(0x7f751c069c90 legacy=0x7f751c13a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2758288151 0 0) 0x7f75040033f0 con 0x7f751c069c90 2023-05-16T07:38:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751bfff700 1 -- 172.21.15.92:0/1712032968 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0bcbf0 con 0x7f751c13a880 2023-05-16T07:38:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751bfff700 1 --2- 172.21.15.92:0/1712032968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c13a880 0x7f751c066e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f7514009e60 tx=0x7f7514009780).ready entity=mon.1 client_cookie=dbabe6bef6c7e207 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751400e070 con 0x7f751c13a880 2023-05-16T07:38:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751400b7a0 con 0x7f751c13a880 2023-05-16T07:38:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.884+0000 7f751a7fc700 1 -- 172.21.15.92:0/1712032968 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510001470 con 0x7f751c13a880 2023-05-16T07:38:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 -- 172.21.15.92:0/1712032968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c13a880 msgr2=0x7f751c066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 --2- 172.21.15.92:0/1712032968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c13a880 0x7f751c066e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f7514009e60 tx=0x7f7514009780).stop 2023-05-16T07:38:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 -- 172.21.15.92:0/1712032968 shutdown_connections 2023-05-16T07:38:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 --2- 172.21.15.92:0/1712032968 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c13a880 0x7f751c066e50 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 -- 172.21.15.92:0/1712032968 >> 172.21.15.92:0/1712032968 conn(0x7f751c063630 msgr2=0x7f751c0c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 -- 172.21.15.92:0/1712032968 shutdown_connections 2023-05-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.885+0000 7f7521caa700 1 -- 172.21.15.92:0/1712032968 wait complete. 2023-05-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.886+0000 7f7521caa700 1 Processor -- start 2023-05-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.886+0000 7f7521caa700 1 -- start start 2023-05-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f7521caa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 0x7f751c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f7521caa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c069c90 0x7f751c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f7521caa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f751c13a880 0x7f751c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f7521caa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c156420 con 0x7f751c0698b0 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f7521caa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c158db0 con 0x7f751c069c90 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f7521caa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c157f20 con 0x7f751c13a880 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 0x7f751c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 0x7f751c157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53890/0 (socket says 172.21.15.92:53890) 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 -- 172.21.15.92:0/1847967838 learned_addr learned my addr 172.21.15.92:0/1847967838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751affd700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c069c90 0x7f751c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751bfff700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f751c13a880 0x7f751c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 -- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f751c13a880 msgr2=0x7f751c155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f751c13a880 0x7f751c155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 -- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c069c90 msgr2=0x7f751c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c069c90 0x7f751c155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 -- 172.21.15.92:0/1847967838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514009a20 con 0x7f751c0698b0 2023-05-16T07:38:47.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751affd700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c069c90 0x7f751c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.887+0000 7f751b7fe700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 0x7f751c157a30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f750c00c4c0 tx=0x7f750c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.888+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00f470 con 0x7f751c0698b0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.888+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750c00f5d0 con 0x7f751c0698b0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.888+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c159240 con 0x7f751c0698b0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.888+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00f970 con 0x7f751c0698b0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.888+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c158480 con 0x7f751c0698b0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.888+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c159090 con 0x7f751c0698b0 2023-05-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.889+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f750c019030 con 0x7f751c0698b0 2023-05-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.889+0000 7f75037fe700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f750805ed90 0x7f7508061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.889+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f750c0791b0 con 0x7f751c0698b0 2023-05-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.889+0000 7f751affd700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f750805ed90 0x7f7508061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.890+0000 7f751affd700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f750805ed90 0x7f7508061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f751c157820 tx=0x7f75100019d0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:47.893+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f750c04a630 con 0x7f751c0698b0 2023-05-16T07:38:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.057+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f751c06c350 con 0x7f750805ed90 2023-05-16T07:38:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.058+0000 7f75037fe700 1 -- 172.21.15.92:0/1847967838 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f751c06c350 con 0x7f750805ed90 2023-05-16T07:38:48.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f750805ed90 msgr2=0x7f7508061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f750805ed90 0x7f7508061250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f751c157820 tx=0x7f75100019d0).stop 2023-05-16T07:38:48.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 msgr2=0x7f751c157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 0x7f751c157a30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f750c00c4c0 tx=0x7f750c007c90).stop 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 shutdown_connections 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f750805ed90 0x7f7508061250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f751c13a880 0x7f751c155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f751c0698b0 0x7f751c157a30 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 --2- 172.21.15.92:0/1847967838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f751c069c90 0x7f751c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 >> 172.21.15.92:0/1847967838 conn(0x7f751c063630 msgr2=0x7f751c153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 shutdown_connections 2023-05-16T07:38:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.060+0000 7f7521caa700 1 -- 172.21.15.92:0/1847967838 wait complete. 2023-05-16T07:38:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2023-05-16T07:38:48.074 INFO:tasks.ceph:Scrubbing osd.4 2023-05-16T07:38:48.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.186+0000 7f1fdbc7b700 1 Processor -- start 2023-05-16T07:38:48.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.186+0000 7f1fdbc7b700 1 -- start start 2023-05-16T07:38:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fdbc7b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154090 0x7f1fd40aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fdbc7b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1fd4154470 0x7f1fd40a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fdbc7b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd40aa570 con 0x7f1fd40aa150 2023-05-16T07:38:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fdbc7b700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40c94d0 con 0x7f1fd4154470 2023-05-16T07:38:48.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fdbc7b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40cb7e0 con 0x7f1fd4154090 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fda218700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1fd40aa150 0x7f1fd4068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36416/0 (socket says 172.21.15.92:36416) 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fda218700 1 -- 172.21.15.92:0/2287139992 learned_addr learned my addr 172.21.15.92:0/2287139992 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd9216700 1 --2- 172.21.15.92:0/2287139992 >> v2:172.21.15.158:3300/0 conn(0x7f1fd4154470 0x7f1fd40a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd9a17700 1 --2- 172.21.15.92:0/2287139992 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154090 0x7f1fd40aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd8a15700 1 -- 172.21.15.92:0/2287139992 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693135250 0 0) 0x7f1fcc001490 con 0x7f1fd40aa150 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd8a15700 1 -- 172.21.15.92:0/2287139992 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc00033f0 con 0x7f1fd40aa150 2023-05-16T07:38:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2287139992 >> v2:172.21.15.158:3300/0 conn(0x7f1fd4154470 msgr2=0x7f1fd40a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd9a17700 1 --2- 172.21.15.92:0/2287139992 >> v2:172.21.15.158:3300/0 conn(0x7f1fd4154470 0x7f1fd40a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2287139992 >> v1:172.21.15.92:6789/0 conn(0x7f1fd40aa150 legacy=0x7f1fd4068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.187+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2287139992 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40c88c0 con 0x7f1fd4154090 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fd9a17700 1 --2- 172.21.15.92:0/2287139992 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154090 0x7f1fd40aaca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1fd0009760 tx=0x7f1fd0009060).ready entity=mon.2 client_cookie=6085a2d048f32b60 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fd8a15700 1 -- 172.21.15.92:0/2287139992 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000d4a0 con 0x7f1fd4154090 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fd8a15700 1 -- 172.21.15.92:0/2287139992 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd000d600 con 0x7f1fd4154090 2023-05-16T07:38:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fd8a15700 1 -- 172.21.15.92:0/2287139992 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000d980 con 0x7f1fd4154090 2023-05-16T07:38:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2287139992 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154090 msgr2=0x7f1fd40aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 --2- 172.21.15.92:0/2287139992 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154090 0x7f1fd40aaca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1fd0009760 tx=0x7f1fd0009060).stop 2023-05-16T07:38:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2287139992 shutdown_connections 2023-05-16T07:38:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 --2- 172.21.15.92:0/2287139992 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154090 0x7f1fd40aaca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 --2- 172.21.15.92:0/2287139992 >> v2:172.21.15.158:3300/0 conn(0x7f1fd4154470 0x7f1fd40a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2287139992 >> 172.21.15.92:0/2287139992 conn(0x7f1fd4067920 msgr2=0x7f1fd40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2287139992 shutdown_connections 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.188+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2287139992 wait complete. 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.189+0000 7f1fdbc7b700 1 Processor -- start 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.189+0000 7f1fdbc7b700 1 -- start start 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fdbc7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd40aa150 0x7f1fd40c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fdbc7b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fd4154090 0x7f1fd40c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fdbc7b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154470 0x7f1fd413e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fdbc7b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40c5510 con 0x7f1fd40aa150 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fdbc7b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40c5230 con 0x7f1fd4154090 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fdbc7b700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4155510 con 0x7f1fd4154470 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd40aa150 0x7f1fd40c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd40aa150 0x7f1fd40c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53902/0 (socket says 172.21.15.92:53902) 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2156615827 learned_addr learned my addr 172.21.15.92:0/2156615827 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9216700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fd4154090 0x7f1fd40c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fda218700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154470 0x7f1fd413e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154470 msgr2=0x7f1fd413e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154470 0x7f1fd413e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fd4154090 msgr2=0x7f1fd40c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1fd4154090 0x7f1fd40c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fd9a17700 1 -- 172.21.15.92:0/2156615827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd00092a0 con 0x7f1fd40aa150 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.190+0000 7f1fda218700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1fd4154470 0x7f1fd413e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.191+0000 7f1fd9a17700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd40aa150 0x7f1fd40c4bc0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f1fd0009d50 tx=0x7f1fd0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.191+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000dd40 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.191+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd000d770 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.191+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd41557f0 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.191+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000dea0 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.191+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd40c2bc0 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.192+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fd4066730 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.192+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1fd0023020 con 0x7f1fd40aa150 2023-05-16T07:38:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.192+0000 7f1fc9ffb700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1fb805ed90 0x7f1fb8061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.192+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1fd007b4d0 con 0x7f1fd40aa150 2023-05-16T07:38:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fc9ffb700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f1fb8065c40 0x7f1fb80680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fd9216700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1fb805ed90 0x7f1fb8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 --> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fb8068690 con 0x7f1fb8065c40 2023-05-16T07:38:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd007db40 con 0x7f1fd40aa150 2023-05-16T07:38:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fda218700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f1fb8065c40 0x7f1fb80680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fd9216700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1fb805ed90 0x7f1fb8061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1fd414cf00 tx=0x7f1fc4009060).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.193+0000 7f1fda218700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f1fb8065c40 0x7f1fb80680b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.194+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== osd.4 v2:172.21.15.158:6800/86891 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f1fb8068690 con 0x7f1fb8065c40 2023-05-16T07:38:48.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.207+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 --> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1fd406c350 con 0x7f1fb8065c40 2023-05-16T07:38:48.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.207+0000 7f1fc9ffb700 1 -- 172.21.15.92:0/2156615827 <== osd.4 v2:172.21.15.158:6800/86891 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1fd406c350 con 0x7f1fb8065c40 2023-05-16T07:38:48.211 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:48.211 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:48.211 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:48.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f1fb8065c40 msgr2=0x7f1fb80680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6800/86891,v1:172.21.15.158:6801/86891] conn(0x7f1fb8065c40 0x7f1fb80680b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1fb805ed90 msgr2=0x7f1fb8061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1fb805ed90 0x7f1fb8061250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1fd414cf00 tx=0x7f1fc4009060).stop 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd40aa150 msgr2=0x7f1fd40c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 --2- 172.21.15.92:0/2156615827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd40aa150 0x7f1fd40c4bc0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f1fd0009d50 tx=0x7f1fd0016000).stop 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fda218700 1 -- 172.21.15.92:0/2156615827 reap_dead start 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 shutdown_connections 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 >> 172.21.15.92:0/2156615827 conn(0x7f1fd4067920 msgr2=0x7f1fd41584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 shutdown_connections 2023-05-16T07:38:48.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.208+0000 7f1fdbc7b700 1 -- 172.21.15.92:0/2156615827 wait complete. 2023-05-16T07:38:48.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-05-16T07:38:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.330+0000 7f21331f5700 1 Processor -- start 2023-05-16T07:38:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.330+0000 7f21331f5700 1 -- start start 2023-05-16T07:38:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21331f5700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f212c069750 0x7f212c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21331f5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21331f5700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212c06a920 con 0x7f212c0c5f70 2023-05-16T07:38:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21331f5700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c0c65f0 con 0x7f212c069750 2023-05-16T07:38:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21331f5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c0c63e0 con 0x7f212c0aab40 2023-05-16T07:38:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21321f3700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f212c069750 0x7f212c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21319f2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21329f4700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f212c0c5f70 0x7f212c153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:40032/0 (socket says 172.21.15.92:40032) 2023-05-16T07:38:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21329f4700 1 -- 172.21.15.92:0/1567673537 learned_addr learned my addr 172.21.15.92:0/1567673537 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21311f1700 1 -- 172.21.15.92:0/1567673537 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850023874 0 0) 0x7f212c06a920 con 0x7f212c0c5f70 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21311f1700 1 -- 172.21.15.92:0/1567673537 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21180033f0 con 0x7f212c0c5f70 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21319f2700 1 -- 172.21.15.92:0/1567673537 >> v2:172.21.15.158:3300/0 conn(0x7f212c069750 msgr2=0x7f212c069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21319f2700 1 --2- 172.21.15.92:0/1567673537 >> v2:172.21.15.158:3300/0 conn(0x7f212c069750 0x7f212c069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21319f2700 1 -- 172.21.15.92:0/1567673537 >> v1:172.21.15.158:6789/0 conn(0x7f212c0c5f70 legacy=0x7f212c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21319f2700 1 -- 172.21.15.92:0/1567673537 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c0bfa30 con 0x7f212c0aab40 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21321f3700 1 --2- 172.21.15.92:0/1567673537 >> v2:172.21.15.158:3300/0 conn(0x7f212c069750 0x7f212c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.331+0000 7f21319f2700 1 --2- 172.21.15.92:0/1567673537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c5a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f211c009760 tx=0x7f211c009060).ready entity=mon.2 client_cookie=cbcc99659ba6a960 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21311f1700 1 -- 172.21.15.92:0/1567673537 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00d4a0 con 0x7f212c0aab40 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21311f1700 1 -- 172.21.15.92:0/1567673537 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c00d600 con 0x7f212c0aab40 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21311f1700 1 -- 172.21.15.92:0/1567673537 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00d980 con 0x7f212c0aab40 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 -- 172.21.15.92:0/1567673537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 msgr2=0x7f212c0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 --2- 172.21.15.92:0/1567673537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c5a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f211c009760 tx=0x7f211c009060).stop 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 -- 172.21.15.92:0/1567673537 shutdown_connections 2023-05-16T07:38:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 --2- 172.21.15.92:0/1567673537 >> v2:172.21.15.158:3300/0 conn(0x7f212c069750 0x7f212c069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 --2- 172.21.15.92:0/1567673537 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c5a80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 -- 172.21.15.92:0/1567673537 >> 172.21.15.92:0/1567673537 conn(0x7f212c0cb550 msgr2=0x7f212c0c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 -- 172.21.15.92:0/1567673537 shutdown_connections 2023-05-16T07:38:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.332+0000 7f21331f5700 1 -- 172.21.15.92:0/1567673537 wait complete. 2023-05-16T07:38:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.333+0000 7f21331f5700 1 Processor -- start 2023-05-16T07:38:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.333+0000 7f21331f5700 1 -- start start 2023-05-16T07:38:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21331f5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f212c069750 0x7f212c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21331f5700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21331f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 0x7f212c14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21331f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c14f110 con 0x7f212c0c5f70 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21331f5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c155f70 con 0x7f212c069750 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21331f5700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c1553a0 con 0x7f212c0aab40 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21329f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 0x7f212c14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21329f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 0x7f212c14e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53922/0 (socket says 172.21.15.92:53922) 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21329f4700 1 -- 172.21.15.92:0/3147049514 learned_addr learned my addr 172.21.15.92:0/3147049514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21319f2700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21321f3700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f212c069750 0x7f212c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.334+0000 7f21329f4700 1 -- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 msgr2=0x7f212c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21329f4700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21329f4700 1 -- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f212c069750 msgr2=0x7f212c156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21329f4700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f212c069750 0x7f212c156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21329f4700 1 -- 172.21.15.92:0/3147049514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21280092a0 con 0x7f212c0c5f70 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21329f4700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 0x7f212c14e950 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f2124005ad0 tx=0x7f21240015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124005d30 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124005e90 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c0092a0 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21240072e0 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.335+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c155640 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.336+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212c083040 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.336+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f2124010070 con 0x7f212c0c5f70 2023-05-16T07:38:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.336+0000 7f21227fc700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f211005edc0 0x7f2110061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.336+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2124079330 con 0x7f212c0c5f70 2023-05-16T07:38:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.336+0000 7f21321f3700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f211005edc0 0x7f2110061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.337+0000 7f21321f3700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f211005edc0 0x7f2110061280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2128009dc0 tx=0x7f2128009060).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.340+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f212404a7b0 con 0x7f212c0c5f70 2023-05-16T07:38:48.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.502+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f212c047150 con 0x7f211005edc0 2023-05-16T07:38:48.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.503+0000 7f21227fc700 1 -- 172.21.15.92:0/3147049514 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f212c047150 con 0x7f211005edc0 2023-05-16T07:38:48.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f211005edc0 msgr2=0x7f2110061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f211005edc0 0x7f2110061280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2128009dc0 tx=0x7f2128009060).stop 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 msgr2=0x7f212c14e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 0x7f212c14e950 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f2124005ad0 tx=0x7f21240015b0).stop 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 shutdown_connections 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f212c0c5f70 0x7f212c14e950 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f212c069750 0x7f212c156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f211005edc0 0x7f2110061280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 --2- 172.21.15.92:0/3147049514 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f212c0aab40 0x7f212c0c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 >> 172.21.15.92:0/3147049514 conn(0x7f212c0cb550 msgr2=0x7f212c150910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 shutdown_connections 2023-05-16T07:38:48.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.505+0000 7f21331f5700 1 -- 172.21.15.92:0/3147049514 wait complete. 2023-05-16T07:38:48.510 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2023-05-16T07:38:48.520 INFO:tasks.ceph:Scrubbing osd.5 2023-05-16T07:38:48.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:48.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.632+0000 7f704d494700 1 Processor -- start 2023-05-16T07:38:48.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.632+0000 7f704d494700 1 -- start start 2023-05-16T07:38:48.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f704d494700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f7048154470 0x7f70480aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f704d494700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480c94d0 con 0x7f7048154090 2023-05-16T07:38:48.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f704d494700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480cb7e0 con 0x7f7048068160 2023-05-16T07:38:48.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f704d494700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480c88c0 con 0x7f7048154470 2023-05-16T07:38:48.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f7046ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7048154090 0x7f70480a9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36442/0 (socket says 172.21.15.92:36442) 2023-05-16T07:38:48.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f7046ffd700 1 -- 172.21.15.92:0/2990711562 learned_addr learned my addr 172.21.15.92:0/2990711562 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:48.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f70467fc700 1 --2- 172.21.15.92:0/2990711562 >> v2:172.21.15.158:3300/0 conn(0x7f7048154470 0x7f70480aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f7045ffb700 1 -- 172.21.15.92:0/2990711562 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384607167 0 0) 0x7f70480c94d0 con 0x7f7048154090 2023-05-16T07:38:48.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.633+0000 7f7045ffb700 1 -- 172.21.15.92:0/2990711562 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70300033f0 con 0x7f7048154090 2023-05-16T07:38:48.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f70467fc700 1 -- 172.21.15.92:0/2990711562 >> v1:172.21.15.158:6789/0 conn(0x7f7048068160 legacy=0x7f70480c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f7045ffb700 1 -- 172.21.15.92:0/2990711562 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504638603 0 0) 0x7f70300033f0 con 0x7f7048154090 2023-05-16T07:38:48.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f70467fc700 1 -- 172.21.15.92:0/2990711562 >> v1:172.21.15.92:6789/0 conn(0x7f7048154090 legacy=0x7f70480a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f70467fc700 1 -- 172.21.15.92:0/2990711562 --> v2:172.21.15.158:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480c8ba0 con 0x7f7048154470 2023-05-16T07:38:48.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f70467fc700 1 --2- 172.21.15.92:0/2990711562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048154470 0x7f70480aaca0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f703c009760 tx=0x7f703c009060).ready entity=mon.1 client_cookie=191b05349579ba14 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f7045ffb700 1 -- 172.21.15.92:0/2990711562 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400017f0 con 0x7f7048154470 2023-05-16T07:38:48.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f7045ffb700 1 -- 172.21.15.92:0/2990711562 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c009130 con 0x7f7048154470 2023-05-16T07:38:48.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f7045ffb700 1 -- 172.21.15.92:0/2990711562 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c012620 con 0x7f7048154470 2023-05-16T07:38:48.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 -- 172.21.15.92:0/2990711562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048154470 msgr2=0x7f70480aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 --2- 172.21.15.92:0/2990711562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048154470 0x7f70480aaca0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f703c009760 tx=0x7f703c009060).stop 2023-05-16T07:38:48.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 -- 172.21.15.92:0/2990711562 shutdown_connections 2023-05-16T07:38:48.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 --2- 172.21.15.92:0/2990711562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048154470 0x7f70480aaca0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 -- 172.21.15.92:0/2990711562 >> 172.21.15.92:0/2990711562 conn(0x7f7048067920 msgr2=0x7f70480cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 -- 172.21.15.92:0/2990711562 shutdown_connections 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.634+0000 7f704d494700 1 -- 172.21.15.92:0/2990711562 wait complete. 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 Processor -- start 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 -- start start 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048068160 0x7f70480a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7048154090 0x7f7048168650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7048154470 0x7f7048164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048167c00 con 0x7f7048154090 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481650a0 con 0x7f7048068160 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f704d494700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048165380 con 0x7f7048154470 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.636+0000 7f70467fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7048154090 0x7f7048168650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7048154090 0x7f7048168650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53934/0 (socket says 172.21.15.92:53934) 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 -- 172.21.15.92:0/805305958 learned_addr learned my addr 172.21.15.92:0/805305958 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f7046ffd700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048068160 0x7f70480a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70477fe700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7048154470 0x7f7048164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 -- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7048154470 msgr2=0x7f7048164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f7048154470 0x7f7048164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 -- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048068160 msgr2=0x7f70480a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7048068160 0x7f70480a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 -- 172.21.15.92:0/805305958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0092a0 con 0x7f7048154090 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f70467fc700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7048154090 0x7f7048168650 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f704806bb30 tx=0x7f703c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c006910 con 0x7f7048154090 2023-05-16T07:38:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c011950 con 0x7f7048154090 2023-05-16T07:38:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7048167e50 con 0x7f7048154090 2023-05-16T07:38:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c011cf0 con 0x7f7048154090 2023-05-16T07:38:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.637+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7048153360 con 0x7f7048154090 2023-05-16T07:38:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.638+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7048066730 con 0x7f7048154090 2023-05-16T07:38:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.638+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f703c01a030 con 0x7f7048154090 2023-05-16T07:38:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f702effd700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f703405ed90 0x7f7034061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f703c07be70 con 0x7f7048154090 2023-05-16T07:38:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f7046ffd700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f703405ed90 0x7f7034061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f702effd700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7f7034065c40 0x7f70340680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 --> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7034068690 con 0x7f7034065c40 2023-05-16T07:38:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f703c07c270 con 0x7f7048154090 2023-05-16T07:38:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f70477fe700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7f7034065c40 0x7f70340680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f7046ffd700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f703405ed90 0x7f7034061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7040000f80 tx=0x7f70400018e0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.639+0000 7f70477fe700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7f7034065c40 0x7f70340680b0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.640+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== osd.5 v2:172.21.15.158:6816/86917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f7034068690 con 0x7f7034065c40 2023-05-16T07:38:48.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.653+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 --> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f704806c350 con 0x7f7034065c40 2023-05-16T07:38:48.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.653+0000 7f702effd700 1 -- 172.21.15.92:0/805305958 <== osd.5 v2:172.21.15.158:6816/86917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f704806c350 con 0x7f7034065c40 2023-05-16T07:38:48.657 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:48.657 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:48.657 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7f7034065c40 msgr2=0x7f70340680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6816/86917,v1:172.21.15.158:6817/86917] conn(0x7f7034065c40 0x7f70340680b0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f703405ed90 msgr2=0x7f7034061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f703405ed90 0x7f7034061250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7040000f80 tx=0x7f70400018e0).stop 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7048154090 msgr2=0x7f7048168650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 --2- 172.21.15.92:0/805305958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7048154090 0x7f7048168650 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f704806bb30 tx=0x7f703c015000).stop 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f70477fe700 1 -- 172.21.15.92:0/805305958 reap_dead start 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 shutdown_connections 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 >> 172.21.15.92:0/805305958 conn(0x7f7048067920 msgr2=0x7f70480acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 shutdown_connections 2023-05-16T07:38:48.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.654+0000 7f704d494700 1 -- 172.21.15.92:0/805305958 wait complete. 2023-05-16T07:38:48.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-05-16T07:38:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.776+0000 7fb2433b0700 1 Processor -- start 2023-05-16T07:38:48.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.777+0000 7fb2433b0700 1 -- start start 2023-05-16T07:38:48.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.777+0000 7fb2433b0700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb23c063e30 0x7fb23c069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb2433b0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb2433b0700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c062960 con 0x7fb23c154040 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb2433b0700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c154420 con 0x7fb23c063e30 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb2433b0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c069fb0 con 0x7fb23c0bf4f0 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24094b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb23c063e30 0x7fb23c069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24094b700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fb23c063e30 0x7fb23c069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:57582/0 (socket says 172.21.15.92:57582) 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c062df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:46516/0 (socket says 172.21.15.92:46516) 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24094b700 1 -- 172.21.15.92:0/658358898 learned_addr learned my addr 172.21.15.92:0/658358898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24114c700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7fb23c154040 0x7fb23c063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:40058/0 (socket says 172.21.15.92:40058) 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 -- 172.21.15.92:0/658358898 >> v2:172.21.15.158:3300/0 conn(0x7fb23c063e30 msgr2=0x7fb23c069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 --2- 172.21.15.92:0/658358898 >> v2:172.21.15.158:3300/0 conn(0x7fb23c063e30 0x7fb23c069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 -- 172.21.15.92:0/658358898 >> v1:172.21.15.158:6789/0 conn(0x7fb23c154040 legacy=0x7fb23c063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 -- 172.21.15.92:0/658358898 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c062350 con 0x7fb23c0bf4f0 2023-05-16T07:38:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb24194d700 1 --2- 172.21.15.92:0/658358898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c062df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fb22c009e60 tx=0x7fb22c009780).ready entity=mon.2 client_cookie=83eb91bfadb2034 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb23bfff700 1 -- 172.21.15.92:0/658358898 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00e070 con 0x7fb23c0bf4f0 2023-05-16T07:38:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb23bfff700 1 -- 172.21.15.92:0/658358898 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c00bb60 con 0x7fb23c0bf4f0 2023-05-16T07:38:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.778+0000 7fb23bfff700 1 -- 172.21.15.92:0/658358898 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c012410 con 0x7fb23c0bf4f0 2023-05-16T07:38:48.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 -- 172.21.15.92:0/658358898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 msgr2=0x7fb23c062df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 --2- 172.21.15.92:0/658358898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c062df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fb22c009e60 tx=0x7fb22c009780).stop 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 -- 172.21.15.92:0/658358898 shutdown_connections 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 --2- 172.21.15.92:0/658358898 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c062df0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 --2- 172.21.15.92:0/658358898 >> v2:172.21.15.158:3300/0 conn(0x7fb23c063e30 0x7fb23c069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 -- 172.21.15.92:0/658358898 >> 172.21.15.92:0/658358898 conn(0x7fb23c0c8aa0 msgr2=0x7fb23c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 -- 172.21.15.92:0/658358898 shutdown_connections 2023-05-16T07:38:48.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.779+0000 7fb2433b0700 1 -- 172.21.15.92:0/658358898 wait complete. 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.780+0000 7fb2433b0700 1 Processor -- start 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.780+0000 7fb2433b0700 1 -- start start 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb2433b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 0x7fb23c0c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb2433b0700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb2433b0700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb23c154040 0x7fb23c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb2433b0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c167ef0 con 0x7fb23c063e30 2023-05-16T07:38:48.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb2433b0700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c168170 con 0x7fb23c154040 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb2433b0700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c168450 con 0x7fb23c0bf4f0 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 0x7fb23c0c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 0x7fb23c0c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53948/0 (socket says 172.21.15.92:53948) 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 -- 172.21.15.92:0/1475758914 learned_addr learned my addr 172.21.15.92:0/1475758914 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24094b700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24194d700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb23c154040 0x7fb23c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 -- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 msgr2=0x7fb23c167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 -- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb23c154040 msgr2=0x7fb23c0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb23c154040 0x7fb23c0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.781+0000 7fb24114c700 1 -- 172.21.15.92:0/1475758914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c009a20 con 0x7fb23c063e30 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.782+0000 7fb24114c700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 0x7fb23c0c2830 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7fb234007fd0 tx=0x7fb234007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.782+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23400ce10 con 0x7fb23c063e30 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.782+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2340072e0 con 0x7fb23c063e30 2023-05-16T07:38:48.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.782+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c168730 con 0x7fb23c063e30 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.782+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2340056e0 con 0x7fb23c063e30 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.782+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c0acf30 con 0x7fb23c063e30 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.783+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c0a9260 con 0x7fb23c063e30 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.783+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fb23401a060 con 0x7fb23c063e30 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.783+0000 7fb2397fa700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb22805ed90 0x7fb228061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.783+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb234079240 con 0x7fb23c063e30 2023-05-16T07:38:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.784+0000 7fb24094b700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb22805ed90 0x7fb228061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.784+0000 7fb24094b700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb22805ed90 0x7fb228061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb23c0a9210 tx=0x7fb2300090f0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.787+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fb23c0a9260 con 0x7fb23c063e30 2023-05-16T07:38:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.949+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb23c0ad150 con 0x7fb22805ed90 2023-05-16T07:38:48.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.950+0000 7fb2397fa700 1 -- 172.21.15.92:0/1475758914 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb230009830 con 0x7fb22805ed90 2023-05-16T07:38:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb22805ed90 msgr2=0x7fb228061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb22805ed90 0x7fb228061250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb23c0a9210 tx=0x7fb2300090f0).stop 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 msgr2=0x7fb23c0c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 0x7fb23c0c2830 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7fb234007fd0 tx=0x7fb234007df0).stop 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 shutdown_connections 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb23c154040 0x7fb23c0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb23c063e30 0x7fb23c0c2830 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fb22805ed90 0x7fb228061250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 --2- 172.21.15.92:0/1475758914 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fb23c0bf4f0 0x7fb23c167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 >> 172.21.15.92:0/1475758914 conn(0x7fb23c0c8aa0 msgr2=0x7fb23c14f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 shutdown_connections 2023-05-16T07:38:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:48.952+0000 7fb2433b0700 1 -- 172.21.15.92:0/1475758914 wait complete. 2023-05-16T07:38:48.957 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2023-05-16T07:38:48.967 INFO:tasks.ceph:Scrubbing osd.6 2023-05-16T07:38:48.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.078+0000 7fe77acd6700 1 Processor -- start 2023-05-16T07:38:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.078+0000 7fe77acd6700 1 -- start start 2023-05-16T07:38:49.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.079+0000 7fe77acd6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 0x7fe7740c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.079+0000 7fe77acd6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740c6660 con 0x7fe7740c61f0 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.079+0000 7fe77acd6700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77406a140 con 0x7fe774153f90 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.079+0000 7fe77acd6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe774069f60 con 0x7fe7740c1160 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.079+0000 7fe77a4d5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7740c61f0 0x7fe774154370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36474/0 (socket says 172.21.15.92:36474) 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.079+0000 7fe77a4d5700 1 -- 172.21.15.92:0/2399170917 learned_addr learned my addr 172.21.15.92:0/2399170917 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe7794d3700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 0x7fe7740c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132414517 0 0) 0x7fe7740c6660 con 0x7fe7740c61f0 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe75c0033f0 con 0x7fe7740c61f0 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098753689 0 0) 0x7fe77406a140 con 0x7fe774153f90 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 --> v1:172.21.15.158:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7740c6660 con 0x7fe774153f90 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286551858 0 0) 0x7fe75c0033f0 con 0x7fe7740c61f0 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 msgr2=0x7fe7740c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 0x7fe7740c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 >> v1:172.21.15.158:6789/0 conn(0x7fe774153f90 legacy=0x7fe7740a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77406a420 con 0x7fe7740c61f0 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fe76c001470 con 0x7fe7740c61f0 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 >> v1:172.21.15.92:6789/0 conn(0x7fe7740c61f0 legacy=0x7fe774154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 0x7fe75c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe75c003fb0 0x7fe75c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe75c004b80 0x7fe75c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774069f60 con 0x7fe75c004b80 2023-05-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c00b990 con 0x7fe75c003fb0 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c00bb50 con 0x7fe7740c1160 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe75c004b80 0x7fe75c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe7794d3700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe75c003fb0 0x7fe75c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe779cd4700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 0x7fe75c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 -- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 msgr2=0x7fe75c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe7740c1160 0x7fe75c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 -- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe75c003fb0 msgr2=0x7fe75c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe75c003fb0 0x7fe75c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 -- 172.21.15.92:0/2399170917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0033f0 con 0x7fe75c004b80 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.080+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe75c004b80 0x7fe75c005000 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7fe76c00c110 tx=0x7fe76c001e10).ready entity=mon.0 client_cookie=e7459972f7a7f39b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00ca40 con 0x7fe75c004b80 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c00cba0 con 0x7fe75c004b80 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe778cd2700 1 -- 172.21.15.92:0/2399170917 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0070d0 con 0x7fe75c004b80 2023-05-16T07:38:49.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77acd6700 1 -- 172.21.15.92:0/2399170917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe75c004b80 msgr2=0x7fe75c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77acd6700 1 --2- 172.21.15.92:0/2399170917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe75c004b80 0x7fe75c005000 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7fe76c00c110 tx=0x7fe76c001e10).stop 2023-05-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77a4d5700 1 -- 172.21.15.92:0/2399170917 reap_dead start 2023-05-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77acd6700 1 -- 172.21.15.92:0/2399170917 shutdown_connections 2023-05-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77acd6700 1 -- 172.21.15.92:0/2399170917 >> 172.21.15.92:0/2399170917 conn(0x7fe7740cb600 msgr2=0x7fe7740c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77acd6700 1 -- 172.21.15.92:0/2399170917 shutdown_connections 2023-05-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.081+0000 7fe77acd6700 1 -- 172.21.15.92:0/2399170917 wait complete. 2023-05-16T07:38:49.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.082+0000 7fe77acd6700 1 Processor -- start 2023-05-16T07:38:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.082+0000 7fe77acd6700 1 -- start start 2023-05-16T07:38:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77acd6700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe774161860 0x7fe7741642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77acd6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe774154450 0x7fe7740aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77acd6700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7740ab490 0x7fe7740ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77acd6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740adca0 con 0x7fe774154450 2023-05-16T07:38:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77acd6700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740ab910 con 0x7fe7740ab490 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77acd6700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe774165230 con 0x7fe774161860 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe774154450 0x7fe7740aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe774154450 0x7fe7740aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53970/0 (socket says 172.21.15.92:53970) 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 -- 172.21.15.92:0/3165224155 learned_addr learned my addr 172.21.15.92:0/3165224155 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe779cd4700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe774161860 0x7fe7741642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7740ab490 0x7fe7740ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 -- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe774161860 msgr2=0x7fe7741642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe774161860 0x7fe7741642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 -- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7740ab490 msgr2=0x7fe7740ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe7740ab490 0x7fe7740ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 -- 172.21.15.92:0/3165224155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7741548d0 con 0x7fe774154450 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.083+0000 7fe7794d3700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe774154450 0x7fe7740aafa0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7fe7680119f0 tx=0x7fe768009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.084+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800f760 con 0x7fe774154450 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.084+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800f8c0 con 0x7fe774154450 2023-05-16T07:38:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.084+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe774164990 con 0x7fe774154450 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.084+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800fc40 con 0x7fe774154450 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.084+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe774164f50 con 0x7fe774154450 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.084+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe774083120 con 0x7fe774154450 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.085+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe76800fda0 con 0x7fe774154450 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.085+0000 7fe771ffb700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe76005ed90 0x7fe760061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.085+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe76807a950 con 0x7fe774154450 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe779cd4700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe76005ed90 0x7fe760061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe771ffb700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fe760065c40 0x7fe7600680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 --> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe760068690 con 0x7fe760065c40 2023-05-16T07:38:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76807ad50 con 0x7fe774154450 2023-05-16T07:38:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fe760065c40 0x7fe7600680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe779cd4700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe76005ed90 0x7fe760061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe774155c50 tx=0x7fe764007f30).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.086+0000 7fe77a4d5700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fe760065c40 0x7fe7600680b0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.087+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== osd.6 v2:172.21.15.158:6824/86922 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fe760068690 con 0x7fe760065c40 2023-05-16T07:38:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.100+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 --> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe7740adde0 con 0x7fe760065c40 2023-05-16T07:38:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.100+0000 7fe771ffb700 1 -- 172.21.15.92:0/3165224155 <== osd.6 v2:172.21.15.158:6824/86922 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe76c00cd10 con 0x7fe760065c40 2023-05-16T07:38:49.103 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:49.104 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.100+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fe760065c40 msgr2=0x7fe7600680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.100+0000 7fe77acd6700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6824/86922,v1:172.21.15.158:6825/86922] conn(0x7fe760065c40 0x7fe7600680b0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe76005ed90 msgr2=0x7fe760061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe76005ed90 0x7fe760061250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe774155c50 tx=0x7fe764007f30).stop 2023-05-16T07:38:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe774154450 msgr2=0x7fe7740aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 --2- 172.21.15.92:0/3165224155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe774154450 0x7fe7740aafa0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7fe7680119f0 tx=0x7fe768009060).stop 2023-05-16T07:38:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77a4d5700 1 -- 172.21.15.92:0/3165224155 reap_dead start 2023-05-16T07:38:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 shutdown_connections 2023-05-16T07:38:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 >> 172.21.15.92:0/3165224155 conn(0x7fe7740cb600 msgr2=0x7fe774152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 shutdown_connections 2023-05-16T07:38:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.101+0000 7fe77acd6700 1 -- 172.21.15.92:0/3165224155 wait complete. 2023-05-16T07:38:49.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-05-16T07:38:49.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.223+0000 7f78e2e58700 1 Processor -- start 2023-05-16T07:38:49.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.223+0000 7f78e2e58700 1 -- start start 2023-05-16T07:38:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e2e58700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc154170 0x7f78dc0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e2e58700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 0x7f78dc0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e2e58700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0c0660 con 0x7f78dc0696a0 2023-05-16T07:38:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e2e58700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0c1b00 con 0x7f78dc154550 2023-05-16T07:38:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e2e58700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0bfd30 con 0x7f78dc154170 2023-05-16T07:38:49.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78dbfff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 0x7f78dc0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e0bf4700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc154170 0x7f78dc0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78dbfff700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 0x7f78dc0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.92:57618/0 (socket says 172.21.15.92:57618) 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e13f5700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f78dc0696a0 0x7f78dc157e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:40096/0 (socket says 172.21.15.92:40096) 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78dbfff700 1 -- 172.21.15.92:0/2532115917 learned_addr learned my addr 172.21.15.92:0/2532115917 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e0bf4700 1 -- 172.21.15.92:0/2532115917 >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 msgr2=0x7f78dc0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e0bf4700 1 --2- 172.21.15.92:0/2532115917 >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 0x7f78dc0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e0bf4700 1 -- 172.21.15.92:0/2532115917 >> v1:172.21.15.158:6789/0 conn(0x7f78dc0696a0 legacy=0x7f78dc157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78db7fe700 1 -- 172.21.15.92:0/2532115917 <== mon.1 v1:172.21.15.158:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754867288 0 0) 0x7f78dc0c0660 con 0x7f78dc0696a0 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e0bf4700 1 -- 172.21.15.92:0/2532115917 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0c0010 con 0x7f78dc154170 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78dbfff700 1 --2- 172.21.15.92:0/2532115917 >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 0x7f78dc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.224+0000 7f78e0bf4700 1 --2- 172.21.15.92:0/2532115917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc154170 0x7f78dc0c5c70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f78d4009760 tx=0x7f78d4009060).ready entity=mon.2 client_cookie=5488c93bee853ca7 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78db7fe700 1 -- 172.21.15.92:0/2532115917 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d40090d0 con 0x7f78dc154170 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78db7fe700 1 -- 172.21.15.92:0/2532115917 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d400f510 con 0x7f78dc154170 2023-05-16T07:38:49.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78db7fe700 1 -- 172.21.15.92:0/2532115917 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d400f890 con 0x7f78dc154170 2023-05-16T07:38:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 -- 172.21.15.92:0/2532115917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc154170 msgr2=0x7f78dc0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 --2- 172.21.15.92:0/2532115917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc154170 0x7f78dc0c5c70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f78d4009760 tx=0x7f78d4009060).stop 2023-05-16T07:38:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 -- 172.21.15.92:0/2532115917 shutdown_connections 2023-05-16T07:38:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 --2- 172.21.15.92:0/2532115917 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc154170 0x7f78dc0c5c70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 --2- 172.21.15.92:0/2532115917 >> v2:172.21.15.158:3300/0 conn(0x7f78dc154550 0x7f78dc0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 -- 172.21.15.92:0/2532115917 >> 172.21.15.92:0/2532115917 conn(0x7f78dc0cb700 msgr2=0x7f78dc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 -- 172.21.15.92:0/2532115917 shutdown_connections 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.225+0000 7f78e2e58700 1 -- 172.21.15.92:0/2532115917 wait complete. 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.226+0000 7f78e2e58700 1 Processor -- start 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.226+0000 7f78e2e58700 1 -- start start 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e2e58700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc0696a0 0x7f78dc143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e2e58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 0x7f78dc0c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e2e58700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78dc154550 0x7f78dc0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e2e58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0adf70 con 0x7f78dc154170 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e2e58700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0b0530 con 0x7f78dc154550 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e2e58700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc149160 con 0x7f78dc0696a0 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78dbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 0x7f78dc0c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78dbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 0x7f78dc0c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53986/0 (socket says 172.21.15.92:53986) 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78dbfff700 1 -- 172.21.15.92:0/3696855530 learned_addr learned my addr 172.21.15.92:0/3696855530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e0bf4700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc0696a0 0x7f78dc143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.227+0000 7f78e13f5700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78dc154550 0x7f78dc0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78dbfff700 1 -- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc0696a0 msgr2=0x7f78dc143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78dbfff700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc0696a0 0x7f78dc143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78dbfff700 1 -- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78dc154550 msgr2=0x7f78dc0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78dbfff700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78dc154550 0x7f78dc0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78dbfff700 1 -- 172.21.15.92:0/3696855530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc009ae0 con 0x7f78dc154170 2023-05-16T07:38:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78e13f5700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78dc154550 0x7f78dc0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78dbfff700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 0x7f78dc0c1d40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f78dc154930 tx=0x7f78cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00ca10 con 0x7f78dc154170 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc00f040 con 0x7f78dc154170 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d40092a0 con 0x7f78dc154170 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00cd80 con 0x7f78dc154170 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.228+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc0b0780 con 0x7f78dc154170 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.229+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc149440 con 0x7f78dc154170 2023-05-16T07:38:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.229+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f78cc01b030 con 0x7f78dc154170 2023-05-16T07:38:49.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.230+0000 7f78d8ff9700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f78bc05ed60 0x7f78bc061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.230+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f78cc079910 con 0x7f78dc154170 2023-05-16T07:38:49.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.230+0000 7f78e0bf4700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f78bc05ed60 0x7f78bc061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.230+0000 7f78e0bf4700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f78bc05ed60 0x7f78bc061220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f78dc0b04e0 tx=0x7f78d4017000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.234+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f78cc04ae40 con 0x7f78dc154170 2023-05-16T07:38:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.396+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f78dc0666c0 con 0x7f78bc05ed60 2023-05-16T07:38:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.396+0000 7f78d8ff9700 1 -- 172.21.15.92:0/3696855530 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f78dc0666c0 con 0x7f78bc05ed60 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f78bc05ed60 msgr2=0x7f78bc061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f78bc05ed60 0x7f78bc061220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f78dc0b04e0 tx=0x7f78d4017000).stop 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 msgr2=0x7f78dc0c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 0x7f78dc0c1d40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f78dc154930 tx=0x7f78cc00b040).stop 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 shutdown_connections 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f78bc05ed60 0x7f78bc061220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f78dc0696a0 0x7f78dc143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f78dc154550 0x7f78dc0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 --2- 172.21.15.92:0/3696855530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78dc154170 0x7f78dc0c1d40 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 >> 172.21.15.92:0/3696855530 conn(0x7f78dc0cb700 msgr2=0x7f78dc0acf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 shutdown_connections 2023-05-16T07:38:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.398+0000 7f78e2e58700 1 -- 172.21.15.92:0/3696855530 wait complete. 2023-05-16T07:38:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2023-05-16T07:38:49.413 INFO:tasks.ceph:Scrubbing osd.7 2023-05-16T07:38:49.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-05-16T07:38:49.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.524+0000 7f1be09d2700 1 Processor -- start 2023-05-16T07:38:49.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.525+0000 7f1be09d2700 1 -- start start 2023-05-16T07:38:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.525+0000 7f1be09d2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1bdc0c4c80 0x7f1bdc152e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.525+0000 7f1be09d2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bdc063690 con 0x7f1bdc0622b0 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.525+0000 7f1be09d2700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bdc153330 con 0x7f1bdc14f990 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.525+0000 7f1be09d2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bdc0c5100 con 0x7f1bdc0c4c80 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd9d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1bdc0c4c80 0x7f1bdc152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bdad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1bdc0622b0 0x7f1bdc063b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36510/0 (socket says 172.21.15.92:36510) 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bdad9d700 1 -- 172.21.15.92:0/3235423908 learned_addr learned my addr 172.21.15.92:0/3235423908 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd9d9b700 1 -- 172.21.15.92:0/3235423908 >> v1:172.21.15.158:6789/0 conn(0x7f1bdc14f990 legacy=0x7f1bdc0c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd9d9b700 1 -- 172.21.15.92:0/3235423908 >> v1:172.21.15.92:6789/0 conn(0x7f1bdc0622b0 legacy=0x7f1bdc063b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd9d9b700 1 -- 172.21.15.92:0/3235423908 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc14fd70 con 0x7f1bdc0c4c80 2023-05-16T07:38:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd959a700 1 -- 172.21.15.92:0/3235423908 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011948154 0 0) 0x7f1bdc063690 con 0x7f1bdc0622b0 2023-05-16T07:38:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd9d9b700 1 --2- 172.21.15.92:0/3235423908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0c4c80 0x7f1bdc152e40 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f1bd0009760 tx=0x7f1bd0009060).ready entity=mon.0 client_cookie=21e87fca4699f478 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd959a700 1 -- 172.21.15.92:0/3235423908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000b520 con 0x7f1bdc0c4c80 2023-05-16T07:38:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd959a700 1 -- 172.21.15.92:0/3235423908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd000b680 con 0x7f1bdc0c4c80 2023-05-16T07:38:49.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.526+0000 7f1bd959a700 1 -- 172.21.15.92:0/3235423908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000ba00 con 0x7f1bdc0c4c80 2023-05-16T07:38:49.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 -- 172.21.15.92:0/3235423908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0c4c80 msgr2=0x7f1bdc152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 --2- 172.21.15.92:0/3235423908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0c4c80 0x7f1bdc152e40 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f1bd0009760 tx=0x7f1bd0009060).stop 2023-05-16T07:38:49.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 -- 172.21.15.92:0/3235423908 shutdown_connections 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 --2- 172.21.15.92:0/3235423908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0c4c80 0x7f1bdc152e40 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 -- 172.21.15.92:0/3235423908 >> 172.21.15.92:0/3235423908 conn(0x7f1bdc05e820 msgr2=0x7f1bdc0c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 -- 172.21.15.92:0/3235423908 shutdown_connections 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.527+0000 7f1be09d2700 1 -- 172.21.15.92:0/3235423908 wait complete. 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.528+0000 7f1be09d2700 1 Processor -- start 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.528+0000 7f1be09d2700 1 -- start start 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0622b0 0x7f1bdc1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1bdc0c4c80 0x7f1bdc155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1bdc14f990 0x7f1bdc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0691c0 con 0x7f1bdc0622b0 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0aea70 con 0x7f1bdc14f990 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc155280 con 0x7f1bdc0c4c80 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0622b0 0x7f1bdc1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0622b0 0x7f1bdc1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53998/0 (socket says 172.21.15.92:53998) 2023-05-16T07:38:49.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 -- 172.21.15.92:0/327740982 learned_addr learned my addr 172.21.15.92:0/327740982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 -- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1bdc0c4c80 msgr2=0x7f1bdc155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f1bdc0c4c80 0x7f1bdc155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 -- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1bdc14f990 msgr2=0x7f1bdc068930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1bdc14f990 0x7f1bdc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 -- 172.21.15.92:0/327740982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd00092a0 con 0x7f1bdc0622b0 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bda59c700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0622b0 0x7f1bdc1563c0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f1bcc00b460 tx=0x7f1bcc006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc012410 con 0x7f1bdc0622b0 2023-05-16T07:38:49.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc012570 con 0x7f1bdc0622b0 2023-05-16T07:38:49.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc1546b0 con 0x7f1bdc0622b0 2023-05-16T07:38:49.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc012930 con 0x7f1bdc0622b0 2023-05-16T07:38:49.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.529+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc154f10 con 0x7f1bdc0622b0 2023-05-16T07:38:49.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.530+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1bdc06c350 con 0x7f1bdc0622b0 2023-05-16T07:38:49.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f1bcc00f030 con 0x7f1bdc0622b0 2023-05-16T07:38:49.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bca7fc700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1bbc05ed90 0x7f1bbc061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1bcc079970 con 0x7f1bdc0622b0 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bca7fc700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7f1bbc065c40 0x7f1bbc0680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 --> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bbc068690 con 0x7f1bbc065c40 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bcc07e220 con 0x7f1bdc0622b0 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bd9d9b700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1bbc05ed90 0x7f1bbc061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bdad9d700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7f1bbc065c40 0x7f1bbc0680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.531+0000 7f1bd9d9b700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1bbc05ed90 0x7f1bbc061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1bdc0ab420 tx=0x7f1bd0017000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.532+0000 7f1bdad9d700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7f1bbc065c40 0x7f1bbc0680b0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.532+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== osd.7 v2:172.21.15.158:6808/86892 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f1bbc068690 con 0x7f1bbc065c40 2023-05-16T07:38:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.545+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 --> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1bdc155560 con 0x7f1bbc065c40 2023-05-16T07:38:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.546+0000 7f1bca7fc700 1 -- 172.21.15.92:0/327740982 <== osd.7 v2:172.21.15.158:6808/86892 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1bdc155560 con 0x7f1bbc065c40 2023-05-16T07:38:49.549 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-05-16T07:38:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-05-16T07:38:49.550 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-05-16T07:38:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.546+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7f1bbc065c40 msgr2=0x7f1bbc0680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.546+0000 7f1be09d2700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6808/86892,v1:172.21.15.158:6809/86892] conn(0x7f1bbc065c40 0x7f1bbc0680b0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.546+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1bbc05ed90 msgr2=0x7f1bbc061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f1bbc05ed90 0x7f1bbc061250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1bdc0ab420 tx=0x7f1bd0017000).stop 2023-05-16T07:38:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0622b0 msgr2=0x7f1bdc1563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 --2- 172.21.15.92:0/327740982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bdc0622b0 0x7f1bdc1563c0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f1bcc00b460 tx=0x7f1bcc006f20).stop 2023-05-16T07:38:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1bdad9d700 1 -- 172.21.15.92:0/327740982 reap_dead start 2023-05-16T07:38:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 shutdown_connections 2023-05-16T07:38:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 >> 172.21.15.92:0/327740982 conn(0x7f1bdc05e820 msgr2=0x7f1bdc151790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 shutdown_connections 2023-05-16T07:38:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.547+0000 7f1be09d2700 1 -- 172.21.15.92:0/327740982 wait complete. 2023-05-16T07:38:49.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-05-16T07:38:49.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.669+0000 7f51748fa700 1 Processor -- start 2023-05-16T07:38:49.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.669+0000 7f51748fa700 1 -- start start 2023-05-16T07:38:49.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.670+0000 7f51748fa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 0x7f51700bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.670+0000 7f51748fa700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f51700bd780 0x7f51700c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.670+0000 7f51748fa700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51700bdbb0 con 0x7f517013a5a0 2023-05-16T07:38:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.670+0000 7f51748fa700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5170154b80 con 0x7f51700bd780 2023-05-16T07:38:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.670+0000 7f51748fa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5170154e60 con 0x7f5170068060 2023-05-16T07:38:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 0x7f51700bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516ed9d700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f51700bd780 0x7f51700c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516e59c700 1 --1- >> v1:172.21.15.158:6789/0 conn(0x7f517013a5a0 0x7f5170068600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.158:6789/0 says I am v1:172.21.15.92:40124/0 (socket says 172.21.15.92:40124) 2023-05-16T07:38:49.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 0x7f51700bd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.158:3301/0 says I am v2:172.21.15.92:46584/0 (socket says 172.21.15.92:46584) 2023-05-16T07:38:49.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516e59c700 1 -- 172.21.15.92:0/2343914884 learned_addr learned my addr 172.21.15.92:0/2343914884 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 -- 172.21.15.92:0/2343914884 >> v2:172.21.15.158:3300/0 conn(0x7f51700bd780 msgr2=0x7f51700c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 --2- 172.21.15.92:0/2343914884 >> v2:172.21.15.158:3300/0 conn(0x7f51700bd780 0x7f51700c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 -- 172.21.15.92:0/2343914884 >> v1:172.21.15.158:6789/0 conn(0x7f517013a5a0 legacy=0x7f5170068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 -- 172.21.15.92:0/2343914884 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170155140 con 0x7f5170068060 2023-05-16T07:38:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516ed9d700 1 --2- 172.21.15.92:0/2343914884 >> v2:172.21.15.158:3300/0 conn(0x7f51700bd780 0x7f51700c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-05-16T07:38:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516dd9b700 1 --2- 172.21.15.92:0/2343914884 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 0x7f51700bd290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f5158009760 tx=0x7f5158009060).ready entity=mon.2 client_cookie=246bfed0ca9d026d server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516d59a700 1 -- 172.21.15.92:0/2343914884 <== mon.2 v2:172.21.15.158:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800f510 con 0x7f5170068060 2023-05-16T07:38:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516d59a700 1 -- 172.21.15.92:0/2343914884 <== mon.2 v2:172.21.15.158:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515800f670 con 0x7f5170068060 2023-05-16T07:38:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.671+0000 7f516d59a700 1 -- 172.21.15.92:0/2343914884 <== mon.2 v2:172.21.15.158:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800f9f0 con 0x7f5170068060 2023-05-16T07:38:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 -- 172.21.15.92:0/2343914884 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 msgr2=0x7f51700bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 --2- 172.21.15.92:0/2343914884 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 0x7f51700bd290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f5158009760 tx=0x7f5158009060).stop 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 -- 172.21.15.92:0/2343914884 shutdown_connections 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 --2- 172.21.15.92:0/2343914884 >> v2:172.21.15.158:3300/0 conn(0x7f51700bd780 0x7f51700c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 --2- 172.21.15.92:0/2343914884 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f5170068060 0x7f51700bd290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 -- 172.21.15.92:0/2343914884 >> 172.21.15.92:0/2343914884 conn(0x7f51700c4430 msgr2=0x7f51700c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 -- 172.21.15.92:0/2343914884 shutdown_connections 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.672+0000 7f51748fa700 1 -- 172.21.15.92:0/2343914884 wait complete. 2023-05-16T07:38:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.673+0000 7f51748fa700 1 Processor -- start 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.673+0000 7f51748fa700 1 -- start start 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 0x7f517014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51700bd780 0x7f51701365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f517013a5a0 0x7f517014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517014caa0 con 0x7f5170068060 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517014be90 con 0x7f517013a5a0 2023-05-16T07:38:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517014c170 con 0x7f51700bd780 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 0x7f517014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 0x7f517014e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54014/0 (socket says 172.21.15.92:54014) 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 -- 172.21.15.92:0/41239755 learned_addr learned my addr 172.21.15.92:0/41239755 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516dd9b700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51700bd780 0x7f51701365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516ed9d700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f517013a5a0 0x7f517014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 -- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51700bd780 msgr2=0x7f51701365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51700bd780 0x7f51701365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 -- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f517013a5a0 msgr2=0x7f517014cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f517013a5a0 0x7f517014cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 -- 172.21.15.92:0/41239755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680092a0 con 0x7f5170068060 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516dd9b700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51700bd780 0x7f51701365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f516e59c700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 0x7f517014e360 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f516000c840 tx=0x7f5160001610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160005d30 con 0x7f5170068060 2023-05-16T07:38:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5160005e90 con 0x7f5170068060 2023-05-16T07:38:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51580092a0 con 0x7f5170068060 2023-05-16T07:38:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160007320 con 0x7f5170068060 2023-05-16T07:38:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.674+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517014c730 con 0x7f5170068060 2023-05-16T07:38:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.675+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517014df10 con 0x7f5170068060 2023-05-16T07:38:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.675+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f5160010030 con 0x7f5170068060 2023-05-16T07:38:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.676+0000 7f51667fc700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f515005ed90 0x7f5150061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.676+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5160079b60 con 0x7f5170068060 2023-05-16T07:38:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.676+0000 7f516dd9b700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f515005ed90 0x7f5150061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.676+0000 7f516dd9b700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f515005ed90 0x7f5150061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5158009760 tx=0x7f5158017000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.680+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f516004af80 con 0x7f5170068060 2023-05-16T07:38:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.841+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f517014cbe0 con 0x7f515005ed90 2023-05-16T07:38:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.842+0000 7f51667fc700 1 -- 172.21.15.92:0/41239755 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f515801f070 con 0x7f515005ed90 2023-05-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f515005ed90 msgr2=0x7f5150061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f515005ed90 0x7f5150061250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5158009760 tx=0x7f5158017000).stop 2023-05-16T07:38:49.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 msgr2=0x7f517014e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 0x7f517014e360 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f516000c840 tx=0x7f5160001610).stop 2023-05-16T07:38:49.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 shutdown_connections 2023-05-16T07:38:49.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f517013a5a0 0x7f517014cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5170068060 0x7f517014e360 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f515005ed90 0x7f5150061250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 --2- 172.21.15.92:0/41239755 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f51700bd780 0x7f51701365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 >> 172.21.15.92:0/41239755 conn(0x7f51700c4430 msgr2=0x7f5170137510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 shutdown_connections 2023-05-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.844+0000 7f51748fa700 1 -- 172.21.15.92:0/41239755 wait complete. 2023-05-16T07:38:49.850 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2023-05-16T07:38:49.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:38:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.966+0000 7efdef59e700 1 Processor -- start 2023-05-16T07:38:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.966+0000 7efdef59e700 1 -- start start 2023-05-16T07:38:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdef59e700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7efdf0150d80 0x7efdf00c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdef59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 0x7efdf0151610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdef59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf00bc5f0 con 0x7efdf00c1550 2023-05-16T07:38:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdef59e700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf0066670 con 0x7efdf0150d80 2023-05-16T07:38:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdef59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf00bbfe0 con 0x7efdf0151160 2023-05-16T07:38:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efde7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efdf00c1550 0x7efdf0065bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36540/0 (socket says 172.21.15.92:36540) 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efde7fff700 1 -- 172.21.15.92:0/322374346 learned_addr learned my addr 172.21.15.92:0/322374346 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdee59c700 1 --2- 172.21.15.92:0/322374346 >> v2:172.21.15.158:3300/0 conn(0x7efdf0150d80 0x7efdf00c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdeed9d700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 0x7efdf0151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145853114 0 0) 0x7efdf00bc5f0 con 0x7efdf00c1550 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efddc0033f0 con 0x7efdf00c1550 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088514554 0 0) 0x7efddc0033f0 con 0x7efdf00c1550 2023-05-16T07:38:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 msgr2=0x7efdf0151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 0x7efdf0151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 >> v2:172.21.15.158:3300/0 conn(0x7efdf0150d80 msgr2=0x7efdf00c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 --2- 172.21.15.92:0/322374346 >> v2:172.21.15.158:3300/0 conn(0x7efdf0150d80 0x7efdf00c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.967+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf0066390 con 0x7efdf00c1550 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7efdd8001470 con 0x7efdf00c1550 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 >> v1:172.21.15.92:6789/0 conn(0x7efdf00c1550 legacy=0x7efdf0065bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efddc0038e0 0x7efddc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 0x7efddc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efddc004b80 0x7efddc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf0066670 con 0x7efddc004b80 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf00bbfe0 con 0x7efddc0038e0 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc00b8e0 con 0x7efdf0151160 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efddc004b80 0x7efddc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdee59c700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efddc0038e0 0x7efddc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efde7fff700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 0x7efddc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 -- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 msgr2=0x7efddc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf0151160 0x7efddc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 -- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efddc0038e0 msgr2=0x7efddc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efddc0038e0 0x7efddc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 -- 172.21.15.92:0/322374346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0033f0 con 0x7efddc004b80 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdeed9d700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efddc004b80 0x7efddc005000 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7efde800e1c0 tx=0x7efde800b040).ready entity=mon.0 client_cookie=c9bfd413fafce937 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8009ac0 con 0x7efddc004b80 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde800f040 con 0x7efddc004b80 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.968+0000 7efdedd9b700 1 -- 172.21.15.92:0/322374346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde800eca0 con 0x7efddc004b80 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdef59e700 1 -- 172.21.15.92:0/322374346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efddc004b80 msgr2=0x7efddc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdef59e700 1 --2- 172.21.15.92:0/322374346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efddc004b80 0x7efddc005000 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7efde800e1c0 tx=0x7efde800b040).stop 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdeed9d700 1 -- 172.21.15.92:0/322374346 reap_dead start 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdef59e700 1 -- 172.21.15.92:0/322374346 shutdown_connections 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdef59e700 1 -- 172.21.15.92:0/322374346 >> 172.21.15.92:0/322374346 conn(0x7efdf0069940 msgr2=0x7efdf00c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdef59e700 1 -- 172.21.15.92:0/322374346 shutdown_connections 2023-05-16T07:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.969+0000 7efdef59e700 1 -- 172.21.15.92:0/322374346 wait complete. 2023-05-16T07:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.970+0000 7efdef59e700 1 Processor -- start 2023-05-16T07:38:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.970+0000 7efdef59e700 1 -- start start 2023-05-16T07:38:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.970+0000 7efdef59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 0x7efdf0135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.970+0000 7efdef59e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efdf00c09f0 0x7efdf00be590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdef59e700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf00bea80 0x7efdf00c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdef59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf00c0e70 con 0x7efdf00be1b0 2023-05-16T07:38:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdef59e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf013fc20 con 0x7efdf00c09f0 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdef59e700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf00bf0a0 con 0x7efdf00bea80 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 0x7efdf0135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 0x7efdf0135de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54034/0 (socket says 172.21.15.92:54034) 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 -- 172.21.15.92:0/866085345 learned_addr learned my addr 172.21.15.92:0/866085345 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdeed9d700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf00bea80 0x7efdf00c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efde7fff700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efdf00c09f0 0x7efdf00be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 -- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf00bea80 msgr2=0x7efdf00c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf00bea80 0x7efdf00c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 -- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efdf00c09f0 msgr2=0x7efdf00be590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efdf00c09f0 0x7efdf00be590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 -- 172.21.15.92:0/866085345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00bf380 con 0x7efdf00be1b0 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efde7fff700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efdf00c09f0 0x7efdf00be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdeed9d700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf00bea80 0x7efdf00c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdee59c700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 0x7efdf0135de0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7efde00119f0 tx=0x7efde0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000f8e0 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde000fa40 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf013fe70 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000fdc0 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.971+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf0138a30 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.972+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf0140310 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.972+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7efde0024070 con 0x7efdf00be1b0 2023-05-16T07:38:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.973+0000 7efde67fc700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efdd005ed90 0x7efdd0061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:38:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.973+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efde007a620 con 0x7efdf00be1b0 2023-05-16T07:38:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.973+0000 7efde7fff700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efdd005ed90 0x7efdd0061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:38:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.973+0000 7efde7fff700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efdd005ed90 0x7efdd0061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7efdd8000e10 tx=0x7efdd8009040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:38:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:49.977+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7efde004baa0 con 0x7efdf00be1b0 2023-05-16T07:38:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.137+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdf0047150 con 0x7efdd005ed90 2023-05-16T07:38:50.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.140+0000 7efde67fc700 1 -- 172.21.15.92:0/866085345 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70908 (secure 0 0 0) 0x7efdf0047150 con 0x7efdd005ed90 2023-05-16T07:38:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:38:50.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efdd005ed90 msgr2=0x7efdd0061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efdd005ed90 0x7efdd0061250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7efdd8000e10 tx=0x7efdd8009040).stop 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 msgr2=0x7efdf0135de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 0x7efdf0135de0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7efde00119f0 tx=0x7efde0009060).stop 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 shutdown_connections 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7efdd005ed90 0x7efdd0061250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7efdf00bea80 0x7efdf00c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:50.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf00be1b0 0x7efdf0135de0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:50.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 --2- 172.21.15.92:0/866085345 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efdf00c09f0 0x7efdf00be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:38:50.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 >> 172.21.15.92:0/866085345 conn(0x7efdf0069940 msgr2=0x7efdf0145990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:38:50.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 shutdown_connections 2023-05-16T07:38:50.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:38:50.142+0000 7efdef59e700 1 -- 172.21.15.92:0/866085345 wait complete. 2023-05-16T07:38:50.148 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:38:50.157 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-05-16T07:38:48.805650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":196288,"kb_used_data":17216,"kb_used_omap":0,"kb_used_meta":179008,"kb_avail":754778432,"statfs":{"total":773094113280,"available":772893114368,"internally_reserved":0,"allocated":17629184,"data_stored":9450896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":183304192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12242,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.170733"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.624797+0000","last_change":"2023-05-16T07:02:22.259510+0000","last_active":"2023-05-16T07:38:41.624797+0000","last_peered":"2023-05-16T07:38:41.624797+0000","last_clean":"2023-05-16T07:38:41.624797+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:41.624797+0000","last_undegraded":"2023-05-16T07:38:41.624797+0000","last_fullsized":"2023-05-16T07:38:41.624797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.714539+0000","last_change":"2023-05-16T07:02:21.778771+0000","last_active":"2023-05-16T07:38:41.714539+0000","last_peered":"2023-05-16T07:38:41.714539+0000","last_clean":"2023-05-16T07:38:41.714539+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:41.714539+0000","last_undegraded":"2023-05-16T07:38:41.714539+0000","last_fullsized":"2023-05-16T07:38:41.714539+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.214226+0000","last_change":"2023-05-16T07:02:21.776034+0000","last_active":"2023-05-16T07:38:41.214226+0000","last_peered":"2023-05-16T07:38:41.214226+0000","last_clean":"2023-05-16T07:38:41.214226+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:41.214226+0000","last_undegraded":"2023-05-16T07:38:41.214226+0000","last_fullsized":"2023-05-16T07:38:41.214226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.715951+0000","last_change":"2023-05-16T07:02:21.777020+0000","last_active":"2023-05-16T07:38:41.715951+0000","last_peered":"2023-05-16T07:38:41.715951+0000","last_clean":"2023-05-16T07:38:41.715951+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:41.715951+0000","last_undegraded":"2023-05-16T07:38:41.715951+0000","last_fullsized":"2023-05-16T07:38:41.715951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.498207+0000","last_change":"2023-05-16T07:02:20.343728+0000","last_active":"2023-05-16T07:38:41.498207+0000","last_peered":"2023-05-16T07:38:41.498207+0000","last_clean":"2023-05-16T07:38:41.498207+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:41.498207+0000","last_undegraded":"2023-05-16T07:38:41.498207+0000","last_fullsized":"2023-05-16T07:38:41.498207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:18.719326+0000","last_clean_scrub_stamp":"2023-05-16T07:02:18.719326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.498325+0000","last_change":"2023-05-16T07:02:21.775370+0000","last_active":"2023-05-16T07:38:41.498325+0000","last_peered":"2023-05-16T07:38:41.498325+0000","last_clean":"2023-05-16T07:38:41.498325+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:41.498325+0000","last_undegraded":"2023-05-16T07:38:41.498325+0000","last_fullsized":"2023-05-16T07:38:41.498325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.497389+0000","last_change":"2023-05-16T07:02:22.259995+0000","last_active":"2023-05-16T07:38:41.497389+0000","last_peered":"2023-05-16T07:38:41.497389+0000","last_clean":"2023-05-16T07:38:41.497389+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:41.497389+0000","last_undegraded":"2023-05-16T07:38:41.497389+0000","last_fullsized":"2023-05-16T07:38:41.497389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:41.625331+0000","last_change":"2023-05-16T07:02:21.774949+0000","last_active":"2023-05-16T07:38:41.625331+0000","last_peered":"2023-05-16T07:38:41.625331+0000","last_clean":"2023-05-16T07:38:41.625331+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:41.625331+0000","last_undegraded":"2023-05-16T07:38:41.625331+0000","last_fullsized":"2023-05-16T07:38:41.625331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-05-16T07:38:40.984303+0000","last_change":"2023-05-16T07:02:21.775145+0000","last_active":"2023-05-16T07:38:40.984303+0000","last_peered":"2023-05-16T07:38:40.984303+0000","last_clean":"2023-05-16T07:38:40.984303+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:40.984303+0000","last_undegraded":"2023-05-16T07:38:40.984303+0000","last_fullsized":"2023-05-16T07:38:40.984303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:02:20.761371+0000","last_clean_scrub_stamp":"2023-05-16T07:02:20.761371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575277,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25392,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":23232,"kb_avail":94346448,"statfs":{"total":96636764160,"available":96610762752,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23789568},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51200000000000001}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.82399999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.755,"15min":0.79800000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73399999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.498},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":4.0970000000000004}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.71499999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.315},"max":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.75},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.82699999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575277,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25200,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":23040,"kb_avail":94346640,"statfs":{"total":96636764160,"available":96610959360,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23592960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64900000000000002,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.622},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.77500000000000002,"5min":1.5,"15min":1.5},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73599999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":1.486,"15min":1.486},"last":0.53900000000000003}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":1.532,"15min":1.532},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61799999999999999}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.878,"15min":0.878},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.316},"max":{"1min":0.873,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575278,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20784,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":18624,"kb_avail":94351056,"statfs":{"total":96636764160,"available":96615481344,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19070976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58899999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.64100000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41899999999999998},"min":{"1min":0.26100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.309},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.67900000000000005},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.872,"15min":0.877},"last":0.5},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.054,"15min":1.054},"last":0.54200000000000004}]},{"osd":6,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.998},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575278,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20464,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":18304,"kb_avail":94351376,"statfs":{"total":96636764160,"available":96615809024,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18743296},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.313},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.223,"15min":0.223},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.30399999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.433,"15min":0.432},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.246}]},{"osd":4,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.98699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.90400000000000003,"15min":1.0549999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":1.0680000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.68000000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.095},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":1.0369999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90300000000000002,"5min":0.91500000000000004,"15min":1.03},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90400000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.71799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575278,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22128,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94349712,"statfs":{"total":96636764160,"available":96614105088,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.314},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.378}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.51500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575277,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28976,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94342864,"statfs":{"total":96636764160,"available":96607092736,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.58999999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.32700000000000001}]},{"osd":1,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.72999999999999998},"last":0.311}]},{"osd":2,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.255},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.94299999999999995,"15min":1.014},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":0.84099999999999997,"15min":1.024},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.97799999999999998,"15min":1.05},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.97299999999999998},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.91600000000000004,"15min":1.0389999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.55800000000000005}]},{"osd":7,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":1.0640000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":1.012},"last":0.504}]}]},{"osd":4,"up_from":13,"seq":55834575277,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":25136,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":22976,"kb_avail":94346704,"statfs":{"total":96636764160,"available":96611024896,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23527424},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.71799999999999997,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.753,"5min":1.359,"15min":1.359},"last":0.64500000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":2,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.81399999999999995},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.755,"15min":0.755},"last":0.623},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575277,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.436},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.496},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:38:50.161 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.162 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.162 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.162 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.162 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-05-16T07:02:18.719326+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=18, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.163 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.163 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.163 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.163 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-05-16T07:02:20.761371+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=2, tm_sec=20, tm_wday=1, tm_yday=136, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=16, tm_hour=7, tm_min=38, tm_sec=45, tm_wday=1, tm_yday=136, tm_isdst=0) 2023-05-16T07:38:50.164 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-05-16T07:39:10.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-16T07:39:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.291+0000 7fe926d27700 1 Processor -- start 2023-05-16T07:39:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.291+0000 7fe926d27700 1 -- start start 2023-05-16T07:39:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.291+0000 7fe926d27700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe920069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.291+0000 7fe926d27700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe920062960 con 0x7fe9200bf4f0 2023-05-16T07:39:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe926d27700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe920069fb0 con 0x7fe920063e30 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe926d27700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe920154420 con 0x7fe920154040 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe9200bf4f0 0x7fe920063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36558/0 (socket says 172.21.15.92:36558) 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91ffff700 1 -- 172.21.15.92:0/3284374130 learned_addr learned my addr 172.21.15.92:0/3284374130 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507603647 0 0) 0x7fe920062960 con 0x7fe9200bf4f0 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe924ac3700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe920069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90c0033f0 con 0x7fe9200bf4f0 2023-05-16T07:39:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3019735957 0 0) 0x7fe90c0033f0 con 0x7fe9200bf4f0 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 msgr2=0x7fe920069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe920069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 >> v1:172.21.15.158:6789/0 conn(0x7fe920063e30 legacy=0x7fe9200bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920062350 con 0x7fe9200bf4f0 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fe908001470 con 0x7fe9200bf4f0 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 >> v1:172.21.15.92:6789/0 conn(0x7fe9200bf4f0 legacy=0x7fe920063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c003910 0x7fe90c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c004690 0x7fe90c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe90c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00b8e0 con 0x7fe90c003910 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00ba50 con 0x7fe90c004690 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00bc10 con 0x7fe920154040 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.292+0000 7fe924ac3700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c003910 0x7fe90c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe91ffff700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c004690 0x7fe90c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe9252c4700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe90c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe924ac3700 1 -- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 msgr2=0x7fe90c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe924ac3700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe90c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe924ac3700 1 -- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c004690 msgr2=0x7fe90c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe924ac3700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c004690 0x7fe90c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe924ac3700 1 -- 172.21.15.92:0/3284374130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0033f0 con 0x7fe90c003910 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe9252c4700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe920154040 0x7fe90c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:39:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe91ffff700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c004690 0x7fe90c004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe924ac3700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c003910 0x7fe90c003d30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7fe910009a60 tx=0x7fe910009060).ready entity=mon.0 client_cookie=4b6be58383a3f679 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910009ea0 con 0x7fe90c003910 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910009430 con 0x7fe90c003910 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe91f7fe700 1 -- 172.21.15.92:0/3284374130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91000e870 con 0x7fe90c003910 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe926d27700 1 -- 172.21.15.92:0/3284374130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c003910 msgr2=0x7fe90c003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe926d27700 1 --2- 172.21.15.92:0/3284374130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c003910 0x7fe90c003d30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7fe910009a60 tx=0x7fe910009060).stop 2023-05-16T07:39:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe9252c4700 1 -- 172.21.15.92:0/3284374130 reap_dead start 2023-05-16T07:39:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe926d27700 1 -- 172.21.15.92:0/3284374130 shutdown_connections 2023-05-16T07:39:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.293+0000 7fe926d27700 1 -- 172.21.15.92:0/3284374130 >> 172.21.15.92:0/3284374130 conn(0x7fe9200c8aa0 msgr2=0x7fe9200c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:39:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.294+0000 7fe926d27700 1 -- 172.21.15.92:0/3284374130 shutdown_connections 2023-05-16T07:39:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.294+0000 7fe926d27700 1 -- 172.21.15.92:0/3284374130 wait complete. 2023-05-16T07:39:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.295+0000 7fe926d27700 1 Processor -- start 2023-05-16T07:39:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.295+0000 7fe926d27700 1 -- start start 2023-05-16T07:39:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c003910 0x7fe9200ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe92014d780 0x7fe920156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 0x7fe920156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920157170 con 0x7fe92014dc00 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920154420 con 0x7fe90c003910 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9200aaaa0 con 0x7fe92014d780 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 0x7fe920156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 0x7fe920156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54056/0 (socket says 172.21.15.92:54056) 2023-05-16T07:39:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 -- 172.21.15.92:0/898603135 learned_addr learned my addr 172.21.15.92:0/898603135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe91ffff700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe92014d780 0x7fe920156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe924ac3700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c003910 0x7fe9200ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 -- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe92014d780 msgr2=0x7fe920156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe92014d780 0x7fe920156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 -- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c003910 msgr2=0x7fe9200ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c003910 0x7fe9200ac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 -- 172.21.15.92:0/898603135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920154670 con 0x7fe92014dc00 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe91ffff700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe92014d780 0x7fe920156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe9252c4700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 0x7fe920156c80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fe92006bfb0 tx=0x7fe918009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918010520 con 0x7fe92014dc00 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe918010680 con 0x7fe92014dc00 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe920154950 con 0x7fe92014dc00 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.296+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918010a20 con 0x7fe92014dc00 2023-05-16T07:39:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.297+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9200ac890 con 0x7fe92014dc00 2023-05-16T07:39:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.297+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe920067940 con 0x7fe92014dc00 2023-05-16T07:39:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.298+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fe918014430 con 0x7fe92014dc00 2023-05-16T07:39:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.298+0000 7fe91cff9700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe90005ed90 0x7fe900061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.298+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe918079950 con 0x7fe92014dc00 2023-05-16T07:39:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.298+0000 7fe924ac3700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe90005ed90 0x7fe900061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.299+0000 7fe924ac3700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe90005ed90 0x7fe900061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fe910006d50 tx=0x7fe91001a000).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.302+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fe9180498f0 con 0x7fe92014dc00 2023-05-16T07:39:10.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:39:10.360 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:39:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.468+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 --> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe92006c350 con 0x7fe90005ed90 2023-05-16T07:39:10.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.472+0000 7fe91cff9700 1 -- 172.21.15.92:0/898603135 <== mgr.8314 v2:172.21.15.158:6832/95623 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70794 (secure 0 0 0) 0x7fe92006c350 con 0x7fe90005ed90 2023-05-16T07:39:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:39:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.473+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe90005ed90 msgr2=0x7fe900061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.473+0000 7fe926d27700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe90005ed90 0x7fe900061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fe910006d50 tx=0x7fe91001a000).stop 2023-05-16T07:39:10.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.473+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 msgr2=0x7fe920156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.473+0000 7fe926d27700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 0x7fe920156c80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fe92006bfb0 tx=0x7fe918009040).stop 2023-05-16T07:39:10.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 shutdown_connections 2023-05-16T07:39:10.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe92014dc00 0x7fe920156c80 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe90c003910 0x7fe9200ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fe90005ed90 0x7fe900061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 --2- 172.21.15.92:0/898603135 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fe92014d780 0x7fe920156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 >> 172.21.15.92:0/898603135 conn(0x7fe9200c8aa0 msgr2=0x7fe9200c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:39:10.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 shutdown_connections 2023-05-16T07:39:10.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.474+0000 7fe926d27700 1 -- 172.21.15.92:0/898603135 wait complete. 2023-05-16T07:39:10.482 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-05-16T07:39:10.490 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-05-16T07:39:08.810076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":196288,"kb_used_data":17216,"kb_used_omap":0,"kb_used_meta":179008,"kb_avail":754778432,"statfs":{"total":773094113280,"available":772893114368,"internally_reserved":0,"allocated":17629184,"data_stored":9450896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":183304192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12242,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.170733"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:48.024632+0000","last_change":"2023-05-16T07:38:48.024632+0000","last_active":"2023-05-16T07:38:48.024632+0000","last_peered":"2023-05-16T07:38:48.024632+0000","last_clean":"2023-05-16T07:38:48.024632+0000","last_became_active":"2023-05-16T07:02:22.259220+0000","last_became_peered":"2023-05-16T07:02:22.259220+0000","last_unstale":"2023-05-16T07:38:48.024632+0000","last_undegraded":"2023-05-16T07:38:48.024632+0000","last_fullsized":"2023-05-16T07:38:48.024632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:48.024539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:48.024539+0000","last_clean_scrub_stamp":"2023-05-16T07:38:48.024539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:50.254883+0000","last_change":"2023-05-16T07:38:50.254883+0000","last_active":"2023-05-16T07:38:50.254883+0000","last_peered":"2023-05-16T07:38:50.254883+0000","last_clean":"2023-05-16T07:38:50.254883+0000","last_became_active":"2023-05-16T07:02:21.778468+0000","last_became_peered":"2023-05-16T07:02:21.778468+0000","last_unstale":"2023-05-16T07:38:50.254883+0000","last_undegraded":"2023-05-16T07:38:50.254883+0000","last_fullsized":"2023-05-16T07:38:50.254883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:50.254760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:50.254760+0000","last_clean_scrub_stamp":"2023-05-16T07:38:50.254760+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:47.776562+0000","last_change":"2023-05-16T07:38:47.776562+0000","last_active":"2023-05-16T07:38:47.776562+0000","last_peered":"2023-05-16T07:38:47.776562+0000","last_clean":"2023-05-16T07:38:47.776562+0000","last_became_active":"2023-05-16T07:02:21.775699+0000","last_became_peered":"2023-05-16T07:02:21.775699+0000","last_unstale":"2023-05-16T07:38:47.776562+0000","last_undegraded":"2023-05-16T07:38:47.776562+0000","last_fullsized":"2023-05-16T07:38:47.776562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:47.776466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:47.776466+0000","last_clean_scrub_stamp":"2023-05-16T07:38:47.776466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:52.259501+0000","last_change":"2023-05-16T07:38:52.259501+0000","last_active":"2023-05-16T07:38:52.259501+0000","last_peered":"2023-05-16T07:38:52.259501+0000","last_clean":"2023-05-16T07:38:52.259501+0000","last_became_active":"2023-05-16T07:02:21.776742+0000","last_became_peered":"2023-05-16T07:02:21.776742+0000","last_unstale":"2023-05-16T07:38:52.259501+0000","last_undegraded":"2023-05-16T07:38:52.259501+0000","last_fullsized":"2023-05-16T07:38:52.259501+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:52.259388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:52.259388+0000","last_clean_scrub_stamp":"2023-05-16T07:38:52.259388+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:54.146695+0000","last_change":"2023-05-16T07:38:54.146695+0000","last_active":"2023-05-16T07:38:54.146695+0000","last_peered":"2023-05-16T07:38:54.146695+0000","last_clean":"2023-05-16T07:38:54.146695+0000","last_became_active":"2023-05-16T07:02:20.343443+0000","last_became_peered":"2023-05-16T07:02:20.343443+0000","last_unstale":"2023-05-16T07:38:54.146695+0000","last_undegraded":"2023-05-16T07:38:54.146695+0000","last_fullsized":"2023-05-16T07:38:54.146695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:54.146576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:54.146576+0000","last_clean_scrub_stamp":"2023-05-16T07:38:54.146576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:56.113639+0000","last_change":"2023-05-16T07:38:56.113639+0000","last_active":"2023-05-16T07:38:56.113639+0000","last_peered":"2023-05-16T07:38:56.113639+0000","last_clean":"2023-05-16T07:38:56.113639+0000","last_became_active":"2023-05-16T07:02:21.775037+0000","last_became_peered":"2023-05-16T07:02:21.775037+0000","last_unstale":"2023-05-16T07:38:56.113639+0000","last_undegraded":"2023-05-16T07:38:56.113639+0000","last_fullsized":"2023-05-16T07:38:56.113639+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:56.113502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:56.113502+0000","last_clean_scrub_stamp":"2023-05-16T07:38:56.113502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:57.157687+0000","last_change":"2023-05-16T07:38:57.157687+0000","last_active":"2023-05-16T07:38:57.157687+0000","last_peered":"2023-05-16T07:38:57.157687+0000","last_clean":"2023-05-16T07:38:57.157687+0000","last_became_active":"2023-05-16T07:02:22.259615+0000","last_became_peered":"2023-05-16T07:02:22.259615+0000","last_unstale":"2023-05-16T07:38:57.157687+0000","last_undegraded":"2023-05-16T07:38:57.157687+0000","last_fullsized":"2023-05-16T07:38:57.157687+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:57.157560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:57.157560+0000","last_clean_scrub_stamp":"2023-05-16T07:38:57.157560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:51.923964+0000","last_change":"2023-05-16T07:38:51.923964+0000","last_active":"2023-05-16T07:38:51.923964+0000","last_peered":"2023-05-16T07:38:51.923964+0000","last_clean":"2023-05-16T07:38:51.923964+0000","last_became_active":"2023-05-16T07:02:21.774659+0000","last_became_peered":"2023-05-16T07:02:21.774659+0000","last_unstale":"2023-05-16T07:38:51.923964+0000","last_undegraded":"2023-05-16T07:38:51.923964+0000","last_fullsized":"2023-05-16T07:38:51.923964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:51.923799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:51.923799+0000","last_clean_scrub_stamp":"2023-05-16T07:38:51.923799+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-05-16T07:38:56.323320+0000","last_change":"2023-05-16T07:38:56.323320+0000","last_active":"2023-05-16T07:38:56.323320+0000","last_peered":"2023-05-16T07:38:56.323320+0000","last_clean":"2023-05-16T07:38:56.323320+0000","last_became_active":"2023-05-16T07:02:21.774517+0000","last_became_peered":"2023-05-16T07:02:21.774517+0000","last_unstale":"2023-05-16T07:38:56.323320+0000","last_undegraded":"2023-05-16T07:38:56.323320+0000","last_fullsized":"2023-05-16T07:38:56.323320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-16T07:38:56.323202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-16T07:38:56.323202+0000","last_clean_scrub_stamp":"2023-05-16T07:38:56.323202+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575281,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25392,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":23232,"kb_avail":94346448,"statfs":{"total":96636764160,"available":96610762752,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23789568},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.78700000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.82399999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.755,"15min":0.79800000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.503},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73399999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71599999999999997}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.71499999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.315},"max":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.75},"last":0.46000000000000002}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.79100000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80800000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.82699999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.84799999999999998},"last":0.40100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575281,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25200,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":23040,"kb_avail":94346640,"statfs":{"total":96636764160,"available":96610959360,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23592960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64900000000000002,"5min":0.83399999999999996,"15min":0.86299999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.622},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.77500000000000002,"5min":1.5,"15min":1.5},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73599999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65500000000000003}]},{"osd":2,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":1.486,"15min":1.486},"last":0.623}]},{"osd":3,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":1.532,"15min":1.532},"last":0.68200000000000005}]},{"osd":4,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.29499999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.505},{"interface":"front","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45700000000000002}]},{"osd":5,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42199999999999999}]},{"osd":7,"last update":"Tue May 16 07:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.878,"15min":0.878},"last":0.496},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.316},"max":{"1min":0.873,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575282,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20784,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":18624,"kb_avail":94351056,"statfs":{"total":96636764160,"available":96615481344,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19070976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58899999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.64100000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.60899999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41899999999999998},"min":{"1min":0.26100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.70199999999999996},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.67900000000000005},"last":0.379}]},{"osd":4,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.872,"15min":0.877},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.91000000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91500000000000004,"5min":1.054,"15min":1.054},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.95399999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.998},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue May 16 07:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46500000000000002,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575282,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20464,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":18304,"kb_avail":94351376,"statfs":{"total":96636764160,"available":96615809024,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18743296},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.442}]},{"osd":2,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.223,"15min":0.223},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.502},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.433,"15min":0.432},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.98699999999999999},"last":0.877},{"interface":"front","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.90400000000000003,"15min":1.0549999999999999},"last":0.755}]},{"osd":5,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":1.0680000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65500000000000003,"15min":0.68000000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.095},"last":0.85699999999999998}]},{"osd":6,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":1.0369999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1},"last":0.77400000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90300000000000002,"5min":0.91500000000000004,"15min":1.03},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90400000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.89400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575282,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22128,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94349712,"statfs":{"total":96636764160,"available":96614105088,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.36099999999999999},"min":{"1min":0.23999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.747},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.504,"5min":0.51800000000000002,"15min":0.65600000000000003},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.67200000000000004},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.69399999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.317,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63200000000000001},"last":0.373}]},{"osd":4,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.90100000000000002,"15min":1.048},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.63400000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35499999999999998},"max":{"1min":0.745,"5min":0.88500000000000001,"15min":1.0069999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.41899999999999998},"max":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.997},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.92700000000000005,"15min":1.01},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76900000000000002,"5min":0.91400000000000003,"15min":1.0229999999999999},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue May 16 07:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.998},"last":0.497},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.84999999999999998,"15min":0.96499999999999997},"last":0.60699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28976,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94342864,"statfs":{"total":96636764160,"available":96607092736,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.58999999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.63200000000000001},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.61299999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.72999999999999998},"last":0.46100000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.68400000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.43099999999999999}]},{"osd":4,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.94299999999999995,"15min":1.014},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":0.84099999999999997,"15min":1.024},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.97799999999999998,"15min":1.05},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.97299999999999998},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.91600000000000004,"15min":1.0389999999999999},"last":0.748},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.0780000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Tue May 16 07:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":1.0640000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":1.012},"last":0.76200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":25136,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":22976,"kb_avail":94346704,"statfs":{"total":96636764160,"available":96611024896,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23527424},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.71799999999999997,"5min":0.79400000000000004,"15min":0.83399999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.753,"5min":1.359,"15min":1.359},"last":0.57599999999999996}]},{"osd":1,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":0.74199999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.80400000000000005},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.627},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.81399999999999995},"last":0.65700000000000003}]},{"osd":5,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.436},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.755,"15min":0.755},"last":0.375},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.318},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue May 16 07:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.505}]}]},{"osd":5,"up_from":13,"seq":55834575281,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":2203648,"data_stored":1181362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.81399999999999995}]},{"osd":2,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.76900000000000002}]},{"osd":3,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.80400000000000005}]},{"osd":4,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue May 16 07:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-16T07:39:10.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-05-16T07:39:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.598+0000 7fbf51d5c700 1 Processor -- start 2023-05-16T07:39:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.598+0000 7fbf51d5c700 1 -- start start 2023-05-16T07:39:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf51d5c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 0x7fbf4c069b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf51d5c700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7fbf4c06a070 0x7fbf4c0c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf51d5c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c06a4e0 con 0x7fbf4c1509b0 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf51d5c700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c0bc1b0 con 0x7fbf4c06a070 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf51d5c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0bc490 con 0x7fbf4c150d90 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf4b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbf4c1509b0 0x7fbf4c065d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36576/0 (socket says 172.21.15.92:36576) 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf4b7fe700 1 -- 172.21.15.92:0/3079018232 learned_addr learned my addr 172.21.15.92:0/3079018232 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.599+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892866618 0 0) 0x7fbf4c06a4e0 con 0x7fbf4c1509b0 2023-05-16T07:39:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf380033f0 con 0x7fbf4c1509b0 2023-05-16T07:39:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 0x7fbf4c069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4bfff700 1 --2- 172.21.15.92:0/3079018232 >> v2:172.21.15.158:3300/0 conn(0x7fbf4c06a070 0x7fbf4c0c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983121260 0 0) 0x7fbf380033f0 con 0x7fbf4c1509b0 2023-05-16T07:39:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 msgr2=0x7fbf4c069b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 0x7fbf4c069b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 >> v2:172.21.15.158:3300/0 conn(0x7fbf4c06a070 msgr2=0x7fbf4c0c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 --2- 172.21.15.92:0/3079018232 >> v2:172.21.15.158:3300/0 conn(0x7fbf4c06a070 0x7fbf4c0c16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0bc770 con 0x7fbf4c1509b0 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7fbf3c001470 con 0x7fbf4c1509b0 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 >> v1:172.21.15.92:6789/0 conn(0x7fbf4c1509b0 legacy=0x7fbf4c065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 0x7fbf380038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf38003ec0 0x7fbf38004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38004b80 0x7fbf38005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0bc1b0 con 0x7fbf38003ec0 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0bc490 con 0x7fbf38004b80 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3800b990 con 0x7fbf4c150d90 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf38003ec0 0x7fbf38004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4bfff700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38004b80 0x7fbf38005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4b7fe700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 0x7fbf380038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 -- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 msgr2=0x7fbf380038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c150d90 0x7fbf380038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 -- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38004b80 msgr2=0x7fbf38005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38004b80 0x7fbf38005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 -- 172.21.15.92:0/3079018232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380033f0 con 0x7fbf38003ec0 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf42ffd700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf38003ec0 0x7fbf38004690 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7fbf3400b9e0 tx=0x7fbf34008040).ready entity=mon.0 client_cookie=cad0c3237436b1b server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34013480 con 0x7fbf38003ec0 2023-05-16T07:39:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf340135e0 con 0x7fbf38003ec0 2023-05-16T07:39:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.600+0000 7fbf4affd700 1 -- 172.21.15.92:0/3079018232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34013980 con 0x7fbf38003ec0 2023-05-16T07:39:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf51d5c700 1 -- 172.21.15.92:0/3079018232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf38003ec0 msgr2=0x7fbf38004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/3079018232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf38003ec0 0x7fbf38004690 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7fbf3400b9e0 tx=0x7fbf34008040).stop 2023-05-16T07:39:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf4bfff700 1 -- 172.21.15.92:0/3079018232 reap_dead start 2023-05-16T07:39:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf51d5c700 1 -- 172.21.15.92:0/3079018232 shutdown_connections 2023-05-16T07:39:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf51d5c700 1 -- 172.21.15.92:0/3079018232 >> 172.21.15.92:0/3079018232 conn(0x7fbf4c0c4620 msgr2=0x7fbf4c066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:39:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf51d5c700 1 -- 172.21.15.92:0/3079018232 shutdown_connections 2023-05-16T07:39:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.601+0000 7fbf51d5c700 1 -- 172.21.15.92:0/3079018232 wait complete. 2023-05-16T07:39:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.602+0000 7fbf51d5c700 1 Processor -- start 2023-05-16T07:39:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.602+0000 7fbf51d5c700 1 -- start start 2023-05-16T07:39:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf51d5c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38003ec0 0x7fbf4c0c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf51d5c700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c0beb60 0x7fbf4c0c0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf51d5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 0x7fbf4c14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf51d5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0bf0e0 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf51d5c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0be4b0 con 0x7fbf38003ec0 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf51d5c700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c2540 con 0x7fbf4c0beb60 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 0x7fbf4c14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 0x7fbf4c14f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54076/0 (socket says 172.21.15.92:54076) 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 -- 172.21.15.92:0/1457472126 learned_addr learned my addr 172.21.15.92:0/1457472126 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf42ffd700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c0beb60 0x7fbf4c0c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 -- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c0beb60 msgr2=0x7fbf4c0c0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c0beb60 0x7fbf4c0c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 -- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38003ec0 msgr2=0x7fbf4c0c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4b7fe700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38003ec0 0x7fbf4c0c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38003ec0 0x7fbf4c0c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.603+0000 7fbf4bfff700 1 -- 172.21.15.92:0/1457472126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c148080 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.604+0000 7fbf4bfff700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 0x7fbf4c14f920 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7fbf4c063c90 tx=0x7fbf44009710).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.604+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44009b10 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.604+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44002d50 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.604+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0c1910 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.604+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44016640 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.604+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c0c1630 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.605+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c148230 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.605+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7fbf44016a70 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.605+0000 7fbf43fff700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf2c05ed90 0x7fbf2c061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.605+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbf4407a990 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.606+0000 7fbf4b7fe700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf2c05ed90 0x7fbf2c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.606+0000 7fbf4b7fe700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf2c05ed90 0x7fbf2c061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fbf3c0011e0 tx=0x7fbf3c009040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.609+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7fbf4c148230 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.771+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fbf4c148230 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.775+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbf4404be10 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.775+0000 7fbf43fff700 1 -- 172.21.15.92:0/1457472126 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7fbf4c148230 con 0x7fbf4c0c0d20 2023-05-16T07:39:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.777+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf2c05ed90 msgr2=0x7fbf2c061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.777+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf2c05ed90 0x7fbf2c061250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fbf3c0011e0 tx=0x7fbf3c009040).stop 2023-05-16T07:39:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.777+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 msgr2=0x7fbf4c14f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.777+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 0x7fbf4c14f920 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7fbf4c063c90 tx=0x7fbf44009710).stop 2023-05-16T07:39:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 shutdown_connections 2023-05-16T07:39:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf4c0c0d20 0x7fbf4c14f920 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbf38003ec0 0x7fbf4c0c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7fbf2c05ed90 0x7fbf2c061250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 --2- 172.21.15.92:0/1457472126 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7fbf4c0beb60 0x7fbf4c0c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 >> 172.21.15.92:0/1457472126 conn(0x7fbf4c0c4620 msgr2=0x7fbf4c0c5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:39:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 shutdown_connections 2023-05-16T07:39:10.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.778+0000 7fbf51d5c700 1 -- 172.21.15.92:0/1457472126 wait complete. 2023-05-16T07:39:10.793 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-16T07:39:10.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-16T07:39:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.901+0000 7f16876f1700 1 Processor -- start 2023-05-16T07:39:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.901+0000 7f16876f1700 1 -- start start 2023-05-16T07:39:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f16876f1700 1 --2- >> v2:172.21.15.158:3300/0 conn(0x7f1680154170 0x7f16800c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f16876f1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800c0660 con 0x7f1680069db0 2023-05-16T07:39:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f16876f1700 1 -- --> v1:172.21.15.158:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800bfd30 con 0x7f1680154550 2023-05-16T07:39:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f16876f1700 1 -- --> v2:172.21.15.158:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800c0010 con 0x7f1680154170 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1685c8e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1680069db0 0x7f16800c0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36598/0 (socket says 172.21.15.92:36598) 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1685c8e700 1 -- 172.21.15.92:0/3651265308 learned_addr learned my addr 172.21.15.92:0/3651265308 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f168548d700 1 --2- 172.21.15.92:0/3651265308 >> v2:172.21.15.158:3300/0 conn(0x7f1680154170 0x7f16800c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460867634 0 0) 0x7f16800c0660 con 0x7f1680069db0 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f166c0033f0 con 0x7f1680069db0 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1736591544 0 0) 0x7f166c0033f0 con 0x7f1680069db0 2023-05-16T07:39:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 >> v2:172.21.15.158:3300/0 conn(0x7f1680154170 msgr2=0x7f16800c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 --2- 172.21.15.92:0/3651265308 >> v2:172.21.15.158:3300/0 conn(0x7f1680154170 0x7f16800c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 >> v1:172.21.15.158:6789/0 conn(0x7f1680154550 legacy=0x7f16800696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16800c02f0 con 0x7f1680069db0 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.902+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3989611242 0 0) 0x7f16780017f0 con 0x7f1680069db0 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 >> v1:172.21.15.92:6789/0 conn(0x7f1680069db0 legacy=0x7f16800c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c0038e0 0x7f166c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f166c004690 0x7f166c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f166c005050 0x7f166c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c0010 con 0x7f166c004690 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c00bfe0 con 0x7f166c005050 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c00c1a0 con 0x7f166c0038e0 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f166c004690 0x7f166c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1685c8e700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f166c005050 0x7f166c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f168548d700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c0038e0 0x7f166c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 -- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c0038e0 msgr2=0x7f166c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c0038e0 0x7f166c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 -- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f166c005050 msgr2=0x7f166c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f166c005050 0x7f166c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1685c8e700 1 -- 172.21.15.92:0/3651265308 reap_dead start 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 -- 172.21.15.92:0/3651265308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0033f0 con 0x7f166c004690 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f168548d700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c0038e0 0x7f166c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-16T07:39:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1684c8c700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f166c004690 0x7f166c004b10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f1680154930 tx=0x7f1670007bb0).ready entity=mon.0 client_cookie=7db9774abfa9a5e3 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670007020 con 0x7f166c004690 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1670007180 con 0x7f166c004690 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.903+0000 7f1677fff700 1 -- 172.21.15.92:0/3651265308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670007540 con 0x7f166c004690 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 -- 172.21.15.92:0/3651265308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f166c004690 msgr2=0x7f166c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f166c004690 0x7f166c004b10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f1680154930 tx=0x7f1670007bb0).stop 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 -- 172.21.15.92:0/3651265308 shutdown_connections 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 --2- 172.21.15.92:0/3651265308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f166c004690 0x7f166c004b10 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 -- 172.21.15.92:0/3651265308 >> 172.21.15.92:0/3651265308 conn(0x7f16800cb700 msgr2=0x7f16800c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:39:10.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 -- 172.21.15.92:0/3651265308 shutdown_connections 2023-05-16T07:39:10.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.904+0000 7f16876f1700 1 -- 172.21.15.92:0/3651265308 wait complete. 2023-05-16T07:39:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.905+0000 7f16876f1700 1 Processor -- start 2023-05-16T07:39:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.905+0000 7f16876f1700 1 -- start start 2023-05-16T07:39:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 --2- >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c004690 0x7f168013b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 0x7f168014d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1680152f30 0x7f1680153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c2730 con 0x7f168014d350 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c32b0 con 0x7f1680152f30 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 -- --> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168014dcc0 con 0x7f166c004690 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 0x7f168014d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 0x7f168014d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54094/0 (socket says 172.21.15.92:54094) 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 -- 172.21.15.92:0/293791663 learned_addr learned my addr 172.21.15.92:0/293791663 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f168548d700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c004690 0x7f168013b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 -- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c004690 msgr2=0x7f168013b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c004690 0x7f168013b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 -- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1680152f30 msgr2=0x7f1680153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1680152f30 0x7f1680153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 -- 172.21.15.92:0/293791663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168013b6b0 con 0x7f168014d350 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f1684c8c700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 0x7f168014d7d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f1670008fd0 tx=0x7f1670007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670007700 con 0x7f168014d350 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1670013740 con 0x7f168014d350 2023-05-16T07:39:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16801508a0 con 0x7f168014d350 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670013ae0 con 0x7f168014d350 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.906+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16800c2c60 con 0x7f168014d350 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.907+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16800666c0 con 0x7f168014d350 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.908+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 87) v1 ==== 75242+0+0 (secure 0 0 0) 0x7f167001c070 con 0x7f168014d350 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.908+0000 7f16757fa700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f166405ed60 0x7f1664061220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.908+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f167007a630 con 0x7f168014d350 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.908+0000 7f168548d700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f166405ed60 0x7f1664061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-05-16T07:39:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.909+0000 7f168548d700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f166405ed60 0x7f1664061220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f16800618e0 tx=0x7f167c008040).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-16T07:39:10.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:10.912+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162005 (secure 0 0 0) 0x7f167004ba50 con 0x7f168014d350 2023-05-16T07:39:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.117+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f168006c350 con 0x7f168014d350 2023-05-16T07:39:11.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.117+0000 7f16757fa700 1 -- 172.21.15.92:0/293791663 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f167004f070 con 0x7f168014d350 2023-05-16T07:39:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-05-16T07:39:11.122 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/2356054425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":2356054425},{"type":"v1","addr":"172.21.15.158:6835","nonce":2356054425}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/3801885908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":3801885908},{"type":"v1","addr":"172.21.15.158:6837","nonce":3801885908}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/374438642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":374438642},{"type":"v1","addr":"172.21.15.92:6834","nonce":374438642}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8823,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/219689849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":219689849},{"type":"v1","addr":"172.21.15.92:6835","nonce":219689849}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[]} 2023-05-16T07:39:11.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f166405ed60 msgr2=0x7f1664061220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:11.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f166405ed60 0x7f1664061220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f16800618e0 tx=0x7f167c008040).stop 2023-05-16T07:39:11.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 msgr2=0x7f168014d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-16T07:39:11.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 0x7f168014d7d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f1670008fd0 tx=0x7f1670007bb0).stop 2023-05-16T07:39:11.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 shutdown_connections 2023-05-16T07:39:11.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:6832/95623,v1:172.21.15.158:6833/95623] conn(0x7f166405ed60 0x7f1664061220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:11.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3301/0,v1:172.21.15.158:6790/0] conn(0x7f166c004690 0x7f168013b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:11.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1680152f30 0x7f1680153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-05-16T07:39:11.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 --2- 172.21.15.92:0/293791663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f168014d350 0x7f168014d7d0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-05-16T07:39:11.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 >> 172.21.15.92:0/293791663 conn(0x7f16800cb700 msgr2=0x7f16800ab4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-16T07:39:11.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 shutdown_connections 2023-05-16T07:39:11.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-05-16T07:39:11.119+0000 7f16876f1700 1 -- 172.21.15.92:0/293791663 wait complete. 2023-05-16T07:39:11.125 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 63 2023-05-16T07:39:11.135 DEBUG:tasks.cephfs.filesystem:already dead... 2023-05-16T07:39:11.135 INFO:teuthology.misc:Shutting down mds daemons... 2023-05-16T07:39:11.135 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-16T07:39:11.136 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.143 INFO:tasks.ceph.mds.a:Stopped 2023-05-16T07:39:11.143 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-16T07:39:11.144 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.191 INFO:tasks.ceph.mds.c:Stopped 2023-05-16T07:39:11.192 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-16T07:39:11.192 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.202 INFO:tasks.ceph.mds.b:Stopped 2023-05-16T07:39:11.202 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-16T07:39:11.203 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.253 INFO:tasks.ceph.mds.d:Stopped 2023-05-16T07:39:11.254 INFO:teuthology.misc:Shutting down osd daemons... 2023-05-16T07:39:11.254 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-05-16T07:39:11.254 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.284 INFO:tasks.ceph.osd.0:Stopped 2023-05-16T07:39:11.285 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-05-16T07:39:11.285 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.356 INFO:tasks.ceph.osd.1:Stopped 2023-05-16T07:39:11.357 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-05-16T07:39:11.357 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.431 INFO:tasks.ceph.osd.2:Stopped 2023-05-16T07:39:11.432 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-05-16T07:39:11.433 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.503 INFO:tasks.ceph.osd.3:Stopped 2023-05-16T07:39:11.504 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-05-16T07:39:11.504 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.538 INFO:tasks.ceph.osd.4:Stopped 2023-05-16T07:39:11.539 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-05-16T07:39:11.539 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.614 INFO:tasks.ceph.osd.5:Stopped 2023-05-16T07:39:11.614 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-05-16T07:39:11.615 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.683 INFO:tasks.ceph.osd.6:Stopped 2023-05-16T07:39:11.684 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-05-16T07:39:11.684 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.756 INFO:tasks.ceph.osd.7:Stopped 2023-05-16T07:39:11.756 INFO:teuthology.misc:Shutting down mgr daemons... 2023-05-16T07:39:11.757 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-05-16T07:39:11.757 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.779 INFO:tasks.ceph.mgr.y:Stopped 2023-05-16T07:39:11.780 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-05-16T07:39:11.780 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.824 INFO:tasks.ceph.mgr.x:Stopped 2023-05-16T07:39:11.824 INFO:teuthology.misc:Shutting down mon daemons... 2023-05-16T07:39:11.824 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-05-16T07:39:11.825 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-05-16T07:39:11.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-05-16T07:39:11.838 INFO:tasks.ceph.mon.a:Stopped 2023-05-16T07:39:11.838 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-05-16T07:39:11.838 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.877 INFO:tasks.ceph.mon.b:Stopped 2023-05-16T07:39:11.877 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-05-16T07:39:11.877 INFO:teuthology.orchestra.run:waiting for 300 2023-05-16T07:39:11.930 INFO:tasks.ceph.mon.c:Stopped 2023-05-16T07:39:11.930 INFO:tasks.ceph:Checking cluster log for badness... 2023-05-16T07:39:11.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-05-16T07:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout:2023-05-16T07:04:27.585584+0000 mon.a (mon.0) 312 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-05-16T07:39:11.958 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-05-16T07:39:11.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-05-16T07:39:12.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-05-16T07:39:12.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-05-16T07:39:12.155 INFO:teuthology.orchestra.run.smithi092.stdout:2023-05-16T07:04:27.585584+0000 mon.a (mon.0) 312 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-05-16T07:39:12.155 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:39:12.156 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-05-16T07:39:12.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:39:12.383 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-05-16T07:39:12.532 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:39:12.532 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-05-16T07:39:12.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:39:12.612 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-05-16T07:39:12.673 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:39:12.673 DEBUG:teuthology.orchestra.run.smithi158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-05-16T07:39:12.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:39:12.874 DEBUG:teuthology.orchestra.run.smithi158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-05-16T07:39:12.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:39:12.953 DEBUG:teuthology.orchestra.run.smithi158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-05-16T07:39:13.071 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:39:13.071 DEBUG:teuthology.orchestra.run.smithi158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-05-16T07:39:13.142 INFO:tasks.ceph:Archiving mon data... 2023-05-16T07:39:13.144 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/data/mon.a.tgz 2023-05-16T07:39:13.144 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-05-16T07:39:13.160 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.stdBHijXtj 2023-05-16T07:39:13.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.stdBHijXtj 2023-05-16T07:39:13.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.stdBHijXtj 2023-05-16T07:39:13.689 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.stdBHijXtj is 2MB 2023-05-16T07:39:13.811 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.stdBHijXtj 2023-05-16T07:39:13.825 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/data/mon.b.tgz 2023-05-16T07:39:13.826 DEBUG:teuthology.orchestra.run.smithi158:> mktemp 2023-05-16T07:39:13.844 INFO:teuthology.orchestra.run.smithi158.stdout:/tmp/tmp.DMT34bYGyp 2023-05-16T07:39:13.844 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.DMT34bYGyp 2023-05-16T07:39:14.324 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 0666 /tmp/tmp.DMT34bYGyp 2023-05-16T07:39:14.404 DEBUG:teuthology.orchestra.remote:smithi158:/tmp/tmp.DMT34bYGyp is 2MB 2023-05-16T07:39:14.515 DEBUG:teuthology.orchestra.run.smithi158:> rm -fr /tmp/tmp.DMT34bYGyp 2023-05-16T07:39:14.529 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/data/mon.c.tgz 2023-05-16T07:39:14.529 DEBUG:teuthology.orchestra.run.smithi158:> mktemp 2023-05-16T07:39:14.582 INFO:teuthology.orchestra.run.smithi158.stdout:/tmp/tmp.wmRQ7Tfnp0 2023-05-16T07:39:14.583 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wmRQ7Tfnp0 2023-05-16T07:39:15.031 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 0666 /tmp/tmp.wmRQ7Tfnp0 2023-05-16T07:39:15.111 DEBUG:teuthology.orchestra.remote:smithi158:/tmp/tmp.wmRQ7Tfnp0 is 2MB 2023-05-16T07:39:15.197 DEBUG:teuthology.orchestra.run.smithi158:> rm -fr /tmp/tmp.wmRQ7Tfnp0 2023-05-16T07:39:15.211 INFO:tasks.ceph:Cleaning ceph cluster... 2023-05-16T07:39:15.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-05-16T07:39:15.214 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-05-16T07:39:15.274 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-05-16T07:39:15.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-05-16T07:39:15.282 DEBUG:teuthology.orchestra.run.smithi158:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-05-16T07:39:15.308 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-05-16T07:39:15.339 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-05-16T07:39:15.344 INFO:tasks.ceph:Archiving crash dumps... 2023-05-16T07:39:15.346 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/remote/smithi092/crash 2023-05-16T07:39:15.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-05-16T07:39:15.384 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/remote/smithi158/crash 2023-05-16T07:39:15.385 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-05-16T07:39:15.408 INFO:tasks.ceph:Shutting down logrotate 2023-05-16T07:39:15.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:39:15.428 DEBUG:teuthology.orchestra.run.smithi158:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-16T07:39:15.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-05-16T07:39:15.552 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-05-16T07:39:15.575 INFO:tasks.ceph:Compressing logs... 2023-05-16T07:39:15.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-05-16T07:39:15.595 DEBUG:teuthology.orchestra.run.smithi158:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-05-16T07:39:16.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-05-16T07:39:16.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-05-16T07:39:23.031 INFO:tasks.ceph:Archiving logs... 2023-05-16T07:39:23.031 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/remote/smithi092/log 2023-05-16T07:39:23.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2023-05-16T07:39:24.166 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/remote/smithi158/log 2023-05-16T07:39:24.167 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/log/ceph -- . 2023-05-16T07:39:24.864 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-05-16T07:39:24.877 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2023-05-16T07:39:24.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2023-05-16T07:39:24.880 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2023-05-16T07:39:24.979 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-05-16T07:39:24.990 INFO:teuthology.task.clock:Checking final clock skew... 2023-05-16T07:39:24.990 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-05-16T07:39:24.993 DEBUG:teuthology.orchestra.run.smithi158:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-05-16T07:39:25.006 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-05-16T07:39:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:210 Number of sources = 4 2023-05-16T07:39:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-05-16T07:39:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-05-16T07:39:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 126 -3330us[-3343us] +/- 85ms 2023-05-16T07:39:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 128 -2038us[-2051us] +/- 100ms 2023-05-16T07:39:25.010 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv03.front.sepia.ceph.com 3 6 377 61 +4433us[+4435us] +/- 74ms 2023-05-16T07:39:25.011 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T07:39:25.011 INFO:teuthology.orchestra.run.smithi158.stderr:bash: ntpq: command not found 2023-05-16T07:39:25.013 INFO:teuthology.orchestra.run.smithi158.stdout:210 Number of sources = 4 2023-05-16T07:39:25.013 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-05-16T07:39:25.013 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2023-05-16T07:39:25.013 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 -3458us[-3448us] +/- 86ms 2023-05-16T07:39:25.013 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -2181us[-2171us] +/- 101ms 2023-05-16T07:39:25.013 INFO:teuthology.orchestra.run.smithi158.stdout:^* hv03.front.sepia.ceph.com 3 6 377 57 +4335us[+4346us] +/- 74ms 2023-05-16T07:39:25.014 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-05-16T07:39:25.014 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-05-16T07:39:25.025 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-05-16T07:39:25.026 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-05-16T07:39:25.038 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-05-16T07:39:25.168 DEBUG:teuthology.orchestra.run.smithi158:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-05-16T07:39:25.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-05-16T07:39:25.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:39:25.294 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-05-16T07:39:25.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:39:25.325 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-05-16T07:39:25.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-05-16T07:39:25.420 INFO:teuthology.task.internal:Duration was 3352.694593 seconds 2023-05-16T07:39:25.420 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-05-16T07:39:25.454 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-05-16T07:39:25.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-05-16T07:39:25.457 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-05-16T07:39:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-05-16T07:39:25.501 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-05-16T07:39:25.965 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-05-16T07:39:25.965 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2023-05-16T07:39:25.965 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-05-16T07:39:25.984 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi158.front.sepia.ceph.com 2023-05-16T07:39:25.985 DEBUG:teuthology.orchestra.run.smithi158:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-05-16T07:39:26.004 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-05-16T07:39:26.004 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-05-16T07:39:26.027 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-05-16T07:39:26.083 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-05-16T07:39:26.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-05-16T07:39:26.112 DEBUG:teuthology.orchestra.run.smithi158:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-05-16T07:39:26.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-05-16T07:39:26.210 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-05-16T07:39:26.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-05-16T07:39:26.234 DEBUG:teuthology.orchestra.run.smithi158:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-05-16T07:39:26.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-05-16T07:39:26.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-05-16T07:39:26.277 DEBUG:teuthology.orchestra.run.smithi158:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-05-16T07:39:26.297 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2023-05-16T07:39:26.321 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = core 2023-05-16T07:39:26.334 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2023-05-16T07:39:26.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:39:26.364 DEBUG:teuthology.orchestra.run.smithi158:> test -e /home/ubuntu/cephtest/archive/coredump 2023-05-16T07:39:26.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-16T07:39:26.387 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-05-16T07:39:26.398 INFO:teuthology.task.internal:Transferring archived files... 2023-05-16T07:39:26.399 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/remote/smithi092 2023-05-16T07:39:26.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-05-16T07:39:26.439 DEBUG:teuthology.misc:Transferring archived files from smithi158:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061/remote/smithi158 2023-05-16T07:39:26.439 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-05-16T07:39:26.471 INFO:teuthology.task.internal:Removing archive directory... 2023-05-16T07:39:26.471 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2023-05-16T07:39:26.475 DEBUG:teuthology.orchestra.run.smithi158:> rm -rf -- /home/ubuntu/cephtest/archive 2023-05-16T07:39:26.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-05-16T07:39:26.532 INFO:teuthology.task.internal:Not uploading archives. 2023-05-16T07:39:26.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-05-16T07:39:26.542 INFO:teuthology.task.internal:Tidying up after the test... 2023-05-16T07:39:26.542 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-05-16T07:39:26.544 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-05-16T07:39:26.556 INFO:teuthology.orchestra.run.smithi092.stdout: 262168 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 16 07:39 /home/ubuntu/cephtest 2023-05-16T07:39:26.574 INFO:teuthology.orchestra.run.smithi158.stdout: 262168 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 16 07:39 /home/ubuntu/cephtest 2023-05-16T07:39:26.575 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-05-16T07:39:26.585 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-05-16T07:39:26.605 INFO:teuthology.nuke:Checking targets against current locks 2023-05-16T07:39:26.632 DEBUG:teuthology.nuke:shortname: smithi092 2023-05-16T07:39:26.632 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-16T07:39:26.654 DEBUG:teuthology.nuke:shortname: smithi158 2023-05-16T07:39:26.654 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-16T07:39:26.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-16 06:37:28.807553', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCogJToiq7Bj+4LXDaTzkRPivwioX4EePZ/pr/wuw7kazuX66qqj+UsTcjkQsxH16yUR7aqgG8BuzZZZot4jagbc2yiK6FpFc1Scs1mug+neYOc7vaRg+4rlSuEDenCXHChSM8Usrys/mmqt+to/W6JC0+F6i8DZT3qlCf2lyYQXNQcUuEJZh7gqtjdGUC6w08V9O9ET18IFk0e67tGdbiY8YhVxW6GiU4chMGdJWqkRHBO5evT5C/fCFScvzT6g+t1XcBYCm5WxcQ7PJg9wDlDIZSTUX0Dudj1h8Wzr5feNqW9/iFhp9RYjAtwWfxy/LLEwhkLY7/aCQwWwChEQCtJs9VzyPPaJbSXkieNPnpqYDbxYwYG9gvPHhKo5/JcuqKtf4ZYv2F+iw9zmMy9c6RWgB3/2tiyAMOPqzHvvdZk+PVySYuV2DfgzwCYCQTIMJp7225sGih4Ns/AQo6NF3qMzEmBIYSmxrfU/CA5DiZ2Rt7PIBQ7wfkzlZdNpG0o548='} 2023-05-16T07:39:26.679 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7275061', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-16 06:37:28.805570', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdf9TST+bZAMveHtQb+bAEliZIsauT4reJIVKcMcsRrC907QFoHkmkAQguH0kVa1zxA4RLPzlufnJ06+syo7rI='} 2023-05-16T07:39:26.711 INFO:teuthology.orchestra.console:Power off smithi092 2023-05-16T07:39:26.712 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-05-16T07:39:26.733 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-16T07:39:26.758 INFO:teuthology.orchestra.console:Power off smithi158 2023-05-16T07:39:26.758 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-05-16T07:39:26.783 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-16T07:39:30.769 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-16T07:39:30.808 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-16T07:39:34.884 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-16T07:39:34.919 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-16T07:39:35.096 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2023-05-16T07:39:35.130 INFO:teuthology.orchestra.console:Power off for smithi158 completed 2023-05-16T07:39:35.234 INFO:teuthology.run:Summary data: description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 3352.6945934295654 failure_reason: '"2023-05-16T07:04:27.585584+0000 mon.a (mon.0) 312 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-05-16T07:39:35.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-05-16T07:39:35.313 INFO:teuthology.run:FAIL